svm & vxvm disk replacement

Solaris: SVM Disk Replacement

1. Detach defective disk:

·         #metadetach d0 d20; 
       #metadetach d1 d21

2. Replace the disk online c1t1d0

3. Allow the disk to be recognized thru devfsadm command:

·         #devfsadm

4. Once disk is recognized, copy the vtoc of other submirror c1t0d0s0 thru fmthard and  prtvtoc commands:volume table of contents (vtoc)

·         #prtvtoc /dev/rdsk/c1t0d0s2 |
      #fmthard -s - /dev/rdsk/c1t1d0s2

5. Recreate metadevices on slice 7:

·         #metadb -d /dev/dsk/c1t1d0s7; 
      #metadb –a –f –c 3 /dev/dsk/c1t1d0s7

6. Clear metadevices d20, d21

·         #metaclear d20; 
       #metaclear d21

7. Recreate the metadevices:

·         #metainit d20 1 1 c1t1d0s0; 
       #metainit d21 1 1 c1t1d0s1

8. Reattach the submirrors:

·         #metattach d0 d20; 
       #metattach d1 d21

Solution2:

1. See disk configuration:

·         #cfgadm -al

2. See replicas of the metadevice state database:

·         metadb –i

3. Remove the faulty disk in metadevice state database:

·         metadb -d c1t1d0s7

4. Unset the faulty disk in the disk configuration:

·         cfgadm -c unconfigure c1::21000004cf96b4d3

5. Replace the disk online c1t1d0

6. echo | format

7. Once disk is recognized, copy the vtoc of other submirror c1t0d0s0 thru fmthard and prtvtoc commands:

·         #prtvtoc /dev/rdsk/c1t0d0s2 
       #fmthard -s - /dev/rdsk/c1t1d0s2

8. Recreate the metadevice state database:

·         #metadb -afc3 c1t1d0s7

9. Check new replicas of the metadevice state database:

·         #metadb –i

10. Check the status of disk errors:

·         #iostat -En c1t0d0

·         #iostat -En c1t1d0

11. Check the status of metadevice:

#metastat

12. Replace the metadevice with “Need maintenance”:

·         #metareplace -e d0 c1t1d0s0

·         #metareplace -e d1 c1t1d0s1

Solaris: Converting VxFS root filesystem to SVM root filesystem

From vivek

NOTE: The assumptions and samples are based on our previews task to convert our Solaris servers having VxFS root filesystem to SVM. The reason for this change was that VxFS root filesystem does not generate any crash dump for analysis when the system reboots unexpectedly due to a hardware or software issue. The experts say that it is a limitation of Solaris system, which has not be address until now. Oh well, let us just do the job and hope for the best!

Assumption:

·        You have the following filesystems on VxFS:

Disk group name: rootdg

Mirrored disks: c0t0d0 and c0t1d0

Other disk groups are data filesystems and must remain untouched

·        For the preparation stage, all private regions of rootdg should only be present in c0t0d0 and c0t1d0 - This is really important since we are going to physically remove and re-insert the two disks for testing. If your setup is different, it will cause your VxVM filesystem to be inaccessible. For this kind of VxVM setup, additional steps for the preparation stage can be found here.

·        You have the following mounted filesystems under rootdg:

/ (rootvol, slice 0)

swap (swapvol, slice 1)

var (var, slice 5)

·         You have the following boot devices:

Primary boot disk: c0t0d0 (vx-rootdisk)

Secondary boot disk: c0t1d0 (vx-rootmirror)

·         You have a spare disk available for your contingency plan

·         You have the Solaris CD to boot from CDROM

Preparation Stage:

1.) Just in case you or I messed up your system, make sure to have a full backup

2.) Gather the current configuration of your system by running SUN explorer

# /opt/SUNWexplo/bin/explorer

3.) Reboot the server to make sure that it boots up properly before any changes

·         Boot up from primary disk (rootdisk)

 #shutdown -i6 -y -g0

4.) Verify that the server also boots up successfully using the secondary disk (rootmirror)

·         First, go to OBP prompt:

# shutdown -i0 -y -g0

·         Physically remove primary disk (rootdisk)

·         Boot to rootmirrot. (Boot alias can also be checked using devalias command)

ok? boot vx-rootmirror

·         Check the state of the server including any running application

5.) Temporarily use a different (spare) disk to act as the secondary disk

·         Go to OBP prompt again:

# shutdown -i0 -y -g0

·         Physically reattached the primary disk (rootdisk)

·         Replace the secondary disk with the spare disk and keep the secondary disk safe. (If things go so bad, you still have this secondary disk to revert back to the original configuration)

·         Boot to rootmirrot (Boot alias can be checked using devalias command)

ok? boot vx-rootdisk

Conversion from VxFS root filesytem to raw root filesystem:

6.) Unencapsulate the rootdisk

(Take note that below are just sample contents and may differ with your current configuration and requirements)

·         Change the /, /var, and swap entry in /etc/vfstab from VxFS devices to raw devices:

From:

/dev/vx/dsk/bootdg/swapvol - - swap - no nologging

/dev/vx/dsk/bootdg/rootvol /dev/vx/rdsk/bootdg/rootvol / ufs 1 no nologging

/dev/vx/dsk/bootdg/var /dev/vx/rdsk/bootdg/var /var ufs 1 no nologging

To:

/dev/dsk/c0t0d0s1 - - swap - no nologging

/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no nologging

/dev/dsk/c0t0d0s5 /dev/rdsk/c0t0d0s5 /var ufs 1 no nologging

·         Edit the /etc/system with the following:

From:

rootdev:/pseudo/vxio@0:0

set vxio:vol_rootdev_is_volume=1

To: (Note: A space is required after “*”)

* rootdev:/pseudo/vxio@0:0

* set vxio:vol_rootdev_is_volume=1

·         Reboot the server to test the changes. Boot from rootdisk (c0t0d0)

# shutdown -i6 -y -g0

·         Check the mounting points, remount all data filesystems if needed

·         Clear up the rootvol, swapvol, var which is under rootdg

# vxdg destroy rootdg

# vxdiskunsetup -C c0t0d0 (This will zero out slice 3 and 4)

# vxdiskunsetup -C c0t1d0 (This will zero out slice 3 and 4)

·         Reboot again the server to check data filesystems are mounted with no fsck errors. Run “vxprint -ht” to check if rootdg has been destroyed and data dgs are mounted in clean state

Conversion from raw root filesytem to SVM root filesystem:

7. Configure OS under SVM

(Take note that below are just sample contents and may differ with your current configuration and requirements)

·         Use the spare disk in creating the submirror of the rootdisk (c0t0d0) in SVM

·         Using format utility, allocate approximately 50MB for slice 7 - This is for metadevice db replica. (Check for any overlapping cylinders, slices 0 and 5 should NOT be edited/adjusted)

o    Boot the system in single user mode using CD/DVD to edit partitions on rootdisk (c0t0d0)

ok? boot cdrom -s

·         Using format utility, edit the partition (c0t0d0). Swap slice (s1) needs to be reduced in order to create metadb slice (s7) with 50MB. (Check all the cylinder sizes for overlaps after reducing swap slice)

From:

Part      Tag    Flag     Cylinders         Size            Blocks

 0       root    wm       0 -  4740       46.01GB    (4741/0/0)   96488832

 1       swap    wu    4741 -  9429       45.50GB    (4689/0/0)   95430528

 2     backup    wu       0 - 14086      136.71GB    (14087/0/0) 286698624

 3          -    wu       0 - 14086      136.71GB    (14087/0/0) 286698624

 4          -    wu   14085 - 14086       19.88MB    (2/0/0)         40704

 5        var    wm    9430 - 14082       45.16GB    (4653/0/0)   94697856

 6 unassigned    wm       0                0         (0/0/0)             0

 7 unassigned    wm       0                0         (0/0/0)             0

To:

Part      Tag    Flag     Cylinders         Size            Blocks

 0       root    wm       0 -  4740         46.01GB    (4741/0/0)   96488832

 1       swap    wu    4741 -  9423     45.50GB    (4689/0/0)   95430528

 2     backup    wu       0 - 14086      136.71GB    (14087/0/0) 286698624

 3          -    wu       0    0       136.71GB    (14087/0/0) 286698624

 4          -    wu       0    0             19.88MB    (2/0/0)         40704

 5        var    wm    9430 - 14082       45.16GB    (4653/0/0)   94697856

 6 unassigned    wm       0                0         (0/0/0)             0

 7 unassigned    wm       9424   9429      59.62MB    (6/0/0)        122112

·         Label the space disk (c0t1d0) using the format utility

·         Copy the vtoc of the rootdisk to its mirror

# prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t1d0s2

·         Create metadevice db for both disk

# metadb -a -f -c 3 c0t0d0s7 c0t1d0s7

·         Create primary root, swap, var submirror

# metainit -f d10 1 1 c0t0d0s0

# metainit -f d11 1 1 c0t0d0s1

# metainit -f d15 1 1 c0t0d0s5

·         Create secondary root, swap, var submirror

# metainit -f d20 1 1 c0t1d0s0

# metainit -f d21 1 1 c0t1d0s1

# metainit -f d25 1 1 c0t1d0s5

·         Create the mirror from the first disk

# metainit d0 -m d10

# metainit d1 -m d11

# metainit d5 -m d15

·         Backup /etc/vfstab and /etc/system

·         Perform metaroot

# metaroot d0

·         Edit /etc/vfstab and /etc/system

From:

/dev/dsk/c0t0d0s1 - - swap - no -

/dev/dsk/c0t0d0s0 /dev/rdsk/c0t0d0s0 / ufs 1 no -

/dev/dsk/c0t0d0s5 /dev/rdsk/c0t0d0s5 /var ufs 1 no -

To:

/dev/md/dsk/d1 - - swap - no -

/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no -

/dev/md/dsk/d5 /dev/md/rdsk/d5 /var ufs 1 no -

·         Edit /etc/system and add the following line in order for Solaris Volume Manager software to boot up with at least 50% of the db replica copied

set md:mirrored_root_flag=1

·         Configure te dump device

# dumpadm -d /dev/md/dsk/d1

·         Reboot the server

# shutdown -i6 -y -g0

NOTE: If SVM encounters problem after booting (i.e stale metadb), perform "metadevadm -u c0t1d0" to update SVM disk device information. Run "metastat" and verify that SVM can see both disks at the end of the output

·         Attach the secondary submirror to sync data

# metattach d0 d20

# metattach d1 d21

# metattach d5 d25

Post - change:

8. Monitor the server for 3 days

9. If not issue after 3 days, you can change spare disk with the actual submirror in safe and do the process of converting it to metadevice and attaching it to your current SVM configuration. (You can check the previous procedures to accomplish this)

10. We are done. Good luck!

Solaris: Patch Procedure for Mirrored SVM

From DarnPedia

Jump to: navigation, search

NOTE: Please find the following as an example only. You may have some restrictions that will not allow you to follow all step and could lead you to a very bad situation.

Assumption:

Note: metadevice d0, d1, d2 are part of the OS filesystem.

metastat -p

d0 -m d10 d20 1

d10 1 1 c1t0d0s3

d20 1 1 c1t1d0s3

d1 -m d11 d21 1

d11 1 1 c1t0d0s1

d21 1 1 c1t1d0s1

d2 -m d12 d22 1

d12 1 1 c1t0d0s0

d22 1 1 c1t1d0s0

vfstab:

/dev/md/dsk/d1  -       -       swap    -       no      -

/dev/md/dsk/d0  /dev/md/rdsk/d0 /       ufs     1       no      logging

/dev/md/dsk/d2  /dev/md/rdsk/d2 /var    ufs     1       no      logging

Preparation:

- You can use my SOLARIS_patch-infoprep to gather SVM information before doing the patch.

1. Make sure that you have a full backup of all filesystems. In case you corrupt your filesystems, you have the option to revert the server to its working state using the backup copy. Having the latest "explorer" output is good to have too.

2. Verify that the server is in good state before patching by doing a normal reboot of server.

# shutdown -i6 -y -g0

3. After boot-up, make sure that no software/hardware issue were detected before proceeding to the next step. You can check for "fail" and "error" entries in dmesg or messages log. You can also use "iostat -En |grep Hard" for any disk failures.

4. Detach the mirror copies. In case you corrupt your filesystems, you have the option to revert the server to its working state using the mirror copies.

# metadetach d20

# metadetach d21

# metadetach d22

5. Check the integrity of the detached mirror copies except for swap mirror. Fix any data errors, at your own risk :)

# fsck -F ufs /dev/md/rdsk/d20

# fsck -F ufs /dev/md/rdsk/d22

6. Edit the vfstab and system files of the mirror copy. This is a necessary step in order for the server to mount the mirror filesystems properly.

# mount /dev/md/dsk/d20 /mnt

# vi /mnt/etc/vfstab

  /dev/md/dsk/d21  -       -       swap    -       no      -

  /dev/md/dsk/d20  /dev/md/rdsk/d20 /       ufs     1       no      logging

  /dev/md/dsk/d22  /dev/md/rdsk/d22 /var    ufs     1       no      logging

# vi /mnt/etc/system

   rootdev:/pseudo/md@0:0,20,blk

# umount /mnt

7. Reboot the server to check the single-copy primary disk.

# shutdown -i6 -y -g0

8. After boot-up, verify the state of the filesystems (single-copy). Make sure that no software/hardware issue were detected before proceeding to the next step.

# df -k / /var (Must reflect d0 and d2)

# swap -l (Must reflect d1)

# iostat -En

9. Test the detached mirror copies. Go to OBP and boot the server on mirror disk. Assuming "disk1" points to your second copy.

# shutdown -i0 -y -g0

ok? boot disk1

10. After boot-up, verify the state of the mirror disk. Make sure that no software/hardware issue were detected before proceeding to the next step.

# df -k / /var (Must reflect d20 and d22)

# swap -l (Must reflect d21)

# iostat -En

11. Reboot the server to boot the primary disk again.

# shutdown -i6 -y -g0

Start Patching

12. After boot-up, verify the state of the filesystems (single-copy). Make sure that no software/hardware issue were detected before proceeding to the next step.

# df -k / /var (Must reflect d0 and d2)

# swap -l (Must reflect d1)

# iostat -En

13. Go to single user mode and start the patching.

# sync; sync; init s

14. Reboot the server to check the current server after patching.

# shutdown -i6 -y -g0

15. After boot-up, verify the state of the server.

16. DONE

When the patching goes bad

Option 1: Revert back the system state using the mirror copy

1. Go to OBP and boot the server on mirror disk.

# shutdown -i0 -y -g0

ok? boot disk1

2. After boot-up, verify the state of the mirror disks. Make sure that no software/hardware issue were detected before proceeding to the next step.

# df -k / /var (Must reflect d20 and d22)

# swap -l (Must reflect d21)

# iostat -En

3. If dumpadm is using SVM (/dev/md/dsk/d1), replace it first with d21.

# dumpadm -d /dev/md/dsk/d21

4. Clear d0, d1 and d2.

# metaclear d0

# metaclear d1

# metaclear d2

5. Attach d20, d21, and d22.

# metainit -f d0 -m d20

# metainit -f d1 -m d21

# metainit -f d2 -m d22

6. Edit the current vfstab and system files.

# vi /etc/vfstab

  /dev/md/dsk/d1  -       -       swap    -       no      -

  /dev/md/dsk/d0  /dev/md/rdsk/d0 /       ufs     1       no      logging

  /dev/md/dsk/d2  /dev/md/rdsk/d2 /var    ufs     1       no      logging

# vi /etc/system

   rootdev:/pseudo/md@0:0,0,blk

7. Go to OBP and boot the server on mirror disk again.

# shutdown -i0 -y -g0

ok? boot disk1

8. After boot-up, verify that the server is now using d0, d1, and d2.

# df -k / /var (Must reflect d0 and d2)

# swap -l (Must reflect d1)

# iostat -En

9. Attach d10, d11, and d12 to sync their data with a working copy.

# metattach d0 d10

# metattach d1 d11

# metattach d2 d12

10. Complete the syncing before proceeding to the next step.

# metastat |grep sync

11. After sync is complete, detach d20, d21, and d22 to make d10, d11, and d12 the primary copy again.

# metadetach d20

# metadetach d21

# metadetach d22

12. If dumpadm is using SVM (/dev/md/dsk/d21), replace it now with d1.

# dumpadm -d /dev/md/dsk/d1

13. Reboot the server.

# shutdown -i6 -y -g0

14. After boot-up, verify that the server is still using d0, d1, and d2.

# df -k / /var (Must reflect d0 and d2)

# swap -l (Must reflect d1)

# iostat -En

15. Attach d20, d21, and d22 to complete the mirroring again.

# metattach d0 d20

# metattach d1 d21

# metattach d2 d22

16. Complete the syncing before proceeding to the next step.

# metastat |grep sync

17. Reboot the server for the last time (Optional).