Data Center

[Lesson Learned] Nexus 7k Upgrade Issue

Ringkasan troubleshooting Nexus upgrade di salah satu customer beberapa waktu lalu. Beberapa credential disamarkan.

Problem:
  1. OS yang diload setelah restart upgrade tetap OS yang lama (6.2.12)
  2. Image dan kickstart image yang baru (6.2.16) telah dimasukkan ke bootflash.
  3. Image dan kickstart image baru (6.2.16) tidak terupload ke dalam Loader Nexus
  4. Boot variable telah di setting ke OS dan kickstart image yang baru
Problem Verification:

Setelah reload:

N7k2# show version 
Cisco Nexus Operating System (NX-OS) Software
<omitted>
Software
BIOS: version 2.12.0
kickstart: version 6.2(12)
system: version 6.2(12)
BIOS compile time: 05/29/2013
kickstart image file is: bootflash:///n7000-s2-kickstart.6.2.12.bin
kickstart compile time: 12/3/2014 18:00:00 [02/03/2015 09:12:42]
system image file is: bootflash:///n7000-s2-dk9.6.2.12.bin
system compile time: 12/3/2014 18:00:00 [02/03/2015 10:43:00]
N7k2# dir bootflash:
<omitted>
218199431 Jun 23 06:36:44 2013 n7000-s2-dk9.6.1.2.bin
425228450 Aug 26 15:10:31 2015 n7000-s2-dk9.6.2.12.bin
425889673 Feb 09 18:50:43 2017 n7000-s2-dk9.6.2.16.bin
36344320 Jun 23 06:35:07 2013 n7000-s2-kickstart.6.1.2.bin
36633088 Aug 26 15:11:33 2015 n7000-s2-kickstart.6.2.12.bin
36697088 Feb 09 18:50:11 2017 n7000-s2-kickstart.6.2.16.bin
<omitted>
Additional Information

Di loader tidak ada image dan kickstart image yang baru (6.2.16):

Autobooting bootflash:/n7000-s2-kickstart.6.2.12.bin bootflash:/n7000-s2-dk9.6.
2.12.bin...
 Filesystem type is ext2fs, partition type 0x83
Booting kickstart image: bootflash:/n7000-s2-kickstart.6.2.12.bin....
........................................Aborting Image Boot 

Error 2: Bad file or directory type

              GNU GRUB  version 0.97 

                Loader Version 4.1.9

loader> dir bootflash:                                                         

bootflash:
  lost+found
  n7000-s2-kickstart.6.1.2.bin
  n7000-s2-dk9.6.1.2.bin
  <omitted>
  n7000-s2-dk9.6.2.12.bin
  n7000-s2-kickstart.6.2.12.bin
Troubleshooting Process
  1. Siapkan USB berisi OS image dan kickstart image baru (6.2.16)
  2. Load OS image dan kickstart dari USB dari loader
  3. Setelah masuk ke CLI admin, cek show version
  4. Cek di loader apakah OS image dan kickstart baru sudah tercopy ke Loader, agar switch tidak harus load image dari USB dengan cara reboot system dan masuk ke Loader (optional)
Detailed Solution
  1. Masukkan USB ke port USB di switch, set agar boot OS image dan kickstart image dari USB
    loader> dir usb1:                                                              
    
    usb1:
     n7000-s2-kickstart.6.2.16.bin
     n7000-s2-dk9.6.2.16.bin
     n7000-s2-dk9.6.2.12.bin
     n7000-s2-kickstart.6.2.12.bin
    	
    loader> boot usb1:n7000-s2-kickstart.6.2.16.bin                                
     Filesystem type is fat, partition type 0xc
    Booting kickstart image: usb1:n7000-s2-kickstart.6.2.16.bin....
    ..........................................
    .....................................
    ........................................
    Kickstart digital signature verification Successful 
    Image verification OK
  2. Setelah masuk ke CLI, cek version dan boot current
    N7k2# show version 
    Cisco Nexus Operating System (NX-OS) Software
    <omitted>
      BIOS:      version 2.12.0
      kickstart: version 6.2(16)
      system:    version 6.2(16)
      BIOS compile time:       05/29/2013
      system image file is:    bootflash:///n7000-s2-dk9.6.2.16.bin
      system compile time:     1/27/2016 9:00:00 [04/06/2016 05:16:06]
    
    N7k2# show boot current 
    sup-1
    kickstart variable = bootflash:/n7000-s2-kickstart.6.2.16.bin
    system variable = bootflash:/n7000-s2-dk9.6.2.16.bin
  3. Secara otomatis OS image dan kickstart baru sudah ter-copy ke loader, jadi USB bisa dilepas
    N7k2# reload 
    This command will reboot the system. (y/n)?  [n] y
    
    loader> dir bootflash:                                                         
    
    bootflash:
      lost+found
      n7000-s2-kickstart.6.1.2.bin
      n7000-s2-dk9.6.1.2.bin
      n7000-s2-dk9.6.2.12.bin
      n7000-s2-kickstart.6.2.12.bin
      n7000-s2-kickstart.6.2.16.bin
      n7000-s2-dk9.6.2.16.bin
    
    loader> reboot
    
    Autobooting bootflash:/n7000-s2-kickstart.6.2.16.bin bootflash:/n7000-s2-dk9.6.
    2.16.bin...
     Filesystem type is ext2fs, partition type 0x83
    Booting kickstart image: bootflash:/n7000-s2-kickstart.6.2.16.bin....
    ...............................................................................
    Kickstart digital signature verification Successful 
    Image verification OK

Semoga membantu

 

Leave a comment