VMware Virtual Hardware Downgrade
Closed     Case # 10016     Affiliated Job:  New Trier Township District 2031
Opened:  Wednesday, October 21, 2009     Closed:  Monday, November 2, 2009
Total Hit Count:  23790     Last Hit:  Wednesday, April 17, 2024 3:59:15 PM
Unique Hit Count:  6123     Last Unique Hit:  Wednesday, April 17, 2024 3:59:15 PM
Case Type(s):  Server, Vendor Support
Case Notes(s):  All cases are posted for review purposes only. Any implementations should be performed at your own risk.

Problem:
After upgrading to VMware vSphere version 4.0, we learned there was a known bug with vSphere which affected web services on Windows servers. It was explained we would need to downgrade the virtual hardware of the affected guest sessions from version 7.0 down to version 4.0. The approach for downgrading virtual hardware (as explained by VMware support) was to perform a v2v (virtual to virtual) migration. This approach meant the guest session would be online during the process and because it is a Shadow Copy - this process could take quite some time with the possibility for the loss of data.

Action(s) Performed:
Total Action(s): 1
Action # Recorded Date Type Hit(s) User Expand Details
10061 2/16/2010 1:52:15 PM Server 3320 contact@danieljchu.com I decided to review an alternative, I found you could create a new guest VM  Collapse ...
Last Hit: Wednesday, April 17, 2024 1:55:53 PM

I decided to review an alternative, I found you could create a new guest VM and select to create it with version 4.0 virtual hardware. So after some testing, I created a plan to perform the downgrade with the guest session offline and the entire process could require as little as 20 minutes downtime. I wrote the procedure up to our VMware tech and they approved the approach.

-   Clone VM guest while online (Backup of VM)
-   Create new VM guest "GUEST-v4" with version 4 hardware
   o   Only associate the C drive during creation
-   Replace GUEST-v4.vmdk with GUEST.vmdk (this is the vmdk config file, the "drive" is labeled GUEST-flat.vmdk)
   o   perform this and the below for all associated hard disk vmdk's
   o   ...coping the additional vmdk's for the other hard disks as needed
   o   ...this must be performed through PuTTY or the local console of the vSphere host server - the vSphere client will not display both the config and "drive" files individually
-   Update all drive config GUEST-v4.vmdk files
   o   ddb.virtualHWVersion (change to "4")
   o   File name change to GUEST-v4-flat.vmdk
-   Update GUEST-v4.vmx (this is the guest VM config file - not the "drive" config file)
   o   Physical adapter address to reflect original MAC address
-   Remove GUEST-v4-flat.vmdk (from the newly created VM guest, this vmdk should be empty since it has never been used)
-   Uninstall VMTools & Power off the original VM guest to be downgraded
-   Move the original GUEST-flat.vmdk to the new GUEST-v4 folder
-   Move any additional VMDK's as needed
-   Bring VM guest GUEST-v4 online & install VMTools
-   Set IP address of guest session
-   Using below commands, open Device Manager - View/Show HiddenDevices - and run through each section removing (Uninstall) the grayed items
   o   Open a Command Prompt
   o   set devmgr_show_nonpresent_devices=1
   o   start devmgmt.msc
-   Final restart of guest session

Resolution:
After a couple evenings, during the days I cloned the original guest and created the new guest profiles; in the evening I took down the original guest and transferred the instance as described in the action item above. In total, we incurred about 20 minutes of downtime per server and have since not had difficulties under version 4 virtual hardware operating on vSphere version 4.0. It is my understanding the second U1 to vSphere version 4 has resolved these issues and the new VMTools along with the update address the web service problems previously described. We retained the original VM guest profiles and simply move the drives back to the original profile path - updating the ddb.virtualHWVersion in the config vmdk file back to "7" and the path as required as well as changing any settings such as upgraded memory/CPUs etc that may have changed since the original profile was last used.

10/15/2010 Update: Last week we underwent the 4.1 upgrade, at which time I felt safe after a few of the servers I had switched back to version 7 virtual hardware have responded well while on U2 for v4.0, so we went ahead and upgraded these remaining servers back to version 7 virtual hardware. All our servers have been working well and have not demonstrated these issues mentioned above. Two of our servers operating TomCat really struggled last time after the upgrade and thus were downgraded, this time around, they are operating absolutely perfectly. Make sure to track those MAC addresses of the cards when switching virtual hardware sets, because if that MAC address does change, any depending services, such as license agents, may become inoperable until they are re licensed for the new MAC address. This can be easily averted simply by recording the MAC address of the virtual NIC as well as any locally administered MAC address set on the adapter in the operating system.



Profile IMG: Footer Left Profile IMG: Footer Right