This guest post is by Steven Kang who blogs at ssbkang.com, where you can find his back catalogue of posts. Find out more about the guest blogger program here.
INTRODUCTION
I’ve worked on rebuilding a vCenter server due to one of the services failing to start and in this blog post, I will try and assist people who is planning to rebuild the existing vCenter server.
Rather than detailed process, it will go through:
1. Preparation
What to prepare before rebuild?
2. High Level Process
3. Roll Back Process
4. Post-Work
Note that the work was done based on the environment stated below. If it differs to your environment, it might not fit for you. However, this could be a good reference for your preparation.
VSPHERE ENVIRONMENT
The following list is the products in place:
1. vCenter server 5.5, no update
2. Remote Windows SQL database running on 2008 R2
3. Remote Single Sign On 5.5 server
4. VMware vCenter Heartbeat 6.6
5. Update Manager 5.5
PREPARATION
There are a number of elements to prepare and check.
Note that the list below is not in priority.
Element
Justification
Check Box
SSO Administrator Password
While installing VMware Inventory Service, Web-client and vCenter server, SSO administrator is required
ODBC Detail
This is to connect new vCenter server to the existing database
The domain user with right permission to add Windows server to the domain
Domain user is required to join the new Windows server to the domain
Backup SQL database
In case anything goes wrong, the database could be restored from the recent backup
Disable monitoring
To avoid false positive alerts due to vCenter server being down, ensure monitoring is disabled
Backup roles & permissions
For this work, there are excellent PowerCLI scripts to export/import roles and permissions written by Alan & Luc.
The scripts could be found below:
1. import => http://blog.vmote.net/documents/Import-vCenter-Permissions.ps1
2. Export => http://blog.vmote.net/documents/Export-vCenter-Permissions.ps1
Backup license keys
In case This could be done by export functionality under Home -> Licensing
VMware vCenter Server & Heartbeat license key
Ensure license keys are in place
ESXi root passwords
Root passwords for ESXi servers are required as all of them will be disconnected from the vCenter server due to the new SSL certificate generated
Disable HA and set DRS to manual on all clusters
Since ESXi servers need to be re-connected, it is recommend to disable HA and set DRS to manual
HIGH LEVEL PROCESS
1. Rename existing vCenter server virtual machines on the inventory (run storage vMotion in order to rename the backend files as well).
2. Un-join existing vCenter server virtual machines from the domain and power-off
3. Deploy a VM and configure network settings
4. Join it to the domain
5. Install SQL Native Client
6. Configure ODBC connection
7. Install vCenter server 5.5
8. Set vCenter server services to manual
9. Power-off the VM and using VMware Converter, clone this machine
10. Install VMware vCenter HeartBeat, documentation could be found here
ROLL BACK PROCESS
1. Un-join newly built vCenter servers from the domain and power-off
2. Join the old vCenter servers back to the domain
3. Start VMware vCenter Heartbeat group
POST-WORK
A number of post works need to be accomplished. This is because the SSL certificate of the vCenter server has been replaced with a new one.
1. SSO needs to be cleaned-up once the vCenter is replaced. vSphere web-client will warn you that it failed to verify vCenter server’s SSL certificate
Steps
- Could be found in this KB
2. All ESXi servers will be disconnected from the vCenter server and they will have to be re-connected.
Error message: “Disconnected from host. Reason: Failed to decrypt password”
Steps
- Right click ESXi server and connect
- Enter root / password
- Accept new SSL certificate
3. Re-register Update Manager
Steps
- Login to Update Manager VM and run cmd
- Run C:\Program Files (x86)\VMware\Infrastructure\Update Manager\VMwareUpdateManagerUtility.exe
- Login with administrator and click re-register to vCenter server
- Restart VMware Update Manager Service
- Login to vSphere client and enable plug-in
4. Re-enable HA and set DRS to fully automated or partially automated
5. Import license keys in if they are missing
Didn’t happen, keys remained
6. Import role and permissions in if they are removed
Didn’t happen, roles & permissions remained
WRAP UP
The rebuilding process is quite simple if the preparation work is done correctly. Plan it out well and it will have no problem. Note that the process would be much simpler if the existing SSL certificate could be used.
If you have specific questions, please ping me.
Hope this helps.