vSphere External Solutions i.e SRM, vCloud Director, Support Assistant SSL Replacement

vSphere External Solutions i.e SRM, vCloud Director, Support Assistant SSL Replacement

If you use many of the external VMWare external solution you may be hit with the issue that they can’t connect after replacing SSL certificates, see for further details

http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=2109074

The KB is quite confusing this details just updating when you have external PCS acting as Intermediate trusted root with Organizational Root CA external i.e Microsoft CA. I don’t think it is too clear whether you need to do the PSC and VC or just the PSC.

I hit the vCenter Support Assistant error (see bottom of the post) and I was able to fix it by running the following just on the external PSC I have.

• Log in to the External Platform Services Controller Appliance via SSH.
• Run this command to enable access the Bash shell:shell.set –enabled true
• Type shell and press Enter.
• Run this command to get the current sslTrust anchor stored for the Platform Services Controller:
  • /usr/lib/vmidentity/tools/scripts/lstool.py list –url https://localhost/lookupservice/sdk –no-check-cert –ep-type com.vmware.cis.cs.identity.sso 2>/dev/null
• This will return the following which I have shortened, but the important section is from SSL Trust onwards
  • 
• Then run, output agin has been shortened but you will see the following under begin Certificate not the SSL Trust and these are different values
  • echo | openssl s_client -connect localhost:443
  • 

  Retrieving the Old Certificate from the Managed Object Browser (MOB)
  

• Create the directory on local workstation and the PSC
  • C:\certificates
  • /certificates
  • Open the PSC Mob browser
    • https://psc.example.com/lookupservice/mob?moid=ServiceRegistration&method=List
    • Login as SSO Admin when prompted
    • Click on retrieveservicecontent
    • Click on Invoke Method
    • Click on “Service Registration”
    • Click on “list”
    • Amend the filtercriteria box to <filtercriteria></filtercriteria>
    • 
    • Click on Invoke Method
    • Lookup the value, https://fqdnpscserver/sts/STSService/vsphere.local
    • Then copy into a text document and save to c:\certificates\old_machine.txt the ArrayofString value above
    • 
• Open the old_machine.txt file and it will be one long string this need to converted into lines that are only 64 lines long, then add —–BEGIN CERTIFICATE—– and —–END CERTIFICATE—–
  • Save the file as old_server.crt

  Extracting the Thumbprint from the Old Certificate
  

• Open the file with a certificate viewer tool. In Windows, double-click the file to open it in Windows Certificate Viewer.
• Get the SHA1 Thumbprint string. In Windows Certificate Viewer, select the SHA1 Thumbprint field.

• Copy the thumbprint string into a plain text editor and replace the spaces with colons or remove the spaces from the string. Save as cl;\certificates\sha1-tp.txt

  Note: With some text editors, invisible characters are added at the beginning. Delete the first character of the thumbprint and any associated spaces, then type, not paste, the character.

  Retrieving the New Certificate
  

• Log in to the vCenter Server or External Platform Services Controller Appliance through console or SSH session.
• Run this command to enable access the Bash shell:
  shell.set –enabled true
• Enter shell and press Enter.
• Run this command to view the new certificate:
  /usr/lib/vmware-vmafd/bin/vecs-cli entry list –store MACHINE_SSL_CERT –-text
• Run ths command to export the new certificate to a file:
  /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert –store MACHINE_SSL_CERT — alias __MACHINE_CERT –output /certificate/new_machine.crt

  Running ls_update_cert on the Appliance
  

  Before completing this ensure you have a valid snapshot of the PSC and VC Appliances
  

• Log in to the External Platform Services Controller Appliance through console or an SSH session.
• Run this command to enable access the Bash shell:
  shell.set –enabled true
• Type shell and press Enter.
• Change directories to /usr/lib/vmidentity/tools/scripts/ with the following command:
  cd /usr/lib/vmidentity/tools/scripts/
• Run this command:
  python ls_update_certs.py –url Lookup_Service_FQDN_of_Platform_Services_Controller –fingerprint Old_Certificate_Fingerprint_from_Task_2 –certfile New_Certificate_Path_from_Task_3 –user Administrator@vsphere.local –password “Password“
• Then it will complete and show the following
  
  • 
    

Errors Seen include

• vSphere ReplicationUnable to obtain SSL certificate: The vCenter Server vCenter_FQDN is not correctly registered in LookupService
• vCenter Site Recovery ManagerSRM server with GUID GUID of vCenter not paired.
  Failed to connect to vCenter Server at vCenter_FQDN:443/sdk. Reason:
  com.vmware.vim.vmomi.core.exception CertificateValidationException: Server certificate chain not verified.
• VMware NSX for vSphere (NSX-v)NSX Management Service operation failed.(Initialization of Admin Registration Service Provider failed. Root Cause: Error occurred while registration of lookup service, com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified)
• VMware Integrated OpenStackConnection failed!
  Please check whether the server has enabled SSO from management server log at:/installer.log.In the VMware Integrated OpenStack installer.log file, you see entries similar to:

  [2015-04-10 14:49:18,848 main ERROR com.vmware.vim.install.impl.AdminServiceAccess] com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
  [2015-04-10 14:49:18,849 main DEBUG com.vmware.vim.install.impl.AdminServiceAccess]
  com.vmware.vim.sso.admin.exception.CertificateValidationException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified

• VMware vCenter Support AssistantSomething failed. Try Again.
  com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
  Server certificate chain not verified
  peer not authenticated

• VMware Customer Experience Improvement Program

  The vSphere Web Client reports:

  Error occurred while processing request. Check vSphere WebClient logs for details.

  The vsphere_client_virgo.log reports an error similar to:

  [2015-10-07T13:08:41.001Z] [ERROR] http-bio-9090-exec-3 70000101 100009 200004 com.vmware.vsphere.client.ceip.impl.CeipServiceImpl Error occurred in showNotification. com.vmware.vim.binding.vmodl.fault.SystemError: Internal server error.