Problem / Solution: ESX server records are missing after VMHA audit

Version 1

     

    Error message:

    In some cases there may not be an error.  In other cases the error below may appear.

     

    Error on Message Queue Journal entry:

    Unable to process LANProbe message for clientId=423AB3B3-70236D25-04FB982F-387B491C, see AssetProcessor log for more details, exception: System.InvalidOperationException: Sequence contains more than one element at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source) at SaaS.IM.DataAdapter.DataAccess.CiAdapter.GetComputerByCIField(String field, String value, Boolean checkAgentNeverInstalled) at SaaS.IM.DataAdapter.AssetDataImporter.AddDiscoveryData(XDocument assetData, String orgUnit, String subTenant, String& error) System.InvalidOperationException: Sequence contains more than one element at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source) at SaaS.IM.DataAdapter.DataAccess.CiAdapter.GetComputerByCIField(String field, String value, Boolean checkAgentNeverInstalled)

    Additional detail in the AssetProcessor Logs:

    <log4net:event logger="FrontRangeSaaS" timestamp="2018-01-09T11:55:43.224938-05:00" level="ERROR" thread="40" domain="/LM/W3SVC/2/ROOT/AssetProcessor-1-131599876614291946" username="{APPPOOLACCT}"><log4net:message>AssetDataImporter.AddDiscoveryData- Discovery item [{FQDN}] [ESX] [] [{MACID}] caused an exception: Sequence contains more than one elementSequence contains more than one element</log4net:message>

     

    Problem:

    Not all ESX servers are showing up in ISM after VMHA audit.  The Message Queue Journal shows all the desired servers in the decompressed message but there is also an error present.

     

    Cause:

    This error is caused when the ISM database already has some ESX entries and they have a duplicate serial number on more than one.  When the Asset Processor attempts to match on Serial Number more than one record is returned from the database.

     

    Solution / Workaround:

    ISM Discovery expects ESX servers to have unique Serial Numbers and will try to match on that criteria.  Duplicates in the incoming data or that already exist in the database will cause ESX servers to fail to import.  In some cases where the incoming data has duplicate you may also see those ESX servers taking over the one matching CI record.  This will also appear as missing data until you closely examine the Audit History on the CI.ESX records and see frequent FQDN / hostname updates changin the value back and forth between two or more ESX servers who share a serial number.

     

    The ultimate fix for these situations is to make certain no two ESX servers share a serial number and then allow VMHA to re-audit them.