About Vulscan and SSL Verification

Version 35

    Verified Product Versions

    Endpoint Manager 9.6Endpoint Manager 2016.xEndpoint Manager 2017.x

    LDMS introduced SSL Validation for Client Certificates when communicating with the core via Vulscan. This can sometimes cause errors when clients attempt to verify using the Client Certificates its been given.

     

    This issue is often caused by those who have upgraded their core to 2016 from an older version, or a side-by-side installation with a database import.

     

     

    Issue

    Vulscan hangs on "Contacting Server..." and never proceeds:

    Contacting Server.png

    Troubleshooting

    In order to identify if Vulscan is failing due to Certificate issues, a number of items can be referenced/tested on the Core and Clients.

     

    Disable Client Certificate Validation for WSVulnerabilityCore

    1. Open Internet Information Services (IIS) Manager on the core.
    2. Drill down to WSVulnerabilityCore under Core Name > Sites > Default Web Site.
    3. Double-Click SSL Settings.
    4. Set Client Certificates to Ignore and click Apply.

    SSL Settings.png

       After this change has been made, attempt to run a vulscan on the client machine. If successful, then the issue is indeed involving Client/Core Certificates.

    Note - Disabling Client Certificate Validation on the core is NOT meant to be a permanent change. Ivanti urges all users to keep the SSL Settings set to "Accept."

     

    Reference Core Logs

    Logs under C:\inetpub\logs\LogFiles\W3SVC1 can be reviewed for HTTP status return codes when workstations attempt to contact WSVulnerabilityCore.

     

    2016-08-26 19:23:34 10.25.26.50 POST /WSVulnerabilityCore/VulCore.asmx - 443 - 10.25.26.55 Microsoft-ATL-Native/11.00 403 16 2148204809 358
    2016-08-26 19:23:34 10.25.26.50 POT  /WSVulnerabilityCore/VulCore.asmx - 443 - 10.25.26.55 Microsoft-ATL-Native/11.00 403 16 2148204809 358
    2016-08-26 19:23:34 10.25.26.50 POST /WSVulnerabilityCore/VulCore.asmx - 443 - 10.25.26.55 Microsoft-ATL-Native/11.00 403 16 2148204809 421
    2016-08-26 19:23:34 10.25.26.50 POST /WSVulnerabilityCore/VulCore.asmx - 443 - 10.25.26.55 Microsoft-ATL-Native/11.00 403 16 2148204809 405
    

     

    Things that can be derived from this information:

    • IIS is returning a 403.16 Status Code which indicates "Client Certificate is Untrusted or Invalid."
    • There is also return code 2148204809. This translates to HRESULT 0x800b0109, which is defined by Microsoft as CERT_E_UNTRUSTEDROOT.

     

    Reference Client Logs

    ProxyHost.log under C:\Program Files (x86)\LANDesk\Shared Files on the client will show errors when attempting to reach WSVulnerabilityCore.

     

    2016-08-26 18:49:28(5144-148) proxyhost.exe:IsProcessSigned succeeded - returning: 1
    2016-08-26 18:49:28(5144-148) proxyhost.exe:Made direct (non-proxy) connection to LDCORE2016:443
    2016-08-26 18:49:28(5144-148) proxyhost.exe:Call UpdateCSAROIFile() with numberofDirectConnectSuccess = 1 numberofDirectConnectFailure = 0  csaName =  bCsaSuccess = 1
    2016-08-26 18:49:28(5144-148) proxyhost.exe:127.0.0.1:57652 Connection close 0 0 0 0
    2016-08-26 18:49:28(5144-148) proxyhost.exe:127.0.0.1:57652 - - [26/Aug/2016:11:49:28 -0800] "POST http://LDCORE2016:443/WSVulnerabilityCore/VulCore.asmx HTTP/1.1" 403 651 469
    

     

    • ProxyHost verifies what is already known - a 403 is returned when attempting to reach WSVulnerabilityCore from the client.

     

    Cause(s)

    As the resolution for this issue isn't reserved to one cause, a few things will need to be verified:

    Verify Core Certificates in the Trusted Root Authority Folder

    According to this Microsoft KB Article, any Non Self-Signed certificates installed in the Trusted Root Certification Authority certificate store on the IIS Server may cause an error when users attempt to authenticate when using a Client Certificate.

     

    Non Self-Signed Certificates will show in the CertMgr tool as having different information in the "Issued To" and "Issued by" columns.

    Trusted Root Folder.png

    Note: Expired Certificates may also cause Client Certification issues; with the exception of Microsoft Required Root Certificates

    You can easily determine if this folder contains any non self-signed certs with this PowerShell command:

    Get-Childitem cert:LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject} | Format-List * | Out-File "c:\certlist.txt"

    This command compares the "Issuer" property and the "Subject" property of each certificate in the store and then outputs details of certificates that do not meet the criteria of a self-signed certificate to c:\certlist.txt.

     

    Remove any Non Self-Signed Certificates in the Trusted Root Certification Authority Folder, if found, and attempt running Vulscan again. If the issue persists, continue to next steps.

     

    Verify Core Certificates and Keys in the Shared Keys Folder

    The certificates in the C:\Program Files\LANDesk\Shared Files\keys and/or C:\Program Files (x86)\LANDesk\Shared Files\keys need to be valid and not expired as well. You can double-click the .CRT and open the .0 file in notepad to verify the information:

     

     

    If there are expired certificates in that folder, back them up and remove them from that folder, including any keys and .0 files associated with that cert.

     

    Verify Client Certificates

    The certificates on the clients will need to be checked in this issue. Certificate information on the client is stored under C:\Program Files (x86)\LANDesk\Shared Files\cbaroot\certs.

     

    In this directory, there are usually one to several ".0" certificates. Its possible one or many of these certificates are not trusted by the core, causing the issue. To verify each certificate:

     

    1. Make a copy of each ".0" certificate in a separate location.
    2. Change the extension of each certificate to ".crt" and double-click on them.
    3. Verify that the LANDESK Trusted Certificate they correspond to exists within the Trusted Root Certification Authority on the core.
      Client Certificate.png

    If a ".0" certificate(s) is found to correspond to a LANDESK Trusted Certificate that does not exist within Trusted Root Certification Authority:

    1. Remove the offending certificate(s) from the C:\Program Files (x86)\LANDesk\Shared Files\cbaroot\certs directory, leaving only the "good" certificates.
    2. Delete the contents of C:\Program Files (x86)\LANDesk\Shared Files\cbaroot\broker. (This folder contains the Broker Certificate, which is the certificate used when communicating with the core).
    3. Run "C:\Program Files (x86)\LANDesk\LDClient\BrokerConfig.exe /r" on the client CMD Window. This will generate a new Broker Certificate.

     

    Attempt running Vulscan again. If all steps have been correctly followed, Vulscan should successfully verify with the core and begin scanning.

     

    Resolution

    If the issue was found to be a client-side issue, a couple of more steps will need to be taken in order to fully resolve the situation. One of two options can be executed:

     

    Import "Old" LANDESK Certificate into the Trusted Root Certification Authority

    If a ".0" certificate was found on the client that didn't correlate with a trusted LANDESK Certificate on the core, its likely due to the client being managed in the past by a core that has since been decommissioned.

     

    The LANDESK Certificate(s) from the old core can be imported into the Trusted Root Certification Authority on the core. This will enable the clients to validate with the core using their existing certificates, provided its the right certificate.

     

    If for whatever reason this is not an option, move on to the next step.

     

    Update Client Certificates to Match Core

    Changes will have to be made to the LANDESK Agent Configuration in order to ensure that clients aren't attempting to connect to the core with Untrusted Certificates.

     

    This is configured through the Client Connectivity Agent Settings on the Core under Configuration > Agent Settings > Client Connectivity.

    Client Connectivity.png

    Ensuring that the correctly configured Client Connectivity Settings are included in the Agent Configuration will ensure that all future agent installs will get the correct certificates.

     

    There are a couple known methods to address the existing clients that already have the untrusted broker certificate on them:

     

    Reinstall agent with updated Client Connectivity Settings

    • Since Vulscan is not able to connect to the core, reinstalling the agent will ensure the new Client Connectivity Settings are applied to the client and new Broker certificates are generated.

     

    Use a script/batch file

    • A script or batch file can be used to clear out the contents of the "certs" and "broker" directory. It can then download the correct ".0" cert and run "brokerconfig.exe /r."
    • This script/batch can be deployed via LANDESK Software Distribution. For more information on deploying Batch Files, please see this Community Document.

     

    Check Default Web Site Bindings

    The Default Web Site configured by the LANDESK install will utilize the LANDESK Secure Token Server SSL Certificate for the Site Binding within IIS. This is the recommended configuration.

     

    To verify the SSL Certificate being utilized, open Internet Information Services (IIS) Manager and go to CoreName > Sites > Default Web Site.

     

    From there, click Bindings.

    Default Web Site.png

    This will bring up the Site Bindings window. Highlight port 443 and select Edit.

     

    In the Edit Site Binding window, the SSL Certificate: dropdown will show the currently assigned binding. Click to expand the dropdown list and verify there aren't any Untrusted Certificates in the list.

     

    • Note: Ignore the WMSVC, LANDESK Secure Token Server and LANDESK_CommandsWS here.

    Site Bindings.png

    Untrusted Certificates here pertains to SSL Certificates found in the Binding Dropdown list, but not found installed in the Trusted Root Certification Authority folder as detailed earlier in this document.

    If there aren't any Untrusted Certificates found in the drop-down, the Site Bindings are good. No other work is needed here. If there are, a couple options are available.

     

    (1) Remove the Untrusted Certificate from Server Certificates

    This is the recommended resolution to this issue as it ensures simplicity and eliminates the possibility of old certificates becoming a problem again in the future.

     

    Open Internet Information Services (IIS) Manager and select Corename. Under the IIS section, double-click Server Certificates.

    Server Certificates.png

    Within the Server Certificates window, select the Untrusted Certficate(s) and click Remove and select Yes when prompted.

    Server Certificates2.png

    Run IISReset in an Administrator Command Prompt on the core and Re-attempt vulscan to verify the issue is resolved.

     

    (2) Install the Untrusted Certificate in the Trusted Root Folder so it is Trusted

    If a copy of the Untrusted Certificate shown IIS is available, it can be imported into the Trusted Root Certification Authority folder so it becomes Trusted.

     

    LANDESK Certificates exist under C:\Program Files\LANDesk\Shared Files\keys on the core by default. They may have been moved to other directories by users for various reasons.

     

    Right-click the currently Untrusted Certificate and select Install Certificate. Follow the configuration detailed below:

    Install Certificate.png

    Re-attempt vulscan to verify the issue is resolved.