How To: Troubleshoot the Cloud Services Appliance (CSA)/Gateway

Version 10

    Verified Product Versions

    LANDESK Management Suite 9.6LANDESK Management Suite 2016.x

    Purpose

     

    A basic troubleshooting guide to help identify the point of failure when agent communications through the CSA fail. This document will only discuss CSA configuration as it pertains to troubleshooting/resolve CSA failures. For configuration information see: https://community.landesk.com/support/docs/DOC-32340

     

    Symptoms


    Anytime agent communications through the CSA fail this document can help identify the point of failure and will contain some of the most common issues surrounding them

     

    Description

     

    The CSA is simply a connection broker. It does not make any outbound connections (with the exception of licensing and CSA updates) it accepts incoming connections and passes data between authorized clients and the core server. Because the CSA doesn’t make outbound connections, it does not rely on DNS, thus any alias’s setup for the core will not affect the CSA. This is an important concept to keep in mind while using/troubleshooting the CSA because once the network is configured there is very little about the CSA configuration that would prevent agents or the core server to connect. It is also important to remember that the connection between core server and CSA is initiated by the core server. See doc https://community.landesk.com/support/docs/DOC-32340 for additional information on configuring the CSA.

     

    Keeping the above in mind and assuming the configuration is good, we can divide the CSA into 2 parts: Client facing and Core Server Facing. We’ll start with the core facing side as it is critical those connections be established before anything else can happen.

     

    Core Server Facing:

    Open the Cloud Services Appliance Connection table from the reports tab

    ss (2014-09-16 at 02.15.44).png

    ss (2014-09-16 at 02.18.08).png

    The Broker service connections are the connections from the core server. Notice the core name is listed for each connection (at least 5 per core) so you can identify the core that is connected if you have multiple core servers connecting to a single CSA.

    If those broker service connections are present in your CSA you can move on to the Agent side section, if not, we will cover a few common issues in the remainder of this section.

     

    If you’re reading this, that probably means those broker service connections aren't present on your connection table. The first thing we want to verify is part of the CSA configuration.

    Within the CSA console browse to the gateway service tab and find the “Additional host names” text box.

    ss (2014-09-16 at 02.20.02).png

    This is a commonly misunderstood and misconfigured section. Essentially, this box should contain any name the CSA could be known as. Typically, we recommend adding the FQDN, internal and external IP addresses. At the very least, this box should contain the same information you used to configure your core server CSA settings. When the core server reaches out to connect to the CSA it checks the name to be sure it’s connecting to the expected CSA. If the name doesn't match the host name field, and is not specified in the ‘Additional host names’ section, the connection will fail.

     

    If you do make a modification to this setting, you’ll need to restart the LANDESK Management Gateway service on your core server, which I will cover in more detail later in this section. Once you restart that service wait at least 5 minutes before you move on, as it can take up to that long to establish those connections.

     

    If you’re still not seeing those connections come online, you’ll want to verify the firewall settings on the gateway. The easiest way to test if this is the point of failure is to just disable the firewall temporarily and see if the core is able to connect.

    ss (2014-09-16 at 02.22.51).png

    If it is determined that the firewall does need to be reconfigured, here are a couple good things to know: The firewall logic is as such(CSA version 4.2 only): Black list then white list then trusted services. Machines in the black list will be blocked even if they are also in the white list. Machines in the white list, and not in the black list, will have full access to the CSA via all services. Machines neither in black nor white lists will only be allowed access via the trusted services. CSA version 4.3 the white list should override the blacklist. I’ve found that it’s helpful adding the specific IP for the core server to the white list and keeping the subnet out of the black list entirely. Firewall settings shouldn’t affect agent communications as they are trusted via client certificates.

     

    If you’re still having problems at this point, you’ll probably want to take a minute to make sure there aren’t any other firewalls or network devices that are blocking communications. If the firewall is turned off you should be able to ping the CSA.

     

    Now we’re going to shift over to the core server for more troubleshooting. First thing here, head over to the broker service log located at "C:\Program Files (x86)\LANDesk\ManagementSuite\log\BrokerService.log" for a 9.5 core (for 9.6 cores: "C:\ProgramData\LANDesk\Log\BrokerService.log") You might consider monitoring this log after restarting the LANDesk Management Gateway service to help determine what’s happening while those connections are being established.

     

    ss (2014-09-16 at 02.31.42).png

    The last common issue we’ll cover in this section is the service account on the CSA. This is especially common as a failure point when there are multiple core servers using the same CSA. The account often will get locked out while trying to reset/change the password either on the CSA itself or at the core server. Best practice is to stop the LANDesk Management Gateway service on the core server, change the password on the CSA, then in the CSA settings on the core. Once the password is set, and you close the CSA configuration window, it should prompt you to restart the gateway service.

     

    To restart the LANDesk Management Gateway service:

    Click ‘Start’

    Type ‘services.msc’

    Find the service labeled LANDesk Management Gateway Service

    ss (2014-09-16 at 02.28.17).png

    Right-click > Restart

    If you’re still having trouble establishing the broker service connections, contact LANDESK Technical Support.

     

    Client Facing

    Once the core broker service connections are established, the agent will be responsible for making the other half of the connection. Most client side troubleshooting will be done in the client application called brokerconfig.exe, and for the purposes of this document there are no additional CSA options that affect the client’s ability to communicate.

     

    The first thing the agent needs in order to send inventory and vulscans through the CSA to the core server, is a core broker certificate. See doc-1888 to automate the process we’re about to discuss.

     

    ss (2014-09-16 at 02.46.45).png

    Brokerconfig.exe is responsible for handling the broker certificate request and is located in the clients LDClient folder. Brokerconfig.exe will also determine the method resident agent uses to connect to the core server. By default broker config is set to dynamically determine if the resident agent should connect directly to the core server or through the CSA; it can also be forced to connect to one or the other.

     

    NOTE: The CSA FQDN should be entered for the CSA name, such as "CSA.LANDESK.com". If you add additional information like "https://CSA.LANDESK.com" you may see "Error 10049 attempting to connect to host" in your test message response on the Certificate request tab.

    ss (2014-09-16 at 02.47.30).png

    When you launch brokerconfig.exe, it is recommended that you right-click and select ‘run as administrator’ in order to avoid any possible file system permissions issues.

     

    You’ll notice that there is a note within the application that tells you when there is or is not a broker certificate present. Even with a certificate present it is still a good troubleshooting step to request a new certificate as this will confirm if that the core is accessible and responsive.

     

    It’s also worth noting here that brokerconfig.exe will require credentials when requesting the broker certificate, but only when the machine is making the request off-core; that is to say when the request has to be sent through the CSA. Any user account that is a member of the LANDESK Management Suite group, see below.

    ss (2014-09-16 at 02.49.08).png

    The broker config test can be very helpful in determining issues with the certificate request. The most common issues in that process relate either to the account used making the request or with the LANDESK Management Gateway service on the core server. We recommend creating a local user account on your core server and assigning it to the LANDESK Management Suite group.

     

    Occasionally the broker service on the core server becomes unresponsive, in which case resetting the LANDesk Management Gateway service would be necessary and check ...\LANDesk\ManagementSuite\brokerreq for unprocessed requests. We’d expect you to need to do this no more often than once or twice a month. If you need to restart that service more often, or if the request fails, please contact technical support.

     

    If the broker certificate request is successful but you’re having problems with other communications, the issue will be more component specific. Here we’re going to split up the client side into 3 separate groups: Inventory Scan, Security Scan and Remote Control.

     

    Inventory Scan

    This information only applies to LDMS 9.5 agents and older. If you’re using a 9.6 agent and have problems sending inventory information through the CSA, please following the steps in the security scan section.

     

    The inventory scanner has a connection string that is set during the agent installation to connect to the inventory service on the core. The CSA is very specific on brokering connections between client and core, this means the core name specified in the connection string must match exactly what the CSA knows the core as. The following steps assume the agent configuration was set to include the start menu items; if they are not visible you’ll not be able to test using the method described below and you’ll have to install a new agent with the recommended resolution below.

     

    Click ‘start>all programs>LANDESK Management’

    Right-Click ‘Inventory scan’

    Copy the text from the Target text box

    Paste the text into text editor, such as notepad

     

    There are 3 switches in the connection string that reference the core server by name. Most often the string uses the Fully Qualified Domain Name (FQDN) of the core server and needs to be changed to use the host name of the server, but the opposite can also be true. Alias’s for the core server will not work, as the CSA does not rely on DNS for connections to the core.

     

    Modify the connection string, apply the change and restart the inventory scan. If the scan still fails please contact Technical Support. If the change resolves the issue you’ll need to change the core name in the agent configuration, rebuild the agent and reinstall the new agent. See the image below.

     

    Security Scan

    The most common issue with the security scanner is identical to the inventory scanner, however the name is configured in the registry. The 9.6 agent’s inventory scanner uses this same registry key to determine the connection string to the core.

     

    Open the registry editor and browse to the following key:

    HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Intel\LANDesk\LDWM

    Modify the value ‘CoreServer’

     

    Usually the value should be the hostname of the core, however the FQDN is sometimes required. For testing, modify the value to use either the hostname or the FQDN, whichever is not already specified.

     

    If the change of the CoreServer value was successful you’ll want to modify the core name in your agent configuration to use the same. See the image below.

     

    If the security scanner is still not able to connect to the core server please contact Technical Support.

     

    If the security scanner can connect to the core for policies and definitions but cannot download packages or software it’s usually due to the configuration of the package location. For software and packages to be downloaded via the CSA the package must be available on the core server, the CSA cannot connect to a preferred server, and must be available via HTTP/HTTPS. The CSA cannot transfer packages via UNC.

     

    If you’re still not able to download packages please contact Technical Support.

     

    Remote Control

    Remote control is a little bit of a black sheep as it’s handled completely differently than security and inventory scanners. On LDMS 9.5 and previous versions, the remote control agent has its own logic to determine its connection method, either Direct or Gateway mode 9.6 agents will rely on resident agent and broker config. I highly recommend configuring your remote control settings to keep the System Tray icon always visible, at least for any agents that will ever use the CSA to connect.

     

    The remote control service will only attempt to determine its connection method when the service starts. For instance, if the laptop boots up while it has a local connection to the core server, or if the core server name can be resolved via DNS, the service will start in Direct mode. If the laptop is then moved off-core but is not rebooted, the remote control service will stay in Direct mode and will not be controllable via the CSA. This commonly provides issues for people, as laptops are suspended and not rebooted when they’re moved, which is why having the remote control system tray icon will allow users to switch modes without rebooting the machine.

     

    The second most frequent point of failure, when remote control is concerned, is related to port 80 being closed between the CSA and the internet. Currently, both ports 80 and 443 are required to be open between the internet and the CSA in order for the remote control agent to automatically switch modes.

     

    A list of error codes you could see in the Remote Control viewer are below:

     

    Error Code

    Description

    0

    StartSession Successful

    1

    Timeout

    2

    Parameter error

    3

    Communication error

    4

    End of file (SSL Stream closed)

    5

    Unexpected content in protocol

    6

    Unable to resolve host/proxy

    7

    Proxy authentication required

    8

    Unrecognized proxy communication

    9

    Error creating SSL session

    10

    SSL Handshake failed

    11

    Peer Certificate not yet valid

    12

    Peer Certificate expired

    13

    Zero Depth Self Signed Certificate

    14

    Certificate common name does not match host name

    15

    This is not a trusted cert chain

     

    If you’re still having trouble with Remote Control, please contact Technical Support.

     

    * UPDATE *

     

    The following steps will help you verify the CSA's connection to our patch server. As per community doc How are the updates checked and downloaded on the Cloud Services Appliance?

     

    1) Configure the appliance to make it able to resolve the hostname patch.landesk.com. This is possible in two ways:

     

         1.1) Configure a DNS server in the System | Network settings section.

         1.2) Manually specify the ip address/hostname in the System | Host names section.

     

    2) Verify that patch.landesk.com is accessible on the tcp port 443, as the list of the available updates is downloaded via https. If there is a firewall between the appliance and the internet, verify that the https protocol in enabled from the Cloud Services Appliance towards the internet.


    To do so, open a terminal on the appliance's local console pressing CTRL+ALT+F2 or access the device via SSH from a remote system and try to download a file via https from patch.landesk.com as shown in the screenshot. In the example we are downloading a gif image pointing to https://patch.landesk.com/index_files/image002.gif

     

     

    csa_check_patch_https.png

     

     

    3) Verify that patch.landesk.com is accessible on the tcp port 80, as the updates are downloaded via http. If there is a firewall between the appliance and the internet, verify that the http protocol in enabled from the Cloud Services Appliance towards the internet.

     

    To do so, repeat the same procedure as per the https protocol, pointing at the same file via http: http://patch.landesk.com/index_files/image002.gif