The Management Gateway becomes unresponsive, the SDA3 partition is full, activation fails, or 503 errors appear.

Version 22

    Subject

    The /SDA3 share on the Management Gateway fills up, dblogger fails, or the Gateway needs to be rebooted from time to time. Also, activation and other applications may have trouble executing.

     

    Description

    Periodically on a handful of Management Gateways the machine needs to be rebooted. The reboot is necessary because applications such as Activation, GSB Web Page, or other functions stop responding. If a Self Test is performed typically the "dblogger" fails. If the status page is checked the /SDA3 share appears full. Other functions of the Gateway may have problems as well. A reboot temporarily fixes the problem.

     

    Cause

    This issue is caused by an overload occurring on the dblogger tunnel. This tunnel funnels logs from different portions of the Gateway into the database. When this overload happens corruption occurs in the Logs table that results in MySQL failing to release INODES. What this means is that applications fail to run because they cannot get an INODE from the kernel. Even though the /SDA3 share shows full there is actually plenty of space and creating a file works fine. This corruption can take a week or more to manifest itself and require the reboot before the process begins again.

     

    Resolution

         Enable MySQL Error Logging:

    1. To change the Configuration file for MySQL run the command:
          
      vi /etc/my.cnf  
    2. Add a line just below "datadir" to read:
           log-error = /home/mysqlerror.log
    3. Check the /tmp folder for permissions. This folder should be 777.
    4. Restart the MySQL Daemon by running the command:
          /etc/init.d/mysqld restart

     

         Non-restart work around:

    From a command line you can call sudo /etc/init.d/mysqld restart

     

         Script controlled work around:

    There is a script that can be  installed on the gateway and set in cron to run hourly, daily, or weekly as is seen fit for the network environment.  This script  will test two conditions and if they are met will restart MySQL. If the issue persists MySQL will attempt to repair the table. If all these attempts fail it will then drop and recreate the table. You can view the code in a text editor and it has documentation explaining each part of the process.

     

    You download the script here: ks-ldcleanup.sh

     

    Instructions for installing the script on to your Management Gateway:

    1. Using a utility such as WinSCP copy the script to the /tmp directory of the Management Gateway.
    2. Using the admin username and password, log into the Command Line Interface using the xterm on the Management Gateway or by using the ssh connection with putty.exe.

      How to use Putty to SSH into the LANDesk Management Gateway Appliance.

    3. Once you are logged into the Gateway type in the following commands.  This will change the ownership, group and permissions to the correct settings for the script.

      sudo chmod 755 /tmp/ks-ldcleanup.sh
      sudo chown root /tmp/ks-ldcleanup.sh
      sudo chgrp root /tmp/ks-ldcleanup.sh

    4. Now that the file has been given the correct permissions, it is time to how often you want the script to run: hourly, daily, or weekly.  This can be done by using the systems internal scheduler CRON.  Cron has 3 folders in the /etc directory where you can place the scriptdepending on the schedule you wish: cron.hourly, cron.daily, or cron.weekly

      Here is an example command for moving the script to the cron.hourly directory.  This way the script will run every hour:

      sudo mv /tmp/ks-ldcleanup.sh /etc/cron.hourly/ks-ldcleanup.sh

      NOTE: If you desire to have more control on when the script runs, you can use this search to find the myriad of tutorials on how to use cron.