9 Replies Latest reply on Jul 19, 2018 3:15 AM by phoffmann

    Patch Detected Linux Vulnerability

    Rodlin Rookie

      With LDMS 2017-3 with Security Suite,

      - it could detect my managed CentOS6.5 endpoint's vulnerability based on downloaded definitions

      select_two_detected_vulnerability_to_repair.png

       

      - by selecting two detected item for Repair (patching the vulnerability) by schedule a task with selected target and start now

      - the schedule task did not succeed and just hanged

      - check on yum deposit site on the endpoint by "ymu -v repolist", it was pointed to the local mirror site for CentOS

      [[email protected] Rodlin]# yum -v repolist

      Loading "fastestmirror" plugin

      Loading "refresh-packagekit" plugin

      Loading "security" plugin

      Config time: 0.033

      Yum Version: 3.2.29

      Loading mirror speeds from cached hostfile

      * base: ftp.ksu.edu.tw

      * extras: ftp.ksu.edu.tw

      * updates: ftp.ksu.edu.tw

      Setting up Package Sacks

      pkgsack time: 0.033

      Repo-id      : base

      Repo-name    : CentOS-6 - Base

      Repo-revision: 1530286202

      Repo-updated : Fri Jun 29 23:37:23 2018

      Repo-pkgs    : 6,713

      Repo-size    : 5.5 G

      Repo-mirrors : http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=os

      Repo-baseurl : http://ftp.ksu.edu.tw/pub/CentOS/6.10/os/x86_64/ (9 more)

      Repo-expire  : 21,600 second(s) (last: Tue Jul 17 14:13:20 2018)

       

      Repo-id      : extras

      Repo-name    : CentOS-6 - Extras

      Repo-revision: 1530611555

      Repo-updated : Tue Jul  3 17:52:37 2018

      Repo-pkgs    : 31

      Repo-size    : 12 M

      Repo-mirrors : http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=extras

      Repo-baseurl : http://ftp.ksu.edu.tw/pub/CentOS/6.10/extras/x86_64/ (9 more)

      Repo-expire  : 21,600 second(s) (last: Tue Jul 17 14:13:26 2018)

       

      Repo-id      : updates

      Repo-name    : CentOS-6 - Updates

      Repo-revision: 1531498395

      Repo-updated : Sat Jul 14 00:15:24 2018

      Repo-pkgs    : 43

      Repo-size    : 539 M

      Repo-mirrors : http://mirrorlist.centos.org/?release=6&arch=x86_64&repo=updates

      Repo-baseurl : http://ftp.ksu.edu.tw/pub/CentOS/6.10/updates/x86_64/ (9 more)

      Repo-expire  : 21,600 second(s) (last: Tue Jul 17 14:13:26 2018)

      repolist: 6,787

      [[email protected] Rodlin]#

      - checking the property for the detected vulnerability found the actual installation command "yum -y update <package_name>"

      detected_vulnerability_property_install_command.png

      - tried the "yum -y update" command for the detected RPM name and found not available

      [[email protected] Rodlin]# yum -v -y update nss-tools-3.28.4-1.el6_9.x86_64.rpm

      Loading "fastestmirror" plugin

      Loading "refresh-packagekit" plugin

      Loading "security" plugin

      Config time: 0.032

      Yum Version: 3.2.29

      rpmdb time: 0.000

      Setting up Update Process

      Setting up Package Sacks

      Loading mirror speeds from cached hostfile

      * base: ftp.ksu.edu.tw

      * extras: ftp.ksu.edu.tw

      * updates: ftp.ksu.edu.tw

      pkgsack time: 5.357

      No Match for argument: nss-tools-3.28.4-1.el6_9.x86_64.rpm

      No package nss-tools-3.28.4-1.el6_9.x86_64.rpm available.

      No Packages marked for Update

      - with the URL found by the "yum -v repolist" command and found only the reversion 3.36 available in base OS (not in updates)

      available_nss_rpm_at_mirror_site.png

      Then the major questions were

      - whether the available reversion 3.36 could fix the vulnerability of 3.28

      - same nss-sysinit rpm with multiple reversions reported/detected, should they all be fixed by the newest patch

      - what should be the correct RPM name that needed to be updated

      - who should make sure the required RPM patch would be available

      - who was responsible to maintain the RPM list for vulnerability patch...

       

      Please advise and/or any comments or suggestions....

       

      Thanks..

        • 1. Re: Patch Detected Linux Vulnerability
          phoffmann SupportEmployee

          So a few things in quick-fire mode (as I'm shot). More detail tomorrow if needed.

           

          Who is responsible for maintaining the YUM repo ... well - that'd be "you" / "your org". "Someone" at your organisation is going to (I would hope?) have to make sure that the local YUM repo is kept up to date?

           

          If in doubt, you *CAN* use the "internat based" YUM repo's (since this is CentOS - it's how I demo patching on my lab kit, for instance). That's not on us - we just "point at YUM" and the rest is up to YUM itself (so to speak) / infrastructure.

           

          ===============

           

          "Multiple vulnerabilities" will be fixed / marked as fixed if you patch to "the latest". We provide "a lot of detail", because not everyone can install "to the latest version" in all cases, and in this case, "information is ammunition" (to either force app devs to get their act in gear / set some sort of corporate policy, etc).

           

          So yeah, if you patch (for instance) to Firefox version 60, the fact that you were vulnerable (hypothetically) for Firefox version 2 -> 59 would "all go away" after you next run the vulnerability scanner.

           

          Does that make sense / address the most burning issues?

           

          =================

           

          If you check in the "DETECTING THE PATCH" section of the vulnerability, you'll see what minimum version is needed to fix that specific vulnerability.

          • 2. Re: Patch Detected Linux Vulnerability
            Rodlin Rookie

            So, based on your comments

            - the public RPM update deposit is based on the best-efforts like the Internet

            - the private RPM update deposit should be the responsibility of each organizations

            - as you mentioned that the property page for each vulnerability had the minimum version below was one sample

            - however, the whether this  minimum version minimum version minimum version RPM was available still really depending on each repository...

            (for examples, the public repository that I had total 43 update PRM's but it did not have the selected sample vulnerability)

             

            • 3. Re: Patch Detected Linux Vulnerability
              phoffmann SupportEmployee

              Most corporate internal repos I've seen tend to be updated pretty regularly ... not sure what the policy is in your place / whether someone just forgot for a bunch of months, or some script broke & no alerting happened or whatnot, but yeah - usually an internal repo should be (vaguely) attempting to grab the newer stuff.

               

              You *USUALLY* don't necessarily care about having "every single" version of *every single* package ... that can vary on some more high-impact things though. What qualifies as "high impact" depends on your org, but common things tend to be among:

              • Java & friends. Various (usually - badly written) "business critical" in-house apps require "specific version X of Java and none other" ... which is a common security nightmare.
              • Specific version of OpenSSL ... another security nightmare if that can't be updated.

               

              ... and so on. So it depends on your org.

               

              In principle, if your NIX machines are fine with it, there's nothing wrong with "just" pointing them at the WWW and running "YUM UPDATE" ... and leech down everything that's up to date.

               

              Now - ONE gotcha to be aware of ... we'll STILL list older kernel versions as being vulnerable (because they're still on the machine, and thus "technically vulnerable") ... so if you want those gone, you'll need to clear those up once they're not needed any more.

               

              But other than that, it's down to "you" (i.e. - "the org" / "the patching startegy") to decide at what patch level your devices should run. We will just make you aware of what list of risks you're potentially exposing yourself to, in effect, until you can get around to patching those devices.

               

              Does that help?

              • 4. Re: Patch Detected Linux Vulnerability
                Rodlin Rookie

                Actually I was testing the LDMS Security Suite's patching capability for Linux endpoints, so

                - there was no private repository for patch update available and forcing me to choose the public ones

                - and Repair Task all got struck any trying to dig out why

                - checking on the Property of the detected vulnerability found the Patch Install Command of "yum -y update <package_name>"

                - whether there was any additional logfile and/or trouble-shoot instructions on how to figure why there Repair tasks failed...?

                • 5. Re: Patch Detected Linux Vulnerability
                  Rodlin Rookie

                  Based on your comments

                  - the minor difference of revisions did not effect the over-all patch RPM availability

                  - because on the LDMS the requested patch revision did not always matching available one(s) on the repository (in my case, the available RPM version is newer than the requested ones)

                  - should the Repair Task could work properly under your comment

                  - or the "yum -y update <package>" command could work in a similar way...

                  • 7. Re: Patch Detected Linux Vulnerability
                    phoffmann SupportEmployee

                    It would help if you'd structure questions a little more coherently - at the moment it seems a little going from place to place, so it's like trying to hit a moving goalpost as it were.

                     

                    It's hard to discern which of the above is "comments in a stream of consciousness" kind of way and/or whether there's clear questions attached. I'll answer as best as I can identify / figure questions out:

                     

                    First up - on "using the existing, in-OS patching tool".

                     

                    We don't have a patch repo of "our own" for NIX-land as we do for Windows, for various reasons:

                    • various NIX distros (RHEL, SUSE for a start) REQUIRE a legal agreement / support agreement & server registration before you're allowed to patch those. So we're not going to be "enabling" potentially illegal activity. That's one.
                    • "Dependency Hell" - patching NIX systems is "a bit more complicated" than patching Windows systems overall due to the aforementioned dependencies. Most NIX Distros have dependencies "figured out already", so there's no real reason to re-invent the wheel if we can leverage existing, in-OS solutions anyway (we use the OS-specific updater, so Ubuntu uses APT-GET and so on for instance).
                    • ... most folks who are serious about NIX tend to already have in-house repos already anyway, so there's little reason not to use the infrastructure anyway. Most folks tend to keep in-house application / packages on them as well, so you can cover them with custom vulnerabilities too.

                     

                    ... so I hope that helps / explains why we're using "in-OS" patching mechanisms ... there's no need to re-invent the wheel & require people to have more infrastructure, if there's perfectly functional solutions already. Plus, the existing approach doesn't require anyone to re-learn anything new ... the existing YUM configuration (or "insert distro-secific patch tool here" whatever) will already apply. Go you.

                     

                    =================

                     

                    In terms of log-files & such. Yep - we have those. Have a look at the following -- "/opt/landesk/logs/" -- where you'll see for example:

                     

                    Now - important reminders:

                    • Remember that the 2017.3 based agent has been re-written from the ground up to be "very linux philosophy friendly". That has implications on various defaults that you will need to change if you want us to be able to patch a box. These include
                    • We don't automatically have access as "root" to the box. You'll need to add the "landesk" user to the SUDOERS file.
                    • You will want to allow "elevate privileges" for specific actions (such as inventory for instance) in the various .CONF-files. For more information see here -- 2017.3+ Linux Agent Conf Files -- which will be quite important. This is significant as "not being allowed to escalate" - while Linux friendly - does have impact. For instance, we can't pick up data on memory banks as part of the inventory scan without elevated privileges - so it affects Inventory as much as other "more obvious" components such as patching / installing software.
                    • On a related note, you can turn on "global debug" logging by un-commenting the "loggingLevel=Debug" line in the LANDESK.CONF (under "/opt/landesk/etc/") and/or you can run the policycheck with verbosity enabled for instance (add "-V" to the command line).

                     

                    There's 2-3 steps involved in patching a machine.

                    • Step 1 (optional) - that's "PUSH"-ing the policy out to devices (i.e. "go check for policies").
                    • Step 2 - client needs to check for policies (running "map-fetchpolicy")
                    • Step 3 - assuming it picked up the policies (and thus the "oh, you need to install X and Y" parts), it'll execute the policy using the software distribution components (who take care of "downloading & running things").

                     

                    • Steps 2 & 3 log their stuff into "/opt/landesk/logs/" -- while "comms with the agent" (i.e. - CBA) gets added/logged into "/var/log/messages/"). That'll get moved to its own dedicated log "at some point" but is quite a bit of effort to change in CBA.

                     

                    You may also want to look at / read the following articles for more information:

                     

                    As a general note - what SU (Service Update) are you on? You'll want to be on at least SU3 for 2017.3 as that includes a few fixes / enhancements.

                     

                    If you're not sure what you're on, check the Console on your Core Server for this part here... (I'm running a limited release patch, in case you're wondering).

                    • 8. Re: Patch Detected Linux Vulnerability
                      Rodlin Rookie

                      Thanks for the comments and recommendations...

                      I got a notification on the LDMS Core Server 2017-3 had issue patching the detected vulnerability on managed Linux endpoints.  

                      And I downloaded and re-installed the LDMS 2016-3, it could patch up the detected vulnerability on the same Linux endpoints (for available RPM on the repository)..

                      So, my issue

                      - caused by bugs on LDMS 2017-3

                      - available of RPM on the repository

                      - if the above issue were not existing, the Patch/Repair could work properly...

                       

                       

                      • 9. Re: Patch Detected Linux Vulnerability
                        phoffmann SupportEmployee

                        So an important point is that the 2016.x and the 2017.3 agents are COMPLETELY different (2017.3 agent was re-written from the ground up). We're talking apples & oranges here.

                         

                        There was a defect with the inital release of 2017.3 that was along the lines of "you can't fix more than 10 vulnerabilities at a time" which was fixed around SU3 ... since there were other things fixed, the "get patched up because this is a brand new agent" is just normal, sensible advice - since you didn't list what service update you were patch ed up to (if any).

                         

                        In your case, your problems are because your repo doesn't have the packages it should have - not because of defects on our side. As you could see, the client TRIED to do the right thing - just that your repo didn' have the required RPM for it to be able to patch. That's "not a bug" at that point  that's "working as would be logical with the conditions provided".

                         

                        If you want to point one of your (test/dev ?) systems at the "internet facing YUM servers" for instance, instead of your local repo - you should be able to validate patching with that? (My thinking being that it should be less likely to run into issues of "I don't have this package available" there). If that works as you expect it to, then the rest - as it were - is "just YUM related stuff" (local config & syncing your local repo some).

                         

                        Does that help / make sense?