7 Replies Latest reply on Dec 6, 2018 9:42 AM by luke.edmonds

    How can I manage multiple sites in multiple countries?

    luke.edmonds Apprentice

      Hi,

       

      I have recently started managing many Ivanti EPM servers globally across many countries including places like UK, Brazil, Australia, UAE, India, South Africa and USA. I have just finished upgrading all servers to Ivanti EPM 2018.3 so they are all on the same software version. The servers are all configured different with different settings and strategies around security patching, software distribution, OS Provisioning, reporting/alerting, etc. Each country and office has it's own Ivanti EPM instance which do not currently communicate with each other.

       

      I am looking to centralise, standardise and simplify everything. I am looking for advise on the best strategy of how to do this. I am currently researching into this to find all the options and then weigh up the options. In the UK I have a distribution point setup locally at one site as a test to see how this works but I am not sure this would be the best option globally when the Core server is in the UK as I imagine the replication would be slow. The other option is to have a rollup core which I don't know much about as yet but I am looking into. As far as I am aware a rollup core communicates with many different instances of Ivanti EPM and stores information from them all based on whatever criteria you set, this might be a better option as at least I could hopefully see what is happening on the servers from a central point. What I was really hoping for though is that I could have an Ivanti EPM core server in each site and they could do some sort of Core Synchronisation back to the Master Core server in the UK similar to a Master and a slave server. I would then make it so each onsite IT contact can only do things relating to clients in their office and things like security patching are completely standardised and the same across countries with exceptions where needed.

       

      Does anyone know if this is possible from a technical point of view or have any ideas of how managing multiple countries can be made easy? I don't know if I am on the right track with the right line of thought or if there are other ways of doing what I want? I know there are other factors to consider like office politics and local laws of what information/data you can retain etc but that is less of an issue right now.

       

      Any help is much appreciated and thank you in advance.

       

      Kind regards,

      Luke

        • 1. Re: How can I manage multiple sites in multiple countries?
          phoffmann SupportEmployee

          Is this possible -- absolutely.

           

          I have been doing similar sorts of scenarios for various TRM accounts for many years already . So "you're not blazing a new trail" is the good news.

           

          The question is - what are you after? If you're JUST looking at centralised reporting ... then that's fairly simple to set up in concept. I'll try to highlight my usual approach here & highlight common pitfalls to watch out for / cope with.

           

          ===============================

           

          At its heart, this is simply a "store & forward" approach, relevant to both INVENTORY and VULNERABILITY scan results. Depending on what you're after, you can "just" do inventory (if you don't care about the vulnerability data) as well -- the difference isn't that big.

           

          Each "Local Core" doesn't just consume the files it gets, but also SAVES a copy of the file. To that end, let's highlight the key things that are needed.

           

          NOTE -- click on the images for a full-scale version

           

          Storing Vulnerability Scans ... (setup == easy).

          • In the "Patch And Compliance" tool, click on the COG icon and select "CORE SETTINGS"


          • These are the options you want to enable:


            Then click "APPLY" ... and that's this part of the operation done (no restarts of anything necessary).

           

          What will this do?

          ==> Under the "ManagementSuite"-directory on the core, you have a "VulscanResults" directory -- the incoming vulnerability scans will now be stored there (usually under "succeeded"), such as in this picture:

          The files are XML format, so you can crack them open if you want to have a peek.

           

          Things to know / watch out for:

          • You MUST have an inventory record for a device first before you can (re-)process a vulnerability scan file (mainly of importance for the reporting server).
          • A device's DEVICEID is part of the file-name ... meaning that if you send 2 vulnerability scans, the "newer one" will overwrite "the older one" (if all else is the same). So you will want to harvest this directory on (probably) a daily basis.
          • Don't FORGET about this. If you have a site with (say) 10,000 devices, that's 10,000 times an additional 1-2 MB of data (varies a lot depending on the OS in question and other factors), but keep in mind that you'll want to have the diskspace available. ESPECIALLY if you're going to be storing inventory scan files as well!
          • Other than that, by and large, you "just" need to gather the files (every evening or whatnot), zip 'em up (as they're text files, they usually compress down to 1% odd of their original size), and send them to "the reporting server" for re-processing (AFTER the inventory scans went through).
          • Whilst you *CAN* run multiple instances of VULSCAN.EXE to process multiple VR-files in parallel, be mindful of the load on your Core (and how well / badly resourced it is). If you have a 4 GB memory server, for instance, that things is going to struggle if it's hosting the database as well, etc ... so "be mindful of your I/O" (disk, memory, database, etc) ... all of this data needs to go somewhere after all. Doesn't make sense to try & send it to a potato of a machine .
          • Would suggest scripting something up to handle this automatically for you. Again - remember to run this AFTER you've processed inventory scans, to ensure you "have a record to update" for any new devices.

           

          How do you re-process a vulnerability results file?

          ==> You run the vulnerability scanner ("vulscan.exe") with the "/i=<path-to-VR-file>" parameter.

           

          So for example (using a switch to force UI to be popped as well, so you can see progress for yourself):

          vulscan /showui /i="C:\Xx\MyResults.vr"

          ... feel free to give that a go with a saved results file.

           

          ====================

           

          I'll cover Inventory scans in a separate post, to keep this reasonable & more visually separate-able.

          • 2. Re: How can I manage multiple sites in multiple countries?
            phoffmann SupportEmployee

            OK - part 2 ... Storing & forwarding & re-processing inventory scans:

             

             

            So - how to store inventory scans? (setup == easy).

             

            1. On the Console on the CORE (this won't work/exist from a remote console) go to CONFIGURE => SERVICES.
            2. And then go to the INVENTORY-tab & click the "ADVANCED SETTINGS"-button...
            3. Scroll to the "STORE SCANS"-option & select it.
            4. Enter a "1" in the value field & click on the "SET"-button.
            5. Click on "OK" to apply/close the advanced settings window.
            6. Click on "OK" to apply/close the CONFIGURE SERVICES window.

            NOTE -- this will require a restart of the INVENTORY SERVICE to be applied (you will be prompted whether you want to restart there & then).

             

            What will this do?

            ==> ANY inventory scan (whether it'll be successfully processed or not) will be stored under "(...)\ManagementSuite\LDSCAN\STORAGE\" with this option applied.

            ==> Note that new inventory scans will NOT overwrite each other (so a single device sending 10 inventory scans will result in 10 saved files)!

             

            See here for instance:

             

            This *WILL* store miniscans (the "hey, here's my updated network info") as well ... so up to you whether you want those. If your aim is "mainly reporting", you may or may not care about "up to the minute" accurate networking information.

             

            What to do in order to "re-process" a scan-file?

            You just need to paste / drop it into the "LDSCAN" folder under the ManagementSuite directory. The Inventory Service will hoover them up automatically.

             

            Things to know / watch out for:

             

            • You MAY want to disable delta scanning on your remote sites (as otherwise you have to have 100% of the inventory scans come in accurately ... as we check each delta scan against its "last time I got data from you" versus "last time you think you sent me data" ... and if they're out of sync, we clearly missed something & will reject the scan). This CAN be a bit of a headache, so we usually disable delta scanning ...

              ... but this *DOES* have an impact on both the network usage of inventory scans (no delta means "if you have 10 MB of scan file, and 9.9 MB of it haven't chaned ... it's still 10 MB of file over the network / 10 MB of file to be stored". So ... be aware of this!

              You set/disable delta-scanning (it's enabled by default) centrally on each Core in the "Advanced Settings" for the Inventory server here:

              where ...
              0 == Delta scanning is disabled.
              1 == Delta scanning is enabled.

              <Changing this option from one setting to another -- like most inventory service changes -- will require a restart of the inventory service to be applied>
            • Scan-files can accumilate QUICKLY ... and you need to remember to delete the ones you've moved over. This is otherwise a "great" way for running out of diskspace (especially since inventory scans won't overwrite each other (which is a good thing overall - and intentional) ).
            • Check with your network infrastructure people in the relevant territories, so that this doesn't hit them as a surprise. Having 40 KB of scan file per device (being "just a delta") suddenly baloon up to 10 MB (for a server, for instance) can make QUITE an impact on network pipes / remote devices across 56 KB modems, for instance especially. So ... "go into this with eyes open" & do your research up-front ... that'll lead to a lot fewer "oh crud... hadn't thought of that" situations.
            • Overall (in my experience), it's PREFERABLE for this situation to disable delta-scanning (if you can afford the network bandwidth), as you only rely on "a single file", rather than a potential chain of files, and if stuff gets lost / you lose a days' feed, it's "not a biggie" as you can just pick up with the next day. But this is very much a YMMV (Your Mileage May Vary) factor!

            • One thing to WATCH OUT FOR is Data Analytics -- if you've got this licensed, check each region's server whether they're using real-time processing / normalisation rules ... and whether their rules "make sense". You MAY want to create / set up real-time processing rules on the reporting server (this would require renaming the files to ".MP" first, so that they'd be processed by the data analytics Core Scan Processor before the inventory service).

              This can be an "interesting" complication, especially if you have "territory 1" doing one thing, and "territory 2" doing another thing alltogether ... (and chances are they will, usually). So you may need to get all your ducks in line (or at least - in coherence) so that you've got a sensible set of consistent data coming in.
            • Other than this, this is by and large a "store & zip & forward" exercise. You just dump the files into LDSCAN, and the inventory server will hoover the files up (starting with the oldest first towards the newest).

            ... remember, the inventory scan files are "JUST" text files ... that can give you a LOT of opportunity to massage them if need be. It's not NECESSARY that you do so - just highlighting that the option exists. (One thing we like to do, for instance, is to inject / include a line about which location / territory each scan file came from).

             

            Right - that should give you plenty to think on / keep you busy for a bit .

            • 3. Re: How can I manage multiple sites in multiple countries?
              luke.edmonds Apprentice

              Hi,

               

              Thank you for providing so much helpful information. Now I just need to take time out to read through it all and work it out. I will let you know once I have read it all and my thoughts.

               

              Kind regards,
              Luke

              • 4. Re: How can I manage multiple sites in multiple countries?
                phoffmann SupportEmployee

                It's "a fair bit to read" (as I assumed you've never done the above), but actual "implementation time" (as far as the Cores are concerned) is "a few minutes".

                 

                The real work is in figuring out / deciding how / when to compress & transfer the data files across & scripting THAT.

                 

                So this is intended as a "writing it once for posterity" type of post, so that all the relevant info is there.

                 

                You don't need to be intimidated by it. Actual work effort (aside from the scripting / decision making process / politics ) is super simple .

                • 5. Re: How can I manage multiple sites in multiple countries?
                  luke.edmonds Apprentice

                  It's good to have all the information rather than implement something and run into problems later. At the moment I seem to have more than enough problems with the Ivanti/LANDesk instances here which is why I have upgraded them all to the same version and am trying to get reports of them all and try standardise everything and make them more a manageable.

                   

                  Thanks for all your help so far. Feel bad for not giving anything back. Especially since I am only going to create more forum posts about more problems.

                  • 6. Re: How can I manage multiple sites in multiple countries?
                    phoffmann SupportEmployee

                    It's OK - it needed to be documented properly anyway (it's not an unusual ask), and now I can just point people to this thread .

                    • 7. Re: How can I manage multiple sites in multiple countries?
                      luke.edmonds Apprentice

                      Fair enough, makes sense. If you are interested I have a problem which I can't put any logic too, is inconsistent and not making much sense to me at the moment to do with device status. I have checked the forums and found nothing that works for me, probably a question for support but I am just gathering information and seeing what I can do first. Device Status not showing correctly