About: "Devices" and "Duplicate ID's" option for Inventory, and how they work

Version 14

    Verified Product Versions

    Endpoint Manager 9.5Endpoint Manager 9.6Endpoint Manager 2016.xEndpoint Manager 2017.xEndpoint Manager 2018.x




    Many users tend to be either confused or uncertain in what the "Devices" and "Duplicate ID's" buttons in the Inventory-section of the "Configure Services" options are for. The most common misunderstanding is that they are merely yet another "two different ways to do one and the same thing".


    Little could be further from the truth.


    This article aims to clarify both how the relevant technologies work conceptually, and how they evolved historically, so as to provide a contextual understanding of this natural progression, so as to leave the reader with a much clearer idea as to the functionality provided here.


    For clarification, a screenshot has been included to clearly mark which options are being discussed.

    1. Towards the "Duplicate Device ID's" options
    2. Towards the options for detection/deletions of duplicate device entries


    Duplicate Device ID's


    What is this?

    This is the historically "oldest" technology of the three working here.


    This technology originates from OSD (OS Deployment) - before even OSD was available in LANDesk as a technology - and one specific scenario in particular. That very specific scenario is the following (and all points have to be true):
    - You use a cloned image (i.e. one non-SYSPREP'ed image of a device that is copied, without change, to other devices).
    - You put the LANDesk agent on your image.
    - You DON'T remove the LANDesk Device ID from the agent on the image.


    Thus, for example:



    If you had - say - 100 devices, this would result in 100 devices all reporting one and the same LANDesk Device ID (the way by which we uniquely identify a device), thus their inventory scans would continuously overwrite each other.


    How & When does it work?

    This piece of technology works at the time an inventory scan is PROCESSED.


    The Inventory Service checks the attributes it is supposed to check for -- by default, these are (based on long experience, we set the defaults that seem to work best) MAC Address + Device Name.


    Now, when a scan-file is processed, the Core checks for the following logical questions to be answered:
    - Do I already have an entry for this Device ID?
    - If there IS already an entry for this Device ID, is the MAC-address different? Is the DeviceName different?


    The last part is based on the configuration in the relevant section -- by default, we require BOTH attributes to be different (since if we have the same deviceID being used by a system that has a different MAC and a different host-name, the chances are high that this is indeed a different physical box). See the screenshot below on how things are configured by default:





    Breaking down the configuration

    The screen is relatively easy to understand, once all the relevant options are put into context. To this end, the following image that takes the different options apart and explains them should help considerably.



    #1 - Attributes List:

    This is a list of all of the fields in inventory (you should recognize its similarity when constructing a LD-query). This is used as the "pool of possible information" to choose from, in order to determine what constitutes a unique device.



    #2 - Identity Attributes:

    This section contains the ACTIVE attributes (from the pool of all possible ones) that are used to determine specifically how to identify a device entry as "unique".


    Note that these should be attributes that remain CONSTANT. A MAC-address or a device-name is not likely to change, whilst for instance, anything relating to timestamps of software would be a bad idea.


    It is also highly recommended to have a MINIMUM of two (2) items here (a device could be reformatted to a different name, for instance, or a NIC might fail and need replacing, changing the MAC-address). With that in mind - one should be careful about using too many attributes as well -- more than 4 is not recommended.


    The reasons for warning about the last part are amongst performance (more attributes means more DB-overhead to check for EVERY inventory scan), as well as an exponential possibility of having mischosen.


    For the vast majority of environments, the defaults we suggest and configure should be quite sufficient (based on many years' experience). It should be contemplated carefully before any changes are enacted here. There can be genuine reasons for this - it is not, however, something that should be done light-heartedly.


    Bad configuration here can cause great problems in the acceptance of inventory scans.


    #3 - Duplicate Device ID Triggers:

    Out of the list of attributes chosen in (#2 - Identity Attributes) above, this sets the # of attributes that need to be 'triggered' for a scan to be detected as coming from a duplicate device.


    In practice -- with the default configuration, this means that BOTH attributes (since 2 out of 2 possible attributes selected) need to be different (whilst the UNIQUEID remains the same) for the Core to detect that this scan is coming from a different device.


    As a hypothetical example:


    A Core has a device in its database with the following information::

    - UniqueID -- {12345678-1234-1234-1234-123456789012}

    - Device Name -- NormalHost

    - MAC -- 00-00-00-00-00-00-00-00


    And a device would send in a scan with the following information:

    - UniqueID -- {12345678-1234-1234-1234-123456789012}

    - Device Name -- DifferentHost

    - MAC -- 11-11-11-11-11-11-11-11


    The Core would note that:

    1 - The UniqueID is the one it already has in the DB (so it is attempting to overwrite/update an existing entry, rather than creating a new one).

    2 - The Core notices that the DEVICENAME is different ("NormalHost" vs "DifferentHost")

    3 - The Core notices that the MAC-address is different (all 00-s vs all 11-s)


    4 - Because the UniqueID is the same (so we're trying to update an existing entry) BOTH of the active attributes for duplicate device ID detection are different. What continues when this happens, is explained further below (in section I.D).


    #4 - Reject Duplicate Identities:

    This is an "on/off" switch, whether to reject duplicate ID's (which is recommended) or whether to just let it happen -- if this is the case, we will just allow the overwriting of a DB-entry by a 2nd (or 3rd or 4th, etc.) device's inventory scan.



    I.D - Assuming we find a duplicate, what happens?

    If we find a duplicate, we immediately reject the scan-file which is being processed, and take note of the Device ID, marking it as bad.


    From this point on, when ANY client tries to send an inventory scan with this device ID, that inventory scan will be rejected, AND any client reporting this DeviceID in will be told to generate a new device ID. Since it is impossible to determine who is the "rightful owner" of this device ID, it's safer to just tell EVERYONE who uses this to generate a new, wholly random Unique ID.


    This will then - over time - correct the problem, by forcing all clients who have duplicate device ID's to generate new, random ones. Seeing as these Device ID's are formed like a Windows SID, the chances of another duplicate are closer to the astronomical (and even if another instance of duplication should occur, all clients involved would shortly after be told to generate another, random ID).

    I.E -- Final notes on this

    This technology should pretty much ONLY ever kick-in if you've got the scenario I outlined in (I.A) -- that should be pretty much the only possibility for you to have duplicates of Device ID's.


    The key point here is that this technology is to remediate duplicate Device ID's, which would manifest itself as multiple devices overwriting each other's inventory entries.


    It's important to be aware that this does not - in any way - address issues where you have multiple entries of a single device in inventory (duplicate device ENTRIES, as opposed to duplicate device ID's).




    The other option needs to split into two sections. I'll go through them historically, that should hopefully make the most sense.




    II - Duplicate Devices


    II.A - What is this?

    This runs during DATABASE MAINTENANCE, so is NOT real-time.


    The key thing on the "remove duplicate when..." window is to be aware that there's ultimately two possible approaches. Either an OR-based logic, and the alternative - an AND-based logic.


    There is no "right" approach here, it's based on circumstance and environment, but first, let's see how it works.


    II.B - How does it work?

    The Inventory Service goes through all the DB-records during DB-maintenance and checks for duplicates based on the settings that are set. There are four possibilities here (the key differentiator being whether you use an OR logic, or an AND logic):


    II.B.1 - A match on only the device name
    II.B.2 - A match on only the MAC address
    II.B.3 - A match on EITHER the device name OR the MAC-address




    II.B.4 - A match on BOTH - the device name AND the MAC-address.



    The duplicates are then deleted from the database, but no preventive measures are taken to stop future duplicates from appearing (with this option alone), but they are deleted.


    II.C -- Final Notes on this

    Again, this is NOT a "real-time process" here at this point. This is important to note, as this part will happen on top of (and independently) of what I'll describe in section III.


    This was a somewhat logical development on the feature we had in section I - previously customers had to filter manually for names and such (or run queries), and delete things by hand -- it made sense to automate this.





    III - Duplicate Devices - Restoring old Device ID's


    III.A - What is this?

    The "Restore old device ID's" function is - in turn - a logical development from the improvements we added in Section II.


    Rather than fight a (nightly) battle against duplicate devices, it made sense to try and be more proactive about it, preventing them before they pollute the database, as it were.


    "Restore old Device ID's" works in conjunction with the feature in Section II. In fact, it gets its own configuration from the same fields.


    Contrary to the 'deleting duplicate devices' process, THIS one does actually happen as inventory scans come in, so is more real-time oriented, rather than a nightly job,


    III.B - How does it work?

    Based on how you configured Duplicate Devices to be detected (see Section II.B), Restore Old Device ID's uses that SAME logic.


    When an Inventory Scan comes in, it uses the same logic (II.B.1 through to II.B.4) to detect whether there already is a copy of this device (but with a different device ID) in the database.


    The normal scenario for this would be if you formatted a device with a clean image - the hardware information would be the same, the device name MAY or MAY NOT be the same - but the LANDesk Device ID would normally be different  (unless you use LANDesk OSD with Sysprep, as we inject the existing DeviceID there).


    III.C - Assuming we find a duplicate, what happens?

    The inventory scan is rejected to begin with, and the clients' Inventory scanner will get told "use this, your old device ID". Once the client adopts its old device ID, it will then - as intended - re-use the existing entry in the database for itself.


    This takes roughly 2-3 "inventory scans" to get this stuff communicated across.


    Once the client uses its old device ID, the scans a processed as normal.


    III.D - Final notes on this

    The "Restore Old Device ID's" feature is a good, active effort to prevent duplicates from occurring in the first place.

    The real problem comes in the form that there isn't a single, right answer on how this should be configured.


    "Normally", Device Name AND MAC-address would be the most sensible, but if you were an organization where any device that gets reformatted absolutely WILL get a new Device Name as well, then this logic isn't going to work.


    Conversely, you CAN have problems if you use the "OR" logic, as you get hung up on a single attribute. One of the more common by which this can be a problem are - for instance - MAC-addresses of virtual machines.


    Virtual machines tend to have a (comparatively) small pool of randomized MAC's - and if you have two (or more) devices that share a trait (such as MAC-address), then this will cause continued problems.




    IV - In Conclusion

    This article should help distinguish between the handling of duplicate ID's and duplicate device entries in LANDesk(R) Management Suite. All aspects of conceptual design and actual configuration have been covered.