1 2 Previous Next 16 Replies Latest reply on Jul 27, 2017 6:22 AM by Lee Richards

New Templates failing after CTOS since SU3

JStevenson Jun 20, 2017 6:58 AM

Hi All,

I'm experiencing a bit of a headache with provisioning. Any new templates I create will not carry on past the CTOS step. This seems to be since we updated our core with SU3 on the 8th of this month. Did know it was happening until very recently.

When I create a new template, I usually just copy one of my previous ones and make changes to it or I create a new one but replicate all the steps making changes where needed. This has always worked for me. Now any new templates I make whether from scratch or by copying a previous one fails to carry on when the computer boots into windows after the Inject LANDesk Provisioning-Agent.

We haven't changed anything else, only done this update (Too my knowledge).

I know the problem is not with the image as it is happening with any image I chuck in a template, even a standard windows 7 wim fresh from Microsoft. Also, all our current existing templates are working fine. I'm really scratching my head with this one. I have tried all of the following.

Recreated the boot.wim from the clean copy.
Edited the boot.wim corename.txt file to have FQDN
I have gone through this article: How to troubleshoot the Configure Target OS (CTOS) Action in Provisioning Templates

Windows 7, 8, 2008, 2012

Change to C:\ldprovisioning directory. Verify the ldprovisioning.cmd is in the folder and the folder is populated. It should contain over 20 files.
Verify that the unattend.xml exists in C:\Windows\Panther
Verify that the commands calling ldprovisioning.cmd exist in the unattend.xml for the platform (x86, amd64, ia64) you have deployed.

C:\ldprovisioning gets created on the device
Unattend.xml gets copied - Either to C:\ or C:\Windows\Panther - depending on the template I'm testing.
There are no commands in the unattend.xml for ldprovisioning. See below *

I have gone through this article - https://community.ivanti.com/docs/47184 and none of this applies or is the problem

One thing I have spotted in the ldProvision log is this :-

2017-06-20 11:33:16(2504-2508) ldProvision.exe:Create process (C:\Program Files (x86)\LANDesk\Shared Files\httpclient.exe) with args ( -f "C:\ldprovisioning\WaitHandler.sig" http://<CORE>/LdLogon/Provisioning/windows/WaitHandler.sig)

2017-06-20 11:33:16(2504-2508) ldProvision.exe:Waiting for process result: 0.

2017-06-20 11:33:16(2504-2508) ldProvision.exe:Process exit code:-1

2017-06-20 11:33:16(2504-2508) ldProvision.exe:Failed to download (C:\ldprovisioning\WaitHandler.sig)

I also noticed, on our core in the folder http://<CORE>/LdLogon/Provisioning/windows/ there is no WaitHandler.sig and in another log I spotted the same thing for ConfigTargetOSHandler.sig which doesn't exist. Maybe it it these missing files???

* I don't see any of this in the unattend.xml

Windows 7, 8, 2008, 2012

Sections are added to the unattend.xml in the Specialize section. This is a special section of the unattend.xml that can also be used for things like setting the home page for Internet Explorer, etc. In this section, the RunSynchronousCommand action is added calling ldprovisioning.cmd from C:\ldprovisioning\. The section of the unattend will look roughly like the following:

<settings pass="specialize" xmlns="" wasPassProcessed="true"> <component name="Microsoft-Windows-Deployment" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <RunSynchronous> <RunSynchronousCommand action="add"> <Description>LANDesk Provisioning Install</Description> <Order>1</Order> <Path>cmd /c %systemdrive%\ldprovisioning\ldprovisioning.cmd</Path> </RunSynchronousCommand> </RunSynchronous> </component> </settings>

I am at a complete loss now.

Oh and to top that. Our current template although is working is causing duplicate entries in our database which I'm sure it wasn't doing before the update. I can't be sure because only by creating a new template to deploy a new image can I test to fix the issue... which I can't do because new templates don't work. Our Inventory Service on the core is set up to deal with duplicate entries, we haven't changed anything there so It looks like that isn't working either.

Help.

Kind Regards

Jamie Stevenson

ldProvision-edited.log.zip 2.7 K
ConfigTargetOSHandler-edited.log.zip 2.0 K
CopyOfTemplateXML.txt.zip 1.7 K

1. Re: New Templates failing after CTOS since SU3

seattleman1969 Jun 20, 2017 9:00 AM (in response to JStevenson)

James,

The .sig files were added for additional security in SU3. If the files do not exist in the location specified then you do indeed have an issue.

If you unpack SU3 and then unpack the 2016.3-SU3.zip you will find the .sig files in the LD2016-3_SU3\LD\1\image directory. the .sig files need to be in the associated directories with their executable files.

However, since this did not occur during install my concern is that your SU3 install was not successful. You may want to call support and open a case on this.

Thanks!

Brandon
Like Show 0 Likes(0)

Actions
2. Re: New Templates failing after CTOS since SU3

JStevenson Jun 20, 2017 9:07 AM (in response to JStevenson)

Thanks Brandon.

The install of the update didn't produce any errors but I think you may be right. Something is going on so maybe the SU3 update didn't go as we thought.

I'll open a support request.
Like Show 0 Likes(0)

Actions
3. Re: New Templates failing after CTOS since SU3

JStevenson Jun 20, 2017 9:25 AM (in response to JStevenson)

Hmmm, having said that, I have just compared the .sig files in the location you suggest and all the ones in there are on the core. So they look to be installed.

Interestingly, I have searched through the LD2016-3_SU3 folder and there is no .sig file for WaitHandler and ConfigTargetOSHandler.

We'll see what support come up with.
Like Show 0 Likes(0)

Actions
4. Re: New Templates failing after CTOS since SU3

seattleman1969 Jun 20, 2017 9:27 AM (in response to JStevenson)

James,

Additionally the inventory issue:

If you go to Configure>Services>Inventory and click on the "Devices" button do you have the checkbox for "Restore Old Device IDs" Checked? If not then you will have multiple entries in the console for a device after imaging, until the daily maintenance runs and cleans up those entries.

In SU3 there is a defect that was "Partially fixed" also that has to do with the order of operations performed by external scanners and inventory when a device is updating it's record in the DB. This keeps the device record from properly updating to the new Ivanti IEM Agent GUID if this device existed in the DB before and the record was not cleared out before provisioning. In the system configuration section of your provisioning template, after agent install, you can add 2 back to back full sync scans and it will correct the record for the device.

This is supposed to be fully corrected in the upcoming 2016.3 SU4 release.

Thanks!
Like Show 0 Likes(0)

Actions
5. Re: New Templates failing after CTOS since SU3

JStevenson Jun 21, 2017 2:18 AM (in response to JStevenson)

Hi Brandon,

We do have the "Restore Old Device IDs" Checked.

We also have the LANDesk agent in stalled on our image. When we started seeing these duplicates I figured I must have somehow forgotten to remove the UniqueIDs and other bits from the image before capture . That is why I was recreating the image to test. Only I can't deploy the new image now. The duplication problem does seem to coincide with when we did the SU3 update though, I cannot see any problems before that and the last image I created has been in use for several months. Unfortunately it has been too long since we did the SU3 update so we can no longer revert to before SU3. Not without completely rebuilding the environment.
Like Show 0 Likes(0)

Actions
6. Re: New Templates failing after CTOS since SU3

Jharris3400 Jun 21, 2017 5:55 AM (in response to JStevenson)

If you agent is in the BASE WIM/Image, get it out now. BKM is to not have that in your base image and install the agent via a provisioning task.
Like Show 0 Likes(0)

Actions
7. Re: New Templates failing after CTOS since SU3

Frank Wils Jun 21, 2017 10:10 AM (in response to JStevenson)

Hi,

Just to chime in on the .sig files issue. Not all action handlers have a matching .sig file right now. However, programmatically it is assumed they all have one to easier add a sig file later without needing to re-code. So nothing to worry there at least

Frank
Like Show 0 Likes(0)

Actions
8. Re: New Templates failing after CTOS since SU3

Frank Wils Jun 21, 2017 10:13 AM (in response to JStevenson)

If you have an agent in your image, the CTOS action will put the command to continue in Windows in the C:\Program Files (x86)\LANDesk\Shared Files\cbaroot\actions.ini

So it should start when the LANDESK Management Agent service starts. You can monitor this. Maybe the service already starts to early?

Frank
Like Show 0 Likes(0)

Actions
9. Re: New Templates failing after CTOS since SU3

JStevenson Jun 23, 2017 9:16 AM (in response to JStevenson)

We still haven't gotten anywhere with this but I have been trying all sorts.
One thing I have noticed is this failure happens even if I schedule an os template to run on a pc that is already built and working.
The Vboot option works at the beginning of a template and will boot a device into winpe but if I schedule something like the following, it fails.

I have searched through logs but I'm not finding anything obvious.

Here is the provision log for this fails schedule.

INFO ProvTaskMan 23/06/2017 16:00:09 : LDTaskIdn: 7953, Reading task information from the database
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, got task status WORKING
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Database read for task data appears to have succeeded
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Reading task configuration info from the database
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, got task status WORKING
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Database read for task config data appears to have succeeded
INFO PROV_SCHEDULE 23/06/2017 16:00:10 : Selected template idn: 494
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Reading task information from the database
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, got task status WORKING
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Database read for task data appears to have succeeded
INFO PROV_SCHEDULE 23/06/2017 16:00:10 : Delivery method IDN: 2
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Reading task information from the database
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, got task status WORKING
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Database read for task data appears to have succeeded
INFO PROV_SCHEDULE 23/06/2017 16:00:10 : Making sure template 494 is flat ...
INFO PROV_SCHEDULE 23/06/2017 16:00:10 : Flattened template idn: 494
INFO ProvTaskManager 23/06/2017 16:00:10 : template:494,Searching for existing locked template
INFO ProvTaskManager 23/06/2017 16:00:10 : template:494,Existing locked template not found. Cloning template.
INFO ProvTaskManager 23/06/2017 16:00:10 : Cloned template ID: 495
INFO ProvTaskManager 23/06/2017 16:00:10 : template:494,Freezing template succeeded.
INFO PROV_SCHEDULE 23/06/2017 16:00:10 : Frozen template idn: 495
INFO ProvTaskMan 23/06/2017 16:00:10 : LDTaskIdn: 7953, Updating task configuration with the template idn: 495
INFO PROV_SCHEDULE 23/06/2017 16:00:10 : iterating through 1 computer(s) ...
INFO PROV_SCHEDULE 23/06/2017 16:00:11 : computer 14282 not busy; creating provisioning job
INFO PROV_SCHEDULE 23/06/2017 16:00:12 : Status: successfully created history task History task idn: 1355
INFO PROV_SCHEDULE 23/06/2017 16:00:12 : First action or first required action in Windows environment. Will attempt to prod.
INFO PROV_SCHEDULE 23/06/2017 16:00:15 : Finished iterating through 1 computer(s)
INFO PROV_SCHEDULE 23/06/2017 16:00:16 : All target machines should now be active
INFO PROV_SCHEDULE 23/06/2017 16:00:16 : Status: PULL_AVAILABLE, prov_schedule ends --------------------
DEBUGGING ProvisioningWebService 23/06/2017 16:00:39 : GetTaskXmlInternal
VERBOSE ProvisioningWebService 23/06/2017 16:00:39 : Identifiers to use - Mac Address: True, Serial Number: True, AMT GUID: True
VERBOSE ProvisioningWebService 23/06/2017 16:00:39 : Identifiers:
VERBOSE ProvisioningWebService 23/06/2017 16:00:39 : MACAddress: F0D5BFB81095
VERBOSE ProvisioningWebService 23/06/2017 16:00:39 : MACAddress: 000C299D10B7
VERBOSE ProvisioningWebService 23/06/2017 16:00:39 : SerialNumber: VMWARE-56 4D 04 8B D9 10 E1 36-2F 61 42 0A 21 9D 10 B7
WARNING ProvisioningWebService 23/06/2017 16:00:39 : Unable to find template for computer IDN 13523
WARNING ProvisioningWebService 23/06/2017 16:00:39 : Unable to find template for computer IDN 14125
WARNING ProvisioningWebService 23/06/2017 16:00:39 : Unable to find template for computer IDN 14207
WARNING ProvisioningWebService 23/06/2017 16:00:39 : Unable to find template for computer IDN 14274
VERBOSE ProvisioningWebService 23/06/2017 16:00:39 : Returning task XML
VERBOSE ProvisioningWebService 23/06/2017 16:00:46 : SetActionStatus (historyTaskIdn 1355, historyEntryIdn 30534, next 30535): state FAILED
VERBOSE PHistoryTask 23/06/2017 16:00:46 : StopProvisioningJob (LDTaskId 7953, computerId 14282, historyTaskIdn 1355), state Failed
VERBOSE PHistoryTask 23/06/2017 16:00:46 : update the history task state
Like Show 0 Likes(0)

Actions
10. Re: New Templates failing after CTOS since SU3

jParnell Jul 5, 2017 9:46 AM (in response to JStevenson)

I'm assuming you rebuilt your agent after the SU3 upgrade, is that correct?

Additionally, what version of Windows are you deploying to? If Windows 10, what build? We're having issues with 1703...
Like Show 0 Likes(0)

Actions
11. Re: New Templates failing after CTOS since SU3

JStevenson Jul 5, 2017 9:50 AM (in response to JStevenson)

Yeah we have rebuilt the agent.

It doesn't matter what version of Windows we are trying to deploy. It doesn't work for Windows 7 SP1 and windows 10 1703.
Like Show 0 Likes(0)

Actions
12. Re: New Templates failing after CTOS since SU3

JStevenson Jul 6, 2017 7:00 AM (in response to JStevenson)

I think I have a work around for this.

As I mentioned before, this problem only happens with new templates going forward so I ran an old template on a device and before it had finished and had time to delete the ldprovisioning folder, I backed up that folder.

Now on another device where the new templates I know fail, I copied that ldprovisioning folder to the C:\ drive and tested a new template. Low and behold, the template ran without issue.

Looking deeper into the logs where the problem occurs, it looks like the ldprovioning agent or Httpclient.exe is failing to download the different handlers the template needs to continue. I think the problem we are having is a permissions issue or maybe something to do with IIS. Something must have changed (or reset) with the patch which has broken this link. Unfortunately I have no idea what that is. By creating the ldprovisioning folder on c:\ and making sure all the necessary files are in there on the image, then when the computer boots into windows, it should carry on because the files it needs are already there and it won't need to download them.

I'm going to test this theory until LD support can come up with a full solution but if this works I may be able to resume with my images for now.
Like Show 0 Likes(0)

Actions
13. Re: New Templates failing after CTOS since SU3

jParnell Jul 6, 2017 8:11 AM (in response to JStevenson)

Your latest reply reminds me of an issue that our Belgium counterparts encountered a few months ago, and now that I think about it, it seems the symptoms are at least familiar (it's been a few months since we had that meeting). It turns out that a firewall patch had inadvertently started causing file copy failures between either the preferred server or the core, and the computer being provisioned. It was working fine in their office, as the source was on the same subnet as their computers being provisioned, but was not working in one of the remote locations where source and destination were on different subnets. After reviewing firewall logs and seeing files being blocked, I believe the solution was to add exemptions from the source (be it your PS or core) directories. The really unique part was that it only affected that specific subnet.

You may want to try sniffing packets between your source and destination, as it could be completely unrelated to LDMS. I will ask our Belgium folks for more details on how they discovered the problem.
Like Show 0 Likes(0)

Actions
14. Re: New Templates failing after CTOS since SU3

jParnell Jul 7, 2017 7:43 AM (in response to jParnell)

OK, got more info from our counterparts in Belgium - here's what he said:

The LDLOGON folder is not available on the preferred servers for [company].
This means that the files used for the CTOS action of LANDESK during provisioning are coming directly from the LANDESK core.
(Not really a problem, these are small files.)

However, since the last upgrade of LANDESK the file CBA8inst.msi is being detected by Meraki as the W32.Triojan.NM.
This file is copied from the LANDESK core during the CTOS action with provisioning.
Meraiki is the hardware used to setup VPN connections with remote offices of Mohawk EU & is running a AMP ( malware protection )

Before the upgrade this was not a problem.
However, the databases with treats change all the time, so the upgrade timeline of LANDESK could not be relevant.

We excluded the file as can be seen in the second screenshot and now provisioning is back ok.
Until we excluded it, the CTOS action failed.
The Firewall Admin over there had this to say about it as well (please keep in mind, this is translated from Dutch to English using Google Translate - I make no guarantees it will make sense):

In the beginning, we need to whitelist a URL in the AMP setting.
If this was not done, each deployment failed and we saw this in the logging / alers
In content filtering we have the same categories on all sites (Palo Alto / Meraki)
One of the categories is "Weapons" but there is apparently also a difference with the setting URL category list size:
If the setting is on "Full list", the Meraki apparently excludes the URL below. Note this is not always the case. In our test cases, this was only in 50% of the deployments.

I don't know if this is related to your issues at all, but I hope it is helpful.
Like Show 0 Likes(0)

Actions