3 Replies Latest reply on Sep 3, 2013 12:08 PM by scots

    Several Issues After Upgrading from 9.0 SP3 to 9.5 SP1

    scots Apprentice

      I upgraded our Core from 9.0 SP3 to 9.5 SP1 on August 11, and we've been having all kinds of problems since.  I've opened tickets with support, but was wondering if anyone else has run into these issues and what they did about them.  Some of them have been corrected/fixed, some are still outstanding:


      • Everything initially seemed OK after the upgrade, however, we’re not “open for business” on the weekends so there wasn’t much going on.  More than half of the PCs are powered off on weekends, nobody is working in most of our locations other than a very small number that are 24/7, and nobody is here in my office other than a couple people on our Help Desk and a couple mainframe people.
      • Monday morning we discovered that we couldn’t get any scheduled tasks to run.  I contacted support, and they asked if the Scheduler service was running on the Core.  I checked, and the Scheduler service wasn’t there at all.  I guess it got removed during the upgrade and didn’t get reinstalled.  They told me how to install the Scheduler service, and scheduled tasks are now running.  This is fixed.
      • After I got scheduled tasks working, ALL machines in ALL tasks were failing.  The machines that were powered on/available were failing with a message that stated “The selected delivery method and or Packages require features that are not provided by the agent installed on this target workstation. This can be resolved by upgrading the agent to the latest version.”  I contacted aupport again, and was told about changing the downgrade option in the delivery method from fail to downgrade.  I don’t know if this is something new in 9.5 (or SP1), or if it has always been there and I just wasn’t aware of it.  If it’s always been there, it’s never been a problem in the past and I’m assuming the upgrade changed the setting to fail for some reason.  This is fixed.
      • It appears that several (not all) scheduled tasks (some existing prior to the upgrade, and some new) have something about them that’s causing the scheduler service to go bonkers and permanently spike the CPU at 100%.  Rebooting the Core was how we were bringing the CPU back to normal after a spike occurs.  Recreating the affected tasks and using those new tasks seems to “fix” the issue sometimes, but not always.  Support has identified this as a bug and say this will be fixed in SP2, but no ETA for SP2 and no word yet as to whether an "out of band" patch just for this issue will be released prior to SP2.  The bug involves ScheduledTaskHandler.exe getting "stuck" and consuming all of the CPU.  The "fix" for now is to go into Task Manager and kill all of the ScheduledTaskHandler.exe processes.  Since we have no clue when this will "strike" and can't constantly monitor the server 24/7, one of our guys has setup something in Windows Scheduler that (via a batch file, I think) regularly kills all running ScheduledTaskHandler.exe processes every 10 minutes.  Not a great "fix" if you ask me.
      • Part of our organization is on a separate domain.  It's a little difficult to explain, but they're part of us but also kinda sorta separate in a way.  There is a limited trust between the domains.  We've been having difficulty upgrading the agent on their machines.  Some appear to have been completely successful, some appear to have had the agent be successful but antivirus not install, and some have been complete failures.  We've never had any issues in the past with installing or upgrading their agent.
      • I have always had my Console view setup to show AV Engine Version and Definition version. Something has changed with 9.5 to where it’s showing two different version numbers for each of these items and causing duplicates of devices to show.  I have to switch to a column view that doesn’t include AV info to not have duplicates.  I mentioned this to support but haven't really gotten an explanation as to why this is happening now and wasn't before.  All he did was basically explain to me why I was seeing duplicate devices (because of my column view).  I've finally identified that the new duplicates are from Windows Defender (which it's now identifying as an antivirus product but previously did not), but have no idea if there’s a way to fix it and have my previous AV view without Windows Defender.
      • We’ve started having a number of PCs have LDAV disabled.  If you attempt to enable it, you get a message that states “Another antivirus product is installed. To avoid compatibility issues, LANDesk(R) Antivirus will not start.”  This started occurring after the upgrade of the Core server.  This is occurring only on PCs that have the 9.0 Agent/AV on them.  I reported this to support, and they had me zip up the contents of the C:\ProgramData\LANDeskAV folder and send to them.  I was then asked me to remove Windows Defender from my PC.  Windows Defender is permanently baked in to Win 7, and there’s no way to remove/uninstall it.  The only thing you can do is open Windows Defender and turn it off, then go into Windows Services and disable the Windows Defender service.  I did this, and now my LDAV is working again on the older agents. They've now identified this as a bug, and offered two options to fix it.  One is to upgrade all the agents to the 9.5 SP1 agent, and the other is to replace the ldavhlpr.dll file on the PCs with the ldavhlpr.dll file from a 9.0 SP3 server and mark it as read-only so it doesn't get replaced again.  We had already made the decison to upgrade the all the agents before they had informed us that this was a bug (and their solutions).
      • The Web Console is doing an upgrade of remote control functionality when our users attempt to initiate a remote control session via the Web Console, which I knew would happen.  In the past we never had any problems with this.  This time the upgrade is failing with a message that states "The remote control viewer was not installed properly. Try reinstalling. Contact your administrator if you are unable to complete the installation."  Many of the users who use the Web Console don't use it very frequently, so I'm not sure exactly how many people are experiencing this.  For right now, I'm assuming it's affecting all of them, but some just haven't tried to use the Web Console yet.  Most of these people are not "computer"/"techie" people, so they have no clue what to do and they open trouble tickets on it...which come to me.  Uninstalling and letting it reinstall seems to fix it.  I reported this, but haven't really gotten an explanation as to why this is a problem now but hasn't been in the past.
      • The upgrade/install of the agent seemed to be working OK at first.  Now when I try to push out the agent it seems to start OK, but then I get the yellow triangle with exclamation point like it's failed after a minute or two.  If I look inside the task it's still showing as active rather than failed, and the result column shows whatever it was showing at the time the yellow triangle appeared.  The result column never changes after that, and the task never appears to complete in any way.  Checking the PCs, it appears that no upgrade/install has occured - even if the result column has gotten to a point that shows the install files had been copied over successfully.  I think this may be related to the killing of the ScheduledTaskHandler.exe, but not sure.  I've disabled the task my team member created and testing right now.


      I *think* that's everything (so far).  We've been using LDMS for probably around 8 years or so now.  I've done both in place upgrades and side by side migrations, many service packs and patches, and have NEVER had problems like this before.  I'm about to rip all my hair out!

        • 1. Re: Several Issues After Upgrading from 9.0 SP3 to 9.5 SP1

          I know I ran into your second problem about task delivery. But it started when I upgraded to 9.5 and was given the same resolution about changing the downgrade. The ones I changed in 9.5 stayed the same when I upgraded to Sp1. I know you have to do a 2-step when going to SP1 meaning 9.5 must be installed first. That may have been when this occured to you.


          I don't run AV so I didn't have any of the issues you described. But I do have a holdover about task distribution. Seems that if you have the wake up on LAN checked in the target page of the task it may start delivery but never finish even a day later. Unchecking that makes the task run normally. But this was again an issue I had previously to the upgrade to SP1 and after I upgraded to 9.5.


          I did run into the issue about the remote control app not upgrading properly on a couple of PCs. I had them uninstall, delete the server manager folder under LANDesk and reinstall. That fixed this issue as well.


          There are additional issue I ran into since 9.5 SP1 which was supposed to be fixed. One item was to fix the task results in the webconsole. It fixed it so I can now actually see the devices that pass/ fail or whatever. But the export as CSV doesn't work. Time to open a trouble ticket about that.


          I also opened an issue in the support portal about a new fix I need which is supposed to be available that you may not have noticed yet. But 32 bit apps aren't showing up in the inventory under add/remove programs after upgrading the agent.

          • 2. Re: Several Issues After Upgrading from 9.0 SP3 to 9.5 SP1

            We are also seeing the issues with antivirus not starting, I will try your fix and report back if it works for us as well.  Either way it sounds like we are going to have to touch every computer with these issues.


            Here are a few other issues we have been experiencing since the upgrade last week...

            -The report template editor is no longer accessible - an error windows pops up saying "an error may have occurred or there may be insufficiant memory to handle a large amount of data".  It also wiped out all of my previous report template changes - now we have the standard LANDesk looking reports that we cannot change - ugly, ugly, ugly!


            -Also having a tough time trying to push agents out to windows 8(also 8.1) machines- Fails everytime.


            -Cannot get a 32-bit agent to work with a push or a local install - nothing installs but it still has me reboot after the failed install.


            So frustrated - I should know better and wait to upgrade next time, the negatives are far outweighing the positives of this upgrade.  Is there any way to downgrade back to 9.5, it had issues but did work better

            • 3. Re: Several Issues After Upgrading from 9.0 SP3 to 9.5 SP1
              scots Apprentice

              We continue to have new issues pop up with this new AV.  I asked if there was any type of documentation on the new AV, other than the "How to troubleshoot LANDesk Antivirus 9.5" article that's posted here in the Community.  I was told no...that article was it.  That article is OK, but isn't helpful for most of the issues we've been having.  I mentioned that it would have been nice to have some sort of documentation - especially something like a "Here's what's new" document that points out all the new/changed features from the previous version of AV.  I've been told that my concerns are valid, and are being pushed up the chain.  Here are some of the new things we've discovered:


              - We had a user who uses FTP, and his FTP quit working.  We figured out that it was the new AV blocking it, and eventually that it was the IM Antivirus component specifically.  This was just standard Windows command prompt FTP, no special software or anything, and not connected to any IM.  The FTP would start, get to a certain point, then just sit there like it was waiting for something.  It would never kick back any error messages or anything.  Since there doesn't appear to be any settings specific to the IM Antivirus component, I just completely turned that part off and that got his FTP working again.  Management here has expressed interest in IM Antivirus, so I guess we'll have to revisit this at a later time.

              - The new AV apparently thinks that the SolarWinds network tools that our network guys use are a bad thing.  At first it was deleting the tools from their PCs, so I setup an exception in AV so it wouldn't scan the directory that SolarWinds tools are installed in.  That fixed the deletion problem, but now when they run the tools they can't use them on anything outside this building (which is most of our network).  I then turned off the Network Attack Blocker feature of the new AV (as with the IM Antivirus portion, there don't seem to be any feature-specific settings available), and that seems to have fixed AV from blocking the use of SolarWinds.

              - The full scan in AV is apparently scanning email, even though we have email scanning turned off.  All attempts to turn this setting off have failed, usually with the setting only turning off temporarily (only for a few seconds in some cases) and then turning back on.  Support was able to duplicate the issue, and is opening a bug with Kaspersky.  No idea how long it'll take for a fix to come back from Kaspersky.