7 Replies Latest reply on Jul 10, 2014 3:39 PM by mmorales

    LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours

    Expert

      I am currently testing our upgrade process in a test environment.  I know that since our VM's are not as highly spec'd as our production environment, we should expect some decreased performance.  However, after scouring the community site to determine why timeouts were occurring, how to possibly avoid them, and how to possibly improve the completion time, I am still at 10 hours or 36656 seconds to complete the upgrade.

       

      Changes applied:

      1. Increase timeout to 600,000, successful(support suggested increasing to 999,999).  Attempted increase from 300 to 600, failed, timeout exceeded.  Attempted increase from 600 to 1200, failed, timeout exceeded.  Attempted increase from 1200 to 12000, failed, timeout exceeded.
      2. Changed database recovery mode from Full to Simple and trimmed the Transaction Log

      USE [DatabaseName];

       

      GO

       

      --(1) first, set recovery mode to SIMPLE

      --(2) run the following to reduce log to 200mb

      CHECKPOINT;

      GO

      CHECKPOINT; -- run twice to ensure file wrap-around

      GO

      DBCC SHRINKFILE(OOTB_60_log, 200);  -- this uses the "logical name," not the actual log file name

      GO

       

        3.   I did not Check user-defined Indexes and Statistics dependent on converted columns, as I did not get a failure due to this.

       

       

      I debated check marking Unicode prior to running any upgrade, but realized it was already too late.  Any opinions or additional information would be greatly appreciated.

       

      Thank you,

      Mike

        • 1. Re: LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours
          Expert

          On my 7.7 upgrade my test upgrade too 4 hours, in production it took about 20 minutes.  The SQL server makes a huge difference.  It will be interesting to see how much faster your upgrade happens on your live server.

          • 2. Re: LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours
            Expert

            I forgot to mention CPU, Memory, and Network utilization were nothing extraordinary and no where near 100%.

             

            Wanted to include, I just ran the scripts related to number 3 above, it came back with nothing.  So at this moment I am at a loss of ideas on how to improve the upgrade performance.  Forgot to mention our .mdf is ~ 53GB

             

            Thanks

            • 3. Re: LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours
              Expert

              Thank you for the feedback Carl.  Did you do anything out of the ordinary on your go live?  Any of the steps I listed above?  Any steps I didn't list, as far as those not included in the upgrade.pdf?

              • 4. Re: LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours
                dmshimself ITSMMVPGroup

                I had a large 74 SQL server DB to upgrade and given that the upgrade changes the type of many of the string attributes I ended up using an SSD to store the DB,  it still took a while, but was much, must quicker than before.  I don't have exact figures but it came down from well over 24 hours to round 1 to 1.5.  As a test that might be worth doing to see what difference it makes

                • 5. Re: LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours
                  Expert

                  Thanks for the reply DMS, but these are running off a SAN and reviewing the performance for the LUN this db is spanned across is showing no issues.

                   

                  On an additional note, I have decided to continue on to take a look at 771.  Per the upgrade.pdf guide, I need to login as SA to "enable certain internal data migrations to happen."  This has timed out several times, so I went back and upped the timeout in configuration centre back up to 12000.  I am seeing some crazy reads on the Disk.  I have upped the cpu's to 8, from 4 on the db server, it has 8GB of mem, with 6GB set for sql only.

                  • 6. Re: LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours
                    Expert

                    Update:

                    Reran DB upgrade after restoration in test.  Prior to doing so, I increased the cores to 8 and set the 6GB limit on SQL to allow the OS to have some breathing room.  This actually took longer than the last upgrade, 11.11 hours.  For the time being this is just going to be acceptable.  I actually am going to be using some other hardware to test the upgrade of the database.  I worked with support and resolved other issues.  These are listed below, just in case someone else runs into the same issues.

                     

                    Issue:

                    Logging into Console after upgrade would time out and prevented login as SA to apply the license.

                     

                    Resolution:

                    On the webserver, changed the consolex64.exe.config setting:

                                   From:   <add key="WebServiceTimeout" value="120000" />

                                   To:       <add key="WebServiceTimeout" value="900000" />

                    Cycle IIS or App pool(iisreset preferred)

                    Was able to log into console as SA without time out and apply license

                     

                     

                    Issue:

                    After the upgrade and cumulative hotfix applied, incidents take a very long time to open under all accounts

                     

                    Resolution:

                    Reindexed the database.  This is not noted anywhere else in documentation, but after doing so, this increased the speed of the console.

                    • 7. Re: LDSD 7.7.1 Upgrade from 7.5 SP1 HF3 took 10 hours
                      Expert

                      Update to time to completion.

                       

                      As part of our upgrade process, we are replacing the current production db server with a similar spec'd server, both are spec'd similar and are same models.  We are going to use a C Name to get around changing reports or anything else pointed at the current database server. 

                       

                      With that said, I was able to run a test upgrade against the  physical hardware prior to go live.  We used a VM App and VM Web server, just like before.  We tested twice only using VM's, the first took 10 hours and the second took 11 hours.  The test upgrade against the physical took 2 hours and 45 minutes.  Needless to say, this is great!

                       

                      We have a production upgrade go live in a couple weeks, I will update then.