14 Replies Latest reply on Nov 16, 2017 8:11 AM by AAmes

    Importing Data from CSV

    noSpam Rookie

      I'm attempting to import data into goldmine, and I can't seem to get very far using the "Wizard".  It has worked on 1 out of 10 times on data from the same data set.  The process monitor tells me that 0 out of x rows has been imported, but doesn't give me any clue as to why it skipped all my data.  It is not duplicate data.  Is there some sort of esoteric logging that is being done to tell me what is wrong with this process?

        • 2. Re: Importing Data from CSV
          JDymon Apprentice

          As someone who has been working with GoldMine for about 20 years, I assure you the import process works. I probably help people import from CSV into GoldMine about 6 times a week on our Help Desk.

          You need to ensure that you remove all comma's, double quotes and single quotes if you are going to use a CSV file and not set the field delimiter. Also yes, I have seen it where GoldMine will cut off the data on import if the columns are not auto-fitted to the full widths, I haven't tested it in a while, but as a standard rule/step I always auto-fit the coulmn widths, Its a simple click of a button in excel and takes 1 second to do.

          • 3. Re: Importing Data from CSV
            John Neighbors Expert

            Gentlemen,

             

            My 2 cents worth, if it's even worth that much ...

             

            (Disclaimer:  I have NOT read through the details of the PDF ultimately referenced here.  I'm purely going off of the commentary itself.)

             

            I agree with Eric (prior comments now deleted) in that a CSV file is simply raw data that does not contain column width information.  So that step is not material to this process, as long as you are using CSV files.  HOWEVER, JEROME is SPOT ON with recognizing "Its a simple click of a button in excel and takes 1 second to do." It sure as heck doesn't hurt.

             

            Back in the day, whenever Excel supported saving as a dBASE (DBF) file, adjusting columns widths in Excel before saving as a PDF was paramount.  However, due to proportional fonts and how Excel does it's column resizing, you still might run into data truncation when saving as a DBF file.  Back then, I would always manually set column widths in Excel based on knowing what the maximum field size was that was being targeted in the GoldMine database.  But, that was another day, before Microsoft blacklisted dBASE (DBF) files, which they never quite figured out how to handle properly to begin with.

             

            Regarding Eric's previous (now deleted) comment of: Most of the time Goldmine doesn't even recognize that I have fields to import.  It shows the data, but on the field mapping step, the "Import Fields" list box is empty.

             

            In my experience, 100% of the time, this is a matter of improperly formatted data, in some shape, form or fashion.  To the best of my failing memory's recollection, I don't know that I have ever experienced this symptom (which I have MANY, MANY times) when it did NOT prove to be something odd with the data. 

             

            Embedded double quotes, commas, something that is not agreeing with GoldMine. It may even be that the line delimiter on the data file is only carriage returns and not carriage returns WITH line feeds or something bizarre like that, but, again, in my experience, it is always something about the data.  I've even had to open CSV files in NOTEPAD and copy/paste to another text editor (like Notepad++) and then SAVE AS, etc. to get GoldMine to accept the data.

             

            Regarding Eric's previous (now deleted) comment of: I assume this is another ancient piece of goldmine that hasn't been updated in decades and doesn't work consistently anymore.

             

            I would personally disagree.  This piece of GoldMine works just as well as it always has.  No, it's not perfect.  Yes, it could be much better.  It would be great if it were the "perfect import tool" for GoldMine (or even just a little bit "smarter", for that matter).  Unfortunately, it's not.  However, there are other great 3rd party import tools that are rock solid.  Yes, they cost extra, in addition to the GoldMine software cost.  If that's a deal breaker, then GoldMine's Import (for what it's designed to truly do) works just as well as it always has AS LONG AS YOU MANIPULATE and MANAGE the data properly.

             

            OK, I agree, probably not quite 2 cents worth.

            1 of 1 people found this helpful
            • 4. Re: Importing Data from CSV
              noSpam Rookie

              I deleted my comment, because after a lot of trial and error, I was able to find what was wrong with my data.  It's frustrating that trial and error is the best way to learn goldmine administrative stuff like this.  Goldmine recognized the number of rows in my data set, but didn't tell me why it skipped them all. 

               

              I'm sure it works as well as it always has, and that's my problem with the whole Goldmine software package.  It doesn't ever seem to get any better.  Most software gets updated to stay modern.  All goldmine updates seem to just barely keep it from breaking, and any new features that are added are so poorly documented that they don't do anyone much good unless you have hours to do testing on it.  Being as we are a small business, we don't have that luxury.

              • 5. Re: Importing Data from CSV
                JDymon Apprentice

                I'm sure a few hours of proper GoldMine training would be better and less frustrating then hours of trial and error, and hopes that you've done it right. I've seen so many users struggle with something and get frustrated with it and then blame the software vs starting out from the start on the right foot with proper training.

                • 6. Re: Importing Data from CSV
                  DCastell Expert

                  I think it's worth pointing out that CSV, as a format, is only slightly more advanced than punched cards. 

                   

                  While the uninitiated think that CSV is a viable data format with well-defined delimiters, the reality is that it could look like any of the below examples and still be called CSV:

                  Bob Smith, (310)555-1212, 155 Oak Ln., Los Angeles, CA

                  'Bob Smith', '(310)555-1212', '155 Oak Ln.', 'Los Angeles', 'CA'

                  "Bob Smith", "(310)555-1212", "155 Oak Ln.", "Los Angeles", "CA"

                  (among other possibilities using the CARAT symbol and so-on.)

                   

                  By necessity, that means the user supplying the CSV file needs to be able to tell GoldMine (or Excel or whatever other program) a little something about the formatting. 

                   

                  Honestly, in my opinion, CSV is a garbage format and it continues to amaze me that it is still used as widely as it is.  Any format that's going to have a problem with commonly-used punctuation is not fit for use on a modern computer. 

                   

                  Imagine, if you will, a contact named Jamie O'Toole who owns a company called The "IT" Guys, LLC.  How will CSV (of any style) handle that?  The answer is, it won't. 

                  • 7. Re: Importing Data from CSV
                    John Neighbors Expert

                    Hear, hear, Mr. Castell.  You are spot on, sir!

                     

                    I've always felt the CSV format was created by someone that couldn't spell "data".   I mean, using quote marks and commas as delimiters?  Come on!

                     

                    I have 3rd parties often asking for data "out of" a GoldMine SQL DB in CSV format.  They want History, emails (including the MAILBOX records), the whole 9 yards. 

                     

                    These are most often DBAs of "other systems".  During our dialogue, we discuss the reality of all the shortcomings of CSV AND I emphasize how I can hand them all the data they want in their own SQL database (AKA ideal format), and encourage them that there would then be NO risk of ANY data loss or craziness (that CSV introduces).

                     

                    This is usually when they explain that they don't know anything about SQL databases or how to deal with them (which then makes me really question the viability of the "system" they are wanting to pump this GoldMine data into).   I tend to just hang my head and whimper quietly.

                    • 8. Re: Importing Data from CSV
                      JErste Specialist

                      This is why I prefer the Excel Add-on for GoldMine that comes with GoldMine for importing rather than relying on CSV.

                      • 9. Re: Importing Data from CSV
                        ivankarp Rookie

                        Hi there! In case you are still interesting in CSV data import into GoldMine CRM, you may take a look at data2crm migration service, from now on it offers direct CSV to GoldMine import Hope this info will be useful! Thanks

                        • 10. Re: Importing Data from CSV
                          Shaul.Bel Apprentice

                          I am not sure, but in the past you should have Company or contact to be able to create a contact record by importing.

                          • 11. Re: Importing Data from CSV
                            BPayne Rookie

                            Doug,

                            While I agree that CSV files are old, they are still extremely useful and work great when they are setup properly. When done right, virtually all text data can be successfully put in a CSV and used without confusion. Using the combination of delimiter (usually a comma or tab character) and separator (my term - the character on both ends of the field data to create a data block) can make all combinations of data work. The problem is that people who create CSV files often do not follow the rules that allow proper interpretation of these files - this includes older versions of Excel. I have worked with CSV files for 30 years and expect that they won't go away any time soon because they are just too easy and useful.

                            • 12. Re: Importing Data from CSV
                              DCastell Expert

                              Oh, I don't deny their usefulness, but there are so many more robust formats...

                               

                              To me, a modern tool/company demanding that CSV it's data interchange format of choice is sort of like demanding that sales people learn and use Morse code to communicate.  Sure, it's possible that the information will get from point A to B, but it will be a bigger PITA than it needs to be.

                              • 13. Re: Importing Data from CSV
                                HGreth Rookie

                                Doug (and anyone who wants to import CSV)

                                 

                                Your example is the reason, why we use the PIPE |  character for field separation in any CSV data. We are doing this since the beginning of IT-time and ist works fine. Should there be any other issue, you can change or add the ^ sign. We like to import CSV with the GoldMine Wizard, if a client does not use one of the professional add-ons we all know  (special thanks to Dave Petonic and David Evans !)

                                • 14. Re: Importing Data from CSV
                                  AAmes Rookie

                                  On occasion I've run into a problem with the wizard importing 0 of X files. Invariably when I have this problem it is because I have forgotten to release my active filter. As soon as I release the filter and try the import again it is able to import normally.