EXPERT RESPONSE
In this case I would recommend creating a temporary table and using the BULK INSERT command to populate it with the data from the text file. BULK INSERT is very fast and if the destination table is empty and has no indexes, you can load millions of rows within seconds. For more details about BULK INSERT, see "BULK insert to import data from file to SQL Server database."
Once you have data in the temporary table, you can use series of INSERT, UPDATE and DELETE statements to populate other tables in the database. In some scenarios I often add an ID column to the temporary table, and then use the UPDATE…FROM statement to join permanent tables on one or more columns in the temporary table and populate the ID column. After that, I know that any rows in the temporary table where the ID equals NULL are new rows and need to be inserted. The rows where the ID column has a value are considered a match. For those rows I use another UPDATE...FROM statement where I join on the ID column and copy the values from the temporary table to the existing rows. This technique is very useful for data imports using set-based operations rather than row-by-row processing.
|
