Monday, March 19, 2012
To speed up processing with SSIS Lookup Data Flow Item use as little columns as necessary
The number of records in a table that was the basis of a Lookup Data flow Item has grown considerateley, whilst the processing speed of the Package was diminishing. In the Lookup item properties for the connection, I had been using the table name as datasource. When running the package interactively the Lookup caching was built up steadily in chuncks until all data had been loaded. I have changed the property to use a SELECT statement that uses only the essential columns. While runnng the package now, the Lookup cache buildup is a lot faster.