Duplicate Check and Importing

During the import process, the Duplicate Check feature plays the major role of determining whether or not the information being imported will be added to an existing Constituent record or requires the creation of a new Constituent record.

The duplicate check button opens the following screen:

The Duplicate Check relies on the information which exists on the Basic Card of a Constituent record to determine if the data to be imported belongs to an existing Constituent or should be used to create a new Constituent record.
Dependent upon your selections on the Duplicate Check screen and the type of data being imported, the resulting duplicate validation screen should reveal the accuracy of the data to be imported when compared to existing Constituent data.

Meaning, if you are confident the data being imported is related to existing constituent records, the value displayed for Number of Duplicates Found should be high or the exact same number as Number of rows in source file.

For example, you are importing 100 gifts received from existing constituents, in which case the Number of rows in source file would display 100 (100 rows of data to be imported), after running the Duplicate Validation you should see a value of 100 next to Number of duplicates found.   The system compared the values in the data file for Last Name (exact), First Name (similiar) and Email (exact) against these fields on every constituent record in your database and found those same values, therefore it was able to ‘match’ the incoming data with existing constituent records.

If you were importing data which you know is for new constituents that do not currently exist in your database, then you would want the Number of Duplicates found value to be zero, this would denote that when the data file Last Name/First Name/Email fields were checked against existing records, no matches were found, so the import will create new Constituent records.

If your file is going to contain a mixture of both new records and data to be added to existing Constituents then you would have a lower number of Duplicates found but it would still be greater than zero.

In the example directly above, the system would create 760 new Constituent records along with whatever other data is in your file (gifts, contact cards, etc) BUT it would only add the ‘other data’ (gifts, contact cards) to 84 existing records that it found in your database.

Again, what the system considers a duplicate (existing) versus a new record depends entirely on the Duplicate Check options you select.
If Last Name (Smitty) is set to Exact but First Name (Tom) is set to similar, that allows the system to make an educated guess that Tom is ‘close enough’ to Thomas to consider this a duplicate or existing record.  If you had exact for both values, then unless the constituent record is set up as Tom Smitty, the import value of Thomas Smitty would create a new record rather than add the data to an existing record.  The addition of the email field would help clarify to the system just which Smitty the data belongs to.

In general, try the following settings, validate and see what the results are: 
 Last Name (exact)/First Name (similar)/Zip Code (exact)
Last Name (exact)/First Name (similar)/Email (exact)

Dependent upon the data fields in the import file, you may have to ‘play’ with the Duplicate check options to get the best results for your import based on how your organization’s Constituent records are established.
As with any import, the most important fact to remember is:  “Garbage in – Garbage out”

Please remember that software is very black and white and what may appear logical in your mind as to why a record should be considered a duplicate or should not have been considered a duplicate, may not equate the same way within the software.  There will always be cases where the Duplicate Check settings may still erroneously create a new record for an existing constituent or vice versa add data to an incorrect existing record.
After importing, use the Duplicate Check Utility (vastly different from the Duplicate Check import validation) to ‘catch’ any erroneously created records.  The Duplicate Check Utility can be found via Admin>Utilities>Duplicate Check.
Insufficient Duplicate Check message

If after the validation you receive a message for certain rows of data that states Insufficient Duplicate Check criteria is available,  this means that based on the data for that row the duplicate check selections did not contain enough fields for the system to verify whether or not a record is a duplicate. 

If you are importing information for Tom Smitty at zip code 78701 and there happens to be two records for Tom Smitty with that same zip code (but their street address is different), the system would return the insufficient data message, meaning you may have to add an additional duplicate check selection such as email or address line 1 so the system can make a distinction between the two records and then compare that to the data being imported. It is the system’s way of saying it cannot determine if there are duplicates because it is not checking enough information in the existing constituent’s records. 

Required Tags:

Article Type: 
Product Info
Product Line: 
Abila Fundraising 50
Product Module/Feature: 

No votes yet