The actual report itself basically states how the study was done. What is more interesting, and significantly longer (90 pages) is the "Contractor’s Report on Initial Pilot Study."

The ultimate goal of the project was to identify the amount of errors within the credit reports from the three major credit reporting agencies (CRAs; TransUnion, Experion, Equifax), and then to see how many of the participants went on to follow procedures to get their information corrected.

The group contracted to perform the study contacted 254 households, and even though 65% indicated interest in participating, only 30 (~12%) participated fully. The contractor worked with the participants closely to help them identify errors within their credit reports. 3 found errors that would have a "material effect" (meaning they could be denied loans, etc. as a result of the error), 25 found errors that would not have a material effect, and 2 found errors that could possibly have a material effect.

7 of the households indicated they planned to file a dispute, but upon followup only 2 of them actually did file, and two indicated that they had bad experiences with the process and never got the dispute filing successfully done.

The report pointed out, "The sample size of 30 is much too small to be a reliable indicator of information in the universe of credit reports. Beyond the assessment of errors as material or not, there are many questions that a national study based on this methodology could address."

So basically the purpose of this project was to validate the methodology before moving on to a new, larger, study group size.

The following describes the security applied to the information collected during the study:

"Data Security
Special consideration must be given to protection of consumers’ confidential information and formal protocols were established for this purpose in the pilot study. In the course of the study, each participating institution and individual members of the research team took care to work in conformity with relevant data safeguards described in "Financial Institutions and Customer Data: Complying with the Safeguards Rule", extracted from http:/www.ftc.gov/bcp/conline/pubs/buspubs/safeguards.htm. Processes were approved by the university’s IRB committee. Each member of the research team was provided with detailed descriptions of the data that will be handled in the course of the study (including prototypical credit reports) and was trained to conform to the research protocols for protecting the consumer against the release or misuse of data. Access to information was limited to persons who had a need to see it.

Information systems. Consumer data in hard-copy form were stored in a secured area (locked office and locked cabinet). Identifying information (consumer SSN and all but the last four digits of credit account numbers) were suppressed in hard copies of the credit reports that were mailed to the consumers and used by the researchers. A unique 29 identifier was used as a cross-reference between consumer contact information (name, address and phone number) and information in the credit report. Electronic copies of consumer contact information were kept in separate computer files from those used to record data derived from reviews of the credit reports. We thus retained background information (such as name and address for contacting the consumer for the credit review) separately from the credit-report and demographic data. Password protection was used to limit access to computerized data. In the course of reviewing the credit files, extracts from the credit reports and consumers’ demographic data and responses to inquiries from the follow-up questionnaire were placed in database for which the myFICO account number and consumer ZIP code were the only consumer identifiers. Upon receiving written notice from the FTC that lists of consumer contacts and crossreferences of consumer information with credit-report information are no longer needed, the research team will permanently delete the computer files that contain consumer names, addresses, etc. They will shred paper records with the consumer contact information."

They didn't mention requiring signed NDAs from those collecting the information; they surely obtained them, though, didn't they?

It's curious that a couple of the exhibits at the end of the report were redacted...the FTC Solicitation Letter (probably to prevent phishing activities using the same letter)...and the Consumer Consent Form (probably also related to preventing phishing and other types of similar fraud prevention).

The purpose of this project was to validate the methodology. However, even with the small sample size, it is interesting and enlightening to see that all 30 participants found errors within their credit reports. This demonstrates the need for not only CRAs, but also any company handling personally identifiable information (PII), to have procedures and tools in place to help ensure the accuracy and integrity of the data. Most do not have effective or comprehensive ways to ensure data accuracy. However, this (ensuring the accuracy of PII) is, after all, a requirement of many laws and regulations throughout the world, such as the European Union Data Protection Directive, Canada's PIPEDA, and many, many others.