In March 2024, I was a panelist at the NICAR2024 conference in Baltimore. NICAR (National Institute for Computer-Assisted Reporting) is the data journalism program of Investigative Reporters & Editors (IRE), the U.S.-based professional organization for investigative journalists. The annual NICAR conference brings together more than 1,000 reporters to share stories, learn skills and hear the latest about the ever-growing specialty of data journalism.
With co-panelists Cam Rodriguez of the Illinois Answers Project and Barbara Gray of the Newmark Graduate School of Journalism at CUNY, our panel aimed to show journalists that by tapping into historical archives and datasets, we can tie the past to the present in a more direct way, pulling it into visuals, fact-checking, data reporting and narrative writing.
I’m sharing here my contribution to this panel, which included:
How to find and use archives, how to find out how they change over time, how to meet the archivists, geneologists and datakeepers you’ll need, how to access records manually or programmatically, how to create your own datasets from archives, how to maintain those datasets, and how to connect the past to your present day reporting.
You can also find Barbara Gray’s tipsheet at this link Archives for Investigative Data.
What’s in an archive?
Archives = primary sources
Example: Manuscripts
Abolitionist John Brown wrote this letter while waiting for his sentence of hanging to be carried out in 1859.
Letter, Charlestown, Jefferson Co., 'Va., to Mrs. George L. Sterns, Boston, Mass. In which John Brown expresses appreciation for Mrs. Stern's letter of the 8th inst. and states his wishes as to attendants at his coming execution.
From the Howard University Moorland-Spingarn Research Center in Washington DC. Fortunately, this letter has been digitized.
Archives = Older documents and materials
Example: Work visa & green card data
On the Department of Labor’s Office of Foreign Labor Certification site, the work visa datasets and green card datasets at include the years 2008-2024 https://www.dol.gov/agencies/eta/foreign-labor/performance
But you can find earlier visa and green card datasets at the National Archives 2000-2018
Archives = Materials marked for retention not destroyed
Look for records systems and retention policies described for the documents and data related to your topic
Record retention policies for U.S. agencies are published periodically in the Federal Register under the Privacy Act of 1974
This notice in the Federal Register relates to the visa and green card data mentioned above. https://www.federalregister.gov/documents/2022/02/14/2022-03063/privacy-act-of-1974-system-of-records
Where are archival materials maintained?
There are various names for archival collections in locations around the United States. They may be in government agency collections, in public or university library collections, in museums, corporations, or in private hands.
Archives: like Federal, state, local government archives
Examples: National Archives (NARA) and Presidential libraries: state archives, city archives
Special collections in libraries
Example: Manuscript Division at Library of Congress
Museums
Corporate Archives
Example: Time Inc archives at the New York Historical Society, includes the Time Inc Research Center (finding aid)
I was surprised to find an archive of my workplace from 1980-1988, the Editorial Reference library at Time Inc.!
College and University Archives
Historical Societies
Example: Central Rappahannock Heritage Center in Fredericksburg, Virginia
How to find archival collections
It may be difficult to locate where the archival collection you are hunting will be found. These tools compiled by archivists and librarians will help:
ArchiveGrid https://researchworks.oclc.org/archivegrid/
Archive Finder http://archives.chadwyck.com/home.do
National Union Catalog of Manuscript Collections https://www.loc.gov/coll/nucmc/
WorldCat https://search.worldcat.org
Online Archives of California http://www.oac.cdlib.org
Reclaim the Records:
a not-for-profit activist group of genealogists, historians, researchers, and open government advocates. We identify important genealogical records sets that ought to be in the public domain but which are being wrongly restricted by government archives, libraries, and agencies. We file Freedom of Information and Open Data requests to get that public data released back to the public. And if the government doesn’t comply, we take them to court. https://www.reclaimtherecords.org
Example of records “freed” by Reclaim the Records: New York City 1924 list of registered voters https://www.reclaimtherecords.org/records-request/8/
I found my immigrant grandparents listed and the address of the Lower East Side tenement where they lived in 1924.
Using archives: FINDING AIDS!
Finding aids are the most valuable tool for locating information within an archival collections. Written by the archivists who maintain the collections, the finding aid will guide you to the specific shelf and box numbers, and often have details on what documents are in each box. Some collections have been digitized, but most, alas, are not.
It’s important to obtain the finding aid, and contact the archivist to request the items, before your trip to an archive. Often, boxes are stored “off-site” and must be retrieved by the archivist in order for the box to be there for you to see.
Sometimes you are able to request that an item be copied for you. Or you can find a local researcher to go through boxes for you for a fee. But it’s best to look for yourself; you may find other items of interest while poring over the boxes. Think of a treasure hunt!
Here’s a clip from the finding aid to files of the Time Inc Editorial Research Library, formerly called The Morgue. https://findingaids.library.nyu.edu/nyhs/timeinc_ms3009_rg13/
Here are some of the guides to help you work in an archive.
“Using Archives: A Guide to Effective Research,” from the Society of American Archivists https://www2.archivists.org/usingarchives
On the National Archives site, there are tools to help find electronic records, hidden in the catalog:
National Archives (NARA) Catalog https://www.archives.gov/research/catalog
Electronic Records in the Catalog https://www.archives.gov/research/electronic-records/access-in-catalog-faqs
List of electronic records in the catalog https://catalog.archives.gov/search?levelOfDescription=series&referenceUnits=%22National%20Archives%20at%20College%20Park%20-%20Electronic%20Records%22
Access to Archival Databases https://aad.archives.gov/aad/
Archives Arrangement and Terminology
Archives are arranged in a different way than books in a library. You find matrials within a specific collection, or — for governments — organized by the administrative unit, for example a department and unit within a department and then, in the case of US federal documents:
Record Group
Collection
Series
File Unit
Alternative Repositories
Beyond archives held in government and university collections, there are innumerable private collections that can be found using the locator guides above. Some examples I use frequently are:
National Security Archive https://nsarchive.gwu.edu Held within the George Washington University’s Gelman Library, a non-profit described as:
Founded in 1985 by journalists and scholars to check rising government secrecy, the National Security Archive combines a unique range of functions: investigative journalism center, research institute on international affairs, library and archive of declassified U.S. documents ("the world's largest nongovernmental collection" according to the Los Angeles Times), leading non-profit user of the U.S. Freedom of Information Act, public interest law firm defending and expanding public access to government information, global advocate of open government, and indexer and publisher of former secrets.
Mary Ferrell Foundation http://www.maryferrell.org, an online-only subscription based archive of digitized records relating to the assassinations of John F Kennedy, Robert F Kennedy and Rev. Martin Luther King Jr.
The Mary Ferrell Foundation (MFF) is a non-profit 501(c)(3) group engaged in an ongoing effort to bring accessible and interactive history to a new generation of critical thinkers.
With a wide topic base including the assassinations of the 1960s, the Watergate scandal, and post-Watergate intelligence abuse investigations, the MFF’s vast digital archive at www.maryferrell.org contains nearly 2 million pages of documents, government reports, books, essays, hours of multimedia, and innovative research tools.
Ancestry.com https://www.ancestry.com The popular fee-based powerhouse includes vast collections of historic records, documents, photos of global interest to genealogy fans, but also of great use to journalists’ investigations involving history: U.S. Census records since 1790, for example, have been digitized and made searchable, even the handwritten entries by census takers.
In a post to come, I’ll dive deeper into research at a few of these collections and how — even in these physical or electronic boxes of history — you can find something new.