Office of the Historian

June 2023

Advisory Committee on Historical Diplomatic Documentation June 12–13, 2023

Minutes

Committee Members

  • James Goldgeier, Chairman
  • Kristin Hoganson
  • William Inboden
  • Adriane Lentz-Smith
  • Sharon Leon
  • Nancy McGovern
  • Timothy Naftali
  • Deborah Pearlstein
  • Sarah Snyder

Office of the Historian

  • Kristin Ahlberg
  • Carl Ashley
  • Margaret Ball
  • Forrest Barnum
  • Sara Berndt
  • Josh Botts
  • Tiffany Cabrera
  • Mandy Chalou
  • Elizabeth Charles
  • Kathryn David
  • Cynthia Doell
  • Thomas Faith
  • Lynette Evans-Tiernan
  • David Geyer
  • Renée Goings
  • Michelle Guzman
  • Charles Hawley
  • Kerry Hite
  • Adam Howard
  • Alina Khachtourian
  • Virginia Kinniburgh
  • Michael McCoyer
  • Brad Morith
  • Christopher Morrison
  • Mircea Munteanu
  • David Nickles
  • Nicole Orphanides
  • Paul Pitman
  • Alexander Poster
  • Kathleen Rasmussen
  • Matthew Regan
  • Amanda Ross
  • Seth Rotramel
  • Daniel Rubin
  • Ashley Schofield
  • Nathaniel Smith
  • Douglas Sun
  • Melissa Jane Taylor
  • Chris Tudda
  • Dean Weatherhead
  • Joseph Wicentowski
  • Alex Wieland
  • James Wilson
  • Louise Woodroofe

Bureau of Administration

  • Jeff Charlston
  • Corynne Gerow
  • Timothy Kootz
  • Thomas Opstal
  • Marvin Russell
  • Eric Stein

National Archives and Records Administration

  • Cathleen Brennan
  • Elizabeth Fidler
  • William Fischer
  • David Langbart
  • Don McIlwain

Public

  • Over 30 members of the public

Open Session, June 12

Presentation on Compiling Foreign Relations

Goldgeier opened the meeting by welcoming all attendees as well as the newest member of the Committee, Sarah Snyder of American University, who will represent the Society for Historians of American Foreign Relations (SHAFR). He also thanked Melani McAllister for her three years of service as SHAFR’s representative to the Committee. He then turned the meeting over to Rasmussen.

Rasmussen introduced Charles and noted her published FRUS volume on the Soviet Union, her INF volume, which is in declassification, and relayed both Charles’s and McCoyer’s resumes. She noted that McCoyer’s Global Issues volume is in declassification and that he and Ahlberg are currently co-compiling Foreign Relations, 1981–1988, Volume XLII, Refugees and Immigration, 1975–1984.

Charles referred to her slides, which describe the thinking that goes into researching, compiling, declassification, and the editing and publishing process of a FRUS volume. She noted that the documents compilers choose for a volume are only the tip of the iceberg of the available documentation throughout various archival repositories. FRUS should therefore be used as a guide for researchers, which is why the footnotes, as well as Editorial Notes, are so important. Those are the breadcrumbs compilers provide for further research.

Charles also encouraged historians to read the front matter in a volume—the preface, the lists of abbreviations, names, and sources—to provide further invaluable information. She then turned it over to McCoyer and said he will show how researchers can use FRUS as a gateway to other online documentation at the NARA and CIA websites.

McCoyer said he would use Foreign Relations, 1977–1980, Volume XV, Central America for this example. McCoyer demonstrated how a researcher can click on an individual chapter, in this case the documents on El Salvador, in 1979. He noted a telegram that had three reference telegrams, and that if a researcher wants to learn more about a telegram referenced in a footnote they can do this with NARA’s Access to Archival Databases (AAD) site. Using the AAD search function, a researcher can click on “Diplomatic Records” then click on “Electronic Telegrams” for the desired year, in this case 1979. They can then copy the D-reel number (the microfilm number originally designated by the Department) from the Office of the Historian (OH) website and paste it into the “Film Number” field. (It will also work in the “General Search” field.) Upon hitting “Enter” McCoyer showed that for his example he could see that the telegram has been declassified and is available in text form. The researcher can repeat this process for other telegrams or put in the specific telegram numbers to get other telegrams.

McCoyer then showed how a researcher can use FRUS with the CIA’s Freedom of Information (FOIA) Reading Room database, available through CIA’s website. This time he used the 1977–1980, Volume XVI, Southern Africa volume. This volume contains a covert action, so if a researcher wants to learn more about it, or to see what else the CIA has published, they can use a similar process as with AAD. He also encouraged users to read the Note on Covert Action that OH includes in each volume that contains a covert action, which provides a brief history of the covert action process.

He noted that in the Angola chapter of the Southern Africa volume Document 16 is a memorandum from Director of Central Intelligence Turner to National Security Advisor Brzezinski. Footnote two to this document notes a document that was found in the CIA records, so a researcher can copy and paste the job number into the (FOIA) Reading Room database in the “advanced search” option. This gives the researcher access to the available records from CIA’s CREST declassification program, which reviews documents under the 25-year program. If the researcher uses an asterisk before and after the job number and adds the box number included in the footnote it will show all the other documents from that box number that are publicly available.

McCoyer also encouraged researchers to look at all the other collections contained on the CIA’s website.

General discussion

There was a round of applause for Charles’s and McCoyer’s presentations. Rasmussen thanked the presenters, noted the utility of the job numbers as a research tool in CIA records, and invited questions from the audience. Wilson, hosting the online forum for the meeting, also invited questions.

Goldgeier asked about the decision-making process for selecting records within a FRUS volume. He noted that the declassification process begins only after the selection has been completed and wondered if there is a way to leave a clue for future readers about documents that were not selected for publication or to be cited in the annotation.

Charles responded that FRUS is the official documentary record of U.S. foreign policy and must therefore include the most essential records, even if they are already fully declassified in full and are readily available. For subjects that already have been heavily declassified this can create a challenge regarding the page limits for a volume. Charles described a strategy that counts the number of pages required for the most essential documentation and then winnows the rest of the documents by their actual impact on the policy process. Charles also noted that large regional volumes tend to be even more condensed than volumes on more narrow subjects, and that the footnotes are incredibly valuable resources for research clues. To sum up, Charles stated: “it is a challenge, but we aim high.”

Leon lauded McCoyer’s presentation for its “esoteric knowledge of citations,” and asked if there is an explanation of all of this on the Department of State or Office of the Historian’s websites. Charles indicated that there is a “How to Search FRUS,” entry on OH’s public website.

Langbart described the detailed guide on the NARA website and opined that no researchers seem to read it. He also added his “two cents” that the D-reel numbers are not as effective citations as the message reference numbers which specify individual documents and have better overall searching power. Rasmussen replied that the message reference numbers are printed within the documents in a FRUS volume and the D-reel numbers are provided in the source notes.

A member of the online audience asked about tips for searching foreign policy documentation from an earlier historical period such as the Theodore Roosevelt presidential administration. McCoyer noted that the indexing for the earlier FRUS volumes was excellent and can be quite helpful. Charles added that FRUS volumes from the earlier years of the twentieth century only included Department of State records. It is after the 1991 FRUS statute that the series included records for other agencies.

Langbart commented that pre-1906 FRUS has no source citations and researchers for this period must understand the Department of State filing system. Post-1906, with the Department’s adoption of a numerical file, FRUS began to include source citations. Furthermore, all FRUS volumes are now digitized and available online.

Goldgeier asked if scholars contact FRUS compilers to seek guidance about the research process. Charles answered affirmatively and said OH historians are gladly willing to share knowledge about research with the community.

William Burr inquired about the status of FRUS, 1977–1980, Volume XIV, Japan; Korea and the Access to Archival Databases (AAD) for electronic Department of State telegrams from the 1980s at NARA. Wilson posted the weblink to the Status of the Series on the OH website. Goings responded that there was no comment from OH.

Langbart responded that there was no comment from NARA.

Rasmussen again thanked Charles and McCoyer for their presentations and there was another round of applause.

Approval of the Record

After brief opening remarks, Goldgeier moved for approval of the minutes from the March 2023 HAC meeting. His motion was seconded and it carried unanimously.

Remarks by the Deputy Director of the Foreign Service Institute

Goldgeier next introduced the Deputy Director of the Foreign Service Institute (FSI), Shelby Smith-Wilson. Smith-Wilson, who noted that this was her first HAC meeting, said that she appreciated the opportunity to meet committee members and welcomed the HAC’s new member, Dr. Sarah Snyder of American University.

Smith-Wilson reviewed some of OH’s recent outreach efforts, including the April 13 webinar on disability history and U.S. foreign policy and noted that on June 22 OH would participate in an event on the Lavender Scare.

Smith-Wilson underlined that FSI’s Front Office strongly supports OH’s modernization and reorganization and presented recent hiring actions. OH hired Dr. Amy Garrett to serve as Director of the Historical Studies Division (HSD) and selected a Director of the Declassification Coordination, Publishing, and Digital Initiatives (DPD) Division, whose name has not yet been released. OH also hired Lynette Evans-Tiernan as Strategic Communications Advisor and Dr. Benjamin Greene as a historian in Special Projects. Smith-Wilson closed by asking for questions from HAC members.

Report by the Executive Director

Goldgeier thanked Smith-Wilson for the good news and turned the session over to OH Director and HAC Executive Secretary Adam Howard, who stated that he had recently done a presentation on OH’s FRUS and policy-support work to the Foreign Policy Association. Goldgeier asked who in the Department can request policy support from OH. Goings replied that anyone in the Department can contact OH and that the office responds depending on priorities and available resources. In some cases, OH helps requesters help themselves by doing their own research.

Report by the General Editor

Following Howard’s comments, FRUS General Editor Kathleen Rasmussen detailed recent progress developments in the FRUS series. Rasmussen noted that the National Archives had restarted the transfer of classified collections from the Presidential Libraries to NARA II. Once the moves have been completed, FRUS research is expected to resume. Rasmussen concluded by inviting questions from HAC members and the audience.

Inboden asked if FSI had been involved with the move; Rasmussen replied that it was a NARA initiative. Naftali asked how much warning NARA had provided, and Goings responded that NARA gave as much warning as possible under the circumstances.

Goldgeier asked Smith-Wilson about the budget for the coming fiscal year. Smith-Wilson briefly outlined the process by which FSI submits its budget for review within the Management family, after which the Department reviews the budget to align it with the Secretary’s priorities.

After thanking the speakers, members of the public who attended the meeting online, and Deputy Director Shelby Smith-Wilson, Goldgeier adjourned the open session of the meeting.

Closed Session, June 12

Briefing on the Status of Department Records

Goings presented William Inboden with a pin to recognize his 20 years of federal service and Howard presented David Langbart with a certificate of appreciation in recognition of the many services he has provided to OH over the years.

Goldgeier then recognized Tim Kootz from IPS for his presentation. Kootz gave a brief recap of his career to introduce himself to the committee. He then stated he would deliver some updates on IPS’s recent efforts.

He announced that IPS’s FOIA program had been moved to Charleston, South Carolina. The Records Service Center (RSC) would not be moved however, which he noted would be a relief to the historians. The RSC might move in the next five years, although talks were in the early stages on this.

Kootz stated that the huge pandemic-induced FOIA backlog is being reduced. Kootz opined that IPS needed to be restructured as it is not optimally organized at present. There needs to be single leadership service areas.

Kootz then turned to technology and machine learning. He noted that the machine learning initiative shows promise in reducing the FOIA backlog. IPS hopes that machine learning declassification would eventually lead to proactive disclosure of records. He noted that a new public access website where researchers could submit FOIA requests and see their status is live. The site is tied into the national FOIA portal for ease of use.

Turning to digitization, he noted that IPS is working with OH to identify issues with born-digital records. The goal is to use metadata and other tools to enrich the researcher experience. He noted that many paper records had not been digitized as it would be cost and time prohibitive however NARA and the federal records center would accession them over the next five years, even in paper form. He noted that the new IPS facility will likely be smaller than the current one, which would mean a focus on digitizing what is possible with space considerations.

He then turned to Jeff Charlston for a presentation on machine learning. Charlston noted that although the machine learning program has only been working for seven months it is already paying dividends. Savings of two-thirds of total labor time are being seen so far. The 1997 cables have been processed; there are some grey areas where the machine makes mistakes but it is still a major time savings. Efforts are now focused on the 1998 cables and some material from the 1995 and 1996 cables. The 1998 cables should be processed by the end of the year.

Charlston then outlined the next steps being taken by IPS. Charlston concluded by noting that the eventual goal is direct release following review. If this could be accomplished the time savings would be impressive.

At 1:20 p.m. the conversation shifted to a question-and-answer session about machine learning and declassification.

Pearlstein asked if Charlston could say more about interagency collaboration efforts regarding machine learning. Charlston stated that regarding declassification, only as of this week had this program started. Charlston added that State is faster than other agencies because the template for cables has been very standardized, and that machine learning would free up labor for MDR backlogs.

Kootz stated that machine learning would contribute to improved efficiency and that State needs new solutions. The fact that the machines help identify agency equities, he added, was very helpful.

Pearlstein asked if Kootz anticipated retaining the savings of labor as time moved forward. Kootz replied that the machine keeps moving faster but it is necessary to tweak the model depending on the set of data. He said it takes one to two months to remodel the machine for each year’s data.

Naftali asked how Charlston knew some documents had been improperly classified by the machine. Charleston said he knew because subsequent human reviews of a sample set of data had been conducted. He stated the next goal is automating interagency referrals for cables.

Naftali asked how Charlston would document the history of machine learning. Charlston said that human labor would document how the machine had been tweaked over time and act as a backstop. Kootz added that machines are consistent, so if they are consistently wrong it is easy to fix.

Goldgeier asked about State documents without other equities. He also inquired about how interagency machine learning collaboration had developed. Charlston said that in the early stages, agencies deliberately refrained from collaborating so they could develop different ideas, especially since each agency had different declassification requirements, and that the agencies are just starting to collaborate. He added that machine learning could also be a valuable tool to prevent human over-classification. Charlston stated that the NDC determines agency equity, and it is a time-consuming process since documents must be sent to different agencies while other agencies wait for a response. He said he is looking for a way to streamline and speed up this process through artificial intelligence.

Pearlstein asked if the reorganization of human labor would slow declassification processes.

Kootz said it would not, and that much of the reorganization would take place in divisions that do not affect the pace of declassification. He said he is trying to shift people to places to offset backlogs.

Leon asked if different interagency Artificial Intelligence (AI) models could be used in combination to make declassification happen more quickly. Charlston said no, stating that each agency has a system tailored to its own declassification guidelines, but in the future a combination strategy could have huge potential. Kootz said such a method could one day be applied to NATO equities.

Inboden asked if the CIA had begun a declassification machine learning project. Charlston said not to his knowledge, but if many agencies adopt such systems, the Intelligence Community might develop one, too.

Goldgeier stated that he would collaborate with IPS on the HAC annual report to make sure all details are accurate. Kootz thanked Goldgeier and Goldgeier thanked Kootz.

The session adjourned at 1:50 p.m.

Briefing from the National Archives and Records Administration

James Goldgeier called the session to order and welcomed William Bosanko, Chief Operating Officer (COO) of the National Archives and Records Administration.

Bosanko, who thanked the HAC for the opportunity to provide an update on NARA, began by welcoming the arrival of the new Archivist of the United States, Dr. Colleen Shogan. According to Bosanko, Shogan is perhaps the best prepared Archivist and has demonstrated strong interest in the needs of researchers.

Turning to the National Declassification Center (NDC), Bosanko reported that the NDC has completed declassification review of the 1980 P- and N-reels (which contain microfilm of paper records and NODIS telegrams) and is starting on 1981. The NDC has received NARA approval to operate the Remote Archive Capture (RAC) system.

The NDC has also logged all outstanding FOIA and MDR requests relating to the classified holdings of the Presidential Libraries that have been consolidated at NARA II. Bosanko stated that most of the backlog cases need agency responses and that the NDC will try to encourage agencies to respond.

Bosanko noted that NARA had restarted the plan to consolidate classified holdings from the Presidential Libraries, which began in 2015. COVID and funding problems caused delays, but NARA now expects to finish by spring 2024 depending on when the Department of Defense can provide transit.

At this point, NARA plans to move the Reagan and Nixon materials in October 2023, the George H.W. Bush and LBJ materials in November, and the Clinton materials in December. NARA plans to complete the project by moving materials from the Hoover, Roosevelt, and Truman Libraries in Spring 2024.

Bosanko noted that the paper records from the Obama administration have already been returned to the Washington area from Chicago, and that most of the classified materials from the Trump administration are stored at NARA I. NARA will need to build more storage space to accommodate the Obama and Trump materials because neither administration plans to construct a library.

NARA plans to hire 15 to 20 archives technicians for the NDC to help with special access requests and declassification review. The additional technicians will free the NDC staff to focus on other work. The NDC has taken over a backlog of about 28,000 MDRs from the Presidential Libraries, more than half of which came from the Reagan and George H.W. Bush libraries. The NDC will process MDRs in the order in which they were received.

Bosanko stated that FRUS researchers should be able to use classified materials at the George H.W. Bush and Clinton Libraries until two or three weeks before their scheduled moves in November and December 2023. He also stated that FRUS researchers should be able to access the collections at the NDC about two or three weeks after they arrive at NARA II. Bosanko stated that the staff at the Presidential Libraries had had years to prepare for the move, so it should not take months for them to get ready.

Goldgeier asked Bosanko how the move would affect existing MDR requests. Bosanko stated that the system should work more efficiently. Goldgeier then asked Bosanko what would count as success in handling the backlog of 28,000 MDR requests. Bosanko stated that he hoped that the NDC would be able to complete better quality reviews, reducing the need for appeals to the Interagency Security Classification Appeals Panel (ISCAP).

Naftali asked what would happen with the collections at the Presidential Libraries that still needed to be processed. Bosanko said that it has not been decided but that the NDC could review collections before they were processed. Bosanko further noted that the Presidential Libraries lack the technical equipment to handle special media, all of which is at NARA II. Naftali offered to meet Bosanko for an informal discussion of practical solutions to some of the problems facing the Presidential Libraries. Bosanko agreed.

Leon asked whether the 15 archives technicians will be internal hires. Bosanko stated that the Presidential Libraries will not lose positions if people transfer to NARA II.

Inboden referred to the MDR requests and asked whether the NDC staff would be ready to process them once all of the materials arrive in the spring of 2024. Bosanko replied that the NDC staff has been ready and is talented and knowledgeable.

Lentz-Smith asked what would happen if NARA could simply focus on the materials that really should still be classified after 30 years. Bosanko stated that this is a moment when there is more interest in questions relating to classified documents. NARA has been leveraging attention on Capitol Hill to get more resources for the NDC, including 15 positions for the NDC and 20 positions for special access.

Bosanko noted that his goal was to build broad support for NARA, and he hopes that the HAC’s report would contribute to that end. David Langbart noted that NARA also lacks the staff to process and declassify agency records. Bosanko stated that NARA lacks staff to handle the wave of electronic records that is coming but that NARA cannot ignore the remaining paper records.

Pearlstein asked if NARA has attempted to use AI to process electronic records. Bosanko stated that the entire executive branch needs to work together to figure out how to use AI.

Goldgeier thanked Bosanko for coming to the HAC to provide so much information and noted that the HAC would look forward to a readout from upcoming discussions between Naftali and Bosanko.

Closed Session, June 13

Demonstration on eRecords

Josh Botts led a discussion and demonstration of the Department’s eRecords system for IPS and OH members.

Discussion on the Foreign Relations Series

The HAC led a discussion for all members of the OH staff on issues related to the Foreign Relations series, the planning for the George W. Bush subseries, and other content relevant to the FRUS series.