Introduction

In 1990, longstanding tensions over U.S. Government transparency policy came to a head. For the preceding 200 years, the executive branch routinely released official diplomatic documents to the congress and the public. Since 1861, the Department of State’s Foreign Relations of the United States (FRUS) series served as the leading instrument of this tradition. While the Department published FRUS volumes nearly contemporaneously with the events they documented in the 19th century, the timeliness of the series receded over the course of the 20th century. Since the 1930s, volumes appeared decades after the events that they documented.

International and bureaucratic dynamics contributed to this shift in U.S. Government transparency. The increasing tardiness of the series coincided with the growth of complex bureaucracies to manage U.S. foreign policy and to maintain, review, and release government records. The Department of State’s influence over foreign affairs—and its control over publishing records relating to foreign policy—diminished as more government agencies assumed international responsibilities. Additionally, as U.S. engagement in global affairs expanded and grew more multilateral, policymakers rebalanced the value of openness in light of the imperative to maintain good relations with other governments.

By the 1980s, those trends brought the U.S. Government’s commitment to openness into question. Guardians of security, representing longstanding concerns that publishing foreign policy documents endangered vital national interests, prevented the release of important records, which jeopardized the credibility of the series. Transparency advocates, who championed equally venerable traditions of open government, tried to protect the FRUS series from these restrictive impulses, but suffered bureaucratic and policy defeats that forced them to adapt to new constraints. Ironically, the Department published volumes marred by these trends at the end of the decade, just as Cold War tensions eased and long-closed Soviet/Russian and East European archives began to open. Liberalization in the Communist bloc, coupled with the post-Watergate erosion of public trust in the U.S. Government, helped empower transparency reformers as they criticized the disturbing trajectory of the FRUS series in the late 1980s.

After considerable debate, Congress affirmed openness as a key tenet of American governmental practice in 1991. The FRUS statute legislated standards and processes for disclosing government records reflecting a “thorough, accurate, and reliable” accounting of past U.S. foreign policy decisions and significant diplomatic activities. The statute also reaffirmed the need to evaluate such records for potential damage their release might cause to diplomatic activities, military operations, intelligence sources and methods, and other sensitivities. In doing so, Congress formalized “responsible transparency” for a new era by building upon two centuries of precedent, pragmatic compromise, adherence to the principle of openness, and evolving perceptions of risk and reward in acknowledging secret deliberations and actions.

This book traces the evolution of “responsible transparency,” as manifested by the Foreign Relations series, from the earliest days of the republic through the efforts undertaken across the U.S. Government to implement the 1991 FRUS statute. The “responsible” in “responsible transparency” references two interrelated dynamics. The most obvious one is substantive. Too much transparency can damage national security and too little can compromise democratic legitimacy. Most advocates of openness accept limitations on disclosure to protect important interests and the safety of individuals. At the same time, most guardians of security acknowledge that government activities cannot be withheld from the public indefinitely. While often employing rhetorical absolutes, both sides in the debate usually accept a middle ground position influenced by both principle and pragmatism informed by shifting geopolitical and institutional contexts.

The other, less obvious, dynamic of “responsibility” in “responsible transparency” relates to the authority of those making decisions about releasing or withholding information. The official character of the Foreign Relations provides a public acknowledgement of U.S. Government decisions and actions. Because the series plays this role, it has always received stricter scrutiny than other mechanisms of disclosure, such as the Freedom of Information Act. One result of this “special treatment” is that the series represents, in aggregate, the evolution of official judgments about the costs and benefits of openness. Although these transparency decisions have often embodied technocratic and bureaucratic perspectives, they also reflect democratic control. Congress plays a critical role, through both legislation and oversight activities. Ultimately, however, the President is accountable for the policies, procedures, and regulations devised and administered by the executive branch that determine the extent of openness about U.S. Government foreign policy.

Although “responsible transparency” is an inherently normative concept—and, indeed, the contested nature of the norms that it embodies is a central theme of this book—we employ the term descriptively as the outcome of evolving efforts to strike the proper balance between security and openness. The principal exception to our descriptive use of the term occurs as we examine the 1980s, when procedural and policy shifts essentially foreclosed informed debate between advocates of transparency and guardians of security. To reflect the diminished effort to balance security and transparency and the resulting circumscribed nature of the openness regime during this period, we describe it as “translucent” rather than “transparent.”

Part I of this book describes the “Contemporaneous FRUS” of a “long 19th century” that resonated into the 1920s. Chapter 1 examines the rise of transparency practices during the early republic, when executive branch officials accepted the legitimacy of congressional demands for records and Congress acceded to executive branch discretion to determine the boundaries of openness. Chapter 2 details how the Lincoln administration formalized ad hoc antebellum precedents to inaugurate the Foreign Relations series during the Civil War. This chapter also reconstructs the de facto declassification and excision criteria employed to sanitize the documents published in the first FRUS volumes and traces their dissemination and consumption. Chapter 3 explores why Secretary of State Hamilton Fish first discontinued and then restarted FRUS. This chapter also recounts how the Department of State learned lessons from publishing Foreign Relations during the Grant administration that shaped the series for a generation. Chapter 4 depicts the production and operation of the Contemporaneous FRUS series during its “golden age” from the 1870s to 1906. Chapter 5 traces the reasons behind the growing FRUS lag in the first decades of the 20th century and reflects upon the lost promise of the 19th century transparency regime.

Part II follows the evolving negotiation of “responsible historical transparency” after the FRUS series acquired its permanent lag from currency in the early 20th century. Chapter 6 covers the formalization of FRUS editorial guidelines and the professionalization of the compiling staff within the Department. This chapter also describes the growing concern about the possible risks of historical transparency among foreign governments and U.S. diplomats, culminating with Franklin Roosevelt’s intervention to veto publication of some volumes during World War II. Chapter 7 recounts controversies surrounding FRUS during the first decade of the Cold War as Congress and the Department of State tried and failed to revive a more contemporaneous mission for the series. The furor over the release of the Yalta papers in 1955 exposed the risks of politicizing FRUS, empowered guardians of security in the U.S. Government, and spurred the Department to invite the academic community to expand its role in assessing the integrity of the FRUS series. Chapter 8 follows the series through two decades of incremental change and relative stability. It also portrays the development, implementation, and consequences of a major acceleration initiative in the 1970s. Chapter 9 illustrates the erosion of transparency of the early 1980s amidst resurgent international tensions, bureaucratic reform, and altered declassification policies. Chapter 10 relates the events leading to the resignation of the chair of FRUS’s scholarly advisory committee and the production of a Foreign Relations volume on Iran that lacked documentation of a significant (and widely-known) covert operation, which sparked a major crisis for the series in 1990. Chapter 11 illustrates the debate surrounding the 1991 legislation that provided a statutory mandate for the Foreign Relations series. Chapter 12 follows the implementation of this law during the subsequent decade and sketches the resulting framework for the current production of the series.

As this work shows, officials throughout the U.S. Government engaged in repeated negotiations over the course of more than two centuries to determine the proper balance between public accountability and the requirements of security. The official publication of documents revealing how U.S. foreign policy is determined and implemented raises questions of fundamental importance to the exercise of democracy. How much do the people need to know, and when? What information must be protected and who should decide which documents to release? What criteria should be employed to determine which records to withhold? And how do the ways Americans address these dilemmas affect foreign perceptions about the United States? Such questions have generated considerable controversy since the dawn of the republic, and, since 1861, that debate has repeatedly raged around, within, and behind the pages of the Foreign Relations of the United States series.

The history of these controversies illuminates the broader evolution of open government in the United States. As political scientist Stephen Skowronek has observed, “state building is most basically an exercise in reconstructing an already established organization of state power. Success hinges on recasting official power relationships within governmental institutions and on altering ongoing relations between state and society.” He concluded that “states change (or fail to change) through political struggles rooted in and mediated by preestablished institutional arrangements.”2 In tracing the evolution of the Foreign Relations series, this book shows how policymakers translated abstract values like “security” and “legitimacy” into concrete practice as they developed institutions to select, clear (or declassify), and evaluate the government’s most important foreign policy records. Over time, this expanding array of FRUS stakeholders inside and outside of the U.S. Government accumulated substantive and procedural knowledge that policymakers employed in their struggles to balance the government’s pursuit of security with its commitment to openness. The outcomes of FRUS debates, ranging from clearances for individual documents to the purpose, size, and scope of the entire series, reflected the relative power, influence, and autonomy of the various FRUS stakeholders. Struggles to define the “soul” of the Foreign Relations of the United States series occurred precisely because of the important issues at stake. Deciding the extent of the “people’s right to know” has fueled lively debate for over two centuries, as this history demonstrates.