Chapter 1: The Parameters of Openness and Executive Discretion, 1790–1860

The Foreign Relations of the United States series, which began publication in 1861,1 drew upon longstanding precedents that established the prerogatives of the executive and legislative branches. Dating from the dawn of the republic, and drawing on earlier English parliamentary practice, Congress exerted its right to inquire into the basis upon which foreign policy decisions were made. The executive branch, in the person of the president, reserved the power to withhold or restrict release of information, with proper justification.2 Much about this dynamic has changed since the 1790s, but the fundamental balancing act between the public’s right to know and the government’s responsibility to protect remains at the center of an ongoing and lively exchange.

During the early republic period, the legislative and executive branches jostled to establish functional intergovernmental communications procedures. The Congress quickly established a habit of calling for reports or executive branch records on a variety of domestic matters. Over time, it became customary for most executive branch agencies to submit annual reports to Congress, although the Department of State never did so. As early as 1790, Congress acceded to the principle that the President might withhold particulars about certain expenditures related to foreign policy. That same year, President George Washington transmitted documents relating to Indian affairs, but insisted that Congress keep the information confidential. Congress also stipulated that certain foreign policy documents, for example, the provisions of treaties ratified by the Senate, should be published at the public expense.

In 1791–1792, an issue encompassing foreign policy, military affairs, and congressional oversight required the two branches to consider key operational and constitutional principles. On November 4, 1791, United States military forces under Major General Arthur St. Clair suffered a crushing defeat on the banks of the Wabash River (in present-day Ohio) at the hands of a coalition of Indian tribes.3 As soon as President Washington received the first report, he voluntarily informed Congress.4 The House of Representatives formed a committee to investigate the causes of the debacle, and, on March 30, 1792, requested that Secretary of War Henry Knox release documents relevant to the inquest. Given the fundamental nature of the questions involved, and aware that his administration’s decisions would set precedents for relations between the branches of government, Washington convened his Cabinet to consider how to reply. The Cabinet examined in detail English Parliamentary precedents to guide their views about prerogatives attached to the executive and legislative functions.5 They decided that the Legislative branch did possess the right to request documents from the Executive. As to who should respond, the Cabinet determined that only the President, as head of the Executive branch and therefore responsible for all executive departments’ operations and records, could reply to congressional requests. Concerning what documents the executive branch might submit, they agreed that “the public good”—as defined by the President—must determine the extent of disclosure. In principle, the President should divulge as much material as possible. Nevertheless, if, in the President’s judgment, release of certain documents might harm the important national interests, those records could be withheld.6

Congress did not challenge Washington’s general approach to questions of principle, and the administration subsequently presented a comprehensive response. On April 4, 1792, the House committee redirected their query to the President, recasting the request in terms that recognized the “public good” criteria by asking Washington to “cause the proper officers to lay before this House such papers of a public nature, in the Executive Department, that may be necessary to the investigation.”7 The President and his Cabinet averred that in this instance, copies of all the relevant documents could be released. Washington even offered to dispatch a clerk to display the original documents so that Representatives could fully satisfy themselves as to the veracity of the records.8

By 1800, this procedure of congressional requests for information with allowances made for reservations had become established practice. The House or Senate asked for documents relating to specific foreign policy issues when they deemed it necessary. Those queries usually deferred to the executive branch by including language to the effect that exceptions could be made if the president judged it necessary to withhold information “in the public interest,” or in consideration of the “public good,” or when “public safety” required. Although much altered in form, this basic approach to sharing information about foreign policy issues continues today.9

Most importantly, the President’s officials possessed the capacity to control the flow of information. Because executive branch agents created, received, retained, and reviewed the documents, little recourse existed for questioning their determinations about what was in the public interest to divulge. Congress could press for the release of more documents or demand explanations as to why records remained sealed, but legislators acceded to the principle that redacting and withholding constituted appropriate executive functions. Neither branch wished to appeal to the judiciary, nor did the courts want to become embroiled in matters likely to involve ephemeral political considerations. Thus, the executive retained the initiative in defining what constituted “the public interest” or “national security” for the purposes of determining responsible U.S. Government transparency.

The earliest evidence of the criteria employed to justify excisions from released material arose during the 1793–1794 Citizen Genet affair.10 After multiple rounds of congressional requests for information and partial executive branch releases, including some transmitted confidentially,11 in January 1794, Secretary of State Edmund Randolph outlined the type of information typically not released. “The parts to be withheld will probably be of these denominations: (1) what related to Mr. G[ene]t; (2) some harsh expressions on the conduct of the rulers in France, which, if returned to that country, might expose [Gouvernor Morris, U.S. representative in Paris] to danger; (3) the authors of some interesting information, who, if known, would be infallibly denounced.”12 Randolph’s criteria incorporated several of the excision categories used today:

  • gossip, accusations, or other information not germane to the issues at hand, especially regarding diplomats representing their country’s interests in the United States;
  • the type of frank assessments diplomats typically convey, but that can prove problematic if revealed to officials of the host nation. Ministers expected their communications with the Secretary of State to remain confidential, at least until the most acute sensitivities receded;
  • “human intelligence” sources—those likely to discontinue cooperation if their communications were revealed.

When considered in conjunction with the “public interest” withholding, the categories of today’s classified information criteria appear similar: material that might compromise national security, intelligence sources and methods, the protection of information provided in confidence by a foreign government, assessments that might damage current relations or compromise ongoing negotiations, and personal information. Sensitive information of these types now comprises the principal categories exempt from release.13

It is important to note when considering the parameters of openness through the early part of the twentieth century that the universe of documents subject to release was much smaller than today. Many types of official records now considered essential to reconstructing the policy making process did not then exist. The large bureaucracies of the modern era create multiple types of records that reveal internal decisionmaking processes such as memoranda of conversation, position papers, decision documents, cross-departmental coordination efforts, interagency task force records, detailed accounts of international negotiations, and the like. Until the early 20th century, the federal government was quite small; the Department of State, for example, totaled seven domestic employees in 1790.14 If a Department head or the President wished to convene a meeting to determine policy, the principals could easily fit into one room. Little need existed to write down the course of the deliberations, since all key players could be present. Additionally, extant records that recounted certain intra-executive branch functions—for example, the deliberations of Cabinet meetings—would have been considered off limits out of respect for the separation of powers. The House and Senate sometimes met in executive session and treated the records of those deliberations as secret; they could hardly ask the executive to surrender similar documents. Finally, conceptions about what constituted a government record were much more circumscribed. Officials routinely retained their “personal” records when leaving government service, which included correspondence considered “private,” diaries, notebooks, and other documents that may reveal much about the internal workings of government. Indeed, the documentary basis of Part One of this book rests largely on the “papers” of leading figures, few of which remained in the possession of the federal government. Until the third decade of the 20th century, Department of State staff assigned to select and transmit documentation to Congress had no authority to consult these records. Consequently, the bulk of the extant record consisted of official correspondence—communications from other governments, messages from U.S. diplomatic officials overseas, instructions written to them, and, sometimes, reports produced for the Secretary of State or the President based largely on those documents. Disclosure of “the record,” then, constituted the release of materials that indicated the inputs to policy as well as the implementation of policy, but which only obliquely revealed the weighing of factors involved in making decisions.

The openness expectations of the era were also conditioned by an essentially unregulated confidentiality regime. Unlike today, no laws prohibited executive branch officials from sharing information with legislators, the press, private citizens, or even foreigners. Leaks of sensitive information occurred frequently. Administration officials sometimes released documents selectively to bolster their position or undermine rivals. The executive branch learned that any information sent to Congress, even confidentially, was liable to become public. In hopes of establishing the trust necessary to receive fuller documentation, both Houses of Congress created standing rules requiring their members to maintain the confidentiality of records transmitted by the executive branch. Nevertheless, both the House and Senate asserted the authority to remove the injunction of secrecy from documents they received by majority vote if legislators deemed it in the public interest to do so. The executive acquiesced, acceding to the principle that the people possessed a sovereign right to make informed judgments about the conduct of their public servants.15

By the early 19th century, a clear set of reciprocal expectations had developed between the executive and legislative branches. Most importantly, a general agreement existed that a functional republic required well-informed public debate; both branches assumed that the President should share documents regarding foreign affairs with representatives, either voluntarily or when asked, and that as much of that information as possible would be made available to the American people. Every year, the executive branch transmitted to Congress foreign policy records, sometimes in very significant amounts, and often on short notice. The nation’s elected representatives uniformly acknowledged the value of openness in government operations.

Both branches (particularly the executive), however, tempered the value ascribed to transparency. All agreed that the indiscriminate release of information could damage the national interest; the executive branch applied the sorts of excisions identified by Secretary Randolph when officials deemed it necessary. In a few high-profile instances, a case can be made that as early as the 1790s Presidents withheld very important information that might have made a difference in congressional support for administration policies that led to hostilities or resulted in territorial expansion.16 Although those exceptions constituted a very small percentage of the overall information exchange, they nevertheless represented an important practice that would occasionally recur after the institution of regularized, contemporary documentary publication in 1861.

Yet even in cases of documents delivered to Congress on a confidential basis, of excisions from released records, and of the withholding of crucial correspondence, Americans assumed as a general rule that those materials would eventually become part of the public record. Beginning in the early 19th century, Congress authorized considerable funding for an extensive program of publishing documents. The U.S. Serial Set, which began publication in 1817, comprised the most comprehensive collection of foreign affairs documentation during the antebellum period.17 The Serial Set included hundreds of thousands of pages covering an incredibly broad range of topics: the slave trade, compensation for consular officials, geographical surveys, land claims, piracy, commercial trade, treaties, fisheries, canals and railroads, boundary disputes, tariffs, and a host of other issues great and small. The releases range from single documents to hundreds of pages.

Congress also authorized additional publications produced by private individuals. For example, in 1829 Joseph Gales and William Seaton proposed that they should be allowed to publish government documents; Congress agreed and in subsequent years they produced American State Papers, volumes that cover foreign relations topics from 1789 to 1828.18 Congress authorized several other publications separately, including 12 volumes containing correspondence of the American Revolution edited by Jared Sparks.19 The legislative branch demonstrated an impressive commitment to governmental openness by funding dozens of projects totaling hundreds of volumes, involving thousands of documents incorporating hundreds of thousands of pages, at the cost of millions.20

Lacking an overarching plan, manifestations of this evolving practice of responsible release emerged sporadically and in piecemeal fashion. Speaking broadly about all government publications (not only those concerning foreign relations), a committee of historians assembled in 1908 reported that “the period from 1829 to 1861 may fairly be declared to have been the most active in historical publication. . . . All this constituted a creditable achievement for a young nation not yet rich. But it is distinctly miscellaneous.”21 “Miscellaneous” describes well the foreign affairs documentation published in the Serial Set. The publications were “so scattered as to be hard to use” and “few libraries” collected the full run of documents. Additionally, “they embrace isolated, selected papers, such as it suited the President or Secretary of State to send to Congress.” The Serial Set and privately produced volumes were not the result of an overarching plan.22 Although those early efforts do not conform to present-day standards of documentary editing and comprehensive coverage, they do reflect the value ascribed to making foreign policy documentation available to Congress and the public.23

This extensive publishing program required federal bureaucracies to incorporate new functions and develop working-level policies that embodied the ethos of openness. Congressional staff and Executive branch employees developed processes for discovering, selecting, organizing, copying, editing, and indexing records. The legislative and executive branches created both a nascent bureaucracy and procedures for what would today be called “declassification” of once-secret documents and for control of restricted information while it was being prepared for publication.24 However rudimentary when compared with modern standards, by the early 1830s, all the key elements of the transparency-secrecy continuum were in existence, as well as mechanisms to resolve the differences arising therefrom. Those procedures were grounded in a profound tradition—the presumption that elected officials had an obligation to inform the people about what the government had done in their name.

President Abraham Lincoln’s covering note accompanying the inaugural FRUS volume included clear evidence that the publication rested on the many well-established practices discussed in this chapter. On December 3, 1861, Lincoln sent his first annual message to Congress with the foreign policy documentation attached. He noted the records briefly at the beginning of his message by stating, “The correspondence itself, with the usual reservations, is herewith submitted.”25 Lincoln’s proviso “with the usual reservations” indicated the evolutionary character of this documentary release. Although we cannot know with certainty what “reservations” Lincoln had in mind, his use of “usual” implied a reliance on established procedures, criteria, and transparency expectations for the release of foreign policy documents.

  1. FRUS has been published regularly and continuously since 1861. Chapter 3 discusses why the Department did not produce a volume covering diplomatic events for 1869. Chapter 4 explains the late appearance of the 1881 and 1898–1900 volumes. Chapter 5 outlines proximate factors contributing to the lapse of annual publication of volumes beginning in 1906. Volume releases further receded from the 19th century standard of currency thereafter; the tension between timeliness and comprehensive coverage comprises a central theme of Part II.
  2. For secondary treatments of the early republic era, see Daniel N. Hoffman, Governmental Secrecy and the Founding Fathers (Westport, CT: Greenwood Press, 1981); Abraham D. Sofaer, “Executive Power and the Control of Information: Practice Under the Framers,” Duke Law Journal 77, no. 1 (March 1977): pp. 1–57; Sofaer, “Executive Privilege: An Historical Note,” Columbia Law Review 75, no. 7, (November 1975): pp. 1318–1321; Louis Fisher, “Invoking Executive Privilege: Navigating Ticklish Political Waters,” William and Mary Bill of Rights Journal 8, no. 3 (1999–2000): pp. 583–629; Mark J. Rozell, Executive Privilege: Presidential Power, Secrecy, and Accountability, 2nd ed. (Lawrence: University Press of Kansas, 2002), pp. 28–37; Lawrence R. Houston, “Executive Privilege in the Field of Intelligence,” paper dated September 22, 1993, Central Intelligence Agency, Center for the Study of Intelligence website, https://www.cia.gov/library/center-for-the-study-of-intelligence/kent-csi/vol2no4/html/v02i4a07p_0001.htm; Brian Balogh, A Government Out of Sight: The Mystery of National Authority in Nineteenth-Century America (Cambridge: Cambridge University Press, 2009), p. 165.
  3. John F. Winkler, Wabash 1791: St. Clair’s Defeat (Oxford: Osprey, 2011).
  4. Washington to the Senate and House, December 12, 1791, The Papers of George Washington, Presidential Series, Vol. 9 (Charlottesville, Virginia: University Press of Virginia, 2000), p. 274. Available at the NARA, Founders Online website, http://founders.archives.gov/documents/Washington/05-09-02-0166.
  5. Knox to Washington, March 30, 1792, The Papers of George Washington, Presidential Series, Vol. 10 (Charlottesville, Virginia: University Press of Virginia, 2002), p. 168 and accompanying notes on pages 168–169. Available at the NARA, Founders Online website, http://founders.archives.gov/documents/Washington/05-10-02-0104. In 1792 the cabinet consisted of Alexander Hamilton (Treasury), Thomas Jefferson (State), Henry Knox (War), and Attorney General Edmund Randolph. For this crucial precedential case, see Jefferson’s March 31 and April 2, 1792 accounts of the Cabinet’s deliberations and implementation of their decisions in Franklin V. Sawvel, ed., The Complete Anas of Thomas Jefferson (New York: Round Table Press, 1903), pp. 70–71. Jefferson’s notes cite “Chandler’s Debates,” referring to The history and proceedings of the House of Commons from the Restoration to the present time, Supplemental Pieces (short title), Vol. 13 (London, 1742), HathiTrust website, http://babel.hathitrust.org/cgi/pt?id=hvd.32044106508997. The specific references address whether the House of Commons possessed the authority to initiate an inquest concerning military affairs (including clandestine spy operations) and how the King’s ministers should respond. The debates took place during the latter period of Robert Walpole’s premiership (late 1730s–early 1740s). The disputants in the Walpole era referenced earlier 18th and 17th century cases that came before Parliament, evidence of the American Founding Fathers’ adherence to English Common Law practices of deference to tradition and precedent.
  6. Hoffman, Governmental Secrecy and the Founding Fathers, pp. 69–83; note 5 above.
  7. Annals of Congress, House of Representatives, 2nd Cong., 1st Sess., April 4, 1792, pp. 535–536 (emphasis added).
  8. Knox delivered the papers to the House on April 9, 1792. See note 5 above and The Papers of George Washington, Presidential Series, Vol. 10, notes on p. 169.
  9. For the legislative branch undulation between acceptance of and challenges to assertions of executive privilege during the 1790s, see note 2 above and especially Hoffman, Governmental Secrecy and the Founding Fathers, pp. 88–118, 124–138, 143–177, 184–196. All parties recognized from the beginning that various actors might derive very different calculations about what constituted “the public good.” Some members of Congress rejected the legislature’s right to demand documents from the executive because they believed it implied impeachment of the President, or because they feared investigations would be instigated for political purposes. See, for example, statement of William Smith in Annals of Congress, 2nd Cong., 1st Sess., House, March 27, 1792, p. 491. Secretary of State Jefferson worried that his rival, Treasury Secretary Alexander Hamilton, contemplated using the procedure to promote bureaucratic independence from the Chief Executive: when confronted with a congressional directive Hamilton did not wish to obey, he would invoke the shield of executive privilege; if, instead, Hamilton did not wish to follow a presidential instruction, he could release material to Congress that promoted Hamilton’s policy preferences. See note 5 above.
  10. Harry Ammon, The Genet Mission (New York: Norton, 1973); Eugene R. Sheridan, “The Recall of Edmond Charles Genet: A Study in Transatlantic Politics and Diplomacy,” Diplomatic History, 18, no. 4 (Fall 1994): pp. 463–488.
  11. Documents detailing the principal submissions to Congress and key and cabinet discussions about release criteria include: Message of the President of the United States to Congress, Relative to France and Great-Britain. Delivered December 5, 1793 (Philadelphia: Childs and Swaine, 1793), Internet Archive website http://archive.org/stream/amessagepreside00conggoog#page/n4/mode/2up (note on p. iii that Washington did not apprise legislators of omissions, merely stating that he forwarded “certain correspondences” pertaining to relations with France and Great Britain); Annals of Congress, 3rd Cong., 1st Sess., Senate, January 17, 1794, p. 34; John C. Hamilton, ed., The Works of Alexander Hamilton; Comprising his Correspondence, and his Political and Official Writings, Exclusive of the Federalist, Civil and Military, Vol. IV, 1851 (New York: John F. Trow), pp. 494–495; American State Papers: Foreign Relations, Vol. 1 (Gales and Seaton, 1833), p. 329. The most important document Washington withheld was Jefferson’s memorandum of conversation with Edmond Charles Genet, July 10, 1793 printed in The Papers of Thomas Jefferson, Vol. 26 (Princeton, N.J.: Princeton University Press, 1995), pp. 463–467, available at the NARA, Founders Online website, http://founders.archives.gov/documents/Washington/05-13-02-0137-0002.
  12. Randolph to Washington, January 26, 1794, The Papers of George Washington, Presidential Series, Vol. 15 (Charlottesville, Virginia: University of Virginia Press, 2009), p. 130, available at the Library of Congress, American Memory Collection website, http://memory.loc.gov/mss/mgw/mgw4/105/0100/0105.jpg.
  13. For careful assessments of the questions surrounding Randolph’s criteria for withholding and material excised from Morris’s communications before transmittal to the Senate, see Hoffman, Governmental Secrecy and the Founding Fathers, pp. 104–116.
  14. Department of State, Office of the Historian website http://history.state.gov/departmenthistory/buildings/section13.
  15. Hoffman, Governmental Secrecy and the Founding Fathers, pp. 238–244.
  16. Key cases include Washington’s handling of records relating to the controversial Jay Treaty, John Adams’s decisions regarding information releases during the “XYZ Affair,” Jefferson’s Louisiana Purchase negotiations, Madison’s and Monroe’s expeditions into the Floridas, and Madison’s withholding of certain correspondence at the time of the 1812 war vote. For a brief overview, see Sofaer, “Executive Power and the Control of Information,” pp. 8–45.
  17. Anne Morris Boyd and Rae Elizabeth Rips, United States Government Publications, 3d ed., revised (New York: H. W. Wilson, 1949), pp. 26–28. For a comprehensive index of materials published between 1828 and 1861, see Adelaide Rosalia Hasse, Index to United States Documents Relating to Foreign Affairs, 1828–1861, 3 vols. (Washington: Carnegie Institution of Washington, 1914–1921). See also Checklist of United States Public Documents, 1789–1909, 3rd ed., revised and enlarged, Vol. I, Lists of Congressional and Departmental Publications (Washington: Government Printing Office, 1911), pp. 892–895.
  18. Martin P. Claussen, “Revisiting America’s State Papers, 1789–1861,” American Archivist 36, no. 4 (October 1973): pp. 523–536; Clarence E. Carter, “United States and Documentary Historical Publication,” Mississippi Valley Historical Review 25, no. 1 (June 1938): p. 7; Richard W. Leopold, “The Foreign Relations Series, A Centennial Estimate,” Mississippi Valley Historical Review 49, no. 4 (March 1963): p. 596.
  19. Leopold, “The Foreign Relations Series,” p. 596, and for other examples of edited volumes, see notes 2 and 6; Message from the President of the United States, Transmitting a Report by the Committee on Department methods on the Documentary Historical Publications of the United States Government, Together with a Draft of a Proposed Bill Providing for the Creation of a Permanent Commission on National Historical Publications (Washington: Government Printing Office, 1909) (hereafter Report by the Committee on Department Methods), pp. 31–33; Laurence F. Schmeckebier, Government Publications and Their Use (Washington: Brookings Institution, 1936), pp. 317–321.
  20. Note also that, in 1800, the legislative branch created the Library of Congress, originally intended primarily as a depository for congressional documents. In 1810, Congress passed an Archives Act that required a systematic approach to the organization and preservation of government records. George H. Callcott, “Antiquarianism and Documents in the Age of Literary History,” American Archivist 21, no. 1 (January 1958), pp. 18–22; Leopold, “The Foreign Relations Series,” pp. 596–597.
  21. Report by the Committee on Department Methods, p. 10.
  22. Report by the Committee on Department Methods, p. 31.
  23. John Bassett Moore, “The Diplomatic Correspondence of the American Revolution,” Political Science Quarterly 8, no. 1 (March 1893), pp. 33–47; Worthington Chauncey Ford, “The Editorial Function in United States History,” American Historical Review 23, no. 2 (January 1918), pp. 273–286; Lester J. Cappon, “American Historical Editors before Jared Sparks: ‘they will plant a forest . . . ’” William and Mary Quarterly, 3d ser., 30 (July 1973), pp. 375–400.
  24. See “Introductory Notice,” American State Papers, Foreign Relations, Vol. I (Washington: Gales and Seaton, 1833), pp. vii–xi at the Library of Congress, American Memory Collection website http://memory.loc.gov/cgi-bin/ampage?collId=llsp&fileName=001/llsp001.db&recNum=6.
  25. Message of the President of the United States to the Two Houses of Congress at the Commencement of the Second Session of the Thirty-Seventh Congress (Washington: Government Printing Office, 1861) (hereafter Message, 1861), p. 4.