User-agent: *
Disallow: /*pg_
Disallow: /search
Crawl-delay: 20

Sitemap: https://history.state.gov/sitemap.xml