# robots.txt for https://bhsg.com/ sitemap: https://bhsg.com/sitemap.xml # live - don't allow web crawlers to index cpresources/ or vendor/ User-agent: * Disallow: /cpresources/ Disallow: /vendor/ Disallow: /.env Disallow: /cache/ User-agent: Bytespider Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Timpibot Disallow: /