User-agent: * Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Sitemap: https://www.vecerni-praha.cz/wp-sitemap.xml # Certain social media sites are whitelisted to allow crawlers to access page markup when links to /images are shared. User-agent: Twitterbot Allow: /images User-agent: facebookexternalhit Allow: /images Crawl-Delay: 5 Request-rate: 18/1m 2100-0529 Request-rate: 6/1m 0530-2059