# Information for robots how to crawl the web User-Agent: * Disallow: # For development and test server un-comment following line in order to stop crawling test sites # Disallow: / User-agent: Seekport Crawler Disallow: / User-agent: SemrushBot Disallow: / User-Agent: MJ12bot Disallow: / User-Agent: Yandex Disallow: / # For productive Buxus server: Disallow: /buxus/admin/ Disallow: /buxus/application/ Disallow: /buxus/dhtml/ #Disallow: /buxus/docs/ Disallow: /buxus/help/ #Disallow: /buxus/images/ Disallow: /buxus/includes/ Disallow: /buxus/lib/ Disallow: /buxus/project_includes/ Disallow: /buxus/project_scripts/ Disallow: /buxus/properties/ Disallow: /buxus/scripts/ Disallow: /buxus/system/ Disallow: /buxus/tmp/ Disallow: /ajax-objednavka-data.html?page_id=393789874 # disallow indexing PDFs Disallow: *&pdf=1 ## TODO odkomentovat #sitemap: http://www.pttours.cz/sitemap.xml