Robots.txt Checker

Validate your robots.txt and see what search engines can crawl