robots.txt Generator

Create a search engine friendly robots.txt file. Choose a preset or build your own crawler rules.