Search engines will look for a file on your website called robots.txt to determine whether and how to crawl and index your site. We make sure one exists and that it has the correct information listed.
Additionally, this file contains a link pointing to the automatic sitemap.xml file we also include on every website.
If you're curious what this file looks like, you can see the one we use on our own website here.