Though the robots.txt file was invented to tell serps what web pages to not crawl, the robots.txt file may also be used to place search engines to the XML sitemap. This is supported by Google, Bing, Yahoo and Ask. Track down the robots.txt file: The robots.txt file need to be https://rankingseotoolz.com/robots-txt-generator