If you are thinking you have developed a truly great keyword-rich-unique-content fully optimized website for the search engines and an attracting site for the visitors - that's fine, but do you know you are missing something? A robots.txt file. Did you include it? By the way do you know what's the importance of a robots.txt file? Success of big companies lies in keeping their confidential data a secret, hidden from all. They tell the world something and do something. This enables them to execu.