The minimum files and structure needed to start a new web design project.
Adding A Bit of Content
We now complete the barebones set up with adding content to the text files.
robots.txt - link
A robots.txt file is not an essential element for your web site but can be useful for asking visiting web robots to not crawl or index parts of your site.
A robots.txt file is a text file that stops web crawler software, such as Googlebot, from crawling certain pages of your site.
This is useful if you have files or folders you would rather not appear in search engine listings. It is included here with no restrictions for robots visiting our site.
Note: You need a robots.txt file only if your site includes content that you don't want Google or other search engines to index. To let Google index your entire site, don't make a robots.txt file (not even an empty one).
# www.robotstxt.org/ # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449 User-agent: *
humans.txt - link
As with the robots.txt file, a humans.txt file is not essential to your web site. What it does do is allow an acknoledgement to be made to all the people who have contributed as authors to the site:
... because it's something simple and fast to create. Because it's not intrusive with the code. More often than not, the owners of the site don't like the authors signing it; they claim that doing so may make the site less efficient. By adding a txt file, you can prove your authorship (not your property) in an external, fast, easy and accessible way.
Visit the humans.txt website above to read more about this initiative.
# humanstxt.org/ # The humans responsible & technology colophon # TEAM - Your details here # TECHNOLOGY COLOPHON HTML5, CSS3