Building/Submitting a site map

My site has been submitted and is being 'crawled' by some search engines.  Google, however, asks for a site map if it can be provided.  I would like to give them one, but am not sure exactly what they are looking for, how to build it, or what all it needs to include.  Can someone explain this to me a little better?  Is it even worth my time?

Travis
Logged


Yes, it is absolutely worth your time. 

Take a look at http://www.sitemaps.org for more information on what an XML sitemap is and http://www.sitemaps.org/protocol.php for the specific format.  Google also has several help pages at https://www.google.com/webmasters/tools/

Google will also accept just a list of urls like...
Code:
http://www.example.com/
http://www.example.com/page1.html
http://www.example.com/page2.html

The Yahoo and Microsoft SEs will also use the XML format, although I'm not sure about the text format.
Logged


Thanks, Ryan.  I will check out the links.  I was able to register on Google for free, can I do the same on Yahoo and MSN?  Everything I found on their sites makes it look like you have to pay.  I already come up on searches there, though.  What is the advantage to submitting my site to all these search engines?

Travis
Logged


You don't actually have to submit.  As long as your site is linked to from somewhere they'll find you. 

What you can do is add a line to your robots.txt file.  Add the line sitemap: /whatever.xml and the 3 main SEs will start to pull your sitemap. 

Yahoo's submission is at http://siteexplorer.search.yahoo.com.
Logged


And thank you very much, again.

Travis
Logged


OK, next question: what should my robots file look like?  What is it, what does it do, what should it have in it, and how do I format it?
Logged


A robots.txt file was originally used to "suggest" to crawlers what sections of your site you didn't want indexed.  There's no spec that says they must follow your requests.  But if you have pages you don't want indexed - login pages, private stuff, that sort of thing - you can include them.  But you probably want your entire site indexed so don't worry about the Disallow settings.

You can get by with a single line
Code:
Sitemap: /sitemap.xml
file.
Logged