An XML sitemap file is a fairly recent standard adopting by the major search engines that can help with indexing. Yahoo recently announced that the big 3 search engines, Google, Yahoo and MSN, have agreed on another standard that relates to how these sitemap feeds can be submitted. Previously they could only be submitted through some site management tools maintained by the search engines, but now your robots.txt file can be used to automate the process a little more. By adding a single line to your robots.txt file you can identify the URL location of your sitemap file for indexing. This is good news for sites with large amounts of pages as it makes it that much easier for the search engines to fully index a site.
Tags: search, engine, indexing, sitemap, xml