What is a Sitemap?
There are two different kinds of sitemaps used on websites. The one most people think of is a sitemap (note the lowercase s) page used to give users (humans) a listing of pages and their relationships. Here's an example sitemap page.
The second type, is a Sitemap (capital S this time) document. This is an XML file used specifically to give search crawlers (robots) a list of what pages are on the site, and some information about them. Typically named sitemap.xml, this document helps search engines crawl the site more intelligently. It gives them a listing of the pages you want them to crawl, their updated time, update frequency, and their level of imprtance in relation to each other.
Building a Sitemap
|<?xml version>||Yes||Declares the XML version and encoding for the file. XML 1.0 and UTF-8 should be used.|
|<urlset>||Yes||The document-level element for the Sitemap. All other elements must be wrapped in this tag.|
|<url>||Yes||The parent element for each entry.|
|<loc>||Yes||This is the URL address of the page. It must be a full path (http://www.example.com/page/) and not a relative (/page/) path and needs to include the protocol (http or https).|
|<lastmod>||No||The date the file was last modified in ISO 8601 format. Examples: YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS|
|<changefreq>||No||Defines how frequently the page is updated. Possible values are always, hourly, daily, weekly, monthly, yearly, never. These are only used for crawlers to know how frequently new content is available on the page.|
|<priority>||No||Priority is a value from 0 to 1, with 1 being the most important. These are relative values to tell crawlers which pages are most important on the site, in relation to one another. For example, you might give a "contact us" page a value of 1, since since you want users to be able to find your contact information, while another page about toe fungus might have a value of .15 since its not as important as the contact page. Setting all pages to the highest priority will not improve search ranking. Remember to include the 0 before the decimal place (use 0.65 instead of .65)!|
Let's look at an example sitemap.xml file. Remember to save this to your website's root directory.
<?xml version="1.0" encoding="utf-8"?>
http://www.example.com/ 2014-10-22 daily 1 http://www.example.com/contact/ 2014-09-30 monthly 0.8 http://www.example.com/about/ 2014-09-12 monthly 0.8 http://www.example.com/toe-fungus/ 2002-04-01 never 0.15
In this example, we've included four pages in our Sitemap. The homepage as identified as the most important page with a priority of 1, followed next by both the /contact/ and /about/ pages which both have a priority of 0.8. The least important page is the /toe-fungus/ page with a priority of 0.15. These priority rankings help search crawlers know which pages are really important. We've also got information telling the crawlers when the pages were updated, and the frequency they will be updated next.
When building a Sitemap, normally you would put all your entries in one sitemap.xml file. However, if you'd like to break them out into separate sitemaps (maybe you hit the 10mb file limit or 50,000 <url> limit) you can do so by building a Sitemap index. A Sitemap index is basically a list that references your other sitemap files.
|<sitemapindex>||Yes||Only used when building a Sitemap index. Sitemap indexes are used to reference other Sitemaps used.|
|<sitemap>||Yes||The parent element for each entry. Similar to <url> used in Sitemaps|
<?xml version="1.0" encoding="UTF-8"?>
http://www.example.com/sitemap1.xml 2014-10-13 http://www.example.com/sitemap2.xml 2014-10-22
In this example, we've got two Sitemaps that we've included in our Sitemap index. The index would be saved as /sitemap.xml, and in this case has references to the Sitemaps sitemap1.xml and sitemap2.xml. Search crawlers will hit the /sitemap.xml file and continue crawling through to the other Sitemaps.
Submit to Search Engines
Once you've got your sitemap.xml file built and uploaded to your site's root directory, you're ready to submit it to search engines. Test it in your browser first to make sure its view-able.
Let search crawlers know where it is by adding it to your robots.txt file. Search engines will see your robots.txt file when they crawl your site, and by telling them where to find the Sitemap you'll help them crawl your site better. To do this, just add the following:
Remember that the directive for the sitemap location is independent of the user-agent, so it can be placed anywhere in the robots.txt file. Make sure you include the full path including the domain. In this example, its http://www.example.com/sitemap.xml, otherwise crawlers like Google can misunderstand the path to your sitemap.xml document.
Updating your robots.txt file will help crawlers next time they crawl your site. Next lets talk about how to submit your Sitemap directly to Google, Bing, and Yahoo.
Google handles Sitemap submissions through the Google Webmaster Tools. Follow these instructions to tell Google about your Sitemap:
- Log in to Google Webmaster Tools
- Open the "Crawl" menu on the left side and choose "Sitemaps"
- On the top right click the "Add/Test Sitemap" button
- Enter the path to your sitemap.xml file
Remember, it will take Google some time to look at and digest your Sitemap, so it could be a while before you see any results. For more information, visit the Google support article.
Bing also allows submissions through its Bing Webmaster Tools. If you haven't created an account yet, the first step of the registration process includes adding a sitemap.
If you've already got a Bing Webmaster Tools account, you can view instructions for submitting a sitemap over at Bing.
In either case, keep in mind it will take some time for Bing to digest your Sitemap.
The Yahoo! search results are powered by Bing, so as long as you've submitted your sitemap.xml to Bing, you're all set!
Creating a Sitemap is a pretty quick process that can really help search crawlers understand what pages are on your site, and which ones are the important ones. Especially if your site is relatively new, building a sitemap.xml file can dramatically improve your search ranking. Even if your site is well established, adding a Sitemap gives you the ability to improve your search results by helping crawlers know which pages really matter.