As you read in our last post, the way a search engine sees your web pages is very different than the way a human sees them. Today, we are going to talk about five ways to help the search engine spiders crawl your website better. The reason this is important is because the spiders don’t want to have to put in a lot of work or effort to find the pages of your website. If you don’t make it easy for them, they will often times leave and not see all of your pages. So let’s take a few minutes and discuss how you can help the spiders find all of your pages, which we call, improved crawlabilty.
Readable Site Navigation
One of the easiest ways to improve the crawlability of your website is by having a full crawlable and indexable site navigation. Building your navigation in flash or javascript limits the ability of a spider to read and move from page to page. Again, when this happens, they might abandon your website all together not even seeing your other pages.
Submit XML Sitemaps
Another easy way to improve crawlablity is to build and submit an XML sitemap… or multiple sitemaps. An XML sitemap is one of the very first things a spider will look at when they come to your site and if updated, will show them every page, image, and video found on your website. Do it… it is easy! Here’s what an XML sitemap should look like:
Build An HTML Sitemap
Just like XML sitemaps help crawlablity, HTML sitemaps do the same thing. An HTML sitemap is a static web page that also lists out every page on your website. When a spider finds this page, they will also find all the other pages of your website. See how awesome that is? Here’s what an HTML sitemap should look like:
Leverage Internal Links
An internal link is a link found in the body of the content on a web page that links to another web page? These are helpful for two reasons. First, it helps the spiders find other pages on your site (seeing a trend yet:)) and second, if you link with the keywords that the page is about, it helps tell the spider what they should find on the page they are going to. Here’s how internal links look:
Publish Lots of Content & Publish Often
Lastly, by publishing lots of awesome content on your site and doing is frequently, you will show the spiders that you have something interesting for them and they will keep coming back for more. If you stop publishing, chances are crawling will slow and if you completely abandon your site, crawling will stop all together. A great way to publish lots of content and do it often is by setting up a blog. Blogs are great for sharing ideas, concepts, and thought leadership.
Well, those are the five best ways to improve your website crawlability. Have others you like? Share them in the comments section below.
