Site Speed and SEO - Does a Slow Site Get Penalized or a Fast Site Rewarded?

Folks often ask me whether site load time affects their SEO, and the short answers is: Yes!

The importance of having a site that loads fast (or at least adequately fast) can be imputed from the fact that search engines want to point users towards reputable and high-quality sites that match their search queries. A slow site that hangs forever is not a good experience for the user, and by extension, for the search engine that led that user there.

The fact that Google's webmaster tools reports on crawl speed (time spent downloading a page), shows that they're watching (aren't they always?). Also, I've noticed a direct correlation between the faster load times translating to more pages crawled per day (which should mean better indexing of your content).

More recently though, Google's top SEO oracle (technically the head of it's web spam team), Matt Cutts, laid it out for us (covered here by SEO Moz): Although slow load times will NOT adversely affect your rankings, fast load times may have a positive effect. So there you have it, straight from the horse's mouth. So, keep those sites scaling and those pages loading as fast as you can!

Number of Pages Indexed: How to Find out How Many Pages You Have Indexed

Ensuring the search engines are properly indexing pages from your website is critical for SEO - after all, if the search engines aren't indexing your pages, those pages will never come up for search results!

So, how do you find out how many pages from your site are getting indexed? The easiest way is to search for the following on Google:
"site:url.com" (no spaces)

SEO Book notes that searching for site:url.com and site:www.url.com can lead to slightly different results. Also, you can use Google Webmaster Tools to get some additional information on the pages Google is crawling on your site and how that changes over time. We'll go over Webmaster Tools in a future post.

Robots.txt and SEO - How to Use Robots.txt

Tonight I ran into my same friend who had previously asked me about hiding text using the same text and background colors, and this time he had another very pertinent question: How is robots.txt to be used for SEO?

Honestly, I have always thought of robots.txt as a way to tell search engines which pages on a website not to index, but hadn't thought about it more broadly than that. So, I figured I'd do some quick web research and let you all know what I learned about using robots.txt for SEO.

First of all, the robots.txt file is used to provide cooperating web robots with instructions on how to crawl the site. If the file is not present, the robot will assume no specific instructions are being given. Also, you need to include a robots.txt file for each subdomain (i.e., subdomain.domain.com/robots.txt).

The main instruction is the "Disallow: /directory/page" command which tells the robot not to crawl those pages. You can also specify that instruction to all robots (with a *) or mention specific robots (or user-agents) (check out this example from CNN).

Another common instruction is to point the robot in the direction of the sitemap file. This is done through the "sitemap: URL location of sitemap" instruction. This is helpful in making sure the robot finds all the pages on your site (more on sitemaps in a later post), but again, not necessary to include.

Breadcrumbing for SEO - Getting your Pages Indexed Properly Via On-Page Links

Sitemaps are one way of getting search engines to understand your site and all the pages you have, but they're not always totally effective, especially as your site grows (in number of pages) and those sitemap files start getting larger, and large.

One way to "show" search engines all the pages on your site as well as to tell them what those pages are about (playing into their relevance for targeted keywords), is by using a practice commonly referred to as breadcrumbing.

Breadcrumbing refers to leading the search engine throughout your site by making it follow specific paths you set up through links between pages. Ensuring that every page on your site is linked to from somewhere is a key component to making sure search engines know that page exists and can index it properly. Not only that, but as you lay breadcrumbs for the search engine to follow, you can use the anchor text in the links (the text that is underlined and links to the other page) to tell the search engine what the destination page is about (giving it context/relevancy).

Breadcrumbing could be done fairly 'dumbly' by simply including a couple of links from every page to the next couple pieces of content in some sort of sequential order. Perhaps by showing a "Next Article" or "Next Products" link(s) at the bottom of the current page. You can also leverage breadcrumbing to improve user navigability of your site. If the links are done in a logical manner that helps the user discover related content, that helps both SEO and navigation. For example "Related Articles" or "Similar Products." To ensure you're linking to all the pages (rather than your algorithm inadvertently leaving out certain pages from the link structure), you may want to inter-mix some kind of relevant linking with some sequential linking. Perhaps a "Next Article" link coupled with a "Related Articles" link.