Recently, I spoke at SES San Jose on “building a search engine friendly website” with emphasis focused on the incorporation and embedding of multimedia. Before narrowing my speech down to multimedia I had researched some common questions beginner SEOs or Web developers diving into SEO have when learning to build search engine friendly.
Below are a few common questions I came across and are addressed specifically:
What are common errors that beginners make when designing their "search friendly" site?
1) Using images, Flash, JavaScript, iFrames for to display navigation or textual content. Bots can't crawl content or links embedded with these technologies though Flash capabilities have been significantly improved lately for Google.
2) Separating words in images/page filenames with underscores instead of dashes. With underscores, bots view your words in a filename as all ran together. With dashes, bots can separate the words in your file name as separate words. It's important to note that Google is close to separating words with underscores but isn't quite there yet.
For Example:
"search_engine_optimization.jpg" is viewed by bots as "searchengineoptimization" as the keyphrase in the image.
As opposed to:
"search-engine-optimization.jpg" is viewed by bots as "search engine optimization" as the keyphrase in the image.
3) Duplicating Titles and/or Meta-Data across the whole Website. Each page should have unique meta tags, and most importantly, unique keyword-rich title tags for each individual page. Duplicate titles and meta-data can cause pages to be put in Google's supplemental index which will kill your search traffic.
How does using frames impact how search engine finds your site?
It used to be that search engine bots couldn't crawl frames and therefore couldn't find your content. A lot of people still believe that and while certain search engine bots might not be able to the major ones can.
The issue isn't a matter of crawlability but a matter of linking. You cannot link to individual pages because they are locked in a frameset. People cannot bookmark pages correctly and cannot link to pages correctly so this creates a huge accessibility issue. People not being able to correctly link to your site drastically effects search rankings. If there's any sort of algorithmic quality score assigned to a site by search engines then it's certainly a ding in the score for the accessibility issues.
What are good ways or methods for a beginner to test how search friendly their site is?
1) Simulate the hierarchical order of how search engine bots are viewing your site with Lynx Viewer. There's a good Mozilla add-on called Yellow Pipe Lynx Viewer that shows you how bots are accessing your content and which order. Ideally, you want to make sure bots are seeing all your content, accessing the most important links first, etc.
2) Web Developer Toolbar allows you to disable JavaScript, Cookies, and CSS to make sure your content is still accessible to bots and people who have disabled or don't have those capabilities.
3) Register and verify your site with Google's Webmaster Tools. Google will provide feedback on any crawling errors it may be having with your site.
What are some of the key issues that affect search engine robots?
Duplicate Content – Bots don't like to have multiple versions of content in their index. If you have multiple versions of a page bots will pick one to index and ignore the rest. Google will put duplicate versions in its supplemental index where they aren't accessible under regular search queries.
The real problem with duplicate content is that your links are diluted across multiple versions. It is wise to consolidate your duplicate pages by 301 redirecting duplicates to just one centralized page.
Some fear a penalty with duplicate content but only in rare cases will you be penalized for it. If you site is heavily duplicating content from other domain names without any of your own unique content then that may be cause for a penalty.
Dynamic URLs – Search engine bots have gotten a lot better at crawling parameters in dynamic URLs now but if you have an excessive amount of parameters most bots will have trouble crawling them. You also lose how on the value of adding keywords to your URLs. It is recommended that you use mod_rewrite to make your URLs easily accessible to spiders and have keyword rich value.
HTTP Headers – It's important to know if bots are accessing your content in the correct manner. Essentially, you want to return an HTTP 200 status OK header to bots for your pages. If you are doing a redirect you want to make sure they're receiving an HTTP 301 header (or 302 in rare occasions). I recommend that people create a custom HTTP 404 header error page so bots can continue to crawl your site if they hit a dead internal link.
Server Load – The lower the file size of your page the better. The less calls to the server for external JavaScript or CSS files the better. If you site is too content heavy you can render bots from fully crawling the deeper levels of your site. Sometimes bots have an allocated spend where they'll only crawl X kilobytes of your site until they move on. You want to make sure your pages are small and fast loading so bots can crawl efficiently.
Following Links – Bots can have trouble crawling links in instances where your links are in JavaScript, Flash (even though there's been vastly improved capabilities), or Frames. Make sure your links are easily accessible to search engines.
Robot Directives – You can use Robots.txt directives or Meta-Tag directives to tell bots whether or not to crawl, index, or follow links on your pages. This comes in very handy when trying to block spiders from duplicate content.
Are there any specific development/optimization tools for beginners to make use of when optimizing their site for search?
I could go on all day. I won't detail each tool but here are a few great resources:
WebCEO
RankSense
Back Link Analyzer v2.0
WebConfs SEO Tools
SEO Chat SEO tools
SEOmoz SEO tools
SEO Oregon
Portland SEO
iWebTool
Complete List of the Best SEO Tools
SelfSEO
136 SEO Tools
We Build Pages SEO Tools
XML Sitemap Generator
SEObook Tools
Search Engine Marketing Company
Mozilla add-ons:
• Web Developer Toolbar
• Tails Export (for microformats)
• User Agent Switcher
• XHTML Mobile Profile
• Yellowpipe Lynx Viewer
• Modify Headers
• Search Status
• SEO for Firefox
• Live HTTP Headers
• Link Diagnosis
• IE View
• Greasemonkey
• DOM Inspector
• Firebug
If there are any further questions anyone has about any of the items addressed please feel free to contact me at my SEO blog or through my SEO firm.
Good tips, I didn’t know the difference between underscore and dashes. Luckily for me I’ve always used dashes.
I’ve found robots.txt to be most helpful, especially to remove duplicate content that can creep up on dynamic sites.
Excellent write up. I didn’t know that about the underscore/dashes thing. I have used the free WebCEO tools and find it very helpful.
Thanks!
Thank you, Jordan!
I’m especially happy to learn about Yellow Pipe Lynx Viewer add-on for Firefox!
A point to add about the value of having a site work well in Lynx is that the site is then accessible. Bots view sites similar to the way those who are visually-impaired view sites using Lynx. (This is imperative if your site receives any type of federal funding, e.g. education.)
Thinking this should be titled “SEO Cliff Notes” for the nice and concise overview.
Those are excellent tools to test search how search friendly your site is: Yellow Pipe Lynx Viewer, Web Developer Toolbar, Google’s Webmaster Tools are reasonably straightforward even for beginners.
Hi Jordan,
Very good SEO articles here. I really love to learn about it.
But why don’t you put axandra IBP products to your SEO tools list?
Is WebCEO better than it?
I think about dynamic URLs you may need to include the new mention from google. As Google is very comfortable crawling, indexing them and they suggest better not to rewrite unless or otherwise we are very good in it. But I personally feel that redirecting to static urls are always better as there are other search engines that do drive traffic to us other than google.
Thanks for the article Justin- did you read the google blog post about dynamic URLs?
http://googlewebmastercentral.blogspot.com/2008/09/dynamic-urls-vs-static-urls.html
what are your thoughts?
Jack
Great advice for some beginners, but with that said, you still got me on a couple pointers.
I was going to make mention of the recent Google post on dynamic URLs, but Mercy beat me to it 🙂 However, speaking further on that topic, for those linking to your site using the URL as the anchor text, the re-written URLs will be more keyword-relevant. Also, the re-written URL could appear more ‘attractive’ to potential linkers. You know, kind of like how clocks sell better if they read 10:10 in the ad (they appear “happier”…ever hear that one?) 🙂
Interesting Jordan, I always thought that underscores or dashes did not make a difference. I will have to do some testing and see if I can improve my results with dashes.
Great list Jordan. I have a number of these issues on one of my sites. Need to give it some attention. Didn’t think the underscore mattered much anymore but I prefer to change it if it’s still a factor.
Hi Jordan,
Very interesting article aimed for beginners and sometimes help even for the so called “SEO Experts” who fail to understand the basics of SEO!
Yellowpipe Lynx Viewer Tool is great; I would also recommend seo-browser.com. Very interesting tool to analyze the web as how bots crawl.
Thanks so much for this informative article.. Have stumbled…!! Very much recommended… looking forward for more such posts.
– Vinay
http://vinay.mp
I added a blog to my website and the pages on my blog always get indexed quicker then my main pages probably because search engines trust the design of blogger templates.
Jordan,
I really do not agree with your Dynamic Url assertion to use mod_rewrite. Unless you know what you are doing, it is not really worth it.
A lot of Urls actually lead to duplicated content, in which case, judicial use of Nofollow can handle a lot of the load and not overburden your Server.
In the case of Google, they have no problems with even the gangliest of Urls.
Read Dynamic URLs vs. static URLs from the Google Blog and it will lay a lot of myths to rest.
WOW Great And Big Seo Tools I Ever Sae Thanx a Lot 🙂 🙂 🙂
Dany
Jordon, found the article on sphinn. A good read indeed and thanks for the links, I will put your post as posts of the week in my blog,
Cheers
Pushkar
It is amazing to note that there are still web designers whom built websites with iframes or frames. How do you expect the robots to follow links? Within frames of another frame? Make the robots crawl with ease and you are half way to a good and friendly design.
Rif Chia
The reason webdesigners use flash is because many sites do not require seo or rankings due to the nature of their business.
Many model agencies and media companies want to look cutting edge and are advertised to be found on the web.
I get bored waiting for the flash graphic to load and often just go elsewhere.
@mujeres
Often what most people would want to do with Flash (fading banner images) can be accomplished in Javascript while staying search engine friendly.
Wonderful tips. Being new to SEO this is exactly the type of thing I need to read. I would love to see a part 2 of common questions on the blog in the near future.