Introducing Blueclaw’s New SEO for Magento Extension & Why you need it.
In the world of eCommerce platforms, Magento stands out as a tantalising offer. Renowned names such as TimeOut and Jack Wolfskin have built their sites with Magento and its highly scalable, open-source nature and dedicated community of extension builders and trouble-shooters have made it an attractive proposition to those looking to take their eCommerce site to the next level. So for those businesses that were looking for a stable, expansive platform for their first steps into the digital marketplace or take an existing site to new heights, the prospect of relaunching on Magento is an exciting one. But once the excitement of launch day has been and gone, when analysing the new site’s performance sometimes it can seem like Magento wasn’t the panacea it was built up to be. If you find that Magento’s out of the box search engine optimisation isn’t working for you, it might need an extra helping hand.
A common feature of Magento builds, the layered navigation, is an excellent tool for users. Where a traditional navigation system may leave your visitors faced with a catalogue of thousands of products, the system of attribute filters allows a user to narrow down their field of view until they find precisely what they were looking for. Great for your customers once they’re on the site, then, but what if your traffic has seen a downturn since launching or relaunching your site on Magento? All the usability in the world isn’t worth much if you’re not attracting visitors to your site and the very same feature that seems like such an advantage might actually be at the root of your issues.
Layered Navigation Filters
To help users navigate through to precisely the shape, size and colour of product they’re looking for, Magento’s layered navigation system creates a vast interlinking mesh of internal links – one for every possible permutation of each category and subcategory on your site. Even on a relatively small site, we’ve seen examples where the total number of pages crawled or indexed has skyrocketed.
That must be great for the long tail, right? Well, no, because this will also mean that Google attempts to plot the flow of your link juice through these thousands of pages, essentially diluting it across the entire site. Google’s Crawl Budget for your site (which is dictated to some extent by a site’s PageRank) will be exhausted long before it fights its way through the complicated net of pages, many of which feature duplicate content, page titles and meta descriptions.
This flood of potentially duplicate, low worth pages combined with the massive dilution of PageRank can really hamper your site’s ability to rank for both long tail searches and short tail, broad match terms. Google Webmaster Tools is an excellent weapon when diagnosing this kind of issue – paying attention to the HTML Improvements section (ideal for detecting how much duplicate content exists on your site), Index Status (for checking how many URLs Google is crawling and indexing across your site) and the Internal Links section (useful for diagnosing where this fracturing of PageRank is occurring) will put you on the right track. An upwardly spiralling number of crawled and indexed pages, an unusually high number of internal links pointing to category or subcategory pages and numerous duplicate meta descriptions and title tags are major symptoms of a layered navigation issue.
So how can I fix this?
If you’ve got links on your site that you want to keep away from search engines, then tagging these links as Rel=”nofollow” seems like a logical approach. The problem with this approach is that Google can still find and index URLs that are set to nofollow, but no PageRank will be passed – resulting in an indexed page that is unlikely to rank and simply wastes even more crawl budget. A further issue is that one reason nofollow exists is to tag untrusted content, and stating to Google that you don’t trust content on your own website is obviously a bad idea.
How about instituting a meta NOINDEX tag on these pages, then? A NOINDEXed page can still be crawled, but won’t be shown in search results. Link juice can still be leaked into these pages, diluting the juice that would otherwise be helping your important pages to rank. On top of that, the pages won’t be indexed, so again this juice is lost. Adding a rule to block parameters in the site’s Robots.txt file will also result in the same issue, where link juice is leeched by the blocked pages.
One tool that should be useful in this instance is the “URL Parameters” section in Google Webmaster Tools. Here you can set up rules that inform Google how you’d like URLs featuring a particular parameter string to be treated.
Unfortunately, while this is a great idea in concept, this rules don’t tend to be particularly well adhered to. By all means, add these to your Webmaster Tools account but don’t expect them to do the job on their own.
Other methods might include using a rel=”canonical” tag to point all the filtered pages back to the main category, but this is arguably going against the intended use of the canonical tag. A canonical should never point to notably different content; while it may be acceptable on some parameter URLs that merely display content in a different order (such as a price or size ‘sort’), filters specifically offer up different content through narrowing the field of visible products.
Canonicals of this sort are likely to be ignored by search engines, the pages will be indexed regardless and your level of duplicate content will shoot up once again.
Thankfully, there does exist a solution which suffers from none of these drawbacks.
The Blueclaw Solution
Blueclaw’s newly released SEO for Magento extension is a fully-featured Search Engine Optimisation module for solving this and other ranking and indexation issues that can cause issues with a vanilla Magento install. Blueclaw SEO for Magento provides an admin interface for the fine control of many under-the-hood aspects of the platform. The extension allows a store owner to hide aspects of the layered navigation from any specified Useragent, or to apply the same rules to any requests coming from a user without cookies.
This allows a store to display a full range of attribute filters to users, retaining their ability to narrow down and filter products, while at the same time hide these aspects from, for example, the GoogleBot. Where rel=”nofollow”, robots.txt and NOINDEX solutions turn these filter attributes into dead ends, where your PageRank simply ceases to take effect, Blueclaw’s SEO for Magento gives you the ability to completely hide the troublesome aspects of the layered navigation from a search engine – the spider won’t crawl the unwanted URLs, as to their eyes the links are effectively no longer present on the site.
With these links removed from the equation, Google’s crawl budget is free to be expended on the pages that you want your link juice to flow to and, ultimately, to rank. Blueclaw’s SEO for Magento also has features to further optimise the level of faceted navigation on your site, with options to remove all but the top level of the navigation from any user identified as a spider.
These features mirror those seen on high profile eCommerce sites such as Amazon.co.uk, which uses a similar approach to reduce the number of internal links they present to search engines in their layered navigation without noticeably changing the on-page content or attempting to serve a noticeably different page to a potential customer versus a UserAgent identified as a search engine.
Blueclaw’s SEO for Magento offers this through the option to remove all but the top level of the navigation when dealing with requests from a user identified as a spider, as well as the functionality to hide the ‘Clear All’ or ‘Clear Filters’ link. Both of these measures are excellent methods to reduce the number of internal links (as seen in Webmaster Tools), which can severely hamper a search engine’s ability to spider your site.
Should you wish to retain some filters, Blueclaw’s SEO for Magento also allows you to hide attributes on a case by case basis, and further optimise these attributes with clean URLs, automatically updated page titles (it also allows you to specify exactly how these should appear in the URL and title slug and whether they should include page numbers, again reducing the chance that any filtered pages will be considered duplicate content) and headers as well as default templates for meta description and on-page category description text.
Blueclaw’s SEO for Magento solves the issues inherent in Magento’s layered navigation by simplifying it for search engines, without losing any of the usability that visitors to your site will enjoy. Aside from the improved handling of layered navigation, the extension also allows for full control and editing of robots.txt and Robots META information, canonical tags for product and categories as well as improved attribute handling. For a more detailed overview of our modules’ features, look out for an extensive series of blog posts that will be released on the Blueclaw Blog over the next few weeks.
To read more about Blueclaw’s SEO for Magento or to purchase the extension, visit the extension website here, where you can also view tutorial videos, read the user guide or explore a live demo install.