RankActive Blog

Duplicate Content: how to find and fix

8 July 2016 George Svash Leave a comment SEO topics

Despite that Google continues to insist there is no penalty for duplicate content, everyone should understand it like this: “yes, your website won’t lose its positions, it just won’t get them”. Which means duplication remains to be a problem, anyway.

Duplicate Content how to find and fix

Duplicate content it is a presence of identical or similar blocks of content within or across domains. It can cause a lot of troubles for any website, starting from absence of the ranking’s growth and ending with the site’s de-indexing even. Although, according to Google, there is no such a thing as penalty for duplication, in the situation when the Search Engine suspects the manipulating of search results through duplicates use, it acts quite predictably and makes certain website to lose its positions, in the best case.

The easiest way to avoid any issues because of duplication is to prevent its appearance, of course. Thankfully, there are straight tips right from Google itself on how to do that.

Avert duplication using the following tips:

1Reduce pages’ similarity. A lot of similar content on numerous site pages is a common problem of travel websites or the food recipes ones, for example. One of Google suggestions is to consolidate similar pages into one, which is, frequently, not an option at all. The most optimal solution is to reduce similarity through increasing a uniqueness of pages content. Alternatively, you can add interesting information on each of the similar pages telling some unique facts about one of the subjects that are present in text.

2Figure out a CMS (Content Management System) you use. Unfortunately, it is a common practice to trust CMSs more than it is necessary. You should always check the display of data you’ve entered. The same content may be shown in multiple formats. A website blog is the best example because its entry is duplicated on a home page of a blog and in an archive page pretty often. So, keep an eye on that.

3Boilerplate. It is a repeated non-content across web pages. For example, website navigation (home, about us) or some special areas (blogroll, navbar). Opinions on boilerplate use differ. Google insists it is a very bad practice but, also, there are a lot of statements that often Google just ignores such things.

4Syndicating. When syndicate your content on other sites you should take into account that Google will pick up a version it considers the most appropriate to display to user. And it is not necessarily be your version. So, make sure to request each site on which your content is syndicated a link back to your original article or ask them to use the noindex meta tag.

5Use 301 redirects if you have restructured your website.

To follow the given hints is a great thing, of course. But if you’ve just got on a board and have no clue how to find out if your website has duplicate content which, as a result, may affect your site’s ranking, for a start, you need to discover how to detect this possible trouble.

Kinds of duplication

If you are not really familiar with duplication issues it should be said that it is not just about content itself, by the way. So, since to check your website for duplication presence you will use one or another tool, would it be free or a paid one, it won’t be extra to make sure it is able to provide you with truly comprehensive information on that point. There are a few pages elements of a site that can be duplicated:

  1. Content. It is the first thing that occurs to one’s mind when hearing about duplication. The similar or identical blocks of a text present on different pages of a website.
  2. Meta duplication. Strange but frequently made kind of mistake. Right here we are talking about two forms of meta duplication:
    • The first one it is a doubling of any meta tag content on a different site pages.
      For example, if you have the same meta description on two or more pages.
    • The second possibility it is a duplication of meta tags themselves. For example, if you have two meta description tags on a single page.

Search manually

Before start to explore the Internet looking for some automated tool you can try to check own website for duplication by yourself. Here are four ways to conduct such a check:

  • intitle:
    This search operator helps you to find duplicate title tags. All you need is the search box. Paste there intitle:”The title that you are interested in” . In just a second you will get the results. Some of them will seem quite reasonable and others may be nothing but total scraping. The last ones should be those you have to be concerned of.
  • inurl:
    The operator search for strings of text in indexed URLs. It is used in the same way as the previous search operator is. Like this, inurl:webpage name with keywords.

    How to choose keywords for SEO

    As usual, automated scraper sites copy each element of a domain, including the URL structure. And because of such behaviour you can easily detect this scrapped content.

  • Webmaster Tools Alerts
    Fairly controversial option, I should say. But, nevertheless, can be helpful sometimes. If you set up a verified Webmaster Tools account Google could notify you via internal message that a duplicate content issue has been found.
  • “Quotes”
    It is quite possible that you have already known about this method of duplication search. But if not, I can assure you that it is the most widely used and simple one. You just copy a part of text you are interested in and paste it in the search box. The thing is that text should be surrounded with quotes. It is done for Google to display the results which exactly match your “query” excluding the variations.


Pity that the results which are found manually are too sketchy and time consuming, and the check of a website on all forms of duplication is too important. Thus, users are forced to look for some automated tool which is able to provide permanent and comprehensive website audit. A couple of words on why an automated tool is more preferable in comparison with manual search. A website check should be conducted on consistent basis, means, daily. And it is not a secret that any website has a tendency to become bigger because pages quantity of which it consists grows all the time. Eventually, the possibility to manually control duplicate issues appearance comes to zero point, really.

That is where an automation is the only way out. Found a free tool on this purpose? Great! Got a full website on-page check in addition? Even better. The main thing is to make sure you’ll be notified immediately if the issue is detected to be able to fix it on time.

Getting rid from duplicates

So, the full website audit is done and duplicate issues are revealed. What’s next? Let’s fix this.

Methods to fight duplication:

1Redirect. Set up 301 redirect from “duplicate” page to the original one. It is not only creates a stronger relevancy and popularity signal for all of those pages, but also allows them not to compete with one another any longer. It happens because they are combined into a single page through the used redirection.

2Canonical. Although, the 301 redirect use is more preferable because of the faster action, the rel=”canonical” tag is also commonly applied thing. The operating principle is practically the same as in the case with 301 redirect use. When a search engine finds this link element on a page it transfers most of the link value gathered by that page to the original one which you’ve pointed in a href section.

3Noindex, follow. The meta robots tag with values “noindex, follow” makes search engine bots crawl the links on the specified page, but keep them from including them in their index. Thus, since unwanted pages are not indexed, they are not a problem anymore.

In conclusion

No matter what Google says, it can and will punish a website for duplicate content if it considers that you are trying to manipulate rankings and deceive Internet users. To minimize such a possibility always pay close attention to your website content. Do as much as possible to not let any doublings appear, regularly check your site on duplicate issues (better automatically) and fix them if they are detected without delay.

Embed This Image On Your Site (copy code below):


Like this article? There’s more where that came from.

mautic is open source marketing automation