SEO Tips – How to Deal with Bots and Why You Should

Author: Mike | Date: August 10, 2017 | Categories:

When it comes to SEO tips, bots are an often overlooked subject. However, they can cause huge fluctuation in reporting leading to less than optimal marketing strategies with resulting decreases in rankings.

What is a bot? Bots are programs that speed up processes, allowing their programmer to complete many tasks in a much shorter amount of time than doing it themselves. There are two kinds of bots. Good bots and bad bots. Good bots are of no concern for your website because they carry out totally harmless tasks such as indexing. Bad bots can hurt your website in a variety of ways. Here’s what they do and what you can do to mitigate their influence and damage.

What type of bot can harm your website?

Theft Bots

As you may have surmised from their title, theft bots are designed to steal information or content. This can include pricing, text, images, and more. Theft bots also gather email contacts for ad spamming. Valuable information or content can be sold to rival websites or posted directly onto their own site. Competitors may seek out stolen data from others, or utilize theft bots of their own in attempt to gain the advantage in their market.

Companies that are guilty of using theft bots to steal competitor information can be sued. Such was the court case between the Metropolitan Regional Information Systems Inc and Maryland based, a real estate listing site, over the use of stolen copyrighted content and unauthorized access to a real estate listing database filled with private information. This was accomplished through the use of theft bots,or what is being called an AgentMatch tool. MRIS has been granted an injunction, which will likely result in a guilty verdict for the owners of

Theft bots also have an adverse SEO effect. When your content gets copied and published on other websites, Google recognizes it as duplicate content, which causes your site to drop down in Google rankings. Bad bots could even cause your site to get blacklisted.

Fraud Bots

Fraud bots are designed to click on paid ads and other clickable items for the purpose of gaining as many hits as possible. They can also carry out tasks such as posting spam to blogs and other web forms. They can create fraudulent user accounts on various types of networks or databases and record sensitive information about customers, clients, associates, or other members of your website. Fraud bots are a danger to any files or data that you wouldn’t want to fall into the wrong hands.

AdWeek estimates that $6 billion are stolen from advertisers annually through the use of fraud bots. And according to Wenda Millard, president of Medialink, approximately one in four PPC hits is fraudulent. These costs add up considerably over time and damage businesses that use PPC advertising. If you use PPC for your business, this includes you and your company as well.

What is a Bot Capable of Other Than Theft and Fraud?

Aside from all of the problems above, bots can also infect user’s devices who are visiting the site, or gain access to their accounts when they log in. They can dramatically impede a site’s performance and make it frustratingly slow. Bad bots cause a reduction in bandwidth, and result in KPIs and analytics being much harder to decipher. They can even control devices remotely and use them as part of a DDoS cyber-attack.

Bad bots have become more and more prevalent over the past few years. Their numbers are increasing at an alarming rate. A Bad Bot Landscape Report recently indicated that the majority are of American origin.

How Common are Bad Bots?

One bad bot named “Pushdo” was found in all major internet providers in the US, Canada, the EU, and more, totaling over 4 million IP addresses affected worldwide. According to, Advanced Persistent Bots (APBs) make up 75% of the bad bots in use today. These advanced bots are capable of loading JavaScript, retaining cookies, and processing external resources. They can automatically randomize their own IP addresses, and conceal themselves by switching user agents.

Bad bots alone were responsible for roughly 20% of all web traffic in 2016, with that number expected to increase. Up to 97% of sites containing proprietary content are falling victim to data scraping from bad bots. Any site with original content or price listings, sign up forms, contact forms, payment processing, and discussion boards are at the greatest risk of bad bot attacks.

SEO Tips on How to Handle Bots

Detecting and Getting Rid of Bad Bots

Bad bots can be identified in several different ways. Each of the following may be a sign of an unwanted bot:

  • They originate from geographic locations suspiciously unrelated to your product or purchase
  • A Google search of their URL brings up reports of malicious activity
  • They spent 0 minutes on your site
  • They only visit one page on your site
  • They don’t seem like the kind of URL that would link to your site

  • Removing these harmful guests on your own isn’t always easy, but it can be done. You can use your .htaccess file to block these URLs from being able to access your site. Doing this is risky though, because if you put in the wrong character, even just one, it can shut down your entire site. Alternatively, you can use Google Analytics filters to check and see if you have removed the bot before making any permanent changes. There is a helpful setting called “Bot Filtering” that you can also use to help.

    There is another, more effective way of preventing and protecting your site from the intrusion of bad bots. With help from the professionals at Course Vector, you can keep your site secure and clear of bad bots without having to tamper with your .htaccess file or fiddle with trial and error adjustment in Google Analytics. Get in touch with Course Vector now to find out how we can comprehensively safeguard your website from all types of bad bots.

    For additional SEO tips check out these other articles from CourseVector:

    On Page SEO

    How an HTML Redirect Can Save Your Website Ranking

    Contact Us to Get Started

    "Your passport to all things web."

    1 Abbey Lane
    Camp Hill, PA 17011
    Phone: (717) 516-6955

    CourseVector Terms & Conditions
    Design and hosting by CourseVector. All rights reserved. Copyright 2021.
    Sitemap | Locations

    To contact us after hours please use the panic button.
    Fees may be incurred depending on reason for support.