What is Google URL Blacklist and How to Avoid It
It’s more or less a given that you will get a great deal of your website visits from search engine users. But have you ever stopped to think about what it is that search engines serve up to their users, exactly? Can a search engine blacklist (i. e. refuse to list a link to) your website, and for what reason?
Google is the world’s most popular search engine, and in this article we will be talking about Google’s blacklist rules and how to avoid falling foul of them. We will discuss what Google’s URL blacklist actually is, how it affects Google’s behaviour with regards to your website, and how to troubleshoot potential problems. A thing to bear in mind is that blacklisting is different from Google deindexing your website and happens for different reasons. The majority of this article deals with what Google blacklisting is and how to avoid it. If your website has somehow got on the blacklist already, the section you need is at the end. Here‘s what we will be discussing:
Contrary to what you may believe, Google does not search the internet the moment a user enters a query. Instead, it searches its own database, called an index, for suitable results. Google’s blacklist (or blocklist) is also a kind of a database of websites – the websites that Google, but also other search engines or antivirus companies consider unsafe for the general user.
Another thing you might want to know is that Google does not compile its indices manually. It uses bots, or crawlers, special programs which explore websites and add them to the index – and the blacklist, too. In fact, Google automatically blacklists up to 10.000 websites a day! So, getting blacklisted can happen by mistake, through no fault or omission of yours, but you can still take steps to prevent it from happening or remove your website from the blacklist if it gets there.
If your website gets blacklisted by Google, it will not appear in Google search results. That means that the website will lose all of its Google traffic, and in turn views, visits, and revenue which stem from it.
Users who attempt to reach the website by entering the website’s URL into their browser’s address field will still be able to reach it. They will, however, get a very dramatic-looking warning screen before they so much as see the website. Even the users who specifically wish to access the website without going through a search engine will likely be dissuaded.
As we said before, you may see a warning about the website being unsafe if you attempt to access it. So, the first thing you should do is to attempt to access your website and see whether a warning appears.
Other than that, there are several online tools which can tell you whether your site has been blacklisted.
Formerly known as Google Webmaster Tools, Google SearchConsole is an easy way to check whether your website is still googlable. Once you create a Google SearchConsole account and verify your website is your property with it, you will be able to find your security notifications in the left-hand side menu, under Security & Manual Actions.
If you are using Google Analytics or another kind of visitor tracker, you may see a sharp drop in visitor traffic. Remember, once your if your website gets blacklisted, its pages will not appear in Google searches. So, if you notice a sudden drop in visits, you can take that as an indication that your website was blacklisted.
Finally, there are online tools which may tell you whether your website is blacklisted and by whom, as different search engines may have different standards and practices for blacklisting. If you suspect your site might be blacklisted, we suggest you try WhatIsMyIPAddress.com’s Blacklist Check, Spamhaus, UltraTools, or similar. You could also try Google’s own Safe Browsing tool, which will tell you whether malicious content – a common reason for blacklisting – had been found on a website.
Websites often get blacklisted when a Google bot detects suspicious code coming from it. This may mean that your website is hacked and is spreading malicious code to your visitors. This could also mean that one or more of your ads link to unsafe content, such as websites distributing malicious code.
For a WordPress website, a common way in for a hacker to get in is a plugin. Think about it: you install a plugin, you are adding software to your website. And plugins can be developed by anyone, even people acting in bad faith. You may have been very scrupulous about which plugins you use on your website, too, but that’s no guarantee. A developer may abandon a plugin (i. e. stop updating it, and, by extension, its security), leaving it vulnerable to malicious hackers.
But why would anyone go to the trouble of installing malicious code on your website? The answer to this is fairly obvious: there is money in it.
When one thinks of website hacking, what typically comes to mind is a hacker gleefully taking down a website’s content and/or replacing it with their own. These hackers may be (relatively) harmless trolls, upset by your website content for whatever reason, or simply acting at random, in which case the hacker attack is similar to their taking down a poster of yours or gluing one of their own over it. This type of attack is called defacement.
However, they may use your website for attacks on your users and their valuable data. Oftentimes a hacker will create a website which looks very similar to the website hacked, right down to the contact forms, and use it to collect data from the website users. This type of online scam is called phishing, and it can be used to obtain bank data or personal logging information which can cause a great amount of damage to the scammed person. A hacker might also trick website users into downloading and installing malware (harmful software) on their computers. This type of software is called a Trojan horse, as the user believes it to be beneficial, recommended or required by the website.
Not all blacklisting comes forth as a result of malicious hackers, though: you could inadvertently be overdoing your own SEO.
If you are familiar with the basics of SEO, you know that links to a website and keywords in the content and headings improve a website‘s ranking in Google search results. You may be tempted to set up multiple websites linking to each other in order to improve their ranking, or to load up your content with popular keywords. This is known as SEO spam, and search engine operators are onto it. And they don‘t like it. SEO spam is one of the reasons you could get blacklisted even if it‘s inadvertent.
There are a couple of things you could do to avoid being blacklisted in the first place.
Firstly, protect your login form. You can make logging in safer by adding a CAPTCHA or by using two-factor authentication for each login. You should also avoid weak passwords, such as words which can be found in dictionaries, or passwords which relate to anything from your personal life, such as dates, or names. These can be easily cracked or even guessed by somebody with access to your personal data. Always use a strong password, and a different password for all accounts. If you have trouble remembering your password or passwords, we recommend you try a password manager.
Secondly, update everything. As we‘ve said before, out of date or obsolete plugins are a common way in for hackers. To avoid your website being hacked, make sure all your plugins have been tested for the version of WordPress you are using, and that you are also using the latest version of each plugin.
Thirdly, keep an eye on user-uploaded content. Your website may get blacklisted for hosting spam or malicious links in user comments, for instance. Additionally, if you allow users to upload files, you may also make it possible for ill-intentioned hackers to influence your website.
Finally, avoid SEO spamming. This means that you should avoid setting up websites and pages specifically for the purposes of linking to a page you wish to promote, and you should absolutely avoid keyword stuffing and using hidden keywords (outside of the normally visible part of your website). Make sure your links are genuine and your keywords organic.
Best Themes for Marketing
View Collection
Okay, but suppose you somehow ended up on Google‘s blacklist. What then?
That depends on why your website had been blacklisted in the first place. To find out why you were blacklisted from Google, you need to access your Google Search Console account. You will find the reasons for your website‘s blacklisting under Security & Manual Actions/Security issues.
You will find that the reason is either injected SQL code, malware, viruses, or SEO spam. This dictates how you should go about fixing things.
If SEO spam is your problem, you will get a clear indication on how to remove the spammy links.
If it is any kind of malicious code, there is a couple of things you should do.
Firstly, check for any unfamiliar users or users with compromised or corrupted accounts. You can find them in the Users section of your website, which you can reach from the left-hand side menu on your WordPress dashboard. You‘d also do well to have all the familiar users change their passwords and submit them to two-factor authentication.
Secondly, you should disable all obsolete plugins and find and install replacements if you really need them.
Next up, you may have gotten notifications about infected databases. You will need to clean up all the tables in your infected databases manually. Be sure to back up your database manually before making any changes, just in case.
Finally, just to be safe, you ought to check again for any unfamiliar changes to your website, and remove them.
Once you have done all within your power to remedy the issues, return once more to your Google Search Console‘s Security issues tab. In the Security issues sections, click I have fixed these issues and then Request a Review.
You will then be prompted to list the steps you have taken in order to resolve the issues with your website, and then go to Manual Actions in order to get a human to review your website again. If you have complied with Google‘s guidelines, your website will be reindexed.
In Conclusion
We hope this little primer on the Google URL Blacklist was of use to you. Whether you feel it‘s right or wrong, Google, as well as other search engines, gets to pretty much write its own rules on how it works, what are its safety standards, and what kind of SEO it allows. As the adage goes, the game may be rigged (against new players, when it comes to search engines), but it‘s the only game in town. It‘s up to you to find a way to the top of the results page – but only by following the rules.
