Sign up for our newsletter and be the first to get all the latest Qode news and updates.
BACK TO TOP

How to Noindex a Page in WordPress

How to Hide a WordPress Page or Post from Search Engines

When you run a website, you want it to rank as high as possible in search engine results pages (a.k.a. SERPs). To achieve that goal, you’ll need to employ all the optimization techniques in your arsenal and do your best to come up with innovative ways to convince search engines your content is relevant and valuable.

The process of getting your website to the top of SERPs starts with the indexation of its individual pages. Letting bots access your content means your pages are ready to be shared with the world.

Of course, you can always lock your pages with a password if you don’t want users to be able to access them. However, that won’t hide those pages from search engines.

But truth be told, there are plenty of times when it’s better to noindex a page in WordPress. And to hide it from search engines, you need to instruct them not to index it. This is done using the noindex directive, which lets bots crawl a page, but prevents them from ranking it.

In this article, we’ll learn the nuts and bolts of how to noindex a page in WordPress and we’ll discover:

Now let’s get started.

6 Reasons to Noindex a Page in WordPress

Reasons to Hide a WordPress Page or Post from Search Engines

While this may seem strange, sometimes it’s perfectly reasonable to noindex a page in WordPress. In fact, we’ll give you 6 reasons why that may be the right to do.

1. Privacy

You don’t want to share pages that contain sensitive information, confidential details, or users’ personal and contact info with the public.

2. Duplicate Content

In case you publish your content in several formats (for instance, Adobe PDF or printer-friendly versions), we advise you to show just one version to search engines in order to avoid duplicate content issues. Furthermore, if you run an eCommerce website, you should always hide the pages that have similar or identical product descriptions (in case only the product sizes or colors vary, for instance).

3. Keyword Cannibalization Issues

If you target one keyword across multiple pages or posts on your website, the page you consider to be the most relevant for a specific query may not be the one that’ll rank the highest in search engines. To avoid this issue, enable indexing only for the pages that are the most important and relevant for specific query terms.

4. Little or No User Value

Thank You pages, admin login pages, and similar types of pages are not that important for users, so it’s not that necessary to have them ranked in search engine results. They hold no real value for your website visitors, and you don’t need them receiving organic traffic.

5. Work in Progress

In case your pages and posts are under development, you should hide them from search bots until you perfect them to your liking.

6. Third-Party Content

If your website features ads and similar content that is generated by third-party sources, you should hide it from search engines. That type of content usually has nothing to do with the primary purpose of your website.

Alright, so we know in which instances we should hide a WordPress page or post from search engines. Now it’s time we learned how to do that.

How to Noindex a Page in WordPress Using Yoast SEO

Hiding your content from search engines using Yoast SEO is the easiest option to follow, regardless of the level of your WordPress proficiency.

To use this method, you need to install and configure the Yoast plugin. Then, from your WordPress dashboard access the post or a page that you’d like to hide and look for the Yoast SEO metabox (it should be right under the page editor). Now, click on Advanced.

Configure the Yoast plugin

Several options will appear, but we’re interested in the top two. The first one allows you to prevent search engines from showing your page in search results, and the second one stops search engines from following links on your page.

Stops search engines from following links on your page

Adding the nofollow tag to your page is usually not necessary. You can use it in case there are paid links on your page or user-submitted links in the comments section. The former are sponsored content and are against Google’s Webmasters Guidelines. As for the latter, the problem is that you can never know how trustworthy the users’ links are, which is why it might be for the best to prevent bots from following them.

Other than that, it’s enough to set your pages to noindex, to keep them off the search index.

When you’re done making changes in Yoast, don’t forget to save them.

Best WordPress Themes for Marketing
Foton WordPress Theme
Foton

Software and App Landing Page Theme

Innovio WordPress Theme
Innovio

Multipurpose Landing Page Theme

Optimize Plugin
Optimize

SEO, Digital Marketing & Social Media Theme

MR SEO WordPress Theme
Mr. SEO

SEO, Marketing Agency and Social Media Theme

How to Noindex a Page in WordPress Using the robots.txt File

This method is not recommended for WordPress beginners since it requires you to alter the robots.txt file.

To access the file, connect to your server via FTP or use the File Manager in your cPanel.

Another way of accessing the robots.txt file is by using the Yoast SEO plugin. Go to SEO > Tools and select File Editor.

Accessing the robots.txt file

If you have the robots.txt file, it’ll be displayed at the top. In case you don’t, you can make one by yourself. Google has a great article on how to create a robots.txt file. To hide a page from search engines via the robots.txt file, add the following lines of code to the file:

User-agent: *
Disallow: /the-URL-of-your-page/

The first line of code refers to specific bots and search engines you may wish to target. To include all search engines, use an asterisk, just like we did. Insert the URL of the page you’d like to hide in the second line of code, but make sure to exclude your website’s domain name.

For instance, if the URL to your page is “http://aweekendinthecity.com/2020/music/indie-rock” you would put:

User-agent: *
Disallow: /2020/music/indie-rock/

When you’re done, you could test your robots.txt file, just to make sure you’ve set everything up correctly. And if you’ve only just created the file, make sure you upload it to the root directory of your website.

The Risks of Using the robots.txt File to Hide Content from Search Engines

Aside from not being quite beginner-friendly, this method comes with a few risks, which is why we’re not its biggest fans.

The robots.txt file is a configuration file that tells search engine crawlers if they should access your pages and posts or not. But not all search engines follow the requests of this file. It’s possible your page will still appear in search results, just without any meta description. Furthermore, if some other website points to yours with descriptive text, your page will be indexed (even if you have just audio and/or video content on it). Plus, malicious bots and content scrapers usually ignore the file.

The robots.txt file is also publicly accessible. This means anyone can see it and easily discover the pages you wish to hide.

How to Noindex a Page in WordPress Using the ‘noindex’ Directive

Another way of hiding your page from search engines is by adding a noindex meta tag in the HTML file of your page or by including a noindex header in HTTP response headers. For this to work, Google insists that you don’t block your page by a robots.txt file or else search engine bots won’t even see the noindex directive.

How to Add the ‘noindex’ Directive as a Meta Tag

To instruct all search engines not to index your WordPress page, add the noindex meta tag to the HTML <head> section of a given page:

<meta name="robots" content="noindex" />

To prevent just the Googlebot from crawling your page, use this tag:

<meta name="googlebot" content="noindex" />

N.B. Even after you modify thesection and add the noindex meta tag to it, some search engines may still crawl your page.

How to Add the ‘noindex’ Directive in the HTTP Response Header

To instruct search engines not to index your page, you can also use the X-Robots-Tag and add the following HTTP response to the head of the header.php file:

header("X-Robots-Tag: noindex", true);

If you’d like to stop bots from following the links on your page as well, go with this HTTP response:

header("X-Robots-Tag: noindex, nofollow", true);

All the directives you can use in a robots meta tag can also be applied to an X-Robots-Tag. But the latter is a lot more flexible. In fact, X-Robots-Tag controls indexing of both a page as a whole and specific elements on a page, while a robot tag lets you manage indexation on a page level only. For instance, if you wish to stop search engine crawlers from indexing just an image or a video file, you would have to use the X-Robots-Tag.

Which Method Should You Opt For?

After all is said and done, we’d recommend using the Yoast SEO plugin to successfully noindex a page in WordPress. It’s simple and effective. No advanced level of expertise is required to use it. Plus, the changes you make with Yoast won’t affect the root directory of your site.

However, if you wish, experiment with other options, too. In the end, all that matters is that you succeed in preventing search engines from accessing and indexing your WordPress page or post using whatever method you feel most comfortable with.

Post your comment

Comments0