How to Exclude WordPress Content from a Google Search
The first question is why a business or organization would want to hide any of its website content from Google. Doesn’t this go against the basics of SEO? You want to do everything possible to attract visitors to your site and see to it that your pages are indexed and ranked well by search engines.
Well, there are some reasons for excluding pages from search engine indexing. Many companies have “members only” pages that cannot even be accessed by the general public, and it would make sense for those not to be indexed and ranked. Other website pages are for staff only and are obviously password-protected with limited access.
It’s important to tell Google, not to index and rank pages that you do not want for public access. If such a page should show up, and a visitor then be turned away, that experience with the company is devalued. The best approach is to “hide” those pages from search engine crawling in advance.
And if you have a WordPress website, this is a relatively easy thing to do, in three different ways. Let’s take a look at each of these so that you can figure out which will be easiest and most convenient for you.
1. Use Yoast SEO Plugin
If you do not have Yoast, you might want to consider checking it out. The primary purpose of Yoast is to help you get a better ranking in search engine results, and it is probably the most popular and most-used plugin for SEO. Even well-known professional writing services, like Trust My Paper and Grab My Essay have WordPress sites and use Yoast to evaluate their website content.
And Yoast is probably the easiest way to exclude content from search engine indexing. If you are a newbie to this practice of “hiding” pages, then this is probably the easiest method.
The best explanation of this process comes from WP Beginner. Here is how you do this:
- Install and activate the Yoast plugin.
- The next step is to edit the content that you want to hide from search engine indexing
- Go to the Yoast meta box underneath the “post editor” and click on “advanced settings”
- In the “meta robots index” choose “noindex.”
- Then click “no follow”
- Now you can “save/publish” your page
- There will be a line of code added to your page: 1<meta name=” robots” content=” noindex.nofollow”/>. This line of code will alert search engines not to follow or index that page.
- Use a Robots.txt File to Hide a Page
Newbies may not know about robots.txt files. In short, it is a file that allows a website to give instructions to search engine bots that are crawling through the web. These bots look for robots.txt files on servers before they read any other files on a website. They are looking for any special instructions on how to crawl through the site. You can set up instructions for a bot to ignore certain files or pages through your robot.txt on WordPress. One thing to remember, though: your excluded pages will still be crawled and indexed if other pages on your site are linked to them.
With that said, here how this works:
- Go to your “File Manager” setting in your WordPress hosting panel
- You will see a choice for “robots.txt file.
- Under “User-agent” you need to include the search engines you wish to target. A “*” will indicate all of them.
- Then where it says “disallow,” you enter the URL that includes the page title – it can be a specific page, a blog post, etc.
- Be certain that you save the changes and then upload your robots.txt file back to your server. This is a matter of a couple of clicks.
If you are new to all of this, you are probably better off sticking with option #1. The other disadvantage of using this method is that it will not keep your robots.txt file from public access. While people cannot access the file/page itself, they can see that you are hiding a particular one. And, non-search engine bots that crawl the web to do bad things (spread malware, get private information, etc.) will ignore these instructions and hack in anyway.
- Protect a Page with a Password
This is a method by which you can allow access to a page only by users who have a password to do so. Obviously, search engine bots cannot get to it either.
Every time you create and finish a page, either on your website or in your blog, you will be prompted to “publish” it. Before you click that “publish” button in the publish meta box, simply click on the “Edit” link that is next to the “visibility” option. The default option is “public.” But when you click the “edit” button, you will have the option to keep it public, make it private, or make it “password protected.” If you make it “private,” it will mean that only you and any others who have administrative control can access it. That is probably not what you want to do. Your goal is to allow access to others, but only those who have a password that you can give them.
Bill Reams, a site manager for the writing service Studicus, puts it this way: “We have pages on our site that are only for our writers, not for our clients. We want our writers to be able to access them, because there is important information about their responsibilities, their current assigned tasks, their earnings, etc. Using password protection on these pages keeps everything private and safe. It’s a perfect solution for us.”
Here is exactly how you do this.
Once you enter your choice as “password protected” and then your password, you simply click “publish,” and it’s done. Any visitor to your site who tries to access that page will have to have the password and that includes search engine bots.
Conclusion
There are certainly good reasons for hiding certain pages on your website – pages that you don’t want search engine bots to index and rank so that they don’t show up in search results. These three methods of hiding those pages should work – choose the one that is most convenient.
Author Bio:
Angela Baker is a self-driven specialist who is currently working as a freelance writer for a number of writing services, as well as review sites like Pick The Writer. Her goal is to become an in-demand blog writer, and she is fast becoming just that, contributing to a number of blogs, such as Live Inspired Magazine, on a wide variety of topics. She is committed to becoming a life-long learner in order to educate and inspire others with her writing.