Grab Your Free 17-Point WordPress Pre-Launch PDF Checklist: http://vid.io/xqRL
Download our exclusive 10-Point WP Hardening Checklist: http://bit.ly/10point-wordpress-hardening-checklist
Check The Robots.txt File For Potential WordPress SEO Issues | WP Learning Lab
The Robots.txt is an important file for controlling access to your website. Inside this file you can block access from different user agents, IP addresses or block access to specific files and folders.
Here’s where it can be a problem.
When a website is in development the developer often restricts access to your site to just a few IPs so that they can work on it and you can view it, but regular traffic isn’t able to access it.
It makes sense, you don’t want your visitors and future customers to see a half-finished website.
On the rare occasion a website will launch without these restrictions being removed from the robots.txt. That destroys your SEO and organic traffic because your site can’t be accessed! Even though everything looks fine to you because your IP is allowed access.
Luckily it’s easy to fix. There are two ways.
You can log into your website’s root folder via FTP or the file manager in your hosting account. Then open the robots.txt file (this file is always in the root folder)
If you have Google Webmaster Tools (GWT) installed you can access and edit your Robots.txt file from there.
For most websites you can delete the robots.txt file or delete it’s contents. If you’re not sure if you should do this check out this website: http://www.robotstxt.org/robotstxt.html
You’ll find all the robots.txt file info you’ll ever need. So if your robots.txt file contains a bunch of code that you want to understand, visit that URL.
I hope this information helps you! If you have any questions leave a comment below or ping me @WPLearningLab on Twitter.
If you want more excellent WordPress information check out our website where we post WordPress tutorials daily.
Connect with us:
WP Learning Lab Channel: http://www.youtube.com/subscription_center?add_user=wplearninglab
Google Plus: http://google.com/+Wplearninglab