Learn How Google Robots View and Index Your Site

Nov 14, 2008   //   by Gerald Weber   //   SEO Blog  //  9 Comments

Getting Google’s view of your website and diagnosing potential problem areas and how Google works Also identify potential SEO problems.

webmaster toolsI have personally been using Google webmaster tools for only a couple of years now, however I could not imagine my life/career as an SEO professional without it now. For those of you who don’t currently use Google webmaster tools you should start right away. You can go here www.google.com/webmasters/tools and simply sign in with your Google account.

What is Google webmaster tools?

Google Webmaster tools is a very powerful tool that was created by Google a few years ago in response to popular demand of many webmasters wanting a better way to become more “Google Friendly”. Google webmaster tools can help you get a great deal of feedback from Google regarding how Google robots view and index your site as well as assisting you in diagnosing potential problem areas in your site.

Key features:

Site verification:

Google Webmaster tools allows you to cofirm ownership of your site by either placing a meta tag in the code on your site or placing an html file in your root directory. This will let Google know you are acting in good faith to become more “Google friendly” and will open up the doors to the other features that Google webmaster tools has to offer.

XML sitemaps Submission:

Google Webmaster tools allows you to register your XML sitemap making it easier for the Google spider to index and crawl your site.

Key Indexing data about your site:

Google webmaster tools gives you very detailed information regarding the indexing of your site. From the last time the Google spider crawled your site to any 404 pages that were found at the time.

Key Statistics:

Google webmaster tools Allows you to see the top search queries that were used to find your site in Google search results.

Other sites linking to you:

Google webmaster tools will display a detailed list of other sites that are linking to your homepage.

Internal links:

Google webmaster tools allows you to easily see the details of your internal linking structure. This is a very important part of on page SEO.

Generate robot.txt file:

Google webmaster tools let’s your create a robot.txt file that allows you to instruct Google as to what directories you don’t’ want crawled by the Google spider.

Set your preferred domain:

You can instruct Google as to how you would like your domain name to appear in the search results i.e. http://domainname.com or http://www.domainname.com.

Geographic target:

Google webmaster tools allows you to set a geographic target. i.e. We are a Houston based company and most of our customers are in Houston so we have our geographic target set to Houston TX.

To summarize if learn to you use Google webmaster tools to it’s potential your SEO will greatly improve and you will have a more “Google friendly” website as a result.

Similar Posts:

Gerald Weber

I founded Search Engine Marketing Group in December 2005. More recently I co-founded viralcontentbuzz.com. which is the free platform that helps bloggers generate REAL "social buzz" on their best content. Feel free to follow me on Google+

More Posts - Website

Follow Me:
TwitterFacebookLinkedInPinterestGoogle PlusStumbleUponDelicious

"Newsletter" Our weekly newsletter features some of the best curated SEO content from around the web!

Comment Policy

  • Abdulrehman says:

    I’d agree with that, I’m usually curious of checking whether I am ranking good for keywords or not, and Google webmasters sure helps me do that!

  • Gerald Weber


    Google Webaster Tools is really an amazing tool!

  • Farrhad says:

    Great post 🙂

    Hope to read more like these!

    <abbr>Farrhad’s last blog post..5 Free SEO Tools You’re Probably Not Using Yet</abbr>

  • Gerald Weber


    Glad you enjoyed this articles. If you would like to read more articles like this simply subscribe to our RSS or you can subscribe by email if you like. We publish weekly articles regarding SEO tips, search engine marketing and just blogging in general.

  • Rob says:

    It can also help you get cached pages that no longer exist to be removed.
    .-= Rob´s last blog ..Lack of Business Plan Could Equal Doom for Twitter =-.

  • Noah says:

    I think that its important to note that even Google webmaster tools doesn't show all your inbound links, so hopefully people don't spend too much time getting obsessed with it. I believe the words Matt Cutts used to describe it was something along the lines of, "It shows a good chunk" of your incoming links, or something along those lines.
    .-= Noah | WealthNet Partners Blog´s last blog ..The Power of Planning for Growing Your Business =-.

  • I always recommend for everyone to install Google webmaster tools on their site. It just shows Google that you are not trying to hide anything from them., especially if you are heavy into SEO efforts.
    .-= Nick Stamoulis´s last blog ..How Do You Become An Authoritative Website? =-.

  • Stephane Bottine says:

    Hi Gerald, Google Webmaster Tool is definitely a great way to put your website on Googlebot’s radar. Another one is linking from other sites to yours. According to Matt Cutts, Googlebot’s crawl priority is proportional to your site’s PageRank.

  • […] Add your site to Google Webmasters tools: There are multiple reasons to add your site to Google Webmaster Tools. For an in-depth look at all of the reasons, check out this post. […]