Early this month Google announced a new user agent for the robots.txt file to direct the Google News bot on what to do. The new user agent, Googlebot-News, is used just the same way you use the Googlebot agent. To make things clear though here are some of the examples given by Google on the use of the new user agent.
Include pages in Google web search, but not in News:
User-agent: Googlebot
Disallow:User-agent: Googlebot-News
Disallow: /Include pages in Google News, but not Google web search:
User-agent: Googlebot
Disallow: /User-agent: Googlebot-News
Disallow:Block different sets of pages from Google web search and Google News:
User-agent: Googlebot
Disallow: /latest_newsUser-agent: Googlebot-News
Disallow: /archives
According to Google “The pages blocked from Google web search and Google News can be controlled independently. This robots.txt file blocks recent news articles (URLs in the /latest_news folder) from Google web search, but allows them to appear on Google News. Conversely, it blocks premium content (URLs in the /archives folder) from Google News, but allows them to appear in Google web search.” Note that you can do this for any specific page.
Stop Google web search and Google News from crawling pages:
User-agent: Googlebot
Disallow: /
In this case since Googlebot is disallowed and there is no specific instruction on what to do with Google News, the news bot will play it safe and simply not crawl the page.
Happy New Year!
Filed in: Google, SEO Tips, SEO lessons




















You might be quite happy with your current pagerank in search engine results pages (SERPs) but have you given consideration to your mobile presence? If you haven’t now is the time to act. We all know how rapid mobile web’s popularity is growing so think of all the opportunities you are missing out on by having no or limited presence in the mobile web.
Michael Jackson’s passing triggered a wave of sadness and remembrance all over the world. People expressed their respect and love for MJ in so many ways, from the usual dancing to SEO parallelisms. Huh? You got it. Not wanting to pirate the original content I will just be giving the outline then ask you to head on to the original post to read it in its entirety.
Pagerank sculpting, which as defined by Matt Cutts is simply trying to change how PageRank flows within your site using different methods such as the “nofollow” tag, is still plausible, however, in his post
Every single link counts. That’s my motto when it comes to link building. While it is true that some links pass more value than other in my opinion you still shouldn’t overlook opportunities to get a backlink, whether it’s just one or a couple or a whole load of links. This is also true whether the page rank of the linking page is zero or ten. A link is still a link.





Recent Comments