Sunday, May 28, 2006 

Opting Out of Open Directory Listings for Webmasters

What has bothered the webmasters previously is that when search engines preferred search result descriptions from dmoz.org, they did not empower webmasters to opt-out of those descriptions. This can be especially annoying if the descriptions from dmoz.org are outdated, or just plain inaccurate.
We had one customer who was frustrated because the ODP description of their site mentioned “favours” and was listed under Canada when their site was actually in the United States and was spelled as “favors”. All they wanted was a way to specify that MSN Search should use the description from their page instead of using ODP.

So what we did was introduce a new option at the page level - a robots meta tag – that tells the MSN search bot not to use the DMOZ site snippet. This is something that only can be done at Web page level, by a webmaster, and is not done as part of the robot.txt file.

So in your Web page you’d put



or

 

New spider Adsbot-Google to check landing page quality

Yet another Google spider is going to be showing up -- AdsBot-Google will be used to monitor AdWords landing page quality.
While we strongly recommend against restricting our system's automatic review of your landing page, you can edit your site's robots.txt file to avoid a review. The file must explicitly exclude your page from our system visits as follows:
To prevent AdsBot-Google from accessing your site, add the following to your robots.txt file:
User-agent: AdsBot-Google
Disallow: /

To prevent AdsBot-Google from accessing parts of your site, add the following to your robots.txt file:
User-agent: AdsBot-Google
Disallow: /exclude/

http://adwords.google.com/support/bin/answer.py?answer=38197




Depending on any current robots.txt restrictions, we also might want to be sure we aren't accidentally excluding the AdsBot. What isn't clear to me is whether AdsBot will also be participating in the big cache sharing free-for-all along with all the other googlebots. Also, I hope the user agent includes the string "googlebot" so various stats packages automatically catch it as a Google spider.

 

Google Pages launches

Google Pages launches
February 23rd, 2006

The rumor mills were right again, Google has launched a web page creation tool called Google Pages. It uses AJAX to smooth along the process and gives you up to 100MB of storage at http://yourgmailusername.googlepages.com. ValleyWag noted this was coming several weeks ago and in the same post hinted at Google Calendar. Maybe it’s not too far off..

Like many recent Google product launches, demand has outpaced infrastructure. Many readers are reporting getting a message that Google has limited accounts and will email when more slots open up. Why they can’t throw more servers at a project before launching remains to be seen. At least you never get a similar message while doing a search.

 

Google Desktop 4

Google Desktop 4

Desktop is becoming more important in every iteration thanks to Microsoft’s upcoming release of Vista that will be the first big shot back from Redmond to Mountain View. Google added its own version of widgets, which it’s calling Gadgets just like Microsoft is in Vista. Yahoo has its own widget app, it’s a popular vertical these days.