Yahoo announced and released new edition of their new spider, Yahoo Slurp. Wait a second wasn’t their spider already called Yahoo Slurp? Yes it was, but this new version is apparently faster and more efficient that the old one. Isn’t that what they said when they built the bionic man? Ah good old Steve Austin, the Six Million Dollar Man. Well this may not be the Six Million Dollar Search Engine Spider, but it is a step in the right direction.

Yahoo states that we can expect a "25% reduction in the number of requests and bandwidth consumed by the crawler." As a result, when you check your log files next month you can expect to see less crawler activity from Yahoo Slurp. Yahoo has been testing the new spider in conjunction with the old one and it wasn’t until earlier this week that it was fully deployed.

So what does this mean for site owners? Well not much yet, with the exception of not getting hammered by page requests from Yahoo Slurp as often. You may however, experience some minor changes in rankings and in the indexing of your site pages in Yahoo.

The “old” Yahoo Slurp already followed redirects and attempted to crawl dynamic URLs. Yahoo claims “Yahoo! Slurp has support for frames and makes an effort to crawl complex URLs such as those generated by forms, content generation systems, and dynamic page generation software.” Does this mean that Yahoo Slurp will attempt to complete forms on the websites it crawls? Probably not, but it is something to think about.

With the launch of the new crawler, Yahoo has retired the old Slurp spider. You may want to keep an eye on Yahoo Slurp through your log files to see if there are any major changes in activity. From here you will want to pay close attention to rankings in Yahoo to see if they are affected by the new crawler activity.

Yahoo Slurp the Second has entered the building... you may now be seated.

Jody Nimetz
Organic Search Strategist

__________________
-
-