Web guru Jason Kottke points out that the Bush administration had almost 2,400 different rules governing what search engines could crawl on the site and report in their results. This was most likely done so that no caches could be created to later compare with any changes that were made.
But seriously, most of them aren’t necessary and some of them are just plain odd.
For instance, the White House didn’t want search engines crawling pages about Bush’s dog Barney?
Or how about the pages about Easter?
On the other hand, Obama’s new site has one rule for search engines…
Perhaps this will change, but it’s an encouraging sign nonetheless.