Google tells us how big the Web is – but not many pages it indexes
By Dave Parrack
How big is the Web? That’s a huge question to ask, and one which Google seems to have the answer to. But while the company may claim the Web is made up of over 1 trillion unique URLs, that is a much larger number than Google actually indexes – a number that remains a mystery.
I’ve pondered this question before, and I was as amazed as anyone when Google announced way back in the year 2000 that its search engine indexed 1 billion pages on the Web. That number is just incredible, and way beyond my limited imagination. Now, in a new post on the Official Google Blog, the company claims to have hit another important milestone, finding 1 trillion (1,000,000,000,000) unique URLs on the Web at the same time.
If 1 billion Web pages is beyond my understanding, then you can imagine that 1 trillion is a number that I can’t even begin to fathom. Luckily, that number only represents the amount of unique pages currently on the Web, and not the amount that Google actually indexes.
The post explains how Google finds these pages, and then presents them to you. But apart from a side note in the fourth paragraph, it’s not really explained that this number doesn’t represent how many pages Google indexes for you and I to be able to find quickly, and easily, using its search engine.
As TechCrunch points out, estimates on how many pages Google indexes are nearer 40 – 60 billion. Not an insignificant number by any means, but a lot less than 1 trillion. Which means that a massive amount of the Web is being left out of Google searches. Some of it is spam, and some of it repeat content with a slightly different URL, but there must still be quite a chunk of the Web out there in no-man’s land.
Google has a brilliant search engine, and one which I use multiple times in a day. And it surely is “the most comprehensive index of any search engine”. But all that claim does is show how much content its rivals are missing out on, as Google itself is missing a huge amount.
Related:
Entry was posted
on Saturday, July 26th, 2008 at 2:35 pm
under
Read more entries by Dave Parrack.
Stumble It!
