Google has launched a new Web indexing system called Caffeine to provide users with fresher search results, the company said on Tuesday.
Caffeine provides 50 percent fresher results for web searches than Google's last index, and it's the largest collection of web content the company has offered, Google said. Whether it's a news story, a blog or a forum post, users will be able to find links to relevant content much sooner after it is published than was possible before, Google added.
Previously, Google crawled a fraction of the Web each night, indexed it and pushed it out in its results. With Caffeine, as Google crawls the Web and finds new information, it indexes it immediately.
The image below illustrates how Gogole's old indexing system worked compared to Caffeine:
Google's old index had several layers, some of which were refreshed at a faster rate than others; the main layer would update every couple of weeks. To refresh a layer of the old index, Google would analyze the entire web, which meant there was a significant delay between when Google found a page and made it available to users.
With Caffeine, Google analyzes the web in small portions and update its search index on a continuous basis, globally. As Google finds new pages, or new information on existing pages, it can add these straight to the index.
Every second Caffeine processes hundreds of thousands of pages in parallel. Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day.