Increasing Index Size and Query Capacity
- Huge increases in index size in ’99, ’00, ’01, …
– From ~50M pages to more than 1000M pages - At same time as huge traffic increases
– ~20% growth per month in 1999, 2000, …
– … plus major new partners (e.g. Yahoo in July 2000 doubled traffic overnight) - Performance of index servers was paramount
– Deploying more machines continuously, but…
– Needed ~10-30% software-based improvement every month