The Internet Archive just saved its one trillionth webpage

The Internet Archive has officially preserved its one trillionth webpage, marking a historic milestone in digital preservation. After nearly 30 years of careful work, the nonprofit organization has protected a vast part of human history, shielding it from the internet’s natural impermanence.

Digital content is famously fleeting. In 2019, an unexpected server migration error mistakenly deleted about 50 million songs from 14 million artists uploaded to MySpace between 2003 and 2015.

To avoid such major data losses, the Internet Archive has used web crawlers since 1996 to archive publicly accessible sites, relying on both automation and volunteer uploads to keep a lasting record of the web’s development.

Today, the archive holds more than one trillion webpages, 41 million texts, and countless other media formats. Storing this enormous collection requires roughly 100,000 terabytes of data—about the capacity of 50,000 high-end iPhones. With around 500 million new websites added daily, the collection continues to grow rapidly.

Although vital for researchers, journalists, and the public, the Internet Archive faces increasing obstacles. The rapid growth of generative AI has prompted tech companies to aggressively scrape the web for training data.

In response, major media outlets such as The New York Times, The Guardian, and Gannett have begun blocking the Archive’s crawlers to prevent their newer content from being used to train AI models.