The Internet Archive plans next week to announce the opening of a new data centre to house two petabytes of information for its Wayback Machine, the digital time capsule that stores archived versions of Web pages dating back to 1996.
The Wayback Machine houses 85bn web pages archived for more than a dozen years, which amounts to three petabytes of data. Only five years ago, the Wayback Machine contained about 30bn web pages. It is expected to continue to grow by 100TB of data per month now that it's live.
The Internet Archive's massive database is mirrored to the Bibliotheca Alexandrina, the new Library of Alexandria in Egypt, for disaster recovery purposes.
According to an event invitation from Sun Microsystems, the Internet Archive is moving from a traditional data centre filled with standard Linux servers to one that runs Solaris 10 with ZFS on Sun Fire x4500s servers inside a Sun Modular Datacenter. The modular system is an all-in-one data center housed in a metal shipping container for mobility.
Because of the modular design, Sun said the data centre was deployed in a tenth the time it would take to build a typical bricks-and-mortar data centre. The Wayback Machine Sun Modular Data Center can service 500 inquiries a second, Sun said. A spokesperson for the Internet Archive said the user interface on the Wayback Machine will not change.
The Internet Archive is a nonprofit organisation located in the Presidio in San Francisco, with data centres in Redwood City and Mountain View, California. The archive not only keeps snapshots of web pages, but software, movies, books, and audio clips.
Users can surf the Wayback Machine by typing in the web address of a Web site or webpage and then choose from a series of dates that reflect the stored images. The site does not currently support keyword search.