We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message
80,259 News Articles

Internet Archive upgrades to 2-petabyte data centre

Online digital time capsule now holds 85bn pages

The Internet Archive plans next week to announce the opening of a new data centre to house two petabytes of information for its Wayback Machine, the digital time capsule that stores archived versions of Web pages dating back to 1996.

For example, this is what PC Advisor's website looked like ten years ago, and what Google looked like in 1998.

The Wayback Machine houses 85bn web pages archived for more than a dozen years, which amounts to three petabytes of data. Only five years ago, the Wayback Machine contained about 30bn web pages. It is expected to continue to grow by 100TB of data per month now that it's live.

The Internet Archive's massive database is mirrored to the Bibliotheca Alexandrina, the new Library of Alexandria in Egypt, for disaster recovery purposes.

According to an event invitation from Sun Microsystems, the Internet Archive is moving from a traditional data centre filled with standard Linux servers to one that runs Solaris 10 with ZFS on Sun Fire x4500s servers inside a Sun Modular Datacenter. The modular system is an all-in-one data center housed in a metal shipping container for mobility.

Because of the modular design, Sun said the data centre was deployed in a tenth the time it would take to build a typical bricks-and-mortar data centre. The Wayback Machine Sun Modular Data Center can service 500 inquiries a second, Sun said. A spokesperson for the Internet Archive said the user interface on the Wayback Machine will not change.

The Internet Archive is a nonprofit organisation located in the Presidio in San Francisco, with data centres in Redwood City and Mountain View, California. The archive not only keeps snapshots of web pages, but software, movies, books, and audio clips.

Users can surf the Wayback Machine by typing in the web address of a Web site or webpage and then choose from a series of dates that reflect the stored images. The site does not currently support keyword search.

Computerworld US


IDG UK Sites

Nexus 6 vs Sony Xperia Z3 comparison: Lollipop phablet takes on KitKat flagship smartphone

IDG UK Sites

Why people aren't upgrading to iOS 8: new features are for power users, not the average Joe

IDG UK Sites

Free rocket & space sounds: NASA launches archive of interstellar audio on SoundCloud

IDG UK Sites

iPad Air 2 review: Insanely fast and alarmingly thin. Speed tests, camera tests, beautiful...