We use cookies to provide you with a better experience. If you continue to use this site, we'll assume you're happy with this. Alternatively, click here to find out how to manage these cookies

hide cookie message
80,258 News Articles

Internet Archive upgrades to 2-petabyte data centre

Online digital time capsule now holds 85bn pages

The Internet Archive plans next week to announce the opening of a new data centre to house two petabytes of information for its Wayback Machine, the digital time capsule that stores archived versions of Web pages dating back to 1996.

For example, this is what PC Advisor's website looked like ten years ago, and what Google looked like in 1998.

The Wayback Machine houses 85bn web pages archived for more than a dozen years, which amounts to three petabytes of data. Only five years ago, the Wayback Machine contained about 30bn web pages. It is expected to continue to grow by 100TB of data per month now that it's live.

The Internet Archive's massive database is mirrored to the Bibliotheca Alexandrina, the new Library of Alexandria in Egypt, for disaster recovery purposes.

According to an event invitation from Sun Microsystems, the Internet Archive is moving from a traditional data centre filled with standard Linux servers to one that runs Solaris 10 with ZFS on Sun Fire x4500s servers inside a Sun Modular Datacenter. The modular system is an all-in-one data center housed in a metal shipping container for mobility.

Because of the modular design, Sun said the data centre was deployed in a tenth the time it would take to build a typical bricks-and-mortar data centre. The Wayback Machine Sun Modular Data Center can service 500 inquiries a second, Sun said. A spokesperson for the Internet Archive said the user interface on the Wayback Machine will not change.

The Internet Archive is a nonprofit organisation located in the Presidio in San Francisco, with data centres in Redwood City and Mountain View, California. The archive not only keeps snapshots of web pages, but software, movies, books, and audio clips.

Users can surf the Wayback Machine by typing in the web address of a Web site or webpage and then choose from a series of dates that reflect the stored images. The site does not currently support keyword search.

Computerworld US


IDG UK Sites

Best Christmas 2014 UK tech deals, Boxing Day 2014 UK tech deals & January sales 2015 UK tech...

IDG UK Sites

Chromebooks: ready for the prime time (but not for everybody)

IDG UK Sites

Hands-on with Sony's latest smartglasses

IDG UK Sites

Apple TV expert tips: get US Apple TV content, watch Google Play, use multiple Apple IDs and more