Engineers at Mitsubishi Electric's main research and development centre have put parallel computing to use in developing an information retrieval system that promises faster searches through large amounts of data.
The system, unveiled yesterday at the company's research and development centre in western Japan, is based on 16 PC servers connected through a gigabit switch and controlled via a host PC. It boasts a free keyword, full-text search of 100 billion characters per second, according to Atsushi Murata, one of the developers who worked on the system.
Traditional information retrieval systems are faltering as the amount of data increases, because of a bottleneck in getting the data out of huge databases, said Murata. Some systems cache heavily used data in memory to get around this, but response gets slower again as soon as the memory cache is full. In contrast, Mitsubishi Electric's new system splits the data between PC servers, which shares the load and leads to increased search speeds.
"We put the CPU (central processing unit) near the storage so we can get data fast from storage," said Murata.
In the demonstration system, the PC servers, which were off-the-shelf, rack-sized 1GHz Pentium III-based models produced by Mitsubishi Electric, had three 36GB hard disks attached for a total 108GB of storage space per server and 1.7TB (terabytes) for the whole system. It can be scaled up to 256 PC servers and 27TB of data and extra servers can be added without the need to update the application software, said the company. All in all it sounds like the computer Deep Thought from Hitchhiker's Guide to the Galaxy.
The servers run Linux while the host computer, which handles the queries and delivers the results, is based on Windows 2000, said Murata. The company has plans to put the system on sale later this year in Japan. Pricing was not disclosed.