Google has revealed a storage software update was responsible for hundreds of Gmail users being unable to access their old emails yesterday.
"We released a storage software update that introduced the unexpected bug, which caused 0.02 percent of Gmail users to temporarily lose access to their email. When we discovered the problem, we immediately stopped the deployment of the new software and reverted to the old version," Ben Treynor, Google's vice president of engineering, said on a blog.
"The good news is that email was never lost and we've restored access for many of those affected. Though it may take longer than we originally expected, we're making good progress and things should be back to normal for everyone soon."
Treynor said Google stores multiple copies of its user's data in a number of different data centres. However, "in some rare instances software bugs can affect several copies of the data".
"That's what happened here. Some copies of mail were deleted. To protect your information from these unusual bugs, we also back it up to tape. Since the tapes are offline, they're protected from such software bugs. But restoring data from them also takes longer than transferring your requests to another data center, which is why it's taken us hours to get the email back instead of milliseconds."
Web users first experienced problems on Sunday morning. One user known as Wienke said on the Gmail help forum: "This morning when I woke up I only saw two mails in my Gmail box that were sent last night. All mail was gone. I also got some notifications which you will get when you have a new account. Seems something must have been reset".
Further complaints flooded in on the same thread and other threads in the forum, leading Google to confirm the bug on its Apps Status Dashboard.
"This issue affects less than 0.08 percent of the Google Mail user base. Google engineers are working to restore full access. Affected users may be temporarily unable to sign in while we repair their accounts," the search engine said at the time.
Treynor also warned users that messages sent between 2am and 10pm (GMT) on Monday February 28 was unlikely to have been delivered.
“Thanks for bearing with us as we fix this, and sorry again for the scare,” he added.