The agreement should see Google delivering search results via Yahoo Japan before the end of this year.
When implemented it will mean Google's technology will be responsible for the vast majority of searches done from home and office PCs in Japan. The two companies taken together handled 90 percent of the 4.5 billion queries sent from PCs in homes and offices in Japan, according to April data from NetRatings Japan.
On the mobile web the deal will result in Google securing roughly half the market.
Softbank Mobile, Japan's number three cellular provider and a sister company of Yahoo, uses Yahoo as its default mobile web service. This will also change to make use of Google's search engine. Number two carrier KDDI already uses Google as its default search engine and the two companies together have about half the market. Leader NTT DoCoMo uses the Goo service from sister company NTT Com.
The deal flies in the face of a 2009 agreement between Yahoo and Microsoft to use the latter's Bing search engine. The deal covered Yahoo's main US site and many of its international services, but Yahoo Japan wasn't covered because Yahoo doesn't hold a majority stake in the company. Softbank, an early investor in Yahoo, holds the largest stake in Yahoo Japan with the U.S. company owning just under 34 percent.
Yahoo Japan said it doesn't expect the Tuesday agreement with Google to affect the financial relationship between it and Yahoo Inc.
For Google the deal represents a further boost for its Japanese operation, which has lagged Yahoo Japan since the launch of Google's Japanese-language search in 2000.
Japanese consumers have remained loyal to Yahoo thanks to a bouquet of popular services that include shopping and auction sites, maps and weather, and a news service that pulls in content from almost all of Japan's major publishers. It is consistently the most popular Web site in the country.
The new deal isn't the first time the two companies have worked together. Yahoo Japan used Google's search engine from 2001 until 2004, when it switched to the engine provided by Yahoo in the U.S.
Tuesday's announcement came as Yahoo Japan reported financial results for the second quarter. The company said net profit jumped 12.6 percent to ¥21.7 billion (£161 million) on a 4.2 percent increase in sales to ¥70.5 billion.