The software, which will be demonstrated at the Black Hat computer security conference in Las Vegas next week, relies on a new type of hybrid file that looks like different things to different programs. By placing these files on websites that allow users to upload their own images, the researchers can circumvent security systems and take over the accounts of surfers who use these sites.
"We've been able to come up with a Java applet that for all intents and purposes is an image," said John Heasman, vice president of research at NGS Software.
They call this type of file a GIFAR, a contraction of GIF (graphics interchange format) and JAR (Java Archive), the two file-types that are mixed. At Black Hat, the researchers will show attendees how to create the GIFAR while omitting a few key details to prevent it from being used immediately in any widespread attack.
To the web server, the file looks exactly like a .gif file, however a browser's Java virtual machine will open it up as a Java Archive file and then run it as an applet. That gives the attacker an opportunity to run Java code in the victim's browser. For its part, the browser treats this malicious applet as though it were written by the website's developers.
Here's how an attack would work: the bad guys would create a profile on one of these popular websites - Facebook for example - and upload their GIFAR as an image on the site. Then they'd trick the victim into visiting a malicious website, which would tell the victim's browser to go open the GIFAR. At that point, the applet would run in the browser, giving the bad guys access to the victim's Facebook account.
The attack could work on any site that allows users to upload files, potentially even on websites that are used to upload banking card photos or even Amazon, they say.
Because GIFARs are opened by Java, they can be opened in many types of browsers.
There is one catch, however. The victim would have to be logged into the website that is hosting the image for the attack to work. "The attack is going to work best wherever you leave yourself logged in for long periods of time," Heasman said.
There are a couple of ways that the GIFAR attack could be thwarted. Websites could beef up their filtering tools so that they could spot the hybrid files. Alternatively, Sun could tighten up the Java runtime environment to prevent this from happening. The researchers expect Sun to come up with a fix not long after its Black Hat talk.
But researchers say that while a Java fix may disable this one attack vector, the problem of malicious content being placed on legitimate web applications is a much larger and thornier issue. "There will be other ways to do this, with other technologies," said GIFAR developer Nathan McFeters, a researcher with Ernst & Young's Advanced Security Center.
"In the long term, web applications are going to have to take control of the content," McFeters said. "It's a web application issue. The Java attack that we're currently using is just one vector."
Ultimately, browser makers will have to make some fundamental changes to their software too, said Jeremiah Grossman, chief technology officer with WhiteHat Security.
"It's not that the internet is broken," he said. "It's that browser security is broken. Browser security is really an oxymoron."