NASA has big plans for DARPA's scary "Deep Web"

NASA is weighing in on the Memex "Deep Web" search project, hoping to harness DARPA's at-times ominous index to crunch vast quantities of space data. A team at the Jet Propulsion Laboratory (JPL) is working with the Defense Advanced Research Projects Agency to add a contextual layer to search, not only allowing the system to view webpages more like humans might, but even capable of drawing links between images and individual frames of videos. If it succeeds, it could be a much-needed blast of positive PR for a project that has become mired in controversy.

Memex was detailed back in February, an attempt to connect the dots within the so-called "Dark Web" or "Deep Web" that would not normally be indexed by spiders from Google or Bing.

In fact, Memex creator Chris White argued, what search engines currently serve up is less than 5-percent of the internet data out there. His tool, in contrast, could analyze images, videos, pop-up ads, forms, and scripts, among other things, to gain a better understanding of how online content is organized and linked.

DARPA and the Department of Defense planned to use Memex to identify possible patterns of sex trafficking, but many were concerned at the potential for privacy intrusion should it be turned to less moral purposes.

Just as it could highlight arms smuggling, critics pointed out, Memex could be used to dig into personal information like an advanced form of tracking and monitoring.

NASA's involvement could give Memex an air of scientific restraint, however. "We're developing next-generation search technologies that understand people, places, things and the connections between them," Chris Mattmann, principal investigator for JPL's work on Memex, said of the team's goals.

For instance, NASA could feed Memex with data returned from the Curiosity rover's various cameras and scientific instruments, using the search engine to more readily spot patterns and connections on Mars.

"Searching visual information about a particular planetary body could greatly facilitate the work of scientists in analyzing geological features. Scientists analyzing imaging data from Earth-based missions that monitor phenomena such as snowfall and soil moisture could similarly benefit" JPL, NASA

On the flip-side, NASA's own data might end up more accessible. By feeding Memex with a diet of published scientific data – such as oceanic records, climate change recordings, and more – others in the field could find it more straightforward to sift through the ever-growing masses of information on each topic.

JPL is one of seventeen groups working on Memex, with the search tool released under an open-source license.