The so-called “dark web” is where the visible Internet – easily accessible by public search engines and web crawlers – ends, and, everything else begins.

The dark web is an unseen iceberg composing more than 95% of the real activity of the web, where databases, password-protected websites, official records from federal, state and local governments, various intranets, messageboards, website archives, forums and vast catalogues of data all reside.

White and black hat hackers, law enforcement agencies and criminal networks all operate there in the shadows.

And now, DARPA, the Pentagon’s secretive Defense Advanced Research Projects Agency, has developed a surveillance tool known as “Memex” that can be used to reveal what is hidden on the dark web, and pursue criminal groups of interest – and perhaps control or even kill the dark web for good.

CBS 60 Minutes reported on DARPA’s Memex, interviewing the project’s director and some of the principles involved. Ostensibly, the effort is focused on tracking and stopping sex trafficking… but obviously so much more is part of this secret military surveillance project. Watch the video report here.

Dan Kaufman, the director of information innovation office at DARPA said:

“The easiest way to think about Memex is: How can I make the unseen seen?.”

“Most people on the internet are doing benign and good things,” he continued, “But there are parasites that live on there, and we take away their ability to use the internet against us — and make the world a better place.”

Curtailing the Wild West of the Internet – whether or not it illegal activity is involved – is clearly within the scope of its possible applications.

Leslie Stahl: Why is DARPA looking at sex trafficking?

Chris White (Inventor of Memex): “We see that with human trafficking, the kinds of groups that do human trafficking are often using the proceeds of that to fund other things are counter to our national security interests. And we also find that people willing to traffic in women are also willing to move drugs and guns, and other sort of contraband.

Chris White: “The internet is much, much bigger than people think. By some estimates Google, Microsoft Bing, and Yahoo only give us access to around 5% of the content on the Web.”

Author Cassius Methyl raised questions about this secret project’s ominous potential for the future of Internet freedom, noting that it can “probably do a lot more than they say it’s capable of”:

As the Scientific American noted, DARPA has said very little about Memex. What does it tell about a person, a group of people, or a program, when they are secretive and operate in the shadows? Why would a body of people doing benevolent work have to do that? I think keeping up with the projects underway by DARPA is of critical importance. This is where the most outrageous and powerful weapons of war are being developed.

She notes a Scientific American article about Memex:

“DARPA has said very little about Memex and its use by law enforcement and prosecutors to investigate suspected criminals.” and ““Memex”—a combination of the words “memory” and “index” first coined in a 1945 article for The Atlantic—currently includes eight open-source, browser-based search, analysis and data-visualization programs as well as back-end server software that perform complex computations and data analysis.”