The US Intelligence Community has announced a new center to exploit publicly available information for intelligence purposes.
“US intelligence chief John Negroponte announced Tuesday the creation of a new CIA-managed center to exploit publicly available information for intelligence purposes. The so-called Open Source Center will gather and analyze information from a host of sources from the Internet and commercial databases to newspapers, radio, video, maps, publications and conference reports. Douglas Naquin, the center’s director, said it will build on the work of the CIA’s Foreign Broadcast Information Service, which once monitored and translated foreign radio broadcasts but has since expanded its reach to other media.
The FBIS’ strength has been in tracking media trends that show, for instance, how foreign publics are responding to particular US policies. But in recent years a group at the FBIS also has been applying data mining techniques to analyze massive volumes of information on the Internet or commercial databases, Naquin said. He said they are “trying to use what we call the volume problem against itself in some type of judo fashion — you know, the more volume the better — so we can kind of get some sense of where the trends are and what the buzz is on certain topics, or who’s connected to whom.” (Link)
I’m certainly not in the loop on what the intelligence community does now and where they are going, but the nature of ‘open source’ information is evolving rapidly on the web and will continue to do so for many years. I suspect that the intelligence community will need to explore new ideas and techniques to deal with and mine the flood of new information coming from web pages, the blogosphere, email, online communities, mailing lists and groups, metadata annotations, open semistructured databases (e.g., Google Base), semantic web data, search engines, etc. The new open sources provide a much clearer and more direct view of what people are thinking, concerned about and, in some cases, doing. Add in increasingly better techniques for text mining, information extraction and language understanding the potential is great– for both good and bad uses. (Link)