Electronic binoculars from Northrop Grumman team to detect threats through brain activity
Everyone who has ever watched the Star Wars films has probably at one moment wished they had Jedi abilities, such as mind control or what Lucas called Jedi reflexes—knowing something will happen a second before it does.
By John McHale
LINTHICUM, Md.—Everyone who has ever watched the Star Wars films has probably at one moment wished they had Jedi abilities, such as mind control or what Lucas called Jedi reflexes—knowing something will happen a second before it does.
A team led by Northrop Grumman’s Electronic Systems Sector is looking to bring a similar threat-detection capability to warfighters as part of an advanced research contract to develop a panoramic day/night optical system that will utilize human brain activity to detect, analyze, and alert foot-soldiers to possible threats.
The Defense Advanced Research Projects Agency (DARPA) in Arlington, Va., awarded the contract, which is for the first phase of the Cognitive Technology Threat Warning System program, or CT2WS. DARPA officials say the CT2WS will function as an intelligent neuro-optical system, using the stimuli sensed by brain activity to detect targets at long-range over a wide field of view.
Not able to rely on that mystical energy field called the Force, as Lucas’s characters do, the academic and industrial consortium is looking to drive a breakthrough in soldier-portable visual threat warning devices using technology that has yet to be invented.
The first step for the Northrop Grumman engineers is developing a system and completing a preliminary design for the company’s Human-aided Optical Recognition/Notification of Elusive Threats (HORNET) system.
“Northrop Grumman’s HORNET system leverages the latest advances in real-time coupling of human brain activity with automated cognitive neural processing to provide superior target detection,” says Michael House, Northrop Grumman’s CT2WS program manager. “The system will maintain persistent surveillance in order to defeat an enemy’s attempts to surprise through evasive move-stop-move tactics, giving the U.S. warfighter as much as a 20-minute advantage over his adversaries.”
The subconscious mind is hypersensitive to visual images. The system will take advantage of that by using non-invasive electro-encephalogram (EEG) measurement sensors inside the warfighter’s helmet and attached to his scalp, House says.
A soldier’s subconscious may detect a threat his conscious mind is not registering, House says. The EEG sensors will read the soldier’s neural responses to potential threats, and then the readings will educate the system’s algorithms. For example, the algorithms will identify when the warfighter’s subconscious is detecting a moving target at 5 kilometers or at 10 kilometers, House continues. As missions and threats evolve, the algorithms will be refined.
One of the main goals of the Hornet is to reduce the false alarm rate for target acquisition, House says. The human brain’s visual process has very few false alarms, he adds. This should also significantly improve the range of standard binoculars. Lowering false-alarm rates is crucial for HORNET’s intended applications, which include improvised explosive device (IED) detection and destruction, aided target recognition, and border surveillance for homeland security scenarios.
House declined to go into specifics on the neurological technology, but referenced a public release from the Georgia Institute of Technology in Atlanta. Northrop Grumman is working with a Georgia Tech team led by Paul Hasler, an associate professor in the Georgia Tech School of Electrical and Computer Engineering. Hasler’s team is researching neuromorphic engineering techniques to emulate how the human brain processes images, House says.
“The idea of this project is to build a visual device that is attentive, that can do the kind of low-level visual processing that your eyes do naturally,” Hasler says. “You would see a certain picture in your field of view, but the device would actually be looking over a much wider space—and if it found something interesting it would present you with that picture as well.”
According to the Georgia Tech release “neuromorphic engineering is interdisciplinary, using fields that include biology, physics, mathematics, and computer science, as well as electrical and other types of engineering. Its aim is to develop artificial systems—such as vision devices, auditory processors, or robotic systems—based to a degree on natural biological systems.”
This technology also could lead to new “neural-inspired” approaches in military as well as commercial applications.
Northrop Grumman will also integrate current wide-angle optics, digital imaging, and low-power digital signal processing (DSP), House says. Low power is an important characteristic as this will be a portable warfighter system with heavy DSP requirements, he adds.
Following the first $6.7 million phase of the project, DARPA has the option to extend the contract for two additional phases to develop the subsystems and final handheld assemblies. A prototype is expected to be ready in 2010, House says.
In addition to Georgia Tech, Northrop Grumman’s multidisciplinary team includes SAIC in San Diego; Theia Technologies LLC in Wilsonville, Ore.; Sensics Inc. in Baltimore; L-3 Communications Infrared Products in Dallas; Georgetown University, in Washington; Portland State University in Portland, Ore.; and the University of Colorado in Boulder, Colo.