Blending different sensors, machine learning, and artificial intelligence (AI) is aim of DARPA research

Sept. 16, 2019
Objective is to develop a holistic object tracking framework using advances in computer-aided tracking to incorporate disparate heterogeneous sensor data.

ARLINGTON, Va. – U.S. military researchers are asking industry to develop find new ways of developing computer-aided target tracking algorithms that can blend data from several different sensors with radically different detection capabilities and meta-information about relative sensor performance in dynamic environments.

Officials of the U.S. Defense Advanced Research Projects Agency (DARPA) in Arlington, Va., issued a Small Business Technology Transfer (STTR) opportunity (HR001119S0035-13) Friday for the Holistic Object Tracking project.

DARPA officials want to develop a sensor framework able to track targets over time, and can learn from extremely limited sets of labeled data, from many disparate sensors, and should incorporate data from large-area sensors with different detection capabilities and sensitivities.

The objective is to develop a holistic object tracking framework that capitalizes on recent advances in computer-aided tracking to incorporate disparate, heterogeneous sensor data to provide a high fidelity fused output.

Related: DARPA pushes ahead with 3-D electro-optical sensors for target identification and tracking

Most object tracking applications today use individual sensors, and most systems that use several different sensors fuse data by correlating tracks on a common geographical display.

The signal-processing subsystems of today's multi-sensor systems, moreover, typically additional multi-sensor disparate sensing modalities and additional, contextual information, which can make multi-object tracking a challenging problem.

This problem can get worse when several disparate sensors with different spatial and temporal sample rates, resolutions, and sensitivities are involved. Instead, researchers want to use artificial intelligence (AI) and deep learning to help sensors with limited a-priori information about the sensor and object appearance track objects.

This project seeks to jump to a Small Business Innovation Research (SBIR) phase-2 level, and from respondents will require descriptions of past efforts and completed work in computer-aided tracking, natural language processing, and computer vision.

Related: Software modelling algorithms for radar detection and tracking introduced by Cambridge Pixel

Companies responding also must identify data sets of their choosing from several open-source data sets, or may suggest their own data sets with detailed descriptions.

Companies chosen to participate will develop a computer-aided tracking framework that can ingest heterogeneous sensor data and form high-fidelity tracks on objects of interest. DARPA is interested in using multi-object and multi-camera video data augmented by simulated open-source data.

Performers must analyze their fused tracker performance compared to current state-of-the-art trackers using metrics like Object Tracking Accuracy (MOTA), Mean Object Tracking Precision (MOTP), and False Alarms per Frame (FAF).

Related: Sensor processing evaluation embedded computer with graphics processing unit (GPU) introduced by Abaco

Applications of this technology primarily will be for military intelligence, surveillance, and reconnaissance (ISR), with an eye to U.S. Navy and Air Force customers. Broader application may include Federal law enforcement agencies, the intelligence community, and the U.S. Federal Aviation Administration (FAA) for improved situational awareness for flight tracking.

Companies interested should submit proposals online at between 27 Sept. and 28 Oct. 2019.

Email questions or concerns to DARPA at [email protected]. More information is online at (

About the Author

John Keller | Editor-in-Chief

John Keller is the Editor-in-Chief, Military & Aerospace Electronics Magazine--provides extensive coverage and analysis of enabling electronics and optoelectronic technologies in military, space and commercial aviation applications. John has been a member of the Military & Aerospace Electronics staff since 1989 and chief editor since 1995.

Voice your opinion!

To join the conversation, and become an exclusive member of Military Aerospace, create an account today!