The UAV video problem: using streaming video with unmanned aerial vehicles

By Tim Klassen

Unmanned aerial vehicles (UAVs) present different video handling requirements to traditional manned vehicles. These demands can be summarized as:

    1) the number of video streams handled;
    2) the criticality of the video data link to the success of the mission;
    3) even greater space/weight/power limitations;
    4) potentially more demands on distribution, such as several remote data consumers; and
    5) varying and, at times, restrictive limitations on data-link bandwidth.

When it comes to adding video data streams in manned vehicles, normally only intra-vehicle distribution is required. For most of the video display need, this can usually be handled by adding wired connections for video sources as necessary and routing physical connections to display destinations. Video transmission to a ground station is either not needed at all or is implemented as an add-on that is non-critical to the safety or functionality of the vehicle. When additional video streams are installed in an unmanned vehicle, the data has to be encapsulated and transmitted to a ground station using a relatively low-bandwidth data link. This adds increasing pressure on the video capture, compression, and transmission solution implemented on the aircraft.

Reconnaissance UAVs like HERTI from BAE Systems use H.263/H.264 codecs for video transmission.
Click here to enlarge image

Additionally, in unmanned air vehicles, the criticality level of the video data is entirely different. For unmanned systems, video data constitutes the only “eyes-in-the-sky” and thus the only link that a ground crew may have with what is actually going on in the vicinity of an unmanned air vehicle.

As it is not unusual for the unmanned air vehicle to be “flown” by a ground crew hundreds or even thousands of kilometers away, the video data link becomes absolutely critical to the safety and success of the mission. The figure on the following page shows a typical scenario with a target, UAV, and ground station configuration. For this arrangement to function safely and effectively, the quality, reliability, and latency of the video stream must be optimized. Also, in years the number and resolution of video sensors mounted on the unmanned vehicle have continued to increase, while the demands on quality, reliability, and latency remain unchanged.

With the addition of weaponized unmanned platforms, the need for more information and, in particular, more video information, has continued to multiply. It has resulted in a massive increase in the amount of video data that must be handled with increasing complexity by the processing hardware installed on the unmanned vehicle payload. Compounding the problem are the ever-present pressures on space, weight, and power (SWaP) with the desire for longer endurance units with greater functionality and lower fuel consumption.

The criticality and computational complexity presented by video data in an unmanned vehicle environment is therefore significant. Augmenting these challenges is the increasing need for flexible and accurate access to video data due to the migration of situational awareness techniques toward net-centric warfare. In this scenario, no longer is the UAV video data only transferred to one ground crew location responsible for controlling the mission.

Now, instead of one data source and one destination (as shown in the figure at right) there may be numerous “sources” as the number and types of image sensors increase, and many “destinations” as many end users request access to the overhead video sensors. Examples of destinations may include central mission control, mobile mission control, mobile ground forces (with handheld devices), or other manned air vehicles. For data that is acquired using UAV missions, the intended uses can span the range of the command, control, communications, computers, intelligence, surveillance, target acquisition, and reconnaissance (C4ISTAR) applications.

The destinations will need to view the data at different resolutions and with varying degrees of criticality. For instance, the pilot and crew of the UAV mission will have the most critical level of access, while an alternate ground destination or other air unit may only need data access for information gathering, not for mission safety. Such a broad range of technical criteria and data use possibilities provides significant challenges for video acquisition, compression, and transmission technologies.

This diagram shows a typical scenario with a target, UAV, and ground station configuration.
Click here to enlarge image

The next level of complication in the UAV video-handling problem is the data link available to a UAV, which usually includes such options as line-of-sight radio link or satellite link. During the course of one mission, the bandwidth available for communication between the UAV and ground crew can vary significantly and unpredictably for many reasons.

As a UAV moves from line-of-sight data links to satellite data links, significant reduction in bandwidth occurs. When operating under satellite data link conditions, environmental or physical obstacles can also influence bandwidth available for data communication. A protocol which allows for automated quality adjustment under limited bandwidth conditions is greatly beneficial. Additionally, capability to remotely control bandwidth requirements or adjust video processing parameters to manage varying bandwidth conditions, enables basic functionality to be maintained under non-circumstances.

It is evident that on UAV platforms, demands on the data handling technology are increased in several dimensions including:

    1) the increasing quantity of sensors,
    2) the increasing quality of sensors (higher resolution),
    3) the likelihood of a limited-bandwidth data link,
    4) the need for several receivers (or consumers) to view the same data, and
    5) the variability in bandwidth over the course of a mission. The complexity of the video handling and compression systems must also be balanced with the limitations of equipment space, weight, and power.

In summary, situational awareness with UAV platforms impose the following set of requirements on the video data handling system:

    1) Rugged products that meet the demanding UAV environment
    2) Efficient and configurable data handling (real-time parameter control)
    3) Several input video format support (RS170, RGB, SMPTE-294M, and so on)
    4) Efficient support for several resolutions
    5) Maximum compression at excellent video quality
    6) Minimized overall latency

The Codec to meet the need

To meet these demanding compression requirements, advances in video compression codecs are being implemented in rugged products. A suitable product must meet the extended temperature and conduction-cooled demands of the military and aerospace industry with performance and flexibility. A video compression codec, such as ITU-T H.263, H.264 (also known as MPEG-4 part 10, or AVC), or JPEG2000 is typically used. These algorithms are widely deployed, mature, and well supported on many platforms. JPEG2000, however, is less appropriate for UAV applications because it consumes more bandwidth than H.263 or H.264.

ITU-T H.263 and H.264 provide various methods to optimize bandwidth use and video quality in a given context. As in all compression algorithms, the goal is to remove redundant information while leaving all salient details untouched. If this goal is achieved then the algorithm is considered “lossless.” In practice, truly lossless compression techniques cannot compress data beyond 3 or 4 times the original image size, under best case conditions, and are frequently impractical in a real-world environment, particularly under non-data-link conditions.

The large majority of applications endure a certain “lossy” nature of the video codec to improve compression ratios to the 20:1 to 100:1 range. This is where full frame rate and full frame size video compression becomes practical for UAV applications. In order to achieve the necessary levels of data compression, it becomes imperative that a lossy algorithm is employed, but with a view to maintaining the maximum possible quality.

The ITU-T H.263 and H.264 codecs provide mechanisms to improve quality or improve compression as the bandwidth available is respectively increased or decreased. It is this flexibility that has resulted in broad acceptance in commercial and consumer applications, and which makes them uniquely suited to demanding UAV applications. One mechanism that is improving bandwidth use is that H.263 and H.264 can encode only regions of a frame that are notably different from a previous frame, and provide a movement vector for regions that have moved within the frame.

These “difference” frames are much smaller than frames which encode data for an entire frame every time, such as is the case with, for example, JPEG2000 codecs. This has the effect of greatly increasing compression without significant loss of quality, since all the pixels of a given frame do not need to be retransmitted from source to destination. Due to the nature of the JPEG2000 codec frame compression algorithm, reductions in bandwidth availability can severely limit the frame rate that can be achieved. For an equivalent low-bandwidth link, H.263 or H.264 compression provides quality.

An example of where the difference frame capability provides massive compression improvement is a UAV transmitting images of gradually changing landscapes where the differences between each frame are minimal, and most of it can be remapped in a frame with a movement vector from the previous frame. Lastly, the H.263 and H.264 codecs provide complete control of the frequency of sending integrated frames, where no previous information is required. The frequency of this “resync” frame can be adjusted from occurring every single frame, in which case no history is ever needed, to over one hundred frames. This level of flexibility of the codec makes it possible to adjust parameters to optimize for a specific system implementation or even “on-the-fly” as conditions change, or target characteristics change.

GE Fanuc Intelligent Platforms has deployed the TS-VID-MP4 product line into many manned and unmanned air vehicle applications which require video compression and transmission. The TS-VID-MP4, for example, implements the ITU-T H.263 codec, can capture up to four RS-170 video input streams, and can provide simultaneous compression for two of them in real time with extremely low latency–as low as three frames of latency (less than 100ms), depending on the data-link technology employed. The TS-VID-MP4 is available in a conduction-cooled, low-power, PMC package, including a complete client/server software solution for quick expansion of any UAV system.

Several consumers with one producer

To further increase system efficiency, an extension of the ITU-T H.264 algorithm, called the H.264 Scalable Video Codec (H.264 SVC), provides mechanisms for one encoded bitstream to be captured by numerous receivers at varying frame resolution and frame refresh rates. The control is handled at the receiver end such that the same video compression server can supply one Ethernet packet stream video at simultaneously high resolution and full frame rates and low resolution or reduced frame rates. This feature is extremely useful in the situational awareness context described earlier where several receivers require access, with varying data-link speeds and the ability to handle the decoding task complexity.

Tim Klassen is video products team leader at GE Fanuc Intelligent Platforms, an embedded computing specialist in Charlottesville, Va.

Military & Aerospace Photos

Most Popular Articles

Related Products

CHAMP-FX4 6U OpenVPX Virtex-7 FPGA Processor Card

The CHAMP-FX4 is the flagship 6U product in Curtiss-Wright Defense Solutions’ family of user-prog...

PMC-E2001 Audio/Acoustic Waveform Generating PMC Card

PMC-E2001 Delta-sigma PMC features 16-bit resolution with 8 Analog Outputs and 4 Analog Inputs wi...

VPX3-453 3U VPX Virtex-6/8640D Digital Signal Processor

The Curtiss-Wright VPX3-453 is a high performance 3U VPX DSP and FPGA processor card that combine...

General Micro "Horizon" C299

The C299 Horizon is a third generation, 6U cPCI SBC module based on GMS’ upgradable CPU technolog...

Rugged Mobile Communications Server

Advanced communications server designed to be deployed in environments where it needs to meet cer...

API DC Link Power Film Capacitors

High reliability DC link capacitors for power inverter applications which require superior life e...

UAS, UAV and Ground Based Robot Antennas

Cobham Antenna Systems have designed and developed a range of antennas for use on UAV, UAS and gr...

Low size, weight, and power (low SWaP) stabilized imaging systems for small manned and unmanned systems Alticam 09MWIR2

Designed for small unmanned aircraft systems. Gyro-stabilized gimbal system featuring mid-wave in...

Electronic Warfare VPX Products

The VPX product line includes three products: Modular Quad IF Up and Down Converter, Modular Dire...

Interpoint™ MTR Series™ 50 Volt

The MTR Series™ MTR 50, 28 volt dc-dc converter offers up to 30 watts of output power from single...

Related Companies

Curtiss-Wright Defense Solutions

About Curtiss-Wright Defense Solutions Curtiss-Wright Defense Solutions (CWDS) is a long established techno...

General Micro Systems Inc

Since 1979, General Micro Systems has been providing the most diverse line of single-board computers in the industry....

Elma Electronic Inc

Who we are...   About Elma Electronic Systems   The Systems division of Elma Electronic Inc. supplies the

API Technologies Corp

Who We Are API Technologies is a dominant technology provider of RF/microwave, microelectronics, and security technol...

Extreme Engineering Solutions Inc (X-ES)

 Extreme Engineering Solutions, Inc. (X-ES) is a leader in the design, manufacture, and support of standard and ...

Crane Aerospace & Electronics

When failure is NOT an option...rely on Crane Aerospace & Electronics. We supply high-density, high-reliability c...

Delta Digital Video

Designs and manufactures video compression and scan conversion products for aerospace applications. Develops standard...

GE Intelligent Platforms

Provides embedded computing solutions. Products include single-board computers, networking products, avionics interfa...

Advanced Microwave Products

Designs and manufactures microwave transmitters and receivers for military, UVS, UAV, UGV, and surveillance applicati...


UAVDirect is a provider of unmanned aerial vehicles specifically designed for aerial photography, aerial cinematograp...
Wire News provided by   

Press Releases

Low Viscosity, One Part Cyanoacrylate Is Non-Toxic and Meets ISO 10993-5 Specifications

Master Bond MB250NT is widely used for a variety of applications ranging from repair to high speed producti...

Thermally Conductive, Two Component Epoxy Passes USP Class VI Tests and ISO 10993-5 Specifications

With biocompatibility and cytotoxicity certifications, Master Bond EP21AOLV-2Med is often selected for bond...

One Component, Snap Cure Epoxy Features High Strength Properties

Suitable for a variety of applications in the electronic, aerospace and OEM industries, Master Bond EP3SP5F...

One Part Epoxy Resists up to 500°F and Meets NASA Low Outgassing Specifications

Master Bond Supreme 12AOHT-LO is a one component epoxy for a variety of bonding and sealing applications in...

VICTORY Shared Processing, Fire Control Computer, and Switch for Ground Vehicles Introduced by Curtiss-Wright

Curtiss-Wright Corporation today announced that its Defense Solutions division has introduced a new fully i...


Curtiss-Wright Corporation’s Defense Solutions division applauds Northrop Grumman Corporation (NYSE: NOC) o...

Curtiss-Wright’s New Rugged Mobile IP Router Subsystem Features an Integrated Cisco® 5915 ESR Router

Curtiss-Wright Corporation today announced that its Defense Solutions division, a Cisco® Systems Solution T...

GE Announces First Sub-Credit Card-Sized Multi-Function High Definition (HD) Video Tracker

HUNTSVILLE, AL.— OCTOBER 13, 2014—GE’s Intelligent Platforms business today announced at AUSA (October 13-...


Meeting the Gen3 backplane challenge with OpenVPX and COTS

Tight Pentagon budgets mean military systems must stay in the field for longer than ever before. This doesn't mean obsolete technology, however. Today's military electronics are being upgraded constantly, an...
Sponsored by:

Design Strategy Considerations for DO-178C Certified Multi-core Systems

Join Wind River to learn how system architecture and design choices can minimize your DO-178C certification challenges.

Sponsored by:

Flying, Sailing or Driving - The Rugged, Embedded Intel-based Server that goes where you need it!Flying Sailing or Driving

Leveraging the power of server-class processors is no longer relegated to the confines of data centers. Through several innovations, Mercury Systems has ruggedized Intel’s server-class chips for deployment. ...
Sponsored by:

All Access Sponsors

Mil & Aero Magazine

April 2015
Volume 26, Issue 4

Download Our Apps




Follow Us On...


Military & Aerospace Electronics

Weekly newsletter covering technical content, breaking news and product information

Cyber Security

Monthly newsletter covering cyber warfare, cyber security, information warfare, and information security technologies, products, contracts, and procurement opportunities

Defense Executive

Monthly newsletter covering business news and strategic insights for executive managers

Electronic Warfare

Quarterly newsletter covering technologies and applications in electronic warfare, cyber warfare, optical warfare, and spectrum warfare.

Embedded Computing Report

Monthly newsletter covering news on embedded computing in aerospace, defense and industrial-rugged applications

Unmanned Vehicles

Monthly newsletter covering news updates for designers of unmanned vehicles