The UAV video problem: using streaming video with unmanned aerial vehicles

By Tim Klassen

Unmanned aerial vehicles (UAVs) present different video handling requirements to traditional manned vehicles. These demands can be summarized as:

    1) the number of video streams handled;
    2) the criticality of the video data link to the success of the mission;
    3) even greater space/weight/power limitations;
    4) potentially more demands on distribution, such as several remote data consumers; and
    5) varying and, at times, restrictive limitations on data-link bandwidth.

When it comes to adding video data streams in manned vehicles, normally only intra-vehicle distribution is required. For most of the video display need, this can usually be handled by adding wired connections for video sources as necessary and routing physical connections to display destinations. Video transmission to a ground station is either not needed at all or is implemented as an add-on that is non-critical to the safety or functionality of the vehicle. When additional video streams are installed in an unmanned vehicle, the data has to be encapsulated and transmitted to a ground station using a relatively low-bandwidth data link. This adds increasing pressure on the video capture, compression, and transmission solution implemented on the aircraft.


Reconnaissance UAVs like HERTI from BAE Systems use H.263/H.264 codecs for video transmission.
Click here to enlarge image

Additionally, in unmanned air vehicles, the criticality level of the video data is entirely different. For unmanned systems, video data constitutes the only “eyes-in-the-sky” and thus the only link that a ground crew may have with what is actually going on in the vicinity of an unmanned air vehicle.

As it is not unusual for the unmanned air vehicle to be “flown” by a ground crew hundreds or even thousands of kilometers away, the video data link becomes absolutely critical to the safety and success of the mission. The figure on the following page shows a typical scenario with a target, UAV, and ground station configuration. For this arrangement to function safely and effectively, the quality, reliability, and latency of the video stream must be optimized. Also, in years the number and resolution of video sensors mounted on the unmanned vehicle have continued to increase, while the demands on quality, reliability, and latency remain unchanged.

With the addition of weaponized unmanned platforms, the need for more information and, in particular, more video information, has continued to multiply. It has resulted in a massive increase in the amount of video data that must be handled with increasing complexity by the processing hardware installed on the unmanned vehicle payload. Compounding the problem are the ever-present pressures on space, weight, and power (SWaP) with the desire for longer endurance units with greater functionality and lower fuel consumption.

The criticality and computational complexity presented by video data in an unmanned vehicle environment is therefore significant. Augmenting these challenges is the increasing need for flexible and accurate access to video data due to the migration of situational awareness techniques toward net-centric warfare. In this scenario, no longer is the UAV video data only transferred to one ground crew location responsible for controlling the mission.

Now, instead of one data source and one destination (as shown in the figure at right) there may be numerous “sources” as the number and types of image sensors increase, and many “destinations” as many end users request access to the overhead video sensors. Examples of destinations may include central mission control, mobile mission control, mobile ground forces (with handheld devices), or other manned air vehicles. For data that is acquired using UAV missions, the intended uses can span the range of the command, control, communications, computers, intelligence, surveillance, target acquisition, and reconnaissance (C4ISTAR) applications.

The destinations will need to view the data at different resolutions and with varying degrees of criticality. For instance, the pilot and crew of the UAV mission will have the most critical level of access, while an alternate ground destination or other air unit may only need data access for information gathering, not for mission safety. Such a broad range of technical criteria and data use possibilities provides significant challenges for video acquisition, compression, and transmission technologies.


This diagram shows a typical scenario with a target, UAV, and ground station configuration.
Click here to enlarge image

The next level of complication in the UAV video-handling problem is the data link available to a UAV, which usually includes such options as line-of-sight radio link or satellite link. During the course of one mission, the bandwidth available for communication between the UAV and ground crew can vary significantly and unpredictably for many reasons.

As a UAV moves from line-of-sight data links to satellite data links, significant reduction in bandwidth occurs. When operating under satellite data link conditions, environmental or physical obstacles can also influence bandwidth available for data communication. A protocol which allows for automated quality adjustment under limited bandwidth conditions is greatly beneficial. Additionally, capability to remotely control bandwidth requirements or adjust video processing parameters to manage varying bandwidth conditions, enables basic functionality to be maintained under non-circumstances.

It is evident that on UAV platforms, demands on the data handling technology are increased in several dimensions including:

    1) the increasing quantity of sensors,
    2) the increasing quality of sensors (higher resolution),
    3) the likelihood of a limited-bandwidth data link,
    4) the need for several receivers (or consumers) to view the same data, and
    5) the variability in bandwidth over the course of a mission. The complexity of the video handling and compression systems must also be balanced with the limitations of equipment space, weight, and power.

In summary, situational awareness with UAV platforms impose the following set of requirements on the video data handling system:

    1) Rugged products that meet the demanding UAV environment
    2) Efficient and configurable data handling (real-time parameter control)
    3) Several input video format support (RS170, RGB, SMPTE-294M, and so on)
    4) Efficient support for several resolutions
    5) Maximum compression at excellent video quality
    6) Minimized overall latency

The Codec to meet the need

To meet these demanding compression requirements, advances in video compression codecs are being implemented in rugged products. A suitable product must meet the extended temperature and conduction-cooled demands of the military and aerospace industry with performance and flexibility. A video compression codec, such as ITU-T H.263, H.264 (also known as MPEG-4 part 10, or AVC), or JPEG2000 is typically used. These algorithms are widely deployed, mature, and well supported on many platforms. JPEG2000, however, is less appropriate for UAV applications because it consumes more bandwidth than H.263 or H.264.

ITU-T H.263 and H.264 provide various methods to optimize bandwidth use and video quality in a given context. As in all compression algorithms, the goal is to remove redundant information while leaving all salient details untouched. If this goal is achieved then the algorithm is considered “lossless.” In practice, truly lossless compression techniques cannot compress data beyond 3 or 4 times the original image size, under best case conditions, and are frequently impractical in a real-world environment, particularly under non-data-link conditions.

The large majority of applications endure a certain “lossy” nature of the video codec to improve compression ratios to the 20:1 to 100:1 range. This is where full frame rate and full frame size video compression becomes practical for UAV applications. In order to achieve the necessary levels of data compression, it becomes imperative that a lossy algorithm is employed, but with a view to maintaining the maximum possible quality.

The ITU-T H.263 and H.264 codecs provide mechanisms to improve quality or improve compression as the bandwidth available is respectively increased or decreased. It is this flexibility that has resulted in broad acceptance in commercial and consumer applications, and which makes them uniquely suited to demanding UAV applications. One mechanism that is improving bandwidth use is that H.263 and H.264 can encode only regions of a frame that are notably different from a previous frame, and provide a movement vector for regions that have moved within the frame.

These “difference” frames are much smaller than frames which encode data for an entire frame every time, such as is the case with, for example, JPEG2000 codecs. This has the effect of greatly increasing compression without significant loss of quality, since all the pixels of a given frame do not need to be retransmitted from source to destination. Due to the nature of the JPEG2000 codec frame compression algorithm, reductions in bandwidth availability can severely limit the frame rate that can be achieved. For an equivalent low-bandwidth link, H.263 or H.264 compression provides quality.

An example of where the difference frame capability provides massive compression improvement is a UAV transmitting images of gradually changing landscapes where the differences between each frame are minimal, and most of it can be remapped in a frame with a movement vector from the previous frame. Lastly, the H.263 and H.264 codecs provide complete control of the frequency of sending integrated frames, where no previous information is required. The frequency of this “resync” frame can be adjusted from occurring every single frame, in which case no history is ever needed, to over one hundred frames. This level of flexibility of the codec makes it possible to adjust parameters to optimize for a specific system implementation or even “on-the-fly” as conditions change, or target characteristics change.

GE Fanuc Intelligent Platforms has deployed the TS-VID-MP4 product line into many manned and unmanned air vehicle applications which require video compression and transmission. The TS-VID-MP4, for example, implements the ITU-T H.263 codec, can capture up to four RS-170 video input streams, and can provide simultaneous compression for two of them in real time with extremely low latency–as low as three frames of latency (less than 100ms), depending on the data-link technology employed. The TS-VID-MP4 is available in a conduction-cooled, low-power, PMC package, including a complete client/server software solution for quick expansion of any UAV system.

Several consumers with one producer

To further increase system efficiency, an extension of the ITU-T H.264 algorithm, called the H.264 Scalable Video Codec (H.264 SVC), provides mechanisms for one encoded bitstream to be captured by numerous receivers at varying frame resolution and frame refresh rates. The control is handled at the receiver end such that the same video compression server can supply one Ethernet packet stream video at simultaneously high resolution and full frame rates and low resolution or reduced frame rates. This feature is extremely useful in the situational awareness context described earlier where several receivers require access, with varying data-link speeds and the ability to handle the decoding task complexity.


Tim Klassen is video products team leader at GE Fanuc Intelligent Platforms, an embedded computing specialist in Charlottesville, Va.


Get All the Military Aerospace Electronics News Delivered to Your Inbox or Your Mailbox

Subscribe to Military Aerospace Electronics Magazine or email newsletter today at no cost and receive the latest information on:


  • C4ISR
  • Cyber Security
  • Embedded Computing
  • Unmanned Vehicles


Get All the Military Aerospace Electronics News Delivered to Your Inbox or Your Mailbox

Subscribe to Military Aerospace Electronics Magazine or email newsletter today at no cost and receive the latest information on:


  • C4ISR
  • Cyber Security
  • Embedded Computing
  • Unmanned Vehicles

Military & Aerospace Photos

Most Popular Articles

Related Products

XCalibur5090 | Dual Virtex-7 Based Digital Signal Processing 6U LRM FPGA with Quad 2500 MSPS DAC and 3200 MSPS ADC

The XCalibur5090 is a high-performance, reconfigurable, conduction-cooled 6U LRM module based on ...

XPedite7574 | 5th Generation Intel® Core™ i7 Broadwell-H Processor-Based Conduction- or Air-Cooled 3U VPX-REDI Module

The XPedite7574 is a high-performance, 3U VPX-REDI, single board computer based on the 5th genera...

XCalibur1645 | Freescale Eight-Core P4080 Processor-Based Conduction-Cooled 6U VPX Module

The XCalibur1645 is a high-performance, 6U VPX, single board computer supporting Freescale QorIQ ...

Ultra-Flex Omni Antennas

The Southwest Antennas Ultra-Flex is a line of S-Band & C-Band omni antennas with a unique enviro...

4G LTE Cellular Omni Concealment Antenna

The 4G LTE Cellular Omni Concealment Antenna from Southwest Antennas measures only 4.70" x 1.70" ...

S & C Dual-Band Omni Antennas

SWA Dual-Band Omni Antennas are designed to cover the S & C bands, and are available with multipl...

DNA-1553-553

Dual Channel MIL-STD-1553 interface board for Cube I/O chassis with 2 independent, dual redundant...

PowerDNR HalfRACK

Half-width 3U Gigabit Ethernet Data Acquisition and control chassis allows the installation of up...

FLATRACK™ Ethernet I/O DAQ Platform

FLATRACK™ Ethernet I/O DAQ Platform accommodates installation of up to four I/O boards, completes...

PowerDNA-MIL

Four-Slot, Military Style I/O Chassis with Military/Rugged 38999 connectivity, 100% COTS solution...

Related Companies

TEN TECH LLC

Is a mechanical engineering consulting company headquartered in Los Angeles, CA with operations in Billerica, MA, pro...

Lexycom Technologies Inc

Manufactures lightweight, miniature software defined radios (SDR) for data, video, and voice. Unlike legacy radios, S...

Curtiss-Wright Defense Solutions

About Curtiss-Wright Defense Solutions Curtiss-Wright Defense Solutions (CWDS) is a long established techno...

MiltechUK

Mil Spec EMC/NEMP/ filters and EMC product solutions for military applications.

PALMARII Dynamics AB

PALMARII Dynamics is a Swedish company incorporated in 2012 as a competence centre for specialist naval architecture ...

PELORUS NAVAL SYSTEMS Inc

Pelorus Naval Systems is a specialist naval defense engineering and support services company with headquarters in Ran...

ISVI Corp

ISVI designs, produces and sells machine-vision camera systems combining high-speed and high-resolution sensors with ...

Southwest Antennas

Southwest Antennas designs and manufactures high-performance RF & Microwave antennas and accessories designed for tod...

CORWIL Technology Corp

CORWIL Technology, the premier US based, IC assembly and test services subcontractor, offering full back-end assembly...

United Electronic Industries Inc

UEI is a leader in the PC/Ethernet data acquisition and control, Data Logger/Recorder and Programmable Automation Con...

Webcasts

Harsh Environment Protection for Advanced Electronics and Components

This webinar will offer an opportunity to learn more about ultra-thin Parylene conformal coatings – how they are applied, applications they protect today, and the properties and benefits they offer, includin...

New Design Tools That Help You Develop Radar That Sees the Un-seeable and Detects the Undetectable

Xilinx EW/ISR System Architect, Luke Miller, has new tricks and he’s going to tell you all about them in a new Xilinx Webinar—for free. His Webinar will cover new ways to implement Radar functions including ...
Sponsored by:

Press Releases

CURTISS-WRIGHT AWARDED CONTRACT BY SIERRA NEVADA CORPORATION

Curtiss-Wright Corporation today announced that its Defense Solutions division has received a contract from Sierra Nevada Corporation (SNC) to supply its small form factor ...

Innovative Integration Announces the FMC-Servo

Camarillo, CA June 19, 2015, Innovative Integration, a trusted supplier of signal processing and data acquisition hardware and software solutions, today announced the FMC-S...

METADATA SUPPORT ENHANCES INDUSTRY-LEADING HD VIDEO MANAGEMENT SYSTEM FROM CURTISS-WRIGHT

Curtiss-Wright Corporation today announced that its Defense Solutions division has further enhanced its innovative VRD1 high definition (HD) video management system (VMS) w...

All Access Sponsors


Mil & Aero Magazine

August 2015
Volume 26, Issue 8
file

Download Our Apps



iPhone

iPad

Android

Follow Us On...



Newsletters

Military & Aerospace Electronics

Weekly newsletter covering technical content, breaking news and product information
SUBSCRIBE

Cyber Security

Monthly newsletter covering cyber warfare, cyber security, information warfare, and information security technologies, products, contracts, and procurement opportunities
SUBSCRIBE

Defense Executive

Monthly newsletter covering business news and strategic insights for executive managers
SUBSCRIBE

Electronic Warfare

Quarterly newsletter covering technologies and applications in electronic warfare, cyber warfare, optical warfare, and spectrum warfare.
SUBSCRIBE

Embedded Computing Report

Monthly newsletter covering news on embedded computing in aerospace, defense and industrial-rugged applications
SUBSCRIBE

Unmanned Vehicles

Monthly newsletter covering news updates for designers of unmanned vehicles
SUBSCRIBE