Although the entertainment industry has been conducting the lion`s share of research in graphics and imaging over the past several years, military officials find that they still must take the lead in simulation development in several key high-end areas
By J.R. Wilson
Computer-aided simulation over the past two decades has developed into a mainstay of military training. During that time it has moved from a largely aviation-oriented application to land and sea weapons systems and even down to the individual soldier and Marine. More recently it has begun to evolve into a key element of mission planning and rehearsal.
The key to how far simulation can go in training and mission rehearsal has always been the capability of graphics software and hardware. When even modestly acceptable resolution requires large expensive image generators (IGs), it was common to hear military commanders dismiss high fidelity and visual acuity as unnecessary luxuries. They insisted that trainees would accept green cones as trees and gray squares as vehicles.
On the eve of the 21st Century, however, Hollywood - not the Pentagon - is defining the level of acceptable reality in computer simulation in movie special effects and computer games. As Hollywood pays the bill to push the envelope of this technology, military officials have adopted a new attitude toward realism. Leaders have found more than enough evidence that the more simulation mimics the real world, the better it serves training - particularly mission rehearsal.
Contrary to this trend, however, the military requirement is now exceeding the limits to which the entertainment industry will go in evolving simulation. This means the 1990s dedication to commercial off-the-shelf (COTS) has hit a wall beyond which the military itself must take responsibility as driver and patron of future technological advances.
"We`re almost at a crossroads," says Pam Woodard, visual systems engineer at the U.S. Army Simulation, Training & Instrumentation Command (STRICOM) in Orlando, Fla. "The military has always taken the lead in developing that kind of technical capability," she continues. "In a couple of more years, we`ll probably start parsing PC-based systems into our training systems. Right now those are capable of doing a lot of polygon visuals, but not the high-end capabilities we need for a lot of our training."
Military officials have a lot of work to do in developing advanced simulation capability, not only to increase fidelity, but also to make these high-end systems more affordable, Woodard says. "We are the only ones paying for the advancements in the high-end systems, where the market is primarily military. But as our budgets go down, it is more difficult for us to be the only ones pushing the state of the art in that area. So we can see in the future the possibility of much lower-cost systems to perform the kinds of things we need to do."
Devil in the details
Today`s advanced simulators feature various degrees of image fidelity. While the simulator`s terrain database may not need to produce precise detail for training aircraft pilots flying high-speed jets at high altitudes, detail grows in importance for slow, low aviation requirements such as helicopters. In fact, the high level of detail necessary for realistic training and mission rehearsal at ground level - be it the deck of an aircraft carrier, the view out a tank hatch, or the walk-through world of the foot soldier - is not yet perfected.
Ideally, what military leaders want is a terrain database with sufficient information to yield credibly realistic simulation detail for an aircraft pilot flying at 25,000 feet or to a tank commander operating on the ground. "If they come at different levels, then there will be a discontinuity," says U.S. Air Force Col. Kenneth Konwin, director of the Defense Modeling & Simulation Office (DMSO) in Alexandria, Va.
In an effort to reduce overall costs in representing the same environment for different combat systems, STRICOM has advised military program managers to use the Synthetic Environment Data Representation and Interchange Specification (SEDRIS), Konwin says.
The ultimate goal of efforts such as SEDRIS is to network simulators for aircraft, ground vehicles, and infantry. The SEDRIS approach seeks to use a database that starts at the same point, which will enable every player to look at the same road in the same place at the same time and a path to exchange the databases.
"To develop a data model that describes all the data we use in building synthetic environments has been the goal," explains STRICOM`s Woodard. "So when that database is built it can be put into a SEDRIS transmittal for someone else to use it or for a variety of systems to use it in a networked exercise. It doesn`t solve intero-perability, but it is a necessary first step to reaching interoperability," Woodard says.
It is that high-end element that defines the dividing line between what simulation developers can and cannot reasonably evolve for military applications from commercial systems (see sidebar page 16).
This new requirement for greater realism is bringing about a new convergence among previously disparate elements of the military. As the imagery improves for ground forces, that new fidelity will be available to aviation - and vice versa.
"One technology we will be pushing is high fidelity visual imagery for air-to-ground," says DMSO`s Konwin. "While you can see the beginnings of representational fly-through scenes at the speeds of fighter airplanes, we still have some challenges in high-definition multispectral images that are clear to the eye of the pilot." Konwin says.
DMSO is the Department of Defense focal point for modeling and simulation; it is the body that articulates, proposes, and staffs for policy across DOD as well as being the steward for the defense modeling and simulation master plan. In that capacity, DMSO officials not only work with the services and other agencies to define the path forward in simulation, but also even push, pay for, and field new technologies.
"We don`t have, within the services and DOD, all the resources needed to attack all the problems that are out there," cautions U.S. Navy Capt. Stan O`Connor, DMSO`s deputy director. "That`s why we have to be patient with the commercial marketplace to bring those things forward and then find out how to exploit them."
Mission rehearsal
While increased realism is a major goal of all training, it is of supreme importance for mission rehearsal, where simulation offers the opportunity to increase the likelihood of mission success and the survival of participants. But such missions rarely involve areas where contemporary databases exist. After all, in a combat theater even an image a few days old may provide dangerously outdated.
"The key to mission rehearsal is getting the terrain database you need," says Col. Louis Gelling Jr., systems manager for the Combined Arms Tactical Trainer, a U.S. Army Training and Doctrine Command (TRADOC) program based at Fort Leavenworth, Kan. "It`s not easy to develop terrain databases you drive over. When you start driving on terrain, there are seven different layers they have to build on - the basic digital data, the buildings, roads, streams and so on."
The challenges, Gelling says, are many. "We`re talking about doing things no one has done before. You take a shot of a piece of ground through a satellite, work that into some kind of digital terrain database, and then make it accessible for someone in a simulator to drive on. That`s not easy."
The need to provide a common virtual world in which a variety of simulators can interact first arose with the introduction of SIMNET, a Defense Advanced Research Projects Agency (DARPA) program that networked simulators produced by the same vendor. In the late 1980s, experts extended this concept to network simulators of differing manufacture, fidelity, and function using a new set of Distributed Interactive Simulation (DIS) protocols.
Common protocols
The U.S. Army has led the services in the integration of simulators and the use of DIS to perform joint training exercises using simulation and live systems. But a DIS exercise remains a complex and time-consuming effort, sometimes requiring weeks of preparation. Reducing that timeline is the goal of a U.S. Air Force effort called Distributed Mission Training (DMT), which is considered the next generation of interactive simulation training for the military.
Under the initial contract, contractors will deliver eight simulators and their support equipment to the Air Force`s Air Combat Command - four to Langley Air Force Base, Va., and four to Eglin Air Force Base, Fla. The Eglin units are to be operational by March 1999 and those at Langley are to be operational by June 1999.
The simulators at the two sites will be interoperable. Furthermore, they will have limited connectivity to an Airborne Warning and Control System aircraft trainer at Tinker Air Force Base, Okla., which is to go on line with Eglin. Options call for 12 additional F-15 jet fighter training sites with a total of 46 crewstation trainers worldwide - a 15-year potential $333 million effort. Prime contractor for the F-15C DMT is Boeing Training & Support Systems in St. Louis.
"Typically it takes months of time and engineering effort to set up an integrated multi-location simulation. But now the Air Force will be able to sit down and do all that at the push of a key without a lot of up-front work every time they want to run the scenario," says Steve Swaine, a Boeing research program manager.
"The Air Force vision is that this will grow into an Air Force-wide training environment," adds Darrell Smith, Boeing`s lead project engineer for the F-15C DMT program. "We`re also executing a C-5 [cargo jet] distributed training contract and a number of others are being competed. The idea is that as time goes by, in short order, there will be multiple platforms available to this distributed environment."
The Air Force is putting an operations and integration contract into place to pull a large exercise together in 24 hours. Device providers such as Boeing are not allowed to compete for the integration contract.
DMT also is innovative in terms of its acquisition strategy. Rather than selling them to the Air Force, Boeing and contractors on future DMT devices will own, maintain, and operate them and the Air Force will pay for usage only. A basic requirement of the entire contract is full compatibility among all devices in the distributed mission environment. Part of the service fee goes to customer-directed technology insertion. This will enable the Air Force to direct industry to pursue new technologies as they evolve and incorporate them into the simulators as warranted.
"We are not sitting back and letting this develop on its own," Swaine says. "We are developing technology not just for the Air Force, but also for our Navy, Army, and Marine Corps contracts. All of the computational processing elements, threat stations, debrief, etc., are being developed generically so they can be applied to all of those distributed environments." If Air Force leaders move to full production of the F-15C DMT, Boeing officials say they hope to receive export permission for a potentially lucrative foreign market.
COTS simulation
The heart of the Boeing DMT is its visual integrated display system (VIDS), a mature commercial device that enables the simulator to operate in an office environment. It is modular, makes extensive use of COTS technology, and is designed to be field maintainable, with all elements easily accessible.
"We`ve thus given the Air Force a lot of technological capability, but also the ability to buy lots of systems without lots of new buildings," notes Darrell Smith, Boeing`s lead project engineer for the F-15C DMT program. "So they can now afford to build the team training facilities they need to train at the squadron level."
Air Force pilots were brought in during several visual evaluations on a variety of F-15 training scenarios. Based on pilot comments after each such session, the engineers refined the technology prior to fielding.
The VIDS itself is a metal frame structure with a set of flat-panel displays. It comes in two sizes capable of handling a variety of aircraft cockpit types. The smaller version (which has been installed aboard an aircraft carrier) has four flat- panel displays, each about 40 inches - one directly in front of the pilot, one on each side, and one overhead. These displays provide a 216-degree horizontal field of view.
The F-15C DMT uses the larger system, which is 7.5 feet high, 24 feet long, and 16 feet wide. The pilot sits in a closed "clamshell," where seven screens surround him in a full field of view. Driving the screens, which are about 20 inches from the pilot, are a red-green-blue background projector, a high-resolution projector, and a second projector driving a head-up display.
The Boeing F-15C DMT uses an ESIG image generator from Evans & Sutherland in Salt Lake City. On a separate contract for the U.S. Navy F/A-18 fighter-bomber, Boeing engineers are using image generators from Silicon Graphics Inc. of Mountain View, Calif.
"You can think of VIDS as the interface monitor to the image generator, Says Boeing`s Smith. "It has a common interface across the board, regardless of the IG. As DMT goes into the air-to-ground training realm, a whole new series of problems arise. So our next generation of IG will address the ability to pick targets out of ground clutter. The visual acuity becomes very critical and is much tougher than the air-to-air solution."
There are four computer-controlled threat servers and four more that are low-fidelity real-time trainers running on PCs. Each of those has databases associated with them. The F-15C DMT also includes brief and debrief stations and a mission observation center - a large-screen theater for viewing what is going on inside the simulation.
"In addition to the visual systems for the pilot, we have correlated visuals and databases throughout the system so all the other players are participating in a fair fight," says Galen Stanley, Boeing`s F-15C DMT program manager. "For example, the threat stations also run a visual scene that is correlated to what the pilot is seeing. The operator station also can run a very good visual of what the pilot is seeing. One of the most significant challenges has been to maintain a consistent set of databases to have a common look to the scenario for all the players."
The computer is a COTS distributed architecture of Motorola PowerPC-based single-board computers and the VxWorks real-time operating system from Wind River Systems in Alameda, Calif. Handling voice communications is a network from ASTi of Herndon, Va.
Another program pushing the same elements of cost, size, and visual technology is the Naval Air Warfare Center`s Transportable Strike Assault/Rehearsal System (TSTARS), on which Evans & Sutherland is prime contractor. Intended to be the first "simulator in a box," the entire system will fit into an 8-by-8-by-20-foot ISO transport container. Reconfigurable cockpits are in a 43-inch-diameter, 360-degree mini-dome.
Designed for training and mission rehearsal, TSTARS will enable pilots to build a simulated mission using the same tools they use aboard ship to input data for a real flight - the Tactical Air Mission Planning System (TAMPS). Once the pilot is satisfied with the mission as planned in the simulator, all waypoints and other TAMPS-created information transfer directly to his aircraft for the actual mission. Upon returning, technicians can download the actual flight data into the simulator for debriefing.
"This is a big step forward for Navy simulation in a number of areas," says Bob Brantley, program manager at Evans & Sutherland. "It puts a mission rehearsal training system on a carrier where the pilots can rehearse their missions and then take off. They will have at their fingertips real world data, real databases based on current satellite data, and up-to-date intelligence and weather data."
The host simulator is running on WindowsNT, and the graphics are based on the open GL commercial standard. TSTARS uses a distributed PowerPC environment and driving the visuals is the Evans & Sutherland Harmony image generator. The COTS element has been carried so far as to use a Sears & Roebuck garage door opener to open and close the mini-dome.
"We can insert those new PCs into the Harmony and flow the old equipment down the chain," Brantley says. "We can upgrade the entire system by purchasing just one new processor."
A crucial aspect of TSTARS is its ability to fit seamlessly into the existing maintenance and engineering structure of a carrier at sea, says Aaron Hutchinson, TSTARS principal investigator for the Naval Air Warfare Center Training Systems Division in Orlando, Fla. Aboard ship, TSTARS will not have its own database engineer, but will use data from the carrier`s existing imaging work suite as well as data available from external networks.
"Everyone on a carrier already has a job, so supportability is meant to happen by someone already supporting other devices already onboard," Hutchinson says. "You don`t want to have to take someone already doing a job off the carrier in order to add a new person for the TSTARS." Current planning calls for at least one TSTARS aboard each U.S. Navy aircraft carrier, he says. "The discussion point has been at about $2.5 million per unit, which is where something of this level and fidelity becomes attractive."
Hutchinson describes TSTARS as a serious effort by the Navy to define where the next generation of training devices are going: "It is a compilation of the mistakes and lessons we`ve learned, with a twist of new technologies." Although TSTARS itself is in an early stage of development, much of the technology involved already has started migrating into other programs.
The recent U.S. Air Force Roadrunner `98 Distributed Mission Training exercise used an operations console consisting of integrated workstations for the simulator operator, test director, and instructor pilot. A video wall, pictured above, shows the forward view from each cockpit.
The Onyx2 family of graphic workstations from Silicon Graphics Inc., pictured above, are popular simulation and training platforms. The F-14 jet fighter depicted on the screen is courtesy of MultiGen-Paradigm Inc. of San Jose, Calif.
How military and commercial simulation differ
The entertainment industry has brought much capability and image fidelity to military simulation and mission rehearsal, yet it still falls short of several high-level military requirements.
"The commercial and entertainment industries will help us get fast rendering visual image scenes, but they won`t help us get multispectral images consistent with ultraviolet, infrared, etc.," says U.S. Air Force Col. Kenneth Konwin, director of the Defense Modeling & Simulation Office (DMSO) in Arlington, Va.
"The physics to stimulate the sensors is a step function in cost and complexity the entertainment industry doesn`t care about," Konwin explains. "That`s why you can by an F-22 off-the-shelf PC simulation for $49.99, but it costs several orders of magnitude more to do a manned flight simulator to do all the sensor stimulations."
The two worlds of military simulation and commercial image generation remain divided over two primary areas: the military needs to incorporate real-world sensor data, while the entertainment industry doesn`t need and to update databases in a matter of days or even hours to reflect the real world.
"Today we can do some amazing things compared to 10 years ago, but we still haven`t reached the goal of making the simulated world indistinguishable from the real world," says Guy Russell, marketing manager for modeling and simulation at Silicon Graphics Inc. of Mountain View, Calif. He cites the recent Roadrunner `98 Distributed Mission Training (DMT) exercise sponsored by the Air Force Directorate of Command and Control.
"The use of satellite imagery and aerial technology throughout the simulation was an important breakthrough," Russell says.
The Roadrunner database is 500 miles on a side, and uses photo-specific imagery throughout. Still, "the resolution is still not quite good enough," he says. "We ran with 5-meter resolution through almost the whole database, with some areas at half-meter, which is pretty good. But there is not enough source data to do that throughout yet."
Imagery comes from satellites, aircraft, ground-based sensors, and human intelligence reports.
"When a pilot flies a simulator, the image generator has a limited number of polygons it can render," Russell explains. "The further you get form the pilot`s eyepoint, the more polygons it takes. Typically, those have been rendered out to about 30 nautical miles, which is considerably less than can be seen out a real aircraft window. In Roadrunner, we rendered the horizon out to 90 nautical miles, which brought in a lot more realism, especially with regard to navigation. That was important in making the pilots believe they were in the real world."
Visual acuity - essentially a factor of the number of pixels on the display screen - is a major area of debate within the simulation world. While military pilots typically are required to have 20/20 vision, simulator displays fall far short of that. For Roadrunner, the F-15C DMT simulator roughly doubled the fielded best systems, running at 1,700 pixels by 1,350 lines. That gave the pilots about 20/45 visual acuity- much closer to what they are used to seeing in the real world, but still not quite there.
"You can solve that with more channels and image generators, but it costs a lot of money," Russell notes. "This is the closest anyone has ever gotten at what is considered a reasonable price point - a couple of million dollars. To represent 20/20 visual acuity takes images that are 5,000 pixels by 4,000 lines per channel in an eight-channel pentagonal screen, he says.
In some ways, the need for such realism is greater for Navy aviation than for the Air Force.
"Accurate representations of a natural environment are going to be increasingly more important to the way we train and develop tactics," says Navy Capt. Stan O`Connor, DMSO`s deputy director. "The ability to provide value-added realistic representations of the carrier environment for practicing approaches to landings and actual touchdown in a simulation hasn`t been seen in the past."
As for the Air Force, Konwin admits, "we can press the state of the art on visual acuity and carrier landings, but it may be something we reserve for live training after a certain level of affordability"
The source of that evolution may be totally unexpected. For example, Konwin says some of the information technologies now being used in military simulation are simply more sophisticated versions of the same technologies being used on the World Wide Web. Compression methods developed to dramatically reduce the size of a Web image without any noticeable degradation in the image also can be used in simulation.
Military leaders still hope to adopt future low-cost commercial systems to meet some of these high-end needs, but in this case, they say, it is a question of COTS moving toward the military requirement technolog-ically, not the military moving toward COTS by lowering the requirement.
"We have some active involvement with the commercial world," notes Pam Woodard, visual systems engineer at the Simulation, Training & Instrumentation Command. "We can`t just sit back and watch and depend on the commercial world to develop exactly what we need in military simulators. We are looking for them to develop some of the advancements in low-end systems, but we will have to fund some of the high-end needs that the entertainment industry just doesn`t care about."
Members of the Roadrunner `98 exercise, which made extensive use of computer-aided simulation and mission-rehearsal capability, review mission tactics. Pictured above are Air Force Capt. Rob Tofil and Lt. Don Kang from Fighter Squadron 522 at Cannon Air Force Base, N.M.
