When real life user interfaces begin to emulate video games

By Ernesto Burden
Posted by Ernesto Burden.

It used to be that video game creators tried to emulate real life experiences. Remember Microsoft's Flight Simulator? It was about as complicated to fly those virtual planes as it is to fly real planes (I spent more time crashing my airliners than landing them as a kid), and that was the goal. But what happens when that paradigm gets flipped on its head - when the real life user interfaces model themselves on rapidly evolving video game standards? I had a first hand taste of this at a recent defense electronics shows I attended - and even got to take the joystick and learn just how easy robot-wranglin' can be at the surreal VR dawn of the 21st century.

If you've played a newer first-person-shooter style video game, you know how powerful the graphics engines are and how immersive the experience is. In a lineage descending from classics such as Doom on through the the most recent Call of Duty or Halo releases, first-person-shooter games have evolved into marvels of 3D graphics rendering, allowing for ultra smooth movement through incredibly detailed environments that draw the player in so well that experiences such as a fall or wild charge down a winding corridor generate actual physical sensations. At the touch a button on the computer keyboard or controller device, you can toggle visual perspectives and control not only forward, backward, left and right movement, but also left and right "strafing" movements, crouching, leaping and in some cases flying or swimming with perfect altitude and depth control.

So what does this embarrassment of riches in entertainment technology mean in aerospace and defense? Well for one thing, it means that the commercial game environment has had generations to explore not only virtual environments, but astoundingly intuitive user interface for control for navigating those environments.

Which brings me to the Association for Unmanned Vehicle Systems International (AUVSI) show in Washington, DC, our Military & Aerospace Electronics Team attended last month, and back to Flight Simulator. I found as a kid - not being a pilot or someone inclined wanted to read a full length instruction manual just to play a game - the planes in Flight Simulator to be difficult to fly. At AUVSI, M&AE chief editor John Keller and I watched a small drone helicopter - the Hornet Micro UAS - demonstration. After the demonstration we visited with the helicopter's maker Adaptive Flight and we got to see how the vehicle was flown. Hint - you didn't need four hands and six feet to operate all the pedals and levers and sticks and whatnot. You used one joystick and a keyboard. The 12-year-old version of you could drop in from the past and fly this thing with about one minute of instruction, no manual required. You'd know exactly how to do it, because it worked just like a video game. In fact, that's pretty much what the UAV's maker says in the product literature: "Advanced flight control technology makes flying the Hornet as easy as navigating a video game with take-offs and landings at the push of a button."

A little bit later that day I was chatting with Andrew Borene at the Recon Robotics booth. Recon makes a tiny little robot called the Recon Scout IR. It's basically a cylinder about the size of a 2.5 lb dumbbell with rubbery wheels on either end and a couple of flexible antennas sticking out. The idea is you can lob this little guy up onto the roof of a building, say, and then drive it around reconnoitering. Andrew gave me one of the robots and let me throw it onto the roof of the booth, which was designed to look like a desert outpost. He then handed me the controller - once again a single joystick was all you needed to drive. The vehicle was designed to self-right and then get on with business, shooting video automatically and transmitting it back to a screen in the controller. Drive into a dark room on the roof? The camera automatically switches to an IR view. I drove a robot about this size in a video game a long time - James Bond Golden Eye - and I say with absolute conviction that the video game version was actually a bit harder to operate. That's how good these things are getting.

So while soldiers are using "serious" games for training and simulation from the cockpit to the battlefield (read this article by M&AE's Courtney Howard for a great overview of this), it's also clear that elements of these games are blurring the line between reality and virtual reality in the other direction as well. And it's fascinating to wonder what impact having grown up in the deeply immersive, super-detailed environments of contemporary video games will have on the engineers and developers of the future in terms of the direction from which they instinctively approach user interface challenges. Link

Ernesto Burden is the publisher of PennWell's Aerospace and Defense Media Group, which include Military & Aerospace Electronics, Avionics Intelligence, and Avionics Europe. Email him at ernestob@pennwel.com.

Previous Blog Posts

Capital Hill budget deal could restore tens of billions of dollars to the Pentagon

Tue Dec 17 13:15:00 CST 2013

Hacker drone story a cautionary tale about the need for unmanned vehicle data security

Tue Dec 10 09:46:00 CST 2013

Lack of money for systems upgrades threatens to maintain wind-farm radar dead spots

Tue Dec 03 10:36:00 CST 2013

Engineering support contracts indicate the Pentagon is sinking into the Mothball Strategy

Tue Nov 26 06:57:00 CST 2013

The revenge of COTS: an ageing commercial technology base complicates military supply chain

Tue Nov 19 08:53:00 CST 2013

Navy's newest destroyers evolve to fill traditional battleship roles

Tue Nov 12 11:54:00 CST 2013

International suspicions of U.S. encryption technology putting defense companies in a bind

Tue Nov 05 11:24:00 CST 2013

Defense industry left guessing as Army struggles forward with an unclear mission

Tue Oct 29 09:45:00 CDT 2013

These are tough times for the combat vehicle and vetronics industries

Tue Oct 22 04:22:00 CDT 2013

Is the government shutdown a harbinger of more ominous things to come?

Tue Oct 15 11:21:00 CDT 2013

Government shutdown reduces military contracting, increasing pressure on U.S. defense industry

Mon Oct 07 12:17:00 CDT 2013

Potential good news: has U.S. defense spending finally bottomed-out?

Tue Oct 01 13:02:00 CDT 2013

Is robotics revolution the first glimpse of a fundamental change in human evolution?

Tue Sep 24 09:46:00 CDT 2013

Obsolescent parts: are we enhancing military readiness or creating a hollow force?

Tue Sep 17 15:46:00 CDT 2013

For the high-tech warfighter, the future of electronics-laden uniforms is here

Tue Sep 10 11:26:00 CDT 2013

New generation of embedded computing thermal management in development at GE

Tue Sep 03 09:44:00 CDT 2013

Trading bus stops for credit cards: how far embedded computing has come in three decades

Tue Aug 27 10:59:00 CDT 2013

Unmanned vehicle industry stands at the doorstep of a fundamental transformation

Tue Aug 20 11:09:00 CDT 2013

AUVSI 2013, one of the biggest unmanned vehicles shows in the world, opens this week in Washington

Tue Aug 13 05:35:00 CDT 2013

The Washington Post, under Jeff Bezos, could lead the way for media in the 21st Century

Tue Aug 06 09:47:00 CDT 2013

Are costs and vulnerabilities making military leaders nervous about satellite communications?

Tue Jul 30 11:07:00 CDT 2013

Unmanned aircraft carrier that travels beneath the waves may be in the Navy's future

Tue Jul 23 05:20:00 CDT 2013

Electronic warfare programs kick into high gear with a flurry of contract activity

Tue Jul 16 08:03:00 CDT 2013

How vulnerable are U.S. Navy vessels to advanced anti-ship cruise missiles?

Tue Jul 09 07:03:00 CDT 2013

First came VHSIC, then came MIMIC, and now comes ACE to push electronics technology

Tue Jul 02 09:16:00 CDT 2013

The Mil & Aero Bloggers

John Keller is editor-in-chief of Military & Aerospace Electronics magazine, which provides extensive coverage and analysis of enabling electronic and optoelectronic technologies in military, space, and commercial aviation applications. A member of the Military & Aerospace Electronics staff since the magazine's founding in 1989, Mr. Keller took over as chief editor in 1995.

Ernesto Burden is the publisher of PennWell’s Aerospace & Defense Media Group, including Military & Aerospace Electronics, Avionics Intelligence and Avionics Europe.  He’s a father of four, a runner, and an avid digital media enthusiast with a deep background in the intersection of media publishing, digital technology, and social media. He can be reached at ernestob@pennwell.com and on Twitter @aero_ernesto.

Courtney E. Howard, as executive editor, enjoys writing about all things electronics and avionics in PennWell’s burgeoning Aerospace and Defense Group, which encompasses Military & Aerospace Electronics, Avionics Intelligence, the Avionics Europe conference, and much more. She’s also a self-proclaimed social-media maven, mil-aero nerd, and avid avionics geek. Connect with Courtney at Courtney@Pennwell.com, @coho on Twitter, and on LinkedIn.

Mil & Aero Magazine

June 2015
Volume 26, Issue 6

All Access Sponsors

Download Our Apps




Connect with Us


Military & Aerospace Electronics

Weekly newsletter covering technical content, breaking news and product information

Cyber Security

Monthly newsletter covering cyber warfare, cyber security, information warfare, and information security technologies, products, contracts, and procurement opportunities

Defense Executive

Monthly newsletter covering business news and strategic insights for executive managers

Electronic Warfare

Quarterly newsletter covering technologies and applications in electronic warfare, cyber warfare, optical warfare, and spectrum warfare.

Embedded Computing Report

Monthly newsletter covering news on embedded computing in aerospace, defense and industrial-rugged applications

Unmanned Vehicles

Monthly newsletter covering news updates for designers of unmanned vehicles