AI will teach soldiers how to dress wounds, fly helicopters in AR

DARPA is developing AI to help soldiers learn new tasks with a step-by-step demonstration in augmented reality

DARPA (Defense Advanced Research Projects Agency) is developing AI assistants that will help them perform complex tasks with a step-by-step guide in augmented reality (AR).

Military personnel can be asked to dress wounds, repair fighter jets or even fly them in the recently created Perceptually-enabled Task Guidance (PTG) program in 2021. Soldiers will wear sensors including microphones and head-mounted cameras, which will collect data on what they see, hear and say.

Instructions to complete tasks and how to do them, will be displayed through AR goggles which they will be required to wear.

Manager of the PTG program, Bruce Draper said, "What we need in the middle is an assistant that can recognize what you are doing as you start a task, has the prerequisite know-how to accomplish that task, can provide step-by-step guidance, and can alert you to any mistakes you’re making,”

DARPA has already awarded PTG contracts and grants to researchers from NYU, Northrop Grumman, and more, to track progress. The demos by the AI assistant would be like following a recipe.

“[Cooking is] visually quite complex,” Draper says. “There’s specialized terminology. There are specialized devices, and there’s a lot of different ways it can be accomplished, so it’s a really good practice domain for all kinds of other highly skilled tasks.”

The AI assistants developed may be released for use by the public soon, just like GPS and internet, which had initially been developed for the military.