The company has developed a system that is based on the Apple Vision Pro AR system. The high-fidelity imagery of aircraft flight decks allows trainees to manipulate all the controls and view the instruments and status indicators in real time as they do so.

The system won’t replace full-on flight training device hours, but it will allow trainees to master homework tasks.

“The Apple Vision Pro App developed by CAE will allow pilots to familiarize themselves with the flight deck, practice critical procedures and develop muscle memory for key functions from anywhere,” Emmanuel Levitte, CAE’s chief technology and product officer, said in a statement. 

The company has fully developed the app for the Bombardier Global 7500 but says it can be adapted to any aircraft.

AVweb’s Russ Niles took the goggles for a spin and said the system is startlingly realistic.

“The fidelity and the virtual manipulation of the controls is hard to describe until you’ve done it,” Niles said. “With some practice, a trainee could be confidently running procedures and dealing with emergencies in a realistic way without leaving the couch. The prospects for this kind of training device are significant.”

Niles spoke with CAE’s Director of Incubation Eric Fortin, who developed the system, about its possibilities.

Editor’s Note: This article first appeared on AVweb.

The post CAE Develops AR For Flight Training appeared first on FLYING Magazine.

The recent exercise is an advancement toward a “military Metaverse” for combat flight training that allows multiple pilots to communicate at the same time while using a synthetic training environment, Red 6 said in an announcement made during the UP.Summit mobility conference underway this week in Bentonville, Arkansas.

During the test flight, pilots in two Berkut 540 experimental aircraft departed Santa Monica Airport (KSMO) in California and entered Red 6’s AR environment, called the Combined Augmented Reality Battlespace Operation Network (CARBON).

• Virtual Reality in Flight Training: More Than a Fad

Once linked in CARBON, the pilots could see and interact with an AR generated Boeing KC-46 Pegasus Tanker. Through AR, one pilot conducted a training mission with the tanker, while the other pilot observed in real time.

“We are thrilled that our thesis has been validated, and this achievement is a major step forward towards the creation of an outdoor Military Metaverse in which all warfighters, across all domains can train together,” said Daniel Robinson, founder and CEO of Red 6, in a statement. 

The test flight comes two years after Red 6 conducted what it said was the “world-first live dogfight” against a virtual aircraft driven by artificial intelligence.

“The flight showcased the groundbreaking display and control systems needed to bring AR into the real world, while simultaneously integrating tactical AI into the Red 6 system,” the company said.

Red 6 is now looking at AR solutions on the ground, according to Robinson.

“We are in the early stages of exploring dismounted solutions and the ability to connect a dismounted soldier to pilots in an airplane to allow them to see a common picture,” he told FLYING.

• Augmented Reality Brings Jets Into The Classroom

The aviation industry is increasingly turning to AR, for applications from air traffic control operations to pilot training. 

Last fall, students at Embry-Riddle Aeronautical University, Prescott, developed AR holograms that create 3D models of the CRJ-700 and allow the student to move around inside the aircraft systems.

The post Military Metaverse: Red 6 Conducts Multi-Aircraft Augmented Reality Training Flight appeared first on FLYING Magazine.

Holograms, used for decades in the entertainment industry, may soon become part of the aviation training environment thanks to a team of students at Embry-Riddle Aeronautical University, Prescott.

The students are part of ERAU’s Simulation Science, Games and Animation program. Over the summer, the student team developed augmented-reality (AR) holograms, creating 3D models of the CRJ-700.

How it Works

Using HoloLens 2 goggles, the user steps through a series of training modules composed of interactive slides.

The AR aspect of the project allows the student to move around inside the aircraft systems, making learning akin to something out of “Fantastic Voyage,” a classic science-fiction movie where a submarine and its crew is shrunk to microscopic size to allow them venture into the body of an injured colleague to provide life-saving medical treatment.

It allows students to explore the CRJ-700 in a way they never could before.

“We encouraged the team to consider the reasons for using augmented reality rather than virtual reality on the project,” associate professor and SSGA program chair Derek Fisher explained. “They determined that while virtual reality might provide a more immersive alternate experience, AR provides opportunities to focus on interacting with the computer-generated jet while collaborating in the familiar environment of the classroom, with real-life colleagues.”

What’s to Come

Another type of hologram model in development could let the students look around the flight deck, manipulating the buttons, switches, and controls as they would in real life. The HoloLens 2 can even “see” their hands. This feature will be a benefit for tactile and kinetic learners, who need to see and touch something for optimum learning to take place.

The holographic slides contain informational labels. These labels can be manipulated by the user, making them larger or rotating them. The idea is that the users will become familiar with the controls of the aircraft and the system schematics, such as the radar danger zones and exhaust system.

The labels are removed when it is time for testing and the students can be asked to identify the parts and systems that they’ve been studying.

How Students are Learning

Evan LaBate, a senior SSGA major, notes he is gaining valuable experience as part of the developmental team, and envisions the simulations will not only enhance flight training on campus, but may also be used in other technical fields.

“Participating on this project has helped me decide that I want to do virtual or augmented reality simulations, either for military or police, as my career,” LaBate said. “I believe this technology can be used to help the military simulate live firefights, or even help with the Air Force, in giving pilots updated heads-up displays.”

“This project was a massive learning experience,” says Josh Snow, a SSGA senior. “Jobs in the field of interactive technology are plentiful but demanding, often working on large projects in small teams. The skills I have picked up from this research opportunity have better prepared me for all facets of working on and or managing a team in my career.”

The initial project was funded by an Undergraduate Research Institute grant. Now the team is focusing on improving the software to allow multiple users to interact with the same jet simulation simultaneously. After testing the new prototype, the group will seek funding to expand the program to a full-class experience.

“This is the first step down a very good road for us as a university, potentially toward an entire class in AR,” said Dr. Michelle Hight, assistant professor of aeronautical science and fixed wing program chair.

“Even if students had to attend class from home, this brings the jet to life. Every student who has a HoloLens 2 could join in 3D, standing or sitting in a virtual classroom, interacting with hologram learning tools rather than a flat screen. This is a timely pedagogical development for a world evolving and responding to a global pandemic.”

The Right Move

ERAU’s application of augmented reality is the next logical step in flight training, said Michael Puoci, a Seattle-area pilot and video game designer with more than 20 years of experience. Aviation games, especially those that involve vintage aircraft cockpits, are a passion for Puoci. During his student pilot days, he often used virtual-reality computer cockpits that he designed to practice real-world flights.

“Augmented reality will be what pushes aviation training devices to a new level,” Puoci said. “With VR, you have a mouse and screens. With AR, the person will put on the glasses and what used to be presented on screens will be a digital fabrication and full immersion and far better fidelity for simulation training for new pilots.”

The post Augmented Reality Brings Jets Into The Classroom appeared first on FLYING Magazine.