Walking through an unfamiliar city, getting directions or simulations of buildings that no longer exist – augmented reality is where virtual content and the real world come together. Scientists at Karlsruhe Institute of Technology (KIT) are developing an assistance system based on this technology which supports engineers in building and maintaining aircraft tanks. The system is currently being tested in industry.
In augmented reality applications (AR), a tablet, smartphone or special video glasses, also called a head-mounted display, provide additional information on reality. On the display, virtual content is mixed with the real world. One of the best-known AR applications is the game of Pokémon GO, a virtual hunt for virtual beings which “emerge” in all kinds of real-world locations. But the technology is gaining more and more in importance also in industry: “We are developing software that helps us build and maintain aircraft tanks. It aims to increase the flexibility of the employees, accelerate the workflow, and connect and optimize processes,” says Christian Tesch from the Institute of Anthropomatics and Robotics at KIT, Chair of Intelligent Sensor-Actuator-Systems (ISAS, headed by Prof. Uwe D. Hanebeck). Many commercial aircraft are at first not equipped for long flights and their fuel tanks are too small. So that they can still travel long distances, additional tanks are required which have to be maintained on a regular basis. To do that, so far engineers have had to climb into the tanks through a small opening. Often, however, they need both hands for assembling components, and new workers in particular also need instructions at the same time.
“The augmented reality glasses – at the moment we are using the HoloLens from Microsoft – display the work to be done in the engineers’ field of vision, who then have their hands free to install or repair components,” says Tesch. The glasses are equipped with cameras. Users scan special markers on the tank in advance using the cameras, which communicate the exact location and the size of the tank to the glasses. A transparent 3D computer model from the inside of the tank is then projected onto the “real” tank; this means that engineers can also look into the closed tank from outside, understand the structure in detail, and get step-by-step instructions on how to install a pipe, for instance. What’s more, with the help of markings on the ground, the glasses show where to find the required components in the warehouse, the location of which the glasses also recognize thanks to markers that the system has already learned. “We connect the actual work on the tank with locating objects which helps us create an overall concept,” says Tesch.
All calculations required for this concept take place directly in the glasses. Additional information about the tank condition, the work progress or the component stock can be supplied through an external database so that users are always up-to-date. They can operate the AR glasses using gesture and voice control.
Researchers are not only developing the system for AR glasses: “Many people today have a smartphone or a tablet; these devices have also found their way into day-to-day work. That’s why our software will also work with standard smartphones,” says Dr Antonio Zea who is in charge of developing the software for mobile devices at the Institute of Anthropomatics and Robotics. In the next few years, the hardware for AR applications will probably also continue to be enhanced; AR glasses could get smaller and more affordable, making them even more versatile.
KIT is working together with PFW Aerospace Gmbh on this project. This allows scientists to test and improve the software in real conditions. They want to make it ready for use by the start of 2019.
Being “The Research University in the Helmholtz Association”, KIT creates and imparts knowledge for the society and the environment. It is the objective to make significant contributions to the global challenges in the fields of energy, mobility, and information. For this, about 10,000 employees cooperate in a broad range of disciplines in natural sciences, engineering sciences, economics, and the humanities and social sciences. KIT prepares its 22,800 students for responsible tasks in society, industry, and science by offering research-based study programs. Innovation efforts at KIT build a bridge between important scientific findings and their application for the benefit of society, economic prosperity, and the preservation of our natural basis of life. KIT is one of the German universities of excellence.