How augmented reality is being used to train the next generation of welders

In this augmented reality gas metal arc welding setup, a student lays down a virtual bead.

It's a black art. I used to hear that a lot on fab shop tours—code for something that took years to learn and only the talented few truly mastered. Why, exactly? Sometimes it had to do with the nature of the skill and the worker's tactile and visual experience welding a workpiece. If something went awry, they’d try again. And again. And again. For years.

Thing is, considering the acute worker shortage, the industry just doesn't have that kind of time. It needs some way to shorten that training cycle while not skimping on process fundamentals, so that students know what works in what situation and why. They don't just show up to the shop, learn a narrow set of skills (push this button, weld this joint), and get to work. They follow procedures, but they also know why those procedures work so well.

Here, augmented reality (AR) might fill a need, especially for one of the most hands-on processes on the fab shop floor: welding.

"We’ve been in the augmented reality space for the past eight years, and it continues to get better and better. We’re trying to get as close to reality as possible, including visual, audio, and tactile elements. Our ability to create an accurate weld puddle in software has come a long way in just the past few years."

That was Steve Hidden, national account manager, welding education and workforce development, at Miller Electric Mfg. LLC in Appleton, Wis. The company offers its AugmentedArc® augmented reality welding system, a technology that merges the visual, aural, and tactile welding experience—the gun, the workpiece, the buzzing, the visual cues—with software that simulates how a bead flows given how the weld is performed.

Students can wield a gas metal arc welding (GMAW) gun, a stinger for shielded metal arc welding (SMAW), or a gas tungsten arc welding (GTAW) torch. The experience isn't a video game. Using a combination of sensors that read the position of strategically placed QR-codes on the weld gun or torch and workpiece, the AR approach to weld training tracks students’ movements throughout the process.

Imagine the first time a student grasps the welding gun and lays a bead on plate. He strikes the arc, hesitates, then moves too fast. He tries again and burns through. Spatter goes everywhere, and coupon after coupon after coupon goes into the recycling bin. The student continues to practice, using up shielding gas and welding wire, and putting the gun nozzle and other consumable components through all sorts of abuse. It's not a pretty site, and—as anyone working at a technical school or fab shop with in-house training knows—it can get quite expensive.

Now imagine that same student donning a welding helmet, only this time he's manipulating an odd-looking GTAW torch and filler rod, each with QR codes. Instead, sensors in the student's helmet read those codes to determine how exactly the student manipulates the "tungsten" tip along a lap joint, creating a virtual fillet on a workpiece that, again, is covered with strategically placed codes, all of which become invisible when the student dons the welding helmet. With the helmet on, he sees a metallic workpiece ready to be joined. He depresses the foot pedal throughout the process, but there are no arcs, no spatter, no metal at all.

A student next to him wields another odd-looking device, this time a stinger for the SMAW process with a code-covered "cube" near the end—again, all invisible through the weld helmet. She manipulates the stinger carefully down a vertical groove joint. Next to her sits another student, this one manipulating the nozzle of a GMAW "gun" around a pipe to create a flange weld.

All three are welding in AR. They see the arc, filler metal, and weld being laid down, and they even hear the weld "buzz," just as a welder would in a real-world application. When the student practicing GTAW dips too much of the filler rod at once, the weld pool reacts. When the student practicing GMAW travels too quickly, again, the weld pool reacts. After completing the practice joint, all three students keep their welding helmets on to view their completed, virtual weld, defects and all.

Through the welding helmet, the student can see a virtual representation of the welding arc, plus certain markers showing ideal work position, travel, and standoff for a particular lesson.

The software points out exactly where and how those errors occurred. The arc length was too long here. The filler rod angle was off here. Your work and travel angles were off. Not only this, the software shows what those angles should have been and exactly how to correct it.

On the next try, the students’ teacher turns on visual aids that they see through the welding helmet, showing what elements should be where. "I call them ‘training wheels,’" Hidden said. "Are they too close? Are they too far way?"

The visual aids are dynamic, changing as needed to direct the student. For instance, a visual cue might show the student where the welding gun should be at a certain point during the weld program; as the student "catches up," the cue changes.

"We also give the instructor the ability to let students make their own mistakes," Hidden said, adding that the software needn't tell them that, say, their gas setting is wrong, that the voltage is set too high, or anything else. The software might not simulate exactly what happens when something goes awry, like an actual burn-through of the material. But it can graphically show something's amiss and leave students to figure out what the problem is on their own. Alternatively, teachers can set the system up to notify exactly what's wrong.

"Teachers can create their own assignments," Hidden said, adding that they can establish what "training wheel" symbols to display and when. The key is knowing when to stop giving assistance and let students fly solo. It all depends on students’ needs and where they are on their training journey.

"[AR] serves as an interim phase between theory and hands-on applications," said Patricia Carr, national manager of education and workforce development at Miller, adding that the technology has helped students uncomfortable with the arcs and sparks, including those with disabilities, gain confidence before practicing the real thing, effectively broadening the recruiting net.

The AR system has been designed around the needs of educators. Over the years, experienced welders and welding engineers within and outside Miller have collaborated with software engineers to make the experience ever more realistic. Students now can see all the puddle dynamics, with molten metal wetting against the joint sidewalls. They see weld pool disturbances that could indicate undercut or porosity, incomplete penetration, as well as over-welding practices that could lead to excessive and costly grinding.

All this begs the question, could AR be used to certify welders? Hidden chuckled a bit. "Not today," he said. "This tool is all about preparation. AR could allow welders to refine their technique and gain muscle memory before practicing in the lab and taking the certification test."

Gaining that muscle memory can be an extraordinary challenge, especially when improper technique can lead to a messy situation—like stick welding overhead, holding an exceedingly long arc, and being caught in a rainstorm of hot sparks.

Using AR, a student can place the workpiece wherever needed to start practicing those challenging weld positions. "We have seven coupons for all positions," Hidden said, "including flat and overhead. And the beauty with AR, I can take my part [coupon], put double sticky tape on it, and put it underneath the table. Students can crawl underneath the table and weld it."

Looking through the welding helmet, a student can inspect his weld and receive specific feedback.

Repetition builds muscle memory and confidence, preparing students for the real world of overhead welding. Once they strike an arc for real, they’re more likely to maintain the right welding technique and produce a clean bead without a plethora of sparks raining down.

Students and professionals using the AR system can practice any technique they wish, but as Hidden explained, software does need to be built around a specific technique to "score it"—that is, building the ability to track the position of the weld and consumables, compare it to an ideal, score it based on that comparison, and pinpoint areas for improvement. For instance, students practicing GTAW might want to "walk the cup" over a certain joint geometry. They can run through the motions to gain the muscle memory, but the system won't be able to give comprehensive feedback, at least not yet.

Though, of course, new software is being written and improved upon all the time. As Carr explained, Miller has been following the voice-of-the-customer methodology, developing software requested by the majority of current and potential users of the technology.

Even in its current state, AR helps demystify a misunderstood, opaque process by identifying exactly what makes a good weld and what doesn't. Skilled people take many paths to achieve a quality weld, so not following exactly what the AR system prescribes doesn't guarantee failure. Still, in the future, those learning and perfecting their skills—and perhaps even experienced welding professionals—might look more and more to AR as a kind of compass, something to reference to make sure their fundamentals are there and that they’re headed in the right direction. And there's an added bonus: They needn't waste consumables and test coupons in the process.