Augmented reality and structural steel fabrication

Using augmented reality goggles, a fabricator is looking at a digital replica of what the workpiece he is about to work on should look like when it is complete. Software from Eterio Realities places a 3D model over the digital version of the structural steel member, giving the wearer of the goggles an idea of what parts should go where on the workpiece. Images: Eterio Realities

The days of staring at a blueprint and then repeatedly glancing over at a steel beam, trying to get it straight in your head how all the parts detailed on the print are supposed to be located on the structural steel workpiece, might be drawing to a close. Augmented reality is here to help.

Admittedly, some structural steel fabricators become very adept at looking at a blueprint, quickly surmising what needs to go where, and then tackling the project. That's not the case for everyone, however, especially those new to the structural steel fabrication industry. The ability to translate 2D drawings into a 3D vision of what the workpiece is supposed to become is a talent that is developed through practice. Some people get to the 3D visioning quicker than others, but it still takes a while before the talent is evident.

Even then, some projects emerge that challenge the most skilled of fabricators. That's what Warnaar Steel Tech Ltd. discovered in 2010 when it tackled a façade fabrication project for the Sparkling Hill Resort in Vernon, B.C. The Swarovski family—a name that might be familiar to those who have ever shopped for crystal jewelry or decorations—developed the resort, and with that prominent brand being part of the development effort, the building itself had to be an attention-getter. That's why 3.5 million Swarovski crystals are incorporated as architectural and artistic elements—some mimicking the movement of waterfalls and others visually replicating fireplaces as daylight dances through them. Then there's the entrance to the resort, which is designed to appear as a giant crystal itself.

Look at any crystal and you see the many angles present. Envision a giant structure of steel pipes and glass aiming to create the sharp lines and unique shape of a crystal, and you can imagine the many challenges that awaited those in charge of bringing those designs to life.

It was crystal clear to the management staff of Warnaar Steel that they needed a better tool than 2D blueprints to help the fabrication team.

"It was very complicated," said Randolph Warnaar, who was then general manager of the Kelowna, B.C., structural steel fabricating company. "As part of this larger project, we had this giant crystal façade, and we had to build the supporting steel structure for it. There were a bunch of guy wire clips that had to be attached to all of these pipes. The 2D drawings just weren't cutting it. The fabricators needed to be able to understand how to fabricate it.

"So this was the first time that we brought a laptop down to the fabrication floor and really had them look at the 3D model in order to understand how it all came together. That made me think: ‘We have all of these 3D assets, so why are we building these 3D structures from 2D drawings?’"

A few years later, in 2015, the first generation of the Microsoft HoloLens augmented reality goggles were made available to software developers. Warnaar said that's when he had the "lightbulb idea." They could take the 3D building information models, which were widely available but mostly used by engineers and architects, and blend that with the augmented reality technology that was emerging at the time.

Today Eterio Realities, led by CEO Warnaar, is marketing the culmination of that early product development and ensuing refinement, done on the shop floor of the family structural steel company, as FabStation-STEEL, an augmented reality (AR) tool that brings a structural steel workpiece job from a blueprint to a 3D model that replicates what the end product should look like. It's also designed to provide real-time reporting and data exchange to front office management systems.

"I guess you can say that our mission statement is to replace all 2D drawings with 3D constructible digital guides," Warnaar said.

The augmented reality software also can be used with a tablet computing device. The fabricator only has to hold the tablet close to the workpiece to get the full effect of the digital replica on the screen.

So how does it work? A shop floor fabricator takes a tablet loaded with the proprietary software and hovers it over the beam structure. The augmented reality technology acts as a digital template that places the 3D model over the workpiece virtually. Looking at the tablet screen hovering over the workpiece, the fabricator can see what parts need to go where and how the build is supposed to go.

"Using the AR feature allows the fabricator to take ownership of their work and catch mistakes before they leave the station," said Chris Thibodeau of Eterio Realities.

With the HoloLens 2, the fabricator obviously doesn't have to hold anything. Using the software, the goggles place the 3D model elements over the workpiece. The user can adjust the view by simply adjusting the viewpoint. Thibodeau said that using the HoloLens requires a bit more training, because the interface is typically new to anyone using it, but with consistent use, the fabricator can progress through jobs simply and quickly, even though those watching might find it strange that the person wearing the augmented reality goggles is punching buttons in the air.

Warnaar said that his company is targeting the use of the Hololens mainly for inspection right now. The software is accurate to within 1/8 in. or 3 mm when properly calibrated, which often is good enough for quality checks because most errors in structural steel fabrication environments are gross errors rather than ones of precision. Instead of seeing if a part is placed within fractions of an inch of where it's supposed to be, quality technicians most of the time are looking to see that the material sizes are correct and that parts are orientated and placed correctly.

The FabStation software also provides access to the job's complete fabrication data, including all of the part and assembly drawings. Production reporting and tracking also are part of the software, giving the front office real-time updates that inspections have taken place and the workpiece is ready for the next process or delivery.

Warnaar described the project data upload as an "easy upload." The 3D models are placed in a compressed file and uploaded to the FabStation cloud. When the upload is complete, the user only needs to click on the newly created file to access the digital overlay or other 2D drawings related to the job.

"Structural steel is the ultimate prefabrication environment. It's somewhat of a controlled environment, and it's a great proving ground for this type of technology," Warnaar said. "From here, we can go on to other things, such as rebar prefabrication, shipbuilding, and even on-site activities."

After more than 18 months of proving out the technology at Warnaar Steel Tech, the Eterio Realities team believes its product can make the shop floor more efficient and put new employees on the fast track to becoming more productive.

The structural steel work, especially the main entrance that mimics the features of a large crystal, at Sparkling Hill Resort in Vernon, B.C., motivated Warnaar Steel Tech Ltd. to find a better way to show its fabricators how to build complex jobs. Instead of relying on blueprints, company management started to investigate ways to use the 3D models that were already part of the building information model and present that in a 3D format. The end result was the FabStation augmented reality software.