In September, packaging machine specialist Harpak-ULMA was honored for its use of Augmented Reality (AR) by PMMI at the association’s Pack Expo show in Las Vegas.
Harpak set the stage for its use of AR in 2018, when it launched a multi-year, multi-phase technology plan, said Matt Bauer, the company’s product manager of smart, connected machines.
The first phase was to re-platform Harpak’s packaging machines using Rockwell Automation controls to establish a foundation of smart connected capabilities. Smart connected machines, Bauer said, allow companies to capture and expose key machine operating data fundamental to any AR experience.
Once phase one was completed, Harpak could start developing AR capabilities and experiences.
“Based on our existing partner relationship with Rockwell Automation, leveraging their intimate relationship with PTC was an obvious choice, particularly considering PTC’s related AR products — Chalk, Vuforia, Thingworx — are all considered market leaders.”
All three products are utilized by Harpak for building and delivering the company’ AR experiences, he said:
- Chalk delivers remote guidance. Bauer compared it to “a Facetime app on steroids.” It allows for on-screen augmentation, such as circling a particular valve during a conversation.
- Vuforia is a comprehensive platform of AR tools that allows users to represent data in augmented reality, depending on their use case. It provides the capability to digitally “overlay” animation and visually guided work instructions in the physical context of the user’s field of view.
- Thingworx is the orchestration layer for Vuforia, bringing together IIoT data, 3D models and the actual AR content together into a platform where different users can access data specific to their use cases.
Working together, Bauer said, Thingworx and Vuforia are key software components that access and display exposed machine data in the context of any particular experience — whether it’s training, troubleshooting, or maintenance related.
The final two phases of Harpak’s AR plan are the introduction of Digital Twins, which will also leverage PTC software; and predictive analytics, which utilize massive amount of machine data combined with ML/AI to predict unplanned downtime events.
Harpak is currently working on the Digital Twins phase.
The purpose of artificial intelligence in an augmented reality experience, Bauer said, is to enforce and validate specific actions taken by the user during the experience.
In these cases, AI utilizes a combination of Vision systems data and data collected by the AR experience to determine if an action has not only been performed — but performed correctly.
“AI gives us the ability to determine, on the fly, if a particular action has been taken,” Bauer said.
For example, during a repair operation, where certain bolts need to be installed in the correct order, AI can recognize whether a procedure was completed “per spec.”
That level of knowledge, combined with guidance and oversight, allows even untrained and unexperienced workers to complete tasks with an unparalleled degree of speed and accuracy, Bauer said.
More accurate, faster performance
In Harpak’s beta deployment, completely untrained individuals with zero expertise performed a complex repair with 100% accuracy and 70% faster than an experienced technician.
“Frankly, we’re just scraping the surface of what is possible here.”
AR delivers a unique combination of “knowledge on demand” in visually oriented guidance within the context of the actual physical world, Bauer said, allowing companies like Harpak to quickly train or provide expert guidance to a workforce, providing them with the information they need at the right time and in the correct context.
“It’s a proven and well accepted precept that that human beings absorb and process visual data at rates far in excess of written instructions. For front-line workers this exponentially shortens learning curves while simultaneously increasing accuracy of any task.”
That, he said, is where most of the value is, especially as producers look for ways to further optimize production processes. Quickly alerting staff to performance issues, visually guiding them through the process to bring a machine back up and running or perform a product line changeover can have a big impact.
“For those of us in the packaging industry, AR offers the opportunity to increase the speed and accuracy with which front-line floor workers accomplish tasks — in other words, improve human productivity. In fact, expectations of 30-60% productivity improvement are well within the bounds of reality.
Internally, Harpak jokingly refer to this particular moment in the company’s history as the “Matrix moment.”
In the famous movie, the characters Trinity and Neo are on a rooftop when Neo asks Trinity if she can fly a helicopter, and her answer is “in a moment.”
She then proceeds to immediately download everything she needs to operate the machine.
While that level of “knowledge on demand” is still only science fiction, Bauer said, the reality is that companies like Harpak are rapidly pushing the boundaries of what’s possible by introducing AI into experiences.
“AR is really capturing the attention of customers,” he said. “Jim Hepplemen, the CEO of PTC, is fond of saying ‘AR is to front-line workers what Microsoft Office has been to white collar workers.’ Visual instructions are not only absorbed better by people, they overcome language barriers — a huge factor in today’s production environments.”
And now, companies like Harpak are making it easy for producers to adopt the technology.
OEMS, Bauer said, are in many ways the companies best suited to provide an AR solution.
“We have access to the CAD drawings of the machines themselves — which are proprietary and required to develop many of the machine-related experiences. We know the machine itself — and how to maintain it. And the skill sets required to build an AR experience are software oriented. Not a skill set you see in many manufacturing environments.”
Having sold and “pushed and prodded” numerous OEM customers over the years to modernize their offering and create value outside of pure cost savings, he added, Harpak now has the opportunity to put those strategies and plans in place.
“Digital transformation is reinventing manufacturing and production on many levels. Harpak-ULMA is uniquely positioned to customers connect their digital threads at the most fundamental level — the plant floor.”
This article is an excerpt from the December 2023 issue of Supermarket Perimeter. You can read the entire Augmented Reality feature and more in the digital edition here.
