New HoloLens 2 gives Microsoft the edge in the next generation of computing

Chelsey Potts felt so much pressure to succeed when she first stepped from the manufacturing floor into the cab of a Kenworth truck.

“When you get a job at Kenworth, most people are doubling what they made at their last job. And, you know, they’re nervous,” she said. “I have a 5-year-old and a 10-month-old, and saying, ‘Mommy’s not going back to work the next day,’ that’s my biggest fear.”

Her new job — outfitting the interior of a sleeper cab — required her to perform an intricate sequence of mechanical and physical tasks that were challenging to learn at first. For workers wielding tools in complex industrial settings, flipping through training manuals or consulting a screen takes focus and hands away from the work.

That’s not the only downside to a paper manual. Few people’s brains are good at translating between paper instructions and three-dimensional objects. New employees worry about slowing production with questions. Even when watching someone else install the bed that allows long-haul truckers to sleep comfortably, it was hard for Potts to see exactly where the other worker was putting his hands or figure out where the screw holes were.

Today, Microsoft is introducing HoloLens 2 — the next generation of its wearable holographic computer — with an integrated suite of new mixed reality services, out-of-the-box apps for businesses and sensors with the capacity to perceive and predict.

It’s exactly the device that Potts said she would have loved to have when she had that nerve-racking experience of first learning how to outfit a sleeper cab.

“If I had had this when I was trained, I would have been less nervous,” she said. “This shows you where the tools go and which way they turn and all the things you can’t see under a truck. It’s just there — step by step — however you want to learn.”

Her employer PACCAR, a global leader in the design and manufacturing of commercial trucks, is one of the first companies to test Dynamics 365 Guides, a new mixed reality app also released by Microsoft today. It allows companies to easily create heads-up, hands-free holographic training materials for employees so they don’t need to flip through a book or consult a search engine to get information they need.

Also today, Microsoft is releasing a new Azure Kinect device providing developers new possibilities for creating AI-powered experiences. Azure Kinect combines the same depth-sensing camera technology found in HoloLens 2 with a circular microphone array and color camera and works with artificial intelligence services in Microsoft Azure. It enables developers to build new perception capabilities like identifying when a saw is operating dangerously based on the sound it makes, enabling robots to judge distance when packing pallets or identifying which item has been selected from a store shelf.

These new technologies are powered by intelligent services that can perform computations wherever it makes the most sense, whether that’s inside a device — so it could quickly spot unsafe conditions — or in the cloud, where virtually limitless computing resources can tackle complex problems. In short, this is the intelligent edge and intelligent cloud made real, Microsoft says.

Microsoft also says these mixed reality and perception tools will make it more practical for companies to adopt an entirely new wave of computing that bridges the digital and physical world. It’s only been made possible by recent advances in the intelligent cloud and at the intelligent edge — a diverse array of increasingly connected sensors and devices in everything from home appliances to warehouse floors to HoloLens 2 that can offer instantaneous insights about their surroundings.

“We are now in a place where this technology is solving real-world problems. You can really begin to see what this new wave of computing looks like and how it translates into real business outcomes, and I love that,” said Julia White, Microsoft corporate vice president for Azure marketing.

From flat screen to hologram

Until now, most people have experienced computing through a flat piece of glass: laptop, computer monitor, phone, tablet, video games on a TV. Microsoft’s mixed reality offerings draw digital information out of rectangular screens and allow people to interact with holograms in physical space. These can exist independently, like a three-dimensional rendering of a human heart that medical students can grab, resize and rotate to see all the structures clearly. Or they can relate to physical objects in the real world, like instructions superimposed on a furnace that show you how to change a filter.

Microsoft says the new HoloLens 2 provides a far more immersive, instinctual and comfortable experience for first-line workers whose hands are occupied by physical tasks. It can help them diagnose a problem with a jet engine or access step-by-step holographic instructions to assemble an electric bus battery. The person using it can go backwards to double-check a step with a nod of her head or a voice command. The person is able to see in three dimensions — on the physical equipment she is working on — precisely where each screw needs to go, or what direction to turn a ratchet.

Compared to the first generation of HoloLens, HoloLens 2 also offers new features like the ability to grab and rotate holograms as you would a real object rather than having to learn specific gestures. Eye tracking can sense when someone’s eyes land on a particular part of a machine and call up useful digital information about it. Words automatically scroll as you read. The end result is like going from watching a cartoon flip book to the truly immersive experience of actual cinema.