Like every self-respecting enterprise software analyst, I love tech for tech’s sake but keep the need for real world affirmation as my lodestar for all supposed innovations (such as blockchain, for crying out loud). So when I first saw a HoloLens four years ago I was sorely tempted to beg ,steal, or borrow one, anything to get my hands on this mixed reality headset/PC/remote device. Even though it was clear at the time the enterprise use cases were more speculative than real.
That’s changed dramatically in the last year, as Microsoft has begun rolling out a series of enterprise-class apps that showcase the spectacular capabilities of mixed reality in the enterprise. The culmination of Microsoft’s efforts came with the rollout this week of HoloLens 2, a significant upgrade that will please nerds and non-techys alike. The result, in a nutshell, is that if there’s a strong business case for hands-free work that involves team or collaborative interactions blending the real world with the virtual one – such as repairing, operating, training, and designing processes for manufacturing shop floors, warehouses, retail stores, or, wait for it, the battlefield – Microsoft’s mixed reality headsets are going to have an impact.
How big an impact is the big question. Thankfully, the US Department of Defense is happy to show us the way.
An interesting sideshow opened up last week in the HoloLens story that actually provides a better guide to HoloLens’ future than Microsoft’s carefully managed H2 release. In case you missed it, a bunch of Microsoft employees – nowhere near a measurable number relative to the company’s 200,000+ worldwide employee base – signed an open letter imploring CEO Satya Nadella to halt a contract for up to 100,000 HoloLens to be delivered in support of the US Army’s Integrated Visual Augmentation System, IVAS. This use of tech in the service of the military was deplorable, claimed the letter’s signatories, and should be halted.
While the debate about the ethics of technology and warfare is not to be discounted, the debate is misses an important truth that goes far beyond H2, this contract, and the signatories’ heartfelt letter. That truth is unequivocal: the Department of Defense has been the handmaiden of modern technological innovation for pretty much as long as the DoD and the tech industry have been around. So why should HoloLens be any different, when the internet, computer graphics, AI, telecommunications, the cellular phone network, and a million other innovations were aided and abetted by the DoD? Indeed, let’s be even more frank – these innovations became even more innovative precisely because of the DoD’s funding and willingness to go out on a limb to see what new tech can add to its missions.
That’s why the number of HoloLens that are being ordered for this project, 100,000, is so significant. It’s one thing that HoloLens is being supported by global companies like Chevron that are planning on deploying a few hundred HoloLens along with the Dynamics 365 Remote Assist app. It’s entirely another that the DoD wants to buy a few order of magnitudes more of the devices. Just the investment in manufacturing and supply chain needed to fulfill the DoD order is huge for a product that had significant manufacturing and supply chain problems in an earlier version (delaminating headset visors) and was even at one time criticized at the highest levels of the company for being yet another hardware product that didn’t make sense for a software company like Microsoft (think Nokia acquisition.)
So whatever Satya Nadella said about supporting democracy by selling to the DoD needs to be filtered through the potential delivery of those 100,000 H2s. Say what you will about the excesses of government procurement and the morality of contributing to the often immoral militarization of technology, IVAS is about to put HoloLens on the map. And how.
A look at some of the IVAS documentation provides a compelling picture of how HoloLens will evolve to be an effective part of the Army’s repertoire of ways to kill without being killed. Again, you may not like it, but the highlights of where IVAS wants to go with military uses of HoloLens are worth taking a look at. Because herein lies the likely evolutionary path of HoloLens in the commercial world, one that users like Chevron will look back on in a few years and be happy they were there when the bandwagon got rolling.
Because this document is a treasure trove of potential ways in which HoloLens may be upgraded beyond version 2, I’ll just cherry pick some of the more interesting highlights. By cherry pick I mean severely limit the immensely cool number of capabilities the DoD wants to add to HoloLens under this program. Really, this is going to a fun product evolution to watch. If you’re a fan of VR and MR films like “Ready Player One,” you’ll see how close this future could be.
Let’s start with support for multi-user capabilities: IVAS is clear that the program will develop the capability for increasingly large units – squads, platoons, companies, all the way up to battalions, which can range from 300 to 800 individuals – to work together in the field. This includes support for a range of collaborative activities such as training, tactical planning, navigation, tracking, and threat detection. In other words, pretty much the stuff that armies on the go tend to do, only with a HoloLens attached to their helmets.
Aiding this mission will be things like 3- D sound, full night vision, 3-D navigation, thermal target recognition (with a threshold of 80% reliability up to 500 meters, a quarter-mile for you non-metrics people, according to the spec). The document also calls for a group communications capability so that multiple users can talk to one another simultaneously. (Note to the DoD – do NOT let Microsoft convince you to use Skype for this. Trust me on this one. Teams would be soooo much better.) There’s even camera-based foreign language translation. And sensors, lots of sensors.
The 48 page document goes on and on, spec after spec, scenario after scenario. One thing you can say about DoD procurement – they sure know how to dream the big dream when it comes to tech innovation, and with the acknowledged fact that our Army is today best equipped to fight the wars of a previous generation, there’s no doubt that the government purse strings will be opened wide for this and other similar projects.
So where does that leave HoloLens? Did I mention that DoD wants to add all this innovation and still keep it weighing in at 1 ½ lbs, which is pretty much what HoloLens 2 weighs out of the box?
IVAS effectively positions HoloLens as the true future of wearable computing, and opens it up for supporting a collaborative work environment that’s an order of magnitude more valuable than the one on one collaboration examples shown in the demos for Remote Assist and the other Dynamics 365 HoloLens products rolled out in the last year. While it’s clear the Microsoft’s is dipping its toe into H2 with a clear understanding of its potential, it’s fascinating to see what an unfettered vision, placed into the public domain by the transparencies of government procurement, can show us about HoloLens’ future.
One interesting potential is that HoloLens will effectively make its wearer a pivot point in the growing IoT-based asset maintenance world, which is where Remote Assist is already headed. A sensor-laden H2 will effectively make its wearer a human IoT end point: The future of H2 will allow a human (and, obviously a machine as well) to detect problems with fixed assets that can be sensed using such techniques as infrared spectrum analysis, audio frequency analysis (a great way to know if a bearing in an engine is about to give out), and pattern recognition in order to identify out of spec behavior in a machine that the human eye or a simple sensor array might not be able to detect (at a threshold of 80% reliability up to 500 meters, remember? Meaning the human would be able to keep a very safe distance from, say, a train wreck full of potentially hazardous materials or an oil pipeline that’s been visibly damaged and still do her job.) The kind of self-detection, diagnosis, and suggested action capabilities the DoD wants for the battlefield will come in handy for the field service repair person who will merely have to “look” at a piece of equipment in order to figure out what’s wrong and how it might be fixed.
The human language processing capabilities also would be amazing in the civilian enterprise world. Imagine having a call on a device that could simultaneously translate a foreign language-speaking team member’s conversation so that everyone could be understood. In fact, the whole notion that a collaborative conversation could take place inside a mixed reality environment would take team collaboration to an entirely new level.
At some point it’s possible to imagine that HoloLens will genuinely redefine the work world by blending the virtual and real world to such a degree that one will not be able to exist without the other. At which point HoloLens won’t just be the go-to tech for the hands-free world, it will permeate the business user world currently defined by Microsoft stalwarts like Windows, Office 365 and Dynamics 365. What will the old transactional world look like with this much power in the hands of an individual and his distributed team? Certainly the DoD is reimagining the battlefield this way, why not the work world too?
The downside risk for Microsoft using the DoD to lead the enterprise to the new world of mixed reality work is pretty much no different than any other tech. The relational database market in the 1980s was totally jumpstarted by its sales to the major defense contractors, a fact that even then many of these vendors took pains to hide. Ask the founders of Sybase, among many others, how hard their jobs would have been without these customers – I’m pretty sure they would agree DoD money was a key component of their success in the commercial world. I don’t see any particular taint left from that experience, and I’m assuming the HoloLens will emerge in future years without being “marred” by the boost given to it by IVAS. On the contrary.
A final personal note to the employees of Microsoft who signed that letter. Back in the mid-1980s, I was fortunate enough to take a graduate course in a nascent field called computer graphics that was taught by Prof. James Foley, one of the pioneers in the field who wrote the textbook on the topic. I acquitted myself well-enough in the class to merit a call one day from him offering me a job doing some cutting edge work at the DoD on battlefield simulations using the graphics technology of the day. I turned him down, despite the fact that he had been more than kind in letting me into the class in the first place (I was missing a few of the academic prerequisites, it’s a long story). I honestly didn’t see myself working in that world, for some of reasons evinced in the letter the employees wrote regarding the HoloLens as well as a few others that I won’t enumerate here except to say that I might not have been allowed to take the job anyway.
The point is this: if you don’t want to be involved in the militarization of tech, don’t do it. Get another job, which at this moment in time shouldn’t be hard with Microsoft on your resume. Or, better yet, take your conscience and get involved in changing the oversight process of how tech is used by governments – and private companies – that might be seen as illegal, illiberal, biased, or whatever else you might find objectional. Frankly, there’s a lot of that going on, and lord knows that lots of oversight is desperately needed. The current stink about bias in facial recognition is a great starting point. Get involved.
But don’t ask the likes of Satya Nadella and Microsoft to take a stand that no other major tech company has ever taken. Working with the military is part of the history of our industry, and even Berkeley hippies like me know that we civilians have benefited tremendously from this connection, and that, behind IVAS is a genuine mission to protect the men and women who engage with enemies we civilians would rather not even think about.
Meanwhile… now I want to get my hands on a HoloLens even more than ever before. Maybe I should ask my old professor for some help getting a job?
Bravo! Great post all around. My husband is currently serving as a physician. Imagine how wearing the H2 can assist medics in the field. The authors of the letter should think about the H2 can help save lives, not just destroy them.
I 100% agree that the medical field is a great place for H2, and, for better or worse, a lot of medical breakthroughs come to civilian medicine via the military’s innovations with battlefield medicine. I’d love to see how Dynamics Remote Assist could be modified for the surgical theater, it could genuinely save a lot of lives.