Most people will have a huge smiley on their face when you show the HoloLens to them. With way over a hundred demos behind me, only 3 or 4 didn’t agree right away, that mixed reality is the future.
So, demonstrating HoloLens is a very grateful job. But it can be pretty frustrating, too, because:
- You can’t see what the user sees, and can’t offer help or explanations;
- If there are more people in the room, they’ll be bored as they have no idea what the current lucky person is experiencing.
Luckily, Microsoft has added a way to wirelessly project the so-called Mixed Reality view from the HoloLens. This displays the GPU-generated holograms that the user sees over the video coming from the RGB camera, and streams it real-time to a computer.
The problem is, that to get this streaming right, you need to have a lot of things working together. Most of all, you need a fast and reliable Wi-Fi connection. But even then, there is usually a delay measurable in seconds, often as much as 5-6 seconds. This makes it extremely difficult to explain what you’re seeing on stage (because you have to explain what you saw 5 seconds ago, which what the audience sees right now). And when you’re trying to help somebody, it can be downright frustrating, because you say stuff like “yes, there is is… oh wait, there it was 6 seconds ago, move back… not there… the other one… now air-tap… let’s wait a little until I can see that you successfully airtapped…”. In a business scenario this even comes along as unprofessional, and can make the HoloLens look like it’s not ready yet.
To illustrate the delay, I put on the HoloLens, and launched the Mixed Reality Capture Live Preview in the HoloLens Device Portal. Then I opened up and closed the Start Menu. Conditions were fairly good, so I “only” experienced a 4 second delay:
After months of experimenting, and dozens of demonstrations, we at 360world found a way to reduce the latency significantly. This has worked for us a dozen times already, under varying circumstances, including our own office, a client’s office and even in very busy conference locations. And the even better news is that it is easy to implement.
Step 1 – Use Windows 10 Anniversary Edition
You should use Windows 10 Anniversary Edition on the computer you want to stream the Mixed Reality Capture to. (you can just use this computer or a projector to share the results with a larger audience). The reason is that for Step 3, you need a new AE feature.
Step 2 – Enable Mobile Hotspot on the Computer
This is a new feature in the Windows 10 Anniversary Edition. You can access it from Settings / Network & Internet, and Mobile Hotspot:
As you can see, the computer has to be connected to the Internet (and has to have a Wi-Fi adapter) for the Mobile hotspot to be enabled. If you can’t see the above warning, all is OK – turn on the “Share my Internet connection with other devices” checkbox.
Step 3 – Connect the HoloLens to the Mobile Hotspot on your Computer
And here comes the trick: once the hotspot is set up, you need to connect your HoloLens directly to the hotspot:
Now air-tap on the Advanced options link, and take a note of the IP address of the HoloLens. You will need it soon.
Step 4 – the HoloLens Windows App
For the best streaming results, forget the device portal. What you need is the Microsoft HoloLens app from the Windows Store. This app has most (but not all) of the features of the Device Portal, and seems to perform much better when it comes to live streaming the Mixed Reality Capture.
Once you have the app installed, click on the + button, and add your HoloLens to the form. This is where you need the IP address of the HoloLens you – hopefully – recorded earlier:
You should now see the HoloLens you just added, and hopefully it is online. If not, make sure that the HoloLens is still turned on. You may need to wait few seconds before the HoloLens app can detect the device.
Click on the connected HoloLens, and
Step 5 – Enjoy!
Click on the first icon called “Live Stream”, and Live Stream should start. To further reduce latency (and not to cause audio issues), you may want to turn off the audio in the … menu.
Here is the result:
As you can see, the latency is well below 1s, in fact, about 0.5s! This is more than satisfactory for any kind of live demo, or helping a first-time HoloLens user.
In fact, depending on how noisy the airwaves around you are, you can even switch it to High Quality mode from the default “Balanced” setting.
This solution is still not foolproof. The connection is wireless, and packet losses occur that can add more and more delays to the stream. If this happens, just navigate away from the live stream and quickly come back, and you’re good as new.
To sum up:
- Use Windows 10 Anniversary Edition
- Turn on Mobile Hotspot
- Connect your HoloLens to the Mobile Hotspot on your computer and make sure you know the IP address
- Install the Microsoft HoloLens app on your computer, and connect it to your HoloLens
- Use “Live Stream” and tune it as you like.
Enjoy demoing HoloLens!
11 thoughts on “HoloLens Mixed Reality Streaming Done Right”
This is very helpful, thanks ! ^_^
is there a way to use the live-stream PLUS recording at the same time? Seems either recording or live streaming works. Thanks! Phil
As far as I know, you can only do one of those at any given time.
As of late, virtual and augmented reality devices have become the new tools of the future. For years classic movies like The Matrix, Tron, Gamer, and even Total Recall have left viewers with the yearning to escape or just experience another world. In fact Nintendo created one of the fist attempts for this reality with the virtual boy which was a little before its time and was not appreciated. As of today in the gaming world we have quite a few devices for virtual reality Oculus Rift, HTC Vive, Playstation VR and a few other. These devices aren’t as immersive as the movies aforementioned but users still have the chance of loosing themselves in these realms to an extent. Even with this small success i fear even today’s version of virtual reality video games will not have the longevity true fans hope for.
Microsoft has taken a different approach to virtual reality with this augmented reality bringing the false world to you instead of taking you to a false world. I suspect this new reality will have the longevity VR dose not because people can use during there everyday lives. Anyone with a single creative bone in there body will get the full experience out of the Hololense device. The biggest reason this device will surpass the others is because it doesn’t immerse the user in virtual reality taking them away from everyday responsibilities in truth it enhances the physical world making everyday activities, duties, and pass time more achievable and tolerable. My only worry is the $3,000 and $5,000 price tag if the device stays at such a ridiculous price regular consumers will be less likely to purchase. For Hololense to succeed the price must be deducted tremendously.
LikeLiked by 1 person
You saved my day! Had to do a presentation of our brand new HoloLens and the stream kept stopping all the time, until I found this article and everything went perfect! Thanks a ton!
Excellent write-up! Thanks for the tip!
Thanks a lot, however I am unable to install the Microsoft HoloLens App you linked in your article. In the Windows Store App it appears to be “currently not available”. I’m not familiar with the Store though, so maybe something else is wrong… My system meets the requirements, I’m logged in with my MSFT account, afaik the app is also free… What else can be wrong so it appears unavailable? Or did MSFT remove it? What’s weird is that, when I enter “Microsoft HoloLens” into the Store App’s search field, the App does not appear in the search results. Can anybody help me out?
LikeLiked by 1 person
The app is still in the Store for me. It may be limited to the US region, so try setting your windows region and language to US English.