Back in November, we reported that Microsoft was sending a couple of its HoloLens augmented reality headsets up to the International Space Station, and a new video from NASA allows us to see them in action. The device simply screams its potential in that kind of environment, but for all that, we mainly get to see Scott Kelly lounging around on and making the first Skype call to Earth from space on February 20.
Not exactly the flashy stuff of modern science fiction, but it shows the tech is largely working as intended. The HoloLens is the key component of NASA and Microsoft’s Project Sidekick, which, in the words of project chief Jeff Norris, empowers “astronauts on the ISS with assistance when and where they need it.” Unlike virtual reality headsets, which largely shut users off from the outside world, augmented reality devices like the HoloLens enhance what we see in the physical world with overlays and similar holographic projections.
Kelly’s anticlimactic Skype call, in fact, demonstrates the effectiveness of Sidekick’s “Remote Expert Mode,” which allows teams on the ground to directly help an astronaut repair the ISS rather than having to rely on mere written or spoken instructions. With the HoloLens and Skype (itself a Microsoft property), the experts in question can circle objects and make annotations within the astronaut’s field of view.
Or put it this way: Experts can directly point out what needs to be done rather than wasting time on questions like, “You mean the red button or the green button?”
The video gives only a brief glimpse of the potential of Sidekick’s “Procedure Mode,” which places holographic information over real-world objects in an attempt to simply training by making the process more immediately hands-on. Here, we only get to see a wall overlaid with the text-only instructions for installing a CBM Controller Panel Assembly, a device used in berthing sequences.
For his part, Kelly was impressed with the technology, even if his response sounds about as riveting as watching his Skype chat.
“We messed around with it for like two hours,” he says in the video, “and immediately I sensed this is a capability we could use right now.”
The video also shows how NASA tested the device on NASA’s “Weightless Wonder” aircraft and the NEEMO underwater space station before shipping it out to space. In time, NASA hopes the technology will be useful on missions to Mars by allowing scientists back on earth to investigate objects the astronauts are looking at.