How to build a holodeck, week 19

This week has been a bust, from the point of view of progress on the holodeck. No pictures or videos until next week - sorry!

Birthdays, colds and toothache

It was my youngest son’s 3rd birthday early this week, so my wife and I spent the day enjoying his company. I’d sort of budgeted this time out of my plan this week, and we had a lovely time. Unfortunately, he’d picked up a cold at the weekend, which then proceeded to devastate my missus (who’s still in bed recovering as we speak). I got a mild dose, but then picked up a tooth infection on Tuesday, and by Friday it had become so painful I decided to get the tooth removed. I have history with this particular tooth - it’s my lucky golden tooth, I’ve had it rooted a few times. None of that helps when the whole side of your head is pounding and you can’t sleep, so out it came. I’m just about at the point now where I don’t need to be on pain meds 24 hours a day, but it’s been a pretty grim week all in all.

If I was working for a salary somewhere, I’d be comfortably resting up, doing my best to get better. It’s very hard to keep that mindset when you’re working solo, setting your own deadlines and timescales. Mentally, I feel like it’s been a wasted week (apart from the birthday trip, obviously!) even though I’ve been in no fit state to do anything other than try and keep the kids alive while the missus rests up.

State of play

As I’ve not managed to get any actual “work” done this week, I thought I’d briefly summarise the state of play as I see it, regarding the hardware, market and technology as it currently stands.


For those who’ve been following along since the start, you’ll remember I mentioned three key areas - displays, input and motion. Hardware cost and capabilities really are the main contributing factor to these three areas right now.


The Tango + Dive combination I’ve been working with is “ok” for prototyping, but after a few month’s experience and trying it out with a variety of volunteers, it’s obviously not a viable form factor for long term (or even medium term, e.g. 10+ minutes) use. The Vive is great as a display, the Rift is also good (minus the god rays) and the other mobile addons/clipons I’ve tried are all pretty ropey. I think I can comfortably say that mobile is not there yet, but it should be there in a year or two, and Rift/Vive are good enough, but there’s plenty of room for improvement.


Regarding input, the Vive wands are the best solution that’s on the market right now. I’ve tried Leap Motion attached to both headsets, and while it technically works, even with the latest Orion drivers, it’s just not stable enough to really be usable as the primary input mechanism. On Android, it’s a long way away from good enough right now, and while I’m hopeful that some form of hand/gesture recognition will become viable, now is not the time. Realsense, Kinect and various other depth-to-gesture sensor combinations are all on the market, and ultimately, they all suck. They’re like speech recognition was a few years ago - good enough to look like they work, until you actually try to use them, at which point they fail in annoying and glitchy ways, which mean they’re actually less viable than the tried-and-true alternatives.

Give it another few years, and the compute / algos should catch up to the aspiration here, and maybe the hardware for the scanners will get good enough to do a proper job, but given the amount of research still happening in this space, I don’t think it’s going to be this year or even next year before they are viable.

My Rift has been gathering a lot of dust compared to my Vive, for two reasons, and one of them is that the input mechanisms are crap compared to Vive’s wands. Hopefully Touch will bring balance to the force here.


Moving in the VR space while you’re stationary in the real world sucks. We can all do it, and some games (driving sims, etc) it feels natural, but honestly, this is the second reason my Rift has been gathering dust - I haven’t found seated VR remotely as compelling as room-scale VR experiences.

Vive works, tracking is stable (for the most part, certainly 98%+ rock solid) and the play space is big enough for a decent amount of moving around in a constrained play space. I’m not a big fan of teleportation mechanics, but I’m a much bigger fan of walk/teleport than I am of sitting still while moving in VR using a joypad.

Tango tracking works for the most part, but it’s just not stable enough to compare to the tracking accuracy you get from the Vive. I really really wanted Tango to be the magical all-in-one solution here, but it’s simply not strong enough yet to be a viable alternative. It will be soon - the question for me is, how soon? Are we talking this year, or next? The first true consumer Tango device (the Lenovo Phab 2 Pro) is out in a month or two - if that makes a splash, then hopefully the market will expand a bit. My gut feeling is that it’s not going to sell like hot cakes, but I really hope I’m wrong.

What I actually want is a proper, mobile form factor headset with Tango built into it. I guess we’ll be waiting until next year for this - I’m assuming that Google have enough sense to roll together the VR and Tango teams into one product group and start bringing all this technology together, but then they also need to nurture the ecosystem, so maybe one of their partners will be the ones to jump in here first. I guess we’ll see.

I’ve not had a chance to try any of the omni-treadmills yet, but from all the reviews I’ve read, they’re just not good enough yet. They don’t accurately track your actual foot motion (or they turn it into a linear input, rather than a properly tracked motion curve). Jumping / crouching etc don’t really work if you’re strapped into a harness and standing on your feet. I’m not going to write them off until I’ve actually tried them, but I’m not aware of anyone I know who’s got one.

Using Kinect for tracking motion of people and joints is working, but again, it’s just too flakey to be something that is worth investing heavily into. The market for the devices is sort of there, there’s a lot of folks with XBox one kits that will have a Kinect (likely gathering dust somewhere) - but no-one that I know of will be bothered to set it up to use in conjunction with e.g. Rift.

So, overall - Display is basically there, device-free input is still a ways away, hand-device input is probably going to be solved next year. Motion is still a long way from being solved, and that won’t change next year imho.

The market

I’ve been tempted to get something “to market” to get some visibility (and potentially some cash). This raises the questions of, which market? Which device? Which product? My initial thoughts were that Tango would be a great space to hit as soon as the market penetration for the device was large enough (250K+ devices, would be a good pool). I’m not sure if that’s going to be this year or next year now.

The Rift and the Vive are both selling around where I’d epected, so I’m guessing there’s maybe 100K devices in consumer hands, and there should be around 200K by the end of the year. I think the PSVR will probably sell well, but I doubt it’s going to be multiple millions, mostly due to their not being any proper killer apps yet.

Gear VR is a larger market, but I’m not really interested in the type of content that’s going to dominate that space (no motion tracking, etc) so while I’m happy to see it grow, it’s not the core space I want to be creating product in.

So, we’re talking about maybe a couple of million devices as the market space over the next year. Not too shabby from a standing start, not earth shattering. Probably not big enough yet to look at spinning up a big team, but probably big enough to cover costs on some smaller products made with smaller teams on a tight timescale.

The technology

What’s out there is cool, undeniably so. An awful lot of the toys I’ve played with recently are just that, though - they’re toys. Proof-of-concept, rather than the tools I was hoping them to be. The consumer-level laser scanners are too short range/inaccurate, the Tango dev kit is likely more powerful than the first gen or two of actual consumer devices, Leap Motion is wonderful in theory but not stable enough to rely on as primary input, and both the Rift and Vive have big bloody cables coming out the back of them.

My gut feeling is that the next year or two is still going to be discovery-time regarding what’s actually required for a proper consumer VR experience (which IMHO means mobile, markerless inside-out tracking to mm accuracy, 90 fps 2K display and device-free input). Theoretically all of this is possible right now, but no-one is building that unicorn device just yet.

AR is a different beast, and I’ve heard fantastic reports of the hololens. I might have to acquire one to see if it’s as awesome as it sounds. I’m assuming Magic leap is still years away, given that there’s still nothing better than a few videos showing what it might possibly look like some day.

So - early days. Hopefully we’re not still a decade away.

Next week …

Hopefully I’ll be through the other side of the extraction, and back to work on getting the holodeck prototype version 1 wrapped up (human avatar with kinect tracking over the network between multiple devices on Vive and Tango).

Written on July 3, 2016