Category Archives: Development/RTsoft

Random stuff I’m working on.

Playing with the HoloLens

At an extremely spendy price of $3,000 I picked up a HoloLens.  Why so much more than a Rift or Vive?

Well, the biggest difference is instead of tethering to your computer, this thing IS a computer+kinect small enough to wear.

Welcome to the world of mixed reality!  I hope I’m keeping up with my jargon correctly

Mobile VR is trash, HoloLens is different

The current mobile VR solutions (GearVR, Google Daydream, etc) are garbage, you know why? They can’t track positional movement.  Take a step sideways or forward – in the game nothing happens.

This means there is an entire category of games they won’t work with – basically nothing where you move around a room naturally.

HoloLens is different (well, it’s AR/MR and not VR for one, but that’s not the point here), it can fully track position/rotation/acceleration, you can even do something a Vive can’t – you can walk BETWEEN ROOMS and it knows.

Without any complicated setup, you can plop this on someone’s head and it just works, anywhere.  Oh, and it’s fast.  As for tracking your hand.. well, not so fast, we’ll get into that later.

Akiko knows kung-fu

It uses multiple cameras (normal and infrared) to figure out where your walls, table, and chairs  are – and is capable of knowing exactly where your head is located in space.

It’s quick enough to feel nearly as smooth as say, a Vive. I think it’s using acceleration/rotation sensors to do accurate predictions while constantly correcting things with the camera based space-mapping but it works well despite the occasional glitch.

I didn’t expect it to track outside (it must use walls to calculate head position, right?) but it worked fine! I guess it’s scanning the ground or something.

Cool, but here are the problems

At 1268×720 resolution per eye, the objects you’ll see overlaid look great – but that’s mostly because the DPI (dots per inch) are so tiny.

If you hold a piece of A4 paper at arm’s length you’ll get an idea of how small the rendering area is.

The actual screen overlay area is tiny

This means when a spaceship tries to run away in a game, it just gets cut-off unless you turn your head to follow it.  A workaround that games are using is they will pop up an arrow “<– It’s over there!” to help you find it again.  Not great, but hey.

It also isn’t able to overlay graphics too close, like if you set your GL near plane too far away.  You can see the image break apart in the gif below.

Click this gif to view it in action. Don’t blame me if you get sick

Kinect delay is back, baby

It can sort of track one of your hands – but it’s too laggy to be of much use for anything.   It’s a better experience to move your head around to put a centered crosshair on an option, then use the included remote’s satisfying clicker to select something.

Random free idea that makes no sense due to the hardware requirements: Use with a Vive hand control to make a spray paint simulator so you can tag up your house.

Seth’s verdict

The good:

  • The wall mapping and fast, accurate positional tracking is amazing.  This is a hard nut to crack and probably the most impressive thing about the entire HoloLens project
  • Totally self contained unit makes it easier to show people.  I mean, good luck bringing your Vive to Kyoto Indie Meetup
  • It barely works, but it can sort of do smart occlusion around physical objects (if a real chair is in front of the hologram, it won’t render the hologram there) – its 3d scanning is too rough but .. it’s still tantalizing us with what the future will hold

The bad:

  • Costly, it’s squarely in “developers only” territory right now. It’s no surprise it’s only sold “thousands”
  • The tiny video overlay area is very limiting
  • Laggy-ass gesture controls are bad for most gaming
  • Can’t walk up close enough to objects, they disappear
  • UE4 doesn’t support HoloLens as an export target, I don’t think there is a way to build it without fighting with Microsoft’s hacked up version of UE4 they did a while back, no thanks

There are rumors that Microsoft has canceled the 2nd generation HoloLens and are skipping straight to a 3rd gen version slated for release in 2019. Interesting.

Playing with the Fove eye tracking VR headset

vove_headset

It came! This VR headset was on kickstarter back in 2015 and has received some serious investor cash since.

FOVE-VR-Headset-Pre-Order-Announced

Sure, that’s exactly how I use VR…

I’ve been working on prototype games with the Vive (in UE4) but really wanted this primarily to play with its unique feature: real eye tracking.  It has cameras that watch your eyes and figure out where you’re looking – and applications can act on this information.
fove_eyes
No, this isn’t from Resident Evil 7… it’s how it watches you watch.  Yes, it’s creepy.

Currently there is only one demo on the Fove website to try out – I expect more will be added soon – hopefully they will show off foveating rendering and depth of field based on gaze. I want to see if it’s all quick enough to “feel right” or not.  I guess we could always write our own tests as well… maybe later.

fove_test_app
This is the sample app.  That little green and red ball?  That’s where your left and right eyes are currently looking.

My impressions

  • I had some trouble getting it going, but after a few reboots, trying different usb ports, and removing my second monitor it kicked into gear
  • Fove is currently marketing this to “developers, creators, researchers” and I agree that it isn’t ready for the general consumer
  • It’s fiddly.  If your headset shifts on your head AT ALL since you’ve calibrated the eye tracking, it will be way off.  I had the best results if I tried not to move my head at all
  • Sadly, they don’t support Valve’s lighthouse tracking (this was something they were talking about earlier) and it comes with a single infrared camera that tracks points on the helmet. I felt it didn’t track rotation/position as well and accurately as Vive or Oculus Rift does
  • It really does work! I could totally write an eye controlled VR web browser or whatever, that rocks

Conclusion

It’s functional and is very useful to experiment with eye tracking VR technology early.  Eye tracking will probably be a standard feature in all the VR headsets soon enough.

If you just want to play some games, get a Vive.

Audiere 64 bit MSVC 2005 development binaries

I needed a Windows 64 bit development version of the audio library Audiere but couldn’t find any precompiled binaries for x64, so I’m posting them here for any future googlers.

http://www.codedojo.com/files/audiere-1.9.4-win32_win64_msvc2005.zip

Why did I need Audiere?  Well, I had switched the Growtopia 64 bit client to use SDL2_mixer and immediately had a rash of complaints from poor audio mixing to no audio at all.

Tried various bit rates, buffer sizes, and driver settings (directsound, etc) , still had weird “drunk” timings and distortion when mixing 4+ sounds.  I got it to the point of “not that bad, on my computer at least” but many players still had major problems.  So back to Audiere we go!

Versions used:

audiere: 1.9.4 (?) forked from Chad Austin’s Audiere on github
flac: 1.3.1
libogg: 1.3.2
libvorbis: 1.3.5

Note:  The 64 bit binaries are missing mod/xm support as I didn’t need them for my project and was too lazy to find and include the Dumb source.

If you’d like an easy way to compile your own 32/64 bit version for MSVC2005, I’ve thrown the entire thing including supporting libraries on github so you can build from source in a single MSVC 2005 solution in Audiere/vc8/audiere.sln:

audiere_solution

Github: https://github.com/SethRobinson/Audiere

Rift CV1 vs Vive

rift_vs_vive

Ok, I finally got my hands on the final consumer models and have given them enough play time to feel like I know what’s going on.

If you’ve been living under a rock and don’t know what these are: they are kits for your computer to enter the magical world of “Virtual reality” apps and games.

Which parts are the same between these devices?

Both are the same resolution.  Both run at 90hz and have roughly the same field of view. Both are USB hubs internally that add a ton of devices. (Vive takes one port, Rift takes two)

Both have their own store/in-between game area that allow you to change and buy games without taking your headset off.

Both screens don’t handle certain high contrast images great, like white text on a black background.  It sort of adds a glare or foggy type of feel due to the lensing structure.  Rift might have a slight advantage in the visuals.

The Rift

rift

The Rift headset is slightly lighter and more comfortable than the Vive. It has built-in (but removable) headphones which simplifies getting in and out of it.

The controllers packed with the Rift

It’s packaged with a tiny remote that reminds me of the Apple TV or FireTV remote.  It’s reported to be usable for 4,000 hours before needing a battery replacement – not surprising considering it doesn’t have a gyro, haptics, or anything else, it’s just for simple selections.

Strangely, it also comes with an Xbox One controller.  I only found it required for a couple games, most are ok with the tiny remote thing. I have to use it wired because they don’t include the wireless adaptor because I’m in Japan and apparently it uses radio frequencies that aren’t allowed here.  (Shhh, I imported a US Xbox One a while back, guess I’m an outlaw now)

Rift technology

This is a decent update to my old Oculus DK2.  We’ve now got 2160×1200 vs 1920×1080, faster refresh, built in mic/audio, and it’s lighter to boot.

You can now swivel a true 360 degrees because the headset has had tracking LEDs placed on the back too.

Unfortunately the tracking technology choice is what potentially dooms it, more on this later.

The Vive

vive2

Ok, now the Vive.

First off, the included earbuds are not a great experience.  They were constantly being pushed and pulled by the cabling which sometimes causes them to pop out.

While demoing the Vive to friends it’s especially awkward to be asked “could you put that back into my ear, I can’t because I’m holding these controllers”.  No way, do it yourself!

I got rid of them (the earbuds, not the friends) and am using a pair of low latency wireless headphones instead, works nicely.

The controllers packed with the Vive

These are what really make the experience something special.  The accuracy and tracking are so good you can toss one up in the air and catch it again with only the VR visuals.  I’ve noticed no jitter or occlusion issues.

Vive technology

Unlike with the Rift, you have to Boba Vila it up a bit and mount the two Vive sensors in opposite corners of your room. (actually, these little boxes just spray non-visible light around your room, it’s the devices you’re wearing/holding that do the actual sensing)

Note: When the “lighthouses” are on, they screw with other IR devices you might be using.  For example, I can’t control my room lights.

Don’t cut corners during the mounting because the moving pieces inside cause these things to slightly vibrate which could cause a shift in position.

My “room VR” space is pretty sad.  Have I mentioned I live in an apartment in Japan?  I now have a garbage can on top of my refrigerator, to give you an idea of the tetris-like compression wonders that had to be achieved to make this possible at all.

With the Oculus DK1 and DK2 I was excitedly telling people “we aren’t quite there yet, but this is going to be amazing someday”.  Kind of a “Marty, your kids are going to love it” thing.

Well, we’re there now, folks.  If you can handle the discomfort of wearing what is essentially a tethered scuba mask, it’s now possible to get your mind blown in VR.

Rift vs Vive

Vive easily wins the VR wars for now because it can also support room scale. You just don’t get sick when playing “room scale” VR (content designed for you to walk around on a 1 to 1 movement basis).

Also, the motion controls being available now would also have put it into the lead, they are a must for VR.

We can’t quite knock the CV1 out of the running yet because soon they are going to release Oculus Touch which includes another sensor and motion tracked controls.  It remains to be seen if it will accurately track at room scale at the quality the Vive does, or even if they suggest trying to set things up that way.

I’ve read that games like Job Simulator and Fantastic Contraption are being re-tooled to be “forward facing experiences” for the CV1 and its new controllers, so this points to room VR/360 degree motion control play not being a main focus for the CV1.

Unfortunately, even assuming the CV1 will eventually be able to do room scale VR there would be a fragmented market between “People who don’t have the new Touch controls”, “People who do but put both sensors on their desk” and “People who tried to setup the sensors for Room VR”.

Rift store vs the Vive store

The Rift store currently has no user rating system so it’s hard to know what the best software is.

Vive uses Steam so of course it has a top notch ratings/community systems. (quiet in back, yeah, it’s not perfect, but much better than no ratings!)

Ease of Development with Vive and CV1

ue4_v4_scene

I had a simple UE4 scene with Vive + Motion controllers being tracked working in 15 minutes using this tutorial.

I found motion controller models included with my normal Steam install here: C:\Program Files (x86)\Steam\SteamApps\common\SteamVR\resources\rendermodels\vr_controller_vive_1_5

On import, I rotated them by 90 degrees on the Roll and Yaw and they perfectly matched.

I unplugged the Vive stuff, plugged in the Rift, checked the “Unknown sources” button in the Oculus settings, restarted UE4 and ran the same test app – it worked first try! I was only 3 inches tall but hey, that could be adjusted.

I don’t know anything about distribution builds or how hard it is to get listed in the various stores, but hey, at least it’s easy to play with out of the box.

Conclusion

Vive rules.

Planning to post some reviews of VR games/apps tomorrow.

Do you have any VR stuff? What do you think?

Try Rogue-Life, a tiny free game I made

rogue-life

Ludumdare is a game jam where they announce a theme and you make a game over the weekend just for fun and practice.

It’s a fantastic way to get your game design juices flowing as hey, no matter how bad you fail, it was only a couple days.  The time-pressure if only having a few days and HAVING to release whatever you have is great for those that like to quit projects halfway, like me.

About the “game” I made:

Rogue-Life is a weird experimental thing that crosses a board game, a rogue-like, and the hollow illusion of controlling that random dungeon of existence we’ve each received.

Can you find meaning before you “Grow” old and die? There are 5 of them to find.

Tips:

  • You are always aging (even while standing still), and you will eventually die
  • Don’t waste your life on the wrong jobs, different jobs give different emotional benefits as well as pay. For instance, a Soup Kitchen pays $0 but raises your Self Acceptance
  • Some lives cannot be happy or successful no matter what you do
  • Overcoming an obstacle (the pad-locked doors) give a bonus to all stats
  • Your stats change with age. For instance, your Physical will go down as you enter old age, and your Self Acceptance takes a hit in your teenage years
  • Meanings are marked by trophies icons – you can zoom out with your mouse wheel to see where they are easier

I’m not ever sure if it’s possible to find all “meaning” in a single life before dying. 3 out 5 is my best.

Post-mortem:

Good:

  • Well, I made what I set out to do
  • It’s funny in weird and unexpected ways, like to accept lack of control and chaos (one of the meanings) you may need to murder someone
  • “Pushing” against a job to continue working there is a nice easy way to control things
  • Got a bit more comfortable with Unity and WegGL

Bad:

  • It’s kind of boring. You end up pushing against everything without even reading what’s happening
  • The more text and numbers I removed, the better it plays. But don’t feel like doing that
  • The random stuff is way too random, each life is completely different and some don’t make sense. Hey, maybe that’s fitting though

For the daring, you can play the game in your web browser here.  Probably doesn’t work right on mobile as you need arrow keys to control stuff.