A talk about EyeSkills at the 35th CCC!

After a stressful Christmas preparing this talk, it finally happened.

Here’s an introduction to EyeSkills, what we’ve achieved, where we’re heading, and what we hope to achieve!

Perhaps the most surprising thing about the talk was the resonance it created.  We’ve had over thirty people sign up to help as volunteers, over 4 thousand views of the uploaded video, and for a couple of days we were at #7 on HackerNews.  Some people are just too kind 😀

The most interesting and important thing of all, was listening to the many people with some form of Lazy Eye share their experiences after the talk.  It’s confirming a suspicion that’s been growing for some time, that the term “Lazy Eye” is intellectually far too lazy.  There is a large amount of variety in the different symptoms individuals experience, but these seem to be clustering so strongly that it seems possible we are looking at a range of individual disorders which have their own specific differences, under the umbrella of “Lazy Eye”.  It is becoming a priority to understand these groupings, so that we can develop more effectively for each specific scenario.

EyeSkills at the HACKademy

I’m very happy that EyeSkills has been selected as one of the four projects which will be represented in Berlin at the world’s first HACKademy!  A team of volunteer specialists will be working to develop our ideas for a VR open-hardware prototype for EyeSkills this March  Here’s the flyer!

Hackademy – hacking Google Cardboard V2

Rather than implementing a new VR Headset for EyeSkills, could we push the price-point down even further, by hacking the standard Google Cardboard V2 design?  That’s one of the questions we explored last weekend at Careable’s first Hackademy!

The standard design has a capacitive button which touches the screen as user input when the user depresses an “origami” lever on the right upper side:

In the manufacturer’s schematics it becomes clear that this is a separate unit…  so after pulling it apart…

…we figured, why not replace this with a new core capable of supporting on-device eye tracking?  Here’s the first mockup.  The tabs at the bottom represent the cameras, the fuzzy tubes are cables, the yellow foam blocks are the USB connectors, and the purple block is the USB hub (all to correct size).

This could be folded back into the existing headset to give us just what we need!

I’m now moving towards programming a parametric model of this interior component which we can prototype using the Hacker Space’s laser cutter.  Very much looking forward to going back there this weekend!!!

At the same time, other members of the group have been exploring how to modify a standard v2 cardboard to improve both the ergonomics and reduce stray light entering the viewing area (after all, people’s foreheads and noses vary more than you might realise until you start looking closely!!!).

Here we are in Potsdam’s wonderful little Maker Space…

And here are some of the wonderful team explaining what we’re doing to members of other teams at the Hackademy….

It was an inspiring weekend, but 7am starts and getting back to bed at midnight after solid work three days in a row have taken their toll a little!

The second Hackademy weekend.

The energy, the passion, the determination of this team never ceases to amaze me. More amazing progress.

Let’s start talking about hardware:

My focus was on continuing the work to produce a new central component for Google Cardboard (v2) capable of supporting eye tracking.

This is an intermediate design – laser cut from card, with one of the first 3D printed custom camera holders mounted and containing the internals of an endoscopic camera.

Laser cutters are a *lot* of fun!

Here’s a closer look at the holder –

A later iteration has mounting slots and a redesign of the cardboard so that (with the help of a 3D printed tool) the mounts are always installed in the right direction and in the right location.  The channel/slot running lengthways up the middle is there to allow heat from the metal backplate of the camera and PCB to escape.

Designing these parts in OpenSCAD (they are fully parameterised so we can cope with different camera parts or headset designs) was a bit of a headscratcher at times.  It took several iterations and alternative designs until I was happy.

…but I’m actually really pleased with the OpenSCAD description.  I built it so it both shows the flattened cut, but also applies all the correct transformations to show what it would look like folded – very helpful when seeing what works best!

At the same time, Moritz was busy working out how to remove the IR filters from the camera. He ended up making a custom tool to unscrew the camera lens assembly, and had to break out the IR filters using brute force :

He also custom shortened and re-soldered the camera cables so we could mockup the prototype, and added an IR light source to replace the LED rings on the original cameras – resulting in our first IR shot of an eye!

Of course, we only managed to get that result because of the hard work put in by Johan, Andre and Rene (left to right) on interfacing Android with those cameras. The two main sticking points which are left are how to address both cameras simultaneously, and integrating OpenCV with Unity.

At the same time, what use is all this when a google cardboard is so uncomfortable and poorly fitting, letting in light and disturbing from what needs to be a concentrated experience?

Asieh’s incredibly intuitive and empathic design sense, coupled with Cong’s ability to synthesise many (often) divergent requirements resulted in what I think is a truly inspiring solution :

This is an add-on to a GCV2 which is amazingly comfortable, fits all heads, and blocks all light!

I also liked this “starwars” style prototype :

So, with hardware and ergonomics, we come to the other essential component which makes for a good experience… the User Experience!  Flo (sadly, no picture?!?!) has an inspirational mind in a super relaxed soul.

After many boards full of analysis :

he drew this little cartoon “Eh?!? Where are you buggering off to?!?” which connected with old ideas I’d had from a story telling perspective many years ago. Together this is going to form the foundation for how we well the EyeSkills story and build the experience!

It’s been fourteen days straight with a level of intensity that is barely sustainable, but it’s been worth it.  I’m looking forward to the third and final event happening on the coming event, and hope that we can all get our tasks finished in time this week!