EyeSkills Infrastructure

3 min read

This is going to be a bit of a geeky post, about how we’re setting the servers which hold and protect your data.  I think it’s quite interesting, and important to get it right and for you to know what we are doing 🙂 

These days most things are offered as online services. Google for email and documents. Dropbox for files. Facebook for socialising. GitHub for storing code. WordPress for writing… but this brings problems with it.

What are these companies really doing with your data?  Can they be trusted? Can an open source project like this afford outgoings which quickly add up to hundreds of €/$ per month? Your data belongs to you, and that goes against the prevailing view in most countries outside the EU

EyeSkills.org server architecture
EyeSkills.org server architecture

Self-hosting everything which touches you, your data, or the code we build on (well, except for the distribution of the app itself, but that – for the moment – is “alternativlos”) seemed a bit ambitious, but after I saw how one of our group (thanks Michi!) had solved it, there was just no going back.    The goal is not only to avoid hosting data where providers can have free and untraceable access, but to make life hard for anybody that wishes to snoop.

We are making sure that all data is cryptographically secured in transit, and where possible, when at rest. We only allow encrypted connections to browsers – everything over https – so that the contents of that traffic cannot be monitored (the same will go for any data sent from the apps in the future to the server).  On the server itself all of our services are containerised (isolated from one another) so that the likelihood of any single compromise of any individual service compromising the whole server is reduced.  We minimise the ports we open to 80 (http) redirected immediately to 443 (https) and SSL.

The SSL port is used to transmit our daily backups to a geographically separated server which also only runs a single SSL connection. Those backups are encrypted on the eyeskills.org server, so now only are they encrypted during transit, they are also encrypted from the very moment they hit the backup server and are stored.

Thanks to the containerisation on eyeskills.org, our backups only store the minimum necessary data (no system data) relating to each container, and a complete description of the container environment.  This, coupled with the deduplicating nature of the backup server (Borgbackup) allows us to keep backup space requirements to the minimum possible (again, we cannot afford to waste money!).  The restore process is interesting, in that our scripts (in the event of a complete server catastrophe) actually restore user data to a completely blank server instance first, then automagically recreate the previous container infrastructure around that data.  It seems to work beautifully! 🙂

Work which is still outstanding: Setting up our own email server and avoiding it getting automatically blacklisted!

There is always room for improvement.  If you happen to be a linux fan with experience of Docker or Borg, why not get in touch 🙂

Would you like to beta test EyeSkills* or just follow what we are doing?

Leave a Reply

Your email address will not be published. Required fields are marked *