server

On demand gaming server for pennies a month!

The only downside with most dedicated gaming servers is the cost. If you need a 24/7 server or someone to manage it for you, you can’t avoid it. However, if you are self sufficient for setting up the gaming software, and only play certain games intermittently, your costs can be significantly reduced.

Let’s use Factorio as an example. To pay for a dedicated server it will cost around $5~10 per month, used or not. Now, if you are like me and play with friends and family only a few nights a month, that’s an unnecessary expense. Sure you can self host, but then everyone is dependent on a single person to have it up and running. Instead, I switched to using a Digital Ocean Droplet, and have paid less than a quarter this past month for an on demand gaming server.

Even if we played every night after work and weekends, it would only be $1 or $2 a month. You may be thinking “oh, you just turn off the droplet when you’re not using it!”, but alas, it is not that simple. A turned off droplet or server is still holding onto resources, so it still incurs cost, which is standard across all the hosting giants I looked into.

The trick on how to save cash? When you’re not playing, turn the droplet off, snapshot it, then destroy the droplet. When you want to play again, restore the snapshot to a new droplet. That way you are only paying server costs while it is running, then paying the super cheap snapshot storage the rest of the time.

This is painful to do by hand, but really easy with a helper script. (If you want to take it further, you could make it a website with access controls so only who you want could control it whenever they wanted.)

Digital Ocean Gaming Service (DOGS)

The code repo is available on github and the instructions are boringly standard.

git clone https://github.com/cdgriffith/dogs.git
# Alternatively, just download and extract the zip file
# https://github.com/cdgriffith/dogs/archive/master.zip

# Create a venv if you are python savvy
pip install requirements.txt
cp config.yaml.example config.yaml
# Update the config file to match your digital ocean settings
python -m dogs

This script does have some prerequisites. You need to have a Digital Ocean account token, and will have to manually create and startup the server once before you can manage it via this script. (Pull Requests always considered if you want to ease that pain for others.)

Droplet setup

When you do create the droplet you want, make sure to pick the smallest specs you think you will need. You can always upgrade to larger disk size, but cannot go back down to smaller. Also during this process, record the hostname which we will use as our server name in the config file.

You will also need your SSH key id, which are the digits at the very end of the public key. For example if the key ends with rsa-key-20190721 the ssh id is 20190721. You can always find this info in the Accounts > Security section as well by selecting a key and hitting “Edit”.

Also if you hook a firewall up to the droplet you will also need that ID, which can you retrieve via the API.

DOGS Server Config

When you got all that info, add it to the config.yaml file.

token: <your 64 character hex string>
 servers:
   ubuntu-1804-factorio:
     region: nyc1
     size: s-1vcpu-2gb
     firewall_id: <unique hex string separated by dashes>
     snapshot_max: 2
     ssh_key: <8 digits from end of ssh public cert>

You can get a full list of regions and sizes from their API as well.

Again, to run you just need to be in the dogs directory and run python -m dogs.

Binary (EXE) files

If you want to package it into an easy to use exe (can also modify for mac or linux binaries), just use the included build scripts.

# Windows specific requirements, otherwise just install 'pyinstaller'
pip install -r requirements-build.txt
python dogs\build.py

And now you should have a super handy dogs.exe in the dist directory. Don’t forget to keep your config.yaml in the same directory as it!

Gaming Setup Scripts

I have a directory to put files for setting up and updating game servers. Right now I only have factorio, but feel free to add your own, a PR would be very appreciated! All you need is a way to near automatically create / install the service and a way to have it auto start (in my example using systemd for standard Ubuntu servers)

Check the scripts out on github.

Personal Cloud Media Server – Encrypted, Streamable, Affordable…Possible?

Is it possible to build a personal media server that is hosted in the cloud, while making privacy, security, and accessibility paramount?I wanted to find out, and this post will dive into the options available to achieve such a possibly as well. (Spoiler: I did end up making my own software to do just this!)  

First, of course, is the why even try this when other options already exist? For example, Plex and Subsonic are some great options if you want to host a media server from your own home. The catch is then you have to have good upload speeds, storage space, an always running server or NAS, and concerned about how private your data really is. Because at the end of the day these are companies, not just software, and they are beholden to the requests of government agencies. They also have user all data in a single, potentially hackable, silo.

Cost Breakdown

So first, fast upload speeds. If you got it, you’re golden, but if your ISP doesn’t offer high enough speeds, you’re screwed. And even if you do have fast upload speeds you now need a server with hard drives that is always being fed electricity. Time to figure out what it costs to remove that need entirely.

On the flip side, a media server is pretty simple, logistics wise. You need a server to host the web page or API, and a storage provider. These could even be the same thing, however I have yet to find a price conscious option that includes both. Instead, for my personal needs, I priced out the difference between buying a 2-bay NAS (as it offers low electric and data redundancy) and using a local server, and constantly paying for an online one storage roughly 2TBs of data (BackBlaze for storage, DigitalOcean for webserver).

 Local Low  High     Cloud Low  High 
2-Bay NAS $150 $300   2TB
Storage 
$140 $600
2TB HDDs $70 $130   Web
server 
$25 $80
Electric $5 $30        
Internet  $0 $50        
             
Immediate
Cost
 $220  $430        
 Yearly Cost $5 $80     $165 $680

And the numbers speak for themselves. It is much, much cheaper,and more viable to buy a NAS and use the established software. The low end of the yearly cloud costs would match the cost of the high end NAS after only five years. Only an idiot with a paranoiac need for total control and security would even think about making their own software and paying so much to host it online.

I naturally started working on the architecture for how to build my cloud media player after the price analysis.

Design

Having content that is both easily streamable and encrypted is a doozy. It wouldn’t really have been possibly for the his and hers at home a few years ago.But thanks to MPEG DASH and HLS, we now have video formats with those features built in!

HLS is far more common, but it is a proprietary format developed by Apple and doesn’t have nearly the same feature set as DASH. (Note: Apple should rename their company to Sour Apple, because they refuse to support the internationally standardized DASH format because they hate competition.) So for my own purposes, I chose MPEG DASH.

The real downsize to either of these formats though, is now you have to re-encode all your videos before uploading them, ugh. But it really can’t be helped, and then at least it standardizes your library. After figuring out a bit more of how DASH works, I created a super basic structure I wanted to follow:

The webserver needs to allow for finding and playing the encrypted movies. DASH supports multiple DRM methods, but the best option for a home user is ‘cleartext DRM’ aka a password. Well, you don’t want to store raw passwords in the database, so that means anything in there also has to be encrypted. Oh, and if you really want to storage provider to have no clue what’son there if they scan your stuff, that means subtitles and cover files need to be encrypted too. Oy.

But I really wanted to see if this was possible and learn this new tech, so I plowed on. I also was heavily working with JavaScript at work,so I wrote it to use Node instead of my beloved Python. Two weeks, forty dependencies and a job change later, I had ZABAVA!

Solution?

Zabava translates to “fun” or “entertainment” from Bosnian / Czech / Croatian(and maybe more?) and I thought it was a cool sounding word. Every time I say it aloud, I imitate Jim Carrey from Ace Ventura saying “shikaka”. (No idea if that’s correct, but no one’s stopped me yet.)

It has user authentication with JWT tokens. Thought, admittedly just single user right now with admin rights.

It of course supports editing video information and changing cover file.

And has a script that allows for automatic converting videos to DASH, adding them to the DB and uploading them to the storage provider. I only designed the backend for BackBlaze B2 currently, as that is what I use, but it has a fairly agnostic provider setup to allow easy creation of others.

Sexy right? I am quite proud I was able to get nearly everything to work as envisioned.

Of course, not all fairy tales have the ending we imagine. Some videos still don’t like being converted or played via DASH format and it takes forever and a day to convert and upload terabytes of media files. The code is also not up to my personal quality standard, as a lot was written while figuring out how the tech worked without consideration to overall architecture.

In the end, after uploading all my media and not using it for a few months, I stopped work on the project and have bought a home NAS anyways (as I needed some solution for tons other files as well.)

I may go back and refactor it some weekend I am in a crazy mood, but I don’t think it will fit my person standard for code quality. However, if you are interested in the code and working on it yourself, you can find it on my github

Summary

I learned a lot about building a streaming site and different security methodologies for it, so I even though I wouldn’t qualify the code as a success, it’s surely a personal win.

If I were to do this again I would of course do things a little differently:

  • Try using HLS instead of DASH for wider support
  • Write the backend in Python instead of Node
  • SQL instead of Mongo (in my defense I was using Node at the time)

So, lets go over the criteria again:

  • Encrypted – Yes!
    • Everything was secure
  • Streamable – Yes!
    • Some conversions might have issues until tweaked right, but majority of the content worked as expected.
  • Affordable – No
    • The NAS that I ended up buying cost more than streaming the media, but it was used for a lot more than just the few VHS backups I have. So not the cheapest option, but not a bank breaker.
    • Oh wait, factor in the time spent building the new app… yeah, it ain’t cheap.

As expected, you can’t have all the perks with no downsides, but if Security and Accessibility are your goals more than cost, something like this might interest you.