The (internet) age old battle between content creators and pirates is an ever escalating arms race. It doesn’t seem to matter how much DRM and encryption creators put on their content, the next day there will be a dozen illegal torrents available. There is no way to fully protect videos shared on the internet from being copied, but there are ways to trip up the digital “porch pirates.”
I personally am not a fan of DRM, however I am also a supporter of content creators. And I want to give any help possible to upcoming artists out there that need some extra protection. If you’re a small content creator with your own site, and keep seeing your stuff being reposed on YouTube or tiktok from punks using simple browser extensions to download your stuff, wouldn’t it be nice to suddenly stop them in their tracks:
This article will show how to prepare videos for streaming and a simple tech demo of how to serve encrypted HLS streams using Python and FastAPI. This is proof of concept work presented as a tech demo and not something I have ever used in a production environment, so use at your own risk!
What you will need
- A video (or my ducks video!)
- FFmpeg (or, you guessed it, my ducks video!)
- openssl (or ((no not ducks)), my crazy insecure pre-made development only cert and key)
- Python 3
End Result
Here is a zip file of the code we will have worked through by the end of the article.
HLS Quick Intro
HLS is one of the most widely used streaming protocol used on the internet. It allows large videos to be chucked into smaller ones for faster and smoother playback.
HLS is composed of two things: a playlist file, the m3u8, which describes the content (aka a list of files) and the content itself.
Example m3u8 file which contains pointers to the three video segments:
#EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:3 #EXT-X-MEDIA-SEQUENCE:0 #EXTINF:8.333000, file_1.ts #EXTINF:8.334000, file_2.ts #EXTINF:8.333000, file_3.ts #EXT-X-ENDLIST
The three files file_1.ts, file_2.ts, and file_3.ts have relative paths in this example, so would be expected to be downloaded from the same path the (or in the same folder) as the .m3u8 file. You will see later how we can change that to suit our need.
HLS DRM and Encryption
HLS supports two different types of encryption, real DRM, which costs money and can still be broken, or “clearkey” symmetrical AES-CBC encryption, which as the name suggests is simply sending the key itself in plain text to the client.
This is then a good time to ask “why bother if we just send them the key?” Simply put, it’s a deterrent. Extensions approved through most marketplaces won’t be allowed to “break” DRM and products like Internet Download Manager avoid it as well. This goes back to the point that DRM for digital content is largely bogus in concept, as at the end of the day someone could just plug in an HDMI recorder even if the DRM was near perfect.
So why bother at all? It does slow down the average content thief, and may even give you better legal re-course if you can show that your content was protected and had to be decrypted.
Now let’s dive into what makes this method a bit more special. Most tutorials I have seen show how to use ffmpeg to encrypt these file the same time you break them into chucks. Which means they would also be encrypted on disk, and would have to be re-split and encrypted again to change the key.
The Seldom Seen Dynamic Encryption
We’ll take the cooler approach of only having to create the chucked files one, and then encrypt them as we serve them to the client. Obviously this may be a poor choice if you’re serving thousands of videos simultaneously, where you would rather want to front load this work fewer times. However it means you don’t need to have the raw content encrypted, which provides a lot more freedom in how you handle that data.
It’s possible to encrypt all chucks with the same key and IV (initialization vector), or switch it up between them. We will take it to the max and encrypt each segment with it’s own key and IV.
Our finished m3u8 will look something more like:
#EXTM3U #EXT-X-VERSION:3 #EXT-X-TARGETDURATION:2 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-KEY:METHOD=AES-128,URI="https://127.0.0.1:8444/key/bd647b13789cf0da94c15847e7cbbbe0/0.key",IV=0x5e36e3dff490c9218267fc846bdcf670 #EXTINF:8.333000, /encrypted_video/bd647b13789cf0da94c15847e7cbbbe0/0/ducks_aac/0000.ts #EXT-X-KEY:METHOD=AES-128,URI="https://127.0.0.1:8444/key/bd647b13789cf0da94c15847e7cbbbe0/1.key",IV=0x9032c1c44f96070b7fb0d04cf1e078d4 #EXTINF:8.334000, /encrypted_video/bd647b13789cf0da94c15847e7cbbbe0/1/ducks_aac/0001.ts #EXT-X-ENDLIST
Here you can see there is a new line before each chuck that starts with #EXT-X-KEY. This provides the details the client needs to know how to decrypt the video.
Preparing your video
If you already have a properly formatted h264 video with aac audio (or other known supported audio), you can simply copy the incoming streams as we make them into the smaller chunks. If you don’t know what that means, we’ll also go over how to convert them!
Pre-prepared video
If you just want to try it out and don’t have a video on hand, download my ducks. Music by AlexGrohl.
When you have that downloaded, you can skip on down to the python server section!
The basic FFmpeg command for HLS
We will be using ffmpeg on the command line to break these files into a basic m3u8 file.
ffmpeg -i my_video.mp4 \
-c copy \
-hls_list_size 0 \
-hls_time 6 \
-hls_base_url "{{ video_path }}/" \
ducks.m3u8
All on one line version for Windows users:
ffmpeg -i my_video.mp4 -c copy -hls_list_size 0 -hls_time 6 -hls_base_url "{{ video_path }}/" ducks.m3u8
Let’s go through this line by line. First is specifying to run the ffmeg command itself, and the input file we will be using.
ffmpeg -i my_video.mp4-idenotes the input file
-c copy-cstands for “codec” and we are saying to copy all the codecs without converting
-hls_list_size 00here says to not cut off the list after a certain number of segments.
-hls_time 6- How long, in seconds, each segment should roughly be.
-hls_base_url "{{ video_path }}/"- This is a trick we will use later in the program. It adds that string before the video names
ducks.m3u8- Name of the output playlist file. There will also be other segment files created.
You can check out all the possible HLS options on the ffmpeg documentation for it.
If you used it on the ducks video, it will look like:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:8
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:8.333000,
{{ video_path }}/ducks0.ts
#EXTINF:8.334000,
{{ video_path }}/ducks1.ts
#EXTINF:8.333000,
{{ video_path }}/ducks2.ts
#EXTINF:8.333000,
{{ video_path }}/ducks3.ts
...
#EXT-X-ENDLIST
In the same directory it created the ducks.m3u8 file there will also be eight other .ts files that are the actual video, now split into small chucks.
Converting video and audio
If you need to make your video “streaming safe” you’ll want to re-encode it to h264 and use aac audio, as they are both widely supported. We will also take it another step and make sure the bitrate is nice and low for slower connections.
ffmpeg -i my_video.mp4 \
-c:v libx264 \
-b:v 3000k \
-bufsize:v 6000k \
-maxrate:v 6000k \
-c:a aac \
-b:a 128k \
-hls_list_size 0 \
-hls_time 6 \
-hls_base_url "{{ video_path }}/" \
ducks.m3u8
Single liner for fellow Windows users:
ffmpeg -i my_video.mp4 -c:v libx264 -b:v 3000k -bufsize:v 6000k -maxrate:v 6000k -c:a aac -b:a 128k -hls_list_size 0 -hls_time 6 -hls_base_url "{{ video_path }}/" ducks.m3u8
- Here we are specifying the
h264encoder for the video track (v) with-c:v libx264 - We are setting the average bitrate of the video
-b:vto3mbps (3000kbps) - Setting a sane bufsize and maxrate for the video
- Encoding the audio to
aacwith-c:a aac - Setting the average bitrate of the audio the
128k
If you know your way around ffmpeg, you’ll know how to eek out a bit higher video quality with a dual pass encode and slower presents, but that is beyond this article.
Python powered FastAPI server
We will walk through each part of setting up the server and going from simply streaming a HLS video to encrypting each segment of it.
Our directory structure will look like this in the end:
> data
> playlists
ducks.m3u8
> video
> ducks
0000.ts
0001.ts
> templates
video.html
> venv
server.py
requirements.txt
certificate.pem
privatekey.pem
Virtual python environment
Lets start by setting up the virtual environment and installing the requirements.
python -m venv venv # Activate on linux with # source ./venv/bin/activate # Activate on Windows bash with # .\venv\Scripts\activate.bat # Activate on Windows powershell with # .\venv\Scripts\activate.ps1 # After activating, run: pip install --upgrade fastapi jinja2 cryptography uvicorn
Create a key pair for using HTTPS connection
HLS will complain if you aren’t using HTTPS. We will quickly create a private key and certificate to use with FastAPI’s server uvicorn. If you don’t have openssl on your server and promise to only use these for local development, these should work until 2031.
openssl genrsa > privatekey.pem openssl req -new -x509 -key privatekey.pem -out certificate.pem -days 365 # Go through the interactive shell and fill in or leave as default
Simple test server
Let’s create a server.py file with minimal setup.
# server.py
from fastapi import FastAPI
app = FastAPI()
@app.get("/")
async def hello():
return {"hello": "world"}
Lets run it and make sure to use our private key and certificate.
python -m uvicorn server:app--ssl-keyfile privatekey.pem --ssl-certfile certificate.pem --port 8444
You should see a few info lines followed with:
INFO: Uvicorn running on https://127.0.0.1:8444 (Press CTRL+C to quit)
Add a webpage to view the video on
Create a directory called templates and an html file called video.html with the following content.
<!DOCTYPE html>
<html lang="en">
<head>
<link href="https://vjs.zencdn.net/7.17.0/video-js.css" rel="stylesheet"/>
<title>Streamer</title>
</head>
<body>
<h2>{{ video_name }}</h2>
<video id=vid1 class="video-js" controls>
<source src="/playlists/{{ video_name }}.m3u8" type="application/x-mpegURL">
</video>
<script src="https://vjs.zencdn.net/7.17.0/video.js" crossorigin="anonymous"></script>
<script>
window.onload = function () {
videojs('vid1');
}
</script>
</body>
</html>
This is a very basic HTML page with some jinja2 variables that we will use to point towards the wanted m3u8 file.
We are using video.js to make sure our HLS video is cross platform safe.
Adding the template to the server
At the top of the server.py file we are going to add a new import and specify our templates directory.
from fastapi.templating import Jinja2Templates templates = Jinja2Templates(directory="templates")
At the end of the file we can then add the route for that template.
@app.get("/{video_name}.html", response_class=HTMLResponse)
async def video_template(request: Request, video_name: str):
return templates.TemplateResponse("video.html", {"request": request, "video_name": video_name})
Basic streaming server
First going to put the full block of text that the updated server.py will look like to make sure we don’t miss anything.
from pathlib import Path
from io import BytesIO
import jinja2
from fastapi import FastAPI, Request
from fastapi.responses import FileResponse, HTMLResponse, StreamingResponse
from fastapi.templating import Jinja2Templates
here = Path(__file__).parent
data = here / "data"
app = FastAPI()
templates = Jinja2Templates(directory="templates")
playlist_templates = jinja2.Environment(loader=jinja2.PackageLoader('data', 'playlists'),)
def sanitize(item: str) -> str:
"""Make sure they can't do a directory transversal attack"""
return item.replace("\\", "").replace("/", "")
def serve_bytes_as_file(content: bytes) -> StreamingResponse:
"""FastAPI doesn't have a handy way of doing this natively so we have to improvise"""
response = BytesIO()
response.write(content)
# Streaming response iterates over the data from the current seeked point
# We need to reset it to start of file, or else it would be empty
response.seek(0)
return StreamingResponse(response)
@app.get("/playlists/{video_name}.m3u8")
async def get_playlist(video_name: str):
"""Return the m3u8 file with the paths updated properly"""
template = playlist_templates.get_template(f"{sanitize(video_name)}.m3u8")
if not template:
return HTMLResponse(status_code=404)
formatted_template = template.render(video_path=f"/video/{video_name}")
return serve_bytes_as_file(formatted_template.encode("utf-8"))
@app.get("/video/{video_name}/{segment_number}.ts", response_class=FileResponse)
async def get_segment(video_name: str, segment_number: str):
segment = data / "video" / sanitize(video_name) / f"{sanitize(segment_number)}.ts"
if not segment.exists():
return HTMLResponse(status_code=404)
return segment
@app.get("/{video_name}.html", response_class=HTMLResponse)
async def video_template(request: Request, video_name: str):
return templates.TemplateResponse("video.html", {"request": request, "video_name": video_name})
We are adding two helper functions, sanitize and serve_bytes_as_file. The first will make sure that we don’t allow for directory transversal attacks. The second will allow us to turn in memory bytes into an object FastAPI will send as a file.
get_playlist function
Let’s look at the interesting part of the get_playlist function.
formatted_template = template.render(video_path=f"/video/{video_name}")
return serve_bytes_as_file(formatted_template.encode("utf-8"))
The template.render will update the {{ video_path }} we put in the m3u8 file earlier with hard paths to the segments. For example with the duck video would point too /video/ducks/0000.ts.
Then we serve that as a file, accessible at /playlists/ducks.3u8
get_segment function
We will be really beefing this function up when we start encrypting these segments. But for right now we are returning them as is, just doing a quick check to make sure the file exists first.
Adding video files
Those video segments and playlist file we created (or downloaded) earlier now need a home. Create a data directory and two directories inside of it, playlists and video.
Place your m3u8 playlist in the playlists directory. Lets assume we called it ducks.m3u8
Then make a sub directory inside of video called the same as the stem of the playlist. So if we called it ducks.m3u8 make the directory name ducks.
Then copy all the .ts files into the ducks directory.
I hope you picked up by now that you can also add more playlists and folders with any name you wish!
Testing the basic streaming server
Make sure that you’re back at the root of the project and still have your venv active, then start up the server.
python -m uvicorn server:app --ssl-keyfile privatekey.pem --ssl-certfile certificate.pem --port 8444
Navigate in a web browser (on the same machine) to https://127.0.0.1:8444/ducks.html
There should be a video file that plays, congrats you have a simple (unencrypted) streaming server!
Now lets add the magic!
Adding HLS Encryption
There is no “special sauce” to HLS encryption. Each segment is simply encrypted in AES-128 CBC mode. You provide the IV for it in the m3u8 file and a path (URI) for it to download the key from. That means we’ll need to generate a cryptographically secure key and IV for each segment. We will also need to generate them when we modify the playlist file. So first, let’s add some imports and global objects.
Imports and Globals
import secrets
from pathlib import Path
from io import BytesIO
import binascii
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from fastapi import FastAPI, Request
from fastapi.responses import HTMLResponse, StreamingResponse
from fastapi.templating import Jinja2Templates
encrypted = {}
MY_URL = "https://127.0.0.1:8444"
We will use the secrets library to create secure keys for each segment, then we will do the encoding with the cryptography library. Do take note we are using the primitives from the hazmat section, which is a reminder that you really shouldn’t play with this unless either you know what you’re doing, or it’s not a high stakes crypto project. Thankfully this sits soundly in the second camp.
Then we will have in memory storage for all those created keys in the encrypted dictionary. This could be beefed up into persistence storage somewhere, but that is beyond scope for now. Similar to how we should really be pulling in the site url for this script from a config file or environment variable.
get_playlist function with encryption
The secret sauce we have to add to the m3u8 file is an encryption line that looks like:
#EXT-X-KEY:METHOD=AES-128,URI="http://example.com/key",IV=0xffffffffffffffff
So we are going to generate a key that we will keep on the server for now, and pass the IV into the m3u8 file itself. We are going to base our location off the “{{ video_path }}” key we added during playlist creation. The encryption line has to go two lines above where the actual “{{ video_path }}” goes, so we need to iterate over the file line by line and create a new output list where we can insert lines where we want.
@app.get("/playlists/{video_name}.m3u8")
async def get_playlist(video_name: str):
playlist = here / "data" / "playlists" / f"{sanitize(video_name)}.m3u8"
if not playlist.exists():
return HTMLResponse(status_code=404)
identifier = secrets.token_hex(16) # Would be best to do check to make sure there is no conflict with existing
m3u8_enc_line = '#EXT-X-KEY:METHOD=AES-128,URI="{base_url}/key/{identifier}/{number}.key",IV=0x{iv}'
keys = []
out_lines = []
instance = -1
for line in playlist.read_text().splitlines():
if line.startswith("{{ video_path }}"):
instance += 1
# Generate a secure key and IV to use for encryption.
key = secrets.token_bytes(16)
iv = secrets.token_hex(16)
keys.append({"key": key, "iv": iv})
out_lines.insert(-1, m3u8_enc_line.format(base_url=MY_URL, identifier=identifier, number=instance, iv=iv))
out_lines.append(line.replace("{{ video_path }}", f"/encrypted_video/{identifier}/{instance}/{video_name}"))
continue
out_lines.append(line)
encrypted[identifier] = keys
return serve_bytes_as_file("\n".join(out_lines).encode("utf-8"), media_type="application/x-mpegURL")
In this loop we will be adding a new entry in the global encrypted dictionary that is a list which contains more dictionaries that are in the {"key": key, "iv": iv} format. The key is in bytes, but the iv is a string that is hexideciamal. They could both be bytes or both be hex, just went with this way because we immediately use the IV as a hex string in the m3u8 whereas we always need the key in raw byes.
Also take note of the path of the video we are replacing. It’s a bit more complex than the simple streaming server.
f"/encrypted_video/{identifier}/{instance}/{video_name}"
We are just giving the server a bit more information on the following calls to get the stream segment. We need to know the “identifier” to match the segment request to the key and IV we have stored. Then we need to know which key and IV to use from the list.
get_segment function with encryption
For the updated function we simply need to make sure the requested file exists, and then use the already saved key and IV to encrypt it with AES in CBC mode. The key length we set was 16, which translates to using AES128.
Ready for testing HLS encryption
Let’s run the server again! Make sure you’re in the root of the project (same location as the “server.py” file) and have your virtual environment enabled.
python.exe -m uvicorn server:app --ssl-keyfile privatekey.pem --ssl-certfile certificate.pem --port 8444
You should see your video there, same as before, just now it’s encrypted! Huzzah!
If you weren’t able to follow along or something went horribly wrong, here is a zip file with the expected end result.
