The GPU hardware encoder in the Raspberry Pi can greatly speed up encoding for H.264 videos. It is perfect to use for transcoding live streams as well. It can be accessed in FFmpeg with the h264_omx encoder. But is it fast enough for live stream a 1080p webcam?
You might have already seen a lot of people using the built-in raspberry pi cameras to stream crisp 1080p video, so why is this even a question? Well the catch there is the Pi Camera itself supports native H.264 encoding. Some webcams do as well, and they are honestly the best choice to use rather than constantly battering the GPU encoder if you don’t need to.
However, you may just happen to have an old cheap webcam that only does MJPEG streams. Those streams are generally too large to pump over the Raspberry Pi’s wifi at full fps. Would using the hardware encoder help you?
The Results First
This is why you’re here, let’s cut to the chase and do a comparison of the two latest Raspberry Pi’s available, the Pi 4 B, and Pi 3 B+ (we’ll throw in the little Pi Zero Wireless for fun too.) We’ll talk about the two videos used later, but suffice to say, Trackday is easier to encode and closer to what an average Webcam would produce. Artist is more of a torture test.
Boom! The Raspberry Pi 4 B is right in the butter zone. Most webcams that are 30fps would be handled just fine with the Pi 4 (depending on the quality of sensor and what you’re filming). The Pi 3 B+ isn’t terrible, but wouldn’t be able to encode a realtime stream smoothly.
The little Pi Zero? Well, it did its best and we’re proud of it!
Test Media
Trackday
The first video I used was a video captured from a car on a racetrack. It is 1920×1080 at 30fps captured from a dash cam.
The original bitrate was a 10.5Mb/s and was cut down to 5Mb/s with all our encodes.
The command used is:
ffmpeg -i trackday.mp4 -c:v h264_omx -b:v 5M -an -sn -dn track_omx.mp4
Artist
The second file, artwork in progress by Clara Griffith, is also 1920×1080 at 30fps. However it is using BT.709 color space and started out at 35Mb/s!
If you see a webcam that advertises as “HDR” it is most likely using the BT.709 color space as well, and may give your Pi a headache.
This one was also compressed down to only 5Mb/s. Why 5Mb/s you ask? Well as it turns out, using the standard 2.4GHz wifi band, the Pi 3 and Pi 4 can each sustain about 6.5Mb/s download speed over my wireless. That means I know these videos could be played smoothly over wifi. The Pi Zero W on the other hand could only sustain around 3Mb/s wifi transfer speed.
All three systems were set up to use 256MB of GPU ram.
Video Quality
This actually took me by surprise to be honest. The quality of the encode is quite good when comparing to what a software encoder could do. I didn’t pull any punches either, the x264 encoder was set to dual pass and using veryslow preset with the film tune set. x264 commands:
ffmpeg -i "artist.mkv" -map 0:0 -c:v libx264 -pix_fmt yuv420p -tune:v film -color_primaries bt709 -color_trc bt709 -colorspace bt709 -pass 1 -passlogfile "pass_log_file_f9e11f23efaa23591fa8" -b:v 5000k -preset:v veryslow -an -sn -dn -f mp4 /dev/null ffmpeg -i "artist.mkv" -map 0:0 -c:v libx264 -pix_fmt yuv420p -tune:v film -color_primaries bt709 -color_trc bt709 -colorspace bt709 -pass 2 -passlogfile "pass_log_file_f9e11f23efaa23591fa8" -b:v 5000k -preset:v veryslow -map_metadata -1 -map_chapters 0 "artist-x264-5M-veryslow-film.mkv"
Of the two videos, Trackday is more realistic to what a webcam would experience and both encoders are near equal. So why was the Artist video so much better quality after encode, even though it started out with a lot higher bitrate? My informed guess on that is how crisp the original was, as well as the content is slow moving enough, the H.264 was able to reuse larger parts of the video for subsequent frames.
That means the software encoder x264 wins by virtue of being able to effectively use B-frames. Whereas the OMX hardware encoder doesn’t have support for B-frames. Therefor the Pi is on even ground when B-frames aren’t effective, but lags behind when they come into play.
A Note on Pi Camera Native H.264
I have found very little information about what Pi Cameras actually support H.264 natively. I only have “knock off” Raspberry Pi cameras that use the ribbon cable. They all support H.264 streams, which you can check with:
v4l2-ctl -d /dev/video0 --list-formats-ext # ... # [4]: 'H264' (H.264, compressed) # Size: Stepwise 32x32 - 2592x1944 with step 2/2 # [5]: 'MJPG' (Motion-JPEG, compressed) # Size: Stepwise 32x32 - 2592x1944 with step 2/2
or
ffmpeg -hide_banner -f video4linux2 -list_formats all -i /dev/video0
# [video4linux2,v4l2 @ 0x22c9d70] Raw : yuv420p : Planar YUV 4:2:0 : {32-2592, 2}x{32-1944, 2}
# [video4linux2,v4l2 @ 0x22c9d70] Compressed: mjpeg : JFIF JPEG : {32-2592, 2}x{32-1944, 2}
# [video4linux2,v4l2 @ 0x22c9d70] Compressed: h264 : H.264 : {32-2592, 2}x{32-1944, 2}
I was kinda worried they were using some hackery to “pretend” to actually have native H.264 but instead using the GPU. However if the Pi Zero has anything to show, it has a really hard time encoding 1080p videos with the GPU encoder, so I do believe they have native support.
Wrap Up
If you already have:
A camera and a Raspberry Pi: you can get started streaming right away.
A 1080p webcam and want to stream from it: consider grabbing a Raspberry Pi 4.
The Raspberry Pi: first always try to grab a camera with built in H.264 support, otherwise, the Pi 4 should support most webcams using hardware accelerated encoding.