nas

16TB Showdown – WD HC550 vs Seagate EXOS 16 vs Toshiba MG08

Hard Drive Interals

The three cheapest SATA 16TB drives available at the moment (March 2021) are all around $350: the Seagate EXOS 16, the WD Ultrastar DC HC550 and the Toshiba MG08 Series. I planned on using two of each in my home NAS, until a shocking discovery while doing a quick benchmark on them! To spoil the suspense, the Toshiba was either defective or was designed with different workloads in mind.

A quick note: these were all purchased out of pocket for my own setup. I do not have any sponsors, nor any advertising on my website. These results are purely my own findings. I simply wanted to share to those who may be interested.

The Contenders

As you may have guessed, these 16TB drives have a lot in similar. Each drive is helium filled, 7200rpm and SATA III (6.0Gb/s). The three of them also offer 2.5M hours of Mean Time Between Failures and max 550TB/yr workload. There are a few subtitle differences though.

Seagate
EXOS 16
WD Ultrastar
DC HC550
Toshiba
MG08 Series
Cache256mb512mb512mb
Block512e / 4Kn512e / 4Kn512e OR 4Kn
TechCMRCMR / EAMRCMR

They also sport a few physical design differences. The WD seems to be “upside down” compared to the others, and the standard middle side screw hole did not line up in my case.

For those who are design conscious, the EXOS is the only one offering a splash of color, and none of them offer RGB lights. Though if you are looking for RGB lights on a HDD, you might need a mental recalibration via wrench to the side of the head. HDDs are like construction workers, they work all day and are only noticed when they are slowing things down.

The Test Machine

This system was not designed to pump out the best benchmark performance. This is a real world NAS I will be using for my house. That said, it may not eek out the most performance on these drives, but it does give us a standard base for comparison.

  • Operating System: TrueNAS SCALE 21.02 Alpha
  • CPU: Ryzen 5 5900x 12-core 3.7GHz
  • Motherboard: ASRock X570 PHANTOM GAMING 4
  • Memory: NEMIX RAM 64GB DDR4-3200 P 2Rx8 ECC Unbuffered Memory
  • PSU: CORSAIR RM Series RM750
  • HBA: Dell H310 LSI IT mode 9211-8i

The hard drives were all connected to the HBA card and tested one at a time. They were setup into single ZFS pools using the TrueNAS UI with record sizes set to 512K.

All three drives are the exact same size (perfect for a NAS setup) and using the same 512e block size.

14.55 TiB, 16000900661248 bytes, 31251759104 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Benchmark Results

Read Speeds

These were gathered by running hdparm -tT twice on each drive and taking the better of the two (they were extremely similar and didn’t seem to merit more tests).

They were all so close we really need to “zoom in” on the differences.

That’s a bit better. However it exaggerates the real world difference because the lack of scale, so keep that in mind (never trust a graph that doesn’t start at zero!) Out of the three, Seagate EXOS 16 looks to be the sweet spot for cached and buffered read speeds. The other two both put in great numbers for spinning platter disks, and may want to stick to the WD if you will be working with frequently cached data.

Write Speeds

Each Benchmark for the write speeds were done using the dd command three times each to pump output from /dev/urandom to each drive. The tests will not encounter compression or caching done during the runs with this design. This means these numbers are for comparison to each other only and not max speed. Each drive was in a single ZFS vdev by itself with a record size of 512k.

Large blocks

Here is where the Toshiba MG08 started to really worry me. Its write speed of large 1M block sizes is half that of the others.

dd if=/dev/urandom  of=/mnt/<dive>/tmp.file  bs=1M  count=1000  oflag=dsync

Medium blocks

The Seagate and WD are still leading by a considerable amount, but this seems to be the most favorable benchmark for the Toshiba.

dd if=/dev/urandom  of=/mnt/<dive>/tmp.file  bs=4096k  count=1000  oflag=dsync

Small blocks

Full on nosedive for the Toshiba, over four times slower than the other drives.

dd if=/dev/urandom  of=/mnt/<drive>/tmp.file  bs=4096  count=1000  oflag=dsync

After reviewing the data, I am convinced that the drive is either defective or is designed for a different type of workload than what is in my setup. I did reach out to other enthusiasts on the /r/homelab discord and nothing seemed to cause that dip due to methodology at least.

Conclusions

Both the WD Ultrastar HC550 and Seagate EXOS 16 seem to be good choices for a home NAS. Others seem to also have great performance with the Toshiba MG08, that I was not able to duplicate in this instance. I wish I had the capability to grab another Toshiba to see if this was just a defective drive or my setup, but I can’t just throw cash at it.

For now I am putting together a combo of WDs and Seagates into my NAS and calling it good. Hope you found this information useful!

Personal Cloud Media Server – Encrypted, Streamable, Affordable…Possible?

Is it possible to build a personal media server that is hosted in the cloud, while making privacy, security, and accessibility paramount?I wanted to find out, and this post will dive into the options available to achieve such a possibly as well. (Spoiler: I did end up making my own software to do just this!)  

First, of course, is the why even try this when other options already exist? For example, Plex and Subsonic are some great options if you want to host a media server from your own home. The catch is then you have to have good upload speeds, storage space, an always running server or NAS, and concerned about how private your data really is. Because at the end of the day these are companies, not just software, and they are beholden to the requests of government agencies. They also have all user data in a single, potentially hackable, silo.

Cost Breakdown

So first, fast upload speeds. If you got it, you’re golden, but if your ISP doesn’t offer high enough speeds, you’re screwed. And even if you do have fast upload speeds you now need a server with hard drives that is always being fed electricity. Time to figure out what it costs to remove that need entirely.

On the flip side, a media server is pretty simple, logistics wise. You need a server to host the web page or API, and a storage provider. These could even be the same thing, however I have yet to find a price conscious option that includes both. Instead, for my personal needs, I priced out the difference between buying a 2-bay NAS (as it offers low electric and data redundancy) and using a local server, and constantly paying for an online one storage roughly 2TBs of data (BackBlaze for storage, DigitalOcean for webserver).

 Local Low  High     Cloud Low  High 
2-Bay NAS $150 $300   2TB
Storage 
$140 $600
2TB HDDs $70 $130   Web
server 
$25 $80
Electric $5 $30        
Internet  $0 $50        
             
Immediate
Cost
 $220  $430        
 Yearly Cost $5 $80     $165 $680

And the numbers speak for themselves. It is much, much cheaper,and more viable to buy a NAS and use the established software. The low end of the yearly cloud costs would match the cost of the high end NAS after only five years. Only an idiot with a paranoiac need for total control and security would even think about making their own software and paying so much to host it online.

I naturally started working on the architecture for how to build my cloud media player after the price analysis.

Design

Having content that is both easily streamable and encrypted is a doozy. It wouldn’t really have been possible for the his and hers at home a few years ago. But thanks to MPEG DASH and HLS, we now have video formats with those features built in!

HLS is far more common, but it is a proprietary format developed by Apple and doesn’t have nearly the same feature set as DASH. (Note: Apple should rename their company to Sour Apple, because they refuse to support the internationally standardized DASH format because they hate competition.) So for my own purposes, I chose MPEG DASH.

The real downsize to either of these formats though, is now you have to re-encode all your videos before uploading them, ugh. But it really can’t be helped, and then at least it standardizes your library. After figuring out a bit more of how DASH works, I created a super basic structure I wanted to follow:

The webserver needs to allow for finding and playing the encrypted movies. DASH supports multiple DRM methods, but the best option for a home user is ‘cleartext DRM’ aka a password. Well, you don’t want to store raw passwords in the database, so that means anything in there also has to be encrypted. Oh, and if you really want to storage provider to have no clue what’son there if they scan your stuff, that means subtitles and cover files need to be encrypted too. Oy.

But I really wanted to see if this was possible and learn this new tech, so I plowed on. I also was heavily working with JavaScript at work,so I wrote it to use Node instead of my beloved Python. Two weeks, forty dependencies and a job change later, I had ZABAVA!

Solution?

Zabava translates to “fun” or “entertainment” from Bosnian / Czech / Croatian(and maybe more?) and I thought it was a cool sounding word. Every time I say it aloud, I imitate Jim Carrey from Ace Ventura saying “shikaka”. (No idea if that’s correct, but no one’s stopped me yet.)

It has user authentication with JWT tokens. Thought, admittedly just single user right now with admin rights.

It of course supports editing video information and changing cover file.

And has a script that allows for automatic converting videos to DASH, adding them to the DB and uploading them to the storage provider. I only designed the backend for BackBlaze B2 currently, as that is what I use, but it has a fairly agnostic provider setup to allow easy creation of others.

Sexy right? I am quite proud I was able to get nearly everything to work as envisioned.

Of course, not all fairy tales have the ending we imagine. Some videos still don’t like being converted or played via DASH format and it takes forever and a day to convert and upload terabytes of media files. The code is also not up to my personal quality standard, as a lot was written while figuring out how the tech worked without consideration to overall architecture.

In the end, after uploading all my media and not using it for a few months, I stopped work on the project and have bought a home NAS anyways (as I needed some solution for tons other files as well.)

I may go back and refactor it some weekend I am in a crazy mood, but I don’t think it will fit my person standard for code quality. However, if you are interested in the code and working on it yourself, you can find it on my github

Summary

I learned a lot about building a streaming site and different security methodologies for it, so I even though I wouldn’t qualify the code as a success, it’s surely a personal win.

If I were to do this again I would of course do things a little differently:

  • Try using HLS instead of DASH for wider support
  • Write the backend in Python instead of Node
  • SQL instead of Mongo (in my defense I was using Node at the time)

So, lets go over the criteria again:

  • Encrypted – Yes!
    • Everything was secure
  • Streamable – Yes!
    • Some conversions might have issues until tweaked right, but majority of the content worked as expected.
  • Affordable – No
    • The NAS that I ended up buying cost more than streaming the media, but it was used for a lot more than just the few VHS backups I have. So not the cheapest option, but not a bank breaker.
    • Oh wait, factor in the time spent building the new app… yeah, it ain’t cheap.

As expected, you can’t have all the perks with no downsides, but if Security and Accessibility are your goals more than cost, something like this might interest you.