Skip to content

What External Data Storage System Do You Use?

edited August 4 in SecuritySpy

What external data storage system do you use?

I just migrated from a (dedicated to SecuritySpy) 2012 Apple Mac Pro that was amazingly bombproof to a new Mac Studio. That Mac Pro was upgraded early on with a smaller SSD hard drive for the operating system, then kept packed full of SATA units for video capture storage. Now that Apple makes almost all of their computers no longer user upgradable for such storage, I usually buy the minimum system storage to save money and rely on various external storage options for better control.

For now, my Mac Studio just has a single Thunderbolt 3 external 1TB SSD hard drive capturing 22 cameras. It works fairly well. However, I need more stored video in the queue before it is overwritten and thinking of stepping down to a USB-C 3.2 (5 Gb/s) to SATA III multi-bay RAID enclosure. Slower, yes, but still overkill for the purpose and more cost effective overall.

Anyhow, feel free to share you own experiences.


Peace, Dr. Z.

Comments

  • I started out with a Synology NAS for Plex. Then I wanted cameras which Synology can do, but decided to move everything, including any upcoming cameras, from the NAS to an M1 Mac mini. I briefly looked into adding RAID to the Mac via software, as opposed to dedicated RAID hardware. Actually, I looked into all the storage options. In the end, I decided to simply hang multiple, cheap, external HDD's and 1 Thunderbolt SSD off the Mac. For SS, I record everything to the SSD. I then use Carbon Copy Cloner to backup everything, which includes copying over my SS footage to one of the external drives where I can store as much as I want. The SSD stores 10 days, but every day CCC copies it over to one of the HDD's

  • Personally, I go for individual external HDDs connected by USB - multiple of these for larger systems, with the cameras split across them. It's an easy and inexpensive solution, and provides a certain amount of redundancy - if one drive fails it doesn't bring down the whole storage solution (unlike RAID-0), don't affect all cameras, and is easy to rectify by simply swapping out that one drive. Any USB 3.x protocol is easily fast enough to cope with a couple dozen cameras capturing continuously to each drive, or many more when capturing based on motion. Desktop Macs give you 4x USB-A ports, and it's easy to add more via Thunderbolt/USB-C hubs if necessary.

    Many customers use RAID setups and they can work well, but they add complexity and therefore failure modes to the system. When customers ask us for help with drive issues that they have had trouble solving themselves, it often turns out they are using RAID setups, and it's then often difficult to pinpoint and fix the problem. One advantage of some specific RAID setups (e.g. RAID-5/6) is that they offer whole-storage redundancy, so that if a drive fails, you don't lose any data. However when this happens, and for a significant period after the failed drive has been replaced while the RAID is recovering, it tends to slow down the whole unit significantly, which can lead to problems capturing video during this period.

    I would say that RAID units are typically the best solution when you really need huge amounts of storage, or whole-storage redundancy, otherwise individual drives are the way to go.

  • edited August 5

    As always, thanks for your expert advice, Ben!

    Yes, I did experiment with all the different RAID configurations a few years ago (using Apple's DiskUtility or Old World Computing's SoftRAID). They all have their pros/cons. Then I gravitated to simply backing up all data to a Mac server once a day. HA, sure enough, I lost some video of an important event when that main hard drive spontaneously failed (and before the daily data transfer). So, I started using a RAID 1 (mirroring) configuration with my older Mac Pro from then on. Now that I have migrated everything over to my new Mac Studio, I plan to implement the same storage configuration via a multi-bay external hard drive enclosure.

    That brings up another line of questions:

    1. Are hard drives specifically designed for video surveillance worth the extra cost? I did start using a Seagate SkyHawk SATA hard drive a couple of years ago and it failed within 16 months. Thankfully it was covered under warranty and I want to think that this was just a fluke. The standard RAID 1 hard drive that I purchased at the same time and had matched to it is still performing just fine.
    2. Would it be worth it to have both volumes the same higher design standard? Regardless, maybe an extra measure of safety would be to stagger the replacement of each unit in a mirroring RAID.
    3. Realistically, when should such video surveillance specific hard drives be replaced? I have read anything from yearly with high security settings to every 3-5 years for residential uses.


    Peace, Dr. Z.

  • BenBen
    edited August 8

    Thanks for reporting your experiences! Yes, HDDs can be relied upon to fail every now and then, and of course especially when there is an important event to record :) RAID 1 is a good solution to provide redundancy in case of a single-drive failure. To answer your specific questions:

    1. I have read the claims by the manufacturers, but have seen no real-world test data comparing surveillance drives with standard drives, so at this point I'm basically agnostic on this. It's plausible that designing drives with certain characteristics will make them better for video surveillance, though the lack of test data means that we basically have to rely on manufacturers' marketing claims.
    2. Do you mean multiple volumes in a RAID? I've always gone for the advice to use identical drives, though this of course is not strictly necessary. High-spec drives aren't too much more expensive than low-spec ones, so it's worth spending the bit extra.
    3. Personally, I wouldn't replace drives too frequently, if they are working well. Once you have a drive that has been working well for 3 years, it's quite likely that it will continue to to work for a few more. Replacing it just means that you're replacing a known good drive for an unknown one. Perhaps after 5 years or so this makes sense, as the probability of failure then starts to increase significantly.
  • I use a 16GB RAM M1 Mac-mini with a 40Gb thunderbolt hub. I have 3 NVMe (2 1TB and 1 2TB) external 10Gb drives plugged into it and a 10Gb ethernet adapter. I have a Synology NAS 1621+ with 32GB ECC RAM and six 10TB HDD and two NVMe drives mirrored for caches and BTRFS metadata pinning. I use file integrity settings and scheduled scrubbing because I dislike bit rot issues. I have two 14TB external drives for backup. I do not back up my video security files. For sequential transfers I can barely exceed 8.5 Gb which is pretty good or just over 1GB/s. For my NAS I shuck Western Digital drives and use the HUGO utility to convert the drives to 4Kn since that's what they are instead of leaving them in the 512e emulation mode.

    I send the videos to the NVME and use the upload feature to send the videos to the Synology by using the local feature.

    I recently read another thread where Ben mentioned you can send the motion database to the NVMe and the video to a NAS by creating two camera objects and it uses one license. So I plan on reworking this since I had no clue I could do this.

    For some reason occasionally the Mac Mini loses connection to the NAS and I just reboot the MAC to quickly fix it and it reconnects at logon again. I would rather have SecuritySpy fix that connection for me or use some other method without having to do crazy stuff in the shell. I do have to say SecuritySpy is great at sending me emails when things screw up (which is not often).

Sign In or Register to comment.