Started working with 4K video, and to be expected I’m chewing through disk space like no tomorrow. On the cusp of replacing the 2012 rMBP with a 27" iMac, and so I’m looking at an external solution. To be exact I’m considering a RAID 1 type solution, so that any video stored on the drive is automatically duplicated.
The questions that I have in my mind at the moment; are as follows:
Which connection? At my disposal right now I have USB-3 (A), TB2, FW800. The 27" iMac has USB-3 (A & C). Obviously I need to use a connection that would easily allow for 4K video to be read and written, basically the fastest I can get.
Capacity v.s. Speed. With spinning drives I can get greater capacity, but with SSDs I can get greater speed. I can’t decide.
Anyone done anything similar, any recommendations? Even just suggestions?
The issue with using external data drives (including raid units) is the speed of getting the data onto/off of the unit. Which is the limitation of the interface you are using. If you can use USB-C connection, it is the fastest way on/off the iMac. Go with that interface.
Now as someone that has 20+ storage certifications, here is my recommendation. Go with RAID-6 (never RAID-5) or RAID-10. There are some odd-ball RAIDs that are executable -Raid-Drobo (or whatever it is called) work well to. Use large HDs for bulk storage. Use SSDs (RAID 10 only) for scratch/workspace. As that is basically memory from the app (it is used as swap). And dont put your SCRATCH drive and BULK DATA drive on the same unit. Put it on separate unit and if all possible separate interfaces.
Also please backup your data. it doesnt have to be a backup software. You can just copy it to a separate drive (unit).
PM me if you want to talk about this at length. I get paid to talk about this stuff and love to. Just dont want to bore people here.
This goes for anyone that wants to talk about storage.
Mainly for storage; I am using Apple’s AVFoundation and so far that seems okay with only 16GB of memory (famous last words). The 2012 RMBP has an internal SSD and the iMac will have an internal SSD (plus 32GB of RAM to start with).
The only thing I’ve not been able to figure out with AVFoundation, is how to create a movie from frames in memory, the API I’m currently using writes the movie to disk as I add frames.
Hence my dilemma as the current machine I’m using doesn’t have USB-C (TB3) only USB 2 and TB2. I don’t have a schedule to replace this machine, only that it needs to be replaced and an iMac is the only suitable solution.
[quote=349003:@scott boss]Now as someone that has 20+ storage certifications, here is my recommendation. Go with RAID-6 (never RAID-5) or RAID-10. There are some odd-ball RAIDs that are executable -Raid-Drobo (or whatever it is called) work well to. Use large HDs for bulk storage. Use SSDs (RAID 10 only) for scratch/workspace. As that is basically memory from the app (it is used as swap). And dont put your SCRATCH drive and BULK DATA drive on the same unit. Put it on separate unit and if all possible separate interfaces.
Also please backup your data. it doesnt have to be a backup software. You can just copy it to a separate drive (unit).[/quote]
Thanks; being old I only recall RAID 0 and RAID 1, I was going for RAID 1 so that I’d automatically have backups of the videos (i.e. the drive would be mirrored). I currently don’t intend to use the drive for scratch, basically for storage, but it would need to be quick enough to read and write 4K video some of the 10 minute clips I have are 10GB in file size.
That said I’m open to suggestions at the moment. Are there any particular brands you can think of that I should be looking for enclosures or brands to avoid?
Sorry I thought I saw you said you had USB A & C my bad. Use whatever the fastest interface you have.
Yeah I am old too. but there is about 3-4 dozen different RAID types now.[quote=349005:@Sam Rowlands] currently don’t intend to use the drive for scratch, basically for storage, but it would need to be quick enough to read and write 4K video some of the 10 minute clips I have are 10GB in file size.[/quote]
doing RAID6 or RAID10, yes it is fast enough. Raid-Drobo is fast enough. As long as the external unit has hardware RAID controller then your bottle neck will be the interface card/port.
Scratch drives are more sensitive to speed. Bulk data space, the 10gig video file is going to be throttled by the USB2 port. It would loaded fasted with USB3 but USB2 is faster than USB1/1.1 unit.
for external units I really like Drobo. I have a Drobo 5D plugged into my Mac Mini (my TV DVR) via TB2. I have over 20TB of usable drive space in it. What I like about Drobo is not all drives have to be the exact same size. It does a weird RAID that can use mixed sized drives. I have 4TB to 8TB drives. And when I am running low on space, I replace the lowest sized drive with a larger one and it rebuilds it Raid-Drobo and I have a bunch more space (nearly) immediately. Very little maintenance you have to do. Just watch the for Orange Lights. All Green means everything is good.
There are plenty of good brands. When you think you have an idea on what you are looking at or have a short list of make/models, let me know and I can advise you.
Not only Drobo supports drives of different sizes, the other NAS manufacturers have caught up on that. On my Synology it is called SHR (Synology Hybrid RAID)
SHR doesnt work as well as the Drobo Raid (cant remember the weird raid name they call it). Give Synology a year or two to work out the bug then it would be as good as the Drobo on that front.
Now Synology with same size drives, using a stand RAID types (Raid0/1/5/6/10) works very well.
yeah they stopped putting money into R&D or cut it way back or something. Their gear isnt bad just not as nice/featureful as some of the others.
Being a founding member of the RAID Advisory Board from 1993 as the RAID project manager at Connor Peripherals (when RAID meant Redundant Array of INDEPENDENT Disks), we’ve always had levels 0, 1, 3, 4, and 5. RAID 6 was introduced in the mid 00’s. The combining of two types to get 10 and 50 (pronounced as one zero and five zero, not ten and fifty) has also long been described and used.