Photo storage management help

[P1505]

Hi all. I need some help or focus.

I currently store my photo library on an SSD, backed up to an HDD, and to Backblaze. With extreme culling I have managed to stay within the 4TB limit of these drives.

However, I now need to think beyond that storage limit. I am shooting GFX and Q2, so the files aren’t small. I would guess I’ll fill on average 3tb per year unless some photo projects drastically ramp up.

Therefore I’m not sure I need a NAS but I need something .

And so I’m thinking of simply adding two more external HDDs to take over. My plan is to keep the SSD as my working drive, and use HDDs as the longer term backup.

This means over time I’ll have my library split across drives. I don’t use Lightroom and don’t see that changing. I’m looking for a way to catalogue and manage files that are on offline drives, I run a Mac system.

Or do I bite the bullet and buy the NAS, and have storage added over time? If I go that route I’d want to edit off the NAS and so still need another backup but my library would remain easy access.


Thoughts?

 

[Alan Edward Klein]

Do you need access to the old projects currently? Can;t you offload them from "current" operational drives? OR can you just move the photos you;ve used to the current drives if you need access to the good pictures?

 

[koraks]

Or do I bite the bullet and buy the NAS, and have storage added over time?

I'd get a 4-drove NAS, fill 2 slots with identical SSD's and the others with bigger HDD's. Mirror both sets (RAID1). Move stuff from the SSD array to the HDD array once it stops being used frequently. This way you can use relatively affordable SSD's for fast, short-term storage and also affordable HDD's for long-term storage. Both sets will be protected against hardware failure of any of the drives. If desired, you could combine this with cloud storage for off-site backup.

 

[P1505]

Do you need access to the old projects currently? Can;t you offload them from "current" operational drives? OR can you just move the photos you;ve used to the current drives if you need access to the good pictures?

Thats a good way to think about it. I arrange photos by camera and date, and use keywords in DXO to call up photos for projects. But if I arranged photos by project, no small feat, I could have drives per project.

The downside of that is it doesn’t cover the full catalogue.

 

[P1505]

I'd get a 4-drove NAS, fill 2 slots with identical SSD's and the others with bigger HDD's. Mirror both sets (RAID1). Move stuff from the SSD array to the HDD array once it stops being used frequently. This way you can use relatively affordable SSD's for fast, short-term storage and also affordable HDD's for long-term storage. Both sets will be protected against hardware failure of any of the drives. If desired, you could combine this with cloud storage for off-site backup.

I’m leaning the NAS route. The up front investment is higher but it should last a long time. I can attach it directly to my Mac for faster throughput.

What this doesn’t solve is the offsite backup in case of burglary. For that I’m tempted to backup the photos per project as Alan suggested and hope Backblaze can keep up.

 

[koraks]

I went thr NAS route many years ago, opted for a modest Synology device, which is still going strong. I upgraded the drives at some point to create more space. The NAS was a good investment in my case; easily accessible online storage.

For off-site backup I'd lean towards cloud storage since it doesn't depend on your own discipline. In my experience, that is always the weakest link.

 

[wiltw]

For off-site backup I'd lean towards cloud storage since it doesn't depend on your own discipline. In my experience, that is always the weakest link.

And emphasis on the word 'backup'...over the years there have been too many cloud storage companies who went backrupt or who suddenly pulled out of the cloud storage business, some of them very unexpectedly and with little notice to its customers. Giant companies, not merely smaller thinly financed companies.
Have your own copies of your data, and assume that your 'backup' on the cloud can be your safety net if your local data suffers catastrophic loss from fire or flood.

 

[P1505]

Possible pivot but thinking of simply getting a 24tb Thunderbolt RAID drive and a 12tb drive. That way I have 12tb of storage, two local copies and the third copy in another building. Plus the cloud backup.

And 4 years maybe to re think this

NAS seems overly complicated.

 

[Sirius Glass]

I make backup copies of my digital photographs on a 2 Terabyte hard drive on a regular basis so that I can always go back and the original RAW and JPEG images later, before I start any post processing.

 

[wiltw]

NAS seems overly complicated.

It not not complicated at all. Your biggest issue might simply be what RAID to use when setting up...RAID 0 thru 10.

The NAS speed of transfer to/from will be dependent upon the speed of the LAN board in your PC.
If you have slower internet 10baseT, it is bestter to keep your 'active' data (e.g., current year) on your local drive that is connected to the motherboard, while 'archive' data (from prior years) is kept on the NAS. Your own situation might be 'active' is current month, and then data is moved to the NAS.

 

[koraks]

RAID 0 thru 10.

Only RAID1 and RAID5 make sense in this application. The choice is simple - RAID1 for 2 drives, RAID5 for more than 2. Personally, I'd stick with 2 drives per array and RAID1.
Note that the RAID 'levels' are a hierarchical/sequential set. They're different ways of distributing data across disks, each with its own trade-off in terms of cost, speed and reliability.

I do get the 'complications' of setting up NAS in relation to a simple external HDD solution. How challenging a NAS is to set up, depends largely on what hard- and software is chosen. Some brands make the process very easy and user-friendly; it's one of the reasons I mentioned Synology.

The NAS speed of transfer to/from will be dependent upon the speed of the LAN board in your PC.

As well as any routers in-between. Fortunately, low-end gigabit devices are very affordable nowadays.

And emphasis on the word 'backup'...over the years there have been too many cloud storage companies who went backrupt or who suddenly pulled out of the cloud storage business, some of them very unexpectedly and with little notice to its customers. Giant companies, not merely smaller thinly financed companies.

Yes, obsolescence of the service is a concern. But even in the rare event (we've had a grand total of 1 occurrence) where a service was discontinued on a very short notice of only a few days, all it takes is to contract a new service and enable it. The rest of the time, the backup is available and it's a "fire and forget" situation. On the other hand, a physical disk backup kept at a second physical location that you have to maintain yourself will always lag behind and involve the risk of "oh hey, when did I last do the disk thing again..was that before Christmas, or just after...?"

Again, the number one cause of data loss is negligence. Ironically, the main factor we have to protect ourselves against, is our own failure...

 

[P1505]

I have my solution. I’m buying 2 32tb RAID arrays. One sits next to my Mac, the other sits in the house attached to an old Mac Mini. I work locally with no delay and the two arrays are synced. I then have Backblaze running.

I insert a card, backup to RAID 1. Do not format card. Should take approx 2 hours to duplicate over the network to second RAID as well as Backblaze. I use FreeFileSync to automate this, it works well.

This way I have access to everything and have backups and it works as my brain expects.

 

[jvo]

...the main factor we have to protect ourselves against, is our own failure...

"we have seen the enemy and they are us!" - Pogo