DropBox, OneDrive, etc: Digital Backup Online

[JWMster]

So I have a few photo files. Curious what folks are using for online backup? I have a number of drives, but each is in one stage or another of getting filled up. Nice to have 5TB of data space, but Microsoft seems to not want to make this available to folks without the right op system. What that is, I dunno. I have 365Office or whatever, but they'll only make 1TB. That's piker stuff. Thoughts?

 

[Poisson Du Jour]

Go out and buy a 1Tb USB drive (so many available for peanuts now). Leave this connected to your PC and get some decent free automated backup software that will periodically make full, incremental and differential backups to the drive according to changes you make and whether you authorise those changes to be reflected in the backup(s) or relabelled as changes of using some personal nomenclature. The only purpose I have for DropBox is when I am using my phone camera to record proceedings and the image is also ghosted to DropBox. There is no sync, so if you delete or change your phone pics, those changes are not reflected automatically on DropBox. Pffft. But such changes will often or optionally be available for PC-based backups.

And what makes you think you will easily expend 1Tb of drive space with "a few photo files"...?

 

[JWMster]

Hmmm let's say my Capture One catalgos fill about 500GB's. Not sure why. But I think they eat space.

 

[Poisson Du Jour]

Editing. The buzz word for all photographers, digital or analogue. Pare out the 'also-rans' from the 'will use' images.
I don't really see any advantages to cloud storage for small-volume occasional users (or those with the baseline problem of accumulating more and more images they will only see or view once before committing them all to the proverbial 'digital black hole'), and even less so among photographers especially who edit their images, so there is no bloating or floss other than images they will most certainly use.

I have never sought any more than my 7.5Gb free allowance on DropBox (only got 12Mb up there are the moment) because my phone camera often gets a good workout during my travels and images are automatically copied to DropBox (when back in the studio and wifi is connected), but with no sync capacity, changes I make on the phone's pics are not reflected on the pics held in DropBox, which is annoying. So I end up deleting what goes to cloud and get by more effectively and efficiently with sequential / incremental / differential / sync USB-drive backups (2 connected: one for the system, one for all images). If synchronisation of images between source and cloud is important/vitally important, your task of getting down to the technical nitty-gritty of choices and how-to is cut out for you. Just remember, KISS.

 

[JWMster]

Y'know you're right... and in the long run, that's the solution. But time is something I don't have a lot of ...especially to go back through and cull the 2nds. Disk space is cheap. And now that I'm shooting film mostly, with film negatives, I find it's really easier... and one of the reasons I'm increasingly out of digital raw files. My Print File of the negatives are my backup. But with the RAW files out of Sony and Fuji.... I made a mess. I haven't had the time to go back and trash the original "seconds" and "duds". Now with film, making contact sheets and scanning only the keepers, it's a lot easier.

If I confess... it's fair to say that some of the best photos I've printed out of my shots were actually ones I passed over initially for some other glitzier thing. But as time passed, months later I changed my mind and the keepers stood out. Passage of time has to be allowed to do its magic.

 

[etn]

Above 1Tb, online backup space rapidly becomes expensive.
I decided for a local NAS (Synology in my case, which I highly recommend, although there are other vendors which are certainly as good).

Advantage: lower cost, higher transfer rates, doubles as a media server, scalable at relatively low cost (you simply add hard disks when your needs grow)
Disadvantages: with online backup, you don’t worry about physical loss of your device (e.g theft) or hard drive crash. Not so with a NAS. Access from outside your local network can be very slow (depends on your internet connection)

Hope this helps!

 

[JWMster]

etn: My current practice is that I have 2 local solid state drives of 1TB each together with a 5TB drive. All 3 drives are external, but I don't have this synched up and run by any automated software, just pretty much done by hand. Looking for something "more elegant, but not too costly". Like you say, it can creep upwards pretty quick. SSD's are sweet... just like RAM in speed. The rest.... you wait for it... wait for it.... still waiting ...ah! there it is.

 

[etn]

SSD's are sweet... just like RAM in speed. The rest.... you wait for it... wait for it.... still waiting ...ah! there it is.

Agree
A NAS connected to a GbE wired connection is still nowhere as fast a locally connected SSD. It is comparable to a locally connected HDD, though.
The best transfer rate I have seen with my Synology is 110 Mbyte/s, with usual rates in the range of 40 to 80 Mbyte/s. I recently installed 2 mirrored SSD caches into my NAS (you need 2 SSD for write caching). This increased an abysmal small files transfer rate of 10~100kbyte/s (!) to somewhere in the range of 5 to 25 Mbyte/s, depending on conditions.
Synology also recently rolled out a service called Drive which, if I am not mistaken, allows to sync data on various devices like Dropbox or Onedrive does. (haven't tried it yet)

I think the final decision regarding a NAS vs an online service depends on how much space you need. I paid approx. $1200 for the NAS (Synology DS918+, $600), 16Tb of HDD and 2 SSD caches. Break-even vs equivalent storage is achieved in about 3 years. (my girlfriend is an avid digital photographer, and we have a lot of digital music.) If you don't need as much space, typically below 1Tb, I think you're better served with Dropbox or similar for about $100/yr/Tb.

If you are into nude photography, be aware that online services have a policy about nudity: e.g. from Onedrive code of conduct: "Don’t publicly display or use the Services to share inappropriate Content or material (involving, for example, nudity, bestiality, pornography, graphic violence, or criminal activity)." I am not sure that the Politruk of online services make a distinction between artistic nudes and pornography.

Hope this helps

 

[JWMster]

etn: Thanks! Seems the cheapest online is Amazon (where else?) where you can buy 1TB for $59 and in increments of 1TB. The trick is that even as a geezer, I have a lot of data files, music files, photo files (and raw files) and then there's Capture One catalogs which seem to eat about 500Gb. Issue came up because I couldn't back my C1 catalogs to the cloud. I think Amazon's the cheapest for now. I'd initially thought to simply add a new 1TB SSD every year, but I think a better idea is to put storage online with period offsite, safety deposit box copies a couple of times a year - of the way old files infrequently accessed. Thanks for the input!

 

[etn]

Sounds good! 1TB for $59 is a good deal indeed. The great advantage is that you don't need to worry about hardware failures, backups, etc - let Amazon take care of all this

 

[Ces1um]

Why not just store your negatives? Why back up a digital file at all?

 

[lantau]

etn: Thanks! Seems the cheapest online is Amazon (where else?) where you can buy 1TB for $59 and in increments of 1TB. The trick is that even as a geezer, I have a lot of data files, music files, photo files (and raw files) and then there's Capture One catalogs which seem to eat about 500Gb. Issue came up because I couldn't back my C1 catalogs to the cloud. I think Amazon's the cheapest for now. I'd initially thought to simply add a new 1TB SSD every year, but I think a better idea is to put storage online with period offsite, safety deposit box copies a couple of times a year - of the way old files infrequently accessed. Thanks for the input!

I'm not using cloud storage in a serious way at the moment. Unless you live in a place where you have a symmetric 100Mbit or faster line into your house it simply isn't practical to move large blocks of data into the cloud and back. At work we have a symmetric 155Mbit fibre line. That's quite nice actually. It also costs around €1200 a month I think.

Two days ago I digitised a few rolls of 135 film with a repro stand and my mirrorless camera and those 230 frames I did that evening came in at around 3.5GB. My upload speed at home is 1.1Mbit. It'd be quicker to mail a DVD to the datacentre. Residential ISPs want you to consume data, not push it out into the world. They hate the idea of high upload speeds.

If you pick the wrong storage provider or other cloud services they might just shutdown business with a few months notice, leaving you scrambling to move your data. OTOH if everyone only goes for the few established big ones there will be little competition. Don't get me wrong, I love the cloud in principle. But if it's not your very own cloud you have to be very aware and informed.

 

[lantau]

etn: My current practice is that I have 2 local solid state drives of 1TB each together with a 5TB drive. All 3 drives are external, but I don't have this synched up and run by any automated software, just pretty much done by hand. Looking for something "more elegant, but not too costly". Like you say, it can creep upwards pretty quick. SSD's are sweet... just like RAM in speed. The rest.... you wait for it... wait for it.... still waiting ...ah! there it is.

For storing large image files there is no real life advantage in using an SSD. The real advantage in most consumer use cases is the speed of small random access transactions. Booting the system, starting software, etc. Of course if you do some magic where you need to randomly read portions from thousands of images for some live art performance, well fair enough. Not a scenario for me, though.

My SSD on the windows side is 1TB. For booting Linux I have a 500GB one. For the big data I have a RAID array of HDDs in the computer. Cheaper, bigger, fast enough.

 

[lantau]

I think the final decision regarding a NAS vs an online service depends on how much space you need. I paid approx. $1200 for the NAS (Synology DS918+, $600), 16Tb of HDD and 2 SSD caches. Break-even vs equivalent storage is achieved in about 3 years. (my girlfriend is an avid digital photographer, and we have a lot of digital music.) If you don't need as much space, typically below 1Tb, I think you're better served with Dropbox or similar for about $100/yr/Tb.

I have a setup similar to what etn described above. A QNAP with 8 bays. Currently it's housing four 6TB NAS grade HDDs and one cache SSD. With RAID6 I have 12TB payload. RAID6 means any two HDD can fail at the same time without data loss. Also I can add another 6TB drive on the fly and have a full 6TB increase in storage.

In my computer I have a RAID5 of 3 cheap desktop HDD. Those are only the data vault, the system drives are extra. Originally setup in December 2012 with 3x2GB I had one fail a bit more than a year ago. Replaced it with a 3TB drive. The next one failed four weeks ago, I also replaced the next day with a 3TB drive. Once the last one is gone it will be 3x3TB and I can increase the storage size of the RAID5 array. I may pull that last one before it goes belly up on its own.

I regularly run a backup (rsync) from the internal raid with single redundancy to the NAS with double redundancy. The only problem now is the place burning down... For that I'm thinking buying a large archival HDD or two, which will take a (non redundant) backup to be stored in the office. Now if the house burned down and the drives in the office happened to be dead I'd know I gave it my best. All my negatives would be gone too. No redundancy there.

Anyway here is the important bit for our sceptical old school friends here: Yes, HDDs will fail and that is not a problem. But you need a redundant system. One drive fails eventually and you replace it quickly. The redundancy is restored. And because redundancy is no backup (only reliability) you backup to the other redundant system. The difference between digital and film/prints is that the latter can be quite stable in the right place. The digital system is more like a living thing requiring some ongoing maintenance/renewal. But the data on it stays the same. One day you'll replace the HDDs with whatever but the data stays save. Fileformats will not a problem either if you stay somewhat mainstream. All the old image formats from the 80s Amiga and Unix systems can still be worked with today.

The negatives from my childhood have been lost forever, btw. There are only some low quality colour prints left in albums. When I was an undergraduate student I didn't have my own camera, but ordered prints from those who took pictures on film. Those prints are what they are. Small and inflexible, hard to reproduce. It's amazing what pictures I still have from the early digicam days. They all survived being having been spread between a few computers and USB sticks. And can be reproduced at anytime to remind someone of those times.

I'm not saying digital storage is absolutely superior to film and prints. I think there is no clear winner: The digital system has some upfront cost and needs ongoing replacement and probably extension. In return you have digital data with no decay. So called bitrot is no issue for the informed photographer.

With film/prints it is possible to practice a store and forget system. But you may have slow decay if the stuff hasn't been processed well enough or the storage isn't good enough. Also it could take a lot of space, increasing like the digital data. For both ways you need to know what you are doing to be safe. Neither is impossible for the dedicated photographer.

 

[juan]

I save everything. I have two 4TB drives that cost around $100 each. I backup the entire computer to each and store one of the drives in a bank safe deposit box that costs $20 a year. Much cheaper than online.

 

[ozphoto]

I too use on-site backups that run automatically - 3 HDDs in case one (or two!) fail.
The main reason for this is that the internet in AUS is sooooo slow, backing up RAW files is a tedious exercise and often unreliable.

If you do still wish to push forward with Cloud back-ups, I use GoogleDrive over Dropbox; DB seems to take forever (always has, even when I was based overseas) when attempting to backup large files.

Note that WD and Seagate, now offer HDDs that are (essentially) your own private Cloud backup. They remain on-site but you plug into your internet connection, for Cloud access. In essence, the best of both worlds without waiting forever to backup in the first instance. From my (limited) experience with this, it works rather well - my sister was able to grab an image she had backed up while we were out. It was a quick an painless exercise.

They chose this over actual Cloud backups/access due to the sensitive nature of their clients' information. They run two - backups are set to run automatically, so if one should fail, they still have data access.

I'm looking at getting a 6TB (WD now has 8TB) - as long as your router is on and operational, access is easy and instantaneous (well, almost).

https://www.wdc.com/products/personal-cloud-storage/my-cloud.html
https://www.seagate.com/au/en/consumer/backup/personal-cloud/

Good luck!

 

[Poisson Du Jour]

Skips,
I think your problem may be rooted more deeply in your large organisation, or, permit me to observe, a lack of it. So many files scattered all over the place! Putting stuff on a cloud will only result in you becoming disconnected from them in a pseudo-physical way. You already have enough drives, but what is needed is to cull the sheer number of files and catalogues that are log-jamming. It requires a dedicated, planned approach and maybe just one gigantic Tb drive with 2-3 redundancy backups and suitable software (my own is the rudimentary and very trusty Todo backup which changes source and destination files according to what changes are made. That Todo software has saved my bacon both for the incoming console computer which is connected to the router and 'net, and a network of satellite computers around the studio and again upstairs in the house. Backups are especially vital in the case of receiving a problematic Windows Update that doesn't quite gel with the system and... oh--! Suddenly, you've got the most attractive blue screen of death or a stop error that says the 'puter can't start. But you can start it from just about any backup sofware out there and actually roll back the 'puter to conveniently pole vault over the midscreant Microshaft gremlins.

I have 2 old Buffalo USB hard drives, one of 500 gigglebites and one of 1Tb (the 500G was bought in 2010). All proof and production .tif print files from the lab are synced between the PC, backup and the lab 75km away so if I make any changes to any file, it is reflected in backups, unless I specifically exclude such a condition. The lab has a further number of drives on which original input files are stored and which are not touched -- a 'fall-back' in the event of multiple failures outside their control (client failures, that is).