A TB of storage

[donbga]

If you live in Europe you might check Dead Link Removed, in CanUSA http://www.frys.com, they offer the WD MyBook Studio 2 TB for only 229 $
(<sigh> I wish we had those bargains in Europe)

Many of these devices have reliability problems. Check out reviews online before buying. I almost purchsed a WD 1TB net drive that got terrible reviews from consumers.

So beware.

 

[toyotadesigner]

Just checked Tom's Hardware guide. Nothing bad about the WD MyBook Studio Editions, so I really don't know why you complain. Compared to the LaCie drives they are good performers.

 

[pellicle]

That's just BS Chris, there is no easy way for a lab to skimp on their C-41 chemistry, in fact it's not in their best interest to do so. They might do so out of neglect but those that do aren't in business for long.

ok, but come and live here for a while and then explain what it is you see ...

 

[donbga]

Just checked Tom's Hardware guide. Nothing bad about the WD MyBook Studio Editions, so I really don't know why you complain. Compared to the LaCie drives they are good performers.

Who's complaining. I just mentioned that some of these products are sketchy based on consumer/user reports.

But what does Tom know and why should we care?

 

[SilverGlow]

what will you do when your drive dies? Will you have redundant backups?

nope, negs will be around in 60 years, drives wont (Ohh, and I'm a database analyst and work for a national library in a mass digitization [news paper archiving] project in my present day job)

You being a database anaylyst I'm exceedingly surprised you would be of that opinion.

You ask what happens if a drive dies? Well what would happen at your work if a drive died? The same thing sould happen: replace drive and copy archived data to it. You think the same archival drives will be used for 10-60 or more years? Of course not! That archived data is moved to newer and better and cheaper storage periodically, and for cheap too. In this way, your 100 year old archive of files is on nearly new media no matter what.

Here's the same type of question back to you: What happens if the house that you keep your negatives in burns down, destroying your negatives along with it? Your compositions gone forever! And your negatives will only last 60 years? That is a VERY short amount of time. The problem with negatives is that you can't keep 100% high-fidelity copies in other locations like you can digital.

It would be far better to scan the negatives with an excellent scanner having very high density (DPI), then keeping those files on more then one drive, and on drives that are in more then one geographical location. In this way, your digital files can last indefinitely (read 1,000s of years). And since image files are just streamed sequential files, there will always be ways to extract the image data.

Again, I am surprised that being a data analyst, you would have such flawed opinions.

Wow!

I too keep my negatives, but for the purpose of longevity, and archive, nothing beats digital content.

I've been an independent IT consultant, and developer for 32 years.

 

[Worker 11811]

My brother's apartment building burned down just a few weeks before Christmas. A guy got drunk and fell asleep in bed with a cigarette.

It was a 2-story, 40-unit complex. Cement block and concrete shell with drywall interior and a wood frame roof. The place went up like a tinderbox. There was nothing left but a burned-out shell. There was nothing left but ashes inside. All but a few people on the opposite end of the complex lost everything. My brother grabbed his lock box which contained his important papers (birth certificate, insurance papers, bank records, etc.) but aside from that, he escaped with his life and the clothes on his back.

The fire marshal let us go back inside to see if we could salvage anything. 90% of everything was burned to ashes. It was surreal! We only got 2 or 3 cardboard boxes full of things out of that scene. Strangely enough, his photo collection was one of the only things that made it out!

They were in the bottom drawer of a his bureau, packed inside two old shoe boxes. I had to pry the dresser apart to get them out but I was able to salvage 90% of his photos.

Most of the photos were okay. Some of them were a bit charred around the edges. The bad parts were cut away and they were all right. A few of them were damaged beyond repair, mostly because the emulsion melted and stuck the photos together. Even soaking them in water would not rescue them.

Almost all of the negatives were okay. A few of them showed shriveling from the heat. Any of them that showed other signs of damage from smoke were fixed up by rewashing in PhotoFlo and drying again.

Almost all of the prints that were lost were recovered from the negatives. I scanned them and printed them out from my inkjet. I also had my brother go through all the photos and give me a list of the top 10% that were the most important. I scanned all those negatives and burned the lot onto CD-Rs then made 2 copies. I also have disk images of those CD-Rs archived on my hard drive then backed up.

It's funny to think about how the photos survived but virtually everything else was destroyed. In the burned-out mess, I saw what looked like 2 or 3 computers in what was left of the neighboring apartments. I don't think anything short of Drive Savers would have recovered any data off them, if at all.

Moral of the story: You never can tell. Data loss or loss of photos due to fire or other catastrophe is essentially a random chance event, as far as I can figure.

 

[toyotadesigner]

I definitely know that more hard drives are dying per day than houses will burn.

Now what?

In the European Community the museums, city archives, etc. participate in a program to transfer all digital media to high resolution film for long term archiving.

I am convinced those scientists who have made this wise decision have more cleverness than you can imagine.

BTW, 2 years ago our computers and external backups had been stolen. But we had the slides and negatives to re-scan everything. Thank God I'm still sticking to film...

 

[David A. Goldfarb]

One of the most fundamental principles of archiving is human readability. Prints and negatives (digital or analogue) are human readable and don't require migration like data (and even with computer data, there is a distinction between human readable document formats like ASCII or RTF that could be printed out and read straightforwardly as opposed to MSWord DOC files, which need software to be interpretable). Of course prints and negatives benefit from optimal storage conditions, but under less than optimal conditions, if a human readable object is found a hundred, five hundred, three thousand, or ten thousand years after it is made, it is possible to understand on some level what it is without additional technology. This is not true of machine readable data.

Libraries and museums digitize collections not to create an archival copy, but to disseminate the material and to create a reference in the event that the artifacts themselves are damaged and need to be restored at some point, just as they have done with microfilm, microfiche, 35mm slides, and large format negatives and transparencies. The archival object is always the original artifact itself.

 

[SilverGlow]

One of the most fundamental principles of archiving is human readability. Prints and negatives (digital or analogue) are human readable and don't require migration like data (and even with computer data, there is a distinction between human readable document formats like ASCII or RTF that could be printed out and read straightforwardly as opposed to MSWord DOC files, which need software to be interpretable). Of course prints and negatives benefit from optimal storage conditions, but under less than optimal conditions, if a human readable object is found a hundred, five hundred, three thousand, or ten thousand years after it is made, it is possible to understand on some level what it is without additional technology. This is not true of machine readable data.

Libraries and museums digitize collections not to create an archival copy, but to disseminate the material and to create a reference in the event that the artifacts themselves are damaged and need to be restored at some point, just as they have done with microfilm, microfiche, 35mm slides, and large format negatives and transparencies. The archival object is always the original artifact itself.

David, ALL digital content needs software to be read by a human. There is no distinction between MS-word docs, RTF docs, PDF's, as they all are ASCII sequential streamed files too. And therefore can be assimilated with software that is fairly easy to re-write should all the Program CD's in the world vanish.

And what about film negatives that manage to make it past 200 years? If there are no longer enlargers, film scanners, readily accessible chemicals to make wet prints, then this is akin to missing software. Sure you can hold a color negative up to the sun but what then?

The problem with film archival is that there is one and only one negative that can be protected, and only in one geographical place. Digital can be archived in unlimited media across unlimited geographical areas and ever single copy is 100% true to the original.

 

[SilverGlow]

I definitely know that more hard drives are dying per day than houses will burn.

Now what?

In the European Community the museums, city archives, etc. participate in a program to transfer all digital media to high resolution film for long term archiving.

I am convinced those scientists who have made this wise decision have more cleverness than you can imagine.

BTW, 2 years ago our computers and external backups had been stolen. But we had the slides and negatives to re-scan everything. Thank God I'm still sticking to film...

I guess the scientists at the Louvre, the Getty and other world class museums are "stupid" because they digitize all their paintings and photographs. They "stupidly" value the fact that with digital you can store the facsimiles in many geographic places with near 100% fidelity, so if the museum gets robbed, a sad day indeed but having digital archival takes the edge off profoundly.

About the burning houses....if mine burns down (or is robbed), all my harddrives will be lost....except for those stored at my parent's house, and my sister's house...they are copies of the originals that got burned.....and provide 100% fidelity....the outcome would be very different if I just had the negatives and prints....now THAT would be a sad day, and I would feel so stupid because this tragedy could have been avoided had I just scanned the negatives and made off-site copies...and for what? A penny per Gigabyte...

Thank God for digital archival of my Tri-X negatives!

 

[David A. Goldfarb]

David, ALL digital content needs software to be read by a human. There is no distinction between MS-word docs, RTF docs, PDF's, as they all are ASCII sequential streamed files too. And therefore can be assimilated with software that is fairly easy to re-write should all the Program CD's in the world vanish.

And what about film negatives that manage to make it past 200 years? If there are no longer enlargers, film scanners, readily accessible chemicals to make wet prints, then this is akin to missing software. Sure you can hold a color negative up to the sun but what then?

The problem with film archival is that there is one and only one negative that can be protected, and only in one geographical place. Digital can be archived in unlimited media across unlimited geographical areas and ever single copy is 100% true to the original.

The Library of Congress certainly distinguishes between an MSWord DOC, which is binary and may only be read with proprietary software, and 8-bit RTF and ASCII files, which may be read using a wide variety of software including any plaintext editor. As I am sure you are aware, if you open an MSWord DOC file as plaintext, you'll see garbage. If you open an RTF, you'll see words, sentences, paragraphs and formatting commands that are not difficult to decode. If you open an ASCII file as plaintext, you can read it directly, presuming you have a way of accessing the storage medium.

As long as you can hold a negative up to the sun or view a print in the light and can identify it as an image, it is human readable. As long as there is some way of reproducing images, then it can be printed. It doesn't require wet printing or scanning or any current method of reproduction. We can reproduce cave paintings, because they are human readable. This is not true of digital files.

If a museum loses an original object, then no, having a digital or other reproduction does in no way "take the edge off." That is not the purpose of the digital reproduction or any other kind of reproduction in the museum world. Museums are about preserving, displaying, and studying original objects, and there is no substitute for the original. Art historians who have seen fifty different reproductions of a work but have never seen the original will say that they have never seen the work. Reproductions make it possible to display work in alternate media, on the internet and in print, and they document the work, but they don't substitute for the work.

 

[toyotadesigner]

Sure you can hold a color negative up to the sun but what then?

You can see the content. Simple as that.

About the burning houses....if mine burns down (or is robbed), all my harddrives will be lost....except for those stored at my parent's house, and my sister's house...they are copies of the originals that got burned.....and provide 100% fidelity

This is tell tale story. I've never met a person who is doing this in reality. 99.99% of the people just theorize about this wanna-be-magic: nobody with a fully functioning brain will backup his data and drive up every day to his parents and sisters house to exchange hard drives. Especially not in the US where most families are separated by hundreds if not thousands of miles.

he outcome would be very different if I just had the negatives and prints

Wrong, because a scan is a scan, and the technology still improves as well as the storage capability of film thanks to R&D.

 

[Worker 11811]

I'm still convinced that it's all up to random chance.

It was mere luck that my brother's photos survived the fire when everything else was incinerated. Those boxes of photos were literally pulled out of a pile of ashes.

It was truly weird to see the burned out remains of the bedsprings and mattress sitting just two feet from a pile of charred timbers lying on top of a chest of drawers which contain an almost unharmed box of photographs. Then, in another room, there is the remains of a computer and a television, reduced to a melted pile of goo.

I checked the hard drive. There was no way that drive would ever operate again. I had hoped that I could buy a USB enclosure and use that to extract the data but, short of sending it to "DriveSavers" (data recovery laboratory) there was nothing that could be done. There was no insurance to cover that cost (several hundred dollars) so we called the drive a total loss.

So, maybe I am a little bit "battle weary" from the experience but it really did open my eyes. No matter what pains you take to save your photos, your data or any other possessions, there's really nothing you can do to protect it against a major fire or a natural disaster.

I am absolutely in favor of making multiple digital backups of everything you think is valuable but when it comes down to the bottom of things, we are all, truly, at the mercy of God. If it is your turn to be struck by disaster, you will be and there is nothing you can do about it.

All you can do is prepare for the worst and hope for the best.

 

[pschwart]

I am absolutely in favor of making multiple digital backups of everything you think is valuable but when it comes down to the bottom of things, we are all, truly, at the mercy of God. If it is your turn to be struck by disaster, you will be and there is nothing you can do about it.

All you can do is prepare for the worst and hope for the best.

Religion and philosophy are not going to protect my data. The whole point is that there *are* things one can do to mitigate risk. Internet storage is cheap (for example, Amazon EC2 or your ISP) and there are other services that provide backup services over the internet. These are super redundant because they have their own backup and disaster recovery. Even a USB key or portable hard disk updated once a month and stored at work provides reasonable protection with very little effort invested. Each of us assumes whatever level of risk we are comfortable with.

 

[toyotadesigner]

How long does it take to backup 2 TB over the Internet with a 3 Mbit/sec connection? Days, weeks, months?

 

[pschwart]

How long does it take to backup 2 TB over the Internet with a 3 Mbit/sec connection? Days, weeks, months?

You do incremental backups. Anyway, you need to arrive at a strategy that works for you; there are lots of options. Even though hugh drives are cheap, most of us don't have terabytes of actual data to back up.

 

[Worker 11811]

Religion and philosophy are not going to protect my data.

Would you like it better if I said that you are at the mercy of the Great Subatomic Whizzing of the Universe?

I do agree that CYA is important and the more CYA the better. (Within reason.)

What I'm getting at is that people have the tendency to think that they are universally protected when they have digital backups.

The only thing that I don't really like about digital data storage is that you really aren't storing anything. You are only storing a set of instructions.

If I sent you a letter in Microsoft Word format, I'm not really sending you a letter. I am only sending you a set of instructions which can be used to reproduce a letter. Until you send that file to the printer and you hold a piece of paper in your hand and read it, you do not have a letter.

When you scan a photo and store it as a file you are only storing a set of computerized instructions with which you can reproduce a photo. Not until it is printed out and you hold it in your hand and look at it do you have a photo. Negatives are photographs AND they are instructions with which you can easily reproduce a photograph.

Scanning a negative and storing it as digital instructions is a good thing to do because an image is now backed up in case of loss or damage but, regardless of whether it is stored on film or on a hard drive, a photograph is just as susceptible to loss, theft or damage.

I don't think one method is inherently better than the other. Each has its strengths and downfalls. What I am getting at is that a photograph can be lost just as easily as a digital copy and there is little we can do to prevent it.

We are at the mercy of The Great Subatomic Whizzing of the Universe.

 

[Curt]

Got a new computer and LED monitor $$$&$$$$ big one. I also got a terabyte of storage drive.

How do I back it up? Another T1 drive or 5 & a 1/4 floppies? No smile face on my iPhone.

Got to finish my room now, I,m down at a carbon workshop and having fun, fun, fun, but nothing lasts forever so home soon. I made a nice carbon print from an old negative of Pt Lobos using my own homemade glop tissue. I'm making more tomorrow, with luck.

Curt

 

[Worker 11811]

Depends on your backup system.

If you are using "Time Machine" or something similar, the computer will keep incremental backups until the drive fills up. Then it will start deleting the oldest incremental backups to make room for new stuff.

*IF* you don't fill up your computer full of junk you can get away with a 500 GB for a while. Time Machine will start deleting old information when the drive gets full. If you end up with 500 GB of new information that the system can't figure out what to delete, it will "choke." But it will buy you some time.

Best solution is to get a 1 TB or larger. (You can get 2 TB drives now!)

 

[David A. Goldfarb]

I've got a routine going now, where I've got several working data drives for different things (writing, photography, video, other archives), and it all gets backed up every night incrementally to a larger drive (currently 1TB), except for the flash drive that contains my academic work, and that gets backed up manually in full as frequently as I think I need to (sometimes daily or more often, sometimes every few days).

When my backup drive starts filling up, I replace it with a larger one, the former backup drive becomes a data drive, and one of the former data drives gets repurposed as an offsite backup with ZIPped copies of my data drives that I leave at my storage unit. I don't do the offsite backup very frequently, but it means that if something catastrophic happens, I won't lose everything.

For the incremental backups, I use Iomega Backup, which is no longer distributed, but I like it, because it stores files in uncompressed format that can be accessed normally in ordinary Windows directories without going through any kind of restore procedure and without using Iomega Backup itself. It can store several generations of updated files if needed and you can set the backup schedule to whatever is convenient (I run it automatically in the middle of the night). The usual reason that I need a backup is because a file has become corrupted, or maybe I want to recover some part of a file I deleted, and this seems to cover my needs.

I've had my share of hard disk crashes and such, but I don't think I've lost more than an hour's work since I started using computers for word processing in 1984 or so. Well, I guess there are a few files I wrote using Wordstar on the Xerox 3030 that are on 8" floppies, and it wouldn't be so easy to recover them today, but I haven't had any interest in doing so. I may even have printouts of some of those files, so if I had to, I'd probably check there first and scan them, if I needed them in digital format.

 

[Worker 11811]

If you are using Mac OS, take a look at "Time Machine." It is very cool.

At first, it backs up the entire computer which, if you have a large hard drive, can take a long time. (A couple-few hours.) Then, after that, it makes hourly incremental backups. At the end of every day, it compresses the hourly backups into a single daily archive. At the end of every week, it compresses the daily backups into weekly archives. Finally, at the end of every month it creates monthly archives.

If, at any time, you need to look back at your archives, you can page through the different hourly, daily, weekly or monthly archives as if you are looking back in time. If you find a file, a group of files, or a folder of files that you want to restore you just highlight them and click the "Restore" button. In a few minutes your hard drive is back to its original state as you chose it to be.

It doesn't matter if you accidentally delete a file, if it is lost or becomes corrupt. You can delete your files on purpose or even restore your whole hard drive, system files and all.

The most you can ever lose is an hour's worth of work.

I am backing up two 1 TB drives onto a single 1 TB drive. I have everything backed up since last October when I started backing up with that drive and I am still only two thirds full on the backup drive.

Those hacks at Microsoft must have figured out a way to copy Time Machine by now.

 

[David A. Goldfarb]

I'm on a PC, but actually, that's the sort of program I don't really like. If I want to restore a file, I don't want to search for it by the time of the last backup. I want to look in an ordinary directory and find the file, and then I might want to look at various stored versions of it, and then copy it using the same method I would use to copy any file. In other words, I like the primary organization to be by filename with the same directory structure as the original files, and then by date, rather than the other way around.

So the way I have it now, my backup drive contains directories that mirror exactly the disks they are backing up, and within each backup set, there is a "revisions" directory" containing older versions of files that have been backed up more than once in different versions, also with the same structure as the original directory.

I can set my backup frequency to any interval I like, but I find it interferes with my work, if it runs continuously or very frequently, so I set it to daily and do a manual backup, when I think I need to backup more frequently.

I also don't want to have to use the backup software to do the restoration, because it is always a possibility if I have a hard drive crash that I might just upgrade the machine to a new OS that can't run the backup software I was using at the time. This probably isn't so much of an issue on a Mac, but on a PC, software going obsolete with the next version of Windows is increasingly a problem.

 

[Worker 11811]

Imagine being able to go back and look at your computer the way it was at a chosen point in time. Choose a folder, choose a file or even choose your whole hard drive. Then click "Enter Time Machine" and select a time period you want to go back to. You will see your computer as it was on your chosen date.

Select a file, folder or group of files then click "Restore." The file is back right where it was. It is as natural to search for a file in the Time Machine as it is to search for it on your normal desktop.

If you don't like to use the graphical interface you can access the backup drive directly. You can search for files just like you always do. If you find the file you are looking for you can just copy it to a new location where you want it to be.

Files are not stored in a compressed format. The catalog structure is NOT in a proprietary format. All files are normally readable, normally searchable and normally copyable.

If you decide to stop using time machine, just go to your control panel and shut it off. You can turn it on or off as you wish. You can tell it to back up certain volumes or not. It backs up your machine hourly. It consolidates backups automatically. It does not delete any files. It only reorganizes the catalog.

I never used automated backups of any kind. I wanted to know what was being backed up and when. The problem was that I got lazy. I would not back up on a regular basis. I ended up with holes in my backups.

Time Machine is so seamless and so simple to use that I don't even have to think about it anymore.

 

[toyotadesigner]

IMHO Time Machine is the ultimate killer app. It is integrated seamlessly into OS X. If you invest into a HFS pre formatted external drive all you need to do is to connect it via FireWire 800 or USB to your machine. The Mac will only ask you if you want to use the drive as a backup and that's it. The rest just works and doesn't require any user input. If you want to, you can exclude any directory or even entire HD from the backup or just keep the default settings.

Time Machine doesn't interrupt your work or workflow, it's running completely in the background. If the Mac goes into standby it sends the connected drive(s) into standby as well. It works on single drives as well as on RAID drives. That's the advantage of a Mac: work, don't worry!

 

[David A. Goldfarb]

Iomega Backup and other backup programs I've seen for Windows run in the background, so theoretically you can keep doing what you're doing, but if you happen to be doing something that needs a lot of computer resources, running large file transfers in the background will noticeably slow things down.