How to create adjustable RAW files from color negative scans

[davidjfthomas]

I have a developed a workflow document detailing how to use an inexpensive flatbed scanner, VueScan Pro, Negfix8, and Adobe Camera Raw to transform color negatives into fully adjustable digital files. I did not invent this method. I merely collected information from various web sources and documented and tested the process.

Please view the document at http://durstac800.blogspot.ca/p/color-negative-scanning.html

Please use the comment facility on that blogsite to contribute corrections and additions.

Cheers,
David Thomas

 

[Eric Rose]

Could you post one of these RAW tiffs so we can open it up in ACR for ourselves to see how it handles please.

 

[pellicle]

What is not fully adjustable with a 16 bit tiff?

 

[pellicle]

Why do you need ACR when you have a scan of a file as an RGB tiff? It is not a raw file in the sence that a digital camera is a raw file.

 

[davidjfthomas]

You can view some screenshots and download sample files here:

https://drive.google.com/folderview?id=0B_-e9Vi-s5r5RGd0bVcwWFpVUVE&usp=sharing

 

[davidjfthomas]

Just as each camera brand has its own linear raw file format, the files we are dealing with here are the direct output of the scanner sensors as captured by VueScan.

VueScan's "raw" file is an unprocessed linear string of pixel data derived directly from the sensor. Same definition applies to raw files from a camera. In fact, VueScan does have the option to save its raw file in Adobe's DNG format, so it is a true raw file.

You need to run the file through Negfix8 to neutralize the orange of the film base and invert the image to a positive.

ACR gives you controls that you just can't get from Photoshop alone.

 

[davidjfthomas]

The point is to do the adjustments with Adobe Camera Raw, rather than the built-in controls of the scanner software or the destructive editing of Photoshop. The ACR controls are much finer than the scanner controls, or Photoshop's, and ACR retains all of the original scan data. You can go back and fine-tune from the original data. You can't do that with a tiff or any other file edited directly in Photoshop.


Finally, you can see your ACR adjustments in real time. Trying to control color with the scanning software is by trial and error.

 

[davidjfthomas]

Here's link to a Google Drive folder containing sample files (the VueScan raw file; the raw file as processed by Negfix8, and the finished file as adjusted by ACR and Photoshop.

There are also screenshot files to show what to expect as you work your way through the procedure.

 

[OzJohn]

Hi David

I’m a little mystified why you introduce into your workflow what to me is the complication of TIFF. If I understand correctly, you scan as a TIFF then presumably somewhere that file gets converted to some type of raw format in order to get it into ACR since ACR, as far I’m aware, will only open raw formats and JPEG. Since your scanning software can do the job in DNG why not just use it and skip TIFF? Is it perhaps that some of the other software you use (none of which I am familiar with) can only work with standard file formats like TIFF, JPEG etc?

I also have a bit of difficulty in accepting that TIFF is a kind of raw format. In its basic form TIFF is just an uncompressed, lossless bitmap format but so is .BMP itself and the only real difference between them is that TIFF can also hold metadata, layers, paths and pages. Today, the main use of TIFF files is in the fields of commercial printing and graphic arts and ever here it is being overtaken by .PDF and the native formats generated by software like InDesign. TIFF file sizes are much larger than DNG, .PSD or any of the camera raw formats and can impact computer performance for no gain in image quality. OzJohn

 

[pschwart]

The point is to do the adjustments with Adobe Camera Raw, rather than the built-in controls of the scanner software or the destructive editing of Photoshop. The ACR controls are much finer than the scanner controls, or Photoshop's, and ACR retains all of the original scan data. You can go back and fine-tune from the original data. You can't do that with a tiff or any other file edited directly in Photoshop.


Finally, you can see your ACR adjustments in real time. Trying to control color with the scanning software is by trial and error.

ACR is certainly more powerful than scanning software, but in my experience doesn't measure up to Photoshop.
Also, layers make it possible to do non-destructive Photoshop editing. Except for some occasional spotting, my base layer remains untouched. I always work on copies and not the original raw files anyway, so this is really a non-issue.

 

[pellicle]

I wonder if he's selling something?

 

[pellicle]

Hi

The point is to do the adjustments with Adobe Camera Raw, rather than the built-in controls of the scanner software or the destructive editing of Photoshop.

photoshop isn't that destructive of any easier to learn than ACR ... although I can see why you may like to "only learn one tool" tis an ideal that has no end in itself.

The ACR controls are much finer than the scanner controls, or Photoshop's,

you seem to have little idea how fine the controls are in photoshop ...

let me recommend some courses to you

and ACR retains all of the original scan data. You can go back and fine-tune from the original data. You can't do that with a tiff or any other file edited directly in Photoshop.

really?
- File Save As?
- work with layers?

sorry ... BWAAAA

Finally, you can see your ACR adjustments in real time. Trying to control color with the scanning software is by trial and error.

true, but capture in scan and tweak in photoshop and you can see the changes in real time there too ... and afterall where does ACR put the file? Photoshop?

Lastly a RAW file is not a TIFF

I suggest you read the following:
Demosaicing - Wikipedia, the free encyclopedia
Raw image format - Wikipedia, the free encyclopedia

an excerpt:

In digital photography, the raw file plays the role that photographic film plays in film photography. Raw files thus contain the full resolution (typically 12- or 14-bit) data as read out from each of the camera's image sensor pixels.

The camera's sensor is almost invariably overlaid with a color filter array, usually a Bayer filter, consisting of a mosaic of a 2x2 matrix of red, green, blue and (second) green filters.

quite unlike the RGB nature of what is already in a TIFF

I guess you'll think I'm some sort of smartarse, perhaps I am ... but thanks for sharing

best of luck selling that stuff in your barrow

 

[artobest]

Pellicle, I don't see that the OP is trying to sell anything. I would take issue with some of his methods (scanning is an art, not a science, after all), but there's no need to question his motives.

 

[Eric Rose]

The point is to do the adjustments with Adobe Camera Raw, rather than the built-in controls of the scanner software or the destructive editing of Photoshop. The ACR controls are much finer than the scanner controls, or Photoshop's, and ACR retains all of the original scan data. You can go back and fine-tune from the original data. You can't do that with a tiff or any other file edited directly in Photoshop.


Finally, you can see your ACR adjustments in real time. Trying to control color with the scanning software is by trial and error.

No link???

 

[davidjfthomas]

Hi OzJohn,

I understand the confusion as indeed TIFF files seem to be disappearing as PDF files dominate in the pre-press world and "raw" files take over as the repositories for rich image storage.
The reality is that that vast majority of raw files are in fact TIFF files.

The TIFF file generated by VueScan is indeed a raw file. Firstly, because raw is a loose definition with no technical specification beyond that it should be a linear string of data drawn directly from the sensor, camera or scanner and unprocessed by the camera or the scanner software. So if VueScan chooses to call its unprocessed linear file a raw file, it is well within its rights. And if VueScan wants to stick a .tif extension on its raw filenames, that's perfectly consistent with what a raw file is. "Raw" is not an acronym for a standard and should not be represented in uppercase; it is merely a broad generic term.

The fact is that, raw files, from most manufacturers including Adobe and its DNG format, are in fact TIFF files. TIFF is not going away; it has merely been re-branded.

From Wikipedia:

"Many raw file formats, including IIQ (Phase One), 3FR (Hasselblad), DCR, K25, KDC (Kodak), CR2 (Canon), ERF (Epson), MEF (Mamiya), MOS (Leaf), NEF (Nikon), ORF (Olympus), PEF (Pentax), RW2 (Panasonic) and ARW, SRF, SR2 (Sony), are based on the TIFF file format.[2] These files may deviate from the TIFF standard in a number of ways, including the use of a non-standard file header, the inclusion of additional image tags and the encryption of some of the tagged data."

and,

"DNG, the Adobe digital negative format, is an extension of the TIFF 6.0 format . . . "


VueScan, when it saves its raw file as a DNG rather than a TIFF is changing a bit of header data when it saves its raw file as a DNG rather than a TIFF. This is an unnecessary step for adjustment with Adobe ACR since ACR does indeed open TIFF files directly.

The big reason for using a TIFF-tagged raw file in this workflow is that the Negfix8 script requires a TIFF file as its input.

I hope this removes the confusion.

 

[davidjfthomas]

Here's the link:"

https://drive.google.com/folderview?id=0B_-e9Vi-s5r5RGd0bVcwWFpVUVE&usp=sharing

 

[davidjfthomas]

I am sorry that my contribution seems to have aroused such animosity, including the wholly false and hurtful accusation that I am selling something. What might that be, pray tell?

I had thought this was a forum for the helpful sharing of experience and information, not for gratuitous insults. Very disappointing. I won't be back.

David

 

[pschwart]

I am sorry that my contribution seems to have aroused such animosity, including the wholly false and hurtful accusation that I am selling something. What might that be, pray tell?

I had thought this was a forum for the helpful sharing of experience and information, not for gratuitous insults. Very disappointing. I won't be back.

David

No animosity here. Everyone has their own preferred workflow and there are lots of ways to achieve the same result. I questioned some of your premises, but welcome the discussion.

 

[L Gebhardt]

I am sorry that my contribution seems to have aroused such animosity, including the wholly false and hurtful accusation that I am selling something. What might that be, pray tell?

I had thought this was a forum for the helpful sharing of experience and information, not for gratuitous insults. Very disappointing. I won't be back.

David

David,

I am hoping to try out your workflow and compare it to some other options. Without trying it I have one minor: relying on VueScan for the scan is both good and bad. It's good because it supports a lot of scanners, but bad because it doesn't support them all. Ideally I would like to develop one workflow that will work with all my scanners from the ScanMate drum to the Epson flatbed.

I happen to really like the Lightroom's interface, so in that way it seems like a positive. I also have VueScan and Epson Scan's interfaces, so again, that's a positive.

Thanks for posting this; the more good methods we have for scanning color negatives the better. Please do come back.

Larry

 

[pellicle]

David

I am sorry that my questioning appeared to be insulting. It did look that you were gearing up to sell something. Please accept my unreserved apology.

as to animosity? well I can only say that if my sentences were brief that may be interpreted as animosity. It was most certainly not intended to be. I did notice however that you did not engage in any discussion on your views to support them (other than assert them) or to engage on why your method was less destructive than the methods I suggested.

I am familiar in science with people questioning my assertions and expecting I can support them.

.. that I am selling something. What might that be, pray tell?

you could be working on a document which you would intend to sell. This has come up in the past (if not here in other places) and people publish in many ways these days.

... not for gratuitous insults. Very disappointing. I won't be back.

if gratuitous insults are there please point them out to me. You had made barely 3 posts when you posted a link to "please visit my site and download my document", yet you speak of "discussion" and "contribution". Where is the discussion? You pull on the cloak of the offended martyr immediately that anyone asks you anything about your methods.

 

[pellicle]

I hope this removes the confusion.

it at least clarifies what you are willing to term a RAW file. I stick with my definition but of course you can stick with yours. Now I know what it is. I personally do not consider a file from a scanner as a RAW file.

Anyway, in case you ever do come back (and you are welcome in my opinion) I would be interested to hear your discussion on the questions I asked. Such as why editing a TIFF is destructive. If handled in the same manner as done with a RAW file (that is, the file is opened -> read -> adjusted, and then the adjustments are stored as meta-data to be reapplied later) the adjustment of a TIFF with layers then stored with the file are indeed non-destructive.

Unless you have some evidence to suggest otherwise?

 

[pellicle]

Hi OzJohn,

I understand the confusion as indeed TIFF files seem to be disappearing as PDF files dominate in the pre-press world and "raw" files take over as the repositories for rich image storage.
The reality is that that vast majority of raw files are in fact TIFF files.

I understand your confusion. the vast majority of RAW files are generated by cameras and are called RAW files. These are not in fact TIFF files, although what is in the Tagged Image File Format is just containers of data. So while the public may get the idea that its just "data" there is more to it than just that.

Digital camera RAW files store information acquired from the camera sensor from the scene. Since the cameras mostly have a BAYER Array they are stored as (say) 14 bit (although this varies) counts of the data the sensor captured. Each "pixel" of the RAW file is either Red Green (and another green) or Blue data. To view this as a image it must be converted in a process called demosiacing where a pixel is created in the middle of the array formed by the GB then RG pixels.

NB in this diagram below, these grey pixels are created and are what you see. They are created based on the RED GREEN and BLUE neighbor information.

This is quite unlike what happens with a scanner. So if you choose to mush them up into the same concept you will be missing information in your understanding.

I hope this helps.

 

[OzJohn]

I am sorry that my contribution seems to have aroused such animosity, including the wholly false and hurtful accusation that I am selling something. What might that be, pray tell?

I had thought this was a forum for the helpful sharing of experience and information, not for gratuitous insults. Very disappointing. I won't be back.

David

Hi David

Since Adobe own the IP to the TIFF format I would not be surprised if DNG is based upon it as I would also not be surprised if other raw formats are based on bitmap which, as I have said before, is what TIFF really is anyway. But camera raw files are still not TIFFs and IMHO using an intermediate TIFF stage where you really need a raw file seems to be a workaround in order to use freebie software to accomplish things like neutralizing the mask colour from colour negs. Like just about everything in digital imaging these tasks can usually be done in Photoshop itself or in combination with ADR.

It is unfortunate and unwarranted that another poster has questioned your motives and it is a natural human reaction to take umbrage but that person has apologised and that should be the end of it. It's just part of the cut and thrust of internet forums. Perhaps you will reconsider and continue to post here. I for one hope so. OzJohn

 

[pellicle]

David, I have a quick question

you say:

I had thought this was a forum for the helpful sharing of experience and information

and it is ... which is why I am curious, in the context of your first post. (Ohh, and have you indeed ever made any previous posts? I couldn't find any)

Please use the comment facility on that blogsite to contribute corrections and additions.

why would you not just like those comments and sharing of experience on this site? For us all to contribte and share.

I mean, if it was your intention to actually participate here? Why should we go to your site to give only to you this experience and our corrections (which seemed to be unwanted by your responces).

Would sharing this among members here not be better done here?

Just a polite question if you would be so kind as to answer that.

Some of us get annoyed by the amount of spam and self promotion that uses forums as the google leaverage for that.

 

[L Gebhardt]

Are we sure that what Vuescan writes out from an Epson scanner isn't the Raw data from the sensor? Sure, it's not in the Bayer array, but the sensor for the Epson has three color elements for each pixel so it could be looked at as raw data.

Using Vuescan it's obvious that what is stored on the disk as a raw file is what the software uses to perform an adjusted scan. In other words all the sensor data (including the IR channel, if scanned) is captured. Then Vuescan processes this in the same manor as Adobe processes a camera generated Raw file (but without needing to do the demosaicing step, but it does need to apply the dust correction step).