Use of IT8 when outputting to RAW with ViewScan

Hallo,

I am just beginning developing slides by myself and I would like to refine my scanning abilities. I came here from APUG as the questions I need to be answered are of "hybrid" nature. I need some clarification regarding the rendering of the RAW scans and its relationship with colour management and I would be grateful of any help. I have read a lot of what already exists in the forum.

FOREWORD
I use VueScan to scan my slides for stock agencies. My monitor is profiled with ColorVision Spyder. I have WindowsXP, the .icm profile thus generated sits in the C:\WINDOWS\system32\spool\drivers\color directory as usual.

I have used the IT8 targets by Faust, and I have created the .icc file with the VueScan procedure.

In VueScan, under Color - Scanner color space I set "ICC Profile". Immediately under this, I point to the .icc profile obtained with Faust targets.
In VueScan - Scanner IT8 data I point to a copy of the data which come with the targets.
I scan "Input" as "Image" (a setting in VueScan).

I did not create any specific "film" profile.

I always scan to a RAW, linear DNG file which I then "develop" with Lightroom 2.3, export to TIFF, work a bit with Photoshop.
The final TIFF file must adopt the AdobeRGB colour space. So in VueScan "Color - Output color space" is set to "AdobeRGB" and both Lightroom and Photoshop are also set to describe colours in the AdobeRGB color space.

I always use VueScan with "Multiexposure".

Given this setup, here are the questions:

Considering that I already have my Monitor profile sitting in the system directory and loaded at system startup, I suppose I must have Vuescan "Color - Monitor color space" simply set to "sRGB". I presume that setting VueScan to "ICC profile" and pointing to my profile would load the monitor profile "twice" and would be wrong. Is this correct?

Considering that VueScan generates DNG RAW files, does it make any sense that I configure 'Color - Scanner color space' to my ICC profile, as I do? Or should I use those profiles somehow inside Lightroom during conversion? Said in other words, does the RAW file which is generated by VueScan "take into consideration" my scanner profile, or is it "pure raw" and totally independent from any settings in the "Color" menu included calibration settings?

Given my setup, am I making any use of Wolf Faust profiles? Or does the use of the RAW format in VueScan defeat the related configuration?

Considering that I always have VueScan output RAW files to be processed with Lightroom, do I have to fiddle with scan exposure values, adjusting black point and white point with the help of the graph after each preview, or can I skip that step altogether? Said in other words: is the RAW file generated by VueScan dependent from the exposure values?

Many thanks for your kind answers.

Fabrizio

PS I have already emailed Ed Hamrick about those questions, he answered that he has no time to read long messages and that I should write a simplified version of it in a few weeks because he's working at VueScan. I did not find anything to dissolve my doubts on the internet, so you probably are my last hope.

I did some research and some experiment and I think I can answers some questions.

Diapositivo said:

Considering that I already have my Monitor profile sitting in the system directory and loaded at system startup, I suppose I must have Vuescan "Color - Monitor color space" simply set to "sRGB". I presume that setting VueScan to "ICC profile" and pointing to my profile would load the monitor profile "twice" and would be wrong. Is this correct?

Click to expand...

It must be correct. The option to pass the monitor profile, which make no sense on the Windows and MacOS platforms, is probably there for the benefit of Unix users where, if I understand correctly, the system is not color managed but the single applications can support color management.
This feature is useless, or actually misleading, on the Windows and MacOS versions.

Diapositivo said:

Considering that VueScan generates DNG RAW files, does it make any sense that I configure 'Color - Scanner color space' to my ICC profile, as I do? Or should I use those profiles somehow inside Lightroom during conversion? Said in other words, does the RAW file which is generated by VueScan "take into consideration" my scanner profile, or is it "pure raw" and totally independent from any settings in the "Color" menu included calibration settings?

Click to expand...

It makes sense, it takes it into consideration. DNG file generated with and without the icc profile are (slightly) different when opened in Lightroom. Basically, the scanner icc profile is the ONLY configuration in the Color section that makes a difference when outputting raw files.

Diapositivo said:

Considering that I always have VueScan output RAW files to be processed with Lightroom, do I have to fiddle with scan exposure values, adjusting black point and white point with the help of the graph after each preview, or can I skip that step altogether? Said in other words: is the RAW file generated by VueScan dependent from the exposure values?

Click to expand...

With my setup, changes to configuration in the Color section do not have any effect on the raw DNG generated by VueScan, so one can just skip any adjustment of White point (%), Black point (%), Curve low, Curve high, Brightness.

So to sum it up: the profile obtained with IT8 targets is embedded into DNG raw files and is interpreted by Lightroom. All other configuration options in the Color section of VueScan are just misleading when outputting raw files and totally irrelevant. Monitor colour space should be set to sRGB on machines which don't have a monitor profile or that have a monitor profile at system level (Windows, MacOS).

Fabrizio

Diapositivo said:

I did some research and some experiment and I think I can answers some questions.



It must be correct. The option to pass the monitor profile, which make no sense on the Windows and MacOS platforms, is probably there for the benefit of Unix users where, if I understand correctly, the system is not color managed but the single applications can support color management.
This feature is useless, or actually misleading, on the Windows and MacOS versions.



It makes sense, it takes it into consideration. DNG file generated with and without the icc profile are (slightly) different when opened in Lightroom. Basically, the scanner icc profile is the ONLY configuration in the Color section that makes a difference when outputting raw files.



With my setup, changes to configuration in the Color section do not have any effect on the raw DNG generated by VueScan, so one can just skip any adjustment of White point (%), Black point (%), Curve low, Curve high, Brightness.

So to sum it up: the profile obtained with IT8 targets is embedded into DNG raw files and is interpreted by Lightroom. All other configuration options in the Color section of VueScan are just misleading when outputting raw files and totally irrelevant. Monitor colour space should be set to sRGB on machines which don't have a monitor profile or that have a monitor profile at system level (Windows, MacOS).

Fabrizio

Click to expand...

FWIW, DNG and RAW camera files have no associated ICC profile. That's the point of processing RAW files.

I've not dorked around with this setting in Vusescan but my understanding of a Vuescan RAW file is to allow Vuescan to rescan the file without having to physically rescan the negative.

I've not updated my version of Vuescan in a while so this maybe some new feature I've not seen (scan to DNG) but that could be an interesting feature, especially since I really like LR 3.2. I may have to update to the latest version since I have some legacy slides I need to scan.

Don

I teach an intro to scanning class for HCP here in Houston, and I always advise the newbies to scan their files into RAW format so they don't have to futz around with anything but the scanner profile created from an IT8 target. The ability to worry about the color balance later in a much more user friendly tool (either Lightroom or Camera RAW) is a huge benefit, IMHO. The only things you need to worry about on the original scanning job is that you don't clip anything and scan it all at 16 bit color depth.

clay said:

I teach an intro to scanning class for HCP here in Houston, and I always advise the newbies to scan their files into RAW format so they don't have to futz around with anything but the scanner profile created from an IT8 target. The ability to worry about the color balance later in a much more user friendly tool (either Lightroom or Camera RAW) is a huge benefit, IMHO. The only things you need to worry about on the original scanning job is that you don't clip anything and scan it all at 16 bit color depth.

Click to expand...

Yeah I just checked out the docos for the latest version Vuescan and real DNG files can be created with the Pro version. I'm glad I bought the Pro version years ago since the user gets free user upgrades. Thanks for verifying that feature Clay.

The TWAIN plugin is also a new twist.

Don

Don Bryant said:

FWIW, DNG and RAW camera files have no associated ICC profile. That's the point of processing RAW files.
Don

Click to expand...

They can have, so that your raw developer will use the profile associated to your scanner in creating the TIFF (or JPEG).

I did some tests yesterday. Raw files with different profiles are differently rendered by Lightroom. I.e., the raw itself is probably identical but the associated scanner (and in case film) profile is honoured by Lightroom during the development.

DNGs created by viewscan are "linear DNG" type and so not the common flavour of DNG. Many programs would not support them (e.g. PhotoMechanics) but Lightroom (and probably many other raw developers) allows you to work with them as if they were raw files from a camera, so you use the same workflow with scans and with digicaptures.

The amount of "rowness" of a VueScan raw file is user-selectable. You can have the raw scan to be written after dust cleaning with infrared, after grain reduction, and also after film curve application by VueScan.

For instance, if you scan negatives, you can have a "positive raw" instead of a "negative row". This, again, will put you in a situation similar to what you normally have when working with raw files generated by a digital camera.

It's a pity VueScan is so horribly documented.

Fabrizio

Diapositivo said:

They can have, so that your raw developer will use the profile associated to your scanner in creating the TIFF (or JPEG).

I did some tests yesterday. Raw files with different profiles are differently rendered by Lightroom. I.e., the raw itself is probably identical but the associated scanner (and in case film) profile is honoured by Lightroom during the development.

DNGs created by viewscan are "linear DNG" type and so not the common flavour of DNG. Many programs would not support them (e.g. PhotoMechanics) but Lightroom (and probably many other raw developers) allows you to work with them as if they were raw files from a camera, so you use the same workflow with scans and with digicaptures.

The amount of "rowness" of a VueScan raw file is user-selectable. You can have the raw scan to be written after dust cleaning with infrared, after grain reduction, and also after film curve application by VueScan.

For instance, if you scan negatives, you can have a "positive raw" instead of a "negative row". This, again, will put you in a situation similar to what you normally have when working with raw files generated by a digital camera.

It's a pity VueScan is so horribly documented.

Fabrizio

Click to expand...

Fabrizio,

Thanks for the additional input, what you say makes more sense to me now that I've had time to reflect on the issue at least in regards to editing DNG/RAW in ACR or LR. These files aren't generated by a camera so LR/ACR recognize a device tag like they would a camera and offer a default profile of variations of a default.

Don

Don Bryant said:

Fabrizio,

Thanks for the additional input, what you say makes more sense to me now that I've had time to reflect on the issue at least in regards to editing DNG/RAW in ACR or LR. These files aren't generated by a camera so LR/ACR recognize a device tag like they would a camera and offer a default profile of variations of a default.

Don

Click to expand...

I now understand what you mean. Well, I'm not sure of what really happens in Lightroom.

If I generate a scanner profile with VueScan, and I configure VueScan to embed this scanner profile, the DNG file that I see in Lightroom has got a different aspect (is actually correctly profiled) than the DNG I see when I use the "built-in" scanner profile.

If I scan a TIFF and generate a profile with SIPC, and I configure it in VueScan, and make a scan, and embed the SIPC-generated profile, the file I see in Lightroom is identical to the "built-in" profile as above.

This means Lightroom, for a reason unknown to me, correctly reads the profile generated by VueScan, but does not read the file generated by SIPC.

If I open a TIFF file in Photoshop and manually apply the scanner profile generated by SIPC I can clearly see the profile in action so the profile generated by SIPC is valid and works.

In Lightroom, Develop - Camera configuration, I have no possible choice. In the drop-down menu I only see one choice, "Embedded", and this is always ticked, and I have no other possibility. This happens in all three cases: DNG with VueScan profile, DNG without profile, DNG with SIPC profile.

Reading the documentation I see that Adobe clearly states that the profiles that Lightroom uses (in Develop - Camera configuration) are NOT common ICC profiles.

So there are a few things to understand here:

1) Why is Lightroom able to interpret the ICC scanner profile generated by VueScan, and not able to interpret the ICC scanner profile generated by SIPC?

2) Is there a way I can generate a SIPC profile (better than the VueScan one, which is a simplified form of ICC profile) and have Lightroom see it?

At the moment I am inclined to think that:

The fact that VueScan profile is interpreted by Lightroom is pure chance. VueScan generates a very simple file (a few rows of text), and probably this "makes sense" to Lightroom. SIPC generates a very long file (491 kB) and this "does not make sense" to Lightroom, which expects a profile in its own format;

If I really want to use the SIPC-generated profile I must have the scanner generate a TIFF and work in Photoshop only. This would be bad because Lightroom allows a much easier correction of white balance. But performing the white balance correction in Lightroom only to see colours change when I pass the file to Photoshop does not make sense. Scanner profiling is to be applied before white balancing.

The culprit in all this is probably Lightroom. I will experiment in the next days with other DNG developers and see if they interpret profiles generated by SIPC.

Fabrizio

Diapositivo said:

If I scan a TIFF and generate a profile with SIPC, and I configure it in VueScan, and make a scan, and embed the SIPC-generated profile, the file I see in Lightroom is identical to the "built-in" profile as above.

This means Lightroom, for a reason unknown to me, correctly reads the profile generated by VueScan, but does not read the file generated by SIPC.
Fabrizio

Click to expand...

Not so. I just made a test. Viewscan has a preview which is aware of the profile. If I scan a preview without profile, and then apply the scanner profile generated by VueScan, the preview changes.

If I scan a preview without profile, and then apply the scanner profile generated by SIPC, the preview does NOT change.

That means that it is VueScan in the first place which does not interpret proper ICC scanner profiles and so probably does not embed them correctly either.

Probably VueScan is able to work only with the (simplified) profiles that generates itself.

Fabrizio

Wow, this is very interesting information. Thanks for diving into the inner workings of Vuescan/Lightroom. I do know that it works well, but from what you have discovered, it may only be just good luck that Lightroom understands how to interpret the Vuescan ICC profiles.

Clay, we have two sets of problems here:

1) VueScan has problems with common ICC profiles. VueScan deals correctly only with the profiles that generates itself.

2) Lightroom has no support for common ICC profiles. Lightroom only supports its own format of colour profile. Nonetheless, Lightroom will correctly interpret the scanner profile generated by VueScan, as both you and me have experienced.

If we want to use a better, proper ICC scanner profile (such as one generated by Lprof, SIPC, Argyll etc.) and want to maintain the convenience of working with a raw file, the only solution we have is to find a raw converter able to read a "proper" profile.

I tried the demo of Bibble yesterday, but it does not have support for DNG. It has support for certain cameras which use DNG as raw format, but no support for "stand-alone" DNG. It just does not show a DNG file in its interface.

I would be grateful if anybody using a DNG-compatible raw developer would make some test and tell me if it supports a "proper" ICC scanner profile. No need to generate a profile, nor to have a scanner.

I can send the SIPC generated profile and the tester should just verify if an image (DNG) changes appearance when the profile is applied to it. Raw developer should support "linear DNG" because that it what scanners produce.

Fabrizio

Sorry for digging up a slightly older thread but I'm currently looking into profiling my scanner and came upon this.

Adobe Camera Raw and Lightroom (I'm pretty sure) can operate on tiff files. There really isn't much reason to have Vuescan save DNG 'raw' files. Save them as tiffs instead. You can use them in almost any program then.

The only advantages working with a DNG file are the following. First, your Camera Raw or Lightroom image settings can be saved in the file itself. However, in my experience, Camera Raw does the same for tiffs, so maybe this isn't really an issue. Secondly, DNGs open by default in Camera Raw/Lightroom. Tiffs open in Photoshop/Preview/etc. However, there is a preference in Camera Raw that lets you toggle which tiff files to open in Camera Raw (all, ones with CR settings, none). To open a DNG in PS initially, go to File>Open, select your file, and then choose Camera Raw as the format. Of course in LR, I would imagine you just drag them into the program to edit.

I recall reading a thread in which Ed Hamrick himself stated that Vuescan uses a Matrix (simple, but not "simplified") profile, as opposed to a LUT (look-up table) profile. As far as I know, which is--caveat emptor--not very far, a matrix profile is entirely appropriate and sufficient for a scanner.

Read: Graphics.com >> Sections >> Insight >> Practical Color Management: Eddie Tapp on Digital Photography

I would imagine that your profiling device/software should be able to make matrix as well as LUT profiles.

Well, I just discovered DPUG today, and found tis topic, which is right at home with me right now.

I am in the middle of a scanning project for a big show, and I have several new variables, a new (used scanner), new software (Vuescan), new targets (faust), and a new printer.

My biggest problem is getting colors to match my slides with the Minolta 5400/Vuescan combo. So far my best attempts are scanning to DNG and manually by eye matching the colors from the slide looking through a loupe. I (stupidly) initially did this to the lamp in my office - and have since invested in a small 5K light table that I can put my slides on and judge color from that.

I also have the Epson V750, and it seems with the epson software, and the bundles Silverfast, I have no issues getting the right colors.

I am wondering what is the best workflow, I have tried several things, and there are so many options in Vuescan that it makes my head spin. Contrary to what most people think that I read about in photo forums, I actually find Silverfast way easier to use. But also very expensive, and not very good about keeping up to date with my OS.

Is it confirmed that you can use the faust targets while scanning to DNG and the correct profile can be viewed in LR/PS?

Also with my current method of matching by eye, when I scan to DNG, and make my adjustments, it seems that the colors are almost too vibrant? I can't explain it, but even just taking a little less saturation out then just makes the image dull. I thought I could master this by just reading some info on-line, but I think I am, for the first time, a bit over my head with this color management stuff, at least with Vuescan... I don't know, maybe it's the minolta scanner... I've never had such troubles with digitally captured files. What I see on my monitor is what I get in print exactly, or at least close enough for me.

gnuyork said:

Well, I just discovered DPUG today, and found tis topic, which is right at home with me right now.

I am in the middle of a scanning project for a big show, and I have several new variables, a new (used scanner), new software (Vuescan), new targets (faust), and a new printer.

My biggest problem is getting colors to match my slides with the Minolta 5400/Vuescan combo. So far my best attempts are scanning to DNG and manually by eye matching the colors from the slide looking through a loupe. I (stupidly) initially did this to the lamp in my office - and have since invested in a small 5K light table that I can put my slides on and judge color from that.

I also have the Epson V750, and it seems with the epson software, and the bundles Silverfast, I have no issues getting the right colors.

I am wondering what is the best workflow, I have tried several things, and there are so many options in Vuescan that it makes my head spin. Contrary to what most people think that I read about in photo forums, I actually find Silverfast way easier to use. But also very expensive, and not very good about keeping up to date with my OS.

Is it confirmed that you can use the faust targets while scanning to DNG and the correct profile can be viewed in LR/PS?

Also with my current method of matching by eye, when I scan to DNG, and make my adjustments, it seems that the colors are almost too vibrant? I can't explain it, but even just taking a little less saturation out then just makes the image dull. I thought I could master this by just reading some info on-line, but I think I am, for the first time, a bit over my head with this color management stuff, at least with Vuescan... I don't know, maybe it's the minolta scanner... I've never had such troubles with digitally captured files. What I see on my monitor is what I get in print exactly, or at least close enough for me.

Click to expand...

I use a Minolta scanner (the cheaper Scan Dual IV), but with the KM Scan utility, and I dare say that I had absolutely no issue with colour, it's always spot on! So far I used it with two colour negative films, Portra and Ektar; the next will be Fuji 400H as I am probing the available films. Apparently there is significant difference how native and third party drivers (like Vuescan) operate. From analogous past experience with KM RAW decoding software I can say that though a bit clumsy and "temperamental" at times it always produced exceptional colour unmatched by anything else.

Diapositivo said:

Clay, we have two sets of problems here:

1) VueScan has problems with common ICC profiles. VueScan deals correctly only with the profiles that generates itself.

2) Lightroom has no support for common ICC profiles. Lightroom only supports its own format of colour profile. Nonetheless, Lightroom will correctly interpret the scanner profile generated by VueScan, as both you and me have experienced.

If we want to use a better, proper ICC scanner profile (such as one generated by Lprof, SIPC, Argyll etc.) and want to maintain the convenience of working with a raw file, the only solution we have is to find a raw converter able to read a "proper" profile.

I tried the demo of Bibble yesterday, but it does not have support for DNG. It has support for certain cameras which use DNG as raw format, but no support for "stand-alone" DNG. It just does not show a DNG file in its interface.

I would be grateful if anybody using a DNG-compatible raw developer would make some test and tell me if it supports a "proper" ICC scanner profile. No need to generate a profile, nor to have a scanner.

I can send the SIPC generated profile and the tester should just verify if an image (DNG) changes appearance when the profile is applied to it. Raw developer should support "linear DNG" because that it what scanners produce.

Fabrizio

Click to expand...

I am sorry to disagree with as what scanners usually produce is a TIFF file, linear or otherwise. The "scanner RAW/DNG" is a bit of a misnomer really, invented I gather by software vendors as a push sales trick.

I.G.I. said:

I use a Minolta scanner (the cheaper Scan Dual IV), but with the KM Scan utility, and I dare say that I had absolutely no issue with colour, it's always spot on! So far I used it with two colour negative films, Portra and Ektar; the next will be Fuji 400H as I am probing the available films. Apparently there is significant difference how native and third party drivers (like Vuescan) operate. From analogous past experience with KM RAW decoding software I can say that though a bit clumsy and "temperamental" at times it always produced exceptional colour unmatched by anything else.

Click to expand...

Thanks for the reply....though my show has since been exhibited and now is down. I ended up still using Vuescan and got through it, but I remember it was more difficult process than I expected. I scanned to DNG with VueScan and made adjustments meticulously by eye in LR4, mostly adjustiung White Balance to match the slide on the light table.

Also I see you mention color neg film, but how about slides?

I have not tried the bundled KM software, because I think it's too old to run on my MAC? Not sure. I think I tried to google search a newer version to download, but I think I didn't have success...can't remember...

Whatever I did ended up working out, but it was quite a bit more of a process than what I would have liked...and I'm sure it's totally not the right way to do it... but I had a deadline and have to sally fourth.

As it was the guy I hired for the framing had me stressed. He waited to the last possible minute (was busy with his own show) to do around 30 pieces, some of them extremely large (24 x48). I ended up having to leave a day later for my show, and the night before and at around 11:00 or midnight I went to his studio to make sure he was going to finish - coach him on... I think I left two hours later with a 40 minuter drive home, then had to get up at 5 am to drive to Alabama...for the hanging of the show that morning. That experience made me a bit older, lol.

I don't shot slides (for the moment) so I can't comment on that.

I run the KM software on a PPC Mac with 10.5.8, and it works; some report it works on Intel too, but I have no direct experience. PPC machines are cheap nowadays to the point it's feasible to buy one just to run the scanner driver.

Well, my experience taught me that there is no substitute for an optimised scan. Photo editors (PS/LR/ and the like) may fine tune it, but are no corrective for improper/wasted previous stage (scanning). I consider myself reasonably adept in PS yet most of the time I could not get the results as good from PS (starting with average/mediocre scan) as from an already perfected scan; when I got close it took me far much time with PS than the few minutes with the Scan utility. To use the parallel, in the analogue-digital domain the scanner plus the scanning software is what the digital camera plus the RAW utility (or the in-camera processing) is in the pure digital. What is missed in capture-conversion stage cannot be restored in post; at most it could be masked to some degree.

I know the above contradicts what many espouse, but that's life...

Sorry

An apology to everybody for drifting off topic