- Joined
- Oct 11, 2006
- Messages
- 2,185
- Format
- Multi Format
My question to Hamrick:
...If I acquire in 8 bits and save raw in 8 bits, is there any gamma applied? ...The raw files don’t have any gamma applied...if I acquire in 16 bits and save as 8 bit raw, is it a straight scaling to 8 bits? His answer: When saved at 8 bits per sample, it’s saved with a gamma 2.2 lookup table...
Yes, your file raw0002 has more noise than raw001.My question to Hamrick:
if I acquire at 16 bits and save raw in 16 bits is there any gamma applied? If I acquire in 8 bits and save raw in 8 bits, is there any gamma applied? Is the behavior the same between color and monochrome capture?
his answer:
The raw files don’t have any gamma applied.
I then asked a follow up question:
if I acquire in 16 bits and save as 8 bit raw, is it a straight scaling to 8 bits?
His answer:
When saved at 8 bits per sample, it’s saved with a gamma 2.2 lookup table.
so, from the files I’ve provided, acquiring in 16 bit mode and saving in 16 bit raw has no gamma, acquiring in 8 bit and saving in 8 bit has no gamma, so raw0001 and raw0002 can be directly compared, and acquiring in 16 bit mode on an Epson V850 Pro has significantly less noise and more DR than acquiring in 8 bit mode.
I'm seeing some very interesting issues with in the scans you made available. The noise isn't random but it's very structured. As I look into this I'm seeing a similar result in some scans on my side.My question to Hamrick:
if I acquire at 16 bits and save raw in 16 bits is there any gamma applied? If I acquire in 8 bits and save raw in 8 bits, is there any gamma applied? Is the behavior the same between color and monochrome capture?
his answer:
The raw files don’t have any gamma applied.
I then asked a follow up question:
if I acquire in 16 bits and save as 8 bit raw, is it a straight scaling to 8 bits?
His answer:
When saved at 8 bits per sample, it’s saved with a gamma 2.2 lookup table.
so, from the files I’ve provided, acquiring in 16 bit mode and saving in 16 bit raw has no gamma, acquiring in 8 bit and saving in 8 bit has no gamma, so raw0001 and raw0002 can be directly compared, and acquiring in 16 bit mode on an Epson V850 Pro has significantly less noise and more DR than acquiring in 8 bit mode.
I have a copy of an image processing program called ImageJ. I just learned that there is a plugin available for ImageJ that allows one to look at TIFF Metadata. The plugin is called "TIFF Tags." This might allow me to figure out some of the details of the file formats that Vuescan saves, such as whether an 8 bit "RAW" file is gamma encoded, and possibly something about whether there is a color profile.
This information might also come in handy for some of you if you are interested in looking at TFF Metadata in images.
Anyway, if a scan is saved as an 8 bit file then I call it an 8 bit scan, and I think most other people would also call it an 8 bit scan. I really don't know what else it would be called.
So if I scan at 8 bit, and save at 16 bit, is that a 16 bit scan?
Sorry, but if you scan at 16 bits, and downsample to 8 bits, that's a different result than scanning at 8 bits and saving at 8 bits.
16 bits converted to 8 bits is 8 bits, so yes, if you acquire an image in 16 bits and store it as an 8 bit image it is an 8 bit scan.So if I scan at 8 bit, and save at 16 bit, is that a 16 bit scan?
Sorry, but if you scan at 16 bits, and downsample to 8 bits, that's a different result than scanning at 8 bits and saving at 8 bits.
16 bits converted to 8 bits is 8 bits, so yes, if you acquire an image in 16 bits and store it as an 8 bit image it is an 8 bit scan.
I'll explain in more detail below, but to keep this simple I'm not going to get into the issue of gamma encoding in this post. If you want to see how gamma encoding would affect the result we can have a separate and longer discussion, but the bottom line is that gamma encoding would not make a noticeable difference.
Consider a simple thought experiment. Suppose you acquire an analog signal (from a single pixel of the image) with an ADC, and it turns out to be 1111001100000111 in its 16 bit binary representation. Suppose you convert it to an 8 bit number and store it. The stored result is 11110011 in its 8 bit binary representation. Now suppose you were to acquire that same signal with an 8 bit digitizer. The value would be 11110011 in its binary representation, and you would store that binary number as 11110011 in an 8 bit word.
So In the first instance you would store the binary number 11110011, and in the second instance you would store the binary number 11110011. Can you explain the difference between those two numbers?
So yes, if you acquire a scan in 16 bits and it store it in 8 bits it is an 8 bit scan.
Thanks. I found exiftools, and also a version of it in a nice GUI wrapper called ExifToolGUI.just use exiftool. You can do a straight dump of all the metadata in any image file (including camera raw files) and inspect the data in the file as closely as you want to.
As my dissertation advisor used to say, theory proposes and experiment disposes. I discussed theory in my last post. Now I will test by experimental results whether scanning an image using an Epson V750 using vuescan and saving the result as an 8 bit scan produces the same result regardless of whether the input of the scan is set at 16 bits or 8 bits.
Here's the experimental setup. I did two scans. For each I scanned an image with transparent and dark regions. (See post #118 to see what the full scan region looks like.) Although previously I scanned the same image in gray scale mode, for these results I scanned in rgb mode. I then selected a 20 x 20 pixel region from the lower left corner of the scan, which is where the image was the darkest possible because it is far from the light portion of the image and in the selected region the light would have had to penetrate a book, a black piece of paper, and the black plastic frame of a film holder before reaching the detector. (That's assuming that no light reaches the detector from an indirect scattered-light path.)
I then generated histograms from each image using photoline software. I zoomed the horizontal scale of the histograms in order to expand it to better determine what the distribution looks like. I zoomed in a single step. The zoom factor was 2.5%. I also extracted the intensity values from the histograms and calculated the means and standard deviations of the two distributions, taking the peaks to be separated by one unit on the zoomed horizontal axis.
Here are the images and the histograms.
View attachment 301684
View attachment 301685
Just Eyeballing the two images I would be hard pressed to tell them apart, so I did some statistics. For the scan with the input set to 8 bits and the output set to 8 bits the mean was 0.9500, and the standard deviation was 0.9526. For the scan with 16 bits in and 8 bits out the mean was 1.0300, and the standard deviation was 0.9078. These results are statistically indistinguishable.
For the scans to be statistically distinguishable would require either that the means from the two scans be further apart or the standard deviations would have to be smaller. Therefore, based on these experimental results if you save the result in 8 bits it does not matter if the input parameter is set to 8 bits or 16 bits because the results are statistically indistinguishable.
This test applies to the specific combination of scanner and software used. It it unknown what would happen with a different scanner/software combination.
Also, this particular test was not designed to determine whether 16 bit scans (which require data to be stored in 16 bit format in order for them to be classified as legitimate 16 bit scans) have lower dark noise than 8 bit scans using this scanner/software combination. That's a different question than the one addressed in this post.
Or to put the results from this test in slightly difference words, a scan with the input parameter set at 16 bits and the output parameter set at 8 bits is an 8 bit scan because It has the same storage requirement as any other 8 bit scan and the results can't be statistically distinguished from any other 8 bit scan.
post your raw files so others can download them.
I zipped four relevant files, but that was also too big to upload.post your raw files so others can download them.
I tried zipping and uploading the two files from which I took the crops and that I analyzed in post #187. The zip file was too big to upload.post your raw files so others can download them.
I zipped the files with the 20x20 pixels I cropped from the two tiff images I discussed in one of my most recent posts. The one that includes images of the cropped regions, the histograms, and the statistical analysis. The files with 0006 in the file names were 8 bit acquire and 8 bit save. The files with 008 in the file names were 16 bit acquire and 9 bit save. Each of those are in pairs, one is with no manipulations done on them other than cropping. The other is after I did a 2.5% histogram zoom on them. This way you can do your own histogram zooms and compare them to mine, or if you prefer you can just use my histogram zooms.post your raw files so others can download them.
I'm not sure you understood the question I was addressing in post #187. Let me be clear about the question I was addressing in that post.Let me be really super specific. Post two raw scans. One acquired at 16 bits and saved as 16 bit tiff raw, and one acquired at 8 bits and saved as 8 bit tiff raw. Scan at the maximum resolution (6400dpi for the 750 right?) and scan a crop out of the middle of the blacked out area that is approximate 24x36mm for each bit depth in full RGB. The only way to get unadulterated raw sensor samples is to save as tiff raw at the same bit depth that you are acquiring at. Plain and simple. If you do that, you can then directly compare the two raw files I posted to your raw files and vice versa. All this other stuff you're doing is just mud in the water. Keep it simple. Generating those two raw files and uploading them should literally take less than 10 minutes.
We want to look at the raw samples acquired at 16 bits and the raw samples acquired at 8 bits. The 16 bit raw should be saved at 16 bits raw and the 8 bit raw should be saved at 8 bits raw. If you don't save as tiff raw, Vuescan applies the output color processing, which is probably totally bunging things up and making any comparisons difficult if not impossible to do. The tiff profile checkbox only tells Vuescan whether to embed the ICC profile into a tiff chunk or not. It doesn't control what color processing Vuescan does on the samples before it saves them.
If you can't manage getting something that Vuescan directly generated being made available, then send it to my public dropbox file request and I'll post it to my download server so everybody can download the Vuescan generated raw files, where the official documentation says that it applies gamma to 8 bit files, but according to Adrian's communication with Hamrick it does not apply gamma to 8 bit raw files if the input parameter is set to 16 bits.
Here's the dropbox link: https://www.dropbox.com/request/4vY3BvYAVZcWBwy4oCNh
No dropbox account is required. Just zip up the two raw files and upload the zip. I'll put it up on my download server and post the URL here when I get the time.
I'm not sure you understood the question I was addressing in post #187. Let me be clear about the question I was addressing in that post.
In post #182 member grat asserted the following:
"So if I scan at 8 bit, and save at 16 bit, is that a 16 bit scan?
Sorry, but if you scan at 16 bits, and downsample to 8 bits, that's a different result than scanning at 8 bits and saving at 8 bits."
I was testing that assertion by doing two scans, one in 16 bit acquire 8 bit save and one is 8 bit acquire and 8 bit save. Those are exactly the conditions addressed by grat in post #182. Grat's post said nothing about raw scans, so I was not testing anything about raw files in that test. I was testing normal scans. All parameters were the same in the two scans except for the input bit width. Since all parameters were the same then vuescan must have given them the same treatment other than converting 16 bits to 8 bits. Otherwise vuescan is a pathelogical program, which I don't believe is the case. If vuescan did give the two scans different treatments (such as applying different color profiles to the two scans) then it is more than a miracle that the two scans turned out be be not distinguishable despite being given different treatments. (Specifically the scans were not distinguishable at the 0.01 significance label as calculated using an only significance calculator found at this website: https://www.socscistatistics.com/tests/studentttest/default2.aspx.)
As for vuescan applying an unrequested color profile to the scans, I can find no documentation that vuescan does that. If you think it does then please supply the documentation that vuescan does it. Otherwise there is no reason to assume that it does. Even if it does give scans an unrequested profile, why would anyone assume that it give different profiles to scans that are performed identically except for the input bit width setting? (By the way, exposure and other relevant parameters were fixed.)
I did notice one thing in my scans where vuescan applies its own correction. I had color balance set to "White Balance". However, since the scans were all of the exact same image any automatic white balance correction would be the same for all. It is possible to set color balance to "None", which is something I could try. I highly doubt that will make a difference, but as I said "Theory proposes, experiment disposes." - Oops, I need to take that back about the color balance setting. That setting is available in vuescan when I use my canon fs4000us scanner, but that setting is not available when I use my Epson v750 scanner.
Ah, one more thing. I see in the vuescan documentation that "saved TIFF files are always gamma corrected according to the Color | Output color space". The gamma used is 2.2 except for a few specific cases. There is nothing there saying that it does anything differently for 8 bit vs. 16 bit images with respect to gamma correction, an exception being for raw files which according to the official documentation vuescan saves 8 bit files in a gamma encoded form. (However, refer to Adrian's communication with Hamrick about how bit width relates to gamma when it comes to raw files, which seems to contradict the official documentation.)
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?