Sensor noise in film scanner

[alanrockwood]

Can anyone comment on sensor noise in a film scanner? I have a Canon FS4000us scanner, which is considered to be an excellent scanner. I have done some rough testing of the scanner, and it looks like the noise level is several bits, even when scanning in 8 bit mode.

I still haven't yet done a rigorous test, but I thought I would see if anyone here has done a rigorous test of their scanner, or even a non-rigorous test, or even has an informed opinion, or even a non-informed opinion.

 

[DWThomas]

Not familiar with that scanner, but that number sounds high. Is the scanner operated via some application on a computer? And does it default to a very automatic mode? If all that is so, what are you scanning for the test -- just a blank ("clear") piece of film or some sort of test frame?

We just had a discussion a few days ago about someone who had some very thin negatives and was getting gobs of noise. The typical "magic" scanning software seems to crank up the curves to try and see a wide dynamic range no matter what. With the scanners I have, if you use an auto-type scan on a piece of clear film, it will be amped up to try to make 10 bits of range out of 2 bits worth of density and show every microscopic defect in sight, but that's not a "real world" situation. I normally use VueScan and adjust the scan parameters manually.

That's my not totally uninformed speculation.

 

[alanrockwood]

Here's what I did. (As I said, so far my testing has been non-rigorous). First I scanned a very dense (fully exposed to room light and developed) with half of the frame being air and half being the exposed film. Both the white and dark parts of the scan had a distribution of values, i.e. the dark parts were not a single value and the light parts were not a single value.

This is a pretty good indication that there is noise in the whites. It is less definitive about the dark region because a little bit of light might be getting through and part of the peak width (i.e. distribution of values) could be film grain.

Then I did a scan and looked ad the region in the frame around the image. That region is imaging thick black plastic, so no light should be getting through. That too had some width in the distribution of values. It is also not quite definitive because the sensor could be picking up some stray light from somewhere.

All of the above were with 8 bit scans. 16 bit scans (actually 14 bit) showed the same thing.

Now, one of the criticisms that some people have leveled at CCD sensors in this class of scanner is that they have sensor noise. This is contrasted with photomultiplier tubes in drum scanners which have very low sensor noise, probably too low to detect would be my guess. If this criticism is true then when scanning a pure black region there should be a distribution of values in the region. The same goes for a pure white region or any region in between.

Now, to bring this around to something practical, there has been considerable discussion about scanning in 16 vs. 8 bit mode. In other threads I pointed out that if film grain is equivalent to about 1 bit of RMS noise (or even a little less) then 8 bit mode captures all of the useful information in the scan. The same thing applies to sensor noise. If RMS sensor noise is of the order of 1 bit then there is virtually no significant advantage to using a higher bit depth. (However, before doing any post processing in photoshop or whatever softwere there is an advantage in converting an 8 bit scan to 16 bits.)

Let me comment on dynamic range. Adding more bits does not improve dynamic range if your sensor noise is of the same order of magnitude as the least significant bit in 8 bit mode.

This all comes back to the question of just how much sensor noise there is in a scanner. If the sensor noise (actually, sensor noise plus film grain combined) is much less than a 1 bit step of an 8 bit analog to digital converter then there is an advantage to scanning in 16 bit mode. If the sensor noise is roughly comparable to a 1 bit step of an 8 bit ADC (or worse) then there is no significant advantage to stepping up to 16 bit mode when scanning. Hence, there is a good reason for trying to determine how much sensor noise a given scanner has.

 

[Ted Baker]

All of the above were with 8 bit scans. 16 bit scans (actually 14 bit) showed the same thing.

What method did you use to make an 8Bit scan? i.e. how did you take output from your scanner which may have a 14bit A/D to convert it 8Bits?

 

[Patrick Robert James]

Sorry I didn't read all of your last post. Not enough coffee today.

I am guessing your problem is flare from dust or the lens or sensor is dirty. These days I use a Nikon 4000 and I have to clean it every now and then. The difference can be startling after it is cleaned. I've never had any issues with clean scanners. Keep in mind the scanner is probably at east 15 years old.

If you are concerned about sensor noise, use multisampling in software if you have it which will deliver a clean file. Vuescan does it if the Canon software doesn't.

I scanned this on a Canon 4000 many years ago, and processed the image in Photoshop 4. I tried to improve it once a couple years ago with a scan on my Nikon and modern software, but couldn't. The Canon is an excellent scanner.

Hope that helps you.

 

[alanrockwood]

What method did you use to make an 8Bit scan? i.e. how did you take output from your scanner which may have a 14bit A/D to convert it 8Bits?

Using Vuescan I can set the acquisition to 8 bit or 16 bit. No doubt the hardware of the FS4000us scanner uses a 14bit ADC and converts the result to 8 bit before storing the data. For example, if the 14 bit binary number from the ADC were 11111011011110 it would be converted
to 11111011 in 8 bit mode for storage. In 16 bit mode the same number would be stored as 1111101101111000.

 

[alanrockwood]

Sorry I didn't read all of your last post. Not enough coffee today.

I am guessing your problem is flare from dust or the lens or sensor is dirty. These days I use a Nikon 4000 and I have to clean it every now and then. The difference can be startling after it is cleaned. I've never had any issues with clean scanners. Keep in mind the scanner is probably at east 15 years old.

If you are concerned about sensor noise, use multisampling in software if you have it which will deliver a clean file. Vuescan does it if the Canon software doesn't.

I scanned this on a Canon 4000 many years ago, and processed the image in Photoshop 4. I tried to improve it once a couple years ago with a scan on my Nikon and modern software, but couldn't. The Canon is an excellent scanner.

Hope that helps you.


View attachment 207328

I had not considered cleaning the scanner.

As you mentioned, I understand that the Canon is considered an excellent scanner, in the same league as the Nikon image quality-wise, though I have read that it may have just a tiny bit more noise in the shadows of a transparency scan.

Also, the photo you posted is a beautiful one.

 

[Ted Baker]

Using Vuescan I can set the acquisition to 8 bit or 16 bit. No doubt the hardware of the FS4000us scanner uses a 14bit ADC and converts the result to 8 bit before storing the data. For example, if the 14 bit binary number from the ADC were 11111011011110 it would be converted
to 11111011 in 8 bit mode for storage. In 16 bit mode the same number would be stored as 1111101101111000.

Except that isn't what vuescan does... If you save it as 8Bit it will gamma encode it BEFORE the 16bit to 8bit conversion.

So in your example 11111011011110 which is 64376 becomes (64376/(2^16))^.45454 which converted to 8bit is 253 or 11111101 instead of 251 i.e. a little brighter.

I hopefully I did not make any mistakes in the calcs but hopefully the point is clear.

 

[alanrockwood]

Except that isn't what vuescan does... If you save it as 8Bit it will gamma encode it BEFORE the 16bit to 8bit conversion.

So in your example 11111011011110 which is 64376 becomes (64376/(2^16))^.45454 which converted to 8bit is 253 or 11111101 instead of 251 i.e. a little brighter.

I hopefully I did not make any mistakes in the calcs but hopefully the point is clear.

I had Vuescan set to record in "Image" mode, which I am pretty sure does not include gamma encoding, though I could be wrong. Wouldn't gamma encoding only apply if scanning in one of the "...negative" modes?

If you are right about gamma encoding I think there is still a slight error in your calculation . The highest number that a 16 bit word can encode is (2^16-1), not 2^16, so I think the denominator in your expression should probably be (2^16-1), although I don't think it would make any difference in the final answer for your calculation once the result is rounded off to 8 bits.

 

[Ted Baker]

Wouldn't gamma encoding only apply if scanning in one of the "...negative" modes?

No gamma encoding always applies, with 1 exception 16bit raw. Even if you select 8bit raw it will gamma encode it BEFORE it divides all the data by 2^8. As the author knows that you do not want to scan using 8bit directly even if you think you do...

Yes I am sure you are right about the calcs, I slightly simplified them.

 

[alanrockwood]

No gamma encoding always applies, with 1 exception 16bit raw. Even if you select 8bit raw it will gamma encode it BEFORE it divides all the data by 2^8. As the author knows that you do not want to scan using 8bit directly even if you think you do...

Yes I am sure you are right about the calcs, I slightly simplified them.

Is there any Vuescan documentation that can verify gamma encoding comments?

 

[alanrockwood]

No gamma encoding always applies, with 1 exception 16bit raw. Even if you select 8bit raw it will gamma encode it BEFORE it divides all the data by 2^8. As the author knows that you do not want to scan using 8bit directly even if you think you do...

Yes I am sure you are right about the calcs, I slightly simplified them.

I found a Vuescan users guide at https://www.hamrick.com (specifically, https://www.hamrick.com/vuescan/vuescan.pdf). The copyright is 2017, so it seems to be a fairly new document. I am looking through the document to try to understand the nuances of how they use gamma.

My impression of Vuescan documentation in the past has been that it is rather terse and not always very completely explained, at least when I have tried to use the Help tabs in the program. I don't know if the current version will be any better, but I will see.

 

[Ted Baker]

am looking through the document to try to understand the nuances of how they use gamma.

Page 90 "The image gamma value is 1.0 when there are two bytes (16- bits) per sample, and 2.2 when there is one byte (8-bits) per sample. Raw files saved with gamma 1.0 will look dark, but this is normal"

Just about everything is gamma encoded, as that IS the standard. It is essential for 8 Bit processing, and is part of the sRGB standard. Which underscores the assertion that 8 bit using a linear A/D is not adequate.

 

[alanrockwood]

Page 90 "The image gamma value is 1.0 when there are two bytes (16- bits) per sample, and 2.2 when there is one byte (8-bits) per sample. Raw files saved with gamma 1.0 will look dark, but this is normal"

Just about everything is gamma encoded, as that IS the standard. It is essential for 8 Bit processing, and is part of the sRGB standard. Which underscores the assertion that 8 bit using a linear A/D is not adequate.

Thanks. This brings up a question. When I look at histograms from 8 bit scans and 16 bit scans (e.g. in photoline or GIMP) they look identical. Given this fact, are TIF files tagged with gamma information so the data can be re-linearized when brought into an image processing program?

 

[Ted Baker]

Thanks. This brings up a question. When I look at histograms from 8 bit scans and 16 bit scans (e.g. in photoline or GIMP) they look identical. Given this fact, are TIF files tagged with gamma information so the data can be re-linearized when brought into an image processing program?

8bit and 16bit TIF is assumed to be gamma encoded, unless an ICC profile is included and the ICC profile says otherwise. Most programs, and I am reasonably sure that includes GIMP will display the histogram data as if it was gamma encoded. i.e. 18% grey will be about half way on the histogram. Depending on the sophistication of the software the internal model should ideally be linear, ideally 32bit floating point, but at least 16bit i.e. will convert the gamma encoded data to a linear representation in memory. But it does depend on the software. I think the current best practise is 32 bit floating point. Which is what the editor I use has. Rawthereapee.

 

[alanrockwood]

8bit and 16bit TIF is assumed to be gamma encoded, unless an ICC profile is included and the ICC profile says otherwise. Most programs, and I am reasonably sure that includes GIMP will display the histogram data as if it was gamma encoded. i.e. 18% grey will be about half way on the histogram. Depending on the sophistication of the software the internal model should ideally be linear, ideally 32bit floating point, but at least 16bit i.e. will convert the gamma encoded data to a linear representation in memory. But it does depend on the software. I think the current best practise is 32 bit floating point. Which is what the editor I use has. Rawthereapee.

Thanks. One of these days I will check it out using my densitometer for comparison. Not for at least a month though due to other time commitments.

 

[alanrockwood]

Except that isn't what vuescan does... If you save it as 8Bit it will gamma encode it BEFORE the 16bit to 8bit conversion.

So in your example 11111011011110 which is 64376 becomes (64376/(2^16))^.45454 which converted to 8bit is 253 or 11111101 instead of 251 i.e. a little brighter.

I hopefully I did not make any mistakes in the calcs but hopefully the point is clear.

Looking at your formula a little closer (and assuming the denominator in the ratio should be 2^16-1 instead of 2^16) it looks like the largest number it can represent is 1. Should your formula look more like 255*(64376/(2^16-1))^0.45454?

Also, assuming my "corrected" form of the equation is right, it looks like the smallest non-zero 8 bit number it can represent (when rounded to an integer) is 2. (The non-rounded number would be 1.649, meaning that if the 16 bit number is 1 it will be encoding in an 8 bit number as 2, which is 1.649 rounded to the nearest integer) Is that right? Could it be that the formula is only an approximation, and the real conversion between 8 bit and 16 bit is a look up table, so that a 16 bit number of 1 gets encoded as an 8 bit number of 1, a 16 bit number of 2 gets encoded as an 8 bit number of 2 (i.3. 2.260 rounded to the nearest integer, 2), a 16 bit number of 3 gets encoded as an 8 bit number of 3 (i.e. 2.727 rounded to the nearest integer), and so forth? This could result in a few hicups, such as a 4 being encoded as 8 in the 8 bit representation (i.e. 3.096, rounded to the nearest integer) which is degenerate with the 16 bit number 3.

I getting the idea that there may be some non-obvious subtleties involved in the encoding of the 16 bit linear space into an 8 bit non-linear space.

 

[Ted Baker]

Looking at your formula a little closer (and assuming the denominator in the ratio should be 2^16-1 instead of 2^16) it looks like the largest number it can represent is 1. Should your formula look more like 255*(64376/(2^16-1))^0.45454?

That was just something I typed out quickly and is close enough and much closer than doing it completely wrong which is without the gamma encoding. The correct formula for sRGB

can be found here http://www.brucelindbloom.com/index.html?Eqn_RGB_to_XYZ.html (see inverse sRGB companding) or https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=12&ved=2ahUKEwjUiLPU4p7dAhVNsKQKHUiWA84QFjALegQIBhAC&url=http://www.color.org/srgb.pdf&usg=AOvVaw3C0jsd_fwAFw6J1nW9-f2B

Not all programs do it correctly in any case.

I getting the idea that there may be some non-obvious subtleties involved in the encoding of the 16 bit linear space into an 8 bit non-linear space.

Exactly! to make your assertions correct you need to be reasonably sure of the intermediate steps that occur. In the case of your scanner which has a 14bit A/D and in the context of this thread. Even in the case of converting 14bit to 16bit how is the data processed, is it for example just a 4X gain adjustment?

The most important thing to remember is that the gamma encoding must be done before any conversion that results in less bit depth, and consequently you will find that IS the way it is done. I can't think of one example where it is done incorrectly.

 

[Alan Edward Klein]

Here's what I did. (As I said, so far my testing has been non-rigorous). First I scanned a very dense (fully exposed to room light and developed) with half of the frame being air and half being the exposed film. Both the white and dark parts of the scan had a distribution of values, i.e. the dark parts were not a single value and the light parts were not a single value.

This is a pretty good indication that there is noise in the whites. It is less definitive about the dark region because a little bit of light might be getting through and part of the peak width (i.e. distribution of values) could be film grain.

Then I did a scan and looked ad the region in the frame around the image. That region is imaging thick black plastic, so no light should be getting through. That too had some width in the distribution of values. It is also not quite definitive because the sensor could be picking up some stray light from somewhere.

All of the above were with 8 bit scans. 16 bit scans (actually 14 bit) showed the same thing.

Now, one of the criticisms that some people have leveled at CCD sensors in this class of scanner is that they have sensor noise. This is contrasted with photomultiplier tubes in drum scanners which have very low sensor noise, probably too low to detect would be my guess. If this criticism is true then when scanning a pure black region there should be a distribution of values in the region. The same goes for a pure white region or any region in between.

Now, to bring this around to something practical, there has been considerable discussion about scanning in 16 vs. 8 bit mode. In other threads I pointed out that if film grain is equivalent to about 1 bit of RMS noise (or even a little less) then 8 bit mode captures all of the useful information in the scan. The same thing applies to sensor noise. If RMS sensor noise is of the order of 1 bit then there is virtually no significant advantage to using a higher bit depth. (However, before doing any post processing in photoshop or whatever softwere there is an advantage in converting an 8 bit scan to 16 bits.)

Let me comment on dynamic range. Adding more bits does not improve dynamic range if your sensor noise is of the same order of magnitude as the least significant bit in 8 bit mode.

This all comes back to the question of just how much sensor noise there is in a scanner. If the sensor noise (actually, sensor noise plus film grain combined) is much less than a 1 bit step of an 8 bit analog to digital converter then there is an advantage to scanning in 16 bit mode. If the sensor noise is roughly comparable to a 1 bit step of an 8 bit ADC (or worse) then there is no significant advantage to stepping up to 16 bit mode when scanning. Hence, there is a good reason for trying to determine how much sensor noise a given scanner has.

So, could you tell me what settings I should use when scanning Tmax 100 on an Epson V600 flat bed scanner?

 

[alanrockwood]

Setting aside the issue of gamma encoding for the moment, does anyone here know what sources of noise dominate in scanner sensors?

Diversion on the physics/electronics/mathematics of noise. In optical systems there are, in general, three main types of noise to consider.

The first is thermal noise (also called Johnson noise) which arises from thermal motion of electrons within the sensor. It follows a Gaussian distribution. It's value is independent of the signal level.

The second type of noise is shot noise. It arises from the randomness of the arrival of photons, and it follows a Poisson distribution, although at high numbers of detected photons its distribution is nearly indistinguishable from Gaussian noise. The magnitude of shot noise increases with the square root of the signal (assuming the signal is proportional to the number of photons), and the relative magnitude (i.e. the value of the noise compared to the signal level) varies one over the square root of the signal level.

The third type of noise is sometimes called pink noise or flicker noise. It tends to dominate if the time scale of the experiment is very long. I am going to ignore it in this discussion.

Shot noise and Johnson noise are generally uncorrelated, i.e. the fluctuations in one noise source are independent of the fluctuations in the other noise source. Under those conditions the variance (a statistical term) of shot noise and the variance of Johnson noise are additive. The standard deviation of the combined noise is the square root of the combined variances. This is a rigorous result from statistics that applies for nearly all combinations of noise sources. (For example, Poisson noise combines with Gaussian noise in this manner.) This has the implication that at very low signal levels Johnson noise dominates, and the noise is almost independent of signal level, but at high signal levels shot noise dominates and the noise is closely approximated as varying with the square root of the signal level.

So, in a CCD sensor what is the dominant noise source, Johnson noise or shot noise? One could break this down into smaller questions. Does Johnson noise typically dominate at low signal levels? How high does the signal need to get before shot noise dominates? My guess is that Johnson noise dominates at low signal levels in a typical scanner. My rationale is that astronomers often use CCD sensors, the same type of sensor found in scanners, and they often cool their sensors to reduce Johnson noise. There would be no need to do this if Poisson noise were dominant at low signal levels.

All this has the goal of better understanding what is limiting scanner noise performance, which can in some cases make it possible to do a better job of mitigating the noise, and more importantly it can help one understand how the dynamic range of the scanner plays into the noise performance of the scanner.

By the way, photo multiplier tubes, such as found in drum scanners, have a type of noise called "dark counts". It is extremely low in most cases and can usually be ignored unless the signal level is extremely low.

 

[alanrockwood]

So, could you tell me what settings I should use when scanning Tmax 100 on an Epson V600 flat bed scanner?

I don't think I have much good advice.

However, I have been recently scanning some very old glass negatives on my Epson V750 scanner. They are very grainy, so I am satisfied with scanning in 8 bit mode.

Tmax 100 is not very grainy, and I think the jury is still out as to whether 8 bit mode is good enough when scanning Tmax 100. Scanning in 16 bit mode might be safer.

By the way, if the resolution of a scanner is high enough then it is generally sufficient to acquire the signal using fewer bits. In the ultimate extreme of super high spatial resolution (not to be achieved under normal conditions) a 1 bit word length would be sufficient, though there are some other considerations that may over-rule this concept, such as diffraction effects limiting the ability to resolve silver particle images.

 

[Alan Edward Klein]

I don't think I have much good advice.

However, I have been recently scanning some very old glass negatives on my Epson V750 scanner. They are very grainy, so I am satisfied with scanning in 8 bit mode.

Tmax 100 is not very grainy, and I think the jury is still out as to whether 8 bit mode is good enough when scanning Tmax 100. Scanning in 16 bit mode might be safer.

By the way, if the resolution of a scanner is high enough then it is generally sufficient to acquire the signal using fewer bits. In the ultimate extreme of super high spatial resolution (not to be achieved under normal conditions) a 1 bit word length would be sufficient, though there are some other considerations that may over-rule this concept, such as diffraction effects limiting the ability to resolve silver particle images.

Thanks. I usually scan MF 6x7 Tmax BW with 16 bits and 2400 resolution. Gives me about 200mb files.

 

[alanrockwood]

Thanks. I usually scan MF 6x7 Tmax BW with 16 bits and 2400 resolution. Gives me about 200mb files.

Hi Alan. I did a little research on your scanner, and here is what I have learned. According to the website https://www.filmscanner.info/en/EpsonPerfectionV600Photo.html
your scanner is capable of an effective resolution of 1560ppi. However, to get this you will need to scan at a setting of 3200, so you are probably not getting quite all of the resolution possible if you are scanning at 2400, though it I suspect it's pretty close. Of course, if you scan at 3200 your file size is going to be almost twice as big, so you would need to decide whether the improvement is worth the increase in file size.

Also, have you tested your scanner to determine the optimum film height above the glass? Due to manufacturing tolerances this is supposed to be important if one wants to get the sharpest results possible with Epson scanners. Some people get lucky and their scanner is already at the optimum.

 

[Alan Edward Klein]

Hi Alan. I did a little research on your scanner, and here is what I have learned. According to the website https://www.filmscanner.info/en/EpsonPerfectionV600Photo.html
your scanner is capable of an effective resolution of 1560ppi. However, to get this you will need to scan at a setting of 3200, so you are probably not getting quite all of the resolution possible if you are scanning at 2400, though it I suspect it's pretty close. Of course, if you scan at 3200 your file size is going to be almost twice as big, so you would need to decide whether the improvement is worth the increase in file size.

Also, have you tested your scanner to determine the optimum film height above the glass? Due to manufacturing tolerances this is supposed to be important if one wants to get the sharpest results possible with Epson scanners. Some people get lucky and their scanner is already at the optimum.

Thanks for the information. I'll have to test it.

 

[Alan Edward Klein]

Why do you have to scan at twice the 1560?