Silverfast 8: 16bit grey scale?

[alanrockwood]

I had another look at Adrians 410 files. One is labeled as a 8 bit scan. The other is labeled as a 16 bit scan. However, when I read them into photoline they are both identified as 16 bit images. At first this caused be to give up on looking at those. However, then it occurred to me that the one labeled as an 8 bit file might have actually been scanned in 8 bit and then converted to 16 bit. In other words, possibly Adrian decided to do the first step in the workflow I have been advocation, i.e. converting an 8 bit file to 1 bit before doing any other operations of the file, so I proceeded on that basis.

I read both files into photoline. Then I adjusted the contrast using the histogram tool. I did this in a series of steps, adjusting the histogram limits to try to keep the contrast in the two files as close to each other as possible. Eventually I was able to pretty well isolate the high step numbers with contrasts that were usable. I was not able to perfectly match the contrast in the two images, but I think I got them pretty close.

Here are the images. The first one is the one identified as 16 bit. The second is the one identified as 8 bit.

In my opinion, for the 16 bit image I think I could say that step #35 can be barely distinguished from the step on the left and the step on the right. Arguably one might bump it to step #36. For the 8 bit image I would say that step #36 can be just barely distinguished from the steps to the left and right.

This would indicate that the 8 bit scan just barely edges out the 16 bit scan as having higher dynamic range by maybe half a step. However, the two are so close that I would say it's a virtual tie. In any case, there is no indication that the 16 bit result has higher dynamic range. Even if one were to fudge the results and give the award to the 16 bit scan by one step, that would amount to only 0.1 density unit, which on a linear scale would mean that the 16 bit scan would have 26% more dynamic range. This is such a small difference that, even if true, it would be lost in the experimental uncertainty.

As a final comment, I'm sure that everyone noticed that the step images here do not look like good bar-shaped steps. I think this probably means that the Epson scanner that Adrian used is not very good at pulling out information from dense parts of the negative. That's to bad because I have a similar scanner (an Epson V/750), and I would be pleased if these scanners would do a little better.

 

[Adrian Bacon]

@alanrockwood I fixed the source file and reuploaded it, and updated my previous post.

Your exercise of seeing if you can distinguish the steps is not a valid test of dynamic range, especially, if the source is less dynamic range than you are acquiring. It is literally the equivalent of expanding the bottom tones of an 8 bit file and saying it has the same amount of dynamic range as a 16 bit file because you can differentiate the distinct tone values. In fact, lets do that now (I included these files in the updated zip file for reference):

Original file:

Expanded file:

Clearly, this 8 bit file has the same dynamic range as a 16 bit file because I can expand the lower tones and see the distinct tone values.

Come on now... We both know that is not a valid way to determine dynamic range.

I'll tell you what is a valid way to determine dynamic range though... how much noise is contained in the lower tone values of the image.... lets revisit your exercise and look at what is really going on shall we?

Starting with the 8 bit scan

I can clearly see the distinct tone values, this has the same dynamic range as the 16 bit scan. Well, does it? Let's look.

Looking at the two of them, the 8 bit scan is way noisier and chunkier than the 16 bit scan. In fact looking at the histograms of both, the 16 bit scan has dramatically more tone values than the 8 bit scan. Why? Because it has more dynamic range, and can therefore acquire dramatically more tone values with dramatically higher fidelity for the same source material, and acquire them with significantly less noise than acquiring at 8 bits. I can push and pull that 16 bit file around way harder than I'd ever dare to do with the 8 bit file, even with converting it to 16 bits first. It just does not have enough tone values to really step on it. In fact, I would venture to say that the chunkiness visible in the 8 bit scan that is not visible in the 16 bit scan is actually posterization and banding happening. The difference is even more absurd with the 410 series files. This is what I mean when I say it's painfully obvious the 16 bit file has more dynamic range.

That's a useful amount of compression, but not all that impressive compared to compression of the 8 bit file. The compression for the 16 bit file was about three times worse than the 8 bit file. Looked at another way, the sized of the compressed 8 bit file only 18% that of the compressed 16 bit file, and the 16 bit file does not contain more useful pictorial information than the 8 bit file (assuming one can even use the term "pictorial" in reference to a step wedge.)

That's because the 8 bit file has less unique data. Whether one considers the more unique data in the 16 bit file pictorially relevant is up to that person, however, based on the two screen shots above, I'll take the 16 bit scan as a starting point over the 8 bit scan without a second thought.

 

[Adrian Bacon]

I think this probably means that the Epson scanner that Adrian used is not very good at pulling out information from dense parts of the negative. That's to bad because I have a similar scanner (an Epson V/750), and I would be pleased if these scanners would do a little better.

Lets put it into perspective.... The V850 is easily capturing 3.5-3.6 log, which is 11-12 stops. For such an old scanner, that's actually very good, and considerably more than most negative material. Also, the 16 bit scan is way cleaner than the 8 bit scan in the lower tone values. The examples you posted don't show it, but it is. Per my previous post, I'll take the 16 bit scan any day of the week. It's not even a contest.

 

[Alan Edward Klein]

Would the format of the film affect DR and banding results? ie. 35mm vs. 4x5?

 

[Alan Edward Klein]

Another question I've asked before but never got an answer to is the following. Do you set the white and black points (levels) for the scan? Or, do you scan flat (0-255 range) and do the levels in post-processing program?

Others have said you should do it for the scan as you will get more data from the scan. But I think the levels if set in the scanning software only edits flat scan results in any case after the scanner scans the full range. So there's only an appearance you're getting more data. At least that's my theory. Does anyone know the actual answer? How and when does the scanner do its leveling if set for the scan?

 

[Adrian Bacon]

Would the format of the film affect DR and banding results? ie. 35mm vs. 4x5?

no. The larger format just makes the grain less apparent.

 

[Adrian Bacon]

Another question I've asked before but never got an answer to is the following. Do you set the white and black points (levels) for the scan? Or, do you scan flat (0-255 range) and do the levels in post-processing program?

Others have said you should do it for the scan as you will get more data from the scan. But I think the levels if set in the scanning software only edits flat scan results in any case after the scanner scans the full range. So there's only an appearance you're getting more data. At least that's my theory. Does anyone know the actual answer? How and when does the scanner do its leveling if set for the scan?

if you scan at 16 bits, you will never get more than 65536 unique tone values. If you scan flat, just get get up to that amount that you can then just push and pull around. If you set the white and black points in the scan, however many unique tone values are between the two points are redistributed between 0 and 65535. If scanning and saving at 16 bits, this doesn’t matter, but if saving at 8 bits, it’d be better to set the white and black points in the scan so that the remaining 16 bit tone values are redistributed between 0-65535 before being gamma encoded and scaled down the 256 unique tone values, so if scanning at 16 bits but outputting 8 bits, it’s better to set the white and black points in the scan, but if scanning and saving at 16 bits, it doesn’t matter, do what you want.

if scanning at 8 bits, you will never have more than 256 unique tone values, and setting the white and black points in the scan isn’t going to do anything but potentially chop that down even further. Why anybody would actually scan at 8 bits is beyond me. It’s way the heck noisier in the lower tone values and an all around inferior scan to begin with. Just don’t do it unless you have no choice.

 

[Alan Edward Klein]

if you scan at 16 bits, you will never get more than 65536 unique tone values. If you scan flat, just get get up to that amount that you can then just push and pull around. If you set the white and black points in the scan, however many unique tone values are between the two points are redistributed between 0 and 65535. If scanning and saving at 16 bits, this doesn’t matter, but if saving at 8 bits, it’d be better to set the white and black points in the scan so that the remaining 16 bit tone values are redistributed between 0-65535 before being gamma encoded and scaled down the 256 unique tone values, so if scanning at 16 bits but outputting 8 bits, it’s better to set the white and black points in the scan, but if scanning and saving at 16 bits, it doesn’t matter, do what you want.

if scanning at 8 bits, you will never have more than 256 unique tone values, and setting the white and black points in the scan isn’t going to do anything but potentially chop that down even further. Why anybody would actually scan at 8 bits is beyond me. It’s way the heck noisier in the lower tone values and an all around inferior scan to begin with. Just don’t do it unless you have no choice.

It's not clear to me. Let';s say the image has data from 25-215. If I sent the scan at 0-255, will I wind up with less data from 25-215 because the scanner is gett data where there is none between 0-25 and from 215-255? Or, if we set the black and white point to 25 and 215, does the scanner actually scan 0-255 and then after it collects the data, it applies the black and white points at 25 and 215. If the latter, than it would seem to matter but if the former, than you;d be losing data?

 

[Adrian Bacon]

It's not clear to me. Let';s say the image has data from 25-215. If I sent the scan at 0-255, will I wind up with less data from 25-215 because the scanner is gett data where there is none between 0-25 and from 215-255? Or, if we set the black and white point to 25 and 215, does the scanner actually scan 0-255 and then after it collects the data, it applies the black and white points at 25 and 215. If the latter, than it would seem to matter but if the former, than you;d be losing data?

The image tooling will only show you 0-255 regardless of how many bits the source is, that 0-255 is internally scaled to whatever the bit depth of the source is, so:

If scanning at 16 bits, and you set the scan from 25-215, the actual range of the underlying data is: 6425-55255 for a total of 48830 discrete tone values that are then stretched back out and redistributed between 0-65535. When that is scaled back down to the 0-255 of 8 bits, all 256 discrete tone values will contain values, because you're scaling 48830 discrete tone values down to 255 discrete tone values, thus maximizing the usage of the 8 bits you're saving to.

If scanning at 8 bits. and you set the scan from 25-215, the actual range of the underlying data is: 25-215 for a total of 190 discrete tone values, that are stretched back out and redistributed between 0-255. Since it's already 8 bits, there is no scaling down to 8 bits, so you end up with an 8 bit file that contains 190 discrete values, instead of an 8 bit file that contains 256 discrete tone values.

Does that make sense?

 

[Alan Edward Klein]

The image tooling will only show you 0-255 regardless of how many bits the source is, that 0-255 is internally scaled to whatever the bit depth of the source is, so:

If scanning at 16 bits, and you set the scan from 25-215, the actual range of the underlying data is: 6425-55255 for a total of 48830 discrete tone values that are then stretched back out and redistributed between 0-65535. When that is scaled back down to the 0-255 of 8 bits, all 256 discrete tone values will contain values, because you're scaling 48830 discrete tone values down to 255 discrete tone values, thus maximizing the usage of the 8 bits you're saving to.

If scanning at 8 bits. and you set the scan from 25-215, the actual range of the underlying data is: 25-215 for a total of 190 discrete tone values, that are stretched back out and redistributed between 0-255. Since it's already 8 bits, there is no scaling down to 8 bits, so you end up with an 8 bit file that contains 190 discrete values, instead of an 8 bit file that contains 256 discrete tone values.

Does that make sense?

I scan in 16 bits and then post-process in Lightroom in 16 bits. Let's forget about 8 bits and only talk in 16 bits.
What happens then if I scan 25-215 vs scanning at 0-255 flat. Which provides more data in the final scanner of the original image?

 

[Adrian Bacon]

I scan in 16 bits and then post-process in Lightroom in 16 bits. Let's forget about 8 bits and only talk in 16 bits.
What happens then if I scan 25-215 vs scanning at 0-255 flat. Which provides more data in the final scanner of the original image?

On a purely technical level, 0-255 flat would, though, on a practical level, you'd be hard pressed to see any difference because you'd presumably then be setting the black and white points in LR instead of in the scanning app, and either way, you have way more discrete tone values than you can actually display, so it really just comes down to preference. If you want more of a finished image coming out of the scanner app, then do it there, if you'd prefer to get to a finished image in LR or PS, then just do a flat scan.

 

[Alan Edward Klein]

On a purely technical level, 0-255 flat would, though, on a practical level, you'd be hard pressed to see any difference because you'd presumably then be setting the black and white points in LR instead of in the scanning app, and either way, you have way more discrete tone values than you can actually display, so it really just comes down to preference. If you want more of a finished image coming out of the scanner app, then do it there, if you'd prefer to get to a finished image in LR or PS, then just do a flat scan.

So if I understand your point, the 25 and 215 points if selected before the scan means that the scanner applies those limits after the scan of 0-255? The scanner always nominally scans flat and applies levels afterward?

 

[brbo]

So if I understand your point, the 25 and 215 points if selected before the scan means that the scanner applies those limits after the scan of 0-255? The scanner always nominally scans flat and applies levels afterward?

Yes*.

* There are exceptions, but it's very unlikely that one would have such a scanner and not be aware of that.

 

[Adrian Bacon]

So if I understand your point, the 25 and 215 points if selected before the scan means that the scanner applies those limits after the scan of 0-255? The scanner always nominally scans flat and applies levels afterward?

what @brbo said.

 

[Alan Edward Klein]

Yes*.

* There are exceptions, but it's very unlikely that one would have such a scanner and not be aware of that.

Well, many people have said the opposite. So because of that, I've been setting the levels for the scan. So now, based on this new info, I will scan everything flat - 0-255 with no other settings used during the scan and do all editing in Lightroom. That will save a lot of time scanning.

 

[Adrian Bacon]

Well, many people have said the opposite. So because of that, I've been setting the levels for the scan. So now, based on this new info, I will scan everything flat - 0-255 with no other settings used during the scan and do all editing in Lightroom. That will save a lot of time scanning.

many people either don’t really understand what is really happening under the covers, or, they are just projecting what they do and think everybody should do it their way. If scanning and saving at 16 bits, it’s purely a personal preference for what workflow you want.

 

[Alan Edward Klein]

many people either don’t really understand what is really happening under the covers, or, they are just projecting what they do and think everybody should do it their way. If scanning and saving at 16 bits, it’s purely a personal preference for what workflow you want.

Well, the beauty of scanning flat, is you shut all edits off and you don't have to set anything for each picture. You press "Go" and go get a cup of coffee while it;s doing it;s job scanning m multiple pictures. So you do all the editing in post. You never have to scan a second time because your editing was wrong if done with the scanner's software. You don;t have to learn a second editing program in the scanner software. You only have to learn how to scan flat which is simply done with the scanner's software and that's free.

 

[alanrockwood]

Continuing the discussion of 8 bit vs. 16 bit mode, the issue of dynamic range has come up, so let's explore that a little deeper.

The most important thing to understand is that having a ADC converter with more bits does not necessarily mean that the dynamic range of a given system is greater than if you have an ADC converter with fewer bits. The reason is that there are at least two things that can limit the dynamic range. The first is how many bits are in the ADC. However, if the signal source is a noisy one, then at some point noise is the limiting factor, not the number of bits in the ADC. This is well-known in signal processing theory.

Since the Epson scanner has come up in the discussion let's explore the question of whether noise is the limiting factor. Here's the experiment. Using my Epson V/750 scanner. I made a scan with half of the image that transmits 100% (i.e. air, with no film) and the other half of the image where the light is perfectly blocked. I did this by laying a thick piece of black paper on the scanners glass. I placed it on the glass instead of the focal plane of the scanner to make sure that there is no texture captured in the image. Above the paper is a standard 35mm multi-frame film holder. Above that I placed a book. There will be no light penetrating through the book and the black paper. All scans were done in 6400 dpi mode.

I selected a crop area and scanned it twice, once saving the scan in 8 bit mode, and the other saving the scan in 16 bit mode. As you can see in the attached photo I included part of the film holder frame, so the light area is actually less than 50% of the image. (Please note. This is actually a image I snipped from a screen shot. It is not the original file, which was too large to upload. In fact, all images shown in this post are screen shots of displayed results, not the original scanned files. However, all work was done on the original files themselves.)

From the 8 bit scan I cropped a small piece from the lower left hand part of the image, containing roughly 9000 pixels. In this region of the image light is not just blocked by the paper and the book, but also the frame of the film holder. Note: this is a snipped image from a screen shot, as are all of the images I am showing.

Then I did a histogram correction. I selected the 2.5% from the left hand side of the histogram correction (the dark end), so one can see the histogram structure in the dark region cropped from the image. Here's what it looks like There are just a few peaks. The spacings of these peaks correspond to one step of an 8 bit word. In addition to the graph I am showing the resulting image of the histogram correction. It's noisy.

This is already a strong hint that 8 bits is more than enough dynamic range to capture the noisy signal. In a scan of film the scanner noise and the film grain will combine to make a scan even noisier, but just the scanner noise alone, without even considering film grain, is sufficient to imply that 8 bits is plenty to capture the dynamic range possible in a scan. Otherwise we would not see multiple peaks in this histogram.

We can make this a little more quantitative by treating the histogram plot as if it were a probability distribution, i.e. normalizing the set of peaks to a sum of 1. The standard deviation of the peak cluster turned out be be 0.55, where 1 is the step size of the digitization. (Note: in the figure the distance between the peaks is 40 arbitrary units of the horizontal axis. The standard deviation in these arbitrary units was 22, but I then renormalized the spacing of the horizontal axis to unit increments, so the normalized standard deviation is 0.55 of the digitization step size.)

Although 0.55 is an important number, it is not to be taken directly as determining the dynamic range limit. A more reasonable number is three sigma. This is the criterion used by analytical chemists to specify a detection limit, which is the value that can just barely be said to be reliably distinguished from a blank. Using this criterion one can say that the 0.55times 3 or .65 digitization steps (out of an 8 bit word size) is where we can just barely say that the image is not quite dark. This means that the dynamic range of the Epson scanner is 155 to 1.

What about if I do the same analysis to scan saved in 16 bit mode? Here's the image of the histogram for that case. This is after we have selected the darks side of the histogram.

Note there are more peaks present because 16 bits can hold more discrete numbers than 8 bits. Of course they are also closer together for the same reason. However, the width of the distribution is comparable to that of the 8 bit scan. (Note: the horizontal axis is of the same relative length the 8 bit and 16 bit histograms.) In fact, if I go through the entire calculation, treating the peak cluster as a probability distribution, calculating the standard deviation, and determining the dynamic range, it works out to a dynamic range of 113 to 1, which is a bit worse than the 8 bit scan. However, I consider this to be within experimental error, meaning that the dynamic range for the 8 bit and 16 bit scans are basically the same.

Just for fun I also took the 16 bit scan and converted it to 8 bits. (These manipulations were all done in photoline, as were all manipulations done in this post.) Here's the result.

Note this result is qualitatively very similar to the 8 bit scan. When I go through the calculation I get a dynamic range of 121 to one. Why did it get a little better after I converted the 16 bit scan to 8 bits? I don't have a full answer to that. However, I suspect that it may be due to the data being in gamma corrected format. Mathematically speaking, this is a non-linear process, and sometimes non-intuitive things can happen when non-linear operations are done on data, and this can be especially true when there is the possibility of roundoff error.

The conclusion from this is that from the point of view of dynamic range there is no advantage to saving scans in 8 bits vs. 16 bits on the Epson scanner because the dynamic range is limited by scanner noise, not by the bit depth of the word used to store the result. If film is scanned this is even more the case because film grain will provide an additional level of effective dithering.

Next comes something very important. The dynamic range discussed above is the single-pixel dynamic range. The effective dynamic range of a full scene can be (and virtually always is) higher than the single pixel dynamic range. This is because when we look at a picture we normally don't see the individual pixels. In fact, our eyes cannot resolve single pixels of most images at ordinary viewing distances. We can think of our visual acuity as sampling an effective patch size in the image. For a given point in the scene our eyes effectively average the result over the pixels in the patch. The dynamic range improves as the square root of this patch area, which means that it improves linearly with the linear patch dimension. For example, if a patch is two times larger in linear dimension it is four times larger in area, and the dynamic range averaged over a patch of this size is twice as high as the single pixel dynamic range. If this were not true then, for example, half-tone printing would not work.

There is also the fact that the film has a gamma associated with it, so when converting the film's gamma to a more linear scale and inverting and adjusting the image, the scene-referred dynamic range is expanded still further.

And as a final comment, although I am not a signal processing engineer, I have spent a good fraction of my profession career as a PhD chemist dealing with the problem of extracting useful information from data having experimental uncertainty, including noise, and along the way I have studied and learned quite a bit about digitization of signals as well, both from an engineering perspective (for example, knowing the difference between a successive approximation ADC vs. a flash ADC, or what's the difference between a pulse counting system and an ADC based system data acquisition system), as well as a theoretical perspective, and an analytical perspective. So if you want to question my credentials to discuss this topic please feel free to do so. I can't stop you, nor do I believe I should stop you. However, I am personally confident of know as much as or more about this topic than 90% or maybe even 99.9% of the people who are likely to be reading this. And I don't know if anyone has noticed, this, but for my part I don't recall having questioned the credentials of any specific persons posting on this topic. I generally believe in a principle of more or less free speech when it comes to scientific or technical matters, especially in the context of informal public forums, and I prefer not to question the credentials of others. It would be nice to give me the same courtesy. I don't want this thread to turn into a battle of credentials.

 

[alanrockwood]

Let me expand a bit on this issue of converting a noisy signal to digital form by illustrating it with an example. If you have appropriate software it's easy to do this on a computer. The idea is to use a random number generator. This falls in the category of a Monte Carlo simulations.

Here's how to do it. Generate a list of numbers containing a ramp running from 0 to 1. It should be a very fine-grained ramp. For example, subdividing the ramp into ten thousand evenly spaced points is a nice number. A million is even better. (I have done both of these, multiple times.)

Use a random number generator to make a list of random numbers having a Gaussian distribution and a standard deviation of 1. I will call this list N.

Add this list to the ramp to make a noisy ramp.

Next, digitize the noisy ramp with an 8 bit word size. You now have a digitized version of the noisy ramp.

Subtract the perfect ramp from the digitized version of the noisy ramp. What you have now is just the noisy part of the digitized signal. I will call this list ND

Calculate the standard deviations of lists N and ND. You will find that the standard deviation of ND is about 1.04, which is about 4% greater than the standard deviation of list N. These standard deviations are so close to each other than for most practical purposes one would not likely consider one to be noisier than the other.

In addition, it can be shown that if there is random noise whose standard deviation is equal to one step size of an ADC it is sufficient to basically suppress the stair-step nature of a digitized signal. This is called dithering, and it is a very well-known principle in digital signal processing theory. In fact, in most respects adding noise whose standard deviation is one half of the ADC step sizer is enough dithering to suppress stair-stepped signals. In image-talk, if there is enough noise (and it doesn't take much) then banding can be eliminated.

As to whether one wants to talk about banding vs. dynamic range, those two concepts are actually two sides of the same coin.

 

[Adrian Bacon]

The most important thing to understand is that having a ADC converter with more bits does not necessarily mean that the dynamic range of a given system is greater than if you have an ADC converter with fewer bits. The reason is that there are at least two things that can limit the dynamic range. The first is how many bits are in the ADC. However, if the signal source is a noisy one, then at some point noise is the limiting factor, not the number of bits in the ADC. This is well-known in signal processing theory.

I think it’s important that we agree on the definition of dynamic range because I suspect that we are not discussing the same thing, but using the same terms to describe what we are talking about. The most common definition for modern ADCs is that of the difference between the maximum recordable signal and the noise floor of the ADC at a given bit depth and signal gain level. It’s also pretty common to pick a maximum acceptable noise level in place of the noise floor.

for example: the canon EOS RP and the EOS R6 both have a 14 bit ADC. However, the R6 has superior dynamic range because it has a significantly lower noise floor than the RP. Per CIPA standards, both sensors top out at ISO 64 as the maximum recordable signal before the sensor clips, but of you capture a black frame on each camera and just look at the raw data coming off the ADC, the RP is dramatically noisier, and the dynamic range measurements bear out that it does indeed have less dynamic range. The cause of that noise comes from a lot of different places, but the point is, the dynamic range is the difference between the maximum recordable signal and the agreed upon noise floor.

an even more important fact is that even though both cameras have different dynamic range, if you take progressively lower and lower light level pictures, there’s a lot of tones that register way below the agreed upon noise floor. You could make the argument that because you can see those tones, no matter how noisy they are, they count as dynamic range. I suspect that this is where you and I are diverging quite a lot.

from where I sit, in 8 bit land, the maximum recordable signal is 255, and the noise floor will generally sit between 0 and 1 (or higher), making the maximum possible dynamic range dynamic range 256:1 with the real world dynamic range more like 128:1 or even a little less. 256:1 is 8 stops. 128:1 is 7 stops, etc. in 16 bit land the maximum recordable signal is 65535, and again, the minimum possible is 0-1 for a maximum possible DR of 65536:1, or 32768:1 real world. 16 bit ADCs never have a noise floor that low, and indeed, it’s really hard to get the electronics feeding into it to be that noise free either, but it is possible to have a solid 11-12 stops (or more) of DR before noise levels are objectionable. The point is: 65536:1 is more dynamic range than 256:1,

so how are we getting to the point where the 8 bit scan appears to have just as much dynamic range as the 16 bit scan? Part of it is we’re not taking into account noise levels between the two, we’re just counting how many steps we can see and asserting that 8 bit is just as much DR as 16 bits, but the fact of the matter is, that’s not how DR is determined. The other part of it is if the resolution at which the noise levels are evaluated at affects the noise floor, and if doing a comparison, it’s important to normalize things so that everything is the same resolution. It’s for this reason that a very high MP camera can claim superior noise levels to lower MP cameras even though the per-pixel noise levels of the higher MP camera are atrocious. Decimating or scaling the captured image down will always lower the noise floor and make for higher dynamic range, in fact, this happening is why you’re claiming 8 bit is all you need.

 

[alanrockwood]

Sometimes it's easier to talk about things in base 10 (ordinary numbers) rather than base 2 (binary numbers), so how about if we shift the discussion to base 10, so that's what I will do in the discussion below.

If there is much noise in your system (total noise, not just the digitizer itself, but the whole system, including the sensor, amplifiers, misc. components, the signal itself, and any other noise sources) then your dynamic range is noise limited, not ADC step size limited. If there is very little noise then dynamic range may be limited by the ADC step size, but only if the ADC step size is much smaller than the noise.

For example, suppose your digitizer has a maximum range of 1 volt. Further suppose that you have noise in the system whose standard deviation is 0.02 volts. Your dynamic range will be noise limited to 50. A digitizer with a step size of 0.01 volt will do an adequate job of capturing the signal and the noise. Going to a digitizer with a step size of .001 or even 0.0001 volts is not going to gain you any useful dynamic range because you are noise limited.

If you have no noise in the system then going to a deeper bit depth will gain you dynamic range, but not if you are noise limited.

Now suppose you have a different system with noise that has a standard deviation of 0.002 volt. Now an ADC with a step size of 0.01 will be inadequate for capturing all of the available dynamic range in the system. But a digitizer with a step size of 0.001 volt is plenty good enough for all practical purposes, and going to a digitizer with a step size of 0.0001 volt buys you only a truly negligible improvement in dynamic range.

Actually, if the standard deviation of the noise is the same value as the ADC step size then for nearly all practical purposes that is also fine because you get a net degradation of dynamic range is only about 4% compared to a digitizer with infinite bit depth, and that is such a small difference that it almost never matters.

If the standard deviation of the noise is even as low as half of the ADC step size then things still aren't too bad. You have about 16% more noise than you would have if you use a digitizer with a higher bit depth. That's not a lot worse, but it might have a small but noticeable impact in some cases.

Once you go much less than half of the ADC step size then things can go bad pretty fast, and in that case you approaching the situation where the dynamic range is limited by the ADC step size rather than the noise.

I illustrate this with a couple of figures below simulating the following cases: 1) The signal is 0.01, the standard deviation of the noise is 0.01, and the ADC step size = 0.01 and 2) The same signal and noise but with the ADC step size = 0.01. Going to a ten times smaller ADC step size bought very little useful improvement, and going to an even smaller ADC step size would buy us virtually no useful improvement.

As for the Epson scans, I scaled both the 8 and 16 bit results so the axis in the figures are directly comparable, and the calculations were also scaled so they are directly comparable. I should point out one subtlety in how I analyzed my Epson scan. I used a 3 sigma criterion, which is typically what an analytical chemist would use for distinguishing signal from noise near the zero-signal threshold. That is for detection limit. For quantitation limit it is customary to use ten sigma. If you want to use a different sigma multiplier then the calculated dynamic range would different, not that it is truly different but rather it relates to what threshold for noise is considered acceptable.

 

[Adrian Bacon]

Note this result is qualitatively very similar to the 8 bit scan. When I go through the calculation I get a dynamic range of 121 to one. Why did it get a little better after I converted the 16 bit scan to 8 bits? I don't have a full answer to that. However, I suspect that it may be due to the data being in gamma corrected format. Mathematically speaking, this is a non-linear process, and sometimes non-intuitive things can happen when non-linear operations are done on data, and this can be especially true when there is the possibility of roundoff error.

The formula for gamma (assuming a pretty common gamma of 2.2) is value divided by bit depth, raised to the inverse of the gamma, then multiplied by the bit depth.

For 8 bits:

(255/255) = 1.0. POWER(1.0, 1/2.2) = 1.0. 1.0 * 255 = 255 <- the maximum recordable signal
(1/255) = 0.003921568627. POWER(0.003921568627, 1/2.2) = 0.08055974814. 0.08055974814 * 255 = 21 <- the minimum signal above zero.

If scanning and saving at 8 bits, if you're not going to save the raw ADC values, you'll only end up with values between 255 and 21, a whole 234 discrete tone values.

For 16 bits:

(65535/65535) = 1.0. POWER(1.0, 1/2.2) = 1.0. 1.0 * 65535 = 65535 <- the maximum recordable signal
(1/65535) = 0.0000152590219. POWER(0.0000152590219, 1/2.2) = 0.006466867352. 0.006466867352 * 65535 = 424 <- the minimum signal above zero.

If scanning and saving at 16 bits, it doesn't matter if you save the raw ADC values or not, with a resultant 65111 discrete values, it's basically more than you can display.

If scanning at 16 bits and saving at 8 bits:

(65535/65535) = 1.0. POWER(1.0, 1/2.2) = 1.0. 1.0 * 255 = 255 <- the maximum recordable signal
(1/65535) = 0.0000152590219. POWER(0.0000152590219, 1/2.2) = 0.006466867352. 0.006466867352 * 255 = 2 <- the minimum signal above zero.

If scanning at 16 bits and saving at 8 bits, it's better overall as you end up with 253 discrete tone values.

@alanrockwood That is why you're seeing that difference. Scanning at a higher bit depth than what you're saving with lets you encode more dynamic range into a lower bit depth with gamma encoding and gives you lower noise results. By your own efforts you can see that difference. The gamma encoding effectively chops your step size down if scanning and saving at 8 bits and gives you lower dynamic range.

The Epson uses the same analog circuits for both bit depths, which is why the 8 bit performance is so good (I also strongly suspect that the 8 bit mode is really still internally sampling at 16 bits and scaling down to 8 bits in hardware before outputting to the driver, but have no way of knowing one way or the other except that to note that it's pretty common to see that as it gives even better 8 bit performance), however, going to 16 bits on the acquisition will give you better results.

 

[Adrian Bacon]

Sometimes it's easier to talk about things in base 10 (ordinary numbers) rather than base 2 (binary numbers), so how about if we shift the discussion to base 10, so that's what I will do in the discussion below.

If there is much noise in your system (total noise, not just the digitizer itself, but the whole system, including the sensor, amplifiers, misc. components, the signal itself, and any other noise sources) then your dynamic range is noise limited, not ADC step size limited. If there is very little noise then dynamic range may be limited by the ADC step size, but only if the ADC step size is much smaller than the noise.

For example, suppose your digitizer has a maximum range of 1 volt. Further suppose that you have noise in the system whose standard deviation is 0.02 volts. Your dynamic range will be noise limited to 50. A digitizer with a step size of 0.01 volt will do an adequate job of capturing the signal and the noise. Going to a digitizer with a step size of .001 or even 0.0001 volts is not going to gain you any useful dynamic range because you are noise limited.

If you have no noise in the system then going to a deeper bit depth will gain you dynamic range, but not if you are noise limited.

Now suppose you have a different system with noise that has a standard deviation of 0.002 volt. Now an ADC with a step size of 0.01 will be inadequate for capturing all of the available dynamic range in the system. But a digitizer with a step size of 0.001 volt is plenty good enough for all practical purposes, and going to a digitizer with a step size of 0.0001 volt buys you only a truly negligible improvement in dynamic range.

Actually, if the standard deviation of the noise is the same value as the ADC step size then for nearly all practical purposes that is also fine because you get a net degradation of dynamic range is only about 4% compared to a digitizer with infinite bit depth, and that is such a small difference that it almost never matters.

If the standard deviation of the noise is even as low as half of the ADC step size then things still aren't too bad. You have about 16% more noise than you would have if you use a digitizer with a higher bit depth. That's not a lot worse, but it might have a small but noticeable impact in some cases.

Once you go much less than half of the ADC step size then things can go bad pretty fast, and in that case you approaching the situation where the dynamic range is limited by the ADC step size rather than the noise.

I illustrate this with a couple of figures below simulating the following cases: 1) The signal is 0.01, the standard deviation of the noise is 0.01, and the ADC step size = 0.01 and 2) The same signal and noise but with the ADC step size = 0.01. Going to a ten times smaller ADC step size bought very little useful improvement, and going to an even smaller ADC step size would buy us virtually no useful improvement.

View attachment 300765


View attachment 300766


As for the Epson scans, I scaled both the 8 and 16 bit results so the axis in the figures are directly comparable, and the calculations were also scaled so they are directly comparable. I should point out one subtlety in how I analyzed my Epson scan. I used a 3 sigma criterion, which is typically what an analytical chemist would use for distinguishing signal from noise near the zero-signal threshold. That is for detection limit. For quantitation limit it is customary to use ten sigma. If you want to use a different sigma multiplier then the calculated dynamic range would different, not that it is truly different but rather it relates to what threshold for noise is considered acceptable.

I don't disagree with any of this except the agreed upon noise limit and possibly the resultant size of the image. I'd prefer to look at per pixel noise as it removes the variable of what resolution you scanned it at and what resolution you scaled the image down to, both of which will affect the resultant dynamic range, either that or agree upon a standard scan resolution and standard output resolution.

 

[Adrian Bacon]

It's generally not a great idea to do dynamic range or noise analysis on image data that has been conformed to a color space as you have to contend with the gamma encoding and the raw to colorspace transform and what effect that has on the resulting image samples.

So...

http://m.avcdn.com/sfl/epson_v850_raw_black_frames.zip

This contains two files: raw0001.tif and raw0002.tif. Both are a ~36x24mm black frame at 6400 dpi from the center of a completely blocked off capture area (except the scanner calibration area). Vuescan was used to capture and output the raw data with all the controls that vuescan can control (analog gain, etc) zeroed out and locked so they are exactly the same between scans with the only difference being raw0001.tif was captured in 16 bit mode and raw0002.tif was captured in 8 bit mode immediately after raw0001.tif.

Anybody can download these raw files and do whatever analysis they want on them.

I've done a very basic and preliminary look using some tooling that I use for my own internal software. The 8 bit samples were scaled up to 16 bit samples to make the comparison a little easier (using sample/255, then result * 65535, all in float to preserve precision).

The per color channel numbers:

raw0001.tif (captured in 16 bits, saved in 16 bits)
resolution: 9064x6046 px
bits per sample: 16 bits
analyze window_size: +-1024, x1: 3508, x2: 5556, y1: 1999, y2: 4047
max_r:1817, max_g:1911, max_b:1714
min_r:0, min_g:0, min_b:0
avg_r:334, avg_g:350, avg_b:313

raw0002.tif (captured in 8 bits, saved to 8 bits, scaled to 16 bits for analysis)
resolution: 9064x6046 px
bits per sample: 16 bits
analyze window_size: +-1024, x1: 3508, x2: 5556, y1: 1999, y2: 4047
max_r:59904, max_g:53248, max_b:32639
min_r:0, min_g:0, min_b:0
avg_r:1699, avg_g:1666, avg_b:1644

Just for edification, the same values, but scaled to 8 bits

raw0001.tif
resolution: 9064x6046 px
bits per sample: 16 bits (scaled to 8 bits)
analyze window_size: +-1024, x1: 3508, x2: 5556, y1: 1999, y2: 4047
max_r:7, max_g:7, max_b:6
min_r:0, min_g:0, min_b:0
avg_r:1, avg_g:1, avg_b:1

raw0002.tif
resolution: 9064x6046 px
bits per sample: 16 bits (scaled to 8 bits)
analyze window_size: +-1024, x1: 3508, x2: 5556, y1: 1999, y2: 4047
max_r:233, max_g:207, max_b:127
min_r:0, min_g:0, min_b:0
avg_r:6, avg_g:6, avg_b:6

Make of it what you will, though, again, I think it's pretty obvious that 16 bit capture has a significantly lower noise floor and therefore more dynamic range than 8 bit capture. Whether that makes it through the raw to colorspace operations and gamma encoding, is harder to tell as the analysis @alanrockwood has done so far, though sound from an analysis perspective, is being done on data that's already been conformed to a color space. Also whether that lower noise floor matters, depends on what you're capturing, so 8 bit capture very well may be adequate for lower contrast materials.

Either way, the raw files are available for anybody to do their own analysis and draw their own conclusions.

 

[faberryman]

Sounds like the issue is choosing between not enough and too much, or, best case scenario, between adequate and overkill. Given all the other things we obsess about, it seems like processing with a few extra bits won't do any harm, and not having to worry about that will allow us to better focus on more important things like resolution and dynamic range. For instance, if you are scanning your 35mm and 120 film with a flatbed scanner instead of a dedicated film scanner, the horse is already out of the barn. If you are not doing your own scanning, you might want to ask your film processor if he is scanning at 8-bit or 16-bit, and what scanner he is using at what settings. You may be able to do a better job yourself. Of course, it may take you a couple hours to do a roll. Lots of trade-offs to consider. I won't even delve into the issue of how much image data you are throwing away when you resize your scans for posting on the web. A parting question: Why are you even scanning your film to begin with?