Anti-aliasing filter?

[alanrockwood]

Most digital cameras have anti-aliasing filters. I suspect that scanners do not have anti-aliasing filters, but does anyone know for sure, and if some scanners do have anti-aliasing filters which scanners would have them?

 

[MattKing]

An anti=-aliasing filter wouldn't have any use for something that scans one line at a time, as all current scanners (of the type that most of us use) actually do.

 

[alanrockwood]

An anti=-aliasing filter wouldn't have any use for something that scans one line at a time, as all current scanners (of the type that most of us use) actually do.

In one way that's good news because it means that certain super-resolution techniques may be feasible. In particular, there is a way of taking multiple shots of an image, combine them, and then apply resolution restoration techniques to improve the resolution of the image. This is the basis of pixel shift methods that some digital cameras apply.

Pixel shift (in the context of what some digital cameras do) probably doesn't apply in this case because I think those pixel shift methods require a known and fixed shift between images. I think this is not actually feasible in a scanner. However, what could be feasible is to do some small random repositioning of the image between scans and then taking quite a few scans so that there are (without going into all of the details) good statistics among the ensemble of scans. It's important that no image processing be done on these raw images, such as no sharpening or smoothing, whether done internally in the scanner or externally.

The next step would be to increase the number of points in each scan. I don't know what the best interpolation method would be to fill in the 'tweener points, but in the absence of additional information I suspect that linear interpolation would be as good as any.

The next step would be to carefully align the images among the group so the registration between the images is as good as possible. There are software packages that can do this.

The next step would be to average the results among all of the scans to produce a single averaged image. This image should contain sufficient information to be able to improve the resolution.

The next step would be to apply an appropriate sharpening method. What you want is to use a method that restores the high frequency components of the scan while introducing minimal artifacts. Plain old unsharp maksing is probably not the method of choice because it often leads to artifacts at edges, which is probably not what you want. Methods based on Fourier transforms (or that can be related back to Fourier transforms) are probably best.

The scheme listed above is not new, at least not new in principle. There was even a commercial software package (no longer available) to do this. However, it was oriented toward camera images rather than scanned images.

If I were to hazard a wild guess it would be that as a practical matter this scheme might be able to increase resolution by a factor of about 1.5X, but this is just a guess.

One downside to this scheme is that it will increase noise, which could tend to increase visible grain. This might be mitigated to some extent if enough scans are averaged, which will tend to minimize noise through the mechanism of "signal averaging". The net result (if it works as I am guessing it would work) is that the grain in the final image might be true to what was in the negative. This could be an improvement over a conventional scan if it mitigates the problem of grain aliasing, which I think would be the case.

I think it's important that the scanner not have an anti-aliasing filter because an anti-aliasing filter would decrease or eliminate the higher spatial frequencies from the image, making them harder to restore. In the worst theoretical case the higher spatial frequencies would be eliminated altogether, in which case it would be impossible to restore them, and the resolution enhancement scheme would be doomed to failure.

Also important is the geometry of the actual sensors in the scanner. Do they approximate point sampling, or do they approximate boxcar sampling? If they approximate true boxcar sampling then the even-order harmonics (as related to the sampling geometry) would be completely eliminated, making them impossible to restore, and the resolution improvement would be only partially successful. If the sensor geometry approximates point sampling then both even order and odd order harmonics would be present and the resolution restoration could be more successful.

Another thing to keep in mind in this scheme is that the sharpening may need to be different in the two directions because resolution degradation in scanners is often different in the horizontal vs. vertical directions.

Yet another thing to consider is that this scheme requires the scanner to be critically focused. Otherwise defocusing will probably overwhelm whatever resolution restoration might otherwise be possible.

 

[brbo]

Pixel shift (in the context of what some digital cameras do) probably doesn't apply in this case because I think those pixel shift methods require a known and fixed shift between images. I think this is not actually feasible in a scanner.

Fuji Frontier scanners do pixel shift. Pixel shift existed in scanners for like two decades before digital cameras reinvented it.

Yet another thing to consider is that this scheme requires the scanner to be critically focused. Otherwise defocusing will probably overwhelm whatever resolution restoration might otherwise be possible.

This.

As for the rest, it's simple oversampling and is used in scanner regularly to overcome various deficiencies or limitations (using much higher mechanical resolution (stepper motor) than lens resolution in desktop flatbed scanners, oversampling with larger than "optimal" aperture in drum scanners...).

 

[Adrian Bacon]

Most digital cameras have anti-aliasing filters. I suspect that scanners do not have anti-aliasing filters, but does anyone know for sure, and if some scanners do have anti-aliasing filters which scanners would have them?

Most digital cameras have an AA filter because they use a Bayer color filter array and the vast majority of camera lenses resolve more detail than the sensor actually resolves, so in those instances, the AA filter is key. It's the exact opposite in scanner land. The scanner doesn't use a bayer CFA, and often the native resolution of the sensor far outpaces what the scanner lens can actually resolve onto it, so in those instances, no AA filter is needed as the lens assembly just doesn't put enough detail onto the sensor to make having an AA filter even necessary. This is why many flatbeds say they're 4800 or 6400 DPI but only really resolve ~2400-3200 DPI.

 

[Adrian Bacon]

The next step would be to increase the number of points in each scan. I don't know what the best interpolation method would be to fill in the 'tweener points, but in the absence of additional information I suspect that linear interpolation would be as good as any.

No. Linear interpolation works, but you get significantly better performance using something like a uniform catmull-rom or centripetal catmull-rom interpolation, combined with something like AHD (Adaptive Homogeny) where you look at the neighboring pixels and determine which interpolation direction would provide the best results, with best results being subjective, based on if you wanted the smoothest or sharpest results (or some combination thereof).

 

[Adrian Bacon]

The next step would be to apply an appropriate sharpening method. What you want is to use a method that restores the high frequency components of the scan while introducing minimal artifacts. Plain old unsharp maksing is probably not the method of choice because it often leads to artifacts at edges, which is probably not what you want. Methods based on Fourier transforms (or that can be related back to Fourier transforms) are probably best.

Sharpening is nothing more than increasing the contrast along the edges of the content of the photo. Many people do a terrible job of it. Best results can be had by making multiple black and white layers of the image (in photoshop or similar), then applying a high pass filter to each BW layer and optimizing the radius for the part of the image you want to sharpen in that layer. From there, set the blend mode to overlay and apply a layer mask to that layer so that you can selectively apply that layer's sharpening to just the part of the image that it is best suited for. Wash, rinse, repeat as many times as needed for other layers to until the image is sharpened to taste. Don't overdo it. It should look pretty sharp, but you shouldn't see any sharpening halos or other artifacts.

The method I described above is done that way because often times, if you just do one sharpen for the whole image, you end up with a middle ground sharpening as many images have many parts that would benefit from different amounts of sharpening, depending on the content of the image, and just doing an unsharp or smart sharpen on the whole image can be OK, but you can generally get superior results by selectively applying a high pass/overlay blend with a layer mask for each part of the image you want to optimize sharpening for.

 

[alanrockwood]

Sharpening is nothing more than increasing the contrast along the edges of the content of the photo. Many people do a terrible job of it. Best results can be had by making multiple black and white layers of the image (in photoshop or similar), then applying a high pass filter to each BW layer and optimizing the radius for the part of the image you want to sharpen in that layer. From there, set the blend mode to overlay and apply a layer mask to that layer so that you can selectively apply that layer's sharpening to just the part of the image that it is best suited for. Wash, rinse, repeat as many times as needed for other layers to until the image is sharpened to taste. Don't overdo it. It should look pretty sharp, but you shouldn't see any sharpening halos or other artifacts.

The method I described above is done that way because often times, if you just do one sharpen for the whole image, you end up with a middle ground sharpening as many images have many parts that would benefit from different amounts of sharpening, depending on the content of the image, and just doing an unsharp or smart sharpen on the whole image can be OK, but you can generally get superior results by selectively applying a high pass/overlay blend with a layer mask for each part of the image you want to optimize sharpening for.

Very interesting.

I like to think about sharpening in the frequency domain (spatial frequency that is) because sharpening relates directly to modifying the high frequency components of the image, although as humans we perceive the result in the spatial domain.

Regarding unsharp masking, if one wants to use that method then based on some one-dimensional numerical experiments I have done on the computer a series of small unsharp masking steps is better than one bigger step. This can produce a similar improvement in sharpness (in terms of what happens to the second moment of the point spread function) while giving less overshoot at edges.

Your comments about sharpening to different degrees in different parts of the picture is well taken. I have not tried that, but it makes a lot of sense. (I have actually thought about it, but wasn't sure how to do it.)

 

[Adrian Bacon]

Your comments about sharpening to different degrees in different parts of the picture is well taken. I have not tried that, but it makes a lot of sense. (I have actually thought about it, but wasn't sure how to do it.)

It can even be as simple as making multiple layers of the original image and doing a basic unsharp mask to varying degrees on each layer, and optimizing the unsharp mask for what you want to sharpen on that layer, then when you're happy with the sharpened parts of all the layers, just apply a layer mask to each layer, invert the masks to black, then with a brush paint white onto each layer mask where you want the sharpening to take effect for that layer. The layers and layer masking stuff is basic photoshop stuff. That's how you control what is visible in each layer in the final "composite" image (made up of all the layers stacked on top of each other).

I prefer to convert each layer to BW and apply the high pass filter and use the "overlay" layer blend mode as this basically applies the sharpening in the luminance part and not the color part without having to convert the image to a different color space (like LAB).

 

[alanrockwood]

A couple of posts have mentioned the lens in the scanner. (Thanks brbo and Adrian Bacon.) Both indicated limitations in the scanner lens and that this can be a limiting factor in scanner resolution.

I should add in passing that some 4000 dpi come close to resolving at the Nyquist limit, notably the Nikon scanners, the Canon fs4000us, and probably some of the Minolta scanners. In those cases the lens is probably not limiting, at least not seriously limiting. However, lets just assume that those scanners may be the exception to the rule. It's probably more common that the lenses in most scanners fall short of supporting the maximum resolution that could otherwise be achieved by the sensor array.

This would apply, for example, to virtually all flatbed scanners that are within the price range of most amateur photographers, as well as some film scanners. For example, the resolution of the top-end Epson scanners seems to top out somewhere in the 2300 to 2600 range in terms of effective dpi (reference: https://www.filmscanner.info/en/EpsonPerfectionV850Pro.html) even though the sensor array is capable of several times that. The Plustek OpticFilm 8100 tops out at 3800 dpi equivalent. That's actually very good for a 35mm film scanner, but it's about half of the sensor's capability.

Anyway, to get to the main point I want to discuss in this post, I wonder what the modulation transfer functions (mtf or mtfs) would look like for scanners that have sub-optimal lenses. Here' why answering this question might be relevant. If the mtfs drop to zero response early in the frequency domain then there is no hope of restoring the highest frequency components of the image, which means that increasing resolution would be a very limited option at best, and hopeless beyond a certain stage. (This is actually a general problem with all lenses. It's only a matter of degree of when it becomes hopeless.) If there are still some residual high frequency components, and if they haven't dropped too far into the noise, then it may be possible to do considerable resolution enhancement before further increases become hopeless. (I am deliberately using the term "resolution enhancement" here rather than "sharpening" because sharpening does not always increase actual resolution, and what I want to talk about here is increasing resolution.)

I mentioned that there was a super-resolution software package available that used a method similar to what I described in an earlier post. The package I had in mind was oriented toward pictorial photography. It was named PhotoAccute. Unfortunately, it is no longer available. However, there is some good info on their website. Their website claims that a new version is under development, but it has been years with no visible movement in that direction.

There are some other super-resolution software packages out there, but so far as I can tell they mostly serve specialized application areas, such as astronomy or microscopy.

Also, there are several different approaches to super-resolution. I have only discussed one, the one I think is most relevant to scanning of film.

I should close with a comment, obvious to some but perhaps needs to be repeated anyway. Resolution is not the be all and end all of image quality, and particularly it is not the be all and end all of scanner performance, but it is one component of image quality and scanner performance, and a fairly important component at that.

 

[Adrian Bacon]

Anyway, to get to the main point I want to discuss in this post, I wonder what the modulation transfer functions (mtf or mtfs) would look like for scanners that have sub-optimal lenses. Here' why answering this question might be relevant. If the mtfs drop to zero response early in the frequency domain then there is no hope of restoring the highest frequency components of the image, which means that increasing resolution would be a very limited option at best, and hopeless beyond a certain stage. (This is actually a general problem with all lenses. It's only a matter of degree of when it becomes hopeless.) If there are still some residual high frequency components, and if they haven't dropped too far into the noise, then it may be possible to do considerable resolution enhancement before further increases become hopeless. (I am deliberately using the term "resolution enhancement" here rather than "sharpening" because sharpening does not always increase actual resolution, and what I want to talk about here is increasing resolution.)

So, a couple of things... I have no doubt higher end flatbed scanners do actually resolve more than what the film scanner review site you cited states, the problem has a couple of intricacies to it. The first being, they use a USAF resolution chart, which is fine, however, it measures the effective MTF by visual inspection by a human eyeball, not with some algorithm that determines a MTF contrast percentage as the cutoff. Again, this is normally fine, except in this particular case, human eyeballs tend to see only high contrast edges and not lower contrast edges (this is why sharpening works so effectively, it boosts the contrast of the lower contrast parts of the image along the detectable edges and brings them up to something that we can see, thus giving the appearance of more resolution, the resolution was there all along, we puny human eye balls just couldn't make it out because the contrast was too low for us to see it). If memory serves, 40-50% contrast response is when we start to have less perception of the contrast, and thus less perception of the available fine detail. It's there, we just don't register it when we're looking at it. Sharpening never increases actual resolution, all it does is boost the contrast of the resolution that is there up to a range that we can more easily see.

So, for most flatbeds, there needs to be a way to measure the MTF programmatically so that we know where the 50% mark is and where the 0% mark is. That will tell us how much resolution we can enhance to bring up to something that we can see. The 50% mark will likely be right around the currently stated performance that many review sites state (simply due to how they're measuring it). I have a v850 and in all honesty, I think it performs better than stated, especially if capturing at the native sensor resolution and then carefully applying sharpening to that image before scaling it down. It's incredible how much the image sharpens up, so I suspect there's a lot of actual resolution (pictorial or otherwise) that the scanner actually is capturing, but it's just low contrast due to the MTF of the lens involved.