lens testing

[BetterSense]

What kind of tests can be performed on lenses?

I imagine that lens makers must have had production line tests used for quality control. They would need a way to detect poor performing lenses. They would want a way to do it without having to shoot a picture and waste time developing it. And they didn't have cheap electronics.

Are there diffraction methods, resolution tests, distortion tests, contrast tests etc that can be performed, possibly with some specialized equipment, and what do those tests look like? I am specifically interested in production QC. People here remark about sample variations in lenses, and if we can detect variations, then one would hope lens manufacturers would be able to detect them, but I honestly don't know what the lens maker's toolbox looked like in the mid 20th century. They didn't even have lasers until the 80s...

 

[Mr Bill]

Norman Goldberg's book, Camera Technology - The Dark Side of the Lens has a handful of test methods that he has used. A few tests, for flare and contrast, for example, include use of a microscope to funnel parts of a test target to a photomultiplier tube whereby the final results can be displayed on an oscilloscope.

He also describes the "time honored" star test, examining the image of an "artificial star" to check for centering and various aberrations.

The tests are too complicated to describe here, but the book may be available through your library.

 

[bernard_L]

Check out these guys. They do not run a production line, but have test equipment far above the brick wall or the USAF chart. And they don't do one-off tests; they can actually do (limited) statistics
http://www.lensrentals.com/blog

 

[Cycler]

I'm reminded of the day I got in bother at work. Nurse said, "Please read off the smallest letters you can read on this chart?" I read off, "3M eye test chart!" at the bottom. Seriously, at one time various photo mags used to have giveaway 'lens test charts' in. Close perusal of the small print often revealed that it had been devised by the promo dept of Nikon or Pentax to do down the competition. Those two were the big promo spenders, until Olympus caught up and overtook.

 

[gone]

These days, I'm more likely to say "what letters"? :[

However, there are things that can mitigate aging eyes, and as soon as I remember them, I'll post them.

 

[RalphLambrecht]

What kind of tests can be performed on lenses?

I imagine that lens makers must have had production line tests used for quality control. They would need a way to detect poor performing lenses. They would want a way to do it without having to shoot a picture and waste time developing it. And they didn't have cheap electronics.

Are there diffraction methods, resolution tests, distortion tests, contrast tests etc that can be performed, possibly with some specialized equipment, and what do those tests look like? I am specifically interested in production QC. People here remark about sample variations in lenses, and if we can detect variations, then one would hope lens manufacturers would be able to detect them, but I honestly don't know what the lens maker's toolbox looked like in the mid 20th century. They didn't even have lasers until the 80s...

If you properly QC the individual components of a lens.Then the assembled lens should perform to specification;therefore,theoretically, no end -of-line-test required.

 

[gorbas]

In Panavison (movie camera equipment manufacturing and rental) lens departments they have entire room with nice set up of very powerful projector on rails with etched glass lens target slide instead of filming test chart you project picture formed by the lens on the wall. Available lens mounts are Panavison, Arri PL and Nikon. Very efficient way of testing lens performance at different distances and apertures. Unfortunately for us photographers, their slide size is 35mm movie frame not 24x36mm.

 

[Dan Fromm]

If you properly QC the individual components of a lens.Then the assembled lens should perform to specification;therefore,theoretically, no end -of-line-test required.

Eh? Wot? Think about tolerancing, Ralphie.

 

[RobC]

I watched a video a while back showing the lens making process, I think it may have been a Leica vid. Today the QC kit they have they have tells them whether the completed lens is within required manufacturing tolerance in few seconds. I'm sure manufacturing tolerances are tighter than they used to be but there is still a "tolerance" and that means some will be fractionally better than others and there is always the possibility that a "lemon" gets through.

But lenses can be dropped or knocked putting things out of whack ever so slightly. There will always be slight variations but if they are within tollerance then no problem.

 

[Dan Fromm]

Rob, Ralphie, every element accepted for use in making a lens is within some tolerance of the design specification. They're rarely exactly to specification. When a lens is assembled it can easily not perform to specifications. That's why testing elements isn't enough.

Rob, I may have traduced you, you and I may have made the same point. If so, apologies.

Real world example. Some years ago I bought 20 38/4.5 Biogons that were used in AGI F.135 aerial cameras. The F.135's body is a lightly machined die casting, the lenses in their shutters bolt to it. The 38/4.5 Biogon's design focal length is 38.5 mm. AGI measured each Biogon's focal length (marked on the lens) and cut a shim (marked with the lens' S/N and with its thickness, to .01 mm) before bolting them to the bodies. The shim guaranteed that its lens was focused to infinity. My 20 Biogons' focal lengths, as measured, ranged from 38.3 to 38.8 mm.

 

[RobC]

yes correct shimming of lenses between front elements and rear elements is rather important, especially non symmetrical types

 

[M Carter]

This is an interesting thread and seems like the OP is interested in discussing manufacturer's techniques.

Still, I'm surprised how many people I know don't test their glass to find optimal f-stop and check for things like fringing or falloff. It doesn't need to be lab-level stuff, just real-world. Put a couple consumer packages on a table and shoot with a post-it note or mini white board showing the f-stop and check fine detail; go shoot an electrical transformer tower-thing for far focus. It's much easier with digital of course, but testing always makes me feel a little badass. I should buy a lab coat and go all the way.

 

[Nodda Duma]

Eh? Wot? Think about tolerancing, Ralphie.

Ralphie is right. Quality control ensures the critical dimensions on parts (barrel and lenses) are within tolerances. There is no use to call out +/-0.001" center thickness tolerance on a lens and then accept it if the vendor can't hit it or you can't verify they hit it. That's where QC comes in, verifying incoming parts do what I need them to do.

Part of optical design is tolerance analysis to ensure the lens can be fabricated and assembled on a production line. Good requirements call out as-built performance. Wherever that requirement is, I want to loosen up tolerances as much as possible to keep production costs down, but not so loose the requirements aren't met. So part of my design effort is optimizing the design for looser tolerances. Engineering is all about "good enough", but that means different things for different level of performance. Also means I need to understand exactly how the tolerances interact (easy enough with experience).

Designers don't throw a design over the wall and let production folks figure it out. We work pretty closely during production transition. In fact all last week I was down on the floor showing the assembly techs how to align a new tight-toleranced lens on a lens centering station. Yes a designer teaching the techs. The tolerances are necessarily tight because it has to resolve 200+ lp/mm @ minimum 30% contrast.

The week before I was stepping them through an eyepiece assembly. I designed that eyepiece so it meets performance requirement by just retaining the elements without any active centering required. Takes 5 minutes to assemble and the view through the it of the micro display is beautiful. Last month it was another new objective lens with moderate tolerances but still aligned by retainers. All meet their performance requirements when measured.

If the assemblers can't put it together and meet the performance then an optic would never make it past the computer screen.

Oh and there are shock/vibe requirements. The optics still meet perf spec after being dropped from make-you-cringe heights onto make-you-cringe surfaces (I was happy to see). But of course those are more stringent than commercial optics.

 

[Dan Fromm]

yes correct shimming of lenses between front elements and rear elements is rather important, especially non symmetrical types

Rob, the shims went between the shutter and the body to adjust flange-to-film distance. AGI didn't take the lenses apart, they collimated the lenses to the bodies.

 

[Dan Fromm]

Jason, on the one hand of course you're right. On the other, in the late '60s Zeiss (BRD) was shipping high end lenses whose focal lengths were within ~ 1% of spec. Good enough for Zeiss, not good enough for the camera manufacturer, hence the need to collimate each lens to a camera.

I have a couple of TTH and one Elcan aerial camera lenses. They all screw into lens cones that allow considerable room for collimating the lenses to their cameras. The lenses were locked in place by set screws. Yes, glued set screws. Aircraft vibrate a lot. I'm not sure that Vinten held F.95 cameras to quite as tight tolerances as AGI did F.135s. Anyway, one of the 12"/4 TTH lenses' mount threads were very badly knackered. It had been used on several camera bodies, recollimated as needed, and the set screws chewed up the mount threads. Not enough to make slight adjustments impossible, more than enough to make unscrewing the lens from its cone very difficult.

 

[Nodda Duma]

Collimating the lens to the camera is another step in the process. I can understand that the manufacturer would collimate each one if Zeiss wasn't designing the lens specifically for that manufacturer. When a lens is designed specifically for an imager, then the collimation process is accommodated for in the design. It sounds like the camera manufacture had a way to address this (and perhaps what you describe was as planned?), so all is good.

To the point you're trying to make: very simply a designer shouldn't be expected to be held to unknown or unstated requirements. But when the requirements are known *and reasonable*, then the designer is expected to accommodate them.

 

[Dan Fromm]

Jason, I take your point but we're not quite together yet. It was that a lens whose elements are all acceptably close to spec need not pass QC/acceptance testing.

My aerial camera lenses all passed manufacturers' Q/C and customers' acceptance testing, so shame on me for bringing in a distraction.

 

[Nodda Duma]

I understand what you're saying Dan, but I disagree based on how things are done. Fundamentally, there isn't any method to determine if the components are in spec without measuring them. So the shop can't claim the elements you've ordered to assemble a lens meets specification unless they've determined (by some measurement or test) that it actually does meet spec. That measurement/test process is quality control. Same thing all the way down to Schott or Ohara or the aluminum foundry. There's no such thing as "close to spec" or "almost spec". It either meets spec or it's rejected.

And by "meet spec" that means within tolerance; there is no such thing as a perfect lens that perfectly matches the ideal prescription.

For lenses, most times a Certificate of Compliance is acceptable from the optical shop, which is how they promise they've built the lens to spec. Lot inspections can also be included in the purchase order. Additional QC includes incoming lot sample inspection (for example the tables in MIL-PRF-13830). After final assembly a functional test (however simple) is made to ensure assembly was performed properly. That's all quality control. The idea --even academically -- of manufacture of precision optics without QC is ludicrous. Ask PE if film can be made without quality control all the way back to the cattle farm and he'd probably say the same thing.

I wouldn't buy elements from a shop which couldn't back up what they say with measurements..That is, without them implementing some type of QC. That's just asking for disaster. The customer won't blame your subcontractors. They'll blame you.

 

[flavio81]

Great posts, Nodda Duma!!

Every time i dissasemble a lens for cleaning, i wonder if i am not decentering the lens when i assemble it back... But most of those lenses seem to have a very tight fit for the elements inside their locations!

 

[Dan Fromm]

I understand what you're saying Dan, but I disagree based on how things are done. Fundamentally, there isn't any method to determine if the components are in spec without measuring them. So the shop can't claim the elements you've ordered to assemble a lens meets specification unless they've determined (by some measurement or test) that it actually does meet spec. That measurement/test process is quality control. Same thing all the way down to Schott or Ohara or the aluminum foundry. There's no such thing as "close to spec" or "almost spec". It either meets spec or it's rejected.

And by "meet spec" that means within tolerance; there is no such thing as a perfect lens that perfectly matches the ideal prescription.

Jason, every responsible lens maker measures lens elements before using them. And yes, testing starts with glass as received. Before grinding it into lens elements, makers recompute the prescription for the glass that's actually going to be used; acceptable but not exactly on spec. One of my friends did this for the CEDIS-Boyer that used to be. All elements that are within tolerance will be used, ones that aren't will be rejected. That said, a lens as assembled can contain some elements that are close to the upper limit on some measurement (radii, thickness) and others that are close to the lower limit.

We're equivocating about "on spec." As I'm using "on spec" it means the prescription, with zero tolerance. The ideal. As you use it you mean a spec with non-zero tolerances. The realizable. That's where you live and work.

Cheers,

Dan

 

[RobC]

isn't the argument about meeting spec about meeting minimum performance target? i.e. What are you testing for, minimum MTF targets? If it is MTF targets or resolution then its quite possible lens would exceed that spec rather hit it 100%. Therefore lens quality would meet a "minimum peformance target" but some could be better than others. i.e. some could hit a higher spec than required. Is that possible and is what we are really talking about rather than "tolerance".

 

[Nodda Duma]

.

We're equivocating about "on spec." As I'm using "on spec" it means the prescription, with zero tolerance. The ideal. As you use it you mean a spec with non-zero tolerances. The realizable. That's where you live and work.

Cheers,

Dan

That's not just where I work. That's where every lens designer works. I never met a lens designer who was paid to dilly dally in Zemax or Code V coming up with optical surface combinations that couldn't be made. Even the professors at Rochester or U of A will stress from day 1 of the fundamental lens design class that the classic designs considered manufacturing.

Using "in spec" to reference only the ideal performance is misleading. You want to use the term "nominal" or "ideal". An "in spec" part is the combination of the basic parameters plus the tolerancing.

You may dismiss this as unimportant but I guarantee it is not. *Every* good lens designer understands this. Consideration of the ability to fabricate a lens influences the design as much as the performance itself does. Let me make it clear: Every lens you've ever reviewed the prescription for had manufacturing taken into account. It is short-sighted to dismiss the consideration of manufacturing. If the lens from nominal prescription can't be built, then you'll never see that lens. Sounds obvious, but you seem to be dismissing that fact.

The prescription, tolerances, and measuring the parts are inexorably intertwined: when talking about a real device you can't discuss one without acknowledging the other. Because during the design all three factors influenced each other from the very start.

isn't the argument about meeting spec about meeting minimum performance target? i.e. What are you testing for, minimum MTF targets? If it is MTF targets or resolution then its quite possible lens would exceed that spec rather hit it 100%. Therefore lens quality would meet a "minimum peformance target" but some could be better than others. i.e. some could hit a higher spec than required. Is that possible and is what we are really talking about rather than "tolerance".

Actually that is exactly what I am talking about. A minimum MTF requirement (specification) drives the design and tolerancing of the lens. The minimum MTF is somewhere lower than the nominal MTF shown by the prescription...tolerances for the parts are determined by the designer (me) to ensure that the as-built lenses meet the minimum MTF with acceptable production yield.

Fun fact: As much effort is expended to determine the tolerancing as it takes to come up with the nominal prescription.

 

[Nodda Duma]

By the way the difference between nominal MTF and minimum MTF is typically 20-40% at the higher spatial frequency. So if the nominal MTF is 85% at 90 lp/mm, I'd expect to see 45-65% for the as-built MTF and reasonable tolerances.

 

[RobC]

So there can be a variation in MTF performance from the minimum required up to some other undefined level and therefore not all lenses that pass QC will be the same except except that they all meet the minimum required. I think that is what Dan and I refer to as tolerance. I don't think either of us was suggesting that anything which is below spec (minimum required) would pass QC. It's just the terminology we used which differs.

 

[Nodda Duma]

Ok fair enough. It seemed that he was saying you don't need QC at all if the parts are in all tolerance, but in reality there's no guarantee the parts are in tolerance without QC all along the supply chain...from Schott or Ohara supplying the melts all the way to the production tech who puts the final screw in.

So let me share the terminology concerning performance metrics, so you can be on the same page as the optical folks (and--stated with the best of intentions--not sound uneducated ).

"Nominal Design" and "Nominal MTF". The optical prescription and it's ideal MTF. I prefer "Ideal" or "theoretical". You won't see this level of performance in a real lens ever.

"As-built" MTF. The range of performance expected from a produced optic. In the charts you see of ideal MTF for camera lenses online (i.e. Generated from the prescription instead of testing), the best performance is probably going to be from 5-30% lower than ideal at the highest presented spatial frequency (50 lp/mm typically right?), depending on how tightly tolerances were controlled in production. I would expect Leica performance to not deviate as much, for example, since they have very tight QC and tolerances. So other than paying for the label, you are also paying for all that extra quality control.

"Typical" MTF. Middle-of-the-road performance amongst produced lenses

"Minimum MTF" or "MTF requirement" or "specification MTF". Lowest MTF to be expected out of the production run. There may be some that get through which are lower, but the intent is that they all perform better than this. A lens that "meets spec" will have an MTF greater than or equal to the "minimum" or "requirement" or "specification" MTF.

The performance distribution is Gaussian of course. The design software provides predicted production yields and as-built expectations of performance (or whatever metric you choose) based upon Monte Carlo simulation of toleranced dimensions. So I put in a tolerance for each parameter, assembly tolerances based on how the lens will be assembled, and even how the actual as-built dimensions will be distributed within the allowable tolerances. Some machinist will polish into the edge of a tolerance range and then stop. Others will try to hit the middle-of-the-road, so I can model all of that and predict production yield. Then I tighten or loosen them as necessary until I can get the required MTF at a required production yield if I'm using that as the metric.

The Monte Carlo simulation technique was developed to predict what happens in a nuclear explosion during development of the H-bomb, interestingly enough. It vastly simplified the calculation effort, and ran on the ENIAC computer.