I think Roger Cicala is as close as we get ...
My pleasure. I think they dig deeper than anyone.Well, I'm impressed. Thanks for pointing out the web site.
Tests are only as good as the methodology, and often interpreting the results requires detail of the methods. How to deal with curvature of field for instance. A lens may be extremely sharp at the edges but not at the same plane of focus as the center, for instance the 2nd version Leica Summilux. You can present accurate information to make that lens look like a dog or a hero.
Many amateur tests end up being a test of focus accuracy more than lens performance. Unless you design your tests to cover misfocus and camera/lens/film focus calibration you are only getting useful information about how that particular system functions for that particular photographer in that particular situation.
I agree. The MF test the OP linked to shows the limitations of this kind of experimenting. The tester apparently couldn't focus the Makina or had a very poor or damaged copy, but since he only tested one unit no conclusion can be made other than the question mark he delivered in the notes section. Ditto the Yashicamat, which is known to have a quality lens especially when stopped down a little, but look at the results: the one he tested showed a significant problem, performing adequately wide open and actually getting worse when stopped down which indicates a problem with the lens, camera, testing methodology or all of the above. Another question mark was offered in the notes since there were apparently no other Yashicamats to test.
Newcomers to reading this sort of lens testing might be forgiven for thinking Makinas and Yashicamats were supplied with crap lenses, but that is most certainly not the case.
Once you test your lens, then what?Over the past many decades there have been many lens test charts. Probably the one most often used is the 1951 USAF lens test chart. Some information on downloading the chart is available at https://jimdoty.org/Tips/Equipment/USAF_Test/usaf_test.html and elsewhere.
Unfortunately, virtually all published lens tests are done on sample size one (or, at very best, n<10) and are therefore completely meaningless.
Assuming that there is SPC in place at manufacturing, there could be a bit more validity than you’re giving credit for. QC done right generally results in a fairly stable product. Statistically speaking, though… you’re quite correct… as you already know.Unfortunately, virtually all published lens tests are done on sample size one (or, at very best, n<10) and are therefore completely meaningless.
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?
We use cookies and similar technologies for the following purposes:
Do you accept cookies and these technologies?