Techniques, Reviews and Commentary
14/07/12Most lens tests on the web seem to fall into two categories, detailed lab style tests that concentrate on numbers and comparisons based on flat field images of charts or short distance studio set-ups and the casual or the“lets shoot a few random pics and call it a test” approach.
For me neither are much use, the former because there is a world of difference between what works for near field imaging in a studio and real world photography needs, whilst the later approach is far too limited to tell you pretty much anything useful and in most cases the views proposed are very subjective and without any point of reference to other lenses etc.
I must add I am not dismissing studio tests like DP reviews etc, any lens that performs excellently in lab test will almost certainly work well in real world situations from a purely measurement point of view but there are a great many lenses that don’t show up well in lab tests yet still produce amazing real world results. In short then I feel we need both lab test and real world tests.
My approach is I think maybe a little unique and no doubt some will wish to take me to task, but hey this is my website and I can do what I like, there are plenty of alternative options for to suit everyones needs. So what is unique about the Zero One Imaging tests.
I do not test jpeg files at all, in fact they are pretty much irrelevant to me. You see I shoot RAW on my DSLRs and Mirrorless cameras virtually 100% of the time, so I am really only interested in what the lens can produce with RAW files as the source. Why? Basically jpeg engines mess with lens resolution and certain types of lenses will produce images that fare badly as jpegs but might be brilliant as RAW files.
As an example the Minolta 28-85 3.5-4.5 is a lens that I love for the way it renders an image, but jpeg' kill its performance stone dead because it is very low contrast lens which exhibits high levels of soft micro detail, especially in the centre, none of which is left in the file by the time the jpeg engine in any Sony camera has had its wicked way.
On the other hand a high contrast optic may have poor micro detail but produce better OOF jpegs simply because it has a higher level of native contrast that does not get crushed in the jpeg engine.
Another factor is that good jpeg files are not ideal raw files, raw files need a bit more exposure if you are going to optimize the data potential they have to offer but high contrast optics can push the file over the edge in the highlights and make the final images hard to edit.
All Images Are Optimized First
Beyond only looking at RAW images, all images are optimized to the best possible final result, this means chromatic aberration is corrected, vignetting is reduced if needed, colour balances are neutralized so that the greys are actually true grey. Sharpening is also fully carried out in the raw convertor right up to the point where all potential clarity in the file is maximized.
Now you might wonder why would I do it this way...well think about it, I make a significant part my living from taking photographs and I’m never going to present a client with an unprocessed RAW file, much less a low grade JPEG. In the end I test lenses for my purposes not for those reading this website, which may sound selfish but that is how it is. I make my money from taking photos and training people in photography, the website is a service that does not make me money and I get very little time to do anything with it so I simply don’t have time to test lenses to answer other peoples needs. In other words I post tests to help those who may have specific interests when I think it may be relevant but I am definitely not in the lens testing business and won’t be changing my methods regardless of what folk reading this site might desire, after all there are plenty of likely more suitable test sources for those who just want the best OOC jpegs.
What counts for me is the potential of a lens to deliver an image once the normal workflows are applied.
If you as a reader of this site are interested in shooting JPEGs or producing work with little or no post production then clearly my lens tests and results are likely not going to be relevant to you. In fact a lens that I claim is brilliant may in fact be totally mediocre for your methods and needs, but likewise a lens that gives you the pop you want I may find completely unsatisfactory as it my be too high in contrast or lacking real micro detail etc.
I am not saying you’d be wrong, nor am I saying I am wrong, we just have different priorities. BUT .... If on the other hand you shoot in the same way I do, RAW, limited numbers of images with a careful approach followed up with targeted post production and an overall desire for image subtlety, then my approach may be just right for you.
I Choose Lenses for Their Rendering
Most lens test concentrate on the standard measurable criteria, ie sharpness, high contrast, lack of distortion etc, however lenses are often chosen for professional work on the basis of their defects or individual rendering characteristics.
On occasions colour rendition, spherical aberration, flare, lack of contrast, vignetting, field curvature and more may all be desirable features for certain types of photos, most of these things defy normal tests and can only be determined by photographing under sympathetic circumstances. In short I often look for these special characteristics but of course I realize most people will disregard many lenses purely because of them.
Price and Reputation is Irrelevant
When it comes to camera gear I have never been much impressed by marketing and brand loyalty carries little credence with me, ultimately I believe that pretty much all makers produce some great, average and some downright poor items, no one has the monopoly. All that matters to me is how well an items works and what it can do photographically. I will call a spade a spade dependent on the images and shooting experience alone, and gush with praise if it is justified, but I will never guild the lily to suit a fan base and I have no sponsors to please.
You will never for example find me buying any Leica digital camera because frankly I think they are all over priced anachronisms (except for the Panasonic clones which are just over priced) which are fundamentally flawed in many important ways that are important to me and no amount of fan boy ranting will change my mind....their lenses however are another matter.
When I see someone carrying a digital Leica I am not at all impressed, honestly I feel a bit concerned for them as I worry about the money they have wasted and the photographic opportunities they are missing in their blind obedience to owning something which really only scores on the basis of perceived snob value. Go ahead, email me with whatever defense/abuse you like, I know I have offended the sacred cow but that’s the way I see it. Honestly if you gave me an M9 I would sell it next week and keep the lens for a mirrorless camera providing it was not a wide angle, and yes I know all about the rangefinder experience, I have owned a few and yes I know Leicas are beautifully made jewels and all that but ultimately the camera needs to be an efficient picture taking tool not a giant jewel around my neck. Oh by the way when I see a Leica film camera in real world use my response is quite the opposite so I am not Leica negative.
Canon L series glass, ah well some is pretty good but a lot of it is overpriced, over heavy and over hyped. All nikon lenses are great, ah well no, some are pretty crook actually, (but I will confess they are probably the most consistent maker across the whole spectrum).
Canons better than Nikons? Which models, when, for what purpose. All makers are pretty dammed good these days and as I constantly tell my students, if there is a problem with your photos, it’s not your camera!
Funny thing, I have people ask me in classes “Brad what camera do you use, Nikon or Canon” as if there is no other alternative, surely you must use one of those esteemed options. I am met with incredulity when I say “well I have owned both systems, but I shoot neither these days” and when I say I am currently shootings Sonys’ they look at me as if I came from another planet. Like I say, all that matters is what it does and how it works for me, if someone builds a mousetrap I like better I will buy it, regardless of who makes it!
Ultimately then there is no point reading my reviews if you need affirmation that you have made the right purchase on any gear, quite likely you will end up offended, not that I set out to be offensive, but like I said I have no sponsors to please.
Clearing Up the Pre and Post Production
I would like to make a few points in explanation of the pre and post production methods that will clarify some things up for you.
Up front, don’t bother with the “classic DP Review forum dismissal” by saying that my testing technique is deficient, frankly I am bloody anal retentive about this stuff and anyone who knows me personally would accuse me of being totally obsessive about getting to the bottom of a lenses performance. If you see a posted review here you can be sure I have probably shot at least 400 images with that lens in a very targeted way. There is nothing casual at all about my approach so if you think I will get insulted by accusations of poor testing approach....your damm right!
Next I pay absolute attention to getting the focus spot on, I never use auto focus, all lenses are focused precisely in live view. For distance shots the focus is adjusted to the point where the horizon just reaches peak sharpness at maximum zoomed view. THIS is very important, time and time again I have seen lenses dissed for being unsharp on the web when clearly if you examine the files its obvious they were simply not properly focused. Don’t for one minute assume that your cameras auto focus system is any where near good enough for rigorous lens testing. I promise if you take the same shot ten times, switching your camera off between each shot you will likely get ten slightly differently focused shots.....one of which might be considerably better than the others.
I never assume that a lens is sharp just because it is focused in the middle of the frame, many lenses have significant field curvature and the last place you might be wanting to focus is right in the frame centre. Yes this a a pain to account for and requires some searching around in live view and testing, but once you are aware of it you will never ceased be amazed at how good some supposedly lousy lenses are simply because the focus was misplaced by the cameras auto focus system.
Once captured all files are extracted using RAW Developer, nothing extracts details from files like this application so I can absolutely be sure that if I am not seeing the details in the resulting TIFF file they are not there to see! Yes Yes I know Lightroom rocks and Aperture is cool and all that stuff, I have them on my system too....but only RAW developer is good enough for this task!
I am very interested in the contrast of the captured raw files, and no I am not looking for high contrast, in fact anything but. I desire lowish filmic contrast but without flare as those files will stand up far better to post editing...PERIOD! Thus many supposed great lenses are for my use simply painful, they push the shadow and highlight details over the edge and make for ultimate images that look, well “digital”. Now trust me on this the differences are very obvious when looking at histograms in the RAW convertor.
Of course I don’t want files that are so flat that they kill the resolution, just a happy medium will suffice.
The greys are always neutralized because you wouldn’t present an off colour image as a final result, but more importantly although different lenses render colour differently you can’t really compare apples to apples unless the greys are balanced in each file. But of course if you only shoot jpegs this is not relevant as you will have to go with what comes out of the camera, fair enough I understand that.
Chromatic Aberration is always fixed, and it is done manually, not using the pre-programmed options as they may not be perfectly accurate for the lens you have.
You cannot judge either edge sharpness or colour properly until this is done, many lenses that seem to be soft on the edges can suddenly come sharp when the CA is sorted and the colour can shift quite a bit out into the corners.
Vignetting is also fixed because once this is done it will make it clear if there are any underlying issues, like too little corner contrast or elevated noise as result of having to push things to far to get a corrected result. Often the correction can reveal greater levels of detail than was first seen.
Along the way common aspects such as bokeh, lens construction and precision, focus shift and a whole raft of other little issues will be dealt with but ultimately it is the lenses potential that I am interested in and most of my testing is aimed at working out where that potential is and deciding if it can be practically liberated on a job or in my personal work.
You might be surprised at just how great some really bad lenses are when used for tasks that match their characteristics and defects, equally you might be surprised at how little you have to spend to get stellar performance and how much of the stellar performance of some very expensive lenses is related to expensive marketing and assumed greatness and reputation rather than actual results!
To finish up I will let you in on a few secrets.
The sharpest lens I have ever tested cost $150.00 new, it was also the worst built lens I have ever used.
The worst overall performing telephoto zoom I have ever tested cost $2500.00 and no I don’t think it was a poor sample.
The best performing zoom I have ever used cost me just $120.00 second hand and the most expensive lens I have ever owned was a big disappointment!
One very expensive zoom lens from a maker starting with “Z” was despite all the rave reviews average at best and downright poor at the maximum focal length!
Anyhow thanks for reading and I hope I can offer some information that is helpful and as always I am glad to answer your question via email.