RAW headroom: why it matters

Comments Off

Posted on 23rd January 2009 by Lars Rehm in Camera reviews |Reviews |Software |Syndicated Press

In the Dynamic Range section of our DSLR reviews we usually look at something that we call ‘RAW headroom’. The RAW headroom could probably be described as the highlight and shadow detail that has been captured in the RAW data but gets lost when a camera’s internal software applies a tone-curve to the RAW image data.

As a rule cameras apply a typical 'S' shaped tone curve to JPEGs to give a visually appealing contrast without harsh clipping of the brightest or darkest tones (the 'shoulders' of the S curve provide a more gentle roll-off than a straight line would). The tone curve applied is often quite steep, sacrificing the tones at the the extremes of highlight and shadow in the pursuit of 'punchy', consumer-friendly out-of-camera results. At the shadow end this is rarely a problem (the very darkest tones are mapped to black, which isn't visually unpleasant and helps reduce noise), but at the highlight end it can mean that what should be a very pale blue sky turns white, or that highlights have harsh edges instead of a gentle gradient. There's no way to get back detail that the camera's tone curve left out of a JPEG, but it is possible to tweak more out of a raw file. Having access to the sensor's full dynamic range also means that you can fix mild under or over exposure by reaching into the headroom.

And so it's very useful to understand what raw headroom is and how you can make use of it to get the maximum out of your images. We regularly receive questions regarding this particular section of our reviews via our feedback system and in the forums, so I thought it would be a good idea to provide some detail about the dpreview approach to raw headroom in a blog article. This isn't a deeply technical post; it's more of a primer on what we do, and why you need to know about the dynamic range hidden away in your raw files.

JPEG vs ‘ACR Best’

For our Dynamic Range test we take an image of a calibrated Stouffer Step Wedge (13 stops total range) which is backlit using a daylight balanced lamp (98 CRI). The exposure is carefully fixed so that wedge no 18 produces the same middle gray. We then feed this image into our in-house software which gives us a Dynamic Range value in stops (EV).

To find out how many extra stops the reviewed camera model is hiding in its raw headroom we measure the dynamic range of a out-of-camera JPEG at default settings and compare the result to the value we get for our ‘ACR Best’ image. The 'ACR Best' image is the test image that gives us the maximum dynamic range result on a particular camera. To create it we tweak the RAW file of our test image (the one of the wedge; whenever possible  we shoot RAW+JPEG) in Adobe Camera RAW until we get an output image that measures maximum Dynamic Range. To achieve this we usually reduce exposure between half a stop and one stop, reduce contrast to a minimum, change the tone curve to ‘linear’ and play a little with the Blacks and Brightness sliders. The optimal settings will be slightly different for each camera though.

JPG_wedgeACR_Best_wedge
When we feed our dynamic range studio test images into our clever software we receive, along with the data, these wedges as an output. The one above illustrates the dynamic range of a Pentax K200D out-of-cam JPEG, the bottom one of the ‘ACR Best’ image. 

Out-of-camera JPG

To illustrate the difference that making use of the raw headroom can make to your images I have picked a sample image from the Pentax K200D review and processed the RAW file with three different sets of parameters in Adobe Camera RAW – ACR 4.6 default settings, our ‘ACR Best’ settings and a set of custom parameters.

Below you can see the original out-of-camera JPG which is very vibrant and saturated with good contrast. On the downside it also  shows some fairly large areas of blown highlights on the statue and some minor clipped shadows in the trees to the left (we’ve opened the JPEG in Adobe Camera RAW to make these areas more visible). On this specific image this is as much a metering issue as a dynamic range one. A third of a stop negative exposure compensation when the picture was taken would certainly not have done any harm but on the other hand this makes the image an ideal demonstration object for this blog article. In our K200D review from September this year the camera’s default JPEG gave us a total Dynamic Range of 9.0 stops (3.0 stops highlight range, 6.0 shadow range).

scene_outofcam_screenshot-001 scene_outofcam-001
Out-of-camera JPEG in ACR 4.6 Click thumbnail for full-size image
scene_outofcam-002 scene_outofcam-003
100% crop 100% crop

Adobe Camera RAW with default settings

For comparison purposes I am also showing the image that is produced out of a RAW file by Adobe Camera RAW using the software’s default settings. The colors are slightly less saturated but the contrast rendition is very similar to the out-of-camera JPEG, showing almost identical amounts of clipped highlights and shadows.

scene_default_screenshot-001 scene_default-001
ACR 4.6 default settings Click thumbnail for full-size image
scene_default-002 scene_default-003
100% crop 100% crop

RAW conversion using ‘ACR Best’ parameters

Let’s have a closer look at the Pentax K200D’s RAW headroom now. Above I described how we determine the RAW headroom by comparing dynamic range of an out-of-camera JPEG to our ‘ACR Best’ image. In the K200D’s case the parameters used to generate the ‘ACR Best’ image are the following:

  • Exposure: –0.85
  • Blacks: 1
  • Contrast: –50
  • Curve: Linear
  • All other parameters ACR default.

Using these parameters on our dynamic range test image we managed to generate an ‘ACR Best’ image that gives us one stop additional dynamic range over the out-of-cam JPEG (additional 0.8 in the highlights, 0.2 in the shadow range). This takes the total achievable dynamic range of the Pentax K200D to 10.0 stops.

scene_best_screenshot-001 scene_best-001
‘ACR Best’ parameters Click thumbnail for full-size image
scene_best-002 scene_best-003
100% crop 100% crop

What happens if we apply these ‘optimal’ parameters to our real-life sample image? As you can see above the label ‘ACR Best’ is slightly misleading in so far that while the image is optimized in terms of dynamic range – there is maximum detail in both shadows and highlights – it is also very ‘flat’ and dull due to a lack of contrast. It’s obvious that simply applying the parameters that guarantee the optimal dynamic range won't make an image that’s visually pleasing.

Custom parameters for optimized output

What we really want in an image is the best of both worlds, the highlight detail of the ‘ACR Best’ image but also good contrast in the mid-tones and shadows. If you’ve been shooting in RAW that’s not a problem. To achieve the image results you can see below we reduced exposure in RAW conversion by approximately half a stop. This first step brings most of the lost highlight detail back. Then we modified the tone curve to increase contrast in the mid- and darker tones. At the last step we ‘fine-tuned’ the result with a levels correction in Photoshop to make sure we use the entire range of available tones.

custom3-screenshot-001 custom3-001
ACR 4.6 Custom parameters Click thumbnail for full-size image
custom3-002 custom3-003
100% crop 100% crop

The table below shows all four images next to each other. You can see that the Custom image shows similar highlight detail to the ‘ACR Best’ image. However, the overall contrast of the picture is much more visually pleasing. Of course, much of this is down to personal taste. If you wanted the Custom image to be more similar to the out-of-camera JPEG for instance, you could simply increase saturation in RAW conversion. However, what these examples are trying to demonstrate is that by shooting in RAW you can preserve highlight detail  and with some minor image modifications create an image that has similar contrast rendition in the mid-tones and shadows to an out-of-camera image. 

scene_outofcam-002 scene_default-002
scene_outofcam-001 scene_default-001
Out-of-cam JPEG ACR 4.6 default settings
   
scene_best-002 custom3-002
scene_best-001 custom3-001
‘ACR Best’ parameters ACR 4.6 Custom parameters

Refining the group test format

Comments Off

Posted on 27th December 2008 by Simon Joinson in Camera reviews |Syndicated Press

It's long been obvious that the existing dpr review format - even in its seriously truncated 'concise' form - is unsuitable for the majority of what can be broadly termed 'point and shoot' compact cameras, and I've been wanting to develop a group test format for some time now. There are simply too many compact cameras released each year for us to keep up with (even the most basic camera takes a minimum of a week to put through our battery of tests and to produce a gallery), and the truth is that image quality amongst competing models doesn't - for the most part - vary that much. But there are always going to be exceptions, cameras with image quality that falls well below or rises way above the mean, and unless we test a far higher proportion of the models on the market there's a risk we'll miss these occasional stars and lemons. The group test format may not give you as much detail about the individual cameras, but it does give you a much better idea of the relative quality of a far greater number of competing models.
The recently published group tests were designed specifically to offer a snapshot of the market in the run up to the peak buying season, and to help the huge number of dpreview visitors who arrive looking for help choosing a camera - those that aren't regulars or forum activists. The format was chosen to reflect the way digital cameras are typically used, and to compare how the various models performed in everyday situations. Unusually for dpreview the image quality was assessed first and foremost in terms of typical viewing magnifications, rather than at a pixel level, with the emphasis on accuracy and reliability of the autofocus, metering and white balance systems and on issues that impact on the quality of a typical print (color, contrast, dynamic range and so on). This doesn't reflect a shift in the dpreview ethos or any intention to change the way our full reviews are conducted; it simply recognizes that the typical buyer of a budget digital camera - or one designed to slip into a shirt pocket - is less likely to be concerned with pixel peeping than the typical dpreview regular.
We've grown to expect anything and everything we publish to receive far more brickbats than bouquets on the forums - a thick skin is an essential attribute for anyone writing for dpreview - but our impression is that the idea of a group test has been fairly well received (and they've certainly had a lot of traffic), even if there are disagreements on the cameras chosen and the conclusions drawn. We're almost certainly going to be doing more of them in the future, and to be honest I can't see many simple point and shoot cameras getting full reviews given the number of DLSR and 'prosumer' compacts launched each year.
So now we're working on refining the group test format to offer a little more than those just published without getting to the point where the articles become unwieldy. I've actually delayed publication of the last group test (the super zooms) so we can add a couple more tests, as befits cameras that in many cases are almost as sophisticated as the typical entry-level DSLR.
I'm sure everyone has their own idea of what's important and of what should be included in the group tests, and we have been following the feedback on the current crop closely. For the most basic cameras I think the format used for the articles just published is probably sufficient, but for more advanced cameras we probably need to add at least some of the following to what we included this time around:
  • Raw comparison
  • Noise analysis
  • Basic performance measurements and timings
  • Optical aberrations (CA etc)
  • Distortion
  • Dynamic range
  • Macro performance
  • Resolution
  • Movie mode
This is what we're currently debating, and hopefully you'll see the group test format developing into the genuinely useful comparison tool we envisage. We also see the group test format as being something we could possibly add to our full SLR reviews, allowing us to, for example, gather together all the entry-level digital SLRs we've already reviewed to produce a more concise comparison article once or twice a year.
    Obviously high-end compacts such as the Canon G10 and Panasonic LX3 will still warrant full reviews, as will many of the superzoom cameras, and the more advanced premium models, but for the scores of other compacts launched each year the group test format will allow us to broaden our coverage to a significantly bigger proportion of the market without compromising our 'get under the skin' approach to reviewing.

    Ten years on

    Comments Off

    Posted on 25th December 2008 by Phil Askey in Camera reviews |Syndicated Press

    Ten years, nobody who starts a web site can honestly claim to have an accurate vision of what that site will be like ten years down the line.  When I wrote my first digital camera article and dropped it onto the now defunct photo.askey.net URL I had no intentions for it to be anything more than an opinion piece of my purchase. But then a funny thing happened, I began getting emails, lots of them, all asking about my article and what I thought of other cameras.  It seemed that this "digital camera thing" was becoming popular and what people were looking for was a site which was always up to date and could deliver honest, detailed reviews.  I went out on a limb and spoke to several manufactures, to my surprise the majority were extremely receptive and before I knew what was happening I had a steady flow of cameras coming my way.  Over the next couple of years I developed the primary site features (camera database, forums, news, etc.) as well as gradually improving the testing methodology (but at all times trying to be consistent).

    The hard work began to pay off, we saw our visitor figures skyrocket; 3 million page views per month after a year, 11 million after two years, today we serve around 140 million pages per month to over 7 million unique visitors (or about 24 million visitor sessions if you prefer).  In 2004 having done virtually a hundred hours a week for five years I decided getting some help may be a good idea and was extremely lucky to hire someone with a long standing digital camera review pedigree; Simon Joinson.  In 2006 I was approached by Amazon to discuss potential acquisition, for me, and for the site, this was actually perfect timing, we were massively resource limited, needing a new office, more staff, more development and new infrastructure.  In the eighteen months since the deal was signed the way we work has changed completely, we now have a large dedicated office and studio, totally new server infrastructure (hosted in the same facility as Amazon) as well as four camera reviews, a lens reviewer, a studio assistant / news author and two (soon to be three) developers.

    Naturally I'm very proud of how far we've come, but I am also aware (especially in tough times such as these) that the need to constantly innovate and change never goes away, proof of this can be seen in our recent group test series which is a totally new format to dpreview but has been delivered to perfection by Simon and the team.  You should also expect to see a lot of new development next year, not least of which our new Challenges system which we'll be launching into public beta on 29th December.

    Downsampling to reduce noise, but by how much?

    Comments Off

    Posted on 26th November 2008 by Phil Askey in Camera reviews |Software |Syndicated Press

    There has been much contention recently that we're being unfair on high megapixel cameras which show higher levels of noise than lower megapixel cameras. The almost universal argument is that "you can downsample the high megapixel image to reduce noise". This statement is often made with no evidence or example, this article is intended to provide some examples so that you can make your own mind up.

    Firstly lets be clear here (and not blind everyone with science), downsampling four pixels into one averages noise - that makes sense without understanding the maths. But downsampling four pixels means you're halving resolution, effectively turning your twenty megapixel camera into a five megapixel (or your G10 into a 3.7 megapixel), ignoring the improvement in sharpness you should see.

    In order to provide some samples I took our standard noise test shot in both JPEG and RAW from a Canon PowerShot G10 at ISO 800 (a sensitivity which is a stretch for almost all compact cameras). I downsampled these 14.6 MP images to three specifically chosen resolutions:

    • 10.0 MP (3648 x 2736)
    • 6.5 MP (2944 x 2208) *
    • 3.7 MP (2208 x 1656) **

    * 1.5 x 1.5 input pixels for one output pixel
    ** 2.0 x 2.0 input pixels for one output pixel

    To be sure I've got a fair cross-section of what the average user (not the average fanboy) would do in downsampling I chose five different methods of downsampling:

    • JPEG: Photoshop Bicubic
    • JPEG: Photoshop Bicubic Sharper
    • JPEG: Photoshop Bilinear
    • JPEG: Canon Digital Photo Professional
    • RAW: Canon Digital Photo Professional

    So lets see what effect this has on noise levels (standard deviation of middle gray):

    G10downnoisegraph_2

    Lets start with the JPEG image (red / orange / green lines), all methods except Bicubic Sharper (no surprise) result in a reduction in noise but it's hardly significant. Indeed even at 3.7 MP (a 4 pixel into 1 reduction) we're seeing very little reduction in measured noise. In RAW things are a bit stranger, using default noise reduction settings DPP delivers a noisier image at full resolution which suddenly dives to 'JPEG levels' at ten megapixels and then tracks as we'd expect it to 3.7 MP.

    If we take the most commonly used downsampling method (Photoshop Bicubic) we get a 4% reduction in standard deviation at 10.0 MP, at 10% reduction at 6.5 MP and a 20% reduction at 3.7 MP. Twenty percent is a nice number and it sounds good except don't forget you've now got an image which is a quarter of its original size.

    Enough graphs and figures, lets have a look at the images, as ultimately that's the most important thing (although some may classify debating as being more important). Below are (a) the most common (Photoshop Bicubic), the best performing (DPP JPEG) and RAW (DPP RAW)

    Photoshop Bicubic downsampling

    (left to right: 14.6 MP original, 10.0 MP downsampled, 6.5 MP downsampled, 3.7 MP downsampled)

    Canon DPP downsampling

    (left to right: 14.6 MP original, 10.0 MP downsampled, 6.5 MP downsampled, 3.7 MP downsampled)

    Canon DPP RAW downsampling

    (left to right: 14.6 MP original, 10.0 MP downsampled, 6.5 MP downsampled, 3.7 MP downsampled)

    As with all of our reviews / articles we provide you with the samples and will let you draw your own conclusions, mine is that you have to downsample a long way (like 4 into 1) before you get any really noticeable gain and even then noise is still visible and you've got a much smaller image. At the end of it all downsampling is no substitute for larger sensors or larger photosites.

    Why theory is great but grain size isn't

    One of the reasons that theories about downsampling reducing noise don’t appear to work in practice is that the theory assumes noise is random. Unfortunately, this isn’t necessarily true. Noise at a single photosite will effect adjacent pixels as part of the demosaicing process. So noise doesn’t occur as individual pixels but as grain. The mathematical theory may tell you that downsampling works but it won’t if your noise grains are any larger than one pixel (and they nearly always are from a camera with a bayer color filter array).

    In the example below we have three noisy images: one with a grain size of 1 pixel (commonly used for non-real-world demonstrations), one with a grain size of 1.5 pixels and one with a grain size of 2 pixels. If we downsample each of these by 50% (using Photoshop Bicubic) we see noise drop substantially for the 1 pixel grain image but much less so for the 1.5 and 2.0 grain size images.

    Original images (crops)


    (1 pixel; 11.4 std dev, 1.5 pixels: 11.1 std dev, 2.0 pixels: 11.1 std dev)

    After 50% Photoshop Bicubic downsampling (crops)


    (1 pixel; 4.9 std dev, 1.5 pixels: 8.0 std dev, 2.0 pixels: 9.9 std dev)

    Compact Camera Reviews – they’re coming (in a way)

    Comments Off

    Posted on 5th November 2008 by Simon Joinson in Camera reviews |Reviews |Syndicated Press

    Around a year ago we had our new reviewers trained to the point where they were ready to produce their own reviews. Before we even started recruiting we knew we'd start them off with compact cameras - partly because a compact camera review is considerably less involved than an SLR, partly because there are so many compact cameras on the market. Perhaps predictably every compact review we released was met with a torrent of complaints suggesting that we shouldn't be wasting time on 'point and shoots' when there were so many SLRs in the queue.

    By the time spring arrived two things were obvious; that we were going to struggle to keep up with the flood of new SLRs, and that the compact camera market had reached something of a plateau, with 'new' models being nothing more than 'old' models with marginally bigger screens or vital new features such as smile detection. So we took the decision (and, I should add, not lightly) to concentrate our limited resources on covering all the SLRs in depth before we went back to compacts.

    To be honest it wasn't that difficult a decision; we get at least 20x more people reading an SLR review than a compact review (with most compacts sold being in the sub-$150 bracket I think it's safe to assume that the amount of pre-purchase research buyers are doing isn't considerable).

    Fast-forward to today and we've finally scaled the mountain of new SLRs and are, save for the couple of models currently mid review (the Panasonic G1, for example), up to date. So what to do about the scores of compact cameras we turned a blind eye to over the last year or so? Compacts still make up the majority of camera sales and we certainly haven't abandoned them (we've already posted our full Panasonic LX3 review and have several others, including the Canon G10, in progress), but the sheer number released each year makes giving them a full review impossible (and, at the entry level, the cameras simply don't warrant a 10 or 12 page review).

    We therefore decided to try an idea Phil and I have been toying with for several years now; a roundup/group test that will, we hope, give a snapshot of the compact camera market as we enter the peak holiday buying period. So we spent a couple of days researching and shortlisting, and ended up with a list of 35 or so cameras to include. We've now got them all in the office and I've spent a week in the studio with them doing the first round of tests. We hope, within the next four weeks or so, to publish all five of the group tests (they're split into budget, 'style', 'advanced', 'high end' and 'SLR-style').

    To give you a taste of what to expect here's the list of cameras we've picked; don't expect a full review of every one of them (though there are some that will eventually make it to full review state), and don't be too surprised if a few of them change if we can't get hold of them....

    • Samsung L210
    • Canon A470
    • Kodak C1013
    • Sony W120
    • Nikon CP L18
    • Fuji FinePix Z20fd
    • Canon A590IS
    • Panasonic LZ8
    • Olympus FE360
    • Canon PowerShot SD790IS / IXUS 90
    • Casio EX-S10
    • Fujifilm Z200fd
    • Nikon CoolPix L210
    • Nikon Coolpix S60
    • Olympus Stylus 1040
    • Panasonic FX37
    • Pentax Optio S12
    • Sony DSC-T700
    • Canon SD880 /IXUS 870
    • Fuji  F100fd
    • Nikon Coolpix S710
    • Pansonic FX150
    • Samsung NV1000HD
    • Sony W300
    • Canon G10
    • Kodak Z8612 IS
    • Nikon P6000
    • Panasonic LX3
    • Ricoh R10
    • Pansonic TZ5
    • Canon SX110 IS
    • Sony H10
    • Canon SX10 IS
    • Fujifilm S2000HD
    • Fuji S8100FD
    • Kodak Z1015 IS
    • Nikon P80
    • Olympus SP565
    • Panasonic FZ28
    • Sony H50

    Adobe Camera RAW in our reviews

    Comments Off

    Posted on 4th November 2008 by Lars Rehm in Camera reviews |Reviews |Software |Syndicated Press

    We have a brand new blog which desperately needs some content before it can be launched and following our recent Canon EOS 50D review there has been quite a lot of discussion in the forums about the use of Adobe ACR as a RAW converter in our reviews. These, I thought, were two good reasons to get into blogging mode and write a brief article that explains why Dpreview is using ACR and not proprietary or other third party converters for its tests.

    So why is it then that we use Adobe Camera RAW and not one of the other converters?

    Firstly we aim to create a level playing field and to make results - as much as possible - comparable between cameras. For this reason we simply have to use a third party converter that can handle most of the many different RAW formats that are out there rather than a proprietary converter. ACR is used as a plug-in in Adobe Photoshop, Photoshop Elements and Lightroom and has the highest market share of all third party converters which makes it the obvious choice. More importantly though Adobe updates ACR very frequently (and we're involved in early releases) to include the latest camera models. This allows us to review a new model fairly promptly after its launch (we often get the cameras before anyone in the RAW development arena, including Adobe).

    We know that Adobe Camera RAW applies different levels of sharpness to different cameras and because of that we use the following workflow when processing our box shots:

    Load RAW file into Adobe Camera RAW (Auto mode disabled)

    • Set Sharpness to 0 (all other settings default)
    • Open file to Photoshop
    • Apply a Unsharp mask: 80%, Radius 1.0, Threshold 0
    • Save as a TIFF (for cropping) and as a JPEG quality 11 for download

    Even by using this workflow we cannot entirely avoid any processing differences between cameras but it's certainly as good as it gets in terms of creating a 'level playing field'.

    Secondly, especially when analyzing image noise, we want to show a camera's image output in its 'purest' form, i.e. as captured by the sensor and as little as possible altered by sharpening algorithms, noise reduction or any other forms of image processing. Again, ACR does a pretty good job in this area. To demonstrate this I have included example crops from two images which both have been processed from an ISO 3200 Canon EOS 50D RAW file, one in ACR 4.6, the other in Canon's Digital Photo Pro 3.5. Sharpening and noise reduction were set to zero in both converters, then we applied an identical unsharp mask to both images, so in theory we would expect pretty similar results.

    Dpp_acr_comp_4

    Reality looks slightly different though. The ACR image is less sharpened and shows visibly larger amounts of chroma noise. That tells us two things. Firstly DPP is doing quite a good job at noise reduction and is probably the better choice for cleaning high ISO pictures of the 50D (ACR's NR can't match DPP's even if you turn it up). However, it also means that DPP applies at least some chroma noise reduction (and sharpening) even when NR is set to zero which renders the software pretty much less useless for our purposes.

    Nevertheless, if you're interested in results that can be achieved with different RAW converters, you can have a look at the 'Software' page in our DSLR reviews where we usually post images of our resolution chart and the studio scene which have been processed in different RAW converters (if available at the time of the review).

    African Photography Blog – South Africa Photographic News, Articles, Tutorials is Digg proof thanks to caching by WP Super Cache