Astrofriend's homepage

Navigation

Advertisement /
Annons:

Project News

Advertisement /
Annons:

Tutorial:
Dark and flat calibration and an illustration and explanation why it's important to do dark and flat calibration


Dark and Flat calibration

  1. The digital DSLR camera features
  2. Linear and nonlinear
  3. The ideal image
  4. The real image and its components
  5. Vignetting
  6. Making of the calibration images
  7. What happens in the stages when we calibrate?
  8. Did we came back to the ideal picture?
  9. The math behind it

1: The digital DSLR camera features

In the examples, the data is partly simulated by a digital DSLR, Canon EOS 350D set to save images in RAW format. The camera records the images with 12-bit precision (raw), gives a scale of 4096 steps per color. Typically jpg-files have 8 bits only and gives you 256 levels, it is not enough for us astronomers and actually we would like to have it in 16-bit, 65000 levels! This camera has its optimal properties for astronomy photo at ISO800, ISO speed is the camera's gain factor. In principle, the described calibration is possibly to do on analog (film) images too, but much more difficult.

Back to contents

2: Linear and nonlinear

Linear and Nonlinear

In order to effectively process a image requires that it is linear and calibrated. What is linear can easily be expressed as:

A star that is 10 times bright as another star also shall provide 10 times the signal. Not so obvious that it is so! More mathematically, a linear function satisfy:

  • f (x + y) = f (x) + f (y) and f (a * x) = a * f (x)

When a camera is set to save images in RAW format is it usually in its linear format. However digital cameras even in raw mode, they have a certain compression in the higher range and then not exactly linear. Normal film is strong nonlinear and also a digital image in jpg format. A nonlinear reproduced image is very difficult to by math "count" back to linear format.

Back to contents

Advertisement / Annons:


3: The ideal image

The ideal image

This image represents the signal strength from various sensors (pixels) from a perfect camera. The grid is built up of columns and rows. In the case of the Canon 350D, 3456 columns and 2304 rows.

If the photographed object is a white board evenly illuminated, all pixels shall have the same signal strength, as shown. The height from the ground is the signal strength. Here exposed to 75% of total the dynamics are exploited, 3000 of 4096 ADU (Analog Digital Unit). Now, unfortunately, not reality itself, optics makes the light diminishes towards the the edges of the sensor, pixels do not have exactly the same sensitivity, we get noise in the system and there are usually dirt (dust) on the sensor and optics. A good camera has low noise, the noise from the light itself can we not do something about.

If we capture N photons this signal will have a noise of root of N (N = number of photons). The level of the signal/noise=snr, (number detected photons = N) divided by square root of N. N / (root (N))=root(N), that's the signal noise relation of the light itself. One sees that the dynamic increases (signal/noise=snr) if we capture many photons.

How will typical signal (picture) taken on this white board look?

Back to contents

4: The real image and its components

The real image

This is how bad it looks in the normal case! The picture illustrates how the signal falls towards the edges because of optical effects, vignetting. Surface roughness is telling us that we have noise and different sensitivity of the different pixels. Sharp peaks are often hot pixels, pixels that give signal without there is any light, also called static pattern. Can we do something about it to get resemble our earlier perfect picture? Yes, it is possible, especially if the image is close to linear as it is from a digital camera that stores images in raw-format. To understand what problems we have to deal with, we divide this image in its separate components.

Bias component

Bias component

In most digital cameras, there is added a offset to the signal (picture), called also bias or constant. This is done to prevent the noise should not give negative values for the lowest levels, negative signal will clipped to zero because we can only handle positive values in raw-format files. There is also technical aspects that make it beneficial that the signal does not begin at zero. How large this constant is are different between camera models, but typically between 50 to 1000 ADU. ADU is one step on the digital scale. This constant is now unfortunately not really a constant but also has a weak noise. Constants we can easily subtract but the noise is worse because it is random. The nosie come from when we read out the bias. The bias component is something we want removing from our image to restore to its original (real) condition. The editing program in the computer can normally handle negative values.

Dark component

Dark component

When the sensor (exposure) is activated it start an unwanted signal to be built up, regardless of the image that we want to capture. There is a thermal current that generates this. We want to also remove this component of our image because it doesn't have any thing to do with the object we taking photo of. If an extra picture taken with camera cover on (no light), we get precisely this picture (plus a bias signal). In the darksignal there could also be sharp peaks that belongs from defect pixels, called hot pixel. Here are two represented but typically it will be 500 or more! As the name thermal current say it is temperature dependent, increase with increasing temperature. More expensive cameras have a built-in cooling device of the sensor. How we handle this problem we will come back to later. Besides in older cameras the amplifier in the sensor emit a light (ampglow) that is not desirable, can be canceled or reduced with darksubtraction.

Note, in modern low noise cameras it could be a good idea to replace dark calibration with dithering technique, especially the static pattern "noise" of the camera must be low, take a look here:

  • Dithering
  • Flat component

    Flat component

    This component shows three of the problems we have at the same time.

    1. Optics decreasing brightness towards the edges, vignetting.
    2. Each sensor (pixel) has different sensitivity.
    3. A noise.

    Decreasing of the brightness (signal) because of the optics towards the edges relative the image's central part. Large aperture lens's usually provides stronger loss on the edges. This is known as vignetting. Choose a lens or telescope that gives an even illumination of the sensor as possible. It is possible to correct but severe vignetting produce more noise outward edges. The use of a lens (a telescope usually has no variable aperture) at full aperture is usually no good ide. Set the lens down one or two steps from full aperture is generally given a considerably better result. Vignetting cause two stars of the same brightness, where one is in the central part of the image and the other at the edge, they will recorded with different brightness. Something we have to correct for.

    Back to contents

    Advertisement / Annons:


    5: Vignetting

    Vignetting

    This is a simulation of a lens to illustrate the lens vignetting at different apertures and the signal we get along a line across the sensor. The advantages obtained at stop down:

      Prons
    • Usually sharper image.
    • More even brightness.
      Cons
    • In the example takes we must compensate with exposures that are 8 or 2 times as long (3 or 1 aperature step) compared with full opening. Aperature steps: 4.0, 2.8, 2.0, 1.4.
    Vignetting normalized

    As you can see in the figure which now is normalized, equal signal at center. It is only in the central parts we get full advantage of more light when we open the aperature of the lens, at the edges we don't get the same improvement.

    Back to contents

    6: Making of the calibration images

    We have now seen how our image is made up of different components. To calibrate our image, we need something that explains how camera and lens/telescope distort our image, it is here the calibration images come in. Here how we "produce" these calibration images.

    Noise

    During the calibration process, we will add, subtract, divide and multiply images with each other. In all these mathematical operations, we add noise to our image. The noise we add will come from our calibration images. We must reduce this to lowest possible level. One way is to take multiple calibration images and taking the average of them. The beauty of this is that after normalization to the same signal level as before the noise is reduced. If we start from 8 pictures it will reduces the noise by a factor of the square root of 8, about the third. So take several of each calibration image, 15 to 20 of each may be just right. More serious might take 50 of each, but also depends much on the quality of the sensor. In the examples here 8x used.

    The creator of DSS has done a very fine example of much it helps to have many darks in the calibration process:
    http://deepskystacker.free.fr/english/Look under map "How to create better images". I normally take 50 of each.

    Master bias frame

    Master bias frame

    A bias frame is taken when the camera lens is covered, use for example lens cap on and have the camera in a dark place. In all astronomy-photography, the camera can be set in manual mode and that also applies to calibration images. Set the exposure in a very short time so that the dark current does not provide any contributions. Choose for example 1/1000 second. Repeat this 8 times to get more images. In the example 8x bias frames averaged and we store it as a master bias. A bias is independent of the frame temperature (almost), exposure and lens. It can thus be used to several other images. If you use different ISO settings it should be taken bias images for each ISO setting.

    Master dark frame

    Master dark frame

    A dark frame is taken similarly to a bias frame. Now, however, you choose the same exposure time that you have for your object image, often called light frame. It is also important that the temperature and gain (ISO) is the same. Repeat this 8 times to obtain multiple images (sub images). The tricky thing about this is that you have to take them at the same temperature as when you took the light frames. There is an another tutorial about this on my homepage that give some tips&tricks, look here:

  • Taken flat calibration images
  • Flat image also need dark frames, and it has to have the same exposure to, but not very temperature dependent because of the short exposures we use for flats normally.

    In the example 8x dark frames averaged and we store it as a master dark frame. Compare with the earlier in the introduction so clearly visible how noise is reduced. From this also our master bias frame is subtracted. In addition, the "hot pixels" fixed. They have been erased and replaced with averages from the nearby pixels with the software you use.

    Master flat frame

    Master flat frame

    Flat frames are very difficult to take, the difficulty has to do how to get an evenly illuminated surface to shoot at. Some examples of how this can be done:

    1. Pictures can be taken of the sky while it is semi-dark, before the stars appear.
    2. A matte disk can be put in front of the lens or telescope.
    3. A white disc with uniform lighting can be photographed.

    It is important that the lens is exactly in the same setup as when you take your object photo. The same aperture and the same focus, if filters is used, these shall also be put on. Repeat this 8 times to obtain multiple images. In the example, 8x flat frames averaged and we get a master flat frame. From this also our master bias frame and matching master flatdark frame shall be subtracted. Alternatively, a master flatdark frame containing the bias is subtracted directly (a-a=0) in this case. It depends on your software how to handle this.

    Histogram on exposures for flat frames

    A digital camera has a limited dynamic range. We must stay inside that. In this example with a Canon 350D, 12 bits (per color). The signal level is divided in 4096 different steps. Exposure must be selected so that the peak value is at about 50 to 75% of the maximum, in this case is about 2000 to 3000.

    Linear hisogram

    The exposure can be viewed in the cameras histogram window. The image above is linear along the x-axis, you see immediately when "Top" end up at 3/4 of the scale length.

    unlinear histogram

    If it is logarithmic as this Canon camera has, then it looks as the signal is compressed to the right. Additionally, some sensors are strongly nonlinear at the higher levels, then it might be good to exposure on a lower level. Important is that the signal does not become clipped, i.e. fall outside the right side of the scale. To weak signal and it end up to the left, the signal becomes unnecessary noisy.

    Back to contents

    Advertisement / Annons:


    7: What happens in the stages when we calibrate?

    How do we use those calibrations images to calibrate our image and get it to be a perfect representation of the object?

    Here it describes how we use the different calibration images, those we made earlier and what happens in the different stages. To help to illustrate what happens when we calibrate our image I only use one line across the sensor, i.e. a cut across the image with the optic center in the middle.

    Real image of a white board, bias+dark+flat component

    Linegraph of bias, dark and flat component

    The picture above illustrate the picture we took in the beginning of a perfectly illuminated white board. All errors are there. The unwanted portions of dark and bias are marked in different colors as well as image brightness decreases towards the edges.

    Bias and dark subtracted from real image

    Bias and dark subtracted from real image

    Here is our picture subtracted with a master dark frame and a master bias frame. Note that the bottom is uneven, this symbolizes a noise that we can not get away from. Note: Now our signal could contain negative values, our our image editing software must handle this! The noise is random and thus we can't subtract it away, it will not disappear, actually we added more random noise to it from our calibration images as was told earlier!

    image / master flat frame * constant

    image / masterflatframe * constant

    The last step is to divide our image with the master flat frame and multiply by a constant, in this case selected so that we normalize for the level of 3000, that was the signal we had at center in the begining, but you can normalize it to whatever level as long your editing software can handle it. Note that a master flat frame has been subtracted with dark frames taken with a different exposuretime than the dark frames was subtracted from the image before.

    Back to contents

    8: Did we came back to the ideal picture?

    The calibrated result

    Here is the final result of all our calibrations. The picture now does look much more as the ideal camera and lens should have taken. Ripples on the surface tell us about the noise. It is only now when we have calibrated (pre-calibrated) our picture we can go further, the image processing we want to do.

    Back to contents

    Advertisement / Annons:


    9: The math behind this calibration

    Astronomy picture AI1, AI2, AI3, ...

    To calibrate the image, we needed to take a large number of extra photos calibration images.

    • 8x dark frames = dai1 ... (which contains bias), need a set for each ISO, exposure time and temperature.
    • 8x dark frames flat = df1 ... (which contains bias) one set for each ISO, flat exposure.
    • 8x bias frames = BF1 ... needed one set for every ISO setting
    • 8x flat frames = FF1 ... needed a set for each telescope/lens and aperture setting and focus setting

    In astronomy we usually focus only on the infinite, however is not always so easy!

    Astronomical objects have an enormous dynamic range and to manage this object can be photographed with multiple exposures, known as HDR (High Dynamic Range) technology. To avoid having to take a set of dark frames for each temperature, one can assume a set and calculate the other. Will not be as good but saves a lot of time.

    It looks like this mathematically for some different approaches:

    A is a constant to optimize the subtraction of the dark frame, D is a constant to normalize the calibrated image.

    One image and no bias:

      Calibrated image no 1 =
    • (ai1-(dai1+dai2+dai3+dai4+ dai5+dai6+dai7+dai8)/8) / ((ff1+ff2+ff3+ff4+ff5+ff6+ff7+ ff8-df1-df2-df3-df4-df5-df6-df7-df8)/8)*D

    One image and with bias separated for optimizing of dark frames:

      Calibrated image no 1 =
    • (ai1- (bf1-bf2-bf3-bf4-bf5-bf6-bf7-bf8) / 8 - (dai1+dai2+dai3+dai4+dai5+dai6+dai7+ dai8 -bf1-bf2-bf3-bf4-bf5-bf6-bf7-bf8)*A/8) / ((ff1+ff2+ff3+ff4+ff5+ff6+ff7+ff8 -df1-df2-df3-df4-df5-df6-df7-df8)/8)*D

    Usually taken several images of the same object in order to keep down the noise, for each of these images must have above process repeated.

    In a modern low noise camera you have alternatives, look under my tutorials about dithering:

  • Dithering
  • Back to contents

    back