2009-07-19

Homepage, community and distribution update

Since February, I'm working hard on new properties of Munipack. A long-time fine tuning of parameters of fitspng was the initial impulse for me. I spend a lot of time (hours) with selection of appropriate values to get a naturally looking picture.

Of course, this is fault of any command line interface. So I've started working on a GUI to Munipack. It inspired me to did some additional changes.

I think, that is one of ways to pass my photometric experiences to next generations. A software in action, together with a source code, may help me describe all details of methods. They will illustrate more preciselly than a natural language. Any white-paper or a web article can't fully describe implementation details. From this point of view, Munipack is a part of my mind and experiences.

Homepage

The past-looking homepage of Munipack has been replaced to a new one. Its style is improved version of Nightview's homepage. All pages are static and they are generated directly from my version system one per day.

I also moved main Munipack site to this new address:


to highlight connection between Munipack and Masaryk university using of .muni.cz domain. The original name is inspired by DAOPHOT originated at Dominion Astronomical Observatory. The -pack suffix means that a set routines isn't focused only to a specific photometry task.


Community site

I founded Munipack's side on Google code:


The site provides two important tools. Wiki which can be used by anybody to write some descriptions, experiences or to summarize some procedure of data processing useful for others. I'm expecting that the wiki will used generally for astronomical image processing. I'm will very gladly when Munipack will used for the processing.

Google code also offers a bug-tracking system. It is an ideal place for reporting bugs and feature requests. I think that will provide for me some better arrangement of development needs.

I think that a mailing list may be very useful to Munipack. So I also founded the Google group for Munipack:


Again, It is intended for a general discuss about the astronomical photometry and I will greet with using of Munipack for this work.


Binary distribution

While average Munipack user is not familiar with a code compilation process, I'm providing binary distribution packages. At the time, only Linux 32-bit and 64-bit packages are generated from Munipack's source tree. The installation itself is really simple like installation of a Linux game. Unfortunately, the distributed binaries occupies a lot of space because they must contain wxWidgets, libpng and cfitsio libraries.

The binary distribution tree stores two sets of binaries. The bin/ directory contains statically linked routines. They will work without any system setup. Opposite with this, the lib/munipack/bin/ contains shared binaries which needs libraries linked against in a system or a specified directory. The setup is done via a shell script in bin/ directory.

Binary installators are created with help of Makeself utility. The packaging itself is done on base of ideas published in that article: Linux Game Development by Troy Hepfner.

The output binaries will probably work on any modern Linux. I haven't resources to directly prepare of packages for a specific Linux distribution like Ubuntu, Mandriva, Fedora etc. The script to generate of the binary packages is included in source distribution in dist/.

In any case, I still recommends source code installation including compilation itself. The advantages are:
  • smaller binaries, optionally with tuning
  • better setup of optimization, be faster
Especially, the compiled binaries are smaller and uses system libraries (usually optimized). They are occupied both less disk storage and a computer memory. The optimization on a target platform and a processor can significantly speed up of run. Especially using of advanced instructions (SSE,..) on 32-bit processors, which are not switched on by default, can be important.

Also, I strongly recommends use of Intel Fortran compiler (ifc) which strongly boost run. Unfortunately, binaries created by ifc can't be freely distributed. Ones are created by GNU tools: gfortran and C, C++ compilers.


Warning

Munipack is in rapid development phase now. You can find bugs, a strange behavior or something like this. Also anything can be changed.

2009-03-28

Recent progress in RAWTRAN

There is a summary of last progress in rawtran (see previous posts: Dec 2008, Jan 2008).

I specified more precissely of the definition of the FITS color format. Be inspired by PNG format (libpng), I introduced a new keyword to the first dummy header

COLORTYP

by PNG_COLOR_TYPE_* to be setup to (string) value RGB for color FITSes. It will be useful for any software to easy recognize of the color format. Also the definition require additional condition on size of the images that is required to be equal for all color bands.

Another change is replace of the CBALANCE header keyword in header by the keyword COLORBAL (COLOR_BALANCE) with the same meaning. I think, it will be put better understanding to users. The CBALANCE is now obsolete and I don't expect any back-compatibility.

I added another type of conversion. The new type 4 produces multi-band FITS file including of all four Bayer colors in separated FITS image arrays. The type provides absolutely raw access to RAW data. It is another multi-band FITS different of the color type.

Some additional changes has been included to rawtran recently:
  • Cleanup of code.
  • CREATOR keyword.
  • EXIF information by dcraw is written in the header block with "begin" and "end" delimiters to be simple parsed.
  • Update of man page.
  • Run-time detection of dcraw.
  • A switch to pass additional parameters to dcraw.


It is still uncertain of the true meaning of Daylight and camera multipliers provided by camera and its meaning on a color white balance.

This post is a small virtual present to anniversary of FITS format which has been established exactly 30 years ago.

2009-01-26

Crumbling walls of Děvín

We had a trip onto ruins of Děvín (Dívčí hrad aka Castle of maid aka Maidberk aka Maidenburg) on mount of Pavlovské vrchy (Pálava, Pavlov's mountains) at Saturday.I acquired a panorama view of dam Nové Mlýny and a part of South Moravia with on top of the crumbling walls. A small village near center of the image is Dolní Věstonice, where prof. K. Absolon found the prehistoric nude sculpture now known as Věstonická venuše. All area under the mounts has been the first part of human settlement on Moravia.

The panoramatic view from Crumbling walls of Děvín. The large version by RAW pictures can be compared to one by JPEGs adjusted by Canon EOS 30D. Is a pastel better than reality?

We also visited a next airport near of Strachotín. It has a nice horizon profile. Unfortunately, a gas station is on the east. Any zodiacal light has not been detected again and the Venus's shadows may be visible, but not by us.

Two Venuses on west in large. The zodiacal light is not visible (?) due clouds.


Pálava at night. Děvín is at peak on left. The clouds on right over Pálava are illuminated by Mikulov town.


Lights on east. The most intensive source is probably a gas station.


Brno on horizon on north at distance about 30km.

Thx for nice accompany to Maceška.

2009-01-18

Approximations of a sidereal time

A precision of computation of a sidereal time has been arouse curiosity of me during last days. So I experimented with formulas on the page Approximate Sidereal Time. I compared of computed GMST, the low GMST version ( GMST = 18.697… + 24.0657…D) and GAST with data by Horizons ephemeris. The enclosed graph represents of final results. The functions shows residuals of the computed sidereal times minus Horizons's ephemeris in time the interval from 2000 to 2050. For example, the red line represents of the difference: GAST - Horizon.

Comparison of sidereal time approximations. Click for SVG version.

As we can see that GAST is a perfect approximation for general purposes. But the GMST's approximations are also acceptable. The GMST is not corrected for a nutation with period of 18.6 year and one is nicely visualised by a big wave. A rapid wave superposed onto the nutation period is demonstration of annual orbit of Earth. Strangle breaks at 2006.0 and 2009.0 are effect of the adding of leap seconds. The low precision GMST approximation has visible differences, with respect to GMST, only at the end of the time interval.


Conclusions
  • To compute of sidereal time with low precision use of the low precision formula: GMST = 18.697374558 + 24.06570982441908 D, where 18.697… is sidereal time of the reference time 2000-01-01 at 0 UT, 24.0657… is a ratio of synodic and sidereal periods of Earth and D is days (and its fractions) since the reference time.

2009-01-03

An evening with Venus

The extraordinary brightness of Venus as the evening star invoke many assorted fantasies. Lovers dreams about a paradise on the Venus, which is an island in the universe darkness. For peoples believing on Martians, Venus setting appears as a landing of its spacecraft. Venus looks as in fast motion, especially, when Venus is observed in some clouds. Venus setting can be also very impressive for painters, photographers and astronomers. The magic performance of ambient light, twilight sky, surroundings and Venus itself offers the absolutely extraordinary experience.

I spend one evening with Venus by observing of the setting and many days and evenings with processing (and related business) of acquired images. My primary goal of the observation was pointed on visual demonstration of an astronomical refraction. The follow-up processing has been inspired me to mining more information from collected images.

Snapshooting

To acquire of nice sequence of refracted Venus, I selected Kotvrdovice's airport what I visited some time ago. The place offers relative good altitude about 560 meters over sea level, so I expected perfect horizon together with a separated place available by a car. Kojál top at proximity has the no separation, place to parking and is close to a road. The expecting Venus setting point is situated in direction of south part of Moravian Karst with low air (light) pollution.


View Larger Map

I pitch my tripod at point 16:47:20.1 E, 49:21:52.1 N. The place is on a rural road and on a local ridge near of a hunter's hide. The first snapped image shows a scenery at moment after my arrival.

The first image.

I manually grabbed a set of images for every minute with start about 15:35 UT and finish at 17:18 UT (but see next note!). Total 101 images has been acquired. Canon EOS 30D has been used with lens EF-S 18-55mm, f/5.7 without any zoom, ISO 1600, a minimal diaphragm. Exposures times has been changed during a light fade of the twilight as I completed into the table:

15:35-15:49 15 1/100
15:50-16:00 9 1/10
16:01-16:15 13 1/2
16:16-16:39 23 4
16:40-17:18 38 8

An air view to Kotvrdovice's airport. The hide is on left part between the airport's facility and a wood. Kojál transmitter at left top corner, a village Krásensko at top, a small part of Senetářov and Kotvrdovice on right edge from top to bottom (in this order).


Time non-synchronisation

Unfortunately, I forget done any time synchronisation of a interval clock of the camera before or after of the trip. So precise time synchronisation is lost forever. Fortunately, I check of time on my mobile phone at start of observation but for information purposes only, without required (on second) precision. It means that all times are considered to be inaccurate (± 30sec). The relative precision is better than 1 sec.

Preprocessing

I acquired all images manually by clicking of button on Canon's body. So I expected that the images can be each other shifted. Therefore, I used the translation property of a cross correlation of Fourier transformation of images to precise co-add of all images to same origin. So, I created of the forward FFT of reference and working copy images. The Hanning window function has been used for bottom part of original images. The phase-correlation and backward FFT followed. Finally, shifts has been computed as centroids of delta function-like peak.

Shifts has been released only for integer segments. I expecting that precision would be not grow, if a non-integer shifts will be used and interpolation would be degrade sharp features of the output image.

An example of sum of images with and without shifts is shown on the zoomed subimage. The image on the left is simple co-add of all images while the right image shows shifted and co-added images. In my opinion, the careful preprocessing leaded to a little bit sharper picture.


The displacement of color layers worked well when I used all colors together of weighted by white multiplication factors (eg. on grayscale images). Initially, I tried to get shifts for every color layer individually and than compute the mean between its. I expected a fine shift due to Bayer mask, but the shift showed some random behavior as a perhaps product of a noise.

The final image has been created by a method analogous to Iris function ADD_MAX by Ch. Buil. Simply, the output image is not only sum (or mean) of two images but one gets the brighter pixel of images. The method grows intensity of a potentially moving inter-image object (Venus). Without this, the Venus trace will be lose on evening sky. However, final colors of the (variable) sky background may be very strange.

The white balance parameters provided by the camera gives not satisfactory color balance. Therefore I equalized the image by hand. Usually on a white object on images like Venus or by automatic way with -fr parameter of fitspng.

The most important part of the processing is on base of my specific routines. I'm attaching link for an inspiration.


Venus setting on 15 Nov. 2008 . This composed image shows track of Venus (central body) above south-west horizon at Kotvrdovice's (CZ) airport. The chain of pearls has origin in short exposures of Venus in one-minute intervals. We can see that the finish of observed path is raised with respect to expected (by following of beginning part) one. It is exhibition of the astronomical refraction of the light in Earth's atmosphere. The track also shows decreasing of amount of the light and its reddening during the setting. Both effects are product of scattering and absorption of light in air. Read the post for detailed description.



Astrometric calibration

The astrometric calibration has been done by Gaia software on grayscale version of one of latest images including some stars. The calibration shows that a projection of the lens is gnomonical with precision better than 2-3 arcmin/pixels. A scale is 0.0165 deg/pix = 0.99 '/pix (minute per pixel). A field of view is 29°×19°. The center of the calibrated image is at coordinates 19:13, -26:00 at 17:17 UT (time epoch). Equatorial coordinates has been rotated about 25° to image's coordinates. The image itself is rotated around vertical (horizontal coordinates) by approximate 1°. The parameters are visualised on the image:




Astronomical refraction

The main goal of my work is to show effect of the astronomical refraction. It can be easy obtained by observing of any point source in time, but when the object downwards really slowly with acute angle to horizon, the observed path may be visibly deviated from non-refracted one. The deflection is nicely demonstrated on annotated images:




Technically, the refraction can be described on a graph of dependence between the observed and the true (not affected by the refraction) zenith distance of some object. The expected values has been computed by a standard way from linearly interpolated coordinates of Venus by NASA's Horizonts. It agrees with values by Xephem.

Measured coordinates has been derived from grayscaled images. I estimated centroids of Venus by the weighted mean. The error of the determination is order of tenth of pixels/arcmin.

The next step is computation of the zenith distance (or elevation) of Venus. In principle we can use two methods:
  • Plain. Using by a scale and a simple geometry, we determine distances in pixels of Venus and horizon. The Euclidean geometry with only rotations and translations is used.
  • Gnomonic. Using by known rectangular coordinates and the gnomonic transformation, we determine equatorial coordinates together with transformed to horizontal coordinates.
Both methods may include some systematic errors. Simple plain geometry has known deviations from spherical coordinates when angles gets values over five degrees. The gnomonic transformation fits spherical coordinates by the better way, but we are not sure that our object lens display some scene by the expected way.

The results of both methods of determination zenith distance from photographic plate are plotted in refraction graph. The plain method shows nonlinear great residua and it is clearly unusable. The gnomonic method is little bit better but there are still great differences.


I also plot of Bennett's approximation of the refraction formula according to Astronomical Algorithms by J. Meeus. Other simple refraction laws on base of tangents of the zenith distance (one or tree order) are unusable in the range of observed quantities.

The visible discrepancy between the approximation and observation has unclear origin. The statistical errors are visualised by random noise. I notices relative higher noise on beginning of observation how it vanished when the ratio of signal to noise is growing. I expected that the refraction will affected by some clouds near of the horizon. Nevertheless, the appropriate part of the measured curve is nicely smooth.

The discrepancy is not due to uncertainly on time determination (the change a few minutes minute slightly modified profile but not importantly). It is also valid for changes in an atmospheric pressure and an ambient temperature.

Only for information, I tried to fit the data by any way. A final fit is a set of points tightly reproducing of the Bennett's approximation. The fit has two free parameters: The relative time between the astrometry approximation and observation times has been changed relative about minute (this is principally incorrectly). The change leaded to a little bit deflection of refractive curve so the profile is more similar to the Bennett's approximation. As second free parameter, a vertical shift of 7 pixels/arcmin has been used (also it is uncorrected again).

My own hypothesis of explaining of the measurements is a non-precise model of the (gnomonical) projection. Firstly, I assumed that the projection is gnomonical on full area of the chip. The assumption may be wrong. The gnomonic projection strongly deforms area elements far from center of projection. Therefore, manufacturers can use any different kind of the projection. For example, some "mean" between the gnomonic and a stereographic projection may be used. Secondly, the fitting routine of Gaia assumed such model of the gnomonical projection which have different scales in Right Ascension and Declination, but my images are deformed in some another direction. The images are squeezed in the vertical direction by the refraction. It leads us to use of a model which will be correctly describe the refraction during of gnomonic projection determination. Simply, we can say that the vertical scale will not be more linear, but it will varied with the zenith distance. The construction of the model will not complicated but I abandoned its due to some uncertainty in the time determination and due to low number of stars near of horizon, so I assume, that the corrected method will not give significantly better results.

The refraction itself is a very fascinating field of applied optics. I recommends visit of wikipedia page and pages of Andrew T. Young about refraction.

Photometry

When the light of a setting object pass trough layers of atmosphere, the observer can detect an exponential attenuation of light together with a color change of the object. Both effect are clearly visible on composed image above.

I think than more better visualisation is represented by a standard photometric analysis of images. By the way, I done an aperture photometry by obvious method (simpler version of one implemented in Munipack). The output magnitudes has been normalized by exposure time and calibrated by the known extratmospheric visual magnitude (again by Horizont) of Venus in G band. I didn't used color balance factors and subtract of a dark frame because the ambient temperature varied about ten degrees. I only subtracted a bias of images which I assumed on level 128 for a RAW image.

The spectral sensitivity of Canon EOS 30D is not available. Therefore, I assumed that the property is similar to one of EOS 300D. I approximated the sensitivity in R band by Gaussian with its center on 600 nm and its half-width about 30 nm, G band as then rectangle with the height 100% and in the range 500 - 600 nm and B band as the rectangle with the height 75% and the width 430 - 480nm. By the way, I get correction color indexes as 0.27, 0, -0.38 for R, G, and B magnitudes and a black body on the temperature of Sun. The reliability is low, so the color indexes may contains systematic deviations of order of tenth magnitude.



The instrumental magnitudes has been calibrated via extinction lines in range of 9 to 30 airmasses. To estimate of the airmass I used formula by (perhaps) Young(1994) , see also article on wikipedia. The least square fitting gives following values extinction coefficients in all bands:

B 0.188± 0.003
G 0.114 ± 0.004
R 0.090 ± 0.003

The first part of graph doesn't follow the extinction law and the magnitudes are perhaps absurd. I think that this is the demonstration of effect described in my previous post. It is simply due to extremal height level of background with comparison to the light signal of Venus.

Sky's brightness

The last graph shows surface magnitude of sky near of top of my images (at elevation ten degrees). I selected a rectangular angular area with constant position. The selected magnitudes shows expected profile. The uncertainty may by a few of tenth of magnitude. The values in minimum are a few magnitudes over natural sky. It may be due to really dark sky when the camera detect thermal noise only or the sky may be illuminated by a near village or a town. Note that when Venus set, the Moon has raised on opposite point of the sky.


Color index

The last graph shows dependence of color indexes on time. I note that the numbers has different base than classical astronomical color bands. Only relative changes can by surly determined from the profiles.

The profiles of Venus shows strong evidence for reddening as we expected from visual determination. The graph shows that the reddening may by two magnitudes which mean 5x more light in blue is absorbed or scattered.

The color index of the sky is strongly different. It means that blue sky has arrived more red color. The color of sky is not more blue during a night. This is also bring out the color of the sky on composed image. The sky color is some kind of mean of the color index.



Thx to Fantom for many suggestive ideas.

The post is dedicated to Vladimír Znojil, my diploma thesis supervisor, which died on 29 Dec. 2008.

2008-12-23

Improvements in FITSPNG.

With two appendixes.

Fitspng has been originally developed as an utility for a fast conversion of FITS images to PNG format. It is required for our Monteboo data archive for observed image display in an usual internet browser. The conversion can be also required for example for color imaging (look forward).

Some time ago, I did the conversion by two steps: to8bit utility converted normal (16-bit) FITS to 8-bit with an intensity scaling and convert (utility by Imagemagick) done FITS to PNG conversion. The way have been simple for me (I didn't have learn of libpng API) but one is slow due to high system load and storing of middle product on hard disk.

A long time (two years at least), the functionality satisfies of me, but now I have new ideas and I extended functionality of fitspng with respect on rawtran utility and some new knowledge.


Color images

If a multi-color FITS or tree normal (gray) FITSes is passed to fitspng as file to convert than 8-bit color png is created by the algorithm:
  1. Fits is opened and recognized as multi-color FITS. All color bands are loaded.
  2. The median and mean deviation of positive differences is computed on 10x10 grid (to speed-up the estimation) for every color band. The computation is abandoned when user specified the -fl switch.
  3. These estimated parameters are merged with an user specification by -fr switch. Also white balance is setup via CBALANCE FITS keywords of every band or by -w switch.
  4. Than every pixel is scaled by specified intensity profile, supplied parameters and white balance.
  5. The file is saved as color PNG with RGB colors (without alpha channel). (May be useful add the alpha to image?)
There are two important requirements for the colorization. A simple way how to convert images from a digital camera processed to FITS by rawtran will be:

bash$ rawtran -t 2 IMG_666.CR2 > color.fits
bash$ fitspng color.fits > color.png

Of course, the direct way by ufraw or dcraw will be faster, but you can't apply some enhancing on images and mouse clicking may be really time consuming.

The most important application is on color composing of image data of CCD. The composition itself for tree images in R,V,B filters can be simple:

bash$ fitspng m1_R.fits,m1_V.fits,m1_B.fits > m1_color.png

The default setup of parameters will produce relative nice images but for more nicer images will be need some fine grained tuning via -fX, -w and by specifying of one from transformation functions: asinh, log (magnitude), error function, sqrt and linear transformation. The setup of various parameters by -fX is usually required when a histogram has and non-usual profile (always for non-stellar images).

If you requires nice publishable images, it is absolutely necessary to done basic processing like, dark and flat field correction and detailed photometric analysis. Ideal way is looking for A0 type (not saturated) star and balancing of the images by hand or by -w.

Note, that all single images has its own header with defined keywords which can/are be different. If any keyword is duplicated we get only the first match.

The multi-color FITS has structure:

FITS
+-----------------------------------+
| Primary array - dummy |
+-----------------------------------+
| Extended array - R image |
+-----------------------------------+
| Extended array - G image |
+-----------------------------------+
| Extended array - B image |
+-----------------------------------+


Intensity (flux) scaling

Inspired by Christian Buil, I had implemented asinh intensity profiles together with logarithmic, square root, error function and reimplemented the linear profile. Results are really nice. The profiles are presented on images. BTW, the intuitive color conversion curves which I used with ufraw are similar.

The conversion can be described as an function between output and input data. The input intensity I can be integer or real number without specified range. The output O level has integer range 0 .. 255. The I is transformed to O by

O = f0*f(x) + z0

where f0 is a scaling constant and z0 is black level. The function f(x) can be one from list:

asinh(x)
log(x)
erf(x)
sqrt(x)
x (f0 = 1)

The argument x is computed by a linear transformation

x = (I - t)/s

where t is a "mean" level and s is "mean" deviation. Values t are estimated by fitspng by median of grid pixels with separation of ten pixels (med). The s is determined by the same way on the same set but from positive deviations of t (not from absolute deviations as usually) (mad). Both parameters are corrected by constants u,v:

t = med - 3*mad
s = mad/15

The specified intensity transformation is a product of black magic on images and it can't be derived from an exact axioms. It empirically describes usual parameters of histograms of astronomical images.

The asinh magnitudes are used as photometric product of the Sloan sky survey. They recommends of use of the magnitudes in Lupton et al.(1999) and for color imaging in Lupton et al.(2004).


Working with full directory of images

Fitspng is ideal utility to done fully automatic conversion of fits images. It can be useful for fast look analysis in anyimage viewer or simple file browser. The thumbnails can be generated by command:

bash$ for A in *.fits; do fitspng -a -s 5 $A > ${A%fits}png; done

The switch -s resize the image 5 times.


How to create an animation

If we have images in png format, it is trivial to create an animated gif (with 50 milliseconds delay between images) via convert command of Imagemagic:

bash$ convert -delay 50 *.png animated_sky.gif

Larger animations is preferred to save to mpeg format. It can be created with small script:

---
#!/bin/bash

I=0
for A in *.png; do
for B in $(seq 1 25); do
I=$((I + 1))
N=$(printf "img%04d.png" $I)
ln -s $A $N
done
done
ffmpeg -f image2 -i img%04d.png animated_sky.mpg
---


The ffmepg requires 25 frames per second (at least 10 fps) so we create 25 times links on a single frame. The conversion requires relative small images with approximate size of a post stamp (about 800x600).


Notes.

I spend some time with optimizing of fitspng for speed because the conversion of normal size image gets a few seconds. Unfortunately, any hooks in main cycle are insignificant and, perhaps, the all actions consuming approximately the same part of execution time. The conclusion is that any (great) numerical operations has negligible fluency on execution duration. It is possible to visually inspect of the duration with setup of -v (verbose) switch.

I'm little bit confused about use of keywords of FITS files. They can be defined arbitrary and fitspng can produce fake outputs. I think that the most important is CBALANCE. Fortunately, the keyword can be replaced by -w switch. All others will only put wrong of description of image included in PNG file. The used keywords are listed in source fitspng.c.

The utility is coded in standard C but the coding style is old Fortran like without extensive usage of some user defined types and complicated structure of functions. I believe that the style is fine for coding and long time maintained of filter type utilities. I will not lost in function tree.

I'm expecting additional changes in future.


Examples of images

First image is composed exposition of field of NGC 7635 (Bubble nebula in Cas) acquired during summer 2006 by me and Exebece on MonteBoo Observatory (0.6m refl.) and HaP MK Brno (0.4m refl.) by non-enhanced cameras ST-8 and ST-7 in R filter. The image has been created by composition of 1084 images of total exposure time 95540(!) seconds. The images has intensity scaled by asinh and linear profiles. The differences are most remarkable at central part. The central star is saturated on linear scale, while faint details are visible.

Bubble nebula NGC 7635. Linear intensity profile.


Bubble nebula NGC 7635. Asinh intensity profile.

Second example shows Dumbell nebula (M27) exposed on MonteBoo (0.6m refl.) by Janapka two months ago. It is composition of 3x11 exposures (3x440 sec) in R,V,B filters by our enhanced ST-8 camera. The images has not flat-field corrected.

Dumbell nebula M27. Linear intensity profile.


Dumbell nebula M27. Asinh intensity profile.

So, I strongly recommends use of asinh intensity profiles.

Post scriptum. Scaling to preserve all colors

Dumbell nebula M27. Asinh intensity profile.
Scaled by intensity. Note better preserving of colors.

The paper Lupton et al.(2004) recommends as an optimal way to scale of output image by intensity rather than every single color. You can use the -f switch to invoke it. The key difference with respect to single color scaling is coloring of the saturated objects. The intensity scale algorithm preserves color ratio in (output) saturated regions. The saturated stars have approximate colors opposite with the white color of previous algorithm. Of course, really (on CCD chip) saturated stars are white again. Unfortunately, the saturated stars can create a strange color defect. To remove it, use another black magic.

Post scriptum. I forget attach a graph of all profiles. Please look on the linear (red) and the logarithmic (blue) and asinh (light green) profiles which ideally "interpolates" beween its.
SVG version.



Download of FITSPNG.

2008-12-20

Improvements of RAWTRAN

While work on processing of set of images of Venus setting (I will it show soon), I found important bugs and revealed some possible improvements in rawtran utility. Moreover, I also found that only the type zero (default) conversion has worked correctly for all versions up to today. The default choice have composed four RGGB pixels of the Bayer color mosaic to one big grayscale pixel by use of unique weights for separate colors.

The Bayer mosaic is defined as red, blue and two green pixels arranged to rectangular matrix. It means that the number of different colors is tree but we have four pixels on detector matrix. Two green pixels are reserved for single color. Now, we have two choices for output intensities: sum of the pixels or their arithmetical mean. The sum gives precise value in integer numbers but one is two times greater than value of other colors. Opposite with this, the mean gives values of the same order as the red and blue pixels unfortunately with a rounding error as result of conversion between real and integer numbers. I think, the use of the mean and real numbers in FITS should be considered as a smooth alternative when a lot of disk space is available. Also sum of two greens and double values of red and blue can be a nice alternative for 12bit and 14bit A/D converters while we are storing data in 16bit FITSes. The arithmetical mean is the most simple alternative with small lost of precision. Please leave a comment about a preferred way to solve the controversial point.


Color weights

As the first noticeable change, I mention about the color balance (weights) used for conversion of a color based to a greyscale image. Generally, the conversion is done by the formula

I = w_R*R + w_G*G + w_B*B

where I is grayscale intensity, R,G,B are proper intensities in appropriate filters and w_i are a suitable weights. The choice of w_i depends on problem to solve. The setup of w_i is supplied by your camera according to spectral profiles of color device elements and to light conditions of an acquired scene. The photographers perhaps uses the term "white color balance" for the procedure, because it can be easy established by balancing of the coefficients on a white. Now, the camera setup is default. The previous versions used w_i = 1 without possibility to specify by hand. Note that both setups are useful in different situations. The camera supplied setup is ideal for images represents intensities close to reality. The 1's weighting can be useful for detailed spectral examination of images.

The color weights are derived from camera supplied data in field "Camera multipliers". The four numbers B1 B2 R1 R2 are used to estimate of color balance as:

w_R = R1/R2,
w_G = 1,
w_B = B1/B2.

The green band is supposed as basic unit and both remain are relatively scaled. The numerical values agrees with ufraw data. I don't know why also provided "Daylight multipliers" are not used for the reason.

For type zero the image is grayscale and the value is indicated by keyword FILTER which is set to GRAY. Weight factors are not directly included in keywords.


Multi-color fits

The most important change in the code is connected to multi-color (band) fits images. Previous versions has been badly decoding of Bayer's mask (the colors has been swapped). I corrected the bug and changed structure of resultant FITS files.

The effect of correct separation of colors can be easy showed on the image of the spectrum of a white source (data projector) observed through a rainbow glasses (famous paper - diffraction foil glasses). Look on included image for details. I'd correctly decoded the structure of image color mask on the image.

Color image and separated color bands.

Since recent issue, the multi-color FITS has following structure: the dummy primary image array (the header without any data) and tree color (band) extended images. Every header of the extended data contains a new keywords FILTER indicating one from set RED, GREEN, BLUE and appropriate color weight factor included as CBALANCE. Other new keyword EXTNAME is the same as FILTER and can be directly used to select the filter. For example, with ds9 you can directly select blue band by the command:
bash$ ds9 x.fits[BLUE]

The same selection can be done also as x.fits[3]. The file names uses the extended file name syntax introduced by cfitsio library.

Note, about multi-color (band) FITSes. The packaging of more images to one file is frequently used in space astronomy, where an experiment on a spacecraft records a radiation on many different wavelengths. The color mode is particular case of the gadget. The idea can be also extended for including of correction or calibration images or data.

Grid mode

The grid mode (type 1) simply arrange of pixels without collecting or space separating of pixels. I have no idea about any usefulness of the mode. Also I don't fully understand why noise (dark) level and white (saturated) regions are correctly displayed but the mid-level intensities produces grid structure.

8,16,32,-32 bits per pixel images

The output FITSes can have specified bitpix parameter (see man page). It can be very useful when you wanna save fine precision of intensities. The non-integer values are common product of white balancing when type zero conversion is selected. Also, the problem extensive discussed above is suppressed.

I suppose that the mode 16-bit is perfect for storing of raw data, the mode -32-bit for further processing and modes 8 and 32 are unusable. Images with 8-bit depth will usually white due to overflow of image's values over 255. The 32-bit mode only grows range of integer numbers and practically only occupied twice more disk space without any growing of stored information.


Examples of conversion

Basic run (equivalent commands):

bash$ rawtran IMG_666.CR2 > IMG_666.fits
bash$ rawtran -o IMG_666.fits IMG_666.CR2

The produced image IMG_666.fits contains grayscale image with half image dimensions of the original. Please, respect rule: first options than an image to convert.

Process all images in directory:

bash$ for A in *.CR2; do rawtran ${A} > ${A%CR2}fits; done

Process all images listed in a file l.lst:

bash$ for A in $(cat l.lst); do rawtran ${A} > ${A%CR2}fits; done

The file l.lst is usually created by command: bash$ ls > l.lst

Produce of multi-color image and show its green channel (band) in ds9:

bash$ rawtran -t 2 IMG_666.CR2 > IMG_666.fits
bash$ ds9 IMG_666.fits[GREEN]

Produce user's white-balanced multi-color image:

bash$ rawtran -t 2 -w 2.0,1.0,1.5 IMG_666.CR2 > IMG_666.fits

Separate blue color (band) from an image:

bash$ rawtran -t 3 -c B IMG_666.CR2 > IMG_666.fits

The output file roughly corresponds to a filtered CCD image with blue band.


Notes

Please, read source code of rawtran.c to get more detailed understanding of my ideas. I know that code is wrongly structured, it uses of poor data structures etc. but it isn't important for me. Valgrind shows defect free code.

I'm little bit confused about handling with output file. The default setup writes data on standard output which can be strangle for unexperinced users. The suffix swaping is not too elegant for C coding (shell replaces strings more effectively) and unique output filename like rawtran.png is confusing too.

All developing is proceed on images produced by Canon 30D. Be careful when your are working with another digital camera.

Download of RAWTRAN.
An older post about RAWTRAN.