This one is under progress.
Recently I was in contact with Martin, he had an heavily optical distorted image and we was not able to align it with the normal affine method. That's because the optical distortion is nonlinear.
I also have a small trace of this problem in my telescope, especially with the 0.75x reducer mounted. I have thought I will address this problem for a long time, but not gone. Now, with the discussion with Martin, interest was raised again. Most camera lenses are rectilinear with a little add of distortion, telescope + corrector could be difference. The distortion if it's big make it impossible to align images with big translations with linear matrixes, mosaics even more problem. If I get rid of the distortion before stacking I can use a simple affine transform to align them. At least when not too wide angle.
For the moment this is not a tutorial, maybe for me because I must learn how to use AstroImageJ and what plugins to use if there are any. But at the end I hope you will find instruction here how to do it.
Below you can see one example how it looks when trying to align an optical distorted image. It's a seven reference point align, in the example I have used a fisheye lens which is not rectilinear and give high distortion.
When doing an align with the affine method you do a linear transform of the images you want to fit each other with translation in x and y axis, scaling in x and y axis and last rotation. You only need three reference points to do this, with a non linear image like the distorted one above and using more then three reference points will be like a mean value to get the best fit. It can not handle nonlinear transformation and that is just that optical distortion are about. In this example I used my 16mm fisheye lens and the images are shifted a big amount.
Common optical distortion are the Barrel distortion and the Pincushion distortion:
Both of them are radius symmetric and is corrected with a second degree function or in more complex distortion a higher degree polynomial, the radius is r = square root (x2 + y2) and the r = 0 at the optical center of the optics.
The free software Hugin:
It transform images in one step when images are stitched together and reasonably also this can be done in Photoshop (which I don't have) and other image software as well. Old honorable astronomy software Iris that I used maybe 15 years ago could do this, but today that program has too big restrictions.
It could be done by warp all sub images to the reference image or by doing a nonlinear transform of all images before align.
I now want to analyze the distortion of the optic in an easy way to get the function needed to transform the image into a linear image. Would also rather do that in AIJ, AstroImageJ so I get a good control over the different steps. After a google, I found this:
It will suffice a 3x3 matrix of images with 50% overlap to get the required parameters. It is not made for astronomical images, it's used to microscopy images, but if it does not work with stars, you can take pictures of something else. Another way to do it are to use images from pro astronomy data bases that are corrected for optical distortion.
Unfortunately, in the field of astronomy, a further distortion is added, which complicates the whole thing. It is the atmosphere that gives a nonlinear distortion that depends on the latitude angle. If you only want to correct for the optics, reference images are taken in direction to the zenith. Has no idea how sensitive it is to the atmosphere, but clearly a problem in the case of wide angle lens images. Requires a correction for each height angle which becomes difficult to handle. However, one might calculate mathematically how the atmosphere affects and apply that transformation on top of the optic distortion transformation. Then you must also know how the camera is oriented relative to the horizon. Now it became quite complex but well worth testing. The tools for AIJ seem to be already there and a 3x3 matrix is easily arranged if we only have clear weather. I think that APT software (camera control) even has a ready-made mosaic feature on the line.