This document describes how corrections for CCD artifacts in PAN
Digital Globe WorldView imagery (WV-1 and WV-2) are are tabulated.

See README_MULTISPECTRAL for how the corrections to multispectral
images are computed.

These documents can be used if desired to compute new corrections.
The existing corrections are already incorporated in the wv_correct
program.

Basic idea
----------

For each TDI and scan direction perform several runs. For each run,
average all rows of the disparity (ignore invalid disparities).
Average these over multiple runs. We get n Vector2 values (since each
valid disparity pixel is a Vector2), where n is the image width. From
these, we will extract the locations of CCD jumps and their values in
x and in y. Those will be hard-coded in asp/Tools/wv_correct.cc.

This text is applicable to WV01 and WV02. For WV03 see the text
on top of ccd_solve.cc. 

Detailed instructions
---------------------

1. Choose a satellite (WV1 or WV2; this approach does not work for WV3
which seems fine as it is), a TDI (values are various multiples of 8
from 8 to 64), and a scan direction (forward or reverse). (All this
information can be found in the XML files.)

2. Create a work directory. Have there at least 5-10 image sets for
the desired satellite/TDI/scan direction combination, with reasonably
flat terrain, little or no clouds, and reasonable amount of detail.

3. Build ASP in dev mode. Ensure that the Tools and WVCorrect
directories are in your path.

4. For a given stereo pair, inspect the left and right images in
stereo_gui. Select with the mouse a region wider than the image in
the left image, and of height say 5000 or more pixels.  Its bounds
will be printed on screen. Select a region from the right image as
well, encompassing with a generous margin all terrain already seen in
the left image (say of heieght 12000 pixels). Make the region again
wider than the image, to not leave any image part out either on left
or on right sides.

Run the script run_lr_wrap.sh (see its source for usage). 

Repeat this excercise for other portions of the left image. 

Repeat this again several times when opening the images in reverse
order. This is important in order to get good disparity coverage.

Before doing all this, in each directory have a dem.tif file pointing
to a pre-existing DEM of the area covered by the images.  This is
optional, but it is queried to find the projection to use in point2dem
during the execution of run_lr_wrap.sh. 

5. Repeat this excercise a few more times for other stereo pairs.

7. Inspect the obtained F.tif disparities. Crop away some top and
bottom rows if they are incomplete or have noise. You will have to
redo the step "disp_avg" from run_lr.sh. The result will be "dx.txt"
and "dy.txt" files having averaged disparities.
   
8. Start Matlab. Add your WVCorrect directory to the path, for example
use 

addpath(genpath('/home/USER/projects/StereoPipeline/src/asp/WVCorrect'))

9. In Matlab, create an entry for each run you made, for example, as

dirs={'pair1/run_lr1', 'pair1/run_rl2', 'pair2/run_lr1', 'pair2/run_rl2'};

The following one-liner can be used to generate the above dirs value:

ls -d pair*/run_lr* | perl -pi -e "s#^(.*?)\n#'\$1\', #g" | perl -pi -e "s#^(.*?),\s*\$#dirs=\{\$1\}\n#g"

Initialize the pitches as: 

pitches=zeros(length(dirs), 1) + 8.0e-3;

It is important to verify that the pitches in each XML file are indeed
8.0e-03 (search for <DETPITCH>). Otherwise fill in the correct values
in the 'pitches' array. 

The script gen_scandir.pl can be helpful in parsing XML files
to extract information on pitches, etc.

10. Run

do_find = 1; fig_no = 1; find_ccds(do_find, dirs, pitches, fig_no);

This will save the CCD corrections to files, from there they can be hard-coded
in wv_correct.cc. 

11. It is quite likely the previous step will result in very noisy curves.
Those need to be eliminated, by excluding the noisy runs altogether, or by
cropping F.tif even further and redoing "disp_avg".

Be prepared for a lot of manual iterative work over many days.

12. The script fix_ccds.m can be used to tweak the CCD jumps from the GUI.

13. Recompile wv_correct.cc. Run it to get corrected images. Redo
stereo and point2dem. Examine if the IntersectionError.tif file looks
better than before. This can be done, for example, by again running
run_lr_wrap.sh but this time setting the "corr" flag to 1. 

An "improved" intersection error file will have less noticeable vertical bands
corresponding to ccd artifacts, and its mean, and especially standard deviation, 
should be lower than with the original uncorrecteed images. Some residual
CCD artifacts should be noticeable, and there should be clear jitter present,
which we do not correct for with wv_correct.

wv_correct will only ever be run once per image.  The Matlab find_ccds tool will 
also only be run once per TDI; it can be re-run to get a better set of fixes, but
corrected data should never fed back in to the find_ccds tool.

To improve the correction as seen in IntersectionError.tif, the following steps can
be taken:
- Use more input data to find_ccds.  This may involve tweaking the ROIs or stereo 
  commands so that more image pairs successfully generate dx/dy files.
- Tune the ROIs more carefully so that mountains or poorly matching regions do not
  get included into the dx/dy averages.
- Compare plots of the newly obtained correction amounts with the corrections from
  other TDIs to get an idea of what shift patterns can be expected.
- Hand-tune individual correction amounts by looking at the curves in Matlab.  The
  jump should be equal in height to the peak-to-peak jump seen in the plot.
- Keep in mind that WV1 is not as well behaved as WV2.  The final errors for WV1 
  will be worse.
  

After correcting the images, you can re-run the process on the corrected images and obtain 
dx/dy curves for them as well.  When plotted, you should see the lack of sharp jumps in the
CCD border areas.  Most likely you will still see jumps, but they will be corrected by an
opposite jump nearby.  This is the effect of our applied correction, and the spacing between
the jumps indicates that our correction is misplaced by some number of pixels.  A difference
in the height of the jumps indicates that our correction amount is also off by some amount.  


Other Observations
---------------------

At least some WV images show a consistent 0.25 pixel oscillation in
the X-direction disparity results.  This oscillation is independent of
the CCD border correction and about half the magnitude.  It is not
noticably affected by the subpixel mode used and it is visible on the
intersection error image.

Seen in WV1 and WV2, multiple directions and TDIs.











