Automatic despiking

At first sight, the automatic despiking of SCUBA data may seem somewhat daunting since there are 4 different tasks provided for this: despike, despike2, scuclip and sigclip. Detailed information on these can be found in the appendix (§) but a direct comparison of the four is provided below:

sigclip

Originally intended for the final clipping of photometry data, this task finds the statistics of the entire data file and clips any point lying more than SIGMA from the mean. This task knows nothing about SCUBA data.

Disadvantages: Should not be used where bolometers see differing signals (i.e. most of the time) since the clipping is then invalid.

Advantages: Will clip any data file. Can be used on reduced photometry data (output of scucat) for clipping since only data for a single bolometer will be present.

scuclip

This task processes each bolometer in turn, finding the mean and removing any points lying more than NSIGMA from the mean for the current bolometer. An iterative clip is used so that the mean is recalculated each time a point lies NSIGMA from the mean until no points are removed. No knowledge of SCUBA is required by this task (except that it knows which quality bit to use for the despiking).

Disadvantages: For JIGGLE/MAPS, on-source bolometers jiggle on and off the source and therefore have a large change in signal (if the source signal is well above the noise level) - the mean and standard deviation calculations therefore have a tendency to remove peak signals from the data. (this can be partly overcome by setting the source bolometer bad, clipping the remaining bolometers and then setting the source bolometer to good.)

Advantages: Can be used for PHOTOM data and weak signals since each bolometers always sees approximately the same signal. Can be used for detecting large spikes on strong sources if a sufficiently large value is chosen for NSIGMA.

despike

This task places each point into a grid cell corresponding to the actual position of the datum on the sky. Each cell is then analysed in turn, any point further than NSIGMA from the mean for a given cell is then treated as a spike¹⁰. All modes are supported with the caveat that SCAN/MAP data should not have been restored (spikes must be removed before the single-beam restoration phase - also EKH data can not strictly be processed in this way beacause the chop angle is not fixed on the sky).

Disadvantages: For small data sets the number of points per bin is not sufficient to perform accurate statistics calculations.

Advantages: Small spikes can be detected in the presence of strong sources since the actual location on the sky is used for the calculation.

despike2

This task is designed specifically for SCAN/MAP data. Each scan for each bolometer is analysed in turn and spikes are detected using a running mean calculation.

Advantages: Finds the large spikes in SCAN/MAP data.

Disadvantages: Care must be taken when despiking bright sources (e.g. planets).

In summary, each mode should probably use different despiking techniques:

photom

scuclip can be used before scuphot and remsky. sigclip should be used after scuphot (or scucat).

scan/maps

despike2 should be used initially. For ``Emerson II'' data it is also possible to use despike since the chop angle is fixed on the sky (only despike data that were taken with the same chop configuration).

Jiggle maps of strong sources

Initially scuclip can be used with a large NSIGMA to remove the obvious spikes. Then despike should be used for the smaller spikes (i.e. those comparable with the source signal).

Jiggle maps of weak sources

Can probably run scuclip as for PHOTOM observations. Here `weak source' means data where the source is not far above the noise level.