NDF format

NDF, the Extensible N-Dimensional Data Format, is a data format for spectra, images, cubes etc. In the first place this is a specification of an HDS object hierarchy. Although Figaro's old data access routines (DSA/DTA) did a reasonably good job at implementing the NDF format, Figaro now uses the official implementation in the form of the NDF subroutine library. So the term `NDF' is not only a recipe for an HDS object tree, it is also the name of a subroutine library to create and access such a tree. And it is customary to call a data set in NDF format `an NDF'.

The minimum content of an NDF is an object with name `DATA_ARRAY'. It must either be a `primitive' array (one that contains numbers etc.), or it must be a structure of type `ARRAY' and contain a primitive array called `DATA'. The two variants are called a primitive NDF and a simple NDF. The following two are equivalent, the first is simple, the second is primitive:

   A_FILE  <NDF>
      DATA_ARRAY     <ARRAY>         {structure}
         DATA(256,256)  <_REAL>         106.2439,126.5268,116.8683,102.3805,
                                     ... 129.4244,134.2537,141.9805,134.2537
   End of Trace.

   A_FILE  <NDF>
      DATA(256,256)  <_REAL>         106.2439,126.5268,116.8683,102.3805,
                                  ... 129.4244,134.2537,141.9805,134.2537
   End of Trace.

The difference can normally be ignored, since the software will sense which variant is present and act accordingly. When you have to specify HDS objects to `hdstrace', `creobj' etc, you must of course check which variant is present. Simple NDFs are more flexible, and are also the standard variant.

There are a number of further components in addition to the data array that an NDF can have. There may be a variance array of the same shape as the data array, or title, label and unit strings. A relatively complete NDF might look like this:

A_FILE  <NDF>

   TITLE          <_CHAR*32>      'krypton singlet'
   UNITS          <_CHAR*32>      'A/D numbers per exposure'
   LABEL          <_CHAR*32>      'OBJECT - DARK'

   DATA_ARRAY     <ARRAY>         {structure}
      DATA(310,19)   <_REAL>         1655.552,1376.111,1385.559,1746.966,
                                     ... 1513.654,1465.343,1446.902,0,0,0,0,0
      BAD_PIXEL      <_LOGICAL>      FALSE

   VARIANCE     <ARRAY>           {structure}
      DATA(310,19)   <_REAL>         87.05064,21.38796,1.088109,14.45454,
                                     ... 286.3985,132.1791,119.0586,0,0,0,0,0

   QUALITY        <QUALITY>       {structure}
      BADBITS        <_UBYTE>        *
      QUALITY(310,19)  <_UBYTE>      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
                                     ... 0,0,0,0,0,0,0,0,0,0,0,0,0,0,*,*,*,*,*

   AXIS(2)        <AXIS>          {array of structures}
   Contents of AXIS(1)
      UNITS          <_CHAR*32>      'microns'
      LABEL          <_CHAR*32>      'Estimated wavelength'
      DATA_ARRAY     <ARRAY>         {structure}
         DATA(310)   <_REAL>            1.85115,1.852412,1.853675,1.854938,
                                        ... 2.237606,2.238869,2.240132,2.241395
   Contents of AXIS(2)
      DATA_ARRAY     <ARRAY>         {structure}
         DATA(19)    <_REAL>         0.5,1.5,2.5,3.5,4.5,5.5,6.5,7.5,8.5,
                                     ... 12.5,13.5,14.5,15.5,16.5,17.5,18.5

End of Trace.

The most remarkable addition, and quite commonly present, is the AXIS. AXIS is an array of structures, even if the data are one-dimensional. Each element AXIS(1), AXIS(2) ... is thus a structure. In fact, it is very similar to a minimal NDF. In the AXIS each pixel is given its own coordinate value. In general AXIS(i).DATA_ARRAY.DATA need neither be linear nor monotonic. However, most software may assume one or the other.

There are a few cases where Figaro treats NDFs differently from other Starlink packages. The details are as follows.

• In Figaro the variance array must not contain bad values. `goodvar' can be used to clean up an offending data set.
• Figaro ignores an origin different from (1,1,...). The origin is the number of the first pixel. It can specify that the first pixel should be counted as pixel 5, or -10, etc. In simple NDFs the origin can be stored, but Figaro will not use such information. As far as it is concerned all data sets begin with pixel (1,1,...). The KAPPA package has an application `setorigin' that allows you to adjust the origin.
• Some Figaro applications may assume that an absent AXIS is equivalent with pixel coordinates 1.0, 2.0, 3.0, ... The default pixel coordinates are in fact ORIGIN-0.5, ORIGIN+0.5, ORIGIN+1.5, ... Sometimes this NDF default is manifested as an actual axis data array, in which case applications may no longer recognise it as a default. For the first and second axes you can use `lxset' and `lyset' to give the data new pixel coordinates.