HDS files

HDS files play a vital role in Figaro. Two data formats, NDF and DST, are realised in HDS files. In addition, the ADAM parameter system uses HDS files to store its information. HDS stands for Hierarchical Data System, and the main feature of HDS files it that there is no fixed order in which items are arranged within the files. Instead HDS files contain a hierarchy of named objects, all that matters are the hierarchy tree and the names of objects, not their sequence.

The actual information is kept in `primitive' objects. These are numbers or strings, scalar or arrays. The other kind of objects are structures, which can contain further objects. These are needed to build the hierarchy tree. Structures can be arrays, too.

Users can inspect and modify the contents and organisation of HDS files. For this a syntax to specify objects has been developed. To practise the syntax, you can use the `hdstrace' utility (see SUN/102) to list the contents of an HDS file:

   ICL> hdstrace a_file full

   A_FILE  <NDF>
      DATA_ARRAY     <ARRAY>         {structure}
         DATA(256,256)  <_REAL>         106.2439,126.5268,116.8683,102.3805,
                                     ... 129.4244,134.2537,141.9805,134.2537
   End of Trace.

This shows that the file contains a top-level structure of type NDF, within it a structure of type ARRAY called `DATA_ARRAY', and within that a primitive array of type _REAL called `DATA'. We also see that this latter array contains 256 by 256 floating point numbers.

You do not have to inspect the whole file, but can concentrate on specific objects within the object tree. For this you use the component names and dots between them. To address array elements, use parentheses:

   ICL> hdstrace @"a_file.sdf".DATA_ARRAY.DATA(2,1)

   A_FILE.DATA_ARRAY.DATA(2,1)  <_REAL>
      DATA         126.5268
   End of Trace.

Here the @ and the double quotes are not necessary, but they show how you can address a DST file instead. The double quotes bind the `.sdf' into the file name so that it does not look like the object `SDF' within the file. The @ makes clear that you talk about an HDS object and not a string or a number. An @ may be useful if you have file names that begin with a digit.

The top-level object is equivalent to the file itself and is not specified. Its name is in fact irrelevant, so long as it is a scalar structure. The other object names are case-insensitive, you can use upper case as in the example, or lower case.

HDS files have names ending with the file name extension `.sdf', which stands for Starlink Data File. But this does not have to be so. Figaro's second data format, DST, also uses HDS files, but their names end in `.dst'. You could use any file name extension, `.sdf' is just the default that is assumed if you don't specify it, such as:

   ICL> hdstrace @"a_file.dst".Z.DATA(2,1)

   A_FILE.Z.DATA(2,1)  <_REAL>
      DATA         126.5268
   End of Trace.

A remark is necessary about data types here. The primitive data types (those that do not make HDS structures, but scalars, vectors and arrays of numbers, strings, etc.) have names beginning with an underscore. The most common type is `_REAL', others are `_DOUBLE', `_INTEGER', `_BYTE', `_UBYTE', and `_CHAR*n'. When an HDS structure manipulator needs to be given a type, then these are to be used. But if a data processing application such as `retype' needs a type specification, then you have to use the traditional Figaro type specifications `FLOAT', `DOUBLE', `INT', `BYTE', and `CHAR'.