libnexus0-dev 4.2.1-svn1614-1 / usr / share / nexus / doc / nxdict / NXdict.tex

%  Copyleft (c) 1997 by Mark Koennecke at PSI, Switzerland.
%
%
%  This software is distributed in the hope that it will be useful,
%  but WITHOUT ANY WARRANTY; without even the implied warranty of
%  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%  GNU General Public License for more details.
%
%  You may already have a copy of the GNU General Public License; if
%  not, write to the Free Software Foundation, Inc., 675 Mass Ave,
%  Cambridge, MA 02139, USA.
%

\documentclass[12pt]{article}

\setlength{\oddsidemargin}{-.1in}
\setlength{\evensidemargin}{0in}
\setlength{\topmargin}{0in}
\addtolength{\topmargin}{-\headheight}
\addtolength{\topmargin}{-\headsep}
\setlength{\textheight}{8.9in}
\setlength{\textwidth}{6.2in}
\setlength{\marginparwidth}{0.5in}

\begin{document}
\title{The NeXus Dictionary API}

\author{Mark K\"onnecke\\
  Labor f\"ur Neutronenstreuung\\
  Paul Scherrer Institut\\
  CH-5232 Villigen PSI\\
  Switzerland\\       
  Mark.Koennecke@psi.ch \\
}



\maketitle

\vskip.3in
\centerline{\large\bf Abstract}
\vskip.2in
\begin{center}
\parbox{.8\textwidth}{
  There is a proposed portable data exchange format for neutron and
  X-ray scattering communities, NeXus (described in a separate
  publication).   Another document describes an application programmers
 interface to NeXus. This is a base level API which hides many of the
 hideous details of the HDF interface from the NeXus programmer. The
 present document   introduces a higher level application programmers
 interface sitting on top of the NeXus API. This API (the NEXDICT-API),
 reads all file structure interface from a dictionary data file and creates
 the structure automatically from that information. The NEXDICT user only
 needs to specify the data to write.
}
\end{center}

\clearpage

\section{Introduction}
 There exists a proposal for a portable data exchange format for neutron and
 X--ray scattering communities, NeXus. NeXus is fully described
 elsewhere$^{1}$. NeXus sits on top of the hierarchical data format (HDF) as
 defined and specified by the National Center for Supercomputer Applications, 
 NCSA, USA. HDF comes with a library of access functions. On top of the
 HDF-library an application programmers interface (API) for NeXus was
 defined which hides many of the low level details and idiosyncracies of
 the HDF interface form the NeXus programmer. However, writing NeXus files stays
 hideous even with this interface due to the amount of repetitive code
 required to implement the NeXus structure. Now, performing repetitive tasks is one
 area a computer is good at. So, why not have the computer take care of all
 the structure associated with the NeXus format? In order to do this two
 components are needed:
\begin{itemize}
\item A language which describes the NeXus file structure to the computer.
  This language will be called the NeXus Data Definition Language (NXDDL).
  NXDDL might also be used as a tool for discussing and defining NeXus
  data structures.
\item An API which works with the NeXus Data Definition Language.
\end{itemize}
Both of the above will be detailed in this document.

\section{The NeXus Data Definition Language}
The NeXus Data Definition Language(NXDDL) has the purpose of defining the structure
and data items in a NeXus file in a form which can be understood by a human
programmer and which can be parsed by the computer in order to create the 
structure. 
For this, a dictionary--based approach will be used. This dictionary
will contain pairs of short aliases for data items and definition strings 
which hold the structure information. This dictionary will
be initialized from a data file, the NXDDL-file. Such a dictionary can be
used in the following way: given an appropriate API function, a NXDICT
programmer  needs to specify only the alias and the data to write and 
everything else is taken care of by the API: Vgroup creation, opening,
SDS definition etc. Another use may involve the creation of a definition string
completely or partly at run time which can then be used by an API function
in order to create the structures defined by the definition string. The same
holds for writing as well.


A NXDDL dictionary is preferably initialized from a file.
Such a NXDDL file has to follow these general structure guidelines:
\begin{itemize}
\item All input is in US--ASCII.
\item A \verb+#+ in the first column denotes a comment and will be ignored.
\item A \verb+\+ at the end of the line means that the current text will be 
 continued with the next non-blank character for the next line.
\item All other entries follow the form: alias = definition string.
 This defines \verb+alias+ as a short form for the definition string  after the
 equality sign.
\end{itemize}         
The addition of a special global Vgroup of class NXdict to the
NeXus API, in order to hold the dictionary information within a NeXus file, might be considered.

 The next thing to define is the content of the definition string. A
 definition string will have the general form: \\
\centerline{\bf PATH/TerminalSymbol}
 This means a definition string will consist of a path specifier which
 describes the position of a data item in the Vgroup hierarchy and a
 terminal symbol which describes the nature of the data item. 

 The path through the Vgroup hierarchy to a data item will be described in a
 manner analog to a Unix directory hierarchy. However, NeXus requires two
 pieces of data in order to fully qualify a Vgroup. This is it's name and
 class. Consequently, both name and class name will be given for each Vgroup,
 separated by a comma. A valid path string then looks like: \\
\begin{verbatim} 
     /scan1,NXentry/DMC,NXinstrument/big_detector,NXdetector/TerminalSymbol
\end{verbatim}
 This translates into: TerminalSymbol in Vgroup big\_detector, class
 NXdetector, which resides in Vgroup DMC of class NXinstrument, which in
 turn is situated in the Vgroup scan1 of class NXentry.

 The terminal symbol in a definition string is used to define the data item
 at the end of the definition. NeXus currently supports only three types of
 data items at the end of the chain: these are scientific data sets (SDS),
 Vgroups and links to other data items or Vgroups. The terminal symbol for a link
 is specified by the keyword \verb+NXLINK+  
 followed
 by a valid alias of another data item or Vgroup. For example the terminal 
 symbol: \\
 \centerline{\bf NXLINK counts}
 would define a link to the data item which is defined by the alias
 \verb+counts+.

 A Vgroup would be denoted by the keyword VGROUP. By then, the Vgroup has
 already been defined by the path string. This form of alias is only useful
 for the definition of links to Vgroups.

 A SDS is more involved. The definition of an SDS starts with the keyword
 \verb+SDS+. This keyword is followed by option value pairs which define the 
  details of the SDS. The following options exist:
 \begin{itemize}
  \item {\bf -rank} defines the rank of the SDS.
  \item {\bf -dim \{dim0,dim1,dim2,...., dimn\} } defines the dimensions of the
       SDS. Exactly the number of rank numbers defining the dimensions
 length is required inside the curly braces. 
  \item {\bf -type} defines the data type of the SDS as a string corresponding
  to the HDF data types.
  \item {\bf -name} Specifies the name of the SDS.
  \item {\bf -attr \{name,value\} } defines an attribute. In the curly braces
   there must be the name and value of the attribute separated by a comma.
  \end{itemize}
  If no options are given a default is used. This will be a single floating
  point number, as this is the most frequently written data item. As an 
  example see the definition of a 3d array of 32 bit integers:
  \begin{verbatim}
   PATHSTRING/SDS -name counts -rank 3 -dim {64,64,712} -type DFNT_INT32 \
                  -attr {Units,Counts}      

  \end{verbatim}

  \section{The NXDICT--API}
  In order to interface with the NeXus dictionary API a set of
  API--functions is needed. All functions and data types belonging to
  this API start with the letters: NXD. The functions belonging to this API
  fall into three groups:
  \begin{itemize}
   \item Dictionary maintainance functions.
   \item Data writing and reading functions.
   \item Utility functions.
  \end{itemize}
  
  One additional data type is needed for this API:
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap1}
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@   typedef void *NXDdict;@\\
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\end{minipage}\\[4ex]
\end{flushleft}
NXDdict will be used as a handle for the dictionary currently in use.

\subsubsection{Dictionary Maintainance Function}
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap2}
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@   NXDdict NXDinitfromfile(char *filename);@\\
\mbox{}\verb@   NXstatus NXDclose(NXDdict handle, char *filename);@\\
\mbox{}\verb@@\\
\mbox{}\verb@   NXstatus NXDadd(NXDdict handle, char *alias, char *DefString);@\\
\mbox{}\verb@   NXstatus NXDget(NXDdict handle, char *alias, char *pBuffer, int iBufLen);@\\
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\end{minipage}\\[4ex]
\end{flushleft}
{\bf NXDinitfromfile} creates a new NeXus dictionary. If filename is NULL, this
  is all that happens. If filename is not NULL, it will be opened and the
  dictionary will be initialized from the file specified.  The return value
  is either 0 for failure or non zero for success. 

  {\bf NXDclose} deletes and writes a NeXus dictionary. If filename is not NULL,
  the dictionary specified by handle is written to the file specified by
  filename. In any case the dictionary specified by handle will be deleted.

  {\bf NXDadd} adds a new alias -- Definition String pair to the dictionary
  specified by handle.

  {\bf NXget} retrieves the definition string for the alias specified as
  the second parameter from the dictionary handle. The definition string
  is copied to pBuffer. Maximum iBufLen characters will be copied.
  
  If a special dictionary Vgroup as extension to NeXus would be accepted,
  two more functions need to be defined which read and write the dictionary 
  from the NeXus file.

\subsubsection{Data Handling functions}
\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap3}
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@   NXstatus NXDputalias(NXhandle file, NXDdict dict, @\\
\mbox{}\verb@                        char *alias, void *pData);@\\
\mbox{}\verb@   NXstatus NXDputdef(NXhandle file, char *pDefString, void *pData);@\\
\mbox{}\verb@@\\
\mbox{}\verb@@\\
\mbox{}\verb@   NXstatus NXDopenalias(NXhandle file, NXDdict dict, @\\
\mbox{}\verb@                        char *alias, void *pData);@\\
\mbox{}\verb@   NXstatus NXDopendef(NXhandle file, char *pDefString, void *pData);@\\
\mbox{}\verb@@\\
\mbox{}\verb@   NXstatus NXDgetalias(NXhandle file, NXDdict dict, @\\
\mbox{}\verb@                        char *alias, void *pData);@\\
\mbox{}\verb@   NXstatus NXDgetdef(NXhandle file, char *pDefString, void *pData);@\\
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\end{minipage}\\[4ex]
\end{flushleft}
The NXDICT data handling functions go in pairs. The version ending in
 alias expects an NXDdict and an alias as input. These routines work
 out the pass from that. The other version ending on def acts upon 
 a definition string specified as second parameter. Using this scheme
 both full dictionary operation is possible, as well as operation with
 program generated definition strings. All routines return the
 usual NeXus status returns. All these routines start at the current Vgroup
 level and return back to it.  

 NXDputalias, NXDputdef write the data element specified by the alias or
 the definition string to the NeXus file specified as first parameter. 
 pData is a pointer to the data to be written. These routines will check for
 the existence of all Vgroups required in the path part of the definition
 string. If a Vgroup  is missing it will be created. 

 NXDopenalias, NXDopendef open the specified data items specified by the
 alias or the definition string. Then the usual NeXus functions can be 
 used to interact with the data. These routines use the same scheme for
 creating Vgroups on the fly as the put routines above.

 NXDgetalias, NXDgetdef read a data item from file. pData MUST point to a
 data area large enough to hold the data read. If a Vgroup is missing in
 the path for one of these routines an error is generated because it is 
 assumed that the data is present if a program wants to read it.  
  
  \subsection{NXCDICT Utility Functions}
  This section list a couple of functions which either perform common 
   tasks on NeXus files or relate
  to aspects of error handling and debugging.

\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap4}
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@   NXstatus NXDwriteglobals(NXhandle file, @\\
\mbox{}\verb@                            char *filename,@\\
\mbox{}\verb@                            char *owner,@\\
\mbox{}\verb@                            char *address,@\\
\mbox{}\verb@                            char *phone,@\\
\mbox{}\verb@                            char *e-mail,@\\
\mbox{}\verb@                            char *place);@\\
\mbox{}\verb@@\\
\mbox{}\verb@   NXstatus NXDmakeentry(NXhandle file, char *pName);@\\
\mbox{}\verb@                            @\\
\mbox{}\verb@   NXstatus NXDsetverbosity(int iLevel);   @\\
\end{list}
\vspace{-1ex}
\footnotesize\addtolength{\baselineskip}{-1ex}
\end{minipage}\\[4ex]
\end{flushleft}
{\bf NXDwriteglobals} writes the global attributes to a newly opened 
 NeXus file. The parameters should be self explaining. In addition 
 the file creation date is automatically written.

 {\bf NXDmakeentry} creates a new entry Vgroup at root level with the
  name specified. The group creation date is automatically written.

  {\bf NXDsetverbosity} sets a verbosity level. Currently only two are
  supported: NXDquiet and NXDalot. If NXDquiet is set (the default) the
  NXD routines perform their tasks quietly, only complaining about
  fatal error conditions. If NXDalot is set, the NXD routines will comment
  on any step they do. This facility will help to debug dictionary files
  and the software. The output happens through the same scheme which
  is used for NeXus error handling, i.e. a configurable error output
  function. 

\begin{flushleft} \small
\begin{minipage}{\linewidth} \label{scrap5}
\vspace{-1ex}
\begin{list}{}{} \item
\mbox{}\verb@@\\
\mbox{}\verb@/*---------------------------------------------------------------------------@\\
\mbox{}\verb@                            NXDICT API header file@\\
\mbox{}\verb@@\\
\mbox{}\verb@   copyleft: Mark Koennecke, March 1997 at LNS,PSI, Switzerland@\\
\mbox{}\verb@@\\
\mbox{}\verb@   No warranties of any kind taken.@\\
\mbox{}\verb@----------------------------------------------------------------------------*/@\\
\mbox{}\verb@#ifndef NXDICTAPI@\\
\mbox{}\verb@#define NXDICTAPI@\\
\mbox{}\verb@#include "napi.h" /* make sure, napi is included */@\\
\mbox{}\verb@@\\
\mbox{}\verb@/*-------------------- NXDict data types & defines ----------------------*/@\\
\mbox{}\verb@#define NXquiet 0@\\
\mbox{}\verb@#define NXalot  1@\\
\mbox{}\verb@/*-------------------- Dictionary Maintainance ----------------------------*/@\\
\mbox{}\verb@/*----------------- Dictionary added data transfer -----------------------*/ @\\
\mbox{}\verb@/*-------------------- Utility Functions --------------------------------*/@\\
\mbox{}\verb@#endif@\\
\end{list}
\vspace{-2ex}
\end{minipage}\\[4ex]
\end{flushleft}
\end{document}