Blacknest BDS API Overview

Date	2009-02-18
Version	0.5

Note that this is a "work in progress" document and is being continually updated.

Introduction

This document give a short overview of the BDS API. It covers the overall concepts of the API.
Seismic sensor data is stored and transfered in a number of different formats. In order to make the BDS system as simple and flexible to use as possible the BDS system has a generalised data access API that allows access to any data format.

Seismic Data

The BDS Data API is based on the following common data file model.

Seismic sensor data is split into individual channels each containing a single data stream.
Data files or streams may contain multiple channels of data.
Each channel has a set of meta data associated with it. This defines things like the TimePeriod, Network, Station, Channel, SampleRate, CalibrationFactor etc.
The data for each channel is split into segments. Each segment may be for a different period of time and/or a different network or source.
Each channels segment data is split into blocks. These may be variable or fixed length. Each block has start and end time stamps associated with it.
Each data block may have special meta data information associated with it depending on the original data format.
If there are multiple channels, these may be synchronously sampled or independently sampled.
Data can be multiplexed by sample or by channel.
Raw data has an original sample format of Int16, Int32 or Float32. This can be extended.
The BDS system allows a set of channel data, from the same time period, to be returned as a single set of sample multiplexed data if the channels are synchonously sampled.

Selecting Data

In order to access data from the BDS system, the set of channels required needs to be selected. The BDS system allows for the selection of multiple channels of data from various sources over a given time period. To achieve this the BDS system is passed a Selection object. The Selection object has the following main fields:

Item	Description
startTime	The Start time to the nearest micro-second
endTime	The End time to the nearest micro-second
channels	The List of SelectionChannel Objects

Each SelectionChannel has the following fields:

Item	Description
network	The network the data is from
station	The Array or Station name
channel	The Channel name
source	The data source (Master, Tape, Processed etc)

Any number of SelectionChannel entries can be included. Each attribute can be set to null, which is used as a synonym for any, or a suitable value for selection. Standard regular expression characters, such as the wild card character "*" can be used in any of the fields. The BDS system will expand the given set of SelectionChannel objects to a set defining unique channel data segments in the system.
The data selection scheme will return a DataInfo object describing the data selected. This will contain a number of separate channels of data. Each channel can have multiple segments of data. Segments of data are based on time periods where the data is split into multiple files or where there are multiple sets of data from different data sources (for example Digital and/or Tape).
It is also possible to get the set of channel MetaData for the given channel selection.

The Bds Data Access API

The BDS Data Access API provides a simple set of functions that allow access to the data within the BDS system. The API allows the user to select a set of data channels and then stream them over either using the BDS data block API or in one of the formats that the BDS system has converters for.

The Bds Data Access API has the following core functions:

Function	Description
BError getSelectionInfo(SelectionGroup group, SelectionInfo selectionInfo)	Returns information on all the Networks, Stations, Channels and Sources that the BDS system knows about. Useful for GUI driven data selectors.
BError getSelections(SelectionGroup group, Selection selectionIn, Selection& selectionOut);	Expands the given selection to match the data contents of the BDS system
BError dataSearch(Selection selection, DataInfo& dataInfo);	Searches for data matching the given selection and returns information on the associated data channels.
BError dataGetChannelInfo(DataInfo dataInfo, BList<ChannelInfo> channelInfos);	Returns the channel MetaData in structured form

BError dataOpen(DataInfo dataInfo, String mode, String format, DataHandle& dataHandle);	Opens a data stream. This will open a data stream which will contain all of the channels listed in dataInfo. The mode will be "w" for writing data and "r" for reading data. The format will be one of: API, BDS, BKNAS, IMS1.0 etc. API defines the BdsApi access scheme.
void dataClose(DataHandle dataHandle)	Closes the stream

API Stream Write Interface
BError dataSetInfo(DataHandle dataHandle, DataInfo& dataInfo)	Provides information on the data. This is used to create file headers etc.
BError dataPutBlock(DataHandle dataHandle, DataBlock& data)	Writes a data block to the file. These have to be sequential.

API Stream Read Interface
BError dataGetInfo(DataHandle dataHandle, DataInfo& dataInfo)	Gets information on the data from the files data header and perhaps the data blocks. This will include the real sample rate and actual number of samples.
BError dataGetWarnings(DataHandle dataHandle, BStringList& warnings);	Return a list or warnings.
BError seekBlock(DataHandle dataHandle, BUInt32 channel, UInt32 segment, BTimeStamp time, BUInt32& blockNumber)	Seeks to a particular data block given a time. This can operate on a single channel if a channel number is given or on multiple channels if channel number is 0. The segment parameter is the data segment number.
BError dataGetBlock(DataHandle dataHandle, BUInt32 channel, UInt32 segment, BUInt32 blockNumber, DataBlock& data);	Reads a data block from the file. If the channel number is given, then it reads data from a given channel. If the channel number is 0 it will read a set of data from all of the channels in sample multiplexd form if this is possible. The segment parameter is the data segment number.

Formatted Stream Read Interface
BError dataFormatedRead(DataHandle dataHandle, UInt32 number, Array<UInt8>& data)	Returns the next set of bytes of pre-formatted data. For example BKNAS, IMS etc.
BError dataFormattedGetLength(DataHandle dataHandle, UInt64& length);	The total length in bytes of the formated data.

BDS DataBlock

When reading/writing data from/to a BDS stream there is a single Object called a BDS DataBlock that stores the data. This object has the following attributes:

Atrribute	Description
startTime	The Start time
endTime	The End Time
channelNumber	The channel number. 0 if all channels
segmentNumber	The segment number. 0 if all segments
channelData	Two dimensional array of data. Each channel can have a different number of samples
info	Extra meta data information. This may be available with particular data formats such as the TapeDigitiser

Further Information

BDS MetaData information