BDS Public

Account

Forgotten password

Blacknest BDS API Overview

Date 2008-03-27
Version 0.1

Note that this is a "work in progress" document and is being continually updated.

Introduction

This document provides an overview of the Blacknest Data System (BDS) API.

API Overview

The BDS API's form the heart of the BDS. All data and meta data access will go through the API's. There will be three API's:

  • BDS Data API: This will provide read only access to the data and meta data. It will be used by the AutoDRM email and Web systems as well as for program access to the data.

  • BDS DataAdd API: This will provide read and restricted write access to enable the adding of data to the system. It will not allow deletions of data to be performed. It is designed to be used by manual and automatic data adding programs.

  • BDS Admin API: This will provide full read/write access to the data and meta data as well as administrative configuration information.

The API's will be implemented using an Object Orientated RPC mechanism over a network socket interface. Access to the API's will be secured by a userid/password system and/or a public/private key system. It is expected to use the Beam BOAP or ICE remote object access systems to implement the API.

API Requirements

This gives an overview of the main API requirements for the separate API's.

Data API

This API will have commands for searching and returning information on the seismic data and meta data available as well as actually returning the data and meta-data.
  • The search system will support the following data selection parameters. A blank entry or a simple wildcard can be used in any of the selectors. The possible data selection parameters are listed below:
TimePeriod The time period of the data. (StartTime, EndTime)
Networks[] A list of Network organisations the data/metadata is from
Array/Stations[] The list of Arrays/Stations to get the data from.
Channels[] List of channels required (or all)
Sources[] A list of possible sources of this data. This would allow multiple sources of data from the same Array and Period. It could also support processed data. the source of this data. This would allow multiple sources of data from the same Array and Period. It could also support processed data. (DIRECT, TAPE, PROCESSED ...)
  • Fetch the seismic data and metadata  in a given format (IMS1.0, BKNAS1.0 etc) given the result returned from the above selection.
  • Fetch the data in a structured data format suitable for direct program access.
  • Fetch Meta-Data including information on Instrument information (Calibration, Frequency response, Locations) for the given selection.

DataAdd API

This API will have commands to be able to add data to the system. It will support streaming data as well as simple file additions. It will also be possible to add MetaData through this interface. The core functionality will be:
  • Add a Siesmic data file to the system. The data file can be in any of the formats supported by the system
  • Add a set of structured data fol the system. The structured data is a set of Objects that define the data and possibly Meta data. This is usefull for programming API access.
  • Add a stream of Siesmic data to the system. This will be used for streams such as the satelite data feeds.

Admin API

This API will have commands to manage the system. It will allow data/meta-data editing and deletions. The core functionality will be:
  • User Management
  • Management of Instrument response and other MetaData.
  • Raw SQL database access.

The API

A prototype API, using the BOAP RPC mechanism is documented at the following link:

BOAP or ICE API's

There are a number of possible methods of implementing the BDS API. The main methods investigated included: BOAP, ICE, SOAP and CORBA.
SOAP was discounted as not being Object Orientated, being to complex and slow. CORBA was discounted as being to complex. So we are left with BOAP and ICE. These are quite similar in overall operation. With both systems an Interface Definition Language file is used to define the API. The API interface code is generated from the IDL file.
Some of the features of BOAP include:
  • Simple Object Orientated protocol.
  • Low level of "boiler" plate code needed to implement.
  • Self documenting system using DOxygen that can be linking with program documentation.
  • Easily extensible for different data types and communication protocols.
  • Efficient communication system.
  • Event and broadcast signal support.
  • Supports multiple language bindings: (C++, Python). Would need to be extended to support PHP.
  • We have used it in a number of applications.
  • Name to IPAddress/socket name server support.
  • Not a mainstream API.
Some features of ICE include:
  • Mainstream API.
  • Quite extensive in abilities.
  • Has a self documenting system but this is separate from DOxygen that would be used for main program documentation.
  • Limited data type support (No unsigned etc).
  • Awkward IDL syntax: No Typedefs, strange sequence syntax etc.
  • Difficult UpperCase/LowerCase behavior.
The choice as to which method to use will need some thinking about ....

Notes