BdsApi

Author

Dr Terry Barnaby

Version

3.1.0

Date

2022-11-21

Introduction

This documention provides detailed reference information for the BEAM BdsApi software API of the Blacknest Data System (BDS). The API provides the ability to store and access seismic sensor data and metadata as well as administer the BDS system. The API is an object orientated API implemented in 'C++' with a number of object classes. It also has bindings for other languages which include Python and PHP.

The API operates over a network interface using an RPC type mechanism implemented by BEAM's BOAP RPC system. The BdsApi API makes use of the BEAM Beamlib 'C++' class library for lower level and system independent functionality. The BEAM Beamlib 'C++' class library provides a small set of low level 'C++' classes for strings, lists and system interface functions. It also implements the BOAP RPC mechanism used to implement the BdsApi. There is some brief information on the BEAM class library later on in this page and the API description is available in the Beamlib documentation.

The BDS Python API is built on top of the standard BDS 'C++' API using the SWIG API generator. Thus all of the standard BDS C++ API documentation applies however there are some differences due to the language facility and syntax differences. The core difference is when returning data from functions. With C++ you can return data by passing references or pointers to objects. In Python this is not generally possible and so objects are returned at the left hand side of functions instead.

This is the reference documentation for the BdsApi. An overall API description and programming manual is provided separately in: BdsDevelopment.pdf

Overview

The BdsApi has been developed using the BOAP (BEAM Object Access Protocol). This provides a simple but powerful Object Orientated RPC mechanism using an efficient binary protocol. The BdsApi is written in a high level interface definition language (IDL). The Beamlib bidl tool generates the client and server side 'C++' interface and implementation files for the API. These are then provided as a set of 'C++' header files and a binary library file for the clients and server programs to link to. The BOAP system employs a simple BOAP name server process that provides a translation between object names and network IPAddress/Socket numbers. The BOAP name server runs on the main BDS Server host. More information on the BOAP system can be found in the Beamlib documentation.

The object orientated BDS API implements a number of data storage classes and three BdsServer interface objects. The interface objects are:

1. Bds::DataAccess BDS Data API: This provides read only access to the data and meta data. It is used by the AutoDRM email and Web systems as well as for user and general program access to the data.
2. Bds::DataAddAccess BDS DataAdd API: This provides read and restricted write access to enable the adding of data to the system. It will not allow deletions of data to be performed. It is designed to be used by manual and automatic data adding programs.
3. Bds::AdminAccess BDS Admin API: This provides full read/write access to the data and meta data as well as administrative configuration information.

These access API's are related in that the DataAddAccess API is a subset of the AdminAccess API and the DataAccess API is a subset of the DataAddAccess API. These API access objects should be consulted to view the functionality provided by the BDS system API's.

This documentation is automatically geneerated from the BDS source code and describes the classes and their member functions. For higher level information on how to use these please consult the BdsDevelopment.pdf document..

C++ Examples

There are some examples of client applications using the BdsApi in the bdsExamples directory of the source code. Some simple Data Access client examples are listed below:

/*******************************************************************************
 *  BdsDataClient1.cpp  BDS API example code for a Data Client
 *              T.Barnaby,  BEAM Ltd,   2008-09-02
 *******************************************************************************
 *
 *  This is a very basic example of using the BdsApi from a data access
 *  perspective. It is designed to give an overview of using the API.
 *  This program gets data in the BKNAS format.
 */
#include <iostream>
#include <stdio.h>
#include <BdsD.h>
#include <BdsC.h>
 
using namespace Bds;
using namespace std;
 
 
// Function to read some data
BError bdsTest(DataAccess& bds){
    BError          err;
    Selection       selection;
    DataInfo        dataInfo;
    DataHandle      dataHandle;
    BArray<BUInt8>      data;
    
    // Set up selection
#ifndef ZAP
    selection.startTime.setString("2002-01-01T00:00:00.000000");
    selection.endTime.setString("2002-01-01T00:01:00.000000");
#else
    selection.startTime.setString("2002-01-01T23:59:00.000000");
    selection.endTime.setString("2002-01-02T00:01:00.000000");
#endif
    selection.channels.append(SelectionChannel("BN", "EKA", "", ""));
 
    // Get list of all data available for the selection     
    if(err = bds.dataSearch(selection, dataInfo)){
        return err.set(1, BString("Error: Searching for data: ") + err.getString());
    }
 
    // We should now choose which set of data we would like from the list, here we just
    // choose the first entry and get the data in appropriate format.
    if(!dataInfo.channels.size())
        return err.set(1, "No data found");
    
    if(err = bds.dataOpen(dataInfo, "r", "IMS", 0, dataHandle)){
        return err;
    }
    
    while(1){
        if(err = bds.dataFormattedRead(dataHandle, 1024, data)){
            return err;
        }
 
        if(data.size() == 0)
            break;
        
        fwrite(data.data(), 1, data.size(), stdout);
    }
    
    return err;
}
 
int main(int argc, char** argv){
    BError          err;
    BString         hostName;
    DataAccess      bds;
 
    hostName = getenv("BDS_HOST");
    if(hostName == "")
        hostName = "localhost";
    
    if(argc == 2)
        hostName = argv[1];
 
    // Connect to the DataAccess service
    if(err = bds.connectService(BString("//") + hostName + "/bdsDataAccess")){
        cerr << "Error: " << err.getString() << "\n";
        return 1;
    }
 
    // Connect to service
    if(!err)
        err = bds.connect("test", "beam00");
    
    // Run a normal data gathering as a normal data access client would.
    if(!err)
        err = bdsTest(bds);
    
    if(err){    
        cerr << "Error: " << err.getString() << "\n";
        return 1;
    }
        
    return 0;
}

/*******************************************************************************
 *  BdsDataClient2.cpp  BDS API example code for a Data Client
 *              T.Barnaby,  BEAM Ltd,   2008-09-02
 *******************************************************************************
 *
 *  This is a very basic example of using the BdsApi from a data access
 *  perspective. It is designed to give an overview of using the API.
 *  This program gets data in raw format and outputs it in ASCII.
 */
#include <iostream>
#include <iomanip>
#include <stdio.h>
#include <BdsD.h>
#include <BdsC.h>
 
using namespace Bds;
using namespace std;
 
 
// Function to read some data
BError bdsTest(DataAccess& bds){
    BError          err;
    Selection       selection;
    DataInfo        dataInfo;
    DataHandle      dataHandle;
    DataBlock       data;
    BUInt32         blockNumber = 0;
    BUInt           c;
    BUInt           s;
    
    // Set up selection
    selection.startTime.setString("2008-01-03T00:00:00.000000");
    selection.endTime.setString("2008-01-03T00:01:00.000000");
    selection.channels.append(SelectionChannel("BN", "EKA", "", ""));
 
    // Get list of all data available for the selection     
    if(err = bds.dataSearch(selection, dataInfo)){
        return err.set(1, BString("Error: Searching for data: ") + err.getString());
    }
 
    // We should now choose which set of data we would like from the list, here we just
    // choose the first entry and get the data in appropriate format.
    if(!dataInfo.channels.size())
        return err.set(1, "No data found");
    
    if(err = bds.dataOpen(dataInfo, "r", "API", 0, dataHandle)){
        return err;
    }
    
    while(1){
        if(err = bds.dataGetBlock(dataHandle, 0, 0, blockNumber, data)){
            return err;
        }
        
        if(data.startTime >= dataInfo.endTime)
            break;
 
        for(s = 0; s < data.channelData[0].size(); s++){
            for(c = 0; c < data.channelData.size(); c++){
                if(c != 0)
                    std::cout << ", ";
                std::cout << setw(8) << data.channelData[c][s];
            }
            std::cout << "\n";
        }
        blockNumber++;
    }
    
    return err;
}
 
int main(int argc, char** argv){
    BError          err;
    BString         hostName;
    DataAccess      bds;
 
    hostName = getenv("BDS_HOST");
    if(hostName == "")
        hostName = "localhost";
    
    if(argc == 2)
        hostName = argv[1];
 
    // Connect to the DataAccess service
    if(err = bds.connectService(BString("//") + hostName + "/bdsDataAccess")){
        cerr << "Error: " << err.getString() << "\n";
        return 1;
    }
    
    // Connect to service
    if(!err)
        err = bds.connect("test", "beam00");
    
    // Run a normal data gathering as a normal data access client would.
    if(!err)
        err = bdsTest(bds);
    
    if(err){
        cerr << "Error: " << err.getString() << "\n";
        return 1;
    }
        
    return 0;
}

/*******************************************************************************
 *  BdsMetaData1.cpp    BDS API example code for a Meta Data Client
 *              T.Barnaby,  BEAM Ltd,   2009-07-01
 *******************************************************************************
 *
 *  This is a very basic example of using the BdsApi from a meta data access
 *  perspective. It is designed to give an overview of using the API.
 *  This program gets information on the data channels.
 */
#include <iostream>
#include <iomanip>
#include <stdio.h>
#include <BdsD.h>
#include <BdsC.h>
 
using namespace Bds;
using namespace std;
 
 
// Function to read some data
BError bdsTest1(DataAccess& bds){
    BError          err;
    Selection       selection;
    BIter           i;
    BUInt           n;
    BList<Station>      stations;
    
    // Set up selection
    selection.startTime.setString("2008-01-03T00:00:00.000000");
    selection.endTime.setString("2008-01-03T00:01:00.000000");
    selection.channels.append(SelectionChannel("BN", "EKA", "", ""));
 
    // Get list of stations available
    if(err = bds.stationGetList(selection, stations)){
        return err.set(1, BString("Error: Getting stations: ") + err.getString());
    }
 
    // This displays some of the information available  
    for(stations.start(i), n = 0; !stations.isEnd(i); stations.next(i), n++){
        Station&    c = stations[i];
        
        cout << n << ": Station: " << c.name << " Type: " << c.type << "\n";
        cout << "    " << "Description: " << c.description
            << " Number of stations " << c.channels.number() << "\n";
    }
    
    
    return err;
}
 
int main(int argc, char** argv){
    BError          err;
    BString         hostName;
    DataAccess      bds;
 
    hostName = getenv("BDS_HOST");
    if(hostName == "")
        hostName = "localhost";
    
    if(argc == 2)
        hostName = argv[1];
 
    // Connect to the DataAccess service
    if(err = bds.connectService(BString("//") + hostName + "/bdsDataAccess")){
        cerr << "Error: " << err.getString() << "\n";
        return 1;
    }
    
    // Connect to service
    if(!err)
        err = bds.connect("test", "beam00");
    
    // Run a normal data gathering as a normal data access client would.
    if(!err)
        err = bdsTest1(bds);
    
    if(err){
        cerr << "Error: " << err.getString() << "\n";
        return 1;
    }
        
    return 0;
}

Python Examples

There are some examples of client applications using the BdsApi in the bdsExamples directory of the source code. Some simple Data Access client examples are listed below:

#!/usr/bin/python
 
import sys
import getopt
from bdslib import *
 
# Function to read some data
def bdsTest(bds):
    err     = BError();
    selection   = Selection();
    
    # Set up selection
    selection.startTime.setString("2008-01-01T00:00:00.000000");
    selection.endTime.setString("2008-01-01T00:01:00.000000");
    print("Selection: StartTime:", selection.startTime.getString(), "EndTime:", selection.endTime.getString());
    
    selection.channels.append(SelectionChannel("TT", "TSB01", "", ""));
    # bdsDumpSelection(selection);
 
    # Get list of all data available for the selection      
    (err, dataInfo) = bds.dataSearch(selection);
    if(err):
        return err.set(1, BString("Error: Searching for data: ") + err.getString());
 
    # We should now choose which set of data we would like from the list, here we just
    # choose all of the data.
    s = dataInfo.channels.size();
    if(s == 0):
        return err.set(1, "No data found");
    
    # bdsDumpDataInfo(dataInfo);
    
    # Open the data file for reading in IMS format
    (err, dataHandle) = bds.dataOpen(dataInfo, "r", "IMS", 0);
    if(err):
        return err;
    
    # Read the formatted data
    while(1):
        # print "Loop";
        (err, data) = bds.dataFormattedRead(dataHandle, 1024);
        if(err):
            return err;
 
        if(data.number() == 0):
            break;
 
        s = "".join(chr(x) for x in data);
        print(s);
 
    return err;
 
def main():
    hostName = "localhost";
    bds = DataAccess();
 
    # Connect to the DataAccess service
    err = bds.connectService("//" + hostName + "/bdsDataAccess");
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
 
    # Connect to service
    err = bds.connect("test", "beam00");
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
    
    (err, version, name) = bds.getVersion();
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
    print("Version:" , version, "Name:", name);
 
    err = bdsTest(bds);
    if(err):
        print("Error:", err.getErrorNo(), err.getString());
        return 1;
        
    return 0;
 
if __name__ == "__main__":
    main();

#!/usr/bin/python
 
import sys
import getopt
from bdslib import *
 
# Function to read some data
def bdsTest(bds):
    err = BError();
    selection = Selection();
    dataInfo = DataInfo();
    
    # Set up selection
    selection.startTime.setString("2008-01-01T00:00:00.000000");
    selection.endTime.setString("2008-01-01T00:01:00.000000");
    selection.channels.append(SelectionChannel("TT", "TSB01", "", ""));
    # bdsDumpSelection(selection);
 
    # Get list of all data available for the selection      
    (err, dataInfo) = bds.dataSearch(selection);
    if(err):
        return err.set(1, BString("Error: Searching for data: ") + err.getString());
 
    # We should now choose which set of data we would like from the list, here we just
    # choose the first entry and get the data in appropriate format.
    s = dataInfo.channels.size();
    if(s == 0):
        return err.set(1, "No data found");
    
    # bdsDumpDataInfo(dataInfo);
    
    (err, dataHandle) = bds.dataOpen(dataInfo, "r", "API", 0);
    if(err):
        return err;
    
    blockNumber = 0;
    while(1):
        # print "Loop";
        (err, data) = bds.dataGetBlock(dataHandle, 0, 1, blockNumber);
        if(err):
            return err;
 
        # print("DataChannels:", data.channelData.size());
        
        print("Data0:", data.channelData[0][0]);
        blockNumber += 1;
 
    return err;
 
def main():
    hostName = "localhost";
    bds = DataAccess();
 
    # Connect to the DataAccess service
    err = bds.connectService("//" + hostName + "/bdsDataAccess");
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
 
    # Connect to service
    err = bds.connect("test", "beam00");
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
    
    (err, version, name) = bds.getVersion();
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
    print("Version:" , version, "Name:", name);
 
    err = bdsTest(bds);
    if(err):
        print("Error:", err.getErrorNo(), err.getString());
        return 1;
        
    return 0;
 
if __name__ == "__main__":
    main();

#!/usr/bin/python
 
import sys
import getopt
from bdslib import *
 
# Function to read display info on Station
def bdsTest(bds):
    err = BError();
    selection = Selection();
    
    # Set up selection
    selection.startTime.setString("2008-01-01T00:00:00.000000");
    selection.endTime.setString("2008-01-01T00:01:00.000000");
    selection.channels.append(SelectionChannel("TT", "TSA", "", ""));
#   bdsDumpSelection(selection);
 
    # Get list of stations available
    (err, stations) = bds.stationGetList(selection);
    if(err):
        return err.set(1, "Error: Getting stations: " + err.getString());
 
    # This displays some of the information available
    for s in stations:
        print("Station: " + s.name + " Type: " + s.type);
        print("    " + "Description: " + s.description + " Number of station/channels " + str(s.channels.number()));
 
    return err;
 
def main():
    hostName = "localhost";
    
    # Create DataAccess object to connect to BDS Server
    bds = DataAccess();
 
    # Connect to the DataAccess service
    err = bds.connectService("//" + hostName + "/bdsDataAccess");
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
 
    # Connect to service
    err = bds.connect("test", "beam00");
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
    
    (err, version, name) = bds.getVersion();
    if(err):
        print("Error: " + str(err) + "\n");
        return 1;
    print("Version:" , version, "Name:", name);
 
    err = bdsTest(bds);
    if(err):
        print("Error:", err.getErrorNo(), err.getString());
        return 1;
        
    return 0;
 
if __name__ == "__main__":
    main();