BDS Public

BDS Release Notes

Release 4.0.1 – 2024-11-11

This adds some small requested changes to the system.

1. 2646: Output from bdsDataAccess metadataGetFormated format ASCII - "source" values. I have modified the bdsDataAccess program to convert any input of ".*" in the selection criteria to a "" (blank). The "" (blank) in a BDS selection is in effect a Wildcard for all items. This fixes the file name creation if a user uses ".*" although really they should simple use a blank. On exporting multiple sources of Metadata I don't think the BDS should do this at the moment. I don't think this makes sense for the way SOURCE is intended and for the Seismic data formats supported. The idea of SOURCE is to handle specific cases where there are multiple sources of the same metadata and sensordata. The main one should always be called Main and that will be the one output if a blank is used. With SOURCE there is no wildcard ability, a "" (blank) equates to "Main". I have added some info on this this to the bdsDataAccess manual. Note that for the filename used when exporting daa using bdsDataAcces, you can use the "-o <fileName>" option to provide the filename you would like to use instead of the generic internally generated one.

2. 2646: Output from bdsDataAccess metadataGetFormated format ASCII - "source" values. Response stage numbering. Some BDS Metadata has response stage numbering with gaps in an integer sequence. It is not known if the stange numbers are important, but there looks to be a bug in ObsPy which throws and error when there are gaps in the response stage numbering. We have changed the bdsExportFix() code to renumber the response stages when exporting to most formats. Note that the BDS Metadata keeps the original imported stage numbering and the API will have this original stage numbering.

3. 2721: make PoleZero response file format identification not case sensitive. Now both the DataFileSac and the legacy DataFileResponse data converters support upper and lower case names in the SAC format response files.

4. 2266: Repeated items and lack of gainFrequency in StationXML file. We have fixed the issue with Metadata export when the source is not specified. The system used to merge the responses from multiple sources if these existed and even the calibration could have selected the wrong source. The system now explicitly looks for Metadata with a source of "Main" if the source has not been specified. Also fixed the issue that the source was only used for the first channel if more than one channel is selected. These fixes are in the Control::metadataGetChannelInfo() function. Note that as the system now only looks for Metadata with the source Main, it could break some Metadata that has been imported without a source of "Main".

5. 2484: Bdsrestart: Add wait for bdsServer. Added a new bdsServerPing program to check if BdsServer is running and this is now used in the bdsRestart script.

6. 2851: bdsDataManage to delete a file specified by ID - ?. This has been implemented. There is now a -dataFileId option that can be used in place of the -select, -startTime and -endTime options to select a data file for listing or deletion.

7. Note that the beamlib-boapns systemd service system has changed a little. You should make sure that the very old boapns is disabled with “systemctl disable boapns” and the new beamlib-boapns is started with “systemctl enable –now beamlib-boapns” if you have issues.

The BDS API is unchanged at version 29.

The database schema is unchanged at version 21.

Release 4.0.0 – 2024-09-06

This is a major new release with API and database schema changes. Currently it is a development only release for Blacknest testing.

1. 2532: Remove arrayOffsetNoth, arrayOffsetEast from Location storage and GUI's. Implemented.

2. 2535: Add getApiVersion() function. Implemented via Beamlibs ping() call.

3. 2545: Event add shot above/below the surface. Implemented.

4. 2530: Network class: Remove stations list. Implemented. Note this drops the NetworkStations database table.

5. 2533: DataInfo stream field. Implemented with new DataChannel member importStream.

6. 2531: Modify metadataGetChannelInfo to allow getting information even if not all of it is present. This has now been implemented. The API has changed and all code using metadataGetChannelInfo() or metadataGetFormatted() will need to be changed. The API is now:

• BError metadataGetChannelInfo(Selection sel, Bool allowIncomplete, ChannelInfos channelInfos);

• BError metadataGetFormatted(Selection sel, Bool allowIncomplete, String format, Array<UInt8> data);

The parameter allowIncomplete if set to 1 allows these functions to return as much Metadata they can without a full error. If there is actually incomplete Metadata they will return an ErrorWarning which can be igored if you wish. All of the internal system uses set allowIncomplete to 0.

7. bdsTestImportStream: Updated to support individual channel imports for testing stream import bugs.

8. 1739: Duplicate dataChannel numbers in DataChannels for BRTR. Basically if there was an existing BDS data file with say a set of 3 channels in it and then another stream import was started with backdated data and with just one channel and that channel did not exist in the original BDS data file 9appending), a new streamlet would be added to the data file and a new database DataChannel entry would be added as expected. However the database DataChannel entry was added a bit to early on, before the BDS data file was opened to actually find out what streamet channels actually existed. In this case the new dataChannelId was added as "1" in the database. The actual BDS datafile however, does have the correct streamlet channel for this new data channel and blocks are added correctly. So it is just the Database dataChannelId is wrong. It will be possible to correct the databases DataChannel dataChannelId's in the affected channels with an appropriate program.

9. 2669: Instrument type (chars 89-94) in IMS WID2 line. Fixed.

10. 2930: -ignoreCorruptions and -fixCorruptions on old BDRS and WRA40 files. Fixed this bdsImportData bug.

11. 2474: Add an API function to return the current client API version. With the latest Beamlib, this can now be done with the following API call: BError ping(BUInt32& apiVersion); ///< Pings the connection and finds the remote version number.

12. 2859: BdsServer validate disk storage once per hour. Now properly checks the available (not read only) BDS stores for available space ignoring those on duplicate file systems.

13. 1154: BdsServer /tmp storage. Checks if there is less than 5 GBytes free in the temp directory. The TMP directory can now be set with the bdsServer.conf parameter: "TmpDir".

14. bdsAutodrm: Print message when no config file is found and exit cleanly rather than segfault.

15. 2664: Use a new library to handle emails rather than mailtools. I have investigated this and it appears work is still going on on the GNU mailtools and now the library side is under the LGPL license rather than the GPL. I have created the code in the bds-third-party GIT repository to create new libmailtools RPM's from the GIT source code but only create and package the mailutils libraries. I have built packages for Rhel7, Rhel9 and Fedora37 and put these in the development repositories. The bds code has been changed to use this new library only package. We will need to perform tests of this when testing the first 4.x.x release.

16. 2941: user-supplied startTime and endTime when accessing metadata. Added multi Metadata support to BdsDataFileStationXml converter and BdsDataFileSeed converters. Modified the API functions, metadataGetChannelInfo() and metadataGetFormatted() to take a more generic MetadataOption bitset to allow multiple options when fetching rather than the simple allowMissing boolean. This now has the multiple options: MetadataOptionAllowIncomplete, MetadataOptionClipTime and MetadataOptionFixedChannels. By default these API functions will no longer clip the start and end times of the Metadata segments to the user supplied start and end times, unless the bit option MetadataOptionClipTime is set.

17. 2655: Input and output of "datum" in StationXML Added support for the import and export of the BDS Location.datum field.

18. 2546: BdsImportData: Only check for valid Metadata over timespan of data import. Now only imports and validates data over the user supplied import timespan. This is currently only implemented for the BdsSeed data convertor.

19. 2398: BDS Metadata export fixes, add comments/warnings somehow. Added internal API’s to describe fixes applied. These are not provided to the user yet.

20. 3012: StationXML output start and end time of Sensor and Datalogger. Added endDate attributes to Station and Channel only when ther are not set to unknown.

21. 3016: dataType in database from SEED and StationXML metadata. If the StationXml does not provide the data type with a "Type" field, we now use the SEED Channel naming convention to set the dataType in the StationXml data converter.

22. 2398: BDS Metadata export fixes, add comments/warnings somehow. The BDS API has been extended with a new "notices" system. This has the API methods:

BError        noticesAdd(BString notice); BError        noticesAdd(const BStringList& notices); BError        noticesGetList(BStringList& notices); BError        noticesClear();

This provides a system where multiple lines of messages can be added to a BDS API connection either by the BdsServer programs internals or indeed a client application. A client application can then get these messages and display them to the user if wanted.

The BDS data converter API has also been extended to allow data convertors to add messages to this system. The only messages currently added are the import/export "fixes" applied to the Metadata/SensorData.

Currently only the bdsDataAccess program can display these messages which are enabled with the new "-v" option.

The BDS API has changed to version 29.

The database schema has changed to version 21.

Release 3.1.9 – 2024-07-30

Minor update for virgin clean build on current Rocky9 systems.

Release 3.1.8 – 2024-04-16

A bugfix version.

1. bdsTestImportStream: Updated to support individual channel imports for testing stream import bugs.

2. 1739: Duplicate dataChannel numbers in DataChannels for BRTR. Basically if there was an existing BDS data file with say a set of 3 channels in it and then another stream import was started with backdated data and with just one channel and that channel did not exist in the original BDS data file 9appending), a new streamlet would be added to the data file and a new database DataChannel entry would be added as expected. However the database DataChannel entry was added a bit to early on, before the BDS data file was opened to actually find out what streamet channels actually existed. In this case the new dataChannelId was added as "1" in the database. The actual BDS datafile however, does have the correct streamlet channel for this new data channel and blocks are added correctly. So it is just the Database dataChannelId is wrong. It will be possible to correct the databases DataChannel dataChannelId's in the affected channels with an appropriate program.

The BDS API is unchanged at version 28.

The database schema is unchanged at version 20.

Release 3.1.7 – 2024-02-21

A special development release that changes the RPM package file layout and provides a new version client RPM that can be installed alongside other BDS versions.

1. 2699: Add support for BDS multiple installed version support. We have modified the BDS RPM file packaging to support a new bds<version>-clients RPM that can be installed alongside any other bds version. This allows the particular versions client programs clients to be accessed at: /usr/bds-<version>/bin. Some of the BDS files have been moved between the various package RPM’s to allow this. The bds<version>-clients RPM also contains the Python API module for that BDS version so that suitably configured Python programs can connect to and use a particular BdsServer version. The Python program should have its PYTHON_PATH search include the /usr/bds-<version>/lib64 directory for this. There are now directories /usr/bds-<version> for each version with /usr/bds being a symbolic link to the latest version. Note that the configuration files are now in /usr/bds-<version>/etc by default although the system will still look in /etc for them if there isn't any config files in /usr/bds-<version>/etc.

The BDS API is unchanged at version 28.

The database schema is unchanged at version 20.

Release 3.1.6 – 2024-02-02

A special development release that disables the bdsMetadataExportFix of response decimation and sample rates.

1. Disabled the bdsMetadataExportFixof response decimation and sample rates.

2. 2175: Decimation factor in StationXML and SEED output. Modified bdsMetadataExportFix to not fix up decimation factors and samplerRates. Also modified StationXml and SEED to to include decimation blocks when the decimation is 0 for a stage.

Release 3.1.5 – 2024-01-09

1. BdsGui5 programs: There was a bug in displaying the FIR a/b coefficients. This has been fixed.

2. 2789: dataFileIdc.getMetaData on FIR filter with denominator ('a' coefficients). The DataFileIdc data converter was not reading FIR coefficient responses correctly when both the a and b coefficients were present. this has been fixed.

3. 2797: bdsImportMetadata SEED import - Trailing space in Station and StationLocation station name. The DataFileSeed data converter was not removing extra trailing spaces on station names in blockette 50. This has been fixed.

Release 3.1.4 – 2023-07-26

1. Added the bdsAddData utility which can add Metadata and example sensor data for a set of channels with any sample rate for test purposes.

2. 2702: Ephemeral(?) failure of SEED-format export on "bds". Fixed the locking of SEED export after a MaxNumSamples error had occurred.

3. 2700: Check that Metadata import divide by zeros are all handled. Fixed the 1.0 / calibrationFactor and 1.0 / calibrationFrequency calculations to return 0 if these parameters are 0. Note we might want to check 1.0 / bdsPoleZeroGain() calcs.

The BDS API is unchanged at version 28.

The database schema is unchanged at version 20.

Release 3.1.3 – 2023-07-02

1. 2620: "source" vs "sourceMeta" in bdsImportMetadata and its output in bdsDataAccess. Changed documentation for this.

2. BdsAutoDrm: Updated help file.

3. Modified to use latest Beamlib paths. These allow specific Beamlib versions to be installed and used.

4. Added initial support for Redhat9 builds.

5. 2218: Segmentation fault when downloading response in GUI. We have now fixed the crash that occurred when using the "Meta Data/Channels/Channel/responses/Download" function.

6. 2692: bdsImportMetadata does not set "network" in Station to user override value. Fixed.

7. 2626: bdsDataAccess - wrong number of samples in IMS-format output. Fixed this. There was a small bug in the bdsServer which was shown up with the 0 sample initial period of 00:00:00 to 00:00:00.799 time period in the 05498678_BN_I51GB_Main_2023-02-16T00:00:00_2023-02-17T00:00:00.bds file. Not sure why the last data block in that file ending at such a strange time, maybe GPS clock sync issues ?

8. Added support for RedHat9 although no bdsImportGui or tapeView applications on this platform yet.

9. Note to build on Fedora37 make sure the idc-cdtools-2.4.30-6.beam.fc37.x86_64 package is installed as earlier packages will return “internal compiler error: in ix86_print_operand_address_as, at config/i386/i386.cc:13872" build errors when building the BDS.

Release 3.1.2 – 2023-02-13

1. 2560: Occasional bad sampling rates in miniSEED data (2). This has been fixed.

2. 2581: "datalogger" in StationXML output. I have added support for import and export of: Sensor.Type, Sensor.SerialNumber, Sensor.InstallationDate and Sensor.RemovalDate and all of these for a DataLogger. InstallationDate are set the the startTime. RemovalDate are set to the endTime unless this is not set or set to the endOfTime. These will now work on both StationXml export and import.

3. 2589: Units of calibrations of infrasound/hydro data (Pa) in IMS output . This has been fixed but see notes in bug item.

4. 2594: field width of "Instrument type" in WID2 line of IMS-format output. Fixed.

5. Don't add "BDS Scaled" to calibration.calibrationUnitsDesc when m**s or m/s are converted to "m".

6. 2616: ChannnelLocation from bdsDataAccess. There was a bug in the BdsServer's metadataGetChannelInfo function that did not correctly choose the correct database entries for the locations when looking for Channel locations. This has been fixed.

The BDS API is unchanged at version 28.

The database schema is unchanged at version 20.

Release 3.1.1 – 2023-01-09

1. 2549: "EOL while scanning string literal" in bdslib.py. This is due to a bug in swig. I have modified the documentation a touch to work around this.

Release 3.1.0 – 2022-12-01

This is a new release based on the new Beamlib class library. The BDS build system has been updated to use this new library. The first release of the Beamlib library, 3.0.0, has an API that closely matches the previous Beam-lib library so the BDS functionality should be unchanged apart from the differences noted below.

Changes:

1. The Events and SpecialChannels tables were not restored when databaseRestore() was used. This has been fixed.

2. Updated BDS to use the new Beamlib class library

3. Beamlib class library has updated BTimeStamp::setString() and BTimeStampMs::setString() functionality to handle fractional seconds completely. See the Beamlib documentation for more details.

4. Documentation build system updated and now uses the new beamlib-bidl functionality.

5. Updated BDS build system to match latest Beam methods. This means that the Makefile.config is now named Config.mk and any Makefile.target files should be renamed Local.mk.

6. Note the CMake system has not been updated.

7. The new Beamlib library uses the beamlib-boapns rather than the boapns programs. These are currently compatible, but ideally you should move to the new beamlib-boapns by running: “systemctl disable boapns; systemctl enable beamlib-boapns;” and stopping/starting these.

8. 2536: bdsDataAccess refuses to export when one channel is faulty even if I don't request it. There was actually a general problem when the BdsServer was asked to select a set of channels when given an Array's name. In this case the bdsServer was expanding the Array to a set of Stations but was ignoring the selections Channel if this was set so it returned all Channels for all Stations in the Array. This occurred when selecting Metadata as well as Sensor data. This has now been fixed.

9. Updated BdsAurodrm help text.

The BDS API is unchanged at version 28.

The database schema is unchanged at version 20.

Release 3.0.12 – 2022-06-23

Various small bug fixes.

1. 2272: AutoDRM fails when wildcard finds LOG files. Added support in a Selection for an excludeChannels comma separate regular expression list of channel names to exclude. The bdsAutoDrm program now setrs this to “LOG” to exclude all channels named LOG. The bdsDataAccess command also now supports the “-excludeChannels” option to use this feature.

2. Updated the beam-lib to the latest 2.20.4 which has support for apiVersion().

The main BDS API has been updated to version 28 and so all clients will need to be rebuilt to use the latest API. We have not incremented the major version number based on this API change as the 3.0.x is a development set of releases.

Release 3.0.11 – 2022-06-16

Various small bug fixes.

1. 2350: complaints from rdseed about output blockette 62 polynomial response. Changes StationXml import to set validFrequencyUnits to “B” rather than “A”. I have also changed the bdsServer to output responses even for channels of type data if they have them but not produce errors if they don't have these.

2. Added the libTapeDigitiser and libbeamdsp libraries to the BDS packages.

Release 3.0.10 - 2022-06-08

Various small bug fixes.

1. 2424: Segmentation fault for bdsImportMetadata on dataless SEED file. There was a bug in the new bdsMetadataImportFix() code with type 'C' symetry conversions. This has been fixed.

2. 2430: crashed bdsServer trying to export metadata in SEED. I have added an error when endTime <= startTime.

3. 2434: IDC format needs to be in displacement. Altered IDC data converter to change seismic channels to displacement if there is a first PoleZero response.

4. 2416: YKA tar SEED fails maxNumChannels. Fixed error message to say “Error,There is data already present for these channels over the time period given.” in this case.

5. 2415: Can't export channels that lack a "Response" entry. Fixed.

6. 2418: bdsMetadata new "source". Now supports adding sources.

7. 2437: question about new items in Response for bdsMetadata. I have updated the Response processing to include the missing fields in API order. I have also updated the Station manipulations to handle the new Network parameter in the Station and ArrayChannels objects.

8. 2363: Add support for FIR symmetry conversions. BdsMetadata has now been updated to support the new BDS responses Metadata system. Included in this is the requirement to set $apiversion in all bdsMetadata command files to the BDS API version the script is intended for. See the updated BdsMetadata manual.

9. 2412: BDS Port to Fedora35. Done. Note that the PHP API library has not been ported to PHP 8 so far.

Release 3.0.9 - 2022-05-24

Various small bug fixes.

1. bdsImportStreamGcf: Added bdsSendStreamGcf.cpp for testing. Fixed a buglet that could cause a CPU core to go 100% for a time when the bdsServer was slow. Improved efficiency of block number store.

2. 2353: BdsDataFileLog Python API. This was a bug in the bdsPython for just the BdsDataLog converter.

3. 2372: IMS output to include full multistage response. This has been fixed.

4. 2375: recalculate A0 normalisation factor when reading in IDC-format responses. The BDS IDC data converter imported the A0 value as the stages gain. The system now ignores this value and the gain is left at 0 (unknown). The BDS doesn't store any A0 values for a PoleZero response anyway. I noticed that on export the BDS data converter outputted the stages gain in this A0 location. I have changed that to be the PoleZero A0 normalisation value and added a comment in the file for that. I have also added the functionality to the IDC data converter to print a warning comment if a Polynomial response stage is present on export similar to the IMS data converter.

5. 2344: StationXML (but not SEED) metadata output has A0 norm factor = stage gain. I have changed the StationXml data converter to output the calculated A0 value at the calibrationFrequency.

6. 2363: Add support for FIR symmetry conversions. The BDS now converts all FIR coefficient imports to the full A format.

7. 2368: BDS export Metadata fixes. General BDS export Metadata fixes changed/added.

8. 2382: BDS Import Metadata fixes. General BDS import Metadata fixes added.

9. 2343: Station description from SEED and XML files. We have added support form importing and exporting the station.description field. This will be in the StationXml Site.Name parameter, we still ignore the Description field. For SEED this is in the blokkete 50's name field. We could store more here (probably matching StationXml, but API changes will be needed for that).

10. 2350: complaints from rdseed about output blockette 62 polynomial response. I have changed StationXml import to set the following on Polynomial import: resp.polynomial.transferType = "P"; resp.polynomial.validFrequencyUnits = "A";

Release 3.0.8 – 2022-05-05

Various small bug fixes.

• 2185: Downloading LOG data causes bdsServer to crash. This has been fixed.

• 1989: minor points on AutoDRM. Updated response command to use metadataGetFormatted API. The format checked against the list of DataFormats, which supports aliases, so IMS2.0 will now work, amongst other things. Updated CMake to install bdsAutodrm.conf.

• 2210: Corrupt SEED metadata from production server. After looking at the generated SEED files when the system returned a good SEED format file and a bad SEED format file it was noticed that the abbreviation codes were incrementing from one export to another. This resulted in an extra byte being added to hold the increased integer code from the 3 specified in the SEED format leading to file format corruptions after around 150 exports from a BdsServer start. This was due to a bug in the BdsSeed data format converter which has been fixed.

• 2162: Discussion on Stage Gains. From this the Polynomial response parameters were wanted to be stored as database columns rather than stored in the responses Polynomial blob. This has now been done. The database schema was updated to 20 to support this. The system will still read old storage style Polynomial parameters although these won’t be seen the the database columns. Whenever a Polynomial response is written the new storage system will be used.

• 2255: StationXML Decimation/Offset field should be integer. Fixed.

• 2290: BdsPython API: No StationXml data conveter present. The BdsPython did not support the DataFileStationXml data file converter. I have added that to the Python API. Also noticed that the DataFileAd22, DataFileIdc, DataFileLac, DataFileCss, BdsDataFileLog were also missing, so I have added those as well. The BdsDataFileAd22.h file was also missing from the BdsLib C++ API. This has been added.

• 2299: Responses: Modify BDS clients and server to expect stage 1 upwards response stages. Modified: bdsImportBlacknestDatabase, bdsTest system updated, bdsGui5 programs: Default Response is now stage 1 with a name of Main. The name field is now a text entry box, bdsServer: When validating Response additions, ignore name field, only the stage should be unique.

• 2301: Responses: Export convertors make sure they match the new Response handling system. Have added the bdsFixMetadata() function that performs the necessary Metadata modifications as needed for the various BDS data converters on metadata export. All of the BDS data converters now use this function with appropriate arguments. The modifications performed are based on the Responses document.

• 2302: Responses: IMS export convert stage 1 response to provide displacement data. When the IMS format is selected, the Response Metadata is now provided as a displacement response for seismic sensor.

• 2305: Responses: Exporting a multistage response to a format that only supports 1 stage. The bdsFixMetadata() function performs this.

BDS database schema has been updated to version 20. The BDS database should be updated using the bdsDatabaseUpdate command.

Release 3.0.7 – 2022-02-24

The 3.0.6 was linked to the old libmseed-beam rather than the new libmseed so any SEED import had major problems. This has been fixed.

Release 3.0.6 – 2022-02-22

1. 2065: Modify web portal to accept start times and durations in seconds.

2. 2075: Fix web portal metadata download.

3. 2074: Allow 'add Network' in bdsMetadata

4. 2076: Event data export test - 2 issues. The bdsDataAccess "-name <name>" event selection was not working. This has been fixed.

5. 2008: CMake build system. This allows the BDS code to be built using the cmake system rather than the simple Makefile based system. Note this also removes "#include <boost/shared_ptr.hpp>" from libbeamdsp/filters/gr_types.h which shouldn't have any negative effects although this has not been tested. It removes the dependency of needing the boost system for BDS builds. Any effect would be in the TapeDigitiser import/view/manage programs.

6. 2038: Metadata import/export. Added support for unit descriptions in Calibrations and Responses.

7. 2038: Metadata import/export. Added unit description import/export support to StationXml and SEED data convertors.

8. 2078: Searching for StationLocations with/without channel names. The Selection has a new parameter "locationSelect" which affects the operation of the locationsGetList() function.

   1. LocationSelectAll: Selects both station and channel locations for the channels in question.

   2. LocationSelectStation: Selects just tation locations for the channels in question.

   3. LocationSelectChannel: Selects just the channel location, if it exists for the channel in question.

   The default is LocationSelectAll. The BdsGui5 programs use the default LocationSelectAll and so the Locations list will show both Station and Channel locations if they exist. I have updated bdsImportMetadata to use this when validating the import of station and channel locations.

9. 2093: Add support for Polynominal responses to the IMS data convertor. For now Polynomial responses are added as a comment “ (comment …)”. Note we are unsure how programs that import IMS format files will cope with this.

10. 2125: Creating a network with empty network code. Adds validation to prevent empty network names, and fixes setting Station.network when reading SEED and StationXML files.

11. 2150: SEED import, metadata ignored when data validation errors occur. BdsDataFileSeed::getMetaData returned prematurely after calling getInfo (before setting channelInfos). If you are importing metadata and ignoring data validation issues, you got no metadata back.

12. 2145: Memory errors in BdsDataFileSeed. Noticed that some SEED files would fail to import on one machine, but succeed on another. Tracked this down to memory errors in the code, sscanf did not have a null termination for the string.

13. 2006: Upgrade libmseed to recent version. Now uses the general libmseed-2.19.5 library rather than libmseed-beam. The bds RPM generation has a requirement for that exact version at the moment as this library contains code that can affect SEED data file imports. As Redhat7/Centos7 does not have this RPM in its normal repositories we have created a Redhat7 libmseed-2.19.5 RPM that is now in the BDS development repository for that system. The code to build this is in the bds-thirdparty. Note there could be library conflicts with this if a libmseed RPM appears in the Redhat7/centos7 repositories.

The main BDS API has been updated to version 27 and so all clients will need to be rebuilt to use the latest API. We have not incremented the major version number based on this API change as the 3.0.x is a development set of releases.

BDS database schema has been updated to version 19. The BDS database should be updated using the bdsDatabaseUpdate command.

Release 3.0.5-1 – 2022-01-31

This is a minor update to development release 3.0.5 building the system using a newer swig-beam.

1. 2046: swig-beam patch introduces segmentation faults. Fixed an issue that would cause semi-random crashes in Python code using the BDS API.

This release was built using swig-beam-4.0.1-5.beam RPM package which is in the development RPM repositories for Fedora33, Fedora35 and Redhat7/Centos7.

Release 3.0.5 – 2022-01-26

This is a minor update to development release 3.0.4 fixing a few items an adding Polynomial response support.

1. Improved error message when Validation of Metadata in import fails.

2. 1919: Operator precedence issue in bdsTapeDigitiserFile. Compiler warning fix.

3. 1954: change in behaviour of AdminAccess.locationGetList v. 2 to v. 3. StationXML Metadata imports were not setting the Calibration.name to Main and hence data imports would fail with Metadata validation warnings of no calibration info. This has been fixed.

4. 1918: Data Import Metadata validation fails with new DataType's on import. The StationXML data converter now sets the dataType to data for unknown channel dataTypes. This allows those channels to be imported. The Metadata validation code in the bdsServer and in bdsImportData has been changed to ignore the dataType unless it is set to unknown.

5. 2012: StationXML import of file did not add Locations for the Station. There was a bug in the BDS StationXML data converter. This has been fixed.

6. 1868: Occasional bad sampling rates in miniSEED data Change of DataFile ReadOptionValidateCorruptions flag to ReadOptionFixCorruptions and reverse its state ie setting ReadOptionFixCorruptions enables the fixing where as before setting ReadOptionValidateCorruptions would disable the fixes. This changes the DataFile API. Also added the ReadOptionFixSampleRate option which only applies to SEED files and rounds the samplerate given in the Metadata. The command line user programs now have the -fixCorruptions and -fixSampleRate options that apply the API flags. The -validateCorruptions flag is kept on for backwards compatibility. The bdsDataImport program now has the -fixList which will list the possible fixes that can be applied to a particular format file on import. This uses the new DataFile API function getFixesInfo().

7. 1911: polynomial responses in SEED headers. Added support for the Polynomial response type. This has involved an update to the BDS API, support in the bdsServer, bdsAdminGui and bdsUserGui support for viewing and editing, StationXML import/export support and SEED import/export support.

8. DataFileSeed: The Type57 blocks sample rate was incorrectly set with a leading space messing up the sample rate and other decimation values on export.

The main BDS API has been updated to version 26 to support the Polynomial response type. The BDS DataFile API has also had a slight change for this release. Any client programs making use of the data converter ReadOption* flags will need modifications and all clients will need to be rebuilt to use the latest API. We have not incremented the major version number based on this API change as the 3.0.x releases are development only releases.

Release 3.0.4 – 2021-12-13

This is a minor update to development release 3.0.3 fixing a few items.

1. 1826: I/O - regressive bug to do with trace labelling (probably) at data export. A problem was introduced where the Metadata ChannelInfos could be out of channel order wrt to its associated Sensor data DataInfo. This could happen if the startTime’s of the Channel Metadata caused the selection ordering to be incorrect. This resulted in export data having incorrectly named channels. This has been fixed.

2. 1865: bdsMetadata problem. There was an issue with selecting some Metadata items using a specific Network. Depending on the selection and database entries this could result in items that were not part of the given network being selected. This has been fixed.

3. BdsMetadata: The Station object now has the Network field included in listings and can now be set with bdsMetadata.

4. 1868: Occasional bad sampling rates in miniSEED data. This is due to resolution errors in the software that created the SEED files providing incorrect SamplingRates in the SEED blockettes. We have added code to round the SEED samplingRates to 7 significant decimal digits which handles the issues in the SEED file where this was seen. This is enabled when the -ignoreCorruptions flag is given to bdsImportData (the API’s ReadOptionValidateCorruptions being 0).

5. 1894: SEED data convertor, support block reordering. An integer overflow bug in the BdsDataFileSeed converter led to 0.1Hz sampling rate data having incorrect time stamps. This has been fixed.

6. Added optional Makefile.target as a local RPM build settings file. Without this it will build RPM’s in the users ~/rpmbuild directory and create repositories in the users ~/bdsdist directory. There is a Makefile.target-template file that provides an example of the options.

Release 3.0.3 – 2021-11-11

This is a minor update to development release 3.0.2 fixing a few items.

1. 1860: Various segmentation faults with empty arguments to BDS client programs and API routines. Fixed a bug in bdsImportData where if no command line arguments beyond the -host and -user were provided the program would segfault.

2. In the BdsPython SWIG generated API the BError::set() function was not returning a copy of the BError object which could cause segmentation faults or incorrect BError contents. this has been fixed.

Release 3.0.2 – 2021-10-21

This is a minor update to development release 3.0.1 fixing a few items and adding some more features to help with Blacknest testing/feedback on the new Event/Metadata API’s.

1. 1824: A0 normalisation factor (poles-zeros constant) output. The BDS::DataFileIms data file format converter actually set "ci.responses[i].gainFrequency = ci.calibration.calibrationFactory" which was wrong.). I have fixed that.

2. 1825: IDC format A0 normalisation factor; put calibration freq into IDC format header: Added setting of PoleZero gain if gain value not set in response and added comments for calibrationFrequency and gainFrequencies used.

3. 1816: -formatList in bdsImportMetadata doesn't work. Added support for -formatList in bdsImportMetadata.

4. 1791: bdsImportMetadata on dataless SEED file. The DataFileSeed data converter was incorrectly decoding the decimationDelay and decimationCor values from the SEED 57 block. Also the bdsGui5 applications were not showing the exponential values. This has been fixed. Also fixed an issue with some SEED files reporting "Error:read block that was not a continuation block".

5. 1815: full SEED file bdsImportMetadata errors. The DataFileSeed converter was not handling the SEED 2.3 miniseed data only extension to change the physical block size for data blocks within a SEED file correctly. Support for this horrible feature has been implemented.

6. 1785: BdsTestServer regression test load complete database. The BDS regression test suite has been updated for the new 3.x.x development release. Note that many of the test results will not match the older 2.x.x release due to the new functionality supported. You will need to go through the results files that fail to compare, manually, to check.

7. 1818: "StationXML: FIR response not supported". The StationXml format now supports the FIR response stage type. Corrected Response inputUnits and outputUnits reading.

8. 1804: Events: Add magnitute units drop down box. Changed magnitude units to be a drop down box with set unit types.

9. 1803: Events: Hide download boxes when in edit mode. Implemented.

10. 1833: Output format of "event" info. Now can export the Event’s metadata in a simple, tabulated, CSV format.

11. BdsGui5: Various GUI fixes.

Release 3.0.1 – 2021-09-20

This is a major update to the BDS system adding the Events system and updating the API for this, improved Metadata storage/access and many other things. Initial releases will be in the Development repository. Quick changes will be made during the 3.0.x development which will mean database schema and API updates may not be reflected properly from one test release to another and manual database schema changes may be required.

1. 1699: BDS API BObj inheritance. A simple BDS API change to allow better access to object members.

2. 1693: Calibration Additions. Add waterLevel parameter. Add “rawCalibrationFactor”, “rawCalibrationUnits” and “rawCalibrationFrequency" to provide information on the raw seismic data ie. Before the data has been processed using the response information. Schema update changes existing Units from "m/s^2" to "m/s**2". BdsGui5: calibrationUnits, rawcalibrationUnits and Sensor::snsorGainUnits changed to use selection of units. Graph plotting now uses rawCalibrationUnits and rawCalibrationFactor if set.

3. The bds.repo file has changed to include the development repo. To install on a Fedora33 system run: “dnf config-manager --add-repo https://portal.beam.ltd.uk/dist/blacknest/fedora/33/bds.repo” for a Rhel7 system first remove the bds-repositories package with “ yum remove bds-repositories” then run “yum-config-manager --add-repo https://portal.beam.ltd.uk/dist/blacknest/rhel/7/bds.repo”. The BdsSystemAdmin manual has been updated to use this method of repository configuration.

4. 1692: Per channel Locations. This adds the ability to have separate locations for individual channels as opposed to only per station locations. This is to more fully support the StationXml and SEED Metadata formats. The Locations type has an additional “channel” field which if set indicates the location is a per channel location. The ChannelInfo type now returns both a stationLocation and a channelLocation. If no location is defined for a channel then both stationLocation and channelLocation are set to the stations location. The GUI editors provide the ability to set a per channel location if desired.

5. Removed the channelInfo.datatype field as there was already a channelInfo.channel.dataType field.

6. 1694: Response changes. Added the Input and Output SI units as used by both StationXml and SEED. If these are unset, then StationXml/SEED Metadata exports will set these to reasonable values so the produced files pass tests. Added the Decimation/Offset, Decimation/Delay, Decimation/Correction parameters. The BdsGui5 programs allows these fields to be viewed and edited. The import and export Metadata systems now use these values if available. If not they will construct a default set of units so that the StationXml and SEED files are valid.

7. 1698: Add BDS metadataGetChannelInfo and metadataGetFormatted export API functions. These provide the ability to get all Metadata for a selection of Channels.

8. 1718: BdsDataFileIms: Improved IMS Response format output. Now supports decimation and decimationCorr and produces DIG2 response when no frequency response information present. Note it can produce multiple Channel response information in a single file, I am not sure other programs can read this ?

9. 1717: BdsDataFileIdc: Datafile converter for IDC response file import/export. Add proper support for IDC response files of the formats IDC, IDC-POLEZERO and IDC-FAP. This takes the code out of the generic BdsDataFileResponse converter.

10. 1727: BdsDataFileSac: Extend to support import as well as export.

11. 1726: Responses: Overall response investigate usage. The function, bdsDataChannelOverallResponse() now looks first for a response named "Overall", but if not found, find the first PoleZero response in stage order. This is used when exporting to Metadata formats that only support a single PoleZero response.

12. BdsAdminGui5 and BdsuserGui5 general fixes.

13. 1738: example python code using DataAvailability API. Fixed some Python API access issues with dataAvailability() and DataFile::getMetaData() API functions.

14. RPM’s now signed. You can updatde the local repository description files with: Fedora “dnf config-manager --add-repo https://portal.beam.ltd.uk/dist/blacknest/fedora/33/bds.repo”, Redhat7/Centos7: “yum-config-manager --add-repo https://portal.beam.ltd.uk/dist/blacknest/rhel/7/bds.repo”.

15. 1702: ArrayOffsets better support. Array offsets are now stored in the ArrayChannels object per array rather than in the individual channel locations. This allows the definition of array offsets to be done per array rather than global for a station. They are now edited as part of a Station object. They are still present in the Locations object. These are filled in with the appropriate values if your are fetching data or Metadata and provide the arrays name as a selector.

• Updated the ArrayChannels class of objects and the associated MySQL table for array offsets.

• Added MySql update code to copy old arrayOffsets to ArrayChannels table.

• Updated API and GUI editors for array offsets in ArrayChannels. Now has array field in Selection and ChannelInfos which is used to select the ArrayOffsets for the given channel selection.

16. 1682: Metadata (response) export investigation. Large scale changes to Metadata API’s. BdsGui programs and bdsDataAccess now support this new system for Metadata export.

17. 1509: BDS: Add configuration API. Added the serverConfigurationGet() API call and updated the bdsDataAccess and bdsGui5 programs to display the servers configuration.

18. 90: Add Options API.

19. 1696: Bds Special Channels API. Have added this API: specialChannelGetList(), specialChannelUpdate() and specialChannelDelete(). The BDS database now holds SpecialChannel entries that are used to mark channels to be ignored during import. Only used currently by the bdsImportMetadata program but its use will be extended in the future. A new bdsLibInit() function now reads this information from the BdsServer to be used by the BDS import converters.

20. 1765: BdsAPI DataFileInfo add stream member. Added to API and to bdsGui5 apps.

21. 1580: BdsServer: Clear realtime flag on DataFiles when starting up. Done.

22. 1615: Support wildcards for network and station in AccessGroups. Done.

23. 1766: Make password stored in Users table more secure. The system now generates a more secure encrypted password that is stored in the database. Note that the existing passwords will only be updated to the more secure form when the user changes/updates their password.

24. 1782: Add network member to Station so that Stations can be only under a particular network if needed.

25. 1710: BdsGui: Allow random unit names in comboboxes. Updated standard SI Units list.

26. Added bdsUserAddGui program to allow certain users to add data including Events and modify Events if they are members of the appropriate groups.

27. Added the “eventsList”, “eventShow”, eventDataGetFormated and eventMetaDataGetFormatted commands to bdsDataAccess to list and provide details of an Event in the system.

28. bdsImportData: Added the “-event <id>” parameter that will update an event with the dataChannels imported. This updates the event’s startTime, endTime and dataChannels fields.

29. Updated the mailutils package to 3.13 and modified the BDS to use this. Now statically links with mailutils libraries for easier BDS install.

30. Added an eventId field to the Notes system so that notes can be attached to a particular event. The bdsNoteAppend command now allows the event id to be set.

31. bdsDataAccess: The dataGetFormatted command now always creates a file rather than sending output via stdout by default. The “-o stdout” option allows the output to be send to stdout.

32. BDS API and database schemas have both been updated. As the API has changed all client programs accessing the bdsServer will need to be updated. The BDS database should be updated using the bdsDatabaseUpdate command.

API’s

• BdsApiVersion: 25

• BdsSqlSchemaVersion: 18

Release 2.2.7 – 2021-08-26

This is a minor update.

1. Added the bdsTimestampUpdate program that can update the timestamps on data blocks in BDS data files and adjust the BDS Metadata in the databases accordingly.

   1. Some of the BdsServer restrictions on setting fields of the DataChannel and DataFile objects has been relaxed to allow the bdsTimestampUpdate program to change these.

Release 2.2.6 – 2021-05-24

1. Added better error message when two overlapping channels entries with the same network:station:channel are selected due to database issues.

2. Validate channelUpdate() when appending a new channel to make sure channel does not already exist over the same time period.

3. Added support for StationXML Metadata file export with bdsDataAccess’s new metadataGetFormatted command.

4. Channels of type “log” did not have location information. Fixed.

5. DataAvailability: Fixed issue where sometimes a segment was marked as fully available when it was only partially available. Also added the ability to get variable period contiguous data segments from this API function when num = 0.

6. bdsDataAccess: Added ability to get data availability information.

7. 1628: Web date/time default to today not 1980?. Implemented.

8. 1627: Hover-over prompts for web interface?. Implemented.

9. 1629: get the Web interface to plot non-synchronously sampled data. Implemented.

10. 1625: Timescale missing for >1 trace displayed on web interface. Implemented.

11. For Data and Metadata selections channel list is now sorted.

12. bdsWeb: Updated data selection and graph drawing.

13. The bds-web RPM package has been split into the bds-web and bds-web-doc packages. The bds-web-doc package contains the latest BDS documentation.

14. 1612: BdsMetadata: Add ability to clone Sensors and Digitisers. Added support for cloning Sensors and Digitisers.

15. 1663: QT5 GUI puts yellow square showing missing blocks in wrong place. Fixed.

16. 1616: bdsAdminGui5: Adding and Access group does not present the comboboxes correctly. Fixed.

17. bdsAdminGui5: File dialogs were misbehaving by opening two dialogs and causing application to not respond to events after the second dialog is closed. This has been fixed.

18. 1563: BdsRestart: Make sure systemctl daemon-reload has been run prior to restarting daemons. Installing/updating the RPM package now calls "systemctl daemon-reload".

19. bdsImportMetadata: Initial StationXML metadata import support with the new bdsImportMetadata program.

20. bdsImportMetadata: Initial SEED metadata import support.

21. bdsServer now supports the “-c <configFile>” argument to aid running test bdsServer instances.

API’s

• BdsApiVersion: 24

• BdsSqlSchemaVersion: 17

Release 2.2.5-1 – 2021-01-06

This release updates the bds-web system to use the latest WebSys PHP engine. No changes to main BDS components.

1. Updated the bds-web WebSys PHP engine to the latest 1.0.0 release. The BdsWebManual has been updated to reflect changes.

2. Build system changes to support builds for Fedora33 and some tidyups.

Notes

• If you have an existing bds-web installation you will need to update the bds-web’s database tables. To do this use the link: http://<bds-web host>/management logging in as an bds-web admin user and use the “Database Update/Create” link in the left hand menu.

API’s

• BdsApiVersion: 24

• BdsSqlSchemaVersion: 17

Release 2.2.4-1 – 2020-11-12

Minor bug fix update. Note this is a tested internal development release.

1. Updated bdsDatabaseUpdate script to Python3 and changed RPM dependencies for Fedora31 so it requires the correct Python MySQL RPM.

2. bdsServer: Now selects the realtime file named on the array (stream) name passed in the DataInfo. DataFile's now have a stream field to match against. This makes sure a single day file is used for the stream. There was a case on streams which had channels come and go that the system could create a second day file for the stream causing issues. Note that at the moment the API does not support access to the DataFile objects stream parameter so we could keep the API unchanged. When the API is next updated we will add this parameter to DataFiles and DataInfo. Note also that stream names must be unique ie. you cannot use two same named real-time streams from different networks currently.

3. Note that RedHat have changed the system distribution name from “el7” to “el7_9” so all Redhat RPMS now have this in their RPM file names.

API’s

• BdsApiVersion: 24

• BdsSqlSchemaVersion: 17

With this release there are updates to the database Schema to version 17. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

• Update the BDS software: "dnf update" or “yum update”

• Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

• Restart the BDS system. "bdsRestart"

Release 2.2.3-1 – 2020-09-30

Minor bug fix update. Note this is a tested internal development release.

3. bdsServer: When the bdsServer is restarted when a DataStore was nearly full, it could open the real-time day files incorrectly. This has been fixed. If the old DataStore still has sufficient space the original files will be opened for appending otherwise it will open the data files in the new store. However if the previous store has been now set as ReadOnly, the bdsServer will create new files in the next DataStore. In this case there will be two day files with partial data from the day.

API’s

• BdsApiVersion: 24

• BdsSqlSchemaVersion: 16

Notes:

• It would be worthwhile setting the ServerRetry and BdsServerRetry parameters in the bdsImportStream*.conf files to 9999999 to reduce the possibility of data loss if the bdsServer crashes.

Release 2.2.2-7 – 2020-09-28

Minor bug fix update. Note this is a tested internal development release.

1. bdsImportStream*: Modified connection to bdsServer to retry timeout period to 60 seconds to same lots of log messages. Also modified default config files so that ServerRetry is set to 9999999 by default.

2. bdsServer: If for some reason there are two day files open, choose the most suitable one.

3. bds*Gui: We have changed the new QT5 GUI programs to be bdsAdminGui5 and bdsUserGui5. The older QT3 ones are bdsAdminGui3 and bdsUserGui3. We have created symbolic links from the bdsAdminGui to the bdsAdminGui3 (and bdsUserGui) for now while the QT5 ones continue to be developed.

4. bdsServer: Now flushes file contents to disk when whole block is flushed. This will almost eliminate BDS file read errors when exporting data from files open for real-time import.

5. bdsImportStreamGcf: Now creates block number file if not present.

6. bdsImportStreamCd: Now closes open files after 1 hour of inactivity rather than 3 hours. This allows all of the data to be available for export quicker.

7. bdsImportStreamGcf/bdsImportStreamCd: Changed close file notification log entries to be of the LOG_INFO level.

8. bdsTest: Updated BDS python test programs from python2 to python3.

9. Various build improvements including the use of “-O2” optimisation on all code.

10. Packaging improvements including tcsh dependency fixes.

API’s

• BdsApiVersion: 24

• BdsSqlSchemaVersion: 16

Notes:

• It would be worthwhile setting the ServerRetry and BdsServerRetry parameters in the bdsImportStream*.conf files to 9999999 to reduce the possibility of data loss if the bdsServer crashes.

Release 2.2.1 – 2020-09-15

Minor update to fix huge data file issue. Note this is a little tested internal development release.

1. bdsServer: We have removed the flush of real-time data import files to disk every 10 seconds as this was causing very large files to be generated.

Release 2.2.0 – 2020-08-14

Major changes to API to support better Python3 API library. So far the BOAP binary API has not changed so this system is compatible with the 2..1.x releases at the moment. Also the database schema is the same as for 2.1.17. Note this is a little tested internal development release.

2. Updated make build system to improve parallel builds and better dependency handling.

3. Cleaned up the build tree of old stuff.

4. Updated BdsLib API to add “cont” to references where they are used as input only variables to help with SWIG Python API production.

5. Modified some little used BdsAPI functionality to clean API.

6. Integrated Python3 SWIG API changes for new BdsPython API library.

7. Updated all Python3 code to Python3.

8. Updated doxygen documentation build system to latest from other projects.

9. Large changes to BDS documentation including doxygen comments and other documents.

10. DataFileResponse: Now allows IMS polezero responses to have 0 poles and 0 zero’s.

11. DataFileResponse: Now ignores any group “delay” information when reading IMS response files.

12. BdsGui: Moved the old QT3 based bdsAdminGui and bdsUserGui programs to the names bdsAdminGu3 and bdsUserGui3.

13. Ported the BdsGui programs to use QT5. This is a major update to these programs so some things may have bugs and/or be slightly different in operation to the older QT3 programs.

14. BdsGui: Modified all GUI tables to provide column header based sorting.

15. BdsGui: Added user configuration settings storage. These are stored in the standard Qt location for user configurations: “~/.config/Beam/BdsAdminGui.conf” and ~/.config/Beam/BdsUserGui.conf”. At the moment this oly stores the size of the GUI’s main window.

16. BdsImportStream*: The bdsImportStreamCd and bdsImportStreamGcf programs now support a simple variable system in their configuration files. The parameter value fields can reference another parameter in the configuration file if the “$” symbol is used. So, for example, you could add a configuration file entry: “ServerHost1: 192.168.0.1” and then reference this in another configuration parameter like: “Stream0.host: $ServerHost1”.

17. bdsImportStreamCd: There was an issue where BDS data file channel numbers were not being correctly set in the MySQL DataChannels information with some specific CD1.1 streams where channels were coming and going. This has been fixed. Also the system no longer creates MySQL DataChannels information for real-time stream day files when there is no data for a particular channel within the day file.

18. bdsImportStreamCd: Added support for CD1.0 streams.

19. bdsServer: Now flushes the real-time data import files to disk every 10 seconds so that the real-time data is available sooner and there is less likelihood of losing data if the system crashes.

20. BDS data files extra channel information. When importing data from various formats, some extra metadata is written to the BDS data files. This includes things like calibrationPeriod and calibrationFactor in CD1.0 streams for example. When using the BDS tools to view the sensor data files metadata, for example with the bdsAdminGui’s “Data Info” button some of this information was not displayed. This has been fixed.

Release 2.1.17 – 2020-05-22

Minor bugfix release.

1. BdsImportStreamGcf: If a system crashed it was possible for the block number file to be empty on the next reboot causing bdsImportStreamGcf not to backfill to where it had been before the system crash. We now synchronise the file to disk on each block number update. The method chosen does have a slight performance overhead, but not as much as a full atomic file update would have although there is stall a very small chance that it could fail if the system crash occurred within a very small time window.

Centos7/Redhat7: bds-lib-2.14.11
Fedora31: bds-lib-2.15.24

Release 2.1.16 – 2020-04-24

Minor bugfix release.

1. BdsImportStreamGcf: This was being started as the root user rather than bds user. The systemctl scripts now starts it running as the bds user. Note its data files will need to have their ownership changed to “bds”. These are the files in the directory /var/cache/bdsImportStreamGcf and /var/log/bds/bdsImportStreamGcf.log by default.

2. bdsImportStreamGcf: Produce error when blocknumber state file cannot be read although it is present and retry server connection.

3. bdsImportStreamGcf: Debug printout when warning "does not have all backfill blocks" happens.

4. bdsServer: When trying to export multiple channels of data in a particular CM format and there is no data from one or more channels of data over the timespan, the system would return the error: “Error: No data segments are available in file for: …”. We have changed this behaviour to only return the error if there is no data in any of the channels selected. We have also introduce the possibility of a 0 numBlocks and zero time period segment to allow this to be used.

Release 2.1.15 – 2020-03-03

Minor bugfix release.

2. BdsImportStreamGcf: Turned of block level debugging that was creating large log files “blockdump-*.txt”.

3. BdsImportStreamGcf: If the remote SCREAM server system died, such as through a power off, the BdsImportStreamGcf did not determine that it was no longer connected and did not receive any more data blocks. This has been fixed.

Release 2.1.14 – 2020-01-14

Minor bugfix release.

1. 1594: Failure to export SEED or miniSEED. Fixed the export of SEED formated data. and metadata.

Release 2.1.13 – 2019-11-11

1. BdsServer: Modified usage of DataStoreMin: configuration parameter. Previously this define the minimum disk space required per open write file in Gigabytes. It now defines just the minimum disk space required in Giagbytes.

2. BdsServer: Added the DataStoreFileMin: configuration parameter. This defines the disk space per open write file in Megabytes on top of the DataStoreMin space.

3. BdsAutodrm: 1586: Have AutoDRM not break data into segments at file boundaries. The bdsAutodrm now uses the DataFlagMergeSegments and DataFlagClipDataToChannels optiond when opening a set of sensor data files. This merges the data into contiguous segments and clips the data so all channel data starts and ends at the same time.

4. 1585: Saving data from GUI in IMS-integer format. The format IMS2.0:INT now returns the data in INT rather than CM6 format.

Release 2.1.12 – 2019-10-25

1. BdsServer: Fix some bugs where a BdsServer crash can occur if there is an error opening a BdsDataFile. Also log the error that has occurred for debugging.

2. TapeView – Ability to delete all export markers on single button press added.

3. TapeView – Ability to spilt exported output into multiple ims files

4. TapeView – Ability to toggle export valid state in the export markers table with the “V” key added

Release 2.1.11 – 2019-10-15

1. BdsImportStreamCd: This now ignores incoming packet validation errors, such as the year 2000 time stamp issue rather than passing an error to the cdtools library. This allows bdsImportStreamCd to continue importing the stream without a reconnection time out on each bad block.

Release 2.1.10 – 2019-09-30

1. Added “systemctl daemon-reload” to RPM installs.

2. 1571: BdsImportScream: Crash or abort on sample rate change. Now reports sample rate change but if -ignoreCorruptions is set continues to import in bdsImportData and bdsImportScream.

3. BdsImportStreamCd: Improved debug print outs.

Release 2.1.9 – 2019-09-27

1. Changes to bdsRestart to not start/stop the old bdsImportCd and bdsImportScream daemons by default.

2. bdsImportGui - User interface enhancement. Middle mouse button can control scroll speed.

3. 1568: BdsImportStreamCd: Import channels mixed data file order. When more than one set of additional channels was added in a realtime stream (CD1.1 or GCF) then the MySQL DataChannels dataFileChannel number was not set correctly and overlapped previously added channels. This has been fixed.

Release 2.1.8-1 – 2019-08-16

Minor bug updates.

2. Changes to bdsRestart to stop client daemons first before restart main bdsServer process.

3. Modified BdsServer to gracefully terminate on the SIGTERM signal so that it flushes and open BDS data files to disk so that data blocks from on-going imports are not lost. This will make sure it does not lose blocks when bdsRestart is used.

4. Modified the BdsPython API to work better with the BdsDataFile* data convertor classes. The API now better handles passing arguments by reference in and out of functions.

5. Realtime import: Now sets times on data channel files to actual contents when closed so across 24:00 boundaries are correctly handled and clears the state flag to ok on file write aborts.

Release 2.1.7-6 – 2019-07-09

Minor packaging updates:

1. Now sets the user/group of /var/cache/bds* to bds/bds.

2. Runs “ldconfig” on library installs (includes libdata-gcf and associated packages) so that libraries are available.

Release 2.1.7 – 2019-07-05

Addition of BdsImportScreamGcf system and some minor bug fixes.

1. TapeView’s IMS export facility was still not working correctly. This has been fixed.

2. Data availability was clipped to the start of the current day. It is now clipped to the end of the current day to better show real-time data availability.

3. Added the new BdsImportScreamGcf system for realtime import of SCREAM GCF streams.

4. BdsPython: There we some issues with accessing BString objects as Python strings. this happened after the 2.1.5 additions of BList API access. This has now been fixed with all BStrings now accessed as native Python strings. So any code that used the <string>.retStr() syntax to return a Python string no longer needs the .retStr() member function call.

This release is in the bds-testing repositories.

Note on Redhat7/Centos7 this now expects the rh-php72 PHP/WEB server system. You will need to add the appropriate repositories (Using centos ones):

• yum -y install centos-release-scl.noarch

Then after the BDS system has been updated you can remove the older rh-php70

• yum remove "rh-php70*"
  
  
  

Release 2.1.6 – 2019-03-11

A minor bug fix release and a release for Fedora29 made.

2. TapeView’s IMS export facility was not working correctly. This has been fixed.

3. Released for Fedora29.

Release 2.1.5 – 2018-10-29

1. BdsPython: Added Bstring class and removed BList::append(BList) function from API.

2. Documentation updates.

3. SEED Converter. Now outputs depth field as “%05.0” (ffff. Or fffff) rather than “%05.1” (fff.f) when the depth is greater than or equal to 999.95 so that hydro-acoustic depths can be passed. Note this will fail in verseed tests.

4. BdsImportStreamCd: Now sets the numBlocks and numSamples fields in the DataChannels table to 0 for realtime data.

5. BdsImportStreamCd: It has been found that occasionally, in some CD1.x data streams, the channels within a CD block have differing time-stamps from each other. The BdsImportStreamCd will now normally send the combined channel DataBlock’s to the bdsServer, as before, unless it sees that the timestamps of the individual channels do not match. In this case it creates and sends one BDS DataBlock per channel for the particular CD blocks in question.

6. BdsDataFileCd: Reading all channels at once from CD1.x files has been disabled due to time stamp issues in some data streams where some channels in a block may have different timestamps to other channels in the same block.

7. BdsSendStreamCd: Updated this tool to be able to send CD1.1 binary files through CD receivers such as bdsImportStreamCd.
   
   
   

Release 2.1.4 – 2018-09-11

1. bdsDataFileDump: Decode CD1.x channel description(mode) and status blocks.

2. 1543: I52 bdsImportStreamCd shows 27 channels - should be 10?. There were some issues with the dataFileChannel number being set incorrectly when channels were added to an existing stream within the day. This has been fixed.

3. BDS data files where large when importing CD1.x data. This was due to duplicated block metadata. This has been fixed.

4. 1536: BdsTestAuto: Ignore data channels that are not marked as seismic. Only runs tests with genuine seismic data files.

5. Changed CD1.1 data block metadata names from "mode" to "cdDescription" and "status" to "cdStatus".

Release 2.1.3 – 2018-08-02

This is a minor update with changes based on Blacknest's feedback of the 2.1.2 release.

1. The CD1.x data converters used in bdsImportData, bdsImportCd and bdsImportStreamCd were not saving the CD1.1 channel status field into the data blocks. This has been fixed.

Release 2.1.2: 2018-08-01

This is a minor update with changes based on Blacknest's feedback of the 2.1.1 release.

1. bdsImportTapeDigitiserData: Segment fault when -verbose is used. Fixed this issue.

2. BdsWeb: Updated to support latest WebSys API.

3. bdsImportStreamCd: Added parameter IgnoreUnknownStations so that warnings on unknown stations is not logged.

4. BdsLib: Bug in bdsDataInfoMergeFlatten() that causes a segment fault fixed.

5. BdsDataFileBds: Now uses last DataInfo in file with getInfo() to allow for channel appending.

6. BdsImportStreamCd: With the above two changes now a a CD1.x stream can be imported when the blocks have different channels to previous imports within the same day file.

7. BdsImportStreamCd: Added open file list/cache to improved efficiency when backfilling. The MaxFilesOpen parameter defines the max number of open files. This should be around 3 x the number of streams in use.

8. Fixed issue with crash on reading null sized CD1.x files.

Release 2.1.1: 2018-06-25

This is a minor update with changes based on Blacknest's feedback of the 2.1.0 release.

1. bdsImportStreamCd: Modified service startup file to start daemon as the user bds.

2. 1506: BdsImportStreamCd: Crashes after a time of running: Added dump of crash info to the log file. need to look at this if it crashes again.

3. 1516: BdsImportStreamCd: Add config option to disable location code usage. Added the configuration file parameters: "Stream0.locationIgnore: 1" which will tell bdsImportStreamCd to ignore location codes when formulating the channel name.

4. 1513: BdsWeb: Trace display not working on centos7. Modified RPM to include packages gnuplot and netpbm-progs. Added setup script: /usr/bds/bdsWebSql/bdsWebSetup.sh to setup selinux context's.

5. 1514: BdsWeb: Clean up temporary web files. Data are temporarily stored in /usr/bds/www/files/tmp. We have added a cron script /etc/cron.d/bdsWeb.conf to clean files older than 2 days in this directory.

6. 1515: BdsWeb: Integerate standard Blacknest help pages in RPM release. The Blacknest user manual and standard BDS documentation has been added to the default BdsWeb information.

7. 1529: BdsImportStreamCd: Hydroacustic data differing block sizes on export.The jitter on the timestamps due to GPS time corrections is now handled within the BdsDataFileBDS BDS seismic data file converter. When data is exported the start and end times of consecutive blocks are checked to see if they are very close in time (within 1% of a blocks time period). If they are they are deemed to be the same then the endtime of the previous block is set to the startTime of the next block.

8. 1521: BdsImportTapeDigitiserData fails with segfault. Fixed this.

9. Added a new BDS regression test system to the bds-test package that uses real data and compares exported data sets from one BDS version to the next.

10. Added the bds-test-data package that contains a base set of data for the BDS regression test system.

11. Documentation updates.

Release 2.1.0-2 – 2018-04-05

This is a minor update that is built with an updated idc-cdtools package for Redhat7 and Centos7. It fixes a crash issue when using on Centos7.

Release 2.1.0 – 2018-02-12

This is a major update adding real-time stream capture and storage capabilities.

1. The BDS data file access code has been changed to support appending data to day files. As this code is used for all BDS seismic data storage it will need a fair degree of testing. Some changes to make the files more robust in the event of the Bds server crash have also been made. See the manual: https://portal.beam.ltd.uk/support/blacknest/files/bds/doc/BdsDataFile.pdf for more details.

2. BdsServer: Added support for storing and appending data into day files for streaming daemons.

3. BdsImportStreamCd: This is a new daemon program to support the storage of the real-time CD1.x cdtools streams into the BDS system. See the manual: https://portal.beam.ltd.uk/support/blacknest/files/bds/doc/BdsImportStream.pdf for more details.

4. BdsDataFileBdsDump: Added this tool to debug the BDS Data File format.

5. BdsImportScream: Fixed file descriptor leak.

6. Updated the BDS system to use the newer systemd system initialisation system.

7. Sets database access mode to allow timestamps of 0000-01-01T00:00:00. Newer database installations default to disabling the ability to store timestamps of this value.

8. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

There are no updates to the database int this release, it uses schema to version 16. If you are updating from a version pre 2.0.38 you will need to update the database howvere. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

• Shutdown bdsServer: "service bdsServer stop"

• Update the BDS software: "yum update"

• Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain" (Passwords as needed)

• Restart the BDS system. "bdsRestart"

Release 2.0.39 – 2017-12-14

This is just a partial package release for RHEL7 to fix a couple of bugs without changing the main bdsServer running system. It includes just the bds-importScream and bds-importCd packages.

1. The CD1.x file converter had a restriction of up to 20,000 Bytes of data before it would return an error. We have increased this limit to 100,000 Bytes.

2. BdsImportScream: Fixed file descriptor leak.

Release 2.0.38 - 2017-05-18

Minor bugfix release.

1. bdsDataFileDump: Fixed loop during data max/min calculation with channel ordered data files.

2. BKNAS-3.0 format, modified calibration factor field to be of format "%12.9e" for greater resolution and to handle exponents.

3. tapeView: When export cursors are used the sample time is also exported to the .tdi file.

4. tapeView:  Multi-recording marker meta-data now includes the ability to specify a start time for each recording segment.

5. Added copy constructors to Python API to allow the return of copies of BDS API objects from Python functions to work around issues when objects from BList's and BArray's are returned from functions that delete the lists.

6. Updated bdsDataAccess to use latest gnuplot when plotting graphs.

7. Added a BdsPhp API to allow PHP programs to access the BDS system directly using a PHP version of the BDS API.

8. New BdsWeb system.

9. BdsServer: Improved the perfromance of SQL lookups when regualr expressions are not used.

10. Database: Added some indexes to the DataChannels table to speed up accesses.

11. BdsMetaData: added "Change Station" function.

12. BdsWeb improvements from Blacknest feedback.

13. DataAvailability system nowrounds to nearest year,week,day or hour.

With this release there are updates to the database Schema to version 16. Actually it is just database information inserts. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

• Shutdown bdsServer: "service bdsServer stop"

• Update the BDS software: "yum update"

• Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain" (Passwords as needed)

• Restart the BDS system. "bdsRestart"

Release 2.0.36 - 2016-09-20

Minor bugfix release.

1. bdsAutodrm: Fixed an issue where an E-MAIL parameter with additional text caused the bdsAutodrm to refuse to send data.

2. bdsDataFileDump: Added ability to print min and max data values.

3. Added support for BKNAS-3.0 format files that provide I12 formatted data.

4. BKNAS: Added clip data function to limit data values to the I6 or I12 data formats as needed.

5. tapeView: Now includes the TapeView program superseding the old tapedigiser-tools package.

6. tapeView: Now supports multi-recording tapes with export of separate tape recording regions for import into the BDS system. (Note no BDS import support for this yet).

Release 2.0.35 - 2016-08-30

Minor bugfix release.

1. Fixed bug in BKNAS export which could cause a crash of the bdsServer program.

Release 2.0.34 - 2016-06-22

Major source code structual updates. This release is designed for the Fedora23 and RHEL7 systems and only releases for those systems have been produced.

1. Uses latest beam-lib.

2. Pulls in libbeamdsp, libTapeDigitiser and tapeView into BDS source tree so a single SVN tree contains all of the BDS and tapeView code.

3. Build changes for RHEL7.

4. Fixed crash in bdsServer when a a set of data was requested in sample sequantial form to a non block syncronised specific end time.

5. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Main thing here is the update to the latest beam-lib. This changes the error numbers. When errors have occured the BDS sometimes stores an error number along with an error message. After installing release 2.0.34 the stored error numbers will be different to those in the database/sensorDataFiles from previous versions of the BDS.

Release 2.0.33 - 2016-06-09

1. Initial release for RHEL7.

Release 2.0.32 - Preliminary

1. Modified the string to time conversion to allow date/times like: 2016-04-22T00:00 ie date/times without the seconds given. This allows that format of date/time to be used in BdsAutoDrm requests and other places.

2. Added a new example bdsMetaData4.py.

Release 2.0.31 - 2016-03-02

1. Modified all *Update() functions to set the objects id to 0 when appending. This is to simply API usaqge.

2. Added support for downloading metadata from the BdsGui program's "Meta Data" tab.

3. Modified bdsDataAccess to return a single SEED metadata file when multiple channels are selected with the format RESPONSE-SEED format.

Release 2.0.30 - 2015-10-20

1. bdsImportGui: Bug 1497. Improvements to  useability of scrolling graphical display.  If mouse is over the scrolling area then the auto-scroll speed can now be set via keyboard short cuts. The following keys are supported. Space - Starts and stops scrolling. Home - Sets scrolling speed to zero. PageUp/PageDown & Cursor keys modify the speed in steps. End - Sets speed to maximum. Additionally while in the plot area the mouse wheel button when rotated will scroll the plot forwards and back 1/10 of a single page on each click (Press and hold the CTRL key with Mouse Wheel spin to move a sample at a time).

2. CSS File convertor: Provide lower level error message on file open errors.

3. SEED Data Format: Added continuation blockettes in the case of large FAP responses.

Release 2.0.29 - 2015-08-17

1. CSS data format: If the data format was big endian, s3 (signed 24bit) the twos compliment data was not handled correctly for negative values, yeilding "square wave" looking data. This has been fixed.

Release 2.0.28 - 2015-06-18

1. 1493: dip and azimuth in dataless SEED files. Changed to use different dip/azimuth algorithem when exporting to SEED.

2. Added the userGetFromId() API function for all interfaces. This allows the user information for a given ID to be accessed.

3. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

4. When exporting small portions of data that has largish original block sizes such that the entire data set is within a single block and clipping the data to the user given start/end times,  the bdsServer did not account for the start sample offset when calculating the end sample. This resulted in an error "Internal error - trying to move to end sample...". This has been fixed.

5. When exporting data it was not always clipped to the user endTime supplied, the data continued to the end of the current block. This has been fixed.

6. bdsImportGui: Bug 1494 Item 3. The plot select time/date input now reflects the current position of the plot.

7. bdsImportGui: Bug 1494 Item 2. The default export file is now data-<exportStartTime>.<ext>

8. bdsImportGui: Bug 1494 Item 1. New functionality. Export Dialog now alway user to export a specified duration from the current plot location.

9. 1495: AutoDRM returns extra channels when "chan_list" not given. Fixed

10. bdsAutodrm: Added new help information.

Release 2.0.27 - 2015-05-19

1. CSS data format: If the dirname field is blank now uses current directory for data file names.

2. CSS data format: Fixed sample rate problem with some data. This was due to incorrect endTimes in *.wfdisc files and an incorrect number of samples calculation for the last block of data. The CSS converter now calculates its own end time based on the starttime, number of samples and samplerate.

3. Added the "-channelRename BHZ=BHZ_US,BHE=BHE_US" option functionality to bdsImportData.

4. bdsImportGui: The X-Axis of the data plot can now be viewed as either by time or sample number offset

5. bdsImportGui: Previously the auto colour button simply auto coloured the plot traces. The button now toggles state allow the user to revert the traces colours to the default

6. bdsImportGui: The maximum speed of the auto scroll feature has been increased

7. bdsImportGui: Display of channels numbers. Channel numbers now start at 1 rather than 0

8. bdsImportGui: New dialog added ExportDialog that allows user to specify export ranges.

9. bdsImportGui: New action added to the edit menu to allow a user to reset some plot parameters to default.

10. Added selection range to dataChannelGetList() function.

11. bdsImportGui: Fixed bug where reset traces to default did not revert to default colours and gain.

12. bdsImportGui: Select plot time widget now limited to actual available time range

13. bdsImportGui: Added menu item to view list of overlapping blocks.

14. bdsImportGui: When opening *.bdrs files the program can automatically create a jobInfo.tdi file importing the date and tapeNumber files from an existing jobInfo.log file.

Release 2.0.26 - 2015-04-22

1. Adds support for the CSS data format for reading CSS files.

Release 2.0.25 - 2015-01-29

1. BDS LAC data convertor allows conversion of 5Hz sampled data.

2. bdsImportGui: Support fo LAC files sampled at 5Hz added.

3. bdsImportGui: Fixed bug where input data overlaps would cause application to crash

4. bdsImportGui: Added command line options support the startup using scripts. Enter bdsAdminGui -help for details

5. bdsImportGui: Added markers to show location of overlapping data

6. bdsImportGui: Fixed problem with file export

Release 2.0.24 - 2014-12-10

1. Modified lowpass filter in VELA decode to improve VELA decoding as used in AD22 data format convertor.

2. bdsImportGui: segment fault on opening files with no data. This has been fixed.

3. Added LAC_BDRS format data converter.

4. bdsImportGui: Added support for LAC_BDRS format data files from the DTPS Digital tape import project.

5. Fixed segment fault in CD1.X format conveter when recovering from currupt frames.

6. 1479: YKA tar SEED failing with "-sequential" option. This has been fixed.

7. Fixed problem with BLists in Python interface. This caused issues when trying to access Response Metadata from the BDS data convertors in Python.

Release 2.0.23 - 2014-10-22

1. Added support for AD22 format file import.

Release 2.0.22 - 2014-10-09

1. SEED: Added code to set block size if different from default 4096 bytes.

2. SEED: Added code to round timestamps to 10us due to some files having incorrect timestamps. This is included if the API's ReadOptionValidateCorruptions flag is not set ie. if the bdsImporData flag "-ignoreCorruptions" is set the the timestamps are corrected.

3. The bdsImportData's "-source" option did not override the default "Main" source when a data file had the necessary metadata to supply the channel information.As I expect that BN would like to use the "-source" option when importing data from other sources I have now implemented this.

Release 2.0.21 - 2014-09-12

1. During export the system allowed a certain degree of timestamp jitter before segmenting the data into chunks. This caused problems with some YKA SEED data that had very small missing chunks of data or timestamp jitter. We have removed this functioinality so data will be segmented if the start/end block time do not match exactly.

Release 2.0.20 - 2014-06-13

1. The bdsImportCd program did not ignore issues due to data file curruptions leading to "Excessive authSize in packet" warnings. This has been fixed.

Release 2.0.19 - 2014-05-21

1. BdsImportScream: Now uses FileAgeMinimum = 864000 by default if not configured in the bdsImportScream.conf configuration file.

2. BdsImportScream: Adding logging of crashes.

3. BdsImportCd: Adding logging of crashes.

4. BDS CD1.x data convertor "Excessive authSize in packet" error. Have fixed this it was due to over agressive validation checks.

Release 2.0.18 - 2014-04-08

1. BdsImportCd: This refused to import CD1.1 files with multiple data segments due to large chunks of missing data. This has been fiexd.

Release 2.0.17 - 2014-04-03

1. BdsImportData: The -printBlocks function is now applied after blocks have been deleted through the use of -deleteDuplicates.

2. BdsImportCd: There was a bug that effects multi-channel CD1.x file import when there are different sample rates per channel. In this case some data blocks near the end of the file would not be imported. This has been fixed.

3. 1471: IMS "STA2 line for hydroacoustic data  Modified STA2 elevation and depth string formats. Depth is now output in km. String formats of floating point elevation and depth now take into account the "-" sign and reduce the number of digits after the decimal point to cope with larger numbers. If a number is outside the range printable and error is returned.

Release 2.0.16 - 2014-03-19

1. 1420: wrong number of samples in IMS header (hydroacoustic data). This has now been fixed. There has also been improvements to BdsServer data export colation. This will result in more accurate data start/end times and calculated sample rates for variable sample rate data and variable sample rate data blocks that stradle file boundaries will be joined into one segment properly.

2. 1413: GCF Unsupported block type: -1. This was not completely fixed. It has been now.

3. Added ability to ignore certain blocks during import with bdsImportData. The option "-ignoreBlocks 69,70,71,72,73,74,75,76,77,78,79" allows this. Note this only works for GCF format files at the moment but it can be exteneded for other file formats. The block numbers are those listed bybdsImportData after any sorting has been performed.

4. BdsImportData: Added -printBlocks option that will print the block headers after sorting and after -ignoreBlocks processing. Only supported for GCF data format at the moment. Can be used with -dryRun for block information.

Release 2.0.15 - 2014-03-04

1. Added MaxSearchTimePeriod configuration parameter and increased max import time from 10 to 31 days.

2. 1420: wrong number of samples in IMS header (hydroacoustic data). Fixed numSamples output when data block sizes just stradle endTime.

3. 1413: GCF Unsupported block type: -1. Have modified the GCF data convertor to ignore corrupted data blocks if ReadOptionValidateCorruptions is not set. ie with "-ignoreCorruptions".

4. 1418: cloning Response in AdminGui. Fixed issue where window was readonly when cloning or other actions called after viewing an item.

5. 1265: configuration file for AccessGroups. Added ability for users in the group "dataAccessAll" to access all data. The database scripts have been set to add the users testAdmin, testAdd, bdsImportCd and bdsImportScream to belong to this group.

With this release there are updates to the database Schema to version 15. Actually it is just database information inserts. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

• Shutdown bdsServer: "service bdsServer stop"

• Update the BDS software: "yum update"

• Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain" (Passwords as needed)

• Restart the BDS system. "bdsRestart"

Release 2.0.14 - 2014-01-29

1. When a Channel is updated or added the channel, channelType and channelAux parameters are checked and setup if they are null from information in the values supplied.

2. The BdsImportCd program could crash with certain heavily currupted data files. this has been fixed.

3. CD1.1 file converter now ignores year 2000 blocks if -ignoreCorruptions is used.

Release 2.0.13 - 2013-12-19

1. Added PythonAPI support for accessing FIR coeficients.

Release 2.0.12 - 2013-12-16

Minor bugfix release:

1. 1408: bdsManage "clean" deletes all of Logs and Changes entries Fixed bug with cleaning Changes and Logs.

2. Fixed bdsCloneStation.py to allow multiple channel clone.

3. 1403: "I4 not supported" for CD-1.1 data from I50. CD1.x added support for "I4" data type.

4. GCF converter: If a file has a block full of 0's it is now ignored if -ignoreCorruptions is used. Peviously it was allowed, but not ignored.

Release 2.0.11 - 2013-11-28

Minor bug fixes.

1. Fixed a bug in SEED export of responses where multiple copies of the response data was apended to previous values.

2. Response import: now supports IMS responses with no space before the "theoretical" keyword.

3. 1397: "load from file" button still active when "Response" window is in "View" mode. Response "Load form file buton" now disabled in view.mode.

4. Fixed a bug in datetime setting from strings in the forat as used in CD1.1.

5. Added the "measured" flag to responses to allow import and export of IDC responses of this type.

6. Added the "sampleRate" parameter to responses to allow import and export of IDC FIR responses with this info.

7. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 14. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. This includes changing the Responses named "Sensor" to "Overall". So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 2.0.10 - 2013-11-14

Major changes to Response system to allow multiple responses and FIR response types.

1. bdsMetadata: Added ability to delete ChannelInstruments.

2. bdsImportData: If trying to import a zero length data file or one with no actual data in it the bdsImportData program would still interact with the bdsServer resulting in a large database search for all of data which increased the memory usage of bdsServer enormously. This has been fixed.

3. The station and channel boxes in the bdsGui programs selector boxes could be very slow with large data sets imported. The speed of these has been improved.

4. GCF converter: If a file has a block full of 0's it is now ignored if -ignoreCorruptions is used.

5. 1390: Data deletion and tidy up of "waste basket" The bds's clean() API call now will delete datafiles that have been previously moved to the Deleted directory. It will delete files, Changes and Logs older than 1 year. The bdsManage command provides a command line access to this function.

6. 252: Database "backout" and "restore" facility Modified database backup system to store compressed backup files. Added the ability to restore portions of the database. The system supports: all – Restores all tables, admin – Restores admin tables, metadata – Restores metadata tables, data – Restores sensor data tables. The bdsMetadata and bdsManage command line tools can be used to access this functionality.

7. BdsGui applications now ignore the text case when pattern matching network/station/channel/source names.

8. Support for the FIR response type has been added.

9. Support for import and export of muliple stage responces has been added.

10. The Response system has had extensive modifications to tidy this aspect up. This includes new response format names, an updated Response format converter and the renaming of stage 0 from Sensor to Overall. Response operation is described in BdsResponses.pdf.

11. Note that bdsMetadata scripts that specify a Response will need updating. There is now a stage number field, a stageType field and a description field. Also the name the primary response should be renamed "Overall" rather than "Sensor".

12. bdsMetadata has been exteneded to support importing a multiple stage response file.

13. Support has been added to the  IMS and SEED data convertors for  FIR filter export and multiple stage response export.

14. Fixed a bug when importing SEED-TAR files with the -sequential option. Now allows this operation to function.

15. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 13. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. This includes changing the Responses named "Sensor" to "Overall". So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 2.0.9 - 2013-09-10

1. 1380: BdsImportScream: Directory layout. The bdsImportScream program now supports GCF files in any format of directory structure. The system now uses the files modified time to determine which files it has already imported. There is a new configuration option in /etc/bdsImportScream.conf, FileAgeMinimum, that defines the minimum age in seconds of a file to import. This is by default set to 864000 so that only files older than 1 day are imported. This allows the SCREAM system, to finish back-filling files.

2. BdsImportScream. Previous versions did not import files that had missing blocks or blocks in the wrong order. The new program will import these after re-ordering the blocks as needed.

3. BdsImportScream. This now logs each file imported with an Ok or Error status to the log file /var/log/bds/bdsImportScream.log as per the bdsImportCd daemon.

4. 1372: Improve implementation of ASCII data log support. The system now supports the storage and retreival of blocked ASCII LOG data in channels contained within the seismic sensor data files. The ASCII log information is stored in BDS data blocks within the info["log"] field. A special channel type "log" (Note this has changed from "ascii" in the previous test version) has been introduced. This channel type can be used for LOG data only channels (LOG data can be attached to normal sensor data blocks if needed). We have added a data file conveter DataFileLog which supports the LOG-SCREAM data format. this is the format used for the SCREAM systems data log files. The DataFileLog can both read and write files of this format and so can be used for import and export of log data. The bdsImportData program will automatically import LOG channels from multiplexed GCF data files. It can also be used to importSCREAM log format data files directly. See the bdsImportData manual for more information. LOG data can be exported using the bdsUserGui, bdsAdminGui and bdsDataAccess programs in the same way as other data when the "LOG-SCREAM" data format is selected. The bdsGui programs also have a "Data Log" button to show the entire data log for the given time period in a pop-up window. Note previous logs imported into the BDS test system are not compatible with the new system.

5. Fixed bug where all Sensors where selected when no channels were matched. This also affected Digitisers and ChannelInstruments.

6. 1362: AdminGui crashed because of too many missing blocks in WRA40 file Have speeded up the "Data Info" window update.

7. bdsImportScream. Added support for LOG file import.

• Note that due to the changes in the bdsImportScream system the configuration file /etc/bdsImportScream.conf should be updated as needed.

Release 2.0.8 - 2013-07-02

1. 1369: "1 not supported" in CD-1.1 import of I52 post-upgrade data Support CD1.1 data format with data transformation 1.

2. 1373: mismatch of channel with sys/stream ID in output from bdsImportData. This has been fixed.

Release 2.0.7 - 2013-06-19

1. 1360: GCF import loses station/channel name in database entries This has been fixed.

2. bdsImportData now accepts channel definitions of the form "=ZIKTW0:5503Z2" to ignore channels in multi channel GCF files.

3. Added the "ascii" channel DataType. This is used for ASCII log streams and the like.

4. 1363: FAP output. Fixed the bug where FAP responses were output in the old IMS format.

5. Fixed a bug where importing Muliplexed GCF files, where a data stream contained ASCII status blocks has been fixed. The GCF data file convertor will now add psuedo channels (The same GCF systemId:streamId with "_log" appende) for any channel containing ASCII data blocks. These channels will contain just the ASCII data with the original channel containing only the seismic data. The bdsImportData's -channel connand can now ignore these channels or allocate them to a BDS channel of appropriate name/type.

6. ASCII data is stored in a standard BDS data stream in the "info["ascii"]" feild. The number of samples in the channel data is set to 0. This data can be viewed by reading the streams data blocks and accessing the inifo["ascii"] field.

Release 2.0.6 - 2013-05-23

1. Added support for importing channel multiplexed GCF files. The GCF file converter has been updated to support this.

2. bdsImportData. The "-channels" syntax has been extended so a particulat GCF stream can be imported as a particular channel using channel information like: WOL:BH1_00=ZIKTW0:5503Z2. This means import the GCF stream with systemId=ZIKTW0 and streamId=503Z2 as the BDS Station=WOL Channel=BH1_00. Also if the BDS Station and Channel is not given then import of this channel is ignored.

3. libgcf2 has been updated to version 2.2.27. This fixes a bug where text status blockes were not processed correctly.

4. Modified IMS data format output to match IDC's change. In the FAP2 output block the number of triplets has been redefined to be an integer of size 5 and the FAP table is now formated as 12.6f 12.6f 12.6f.

5. Note that with the Python version on RHEL5/CentOS5  exeptions have a different syntax to the BDS examples provided.

Release 2.0.5 - 2013-04-10

1. 1347: AutoDRM format issues Have added support for IMS2.0, IMS2.0:CM6 and IMS2.0:INT format names and allow "TO" to be in lowercase.

2. The BDS API call calibrationsGetList() was not expaning Arrays to the set of Stations. this has been fixed.

3. 829: default filenames for miniSEED and dataless SEED. BdsGui programs now use the default extensions of "seeddata" and "seedmetadata" for SEED-MINI and SEED-METADATA exports.

4. 780: be able to change "source" of all channels in a file at once The DataChannel edit window now provides a combo bix for the source selection rather than a free from text field.

5. Added Python API and some Python API example to the system.

6. Added the bdsImportGui application. This supports the viewing of DigitalTape BDRS data files. It can be later expanded to support BDS data file import.

7. 1334: bdsMetadata let pole-zero or FAP filename be variable. This did not work for FAP files. Fixed.

8. 1164: in addWarnings, if no endTime is specified, it defaults to 0000-01-01. With multi-session TapeDigitiser imports the addWarning system now sets the start/end times for each session separately.

9. Fixed problen with CD1.0 files where data format's authentication field is set to a value other that 1 or 0. Also improved error recovery.

10. BdsAutodrm: Now ignores the case of the given format.

11. Added channel name comments to SAC response files.

12. 1358: IMS response output - space for "auxiliary identification code". Fixed.

The BDS system is now available in 64bit as well as 32bit versions for RHEL6 and Centos6. When updating the system will update based on the packages that are already installed. To convert a 64bit system that is running bds 32bit packages there is a script bdsUpdateTo64.sh which is in the /usr/bds/config directory as part of this release. A 64bit BDS installation is required on 64bit systems where Python programs using the BDS API need to be run.

Release 2.0.4 - 2012-12-10

Minor bugfix release.

1. bdsGui programs: When converting a PoleZero response to an FAP table from response Metadata, the system would use a calibration and sample frequency of 1 Hz. It now uses the values from the first calibration entry starting from the responses start time.

2. 1334: bdsMetadata let pole-zero or FAP filename be variable

3. 1335: bdsMetadata using variable for "split time" this has now been implemented.

4. 1320: When creating a new channel with a "network" other than "BN", GUI queries Do you Really...

5. 1319: Allow "network" to be changed in Clone Channel dialogue

6. 1318: bdsImportTapeDigitiserData sample rate error Added -ignoreSampleRate flag.

7. 1336: bdsMetadata "list" and "find" on Sensors and Digitisers tables Added support for normal channel selections.

8. 1118: allow "maxChanges" in bdsMetadata to be changed within a script file Added ability to set maxChanges from program using the $maxChanges variable.

9. bdsMetadata: Now issues an error if an unknown selection parameter is given.

10. 1164: in addWarnings, if no endTime is specified, it defaults to 0000-01-01 Now sets endTime equal to data's end time if not set. Also checks users times given are within the file.

Release 2.0.3 - 2012-10-24

Minor bugfix release.

1. bdsMetadata: "add Response" is not working with variables for the reponse to use. Fixed.

2. bdsImportTapeDigitriserData: Session numbers above 7 failed due to reading sessions numbers as octal. Fixed.

3. 1321: mport FAP using AdminGUI The BDS database used a "text" field type for the response storage. this was too small so the database sechma has been updated to version 12 with a "mediumtext" field.

With this release there are updates to the database Schema to version 12. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 2.0.2 - 2012-10-03

Minor bugfix release.

1. BdsMetadata: "add ChannelInstrument" was expaning member variables twice.

2. BdsMetadata: The split time argument can now be a variable.

3. BdsImportCd: Changed to ignore imports from 23:00:00 to 01:00:00 to allow for end of day file sync.

4. BdsImportTapeDigitiserData: The -timecodeIgnoreTimeJumps flag failed to work.

5. BdsDataFileCd: Now returns an error when no Data Format block is present at the start of a CD1.0 file.

Release 2.0.1 - 2012-08-13

Minor bugfix release.

1. bdsImportTapeDigitiserData would segment fault on older TapeDigitiser data files which had not been re-processed with TapeView and where the cursors startSample and endSample fields were not set.

2. BDS SEED export. Added a 58 blockette for the stage 1 gain.

3. 1259: msr_pack: No samples to pack when downloading H08 data as SEED Have fixed the "Error: BdsDataFileSeedError: msr_pack(BN_H11C2_US_LEA_): No samples to pack" problem. Was caused due to a block time calculation that in this instance of data request, block sizes, and time offsets caused a block of 0 bytes to be returned.

4. BdsSeed: Fixed a bug where if there were multiple channels per station in the data then the Station headers would be incorrectly output.

5. 1270: rdseed failure for SEED files with many data segments There was a minor bug with continuation of SEED data records which should make sure at least the blockette type and length fields were not split across records.

Release 2.0.0 - 2012-07-27

1. 1198: Cannot delete StationLocation from AdminGUI. The BdsServer now deletes StationLocations when station is deleted.

2. 1199: be able to export responses even where there are no data. BdsDataAccess: Modified response output to work when no data is present.

3. 1200: warnings issued during TapeDigitiser import and export. BdsDataLib: Fixed bug where Qaulity Notes were placed under the source Main rather than TapeDigitiser. This does mean than any existing imports will have the Notes incorrectly set in the BDS database. This can be corrected with a database script (setting the source to "TapeDigitiser" for all Notes where the title starts with "Quality" will probably be fine). The warnings "Warning in executing bdsTapeDigitiserFile" which occuerd are no longer stored.

4. 1201: v 1.2.22 does not write responses. BdsDataLib: Fixed bug 1201: v 1.2.22 does not write responses. Now writes IMS responses when set to IMS-2.0-CM6 and other sub formats.

5. 1207: "name" in "Response" edit window. Fixed BWEditComboBox bug.

6. BdsTapeDigitiserFile: Fixed bug with small files when wrong timecode type was set. In this case the program did not copy any data to the output file.

7. 1203: Output SEED format data with FAP files; verseed output; rdseed. BdsSeed data format was not handling large FAP tables properly, the data was crossing block boundaries without a continuation block indicator.

8. 1219: After "split at time", "Create New Digitiser/Sensor" button overwrites old. This has been fixed.

9. Modified TapeDigitiser import. Now responds to tapeView introduced: startSample,endSample,ignore and allows VelaPll and VelaStd timecode types.

10. 1223: bdsMetadata "list" on ChannelInstrument. Added the ability to get lists of ChannelInstruments without specifying the channelId.

11. 1221: plit calibration in bdsMetadata bdsMetaData should support "split Calibration", and in fact does do so but will return the wrong error message when the number of changes is exceeded have fixed this. Also added support for spliting ChannelInstruments.

12. BdsMetadata: Added support for varibles to be used in the parameter seting section of BdsMetadata commands. Also have added the "print" command (with no arguments) to print all variables.

13. BdsImportBlacknestDatabase: Added verfification of station and channel names during import.

14. 1231: BdsGui: File selection lost when you save a mod to a channel in Data Files tab This has been fixed.

15. 1232: bdsImportBlacknestDatabase interpretation of "AHT" and "AST" This has been implemented.

16. BdsServer: Modified missing data errors to list channel information.

17. 1249: WRA_AGSO sampling rate There was an issue in the BdsServer data gathering and export code to do with time calculations. This would have affected all exports of all data types where there was a lot of separate segments of data. This has been fixed.

18. 1248: YKA tar SEED all except first file have locn code "YK" BdsImportData: Fixed a bug where the network and station location where not
    overriden in the special case for YKA SEED data from 1991-07-10 to 1995-09-01.

19. 1253: Cloning calibrations Found that this command had accidentialy been effectively removed when the command "clone Location" was added.

Release 1.2.22 - 2012-05-08

1. Added support to BDS GUI applications to show FAP responses when exporting from seismic data tab.

2. Added ability to export FAP responses in SEED sensor data exports.

3. Added ability to export in EVALRESP (FAP) format from PoleZero responses.

4. Added mode and status metadata to CD1.x imports so that calibration info can be retrieved from data blocks.

5. Changed some channel types in bdsImportBlacknestDatabase for Bug: 1172.

6. IMS format export now includes Responses in PoleZero or FAP by default.

7. BdsGui applications: Added "No Metadata" tickbox on data export. This reduces the amount of Metadata included in the export files. Mainly this means no response data.

8. BdsDataAccess: Added the "-noMetadata" flag. This reduces the amount of Metadata included in any export files. Mainly this means no response data.

9. BdsGui: Changed "Response" field name to "Name" in Response dialog and made this an editable field so any value cane be entered.

10. BdsMetadata: Added ability to change Response.

11. CD1.x data import now imports channels marked as down and/or zeroed with a data quality warning included in the BDS data file.

12. Now only reads and uses the first PoleZero response in IDC-format PAZFIR files.

13. BdsGui: Added ability to download responses from the MetaData information tabs.

14. Now prefixes the response file's extention with "fap" or "polezero" as appropriate by default.

15. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Release 1.2.21 - 2012-04-24

1. bdsImportCd: Now only looks in directories that have a digit as their first character. This stops warnings about trying to access lost+found.

2. Have set bdsImportTapeDigitiser to continue with the Timecode processing even if no Timecode has been found for 4 hours.

3. Modified wording for "Duplicate Block" during data conversion.

4. BdsMetadata: Added support for FAP and FIR responses.

5. BdsMetadata: Modified "list response" to return actual responce data.

6. BdsMetadata: Added "read" command to read responses from files.

7. BdsGui programs: Added support for FAP responses.

8. Modified BdsDataFileCd to only check relevent channels during duplicate block check.

9. Added ability to export FAP responses in EVALRESP format in BdsGui applications and bdsDataAccess.

10. Modified default response file name to include response type  (polezero, fap).

11. CD1.x data convertor: Added support for "-ignoreCorruptions" flag in bdsImportData. If a currupt (short) block is found, it will be ignored and the next valid block found. This is now the default in bdsImportCd.

12. The CD1.x imported was setting the number of blocks in a data files channel to one less than there were. This would have affected sample rate calculations.

13. Added support for multi-segment data import. This allows CD1.X data files to be imported as separate segments of data into one BDS data file. So CD1.x data files that have sets of data over different time periods can be imported and used easily without the issue of "missing blocks" errors. Currently gaps of greater than 1 hour result in these database segments.

Release 1.2.20 - 2012-03-19

1. Fixed bug where CD1.x file import would fail with "std::length_error" on cd1.0 files without channel authorisation.

2. Fixed file descriptor leak when file import errors.

3. Fixed time stored in bdsImortCdTimes.cache.

4. BdsDataFileDump: Added support for -blocks command to display block timestamps etc.

5. Fixed problem with DataInfo startTimes not being set correctly when blocks ordering is completely incorrect on CD1.x and GCF files.

6. BdsImportCd: Added file import log in /var/log/bds/bdsImportCd.log.

7. Moved logrotate bits to bds package.

Release 1.2.19 - 2012-03-15

1. Fixed crash bug in bdsImportCd/bdsImportData.

2. Added validation of authSize filed in CD1.1 data frames to eliminate a crash whith currupt data files.

3. Modified bdsTestCreateMetaData to create responses with equal poles/zeros.

4. Fixed a crash in the BdsServer where the incoming data block was not being fully validated.

5. Modified the CD1.X data converter to use ChannelMultiplexed data rather than SampleMultiplexed data due to a feq data files having missing chunks of data on a per channel basis.

6. BdsServer: Added support for calibrations with name = Measured. These entries can overlap normal calibrations and have there startTime and endTime equal if desired. All data exports ignore calibrations with a name of Measured.

7. Added calibrationName to selection mechanism.

8. Added calibrations command to bdsDataAccess.

9. CD1.x data convertor: Set the maximum TIMESTAMP_JITTER to 1% of the block time rather than 0.1%.

10. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Release 1.2.18 - 2012-03-12

This is a minor update, fixing some small bugs.

1. BdsImportCd has been modified to ignore the curent days files when importing.

2. Fixed a bug in CD1.1 import where a block with missing channels and short numbers of samples would cause a samplerate validation error.

3. The CD1.1 data convertor now ignores "Type 7" frames. These are an alert frame that indicates that the connection is about to be terminated.

4. BdsMetadata added support for listing and changing ChannelInstruments. This requires the updated bdsServer program as well as bdsMetadata.

5. Changed SEED export to set Blockette 53's normFactor to be 1/gain.

6. Modified VELA PLL Timecode system for more efficient memory usage. This now allows at least 48 hours with no time codes.

7. Added code to allow continuation of VELA PLL Timecode system beyond 48 hours of no time codes.

8. Modified tapeView to not recode timecodes before start cursor.

Release 1.2.17 - 2011-10-07

This is a minor update, fixing some small bugs found in the 1.2.15 release and adding Hutchins timecode support.

1. BdsImportTapedigitiserData and bdsTapedigitiserFile: Modified "-timecode" timecode type fields to "Manual, Vela or Hutchins" with parameter checking.

2. BdsImportTapedigitiserData and bdsTapedigitiserFile: Changed -velaStartTime to -timecodeStartTime etc.

3. Fixed bug in bdsImportTapeDigitiserData where it crashed with data files > 4GByte.

Release 1.2.16 - 2011-09-19

This is a minor update, fixing some small bugs found in the 1.2.15 release and adding Hutchins timecode support.

1. 1144: Problm exporting CD-1.1 data from hydrophone array H08  Have found this bug due to the timestamp jitter algorithm not resetting new endblock times. Have fixed this.

Release 1.2.15 - 2011-09-14

This is a minor update, fixing some small bugs found in the 1.2.14 release and adding Hutchins timecode support.

1. Added support for Hutchins and Manual Timecode re-processing in bdsImportTapeDigitiser and bdsTapeDigitiserFile.

2. Fixed CD1.X data import bug were endian was incorrect.

3. Modified CD1.X data converter to issue MissingBlocks errors/warnings when Channel SubFrames have no data (numSamples == 0).

Release 1.2.14 - 2011-08-31

This is a minor update, fixing some small bugs found in the 1.2.13 release.

1. 1138: Problems specifying "Station" This has been fixed.

2. BdsServer: Added ability to select sensors and digitisers from channel selection criteria. This allows sensorsGetList() and digitiserGetList() to return a selection based on the channel selection crteria.

3. bdsMetadata: Changed bdsMetadata to set the sensors oldId field to 0 on cloning.

Release 1.2.13 - 2011-08-11

This is a minor update, fixing some small bugs found in the 1.2.12 release.

1. 1110: Signs of poles and zeros entered in bdsMetadata This has been fixed.

2. 1113: inconsistent channelType and channelAux in bdsMetadata "clone" This has been fixed.

3. 1116: Two queries about interpretation of command files by bdsMetadata I have fixed the line number error reporting when "/* */" comments are used.

4. bdsMetdata, the default variables were not setup. These are now setup as per the manual.

5. 1114: Cloning in bdsMetadata does not alter dates for all the cloned lines. The system now sets the start and end times of the calibrations, responses, and instruments to the new channel settings. It also only clones the Calibrations,Reponses and Instruments within the newly created Channels time range if more than one Calibrations,Reponses and Instruments are selected.

6. 1131: BdsMetadata: Add flag to give BdsServer name. The bdsMetadata program now has to be given the "-name <server name>" flag with the name of the server beeing connected to. This is an assitional safety check.

Release 1.2.12 - 2011-07-25

This update adds support for the CD1.x data format and adds the bdsMetadata manipulation program. As well as these and a few extra small features, it fixies some small bugs found in the 1.2.11 release.

1. 1093: Add ability for normal users to change their password. There is now a menu item in the File menu allowing the user to change their details in the bdsUserGui and bdsAdminGui programs. There is no command line support for this as yet.

2. 1091: Non-synchronously sampled data from two or more 3-component stns served in BKNAS-1. Fixed this.

3. Added the bdsDataManage utilty in the package bds-extra.

4. Fixed bdsDataManage wild cards and set dryRun as default.

5. Added the "-ignoreVelaReprocessingErrors" flag to bdsImportTapeDigitiser to ignore VELA re-processing errors.

6. Fixed bug on import wehre imp[ort would loop on first file in directory.

7. Added data convertor for CD1.0 and CD1.1 support.

8. Added bdsImportCd a daemon to import CD1.X data files recieved over the network.

9. Modified bdsImportScream to re-order blocks, allow duplicate blocks and ignore missing blocks during import.

10. Modified bdsImportScream to also add import validation warnings to the BDS Notes system.

11. Added the ability for the bdsServer to ignore minor time discontinuities when exporting data. Any discontinutity less than 0.1% of a blocks time period will be taken to be a clock resynchronisation event.

12. Fixed bug in BdsServer where ArrayChannels were not deleted when an array was deleted.

13. Changed the bdsServer to delete the Calbrations,Responces and Instruments for a channel when a channel is deleted.

14. Added database backup and restore API to the BdsApi. This allows the BDS database and server configuration to be backed up into the Backup directory of the currently writeable datastore.

15. Added the bdsManage program. This runs as an Admin user and peforms the work of bdsControl but with additional functionality. This includes database backup and restore.

16. 1016: BDS database looses data if a transaction from another program is cancelled. Each BDS client connection now has its own database connection and so is independent of other clients. This also has the side effect of speeding up BDS access when another client is performing a large database search.

17. Added the "readOnly" option to the BDS data store configuration in the bdsServer. This allows a set of datastores to be marked read-only. These will not be used for wriring new data. This is useful for the creation of a test server sharing the seismic data of a main server in read-only mode.

18. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 11. Actually this is only the addition of the user bdsImportCd. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 1.2.11 - 2010-12-14

This is a minor update, fixing some small bugs found in the 1.2.10 release.

1. 1034: IMS response calib zero. The formats of the calibrationfactor CAL2 and pole zero values in the PAZ2 lines have been change from %f to %e. The calibrationfactor is now output in nm/count rather than m/count in IMS exports (both sensor data and meta data). This also applies to the ASCII and BKNAS data file output convertors.

2. BdsImportData/BdsImportTapeDigitiserData: The user supplied -addWarning Network:Station:Channel:Source parameter was overriden with values obtained from the data. The user setting now takes presidence. Also added validation of Network,Station,Channel,Source parameters passed against metadata.

3. BdsAdminGui/BdsUserGui: Notes deletion was poping up warning/error dialogs when deleting notes without documents attached. This has been fixed.

4. BdsImportTapeDigitiserData: We have added a test to make sure memory is not eausted when the data's VELA year is set to 0.

5. 1039: Date-time widget "end time before start time" BdsAdminGui/BdsUserGui: Modified to not test for endTime < startTime while entering dates.

6. Improved the performance of the dataSearch() API call when using arrays with multiple station/channel pairs.

7. 1041: User GUI modification/delete buttons active

8. 1047: trace viewer scaling BdsAdminGui/BdsUserGui: Now only auto scales data of type seismic together.

9. 1054: bdsDataAccess reports zero calibs Changed to use scientific notaion for calibrationFactor.

10. 1055: "data already present" error while importing TapeDigitiser data This has been fixed.

11. 1057: TapeDigitiser multisession data with missing blocks - Notes table contains too many entries This has been fixed, also fixed in bdsImportData.

12. bdsTapeDigitiserFile: Added ability to invert VELA and Error tracks.

13. 1066: EKA3617 TapeDigitiser timecode problems with day 367 Fixed bug with VELA year wrap.

14. 414: Database does not keep up with station/channel name change Modified API function stationUpdate to also modify all of its StationLocation entries.

15. 1071: Make restricted stations invisible to unprivileged users The data access bug 1072 has been fixed (Not checking all of the segments in a multi-segment data request). We have added support for completely masking stations if they are added to AccessGroups with a time period 0001-01-01 to 9999-01-01.

16. 1078: Tab name DataSelect in UserGUI or Sensor Data in AdminGUI

17. 1073: DataPlot in bdsDataAccess fails for multichannel data with non-aligned block boundaries Now supports accessing random block sized data (GCF) in a sample multiplexed manner.

18. The segment end time was not being set correctly on channels when a MetaData occurred and the data blocking amougst the channels was un-even.

19. 943: calculated sampling rate does not match metadata when "Calibration" split not on block boundary

20. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Release 1.2.10 - 2010-11-04

This is a minor update, fixing some small bugs found in the 1.2.9 release.

1. BdsServer: Changed default "DataStoreMin" size to be 32GBytes.

2. BdsServer: Fixed bug in data store allocation with multiple data stores wherea data file write error could occur due to insufficient space being available.

3. BdsImportBlacknestDatabase: Modified to set endTimes of "EKA:BHZ" channels rather than "EKA:BHZ_10" channels.

4. BdsImportBlacknestDatabase: Modified to add location codes to channel types in ArrayChannels table.

5. Modified Notes/Warnings system to include particular Station and Channel names when imporing from multiple channel files.

6. 1003: incomplete "Data Info" Each files individual info is now preceeded by "file<n>" when multiple files are involved.

7. 1006: Start time becomes end time in -addWarning

8. 1009: pole-zero constant is inverse of correct value

9. 1017: add oldId to list for "clone" or "share" an existing sensor

10. 1016: Export error when "full blocks" not set (ref 952, 953, 965)

11. 1029: BdsAdminGui: Raise Data info windows when appropriate button is pressed

12. 1015: original file name of tar SEED files is given as the first /tmp file into which the tar SEED was untarred Now uses the original tar file name for all filename entries.

13. Notes documents are now stored in a documents directory within the main data stores.

14. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 10. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 1.2.9 - 2010-10-21

This is a minor update, fixing some small bugs found in the 1.2.8 release.

1. 995: out-of-place WID2 lines in IMS data exported with "segment merge" ticked Fixed a bug in merged segment output.

2. 997: "SeismicUnknown" channels not finding metadata but spurious backslashes Now allows import of data into "seismicUknown" and "empty" data typed channels.

3. 994: Duplicating/conjugating poles and zeros (see 784/792) Have changed this to add after the selected item.

4. Added multiple hour long GCF like test data.

Release 1.2.8 - 2010-10-15

This release has a number of modifications for the first live BDS release. The core features added or changed are:

1. The Warnings/Notes system has been overhauled. Note that the bdsImportData's "-addWarning" syntax has changed. Note that due to this there will be no Notes from previously imported data.

2. Note that the /etc/bdsServer.conf file should be updated with the "DocDir:" entry. Info in /etc/bdsServer.conf.rpmnew.

3. The bdsAdminGui's tab layout has been changed by using top level windows for some of the previous tabs. The selectors have been merged into one set of code although there are still separately settable selectors. There are also quite a few changes to support the bdsUSerGui program sharing the code base.

4. The bdsUserGui progam and bdsNoteAppend programs has been added.

5. Instrument response export has been added.

Detailed list of changes:

1. Added support for -includeSessions flag in bdsImportTapeDigitiserData.

2. 925: Unable to edit location of an array in bdsAdminGui Fixed.

3. Have changed error messages in bdsTapeDigitiserFile to include "Error:" or "Warning:" at start.

4. The error "ErrorDataPresent" is now returned when bdsImportData or bdsImportTapeDigitiserData fails to import when data is already present.

5. 932: bdsImportTapeDigitiserData reports temporary filename Changed bdsImportTapeDigitiserData to report original file name in errors.

6. Modified SEED export to only list one station type 50 block per station.

7. 944: TapeDigitiser import with extra files in directory bdsImportTapeDigitiserData now only looks of files of the name format "data-*.bs".

8. 931: bad_alloc while importing TapeDigitiser data bdsTapeDigitiserFile: Added support for setting the error and vela track numbers with flags. Returns an error if the VELA code could not be read for more than 4 hours. Added support for setting the correct VELA and Error channels from the info in the JobInfo file to handle files with unusual VELA code tracks..

9. 945: YKA SEED TAR files with currupted ends Added the -ignoreTarErrors flag to bdsImportData command.

10. 946: SEED: Improve compatibility of output SEED export: The "#" can be removed by setting the station alias. I have modified bdsImportBlacknestDatabase to do this by default. The station block numbers in blockette 11 have been fixed.

11. bdsImportBlacknestDatabase: Now adds a station alias for all stations with a '#' character. This station alias has the '#' removed. This allows export in all formats with station names not having a '#' character.

12. 36: Instrument Response Information: Accessing Added support in the BdsUSerGui and BdsAdminGui programs for exporting the sensors response in SAC-POLEZERO, IMS-POLEZERO and IMS-FAP formats.

13. Added support to the bdsAdminGui, bdsUserGui, bdsDataAccess and bdsAutodrm programs to export sensor responses.

14. 941: DataStoreMin ineffective if data file larger than value set, can cause data loss

15. 952: Modify Full-blocks data export feature The new normal operation is to return all of the data requested between the times stated to the nearest sample. Some channels may not contain data at the start or end times (short channels). The FullBlocks mode is the same as it was. There is an additional option in the API to clip the data so that all channels are complete. Block Start/End times were incorrect with time clipping, this has also been fixed.

16. Added support to the bdsAutodrm program to export sensor responses. Supports "GSE2.0"/"IMS" and "SAC" pole zero formats.

17. The bdsAutodrm now support sthe "EMAIL" as well as "E-MAIL" commands.

18. The bdsAutodrm now logs to the log system rather than the notes system.

19. 950: Rationalise warnings/notes/logs Major changes to data warnings and Notes system. All data import warnings both by the system and user added ones, with -addWarning, are now added to the BDS sensor data files and stored in the Notes database table. Data warnings now have a start and end time so that only the warnings relavent to the data being accessed will be shown on export. Only the notes stored in the database are used for data notes information. The warnings store in the file can be viewed in the DataInfo tab wich shows all of the MetaData stored in the actual data files. When accessing sensor data the GUI programs now have a "DataNotes" window to view these notes. The Notes tab has also been updated with a selector to select the notes required.

20. The Notes system now has the ability to store documents in various formats. The Notes database stores information on the format (mime format info and normally the filename extension). The actual document files are stored in a documents directory on the BdsServer which is defined using the "DocDir" configuration parameter. There is a new, command line, program named bdsNoteAppend that has the ability to append notes with and without documents. Note that there is no way of viewing these documents as yet although the API provides the mechanism for document reading.

21. BdsAdminGui: Changed tab system. MetaData is all under one tab with sub tabs. There is now one Sensor data tab that brings up top level windows for the Traces, DataInfo etc.

22. BdsAdminGui: Single selector widget for all selections. Separate instantiations for MetaData, SensorData and DataFiles.

23. BdsAdminGui: Improved selection performance.

24. BdsAdminGui: Disabled the ability to change the file DataChannel sample rate parameter.

25. 49: BdsDataView GUI program Added BdsUserGui program for user access to the BDS system.

26. 940: date-time widget sometimes has 5-digit year This has been fixed. It occured when there was no data in the system.

27. 923: "Logs" tab info - should this be so easy to alter? And bugs in deleting and accepting new items I have fixed the delete bug and added checking for the priority value. I can remove the ability to delete/update log entries if wished.

28. 916: Cannot display info in DataViewInfo tab if channel lacks metadata  We have added this ability. If the CalibrationFactor or calibrationFrequency values are 0 then the system will set them to 1.0. It will also use the samplingRate as defined in the data file and ignore the rate in the metadata if it is 0. Instrument MetaData data is currently ignored.

29. BdsAutodrm: Added new help file. This file is stored in /usr/bds/config/Help.txt and has been set as a package "noreplace" file so that local changes will not be overwritten on BDS package updates.

30. 890: Merge consequtive data segments on export. This has been added. There is an option in the bdsGui programs for this and bdsDataAccess has it enabled by default.

31. 934: BdsImportData: only validate between startTime and endTime. The bdsImportData command now only performs data validation between the startTime and endTime values if set. Note that data curruption, data reordering and filename time errors are still reported and noted.

32. 982: is downloading in "BDS" format meant to work? We have disabled the export of BDS files. This was intended to be an internal format only. If wanted we can add support for exporting in this format.

33. TapeDigitiser data import. The block time stamps were not quiote corrrect due to a bug in the ms to us conversion (could have been out by up to 500ms).

34. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 9. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 1.2.7 - 2010-07-16

1. Fixed the database creation scripts.

2. 892: database table ArrayChannels errors for "BKNI" entry: Modified bdsImportBlacknestDatabase to set up the correct channels in the ArrayChannels database table. Fixed a bug where spaces were included in ArrayChannels station names.

3. The admin email address list and AutoDrom station and channel lsits also sufferend from having spaces included in the list components.

4. Modified bdsImportBlacknestDatabase to ignore metadata entries for channels with the formats "GCFEKABBCD" and "GCFEKBCD". The source "EkaCD" will probably be used for this data and has been added to the inital database setup scripts. Note that for an existing database the souce "EkaCD" will need to be added.

5. 898: start time when requested timespan includes staggered beginnings of data: There was a bug in the BdsServer where if the user selected muliple channels of channel multiplexd data begining or ending within a missing block of data the BdsServer would segment fault. This has been fixed.

6. No API or Database changes have been made with this release.

Release 1.2.6 - 2010-06-30

1. Added "-allowOverlap" flag to bdsImportData program.

2. Now stores the original import file name in the DataChannels database table. This is accessible from the BDS API in the DataInfo object. The bdsDataAccess programs "dataInfo" command will return this information. There were also changes to bdsAdminGui to support this feature.

3. 887: adminGui hangs during export of mixed-"source" data when timespan includes start of one source

4. Modified to store the full path name of the original import file.

5. Added code to setup stdout to be in line buffered mode for command line programs.

6. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 8. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 1.2.5 - 2010-06-21

1. Fixed bug where data files were not moved to ./Deleted when deleted.

2. Modified data export to clip data to the sample level at requested times rather than just make sure the data starts abd ends at the sample sample.

3. Disabled changing of DataChannel.importFormat field.

4. Added support for calculating overal TapeDigitiser digitisation quality values and displaying warnings on import and export.

5. Fixed a bug where warnings added with a time set to "unset" where not displayed on export.

6. 881: failure to open temporary file while reading tar SEED data When importing SEED TAR files now sets the permisions on the un-tared files to allow reading.

7. When importing GCF files, the filename time comparison now only compares the year and day.

8. 880: very minor bug in error msg from non-existent file channel

9. Fixed a bug that did not allow 0 length strings in comma separet lists. this apllied to -addWarning fileds and other cases.

10. Fixed a few minor bugs.

Release 1.2.4 - 2010-06-03

1. 828: Database: Move to transaction based database The MySql database has been changed to be a Transaction based database using the InnoDB engine. The Bds API has been updated to support Transactions and some of the BdsAdminGui's operations have been modified to use this feature. We have also updated the database to use the utf8 character set by default.

2. 85: Data per sample access and data block interpolation

3.  The system now has an option for "Clipping" the output data so that the start and end times of all of the channels export is the same. The data blocks of channels that are beyond this period are truncated to the appropriate sample. The truncation is performed based on the blocks time stamp and the sample rate in use.

4. 496: WRA_AGSO won There was a bug in the WRA-AGSO file convertor where the first block was repeated and the last block lost on each channel.

5. Modified bdsImportTapeDigitiserData to treat bdsTapeDigitiserFile processing warnings as warnings not errors.

6. 86: Notes: Add information on TimePeriod, Network, Station, Channel andSource

7. 685: Add GCF like variable block size data to BdsTest system

8. 850: BdsAdminGui title to reflect connection to server

9. 854: Add BDS Clean API call to clean logs and changes database entries

10. 59: Error and Warning event system

11. 665: Missing block warnings

12. Modified dataType field to support seismic,seismicUnknown,data,unkown and empty. The bdsImportBlacknestDatabase program now sets these based on the channelType.

13. The bdsImportBlacknestDatabase program did not import calibration info for channels without sensors. This has now been done.

14. 837: TapeDigitiser import: Add flag to ignore sessions

15. 701: Add support for source names in bdsDataAccess -channel options bdsImportData and bdsImportTapeDigitiserData now allow the source to be specified in the -channels flag.

16. 700: Export sample rate information

17. 820: "hang" for horizontal seismos of unknown orientation

18. 777: BdsAdminGui: Improve Channel edit dialog , 522: Changing channel name loses calibrations/responses. Many improvements to the Channel editing system. The core difference is changing a Network/Station/Channel name changes all of the metadata and sesimic data in the system to match. Note that the channel information in the sesimic data files is not affected although this is not used anywhere. As we are now using a transactional database all of the changes to the channel,calibrations,instruments,responses is only commited when the "Save" button is pressed.

19. 838: BdsAdminGui: Improve Instrument Sensor and Digitiser entry. Improved this in line with sugestions from Sheila.

20. 77: Add support for easy import of Meta Data working at Array, Station and Channel levels

21. 768: Instrument "split at time" when split time is before opening date/time of instrument metadata

22. 694: Change file URL locations to just specify relative path within the data store

23. Added support for 'M' type data records in SEED files.

24. There have been a few parameters added to the /etc/bdsServer.conf file. It will be worth taking the new version and editing this with the desired parameters.

25. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 7. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

The new release uses a Transactional database. By default the MySQL system stores all InnoDB database tables in a single file. If desired, the system can store each table in a separate file. To do this edit the /etc/my.conf file and add the option "innodb_file_per_table" in the "[mysqld]" section and restart the mysql daemon with the "service mysqld restart" command. If this is done before bdsDatabaseUpdate is run the new transactional database tables will be stored in separate files. See info in: https://portal.beam.ltd.uk/support/blacknest/info?cmd=infoView&id=828 for more details.

Release 1.2.3 - 2010-05-11

1. The bdsImportScream program now uses the import validation options: "OptionReorder" and "OptionDeleteDuplicates". So any SCREAM backfilling errors should be handled as best as is possible given the faulty SCREAM system.

2. 700: Export sample rate information. The sample rates reported in IMS,BKNAS and SEED files are now the calculated sample rates rather than the MetaData specified sample rates. This handles variable sample rate data better, such as TapeDigitiser data.

3. 703: Save dialog, always defaults to save in ~ FIle dialog now keeps last uesed directory.

4. 708: "dip" in SEED blockette 52 should be -90 usually for vertical seismometers. Swaped sign of dip calculation.

5. 704: sensitivity "inf" in SEED files when calib = 0. SEED gain is set to 0.0 if the calibrationFactor is 0.0.

6. 724: Group checkboxes dissappear on View/Append/Modify Fixed bug with Users group boxes dissappearing.

7. 725: Admin GUI Segfault in Data Files tab

8. 726: Channels selection View/Modify do not work with mutiple selections

9. 728: creating array in AdminGui - added array stations without setting "type" to "array"

10. Fixed a bug in bdsServer in array to station expander. This could cause no stations to be selected if the first station in the database was an array.

11. 716: -duration flag doesn't work on bdsDataAccess

12. 739: Station description rendered badly Modified all tables to display first line of multiline descriptions.

13. 747: BKNAS1 number of channels must be < 32 BKNAS convertor restricts number of cahnnels to 31 with an error.

14. 754: Changes tab - minor hiding issue

15. 755: Missing control on certain forms

16. 763: Save warning dialog has two buttons

17. 769: bdsDataAccess reports warnings to stdout

18. 550: Problem with multiple Pause/UnPause Fixed.

19. 736: Restricted-length character fields Added name length checking for Network, Station, ChannelType, ChannelAux and Source fields.

20. 784: "conjugate" pole has zero imaginary part if "return" or "tab" not hit; window focus

21. 783: wrong "calib" in data exported over mid-block metadata change. Modified to start next segment on block following MetaData change.

22. 781: Reconecting from Admin GUI after server restart causes bds process to fail

23. 774: Updating list when new channel is added

24. 705: "#" in station name where otherwise station name = array name

25. 671: Failure to export overlapping data from EKA3566

26. bdsImportTapeDigitiserData: There have been major changes to the bdsTapeDigitiserFile program used by bdsImportTapeDigitiserData to re-process the VELA timecode track. This now uses a Phase Locked Loop algorithm to more accurately set the data block start and end times. The system now calculates a VelaQuality based on the number of VELA codes decoded versis the number expected. This is reported as an Import Warning.

27. 770: Filename of imported data states nearly 8 months Added a check to make sure the date range is less than 10 days in one import file.

28. 804: Output SEED data have duplicate blocks at end

29. 805: BDS SEED files in "Jrdseed" Now outputs channels in Station order with a single blockette 50 per station.

30. Source names are now validated and BdsAdminGui uses comboboxes for the available source names.

31. 810: bdsImportData with -dryRun does not issue "data already present" error Added checkif if data is present when -dryRun is used.

32. 808: Dummy channels Added support for a channel "noMetaData" boolean. When this set on a channel it states that there is no MetaData present (Calibrations, Responses, Instruments). When set data can be imported, with a warning, however date export of the appropriate channels is not possible as an error of missing MetaData is given.

33. 79: Add meta data validation checks.

34. 733: Add -startTime and -endTime options to bdsImportData It is now possible to use -startTime and -endTime options with the bdsDataImport program to imports time spans of data from files.

35. 690: default source "Main" for Responses

36. 823: GCF files: Filename time errors when hour is not 0

37. 456: Certain windows/dialogues incompatible with 1024x768 resolution

38. 825: BDS server behaviour under load This has been fixed.

39. 835: Channel DataType field This has been added.

40. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 6. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

4. Restart the BDS system. "bdsRestart"

Release 1.2.2 - 2010-03-16

1. Minor change to BdsImportBlacknestDatabase to modify MetaData source allocation to use new EkaDig2 scheme. Set database initial source data to use "EkaDig2" rather than "Scream".

2. Now handles MetaData for EKA:SHZ around GCF24EKA0811 better by merging MetaData either side for the source Main.

3. Extends the EKA:SHZ:Main and EKA_BHZ_10 data to end of time.

4. Extends the EKA:SHZ:EkaDig2 to cover all time

5. Improved Sensor selection algorithm to share sensors for channels better.

6. 695: preicision of longitude for WRA and YKA

7. Extended bdsImportScream to support source names in configuration file on a per import source basis and modified default to use a source of "Main".

Release 1.2.1 - 2010-03-12

This is a minor update, fixing some small bugs found in the 1.2.0 release.

1. BdsAdminGui: Fixed bug with data viewer where missing block data at the very start and very end was not highlighted as missing.

2. SEED blockette MetaData is now stored in infoExtra to reduce normal data info clutter.

3. BdsAdminGui: Added Splitter bar to DataSelect tab.

4. BdsAdminGui: Fixed bug in DataFiles window where selected channels where not viewed correctly when a File was selected.

5. BdsAdminGui: Changed default filename format used for downloads.

6. BdsAdminGui: Added scroll bars to DataSelect warnings dialog.

7. BdsImportBlacknestDatabase: Modified to ignore "SCREAM.*" format channels on import and extend MetaData endTimw's for EKA to be end of time.

8. 657: Exporting EKA BB data with gaps: (ChannelInfo not matching actual data blocks) partially fixed.

9. 658: wrong number of samples in IMS format data originally GCF from EKA BB array

10. 673: SEED reader "unknown block type: R" Added support for 'R' and 'Q' SEED data blocks and type ' ' padding blocks.

11. 668: AdminGui DataSelect output file navigator GUI error

12. 670: "Update Select Fields" button in DataSelect window resets times

13. 669: output data file is not completed until a warning popup is "OK"d/closed

14. Added endTime to ASCII format MetaData.

15. BdsAdminGui: Improved DataSelect and display to keep segementation of data rather than create multiple channels of data.

16. 690: default source "Main" for Responses

17. Validates user entered time in Respose,Calibration and Instrument "split" operations.

Release 1.2.0 - 2010-03-01

This is a major update to the system, fixing and changing some of the core internal structure.

1. Changes to BDS data file format. This adds channel and segment info to the data blocks to improve the performance of dealing with segments of data when there are missing blocks or in other multi-segmented data applications. Also improved block searching performance and other minor improvements.

2. Major changes to the DataSet class. This is responsibe for gathering the data from multiple channels of multiple segments in multiple files and providing access top the seismic data conatined. This will have fixed many issues with data access.

3. 89: Automatically break data into segments when there are missing blocks

4. 526: DataAccess wild card matching not supported

5. 498: non-synchronously sampled" error when exporting (non-multiplexed) ASCII data over a file boundary

6. 84: Improved operation when there are multiple segments of data

7. 330: Break data into segments on MetaData changes

8. 303: Automatically break data into segments when there are metadata changes

9. 498: "non-synchronously sampled" error when exporting (non-multiplexed) ASCII data over a file boundary

10. Modified the ASCII data format to support Channel Multiplexed: ASCII-CM and Sample Multiplexed: ASCII-SM formats and multiple segments. Multiple segments are implemented by adding a comment line starting with a "#" with channel, segment, time and other information.

11. 338: failure to export EKA Red line

12. 404: tapeDigitiser data import: Delete multiple FM deviation errors. Added InfoExtra storage in BDS data file format in in BDS API to store the extra TapeDigitiser error information. This speeds up data access but still allows full storage of the contents of a TapeDigitiser data file.

13. 385: TapeDigitiser import fails when there is an "empty" data file

14. 371: TapeDigitiser import failing for lack of metadata

15. 336: TapeDigitiser data with huge numbers of errors

16. 597: Add Calibrations name field

17. 513: Metadata overlap error message when modifying "response" information

18. 588: poles/zeros in output SEED files read by rdseed The system now uses base SI units of all data and uses calibrationFactors that will convert the data samples to the base SI units in question. When exporting in a particular data format unit/scaling conversion is done as appropriate.

19. 507: BKNAS format when many poles/zeros

20. Fixed crash in BdsServer when trying to export SEED-MINI data.

21. Fixed problem with SEED sensitivity value.

22. Changed SEED dip angle and gain calculations. Set Stage Number to 0.

23. Changes units operation.

24. Fixed problem with dataImport continuing when SEED data format errors were present.

25. 634: Add support for SEED-METADATA file format

26. Modifications to BdsTest system to help test the above changes.

27. 432: Login window disappears after one failed attempt

28. 433: TCP reset requires to have to be restarted, Access Denied error

29. 488: DataFiles tab shows meaningless import data

30. 537: File channel metadata editing

31. 425: DataSelect tab multi select

32. 427: bdsAdminGui dataselect->dataview usability issues

33. 497: "Update" button on channels window didn't work

34. 430: DataView channel shift, 431: DataView seems not to work when displaying multiple channels A simplified DataView tab has been implemented. This allows the viewing of multiple data channels with multiple data segments, possibly overlapping as well as channels at differeing sample rates. It has a simple autoscaling function.

35. 647: default file name formation fails when station name includes #

36. Numerous higher level MetaData editing functions added to "Options" menus.

37. BdsImportBlacknestDatabase: Added support for importing Pseudo stations.

38. Removed SensorLocations table and added depth and sensor angles to Calibrations.

39. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

40. Note that the BDS data file format has changed and, for simplicity at this stage of development, no compatibility with the previous file format has been provided. So all data files will need to be re-imported.

With this release there are updates to the database Schema to version 5. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:
Shutdown bdsServer: "service bdsServer stop"
Update the BDS software: "yum update"
Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"
Restart the BDS system. "bdsRestart"

Release 1.1.9 - 2010-01-26

1. Changed default import data source to "Main" from "Master".

2. System now sets importUserId and importTime fields in DataFiles information.

3. Preliminary SEED data file import.

4. BdsImportData: added "-sequential" flag. When used it will import the given set of files sequentially into a single import file. This is especially usefull for SEED-TAR format files. The files have to contain the same channels in the same order and be completely sequential in time.

5. 570: SEED Export converter. The system can now export data in SEED 2.4 and SEED-MINI 2.4 formats.

6. DataSelections invovling an array in combination with a set of individual stations did not work correctly. This has been fixed.

7. Data selections within a single file that contained non-synchronously sampled data at different sample rates did not work corectly. This has been fixed.

8. Added -formatList option to bdsImportData.

9. 428: bdsAdminGui long processes need hourglass cursor

10. 104: Add ability to manipulate Sensor Data information and delete sensor data files

11. 405: New Data files window - various

12. 523: Delete data files. System now deletes data files by moving them to a directory "Deleted" on the store in question. These will need to be actually deleted at a later date by an operator.

13. 350: Improve data access performance by elimimanting second getInfo

14. 27: default filename in GUI

15. 417: Time date widget defaults to 2002

16. 424: DataSelect tab usability issues

17. Changed DataSelection system in bdsAdminGui. Now provides wildcard filter on each item and multi-selection restricted to database entries. Note that selection restriction for data ignores the time fields. This is for performance reasons. It would be possible to implement a cache of distinct, time based, channel information to allow selection restrictions based on actual data time periods if required.

18. Changes to use "*" wild carding system in bdsDataAccess and bdsAdminGui.

19. System now checks the "state" parameter on data files before serning the data.

20. Added ability to list default file format extensions.

21. Added BdsDataFormats manual and updated other manuals and documents.

22. A few minor bugs have been fixed.

23. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:
Shutdown bdsServer: "service bdsServer stop"
Update the BDS software: "yum update"
Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"
Restart the BDS system. "bdsRestart"

Release 1.1.8 - 2009-12-14

This is an interim bug release version. The BDS API has changed so that any client machines will need to be updated as well.
This release is now built on Red Hat Enterprise Linux 5.4 and has a new yum repository. The repository change will be automatic, but two yum updates are required, one to update the repository info and one to actually perform the update.

1. 455: Window hiding issue on certain window managers

2. 464: DataViewInfo DataViewChans inconsistent inital state

3. 465: UserInfo tab, User Edit dialog group selection disappears

4. 461: Dialogue window on Changes tab list

5. 411: BdsAdminGui: Update selection lists in channel on updates

6. 414: Database does not keep up with station/channel name change

7. 474: Number of channels incorrectly calculated?

8. 475: Wrong hang & vang

9. Added "datum" import from Autodrm database in bdsImportBlacknestDatabase.

10. Removed wild card from stations in Autodrm.

11. 416: System puts all data files in a single directory Note that this does not update the location of any existing files imported into the BDS system.

12. 401: Change source "Default" to "Prioritised"

13. 460: Inconsistent list behaviour with modify

14. 494: BDS fills /tmp

15. 495: bdsControl exectuable missing from rpms

16. 511: BKNAS instrument number

17. 508: BKNAS format 1st line of Channel description

18. 501: precision of "calib" for BKNI data Note that this bug affeced MetaData imports so values in the BDS SQL database for calibrationFactor could be wrong.

19. 355: Add a test Array/Stations/Channels for performing automatic regression tests The first pass system has been created. This just creates BDRS files at the moment but export tests are generic.

20. Added the "NETWORK" keyword to the BdsAutodrm. This allows the network to be set for test puposes.

21. 492: StationLocation selection in Stations needs Network This has been added.

22. 453: BDS AutoDRM acts as a relay The system now ignores emails from users not registered in the BDS users database. It checks the "ReturnPath" and "From" field headers for the users email address. Errors are looged into the syslog of the bdsAutodrm daemon's host.

23. 514: Interrupted data import results in bad data status. This has been fixed.

24. 525: DataAccess error message Error text changed.

25. 402: Data import error/warning message improvement Warnings are now presented as a comma separated list.

26. 403: BdsImportData: Add "-info" flag. Added -description and -addWarning flags to system. See the documentation in https://portal.beam.ltd.uk/support/blacknest/files/bds/doc/BdsImport.pdf for more information on the formats of the warning messages.

27. 545: bdsImportTapeDigitiserData. Added ability to read jobInfo.tdim if it exists instead of jobInfo.tdi.

28. Added import warnings when blocks are re-ordered. The warning includes a description of the block re-ordering performed in the form of an ASCII string. The format of this string is a list of block ranges and the re-ordered position. For example "0-10:30" means blocks 0 through 10 have been moved to position 30.

29. Added the Blacknest-testing yum repository for test updates. This is not enabled by default.

Release 1.1.7 - 2009-10-02

1. Added support for setting source names from format in bdsImportBlacknestDatabase program.

2. Fixed a bug where the source field was not being used to validate the addition of new ChannelInstruments.

Release 1.1.6 - 2009-09-25

1. Fixed crash on DataFile deletions.

2. 282: multi-multiplexed data in channels 19 and 20 of BDRS_UKNET data Added support for BDRS-MM data files.

Release 1.1.5 - 2009-09-25

1. 381: BOAP connects to localhost. This has been fixed

2. 367: BDRS UKNET 10-Hz data import Added support for BDRS files with 10Hz sampling rate.

Release 1.1.4 - 2009-09-16

1. 338: CalibrationUnits now set from channel name.

2. 389: BKNAS-1 output calibration factor is written where pole-zero constant should be Now calculates the PoleZero constant for the PoleZero output in BKNAS format from the PoleZero table.

3. 385: TapeDigitiser import fails when there is an "empty" data file Added support for zero data length TapeDigitiser files.

4. 390: EKA BB GCF corrupt data This has been fixed.

5. Added, first pass, DataFile and DataChannel API functions and editing ability.

Release 1.1.3 - 2009-09-08

• Removed debug print from bdsImportData with sample multiplexed file import.

Release 1.1.2 - 2009-09-07

• Created new BTimeStamp editor widget. This allows timestamps in the range 0001-01-01 -> 9999-01-01. The TimeStamp 0000-01-01 is used to define a null/unset date. All BDS timestamps now use this format.

• The bdsDatabaseUpdate command should be run to convert all old start of time/end of time time stamps to the new values.

• bdsImportBlacknestDatabase: Now sets the datum to be blank by defaut rtaher than "WGS85" which was incorrect.

• 365: crashed server by asking for equal start and end times This has been fixed.

• 373: Export of WRA64 data fails with "Channel number out of range" fixed by increasing the maxinum number of BDS file streams to 256 from 64.

• Fixed bug that caused bdsAdminGui to crash when displaying data in graph form sometimes.

• 372: Crashed server with WRA_AGSO import This has been fixed.

Release 1.1.1 - 2009-08-17

1. 341: WRA_AGSO alias in bdsImportData "-format" option. This has been added.

2. 343: MaxNumSamples in bdsServer.conf. Changed to 1024 by default.

3. Fixed a bug in calculating the number of datasamples/sample rate in a set of data.

4. 342: "sample rate incorrect" for WRA40 and WRA_AGSO data There was a bug in the WRA-AGSO sample rate calculation routine when crossing a year boundary, This has been fixed.

5. 340: Add -listFormats option to bdsDataAccess and use in bdsAdminGui

6. There was a bug that caused data import to fail with the error "Unknown output format:" this has been fixed.

Release 1.1.0 - 2009-08-06

1. 57: Added snapshotPause call. The API function takes a boolean argument, on or off to set the BdsServer into pause mode for backup puposes. See the BdsServer manual for more information.

2. 57: Added the modeSet API call. This allows the BdsServer to be put into a read-only slave mode. See the BdsServer manual for more information.

3. 57: Added the security group "control". The snapshotPause and modeSet calls require the user to belong to this group.

4. 57: Adde the user "bdsBackup" for backup purposes. This user belongs to the control group but has no other abilities.

5. 57: Added "state" field to DataFiles table to indicate the data files state. This can be set to "importing" or "ok" and can be used for data synchronisation puposes.

6. 270: Added the "Config" tabale to the database to contain configuration parameters. One configuration parameters, "schemaVersion" has been added to this to track the BDS MySQL data base schema version. This allows for automated database schema updates on software updates. Add the command line program bdsDatabaseUpdate command to update the ruuning database's schema to the current version.

7. 253: Change the BdsServer's database user to "bdsMain" and set the older "bds" user for read-only access. This allows external programs to read the BDS's database safely. The database grant's need to be updated with the following commands for this to work and the /etc/bdsServer.conf file needs modification.
   grant all on BDS.* to bdsMain@"%" identified by "bdsMain";
   grant all on BDS.* to bdsMain@"localhost" identified by "bdsMain";
   grant select on BDS.* to bds@"%" identified by "bds";
   grant select on BDS.* to bds@"localhost" identified by "bds";

8. 268: Added support for re-ordering data blocks and deleting duplicate data blocks on import. The "-reorder" and "-deleteDuplicate" flags have been aded to bdsDataImport for this. Currently only the GCF format convertor supports these options, but they can be added to each or all data convertors relatively easily if required.

9. Fixed: 278: cannot add new SensorLocation without saving-and-reentering

10. Fixed: 264: AdminGui update bug in DataSelect window

11. Fixed: 283: extracting "filename time" from EKA SCREAM data files

12. Fixed: 289: AdminGui bug for a particular GCF file - "QG Vector::insert: index xx out of range"

13. Fixed: 291: BKNAS1 format

14. Fixed: 306: AdminGui editing channel responses

15. Fixed: 265: No. of samples in 2nd segment of 2-segment IMS exported file

16. Fixed: 286: sampling rate for EKA GCF data

17. Fixed: 210: TapeDigitiser import two-tape sets

18. 311: On importing TapeDigitiser files keep original file name The BdsImportTapeDigitiserData program now lists the actual file import in the Description field.

19. 325: Fixed: Database description entries should allow the "' " character

20. 336: TapeDigitiser data with huge numbers of errors We have improved the perormance of the BDictString class to help this. Howvere the number of errors produced by the tapeDigitisation process needs to be looked at.

21. Added support for Station Aliases. These can be used to rename the pusedo stations used for extra data channels that are not associated with any physical stration, to the Array name as well as for other puposes. The Aliases are only normally used in data output formats.

22. Added support for MetaData sources that match the data sources. This allows different MetaData parameters to be attached to different data sources.

23. Added support for Source Priority to decide which data Source to select if a data source of "Default" is used.

24. General bug fixing.

25. Modified bdsImportBlacknestDatabase to suit more recent Blacknest Autodrm database schema.

With this release there are updates to the database Schema. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

1. Shutdown bdsServer: "service bdsServer stop"

2. Update the BDS software: "yum update"

3. Update database users: "mysql -uroot -p<root-password> < /usr/bds/bdsSql/permissionsFix.sql" (Special fix to database user names)

4. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

5. Restart the BDS system. "bdsRestart"

Release 1.0.7 - 2009-06-03

1. BDRS data converter: Added support for reading files with corrupted first block and some other data validation improvements.

Release 1.0.6 - 2009-06-01

1. 215,220,225,229: VELA timecode track, end of year problem. If a new tape was started when the VELA time code generator had gone beyond the end of the previous year, the year would be decoded incorrectly. This has been fixed.

2. Added data converters for WRA-40, WRA-64 and WRA-AGSO data formats.

3. Added "-ignoreCorruptions" option to BdsImportData program to fix data files by ignoring corrupted blocks. The data converters will scan for the next valid block when an invalid block is found on a byte by byte basis.

4. Added "-dryRun" option to BdsImportData program to perform import test without actually importing the data.

5. BdsImportData now as a "-warnings" flag which will list the data import warnings. The warnings are also added to the BDS MetaData for the data file so that they can be investigated at a later date.

6. Various data import validation improvements.

Release 1.0.5 - 2009-04-02

Minor bug fixes:

• 212: BDS Instrument name Metadata issue. This was a bug in the BDS server when there were multiple instruments used on a
  channel at different time periods. The system was choosing the first entry rather than the one for the time period in question.

• 217: BdsAdminGui: Adding new stations problem

• Added the utility program bdsDataFileCreate. This program can create test sensor data files in any of the formats supported by the BDS system with write data convertors.

• Added the ability for bdsImportData to import sensor data files in the BDS format.

• Made boapns daemon run as the daemon user by default.

Release 1.0.4 - 2009-03-27

Minor bug fixes:

• Bug in BdsImportData when ignoring missing data blocks. This would cause a segment fault.

• 195: Fix to TapeDigitiser import. VELA timecode track, end of year problem. This problem was caused by problems in the VELA timecode track near the year end on tape EKA 3616.There was some curruption in the VELA timecode track and then a missing bit of VELA time code followed by and incorrect year wrap. The software could not cope with this unusual circumstance. We have added some code to handle this case.

• Added Channel Aux field to BKNAS channel output lines based on Sheilas BKNAS modification.

• Modified BdsAutodrm so that all channel names have the ".*" postfix added so that channel types given will match all channel names of that type irrelevent of the AuxId.

Release 1.0.3 - 2009-03-23

Minor bug fixes:

• 158: End of day/End of year crossing time stamp errors in formatted data output. The time stamps listed in the data output where set to the user requested time not the actual data times from the blocks. This has been fixed.

• 187: Problem with bdsImportTapeDigitiserData validating meta data. This has been fixed.

Release 1.0.2 - 2009-03-16

Minor bug fixes:

• Fixed bdsImportScream file handle leak.

• Fixed bdsImportTapedigitiserData to find supporting programs in standard path.

• Improved Pole/Zero editor.

Release 1.0.1 - 2009-03-10

Minor bug fixes.

• 145: IMS data spanning midnight: There was a problem with the second sections start time. This has been fixed.

• 148: Loaded WOL but bdsAdminGui doesnt: We have added a button "UpdateSelection" to update the networks/stations/channels and sources selection information from the database. This can be used when new data has been added.

Release 1.0.0 - 2009-03-04

This is the first release of the BDS software.

This site including all text and images is © 2023 Beam or its clients.

Websys © BeamWeb 2023