BDS Release Notes



Release 2.1.17 – 2020-05-22

Minor bugfix release.

  1. BdsImportStreamGcf: If a system crashed it was possible for the block number file to be empty on the next reboot causing bdsImportStreamGcf not to backfill to where it had been before the system crash. We now synchronise the file to disk on each block number update. The method chosen does have a slight performance overhead, but not as much as a full atomic file update would have although there is stall a very small chance that it could fail if the system crash occurred within a very small time window.

Release 2.1.16 – 2020-04-24

Minor bugfix release.

  1. BdsImportStreamGcf: This was being started as the root user rather than bds user. The systemctl scripts now starts it running as the bds user. Note its data files will need to have their ownership changed to “bds”. These are the files in the directory /var/cache/bdsImportStreamGcf and /var/log/bds/bdsImportStreamGcf.log by default.

  2. bdsImportStreamGcf: Produce error when blocknumber state file cannot be read although it is present and retry server connection.

  3. bdsImportStreamGcf: Debug printout when warning "does not have all backfill blocks" happens.

  4. bdsServer: When trying to export multiple channels of data in a particular CM format and there is no data from one or more channels of data over the timespan, the system would return the error: “Error: No data segments are available in file for: …”. We have changed this behaviour to only return the error if there is no data in any of the channels selected. We have also introduce the possibility of a 0 numBlocks and zero time period segment to allow this to be used.

Release 2.1.15 – 2020-03-03

Minor bugfix release.

  1. BdsImportStreamGcf: Turned of block level debugging that was creating large log files “blockdump-*.txt”.

  2. BdsImportStreamGcf: If the remote SCREAM server system died, such as through a power off, the BdsImportStreamGcf did not determine that it was no longer connected and did not receive any more data blocks. This has been fixed.

Release 2.1.14 – 2020-01-14

Minor bugfix release.

  1. 1594: Failure to export SEED or miniSEED. Fixed the export of SEED formated data. and metadata.

Release 2.1.13 – 2019-11-11

  1. BdsServer: Modified usage of DataStoreMin: configuration parameter. Previously this define the minimum disk space required per open write file in Gigabytes. It now defines just the minimum disk space required in Giagbytes.

  2. BdsServer: Added the DataStoreFileMin: configuration parameter. This defines the disk space per open write file in Megabytes on top of the DataStoreMin space.

  3. BdsAutodrm: 1586: Have AutoDRM not break data into segments at file boundaries. The bdsAutodrm now uses the DataFlagMergeSegments and DataFlagClipDataToChannels optiond when opening a set of sensor data files. This merges the data into contiguous segments and clips the data so all channel data starts and ends at the same time.

  4. 1585: Saving data from GUI in IMS-integer format. The format IMS2.0:INT now returns the data in INT rather than CM6 format.

Release 2.1.12 – 2019-10-25

  1. BdsServer: Fix some bugs where a BdsServer crash can occur if there is an error opening a BdsDataFile. Also log the error that has occurred for debugging.

  2. TapeView – Ability to delete all export markers on single button press added.

  3. TapeView – Ability to spilt exported output into multiple ims files

  4. TapeView – Ability to toggle export valid state in the export markers table with the “V” key added

Release 2.1.11 – 2019-10-15

  1. BdsImportStreamCd: This now ignores incoming packet validation errors, such as the year 2000 time stamp issue rather than passing an error to the cdtools library. This allows bdsImportStreamCd to continue importing the stream without a reconnection time out on each bad block.

Release 2.1.10 – 2019-09-30

  1. Added “systemctl daemon-reload” to RPM installs.

  2. 1571: BdsImportScream: Crash or abort on sample rate change. Now reports sample rate change but if -ignoreCorruptions is set continues to import in bdsImportData and bdsImportScream.

  3. BdsImportStreamCd: Improved debug print outs.

Release 2.1.9 – 2019-09-27

  1. Changes to bdsRestart to not start/stop the old bdsImportCd and bdsImportScream daemons by default.

  2. bdsImportGui - User interface enhancement. Middle mouse button can control scroll speed.

  3. 1568: BdsImportStreamCd: Import channels mixed data file order. When more than one set of additional channels was added in a realtime stream (CD1.1 or GCF) then the MySQL DataChannels dataFileChannel number was not set correctly and overlapped previously added channels. This has been fixed.

Release 2.1.8-1 – 2019-08-16

Minor bug updates.

  1. Changes to bdsRestart to stop client daemons first before restart main bdsServer process.

  2. Modified BdsServer to gracefully terminate on the SIGTERM signal so that it flushes and open BDS data files to disk so that data blocks from on-going imports are not lost. This will make sure it does not lose blocks when bdsRestart is used.

  3. Modified the BdsPython API to work better with the BdsDataFile* data convertor classes. The API now better handles passing arguments by reference in and out of functions.

  4. Realtime import: Now sets times on data channel files to actual contents when closed so across 24:00 boundaries are correctly handled and clears the state flag to ok on file write aborts.

Release 2.1.7-6 – 2019-07-09

Minor packaging updates:

  1. Now sets the user/group of /var/cache/bds* to bds/bds.

  2. Runs “ldconfig” on library installs (includes libdata-gcf and associated packages) so that libraries are available.

Release 2.1.7 – 2019-07-05

Addition of BdsImportScreamGcf system and some minor bug fixes.

  1. TapeView’s IMS export facility was still not working correctly. This has been fixed.

  2. Data availability was clipped to the start of the current day. It is now clipped to the end of the current day to better show real-time data availability.

  3. Added the new BdsImportScreamGcf system for realtime import of SCREAM GCF streams.

  4. BdsPython: There we some issues with accessing BString objects as Python strings. this happened after the 2.1.5 additions of BList API access. This has now been fixed with all BStrings now accessed as native Python strings. So any code that used the <string>.retStr() syntax to return a Python string no longer needs the .retStr() member function call.

This release is in the bds-testing repositories.

Note on Redhat7/Centos7 this now expects the rh-php72 PHP/WEB server system. You will need to add the appropriate repositories (Using centos ones):

Then after the BDS system has been updated you can remove the older rh-php70

Release 2.1.6 – 2019-03-11

A minor bug fix release and a release for Fedora29 made.

  1. TapeView’s IMS export facility was not working correctly. This has been fixed.

  2. Released for Fedora29.

Release 2.1.5 – 2018-10-29

  1. BdsPython: Added Bstring class and removed BList::append(BList) function from API.

  2. Documentation updates.

  3. SEED Converter. Now outputs depth field as “%05.0” (ffff. Or fffff) rather than “%05.1” (fff.f) when the depth is greater than or equal to 999.95 so that hydro-acoustic depths can be passed. Note this will fail in verseed tests.

  4. BdsImportStreamCd: Now sets the numBlocks and numSamples fields in the DataChannels table to 0 for realtime data.

  5. BdsImportStreamCd: It has been found that occasionally, in some CD1.x data streams, the channels within a CD block have differing time-stamps from each other. The BdsImportStreamCd will now normally send the combined channel DataBlock’s to the bdsServer, as before, unless it sees that the timestamps of the individual channels do not match. In this case it creates and sends one BDS DataBlock per channel for the particular CD blocks in question.

  6. BdsDataFileCd: Reading all channels at once from CD1.x files has been disabled due to time stamp issues in some data streams where some channels in a block may have different timestamps to other channels in the same block.

  7. BdsSendStreamCd: Updated this tool to be able to send CD1.1 binary files through CD receivers such as bdsImportStreamCd.


Release 2.1.4 – 2018-09-11

  1. bdsDataFileDump: Decode CD1.x channel description(mode) and status blocks.

  2. 1543: I52 bdsImportStreamCd shows 27 channels - should be 10?. There were some issues with the dataFileChannel number being set incorrectly when channels were added to an existing stream within the day. This has been fixed.

  3. BDS data files where large when importing CD1.x data. This was due to duplicated block metadata. This has been fixed.

  4. 1536: BdsTestAuto: Ignore data channels that are not marked as seismic. Only runs tests with genuine seismic data files.

  5. Changed CD1.1 data block metadata names from "mode" to "cdDescription" and "status" to "cdStatus".

Release 2.1.3 – 2018-08-02

This is a minor update with changes based on Blacknest's feedback of the 2.1.2 release.

  1. The CD1.x data converters used in bdsImportData, bdsImportCd and bdsImportStreamCd were not saving the CD1.1 channel status field into the data blocks. This has been fixed.

Release 2.1.2: 2018-08-01

This is a minor update with changes based on Blacknest's feedback of the 2.1.1 release.

  1. bdsImportTapeDigitiserData: Segment fault when -verbose is used. Fixed this issue.

  2. BdsWeb: Updated to support latest WebSys API.

  3. bdsImportStreamCd: Added parameter IgnoreUnknownStations so that warnings on unknown stations is not logged.

  4. BdsLib: Bug in bdsDataInfoMergeFlatten() that causes a segment fault fixed.

  5. BdsDataFileBds: Now uses last DataInfo in file with getInfo() to allow for channel appending.

  6. BdsImportStreamCd: With the above two changes now a a CD1.x stream can be imported when the blocks have different channels to previous imports within the same day file.

  7. BdsImportStreamCd: Added open file list/cache to improved efficiency when backfilling. The MaxFilesOpen parameter defines the max number of open files. This should be around 3 x the number of streams in use.

  8. Fixed issue with crash on reading null sized CD1.x files.


Release 2.1.1: 2018-06-25

This is a minor update with changes based on Blacknest's feedback of the 2.1.0 release.

  1. bdsImportStreamCd: Modified service startup file to start daemon as the user bds.

  2. 1506: BdsImportStreamCd: Crashes after a time of running: Added dump of crash info to the log file. need to look at this if it crashes again.

  3. 1516: BdsImportStreamCd: Add config option to disable location code usage. Added the configuration file parameters: "Stream0.locationIgnore: 1" which will tell bdsImportStreamCd to ignore location codes when formulating the channel name.

  4. 1513: BdsWeb: Trace display not working on centos7. Modified RPM to include packages gnuplot and netpbm-progs. Added setup script: /usr/bds/bdsWebSql/bdsWebSetup.sh to setup selinux context's.

  5. 1514: BdsWeb: Clean up temporary web files. Data are temporarily stored in /usr/bds/www/files/tmp. We have added a cron script /etc/cron.d/bdsWeb.conf to clean files older than 2 days in this directory.

  6. 1515: BdsWeb: Integerate standard Blacknest help pages in RPM release. The Blacknest user manual and standard BDS documentation has been added to the default BdsWeb information.

  7. 1529: BdsImportStreamCd: Hydroacustic data differing block sizes on export.The jitter on the timestamps due to GPS time corrections is now handled within the BdsDataFileBDS BDS seismic data file converter. When data is exported the start and end times of consecutive blocks are checked to see if they are very close in time (within 1% of a blocks time period). If they are they are deemed to be the same then the endtime of the previous block is set to the startTime of the next block.

  8. 1521: BdsImportTapeDigitiserData fails with segfault. Fixed this.

  9. Added a new BDS regression test system to the bds-test package that uses real data and compares exported data sets from one BDS version to the next.

  10. Added the bds-test-data package that contains a base set of data for the BDS regression test system.

  11. Documentation updates.



Release 2.1.0-2 – 2018-04-05

This is a minor update that is built with an updated idc-cdtools package for Redhat7 and Centos7. It fixes a crash issue when using on Centos7.

Release 2.1.0 – 2018-02-12

This is a major update adding real-time stream capture and storage capabilities.

  1. The BDS data file access code has been changed to support appending data to day files. As this code is used for all BDS seismic data storage it will need a fair degree of testing. Some changes to make the files more robust in the event of the Bds server crash have also been made. See the manual: https://portal.beam.ltd.uk/support/blacknest/files/bds/doc/BdsDataFile.pdf for more details.

  2. BdsServer: Added support for storing and appending data into day files for streaming daemons.

  3. BdsImportStreamCd: This is a new daemon program to support the storage of the real-time CD1.x cdtools streams into the BDS system. See the manual: https://portal.beam.ltd.uk/support/blacknest/files/bds/doc/BdsImportStream.pdf for more details.

  4. BdsDataFileBdsDump: Added this tool to debug the BDS Data File format.

  5. BdsImportScream: Fixed file descriptor leak.

  6. Updated the BDS system to use the newer systemd system initialisation system.

  7. Sets database access mode to allow timestamps of 0000-01-01T00:00:00. Newer database installations default to disabling the ability to store timestamps of this value.

  8. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

There are no updates to the database int this release, it uses schema to version 16. If you are updating from a version pre 2.0.38 you will need to update the database howvere. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

Release 2.0.39 – 2017-12-14

This is just a partial package release for RHEL7 to fix a couple of bugs without changing the main bdsServer running system. It includes just the bds-importScream and bds-importCd packages.

  1. The CD1.x file converter had a restriction of up to 20,000 Bytes of data before it would return an error. We have increased this limit to 100,000 Bytes.

  2. BdsImportScream: Fixed file descriptor leak.

Release 2.0.38 - 2017-05-18

Minor bugfix release.

  1. bdsDataFileDump: Fixed loop during data max/min calculation with channel ordered data files.

  2. BKNAS-3.0 format, modified calibration factor field to be of format "%12.9e" for greater resolution and to handle exponents.

  3. tapeView: When export cursors are used the sample time is also exported to the .tdi file.

  4. tapeView:  Multi-recording marker meta-data now includes the ability to specify a start time for each recording segment.

  5. Added copy constructors to Python API to allow the return of copies of BDS API objects from Python functions to work around issues when objects from BList's and BArray's are returned from functions that delete the lists.

  6. Updated bdsDataAccess to use latest gnuplot when plotting graphs.

  7. Added a BdsPhp API to allow PHP programs to access the BDS system directly using a PHP version of the BDS API.

  8. New BdsWeb system.

  9. BdsServer: Improved the perfromance of SQL lookups when regualr expressions are not used.

  10. Database: Added some indexes to the DataChannels table to speed up accesses.

  11. BdsMetaData: added "Change Station" function.

  12. BdsWeb improvements from Blacknest feedback.

  13. DataAvailability system nowrounds to nearest year,week,day or hour.

With this release there are updates to the database Schema to version 16. Actually it is just database information inserts. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

Release 2.0.36 - 2016-09-20

Minor bugfix release.

  1. bdsAutodrm: Fixed an issue where an E-MAIL parameter with additional text caused the bdsAutodrm to refuse to send data.

  2. bdsDataFileDump: Added ability to print min and max data values.

  3. Added support for BKNAS-3.0 format files that provide I12 formatted data.

  4. BKNAS: Added clip data function to limit data values to the I6 or I12 data formats as needed.

  5. tapeView: Now includes the TapeView program superseding the old tapedigiser-tools package.

  6. tapeView: Now supports multi-recording tapes with export of separate tape recording regions for import into the BDS system. (Note no BDS import support for this yet).

Release 2.0.35 - 2016-08-30

Minor bugfix release.

  1. Fixed bug in BKNAS export which could cause a crash of the bdsServer program.

Release 2.0.34 - 2016-06-22

Major source code structual updates. This release is designed for the Fedora23 and RHEL7 systems and only releases for those systems have been produced.

  1. Uses latest beam-lib.

  2. Pulls in libbeamdsp, libTapeDigitiser and tapeView into BDS source tree so a single SVN tree contains all of the BDS and tapeView code.

  3. Build changes for RHEL7.

  4. Fixed crash in bdsServer when a a set of data was requested in sample sequantial form to a non block syncronised specific end time.

  5. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Main thing here is the update to the latest beam-lib. This changes the error numbers. When errors have occured the BDS sometimes stores an error number along with an error message. After installing release 2.0.34 the stored error numbers will be different to those in the database/sensorDataFiles from previous versions of the BDS.

Release 2.0.33 - 2016-06-09

  1. Initial release for RHEL7.

Release 2.0.32 - Preliminary

  1. Modified the string to time conversion to allow date/times like: 2016-04-22T00:00 ie date/times without the seconds given. This allows that format of date/time to be used in BdsAutoDrm requests and other places.

  2. Added a new example bdsMetaData4.py.

Release 2.0.31 - 2016-03-02

  1. Modified all *Update() functions to set the objects id to 0 when appending. This is to simply API usaqge.

  2. Added support for downloading metadata from the BdsGui program's "Meta Data" tab.

  3. Modified bdsDataAccess to return a single SEED metadata file when multiple channels are selected with the format RESPONSE-SEED format.

Release 2.0.30 - 2015-10-20

  1. bdsImportGui: Bug 1497. Improvements to  useability of scrolling graphical display.  If mouse is over the scrolling area then the auto-scroll speed can now be set via keyboard short cuts. The following keys are supported. Space - Starts and stops scrolling. Home - Sets scrolling speed to zero. PageUp/PageDown & Cursor keys modify the speed in steps. End - Sets speed to maximum. Additionally while in the plot area the mouse wheel button when rotated will scroll the plot forwards and back 1/10 of a single page on each click (Press and hold the CTRL key with Mouse Wheel spin to move a sample at a time).

  2. CSS File convertor: Provide lower level error message on file open errors.

  3. SEED Data Format: Added continuation blockettes in the case of large FAP responses.

Release 2.0.29 - 2015-08-17

  1. CSS data format: If the data format was big endian, s3 (signed 24bit) the twos compliment data was not handled correctly for negative values, yeilding "square wave" looking data. This has been fixed.

Release 2.0.28 - 2015-06-18

  1. 1493: dip and azimuth in dataless SEED files. Changed to use different dip/azimuth algorithem when exporting to SEED.

  2. Added the userGetFromId() API function for all interfaces. This allows the user information for a given ID to be accessed.

  3. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

  4. When exporting small portions of data that has largish original block sizes such that the entire data set is within a single block and clipping the data to the user given start/end times,  the bdsServer did not account for the start sample offset when calculating the end sample. This resulted in an error "Internal error - trying to move to end sample...". This has been fixed.

  5. When exporting data it was not always clipped to the user endTime supplied, the data continued to the end of the current block. This has been fixed.

  6. bdsImportGui: Bug 1494 Item 3. The plot select time/date input now reflects the current position of the plot.

  7. bdsImportGui: Bug 1494 Item 2. The default export file is now data-<exportStartTime>.<ext>

  8. bdsImportGui: Bug 1494 Item 1. New functionality. Export Dialog now alway user to export a specified duration from the current plot location.

  9. 1495: AutoDRM returns extra channels when "chan_list" not given. Fixed

  10. bdsAutodrm: Added new help information.

Release 2.0.27 - 2015-05-19

  1. CSS data format: If the dirname field is blank now uses current directory for data file names.

  2. CSS data format: Fixed sample rate problem with some data. This was due to incorrect endTimes in *.wfdisc files and an incorrect number of samples calculation for the last block of data. The CSS converter now calculates its own end time based on the starttime, number of samples and samplerate.

  3. Added the "-channelRename BHZ=BHZ_US,BHE=BHE_US" option functionality to bdsImportData.

  4. bdsImportGui: The X-Axis of the data plot can now be viewed as either by time or sample number offset

  5. bdsImportGui: Previously the auto colour button simply auto coloured the plot traces. The button now toggles state allow the user to revert the traces colours to the default

  6. bdsImportGui: The maximum speed of the auto scroll feature has been increased

  7. bdsImportGui: Display of channels numbers. Channel numbers now start at 1 rather than 0

  8. bdsImportGui: New dialog added ExportDialog that allows user to specify export ranges.

  9. bdsImportGui: New action added to the edit menu to allow a user to reset some plot parameters to default.

  10. Added selection range to dataChannelGetList() function.

  11. bdsImportGui: Fixed bug where reset traces to default did not revert to default colours and gain.

  12. bdsImportGui: Select plot time widget now limited to actual available time range

  13. bdsImportGui: Added menu item to view list of overlapping blocks.

  14. bdsImportGui: When opening *.bdrs files the program can automatically create a jobInfo.tdi file importing the date and tapeNumber files from an existing jobInfo.log file.

Release 2.0.26 - 2015-04-22

  1. Adds support for the CSS data format for reading CSS files.

Release 2.0.25 - 2015-01-29

  1. BDS LAC data convertor allows conversion of 5Hz sampled data.

  2. bdsImportGui: Support fo LAC files sampled at 5Hz added.

  3. bdsImportGui: Fixed bug where input data overlaps would cause application to crash

  4. bdsImportGui: Added command line options support the startup using scripts. Enter bdsAdminGui -help for details

  5. bdsImportGui: Added markers to show location of overlapping data

  6. bdsImportGui: Fixed problem with file export

Release 2.0.24 - 2014-12-10

  1. Modified lowpass filter in VELA decode to improve VELA decoding as used in AD22 data format convertor.

  2. bdsImportGui: segment fault on opening files with no data. This has been fixed.

  3. Added LAC_BDRS format data converter.

  4. bdsImportGui: Added support for LAC_BDRS format data files from the DTPS Digital tape import project.

  5. Fixed segment fault in CD1.X format conveter when recovering from currupt frames.

  6. 1479: YKA tar SEED failing with "-sequential" option. This has been fixed.

  7. Fixed problem with BLists in Python interface. This caused issues when trying to access Response Metadata from the BDS data convertors in Python.

Release 2.0.23 - 2014-10-22

  1. Added support for AD22 format file import.

Release 2.0.22 - 2014-10-09

  1. SEED: Added code to set block size if different from default 4096 bytes.

  2. SEED: Added code to round timestamps to 10us due to some files having incorrect timestamps. This is included if the API's ReadOptionValidateCorruptions flag is not set ie. if the bdsImporData flag "-ignoreCorruptions" is set the the timestamps are corrected.

  3. The bdsImportData's "-source" option did not override the default "Main" source when a data file had the necessary metadata to supply the channel information.As I expect that BN would like to use the "-source" option when importing data from other sources I have now implemented this.

Release 2.0.21 - 2014-09-12

  1. During export the system allowed a certain degree of timestamp jitter before segmenting the data into chunks. This caused problems with some YKA SEED data that had very small missing chunks of data or timestamp jitter. We have removed this functioinality so data will be segmented if the start/end block time do not match exactly.

Release 2.0.20 - 2014-06-13

  1. The bdsImportCd program did not ignore issues due to data file curruptions leading to "Excessive authSize in packet" warnings. This has been fixed.

Release 2.0.19 - 2014-05-21

  1. BdsImportScream: Now uses FileAgeMinimum = 864000 by default if not configured in the bdsImportScream.conf configuration file.

  2. BdsImportScream: Adding logging of crashes.

  3. BdsImportCd: Adding logging of crashes.

  4. BDS CD1.x data convertor "Excessive authSize in packet" error. Have fixed this it was due to over agressive validation checks.

Release 2.0.18 - 2014-04-08

  1. BdsImportCd: This refused to import CD1.1 files with multiple data segments due to large chunks of missing data. This has been fiexd.

Release 2.0.17 - 2014-04-03

  1. BdsImportData: The -printBlocks function is now applied after blocks have been deleted through the use of -deleteDuplicates.

  2. BdsImportCd: There was a bug that effects multi-channel CD1.x file import when there are different sample rates per channel. In this case some data blocks near the end of the file would not be imported. This has been fixed.

  3. 1471: IMS "STA2 line for hydroacoustic data  Modified STA2 elevation and depth string formats. Depth is now output in km. String formats of floating point elevation and depth now take into account the "-" sign and reduce the number of digits after the decimal point to cope with larger numbers. If a number is outside the range printable and error is returned.

Release 2.0.16 - 2014-03-19

  1. 1420: wrong number of samples in IMS header (hydroacoustic data). This has now been fixed. There has also been improvements to BdsServer data export colation. This will result in more accurate data start/end times and calculated sample rates for variable sample rate data and variable sample rate data blocks that stradle file boundaries will be joined into one segment properly.

  2. 1413: GCF Unsupported block type: -1. This was not completely fixed. It has been now.

  3. Added ability to ignore certain blocks during import with bdsImportData. The option "-ignoreBlocks 69,70,71,72,73,74,75,76,77,78,79" allows this. Note this only works for GCF format files at the moment but it can be exteneded for other file formats. The block numbers are those listed bybdsImportData after any sorting has been performed.

  4. BdsImportData: Added -printBlocks option that will print the block headers after sorting and after -ignoreBlocks processing. Only supported for GCF data format at the moment. Can be used with -dryRun for block information.

Release 2.0.15 - 2014-03-04

  1. Added MaxSearchTimePeriod configuration parameter and increased max import time from 10 to 31 days.

  2. 1420: wrong number of samples in IMS header (hydroacoustic data). Fixed numSamples output when data block sizes just stradle endTime.

  3. 1413: GCF Unsupported block type: -1. Have modified the GCF data convertor to ignore corrupted data blocks if ReadOptionValidateCorruptions is not set. ie with "-ignoreCorruptions".

  4. 1418: cloning Response in AdminGui. Fixed issue where window was readonly when cloning or other actions called after viewing an item.

  5. 1265: configuration file for AccessGroups. Added ability for users in the group "dataAccessAll" to access all data. The database scripts have been set to add the users testAdmin, testAdd, bdsImportCd and bdsImportScream to belong to this group.

With this release there are updates to the database Schema to version 15. Actually it is just database information inserts. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

Release 2.0.14 - 2014-01-29

  1. When a Channel is updated or added the channel, channelType and channelAux parameters are checked and setup if they are null from information in the values supplied.

  2. The BdsImportCd program could crash with certain heavily currupted data files. this has been fixed.

  3. CD1.1 file converter now ignores year 2000 blocks if -ignoreCorruptions is used.

Release 2.0.13 - 2013-12-19

  1. Added PythonAPI support for accessing FIR coeficients.

Release 2.0.12 - 2013-12-16

Minor bugfix release:

  1. 1408: bdsManage "clean" deletes all of Logs and Changes entries Fixed bug with cleaning Changes and Logs.

  2. Fixed bdsCloneStation.py to allow multiple channel clone.

  3. 1403: "I4 not supported" for CD-1.1 data from I50. CD1.x added support for "I4" data type.

  4. GCF converter: If a file has a block full of 0's it is now ignored if -ignoreCorruptions is used. Peviously it was allowed, but not ignored.

Release 2.0.11 - 2013-11-28

Minor bug fixes.

  1. Fixed a bug in SEED export of responses where multiple copies of the response data was apended to previous values.

  2. Response import: now supports IMS responses with no space before the "theoretical" keyword.

  3. 1397: "load from file" button still active when "Response" window is in "View" mode. Response "Load form file buton" now disabled in view.mode.

  4. Fixed a bug in datetime setting from strings in the forat as used in CD1.1.

  5. Added the "measured" flag to responses to allow import and export of IDC responses of this type.

  6. Added the "sampleRate" parameter to responses to allow import and export of IDC FIR responses with this info.

  7. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 14. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. This includes changing the Responses named "Sensor" to "Overall". So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 2.0.10 - 2013-11-14

Major changes to Response system to allow multiple responses and FIR response types.

  1. bdsMetadata: Added ability to delete ChannelInstruments.

  2. bdsImportData: If trying to import a zero length data file or one with no actual data in it the bdsImportData program would still interact with the bdsServer resulting in a large database search for all of data which increased the memory usage of bdsServer enormously. This has been fixed.

  3. The station and channel boxes in the bdsGui programs selector boxes could be very slow with large data sets imported. The speed of these has been improved.

  4. GCF converter: If a file has a block full of 0's it is now ignored if -ignoreCorruptions is used.

  5. 1390: Data deletion and tidy up of "waste basket" The bds's clean() API call now will delete datafiles that have been previously moved to the Deleted directory. It will delete files, Changes and Logs older than 1 year. The bdsManage command provides a command line access to this function.

  6. 252: Database "backout" and "restore" facility Modified database backup system to store compressed backup files. Added the ability to restore portions of the database. The system supports: all – Restores all tables, admin – Restores admin tables, metadata – Restores metadata tables, data – Restores sensor data tables. The bdsMetadata and bdsManage command line tools can be used to access this functionality.

  7. BdsGui applications now ignore the text case when pattern matching network/station/channel/source names.

  8. Support for the FIR response type has been added.

  9. Support for import and export of muliple stage responces has been added.

  10. The Response system has had extensive modifications to tidy this aspect up. This includes new response format names, an updated Response format converter and the renaming of stage 0 from Sensor to Overall. Response operation is described in BdsResponses.pdf.

  11. Note that bdsMetadata scripts that specify a Response will need updating. There is now a stage number field, a stageType field and a description field. Also the name the primary response should be renamed "Overall" rather than "Sensor".

  12. bdsMetadata has been exteneded to support importing a multiple stage response file.

  13. Support has been added to the  IMS and SEED data convertors for  FIR filter export and multiple stage response export.

  14. Fixed a bug when importing SEED-TAR files with the -sequential option. Now allows this operation to function.

  15. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 13. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. This includes changing the Responses named "Sensor" to "Overall". So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 2.0.9 - 2013-09-10

  1. 1380: BdsImportScream: Directory layout. The bdsImportScream program now supports GCF files in any format of directory structure. The system now uses the files modified time to determine which files it has already imported. There is a new configuration option in /etc/bdsImportScream.conf, FileAgeMinimum, that defines the minimum age in seconds of a file to import. This is by default set to 864000 so that only files older than 1 day are imported. This allows the SCREAM system, to finish back-filling files.

  2. BdsImportScream. Previous versions did not import files that had missing blocks or blocks in the wrong order. The new program will import these after re-ordering the blocks as needed.

  3. BdsImportScream. This now logs each file imported with an Ok or Error status to the log file /var/log/bds/bdsImportScream.log as per the bdsImportCd daemon.

  4. 1372: Improve implementation of ASCII data log support. The system now supports the storage and retreival of blocked ASCII LOG data in channels contained within the seismic sensor data files. The ASCII log information is stored in BDS data blocks within the info["log"] field. A special channel type "log" (Note this has changed from "ascii" in the previous test version) has been introduced. This channel type can be used for LOG data only channels (LOG data can be attached to normal sensor data blocks if needed). We have added a data file conveter DataFileLog which supports the LOG-SCREAM data format. this is the format used for the SCREAM systems data log files. The DataFileLog can both read and write files of this format and so can be used for import and export of log data. The bdsImportData program will automatically import LOG channels from multiplexed GCF data files. It can also be used to importSCREAM log format data files directly. See the bdsImportData manual for more information. LOG data can be exported using the bdsUserGui, bdsAdminGui and bdsDataAccess programs in the same way as other data when the "LOG-SCREAM" data format is selected. The bdsGui programs also have a "Data Log" button to show the entire data log for the given time period in a pop-up window. Note previous logs imported into the BDS test system are not compatible with the new system.

  5. Fixed bug where all Sensors where selected when no channels were matched. This also affected Digitisers and ChannelInstruments.

  6. 1362: AdminGui crashed because of too many missing blocks in WRA40 file Have speeded up the "Data Info" window update.

  7. bdsImportScream. Added support for LOG file import.

Release 2.0.8 - 2013-07-02

  1. 1369: "1 not supported" in CD-1.1 import of I52 post-upgrade data Support CD1.1 data format with data transformation 1.

  2. 1373: mismatch of channel with sys/stream ID in output from bdsImportData. This has been fixed.

Release 2.0.7 - 2013-06-19

  1. 1360: GCF import loses station/channel name in database entries This has been fixed.

  2. bdsImportData now accepts channel definitions of the form "=ZIKTW0:5503Z2" to ignore channels in multi channel GCF files.

  3. Added the "ascii" channel DataType. This is used for ASCII log streams and the like.

  4. 1363: FAP output. Fixed the bug where FAP responses were output in the old IMS format.

  5. Fixed a bug where importing Muliplexed GCF files, where a data stream contained ASCII status blocks has been fixed. The GCF data file convertor will now add psuedo channels (The same GCF systemId:streamId with "_log" appende) for any channel containing ASCII data blocks. These channels will contain just the ASCII data with the original channel containing only the seismic data. The bdsImportData's -channel connand can now ignore these channels or allocate them to a BDS channel of appropriate name/type.

  6. ASCII data is stored in a standard BDS data stream in the "info["ascii"]" feild. The number of samples in the channel data is set to 0. This data can be viewed by reading the streams data blocks and accessing the inifo["ascii"] field.

Release 2.0.6 - 2013-05-23

  1. Added support for importing channel multiplexed GCF files. The GCF file converter has been updated to support this.

  2. bdsImportData. The "-channels" syntax has been extended so a particulat GCF stream can be imported as a particular channel using channel information like: WOL:BH1_00=ZIKTW0:5503Z2. This means import the GCF stream with systemId=ZIKTW0 and streamId=503Z2 as the BDS Station=WOL Channel=BH1_00. Also if the BDS Station and Channel is not given then import of this channel is ignored.

  3. libgcf2 has been updated to version 2.2.27. This fixes a bug where text status blockes were not processed correctly.

  4. Modified IMS data format output to match IDC's change. In the FAP2 output block the number of triplets has been redefined to be an integer of size 5 and the FAP table is now formated as 12.6f 12.6f 12.6f.

  5. Note that with the Python version on RHEL5/CentOS5  exeptions have a different syntax to the BDS examples provided.

Release 2.0.5 - 2013-04-10

  1. 1347: AutoDRM format issues Have added support for IMS2.0, IMS2.0:CM6 and IMS2.0:INT format names and allow "TO" to be in lowercase.

  2. The BDS API call calibrationsGetList() was not expaning Arrays to the set of Stations. this has been fixed.

  3. 829: default filenames for miniSEED and dataless SEED. BdsGui programs now use the default extensions of "seeddata" and "seedmetadata" for SEED-MINI and SEED-METADATA exports.

  4. 780: be able to change "source" of all channels in a file at once The DataChannel edit window now provides a combo bix for the source selection rather than a free from text field.

  5. Added Python API and some Python API example to the system.

  6. Added the bdsImportGui application. This supports the viewing of DigitalTape BDRS data files. It can be later expanded to support BDS data file import.

  7. 1334: bdsMetadata let pole-zero or FAP filename be variable. This did not work for FAP files. Fixed.

  8. 1164: in addWarnings, if no endTime is specified, it defaults to 0000-01-01. With multi-session TapeDigitiser imports the addWarning system now sets the start/end times for each session separately.

  9. Fixed problen with CD1.0 files where data format's authentication field is set to a value other that 1 or 0. Also improved error recovery.

  10. BdsAutodrm: Now ignores the case of the given format.

  11. Added channel name comments to SAC response files.

  12. 1358: IMS response output - space for "auxiliary identification code". Fixed.

The BDS system is now available in 64bit as well as 32bit versions for RHEL6 and Centos6. When updating the system will update based on the packages that are already installed. To convert a 64bit system that is running bds 32bit packages there is a script bdsUpdateTo64.sh which is in the /usr/bds/config directory as part of this release. A 64bit BDS installation is required on 64bit systems where Python programs using the BDS API need to be run.

Release 2.0.4 - 2012-12-10

Minor bugfix release.

  1. bdsGui programs: When converting a PoleZero response to an FAP table from response Metadata, the system would use a calibration and sample frequency of 1 Hz. It now uses the values from the first calibration entry starting from the responses start time.

  2. 1334: bdsMetadata let pole-zero or FAP filename be variable

  3. 1335: bdsMetadata using variable for "split time" this has now been implemented.

  4. 1320: When creating a new channel with a "network" other than "BN", GUI queries Do you Really...

  5. 1319: Allow "network" to be changed in Clone Channel dialogue

  6. 1318: bdsImportTapeDigitiserData sample rate error Added -ignoreSampleRate flag.

  7. 1336: bdsMetadata "list" and "find" on Sensors and Digitisers tables Added support for normal channel selections.

  8. 1118: allow "maxChanges" in bdsMetadata to be changed within a script file Added ability to set maxChanges from program using the $maxChanges variable.

  9. bdsMetadata: Now issues an error if an unknown selection parameter is given.

  10. 1164: in addWarnings, if no endTime is specified, it defaults to 0000-01-01 Now sets endTime equal to data's end time if not set. Also checks users times given are within the file.

Release 2.0.3 - 2012-10-24

Minor bugfix release.

  1. bdsMetadata: "add Response" is not working with variables for the reponse to use. Fixed.

  2. bdsImportTapeDigitriserData: Session numbers above 7 failed due to reading sessions numbers as octal. Fixed.

  3. 1321: mport FAP using AdminGUI The BDS database used a "text" field type for the response storage. this was too small so the database sechma has been updated to version 12 with a "mediumtext" field.

With this release there are updates to the database Schema to version 12. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 2.0.2 - 2012-10-03

Minor bugfix release.

  1. BdsMetadata: "add ChannelInstrument" was expaning member variables twice.

  2. BdsMetadata: The split time argument can now be a variable.

  3. BdsImportCd: Changed to ignore imports from 23:00:00 to 01:00:00 to allow for end of day file sync.

  4. BdsImportTapeDigitiserData: The -timecodeIgnoreTimeJumps flag failed to work.

  5. BdsDataFileCd: Now returns an error when no Data Format block is present at the start of a CD1.0 file.

Release 2.0.1 - 2012-08-13

Minor bugfix release.

  1. bdsImportTapeDigitiserData would segment fault on older TapeDigitiser data files which had not been re-processed with TapeView and where the cursors startSample and endSample fields were not set.

  2. BDS SEED export. Added a 58 blockette for the stage 1 gain.

  3. 1259: msr_pack: No samples to pack when downloading H08 data as SEED Have fixed the "Error: BdsDataFileSeedError: msr_pack(BN_H11C2_US_LEA_): No samples to pack" problem. Was caused due to a block time calculation that in this instance of data request, block sizes, and time offsets caused a block of 0 bytes to be returned.

  4. BdsSeed: Fixed a bug where if there were multiple channels per station in the data then the Station headers would be incorrectly output.

  5. 1270: rdseed failure for SEED files with many data segments There was a minor bug with continuation of SEED data records which should make sure at least the blockette type and length fields were not split across records.

Release 2.0.0 - 2012-07-27

  1. 1198: Cannot delete StationLocation from AdminGUI. The BdsServer now deletes StationLocations when station is deleted.

  2. 1199: be able to export responses even where there are no data. BdsDataAccess: Modified response output to work when no data is present.

  3. 1200: warnings issued during TapeDigitiser import and export. BdsDataLib: Fixed bug where Qaulity Notes were placed under the source Main rather than TapeDigitiser. This does mean than any existing imports will have the Notes incorrectly set in the BDS database. This can be corrected with a database script (setting the source to "TapeDigitiser" for all Notes where the title starts with "Quality" will probably be fine). The warnings "Warning in executing bdsTapeDigitiserFile" which occuerd are no longer stored.

  4. 1201: v 1.2.22 does not write responses. BdsDataLib: Fixed bug 1201: v 1.2.22 does not write responses. Now writes IMS responses when set to IMS-2.0-CM6 and other sub formats.

  5. 1207: "name" in "Response" edit window. Fixed BWEditComboBox bug.

  6. BdsTapeDigitiserFile: Fixed bug with small files when wrong timecode type was set. In this case the program did not copy any data to the output file.

  7. 1203: Output SEED format data with FAP files; verseed output; rdseed. BdsSeed data format was not handling large FAP tables properly, the data was crossing block boundaries without a continuation block indicator.

  8. 1219: After "split at time", "Create New Digitiser/Sensor" button overwrites old. This has been fixed.

  9. Modified TapeDigitiser import. Now responds to tapeView introduced: startSample,endSample,ignore and allows VelaPll and VelaStd timecode types.

  10. 1223: bdsMetadata "list" on ChannelInstrument. Added the ability to get lists of ChannelInstruments without specifying the channelId.

  11. 1221: plit calibration in bdsMetadata bdsMetaData should support "split Calibration", and in fact does do so but will return the wrong error message when the number of changes is exceeded have fixed this. Also added support for spliting ChannelInstruments.

  12. BdsMetadata: Added support for varibles to be used in the parameter seting section of BdsMetadata commands. Also have added the "print" command (with no arguments) to print all variables.

  13. BdsImportBlacknestDatabase: Added verfification of station and channel names during import.

  14. 1231: BdsGui: File selection lost when you save a mod to a channel in Data Files tab This has been fixed.

  15. 1232: bdsImportBlacknestDatabase interpretation of "AHT" and "AST" This has been implemented.

  16. BdsServer: Modified missing data errors to list channel information.

  17. 1249: WRA_AGSO sampling rate There was an issue in the BdsServer data gathering and export code to do with time calculations. This would have affected all exports of all data types where there was a lot of separate segments of data. This has been fixed.

  18. 1248: YKA tar SEED all except first file have locn code "YK" BdsImportData: Fixed a bug where the network and station location where not
    overriden in the special case for YKA SEED data from 1991-07-10 to 1995-09-01.

  19. 1253: Cloning calibrations Found that this command had accidentialy been effectively removed when the command "clone Location" was added.

Release 1.2.22 - 2012-05-08

  1. Added support to BDS GUI applications to show FAP responses when exporting from seismic data tab.

  2. Added ability to export FAP responses in SEED sensor data exports.

  3. Added ability to export in EVALRESP (FAP) format from PoleZero responses.

  4. Added mode and status metadata to CD1.x imports so that calibration info can be retrieved from data blocks.

  5. Changed some channel types in bdsImportBlacknestDatabase for Bug: 1172.

  6. IMS format export now includes Responses in PoleZero or FAP by default.

  7. BdsGui applications: Added "No Metadata" tickbox on data export. This reduces the amount of Metadata included in the export files. Mainly this means no response data.

  8. BdsDataAccess: Added the "-noMetadata" flag. This reduces the amount of Metadata included in any export files. Mainly this means no response data.

  9. BdsGui: Changed "Response" field name to "Name" in Response dialog and made this an editable field so any value cane be entered.

  10. BdsMetadata: Added ability to change Response.

  11. CD1.x data import now imports channels marked as down and/or zeroed with a data quality warning included in the BDS data file.

  12. Now only reads and uses the first PoleZero response in IDC-format PAZFIR files.

  13. BdsGui: Added ability to download responses from the MetaData information tabs.

  14. Now prefixes the response file's extention with "fap" or "polezero" as appropriate by default.

  15. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Release 1.2.21 - 2012-04-24

  1. bdsImportCd: Now only looks in directories that have a digit as their first character. This stops warnings about trying to access lost+found.

  2. Have set bdsImportTapeDigitiser to continue with the Timecode processing even if no Timecode has been found for 4 hours.

  3. Modified wording for "Duplicate Block" during data conversion.

  4. BdsMetadata: Added support for FAP and FIR responses.

  5. BdsMetadata: Modified "list response" to return actual responce data.

  6. BdsMetadata: Added "read" command to read responses from files.

  7. BdsGui programs: Added support for FAP responses.

  8. Modified BdsDataFileCd to only check relevent channels during duplicate block check.

  9. Added ability to export FAP responses in EVALRESP format in BdsGui applications and bdsDataAccess.

  10. Modified default response file name to include response type  (polezero, fap).

  11. CD1.x data convertor: Added support for "-ignoreCorruptions" flag in bdsImportData. If a currupt (short) block is found, it will be ignored and the next valid block found. This is now the default in bdsImportCd.

  12. The CD1.x imported was setting the number of blocks in a data files channel to one less than there were. This would have affected sample rate calculations.

  13. Added support for multi-segment data import. This allows CD1.X data files to be imported as separate segments of data into one BDS data file. So CD1.x data files that have sets of data over different time periods can be imported and used easily without the issue of "missing blocks" errors. Currently gaps of greater than 1 hour result in these database segments.

Release 1.2.20 - 2012-03-19

  1. Fixed bug where CD1.x file import would fail with "std::length_error" on cd1.0 files without channel authorisation.

  2. Fixed file descriptor leak when file import errors.

  3. Fixed time stored in bdsImortCdTimes.cache.

  4. BdsDataFileDump: Added support for -blocks command to display block timestamps etc.

  5. Fixed problem with DataInfo startTimes not being set correctly when blocks ordering is completely incorrect on CD1.x and GCF files.

  6. BdsImportCd: Added file import log in /var/log/bds/bdsImportCd.log.

  7. Moved logrotate bits to bds package.

Release 1.2.19 - 2012-03-15

  1. Fixed crash bug in bdsImportCd/bdsImportData.

  2. Added validation of authSize filed in CD1.1 data frames to eliminate a crash whith currupt data files.

  3. Modified bdsTestCreateMetaData to create responses with equal poles/zeros.

  4. Fixed a crash in the BdsServer where the incoming data block was not being fully validated.

  5. Modified the CD1.X data converter to use ChannelMultiplexed data rather than SampleMultiplexed data due to a feq data files having missing chunks of data on a per channel basis.

  6. BdsServer: Added support for calibrations with name = Measured. These entries can overlap normal calibrations and have there startTime and endTime equal if desired. All data exports ignore calibrations with a name of Measured.

  7. Added calibrationName to selection mechanism.

  8. Added calibrations command to bdsDataAccess.

  9. CD1.x data convertor: Set the maximum TIMESTAMP_JITTER to 1% of the block time rather than 0.1%.

  10. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Release 1.2.18 - 2012-03-12

This is a minor update, fixing some small bugs.

  1. BdsImportCd has been modified to ignore the curent days files when importing.

  2. Fixed a bug in CD1.1 import where a block with missing channels and short numbers of samples would cause a samplerate validation error.

  3. The CD1.1 data convertor now ignores "Type 7" frames. These are an alert frame that indicates that the connection is about to be terminated.

  4. BdsMetadata added support for listing and changing ChannelInstruments. This requires the updated bdsServer program as well as bdsMetadata.

  5. Changed SEED export to set Blockette 53's normFactor to be 1/gain.

  6. Modified VELA PLL Timecode system for more efficient memory usage. This now allows at least 48 hours with no time codes.

  7. Added code to allow continuation of VELA PLL Timecode system beyond 48 hours of no time codes.

  8. Modified tapeView to not recode timecodes before start cursor.

Release 1.2.17 - 2011-10-07

This is a minor update, fixing some small bugs found in the 1.2.15 release and adding Hutchins timecode support.

  1. BdsImportTapedigitiserData and bdsTapedigitiserFile: Modified "-timecode" timecode type fields to "Manual, Vela or Hutchins" with parameter checking.

  2. BdsImportTapedigitiserData and bdsTapedigitiserFile: Changed -velaStartTime to -timecodeStartTime etc.

  3. Fixed bug in bdsImportTapeDigitiserData where it crashed with data files > 4GByte.

Release 1.2.16 - 2011-09-19

This is a minor update, fixing some small bugs found in the 1.2.15 release and adding Hutchins timecode support.

  1. 1144: Problm exporting CD-1.1 data from hydrophone array H08  Have found this bug due to the timestamp jitter algorithm not resetting new endblock times. Have fixed this.

Release 1.2.15 - 2011-09-14

This is a minor update, fixing some small bugs found in the 1.2.14 release and adding Hutchins timecode support.

  1. Added support for Hutchins and Manual Timecode re-processing in bdsImportTapeDigitiser and bdsTapeDigitiserFile.

  2. Fixed CD1.X data import bug were endian was incorrect.

  3. Modified CD1.X data converter to issue MissingBlocks errors/warnings when Channel SubFrames have no data (numSamples == 0).

Release 1.2.14 - 2011-08-31

This is a minor update, fixing some small bugs found in the 1.2.13 release.

  1. 1138: Problems specifying "Station" This has been fixed.

  2. BdsServer: Added ability to select sensors and digitisers from channel selection criteria. This allows sensorsGetList() and digitiserGetList() to return a selection based on the channel selection crteria.

  3. bdsMetadata: Changed bdsMetadata to set the sensors oldId field to 0 on cloning.

Release 1.2.13 - 2011-08-11

This is a minor update, fixing some small bugs found in the 1.2.12 release.

  1. 1110: Signs of poles and zeros entered in bdsMetadata This has been fixed.

  2. 1113: inconsistent channelType and channelAux in bdsMetadata "clone" This has been fixed.

  3. 1116: Two queries about interpretation of command files by bdsMetadata I have fixed the line number error reporting when "/* */" comments are used.

  4. bdsMetdata, the default variables were not setup. These are now setup as per the manual.

  5. 1114: Cloning in bdsMetadata does not alter dates for all the cloned lines. The system now sets the start and end times of the calibrations, responses, and instruments to the new channel settings. It also only clones the Calibrations,Reponses and Instruments within the newly created Channels time range if more than one Calibrations,Reponses and Instruments are selected.

  6. 1131: BdsMetadata: Add flag to give BdsServer name. The bdsMetadata program now has to be given the "-name <server name>" flag with the name of the server beeing connected to. This is an assitional safety check.

Release 1.2.12 - 2011-07-25

This update adds support for the CD1.x data format and adds the bdsMetadata manipulation program. As well as these and a few extra small features, it fixies some small bugs found in the 1.2.11 release.

  1. 1093: Add ability for normal users to change their password. There is now a menu item in the File menu allowing the user to change their details in the bdsUserGui and bdsAdminGui programs. There is no command line support for this as yet.

  2. 1091: Non-synchronously sampled data from two or more 3-component stns served in BKNAS-1. Fixed this.

  3. Added the bdsDataManage utilty in the package bds-extra.

  4. Fixed bdsDataManage wild cards and set dryRun as default.

  5. Added the "-ignoreVelaReprocessingErrors" flag to bdsImportTapeDigitiser to ignore VELA re-processing errors.

  6. Fixed bug on import wehre imp[ort would loop on first file in directory.

  7. Added data convertor for CD1.0 and CD1.1 support.

  8. Added bdsImportCd a daemon to import CD1.X data files recieved over the network.

  9. Modified bdsImportScream to re-order blocks, allow duplicate blocks and ignore missing blocks during import.

  10. Modified bdsImportScream to also add import validation warnings to the BDS Notes system.

  11. Added the ability for the bdsServer to ignore minor time discontinuities when exporting data. Any discontinutity less than 0.1% of a blocks time period will be taken to be a clock resynchronisation event.

  12. Fixed bug in BdsServer where ArrayChannels were not deleted when an array was deleted.

  13. Changed the bdsServer to delete the Calbrations,Responces and Instruments for a channel when a channel is deleted.

  14. Added database backup and restore API to the BdsApi. This allows the BDS database and server configuration to be backed up into the Backup directory of the currently writeable datastore.

  15. Added the bdsManage program. This runs as an Admin user and peforms the work of bdsControl but with additional functionality. This includes database backup and restore.

  16. 1016: BDS database looses data if a transaction from another program is cancelled. Each BDS client connection now has its own database connection and so is independent of other clients. This also has the side effect of speeding up BDS access when another client is performing a large database search.

  17. Added the "readOnly" option to the BDS data store configuration in the bdsServer. This allows a set of datastores to be marked read-only. These will not be used for wriring new data. This is useful for the creation of a test server sharing the seismic data of a main server in read-only mode.

  18. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 11. Actually this is only the addition of the user bdsImportCd. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 1.2.11 - 2010-12-14

This is a minor update, fixing some small bugs found in the 1.2.10 release.

  1. 1034: IMS response calib zero. The formats of the calibrationfactor CAL2 and pole zero values in the PAZ2 lines have been change from %f to %e. The calibrationfactor is now output in nm/count rather than m/count in IMS exports (both sensor data and meta data). This also applies to the ASCII and BKNAS data file output convertors.

  2. BdsImportData/BdsImportTapeDigitiserData: The user supplied -addWarning Network:Station:Channel:Source parameter was overriden with values obtained from the data. The user setting now takes presidence. Also added validation of Network,Station,Channel,Source parameters passed against metadata.

  3. BdsAdminGui/BdsUserGui: Notes deletion was poping up warning/error dialogs when deleting notes without documents attached. This has been fixed.

  4. BdsImportTapeDigitiserData: We have added a test to make sure memory is not eausted when the data's VELA year is set to 0.

  5. 1039: Date-time widget "end time before start time" BdsAdminGui/BdsUserGui: Modified to not test for endTime < startTime while entering dates.

  6. Improved the performance of the dataSearch() API call when using arrays with multiple station/channel pairs.

  7. 1041: User GUI modification/delete buttons active

  8. 1047: trace viewer scaling BdsAdminGui/BdsUserGui: Now only auto scales data of type seismic together.

  9. 1054: bdsDataAccess reports zero calibs Changed to use scientific notaion for calibrationFactor.

  10. 1055: "data already present" error while importing TapeDigitiser data This has been fixed.

  11. 1057: TapeDigitiser multisession data with missing blocks - Notes table contains too many entries This has been fixed, also fixed in bdsImportData.

  12. bdsTapeDigitiserFile: Added ability to invert VELA and Error tracks.

  13. 1066: EKA3617 TapeDigitiser timecode problems with day 367 Fixed bug with VELA year wrap.

  14. 414: Database does not keep up with station/channel name change Modified API function stationUpdate to also modify all of its StationLocation entries.

  15. 1071: Make restricted stations invisible to unprivileged users The data access bug 1072 has been fixed (Not checking all of the segments in a multi-segment data request). We have added support for completely masking stations if they are added to AccessGroups with a time period 0001-01-01 to 9999-01-01.

  16. 1078: Tab name DataSelect in UserGUI or Sensor Data in AdminGUI

  17. 1073: DataPlot in bdsDataAccess fails for multichannel data with non-aligned block boundaries Now supports accessing random block sized data (GCF) in a sample multiplexed manner.

  18. The segment end time was not being set correctly on channels when a MetaData occurred and the data blocking amougst the channels was un-even.

  19. 943: calculated sampling rate does not match metadata when "Calibration" split not on block boundary

  20. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

Release 1.2.10 - 2010-11-04

This is a minor update, fixing some small bugs found in the 1.2.9 release.

  1. BdsServer: Changed default "DataStoreMin" size to be 32GBytes.

  2. BdsServer: Fixed bug in data store allocation with multiple data stores wherea data file write error could occur due to insufficient space being available.

  3. BdsImportBlacknestDatabase: Modified to set endTimes of "EKA:BHZ" channels rather than "EKA:BHZ_10" channels.

  4. BdsImportBlacknestDatabase: Modified to add location codes to channel types in ArrayChannels table.

  5. Modified Notes/Warnings system to include particular Station and Channel names when imporing from multiple channel files.

  6. 1003: incomplete "Data Info" Each files individual info is now preceeded by "file<n>" when multiple files are involved.

  7. 1006: Start time becomes end time in -addWarning

  8. 1009: pole-zero constant is inverse of correct value

  9. 1017: add oldId to list for "clone" or "share" an existing sensor

  10. 1016: Export error when "full blocks" not set (ref 952, 953, 965)

  11. 1029: BdsAdminGui: Raise Data info windows when appropriate button is pressed

  12. 1015: original file name of tar SEED files is given as the first /tmp file into which the tar SEED was untarred Now uses the original tar file name for all filename entries.

  13. Notes documents are now stored in a documents directory within the main data stores.

  14. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 10. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 1.2.9 - 2010-10-21

This is a minor update, fixing some small bugs found in the 1.2.8 release.

  1. 995: out-of-place WID2 lines in IMS data exported with "segment merge" ticked Fixed a bug in merged segment output.

  2. 997: "SeismicUnknown" channels not finding metadata but spurious backslashes Now allows import of data into "seismicUknown" and "empty" data typed channels.

  3. 994: Duplicating/conjugating poles and zeros (see 784/792) Have changed this to add after the selected item.

  4. Added multiple hour long GCF like test data.

Release 1.2.8 - 2010-10-15

This release has a number of modifications for the first live BDS release. The core features added or changed are:

  1. The Warnings/Notes system has been overhauled. Note that the bdsImportData's "-addWarning" syntax has changed. Note that due to this there will be no Notes from previously imported data.

  2. Note that the /etc/bdsServer.conf file should be updated with the "DocDir:" entry. Info in /etc/bdsServer.conf.rpmnew.

  3. The bdsAdminGui's tab layout has been changed by using top level windows for some of the previous tabs. The selectors have been merged into one set of code although there are still separately settable selectors. There are also quite a few changes to support the bdsUSerGui program sharing the code base.

  4. The bdsUserGui progam and bdsNoteAppend programs has been added.

  5. Instrument response export has been added.

Detailed list of changes:

  1. Added support for -includeSessions flag in bdsImportTapeDigitiserData.

  2. 925: Unable to edit location of an array in bdsAdminGui Fixed.

  3. Have changed error messages in bdsTapeDigitiserFile to include "Error:" or "Warning:" at start.

  4. The error "ErrorDataPresent" is now returned when bdsImportData or bdsImportTapeDigitiserData fails to import when data is already present.

  5. 932: bdsImportTapeDigitiserData reports temporary filename Changed bdsImportTapeDigitiserData to report original file name in errors.

  6. Modified SEED export to only list one station type 50 block per station.

  7. 944: TapeDigitiser import with extra files in directory bdsImportTapeDigitiserData now only looks of files of the name format "data-*.bs".

  8. 931: bad_alloc while importing TapeDigitiser data bdsTapeDigitiserFile: Added support for setting the error and vela track numbers with flags. Returns an error if the VELA code could not be read for more than 4 hours. Added support for setting the correct VELA and Error channels from the info in the JobInfo file to handle files with unusual VELA code tracks..

  9. 945: YKA SEED TAR files with currupted ends Added the -ignoreTarErrors flag to bdsImportData command.

  10. 946: SEED: Improve compatibility of output SEED export: The "#" can be removed by setting the station alias. I have modified bdsImportBlacknestDatabase to do this by default. The station block numbers in blockette 11 have been fixed.

  11. bdsImportBlacknestDatabase: Now adds a station alias for all stations with a '#' character. This station alias has the '#' removed. This allows export in all formats with station names not having a '#' character.

  12. 36: Instrument Response Information: Accessing Added support in the BdsUSerGui and BdsAdminGui programs for exporting the sensors response in SAC-POLEZERO, IMS-POLEZERO and IMS-FAP formats.

  13. Added support to the bdsAdminGui, bdsUserGui, bdsDataAccess and bdsAutodrm programs to export sensor responses.

  14. 941: DataStoreMin ineffective if data file larger than value set, can cause data loss

  15. 952: Modify Full-blocks data export feature The new normal operation is to return all of the data requested between the times stated to the nearest sample. Some channels may not contain data at the start or end times (short channels). The FullBlocks mode is the same as it was. There is an additional option in the API to clip the data so that all channels are complete. Block Start/End times were incorrect with time clipping, this has also been fixed.

  16. Added support to the bdsAutodrm program to export sensor responses. Supports "GSE2.0"/"IMS" and "SAC" pole zero formats.

  17. The bdsAutodrm now support sthe "EMAIL" as well as "E-MAIL" commands.

  18. The bdsAutodrm now logs to the log system rather than the notes system.

  19. 950: Rationalise warnings/notes/logs Major changes to data warnings and Notes system. All data import warnings both by the system and user added ones, with -addWarning, are now added to the BDS sensor data files and stored in the Notes database table. Data warnings now have a start and end time so that only the warnings relavent to the data being accessed will be shown on export. Only the notes stored in the database are used for data notes information. The warnings store in the file can be viewed in the DataInfo tab wich shows all of the MetaData stored in the actual data files. When accessing sensor data the GUI programs now have a "DataNotes" window to view these notes. The Notes tab has also been updated with a selector to select the notes required.

  20. The Notes system now has the ability to store documents in various formats. The Notes database stores information on the format (mime format info and normally the filename extension). The actual document files are stored in a documents directory on the BdsServer which is defined using the "DocDir" configuration parameter. There is a new, command line, program named bdsNoteAppend that has the ability to append notes with and without documents. Note that there is no way of viewing these documents as yet although the API provides the mechanism for document reading.

  21. BdsAdminGui: Changed tab system. MetaData is all under one tab with sub tabs. There is now one Sensor data tab that brings up top level windows for the Traces, DataInfo etc.

  22. BdsAdminGui: Single selector widget for all selections. Separate instantiations for MetaData, SensorData and DataFiles.

  23. BdsAdminGui: Improved selection performance.

  24. BdsAdminGui: Disabled the ability to change the file DataChannel sample rate parameter.

  25. 49: BdsDataView GUI program Added BdsUserGui program for user access to the BDS system.

  26. 940: date-time widget sometimes has 5-digit year This has been fixed. It occured when there was no data in the system.

  27. 923: "Logs" tab info - should this be so easy to alter? And bugs in deleting and accepting new items I have fixed the delete bug and added checking for the priority value. I can remove the ability to delete/update log entries if wished.

  28. 916: Cannot display info in DataViewInfo tab if channel lacks metadata  We have added this ability. If the CalibrationFactor or calibrationFrequency values are 0 then the system will set them to 1.0. It will also use the samplingRate as defined in the data file and ignore the rate in the metadata if it is 0. Instrument MetaData data is currently ignored.

  29. BdsAutodrm: Added new help file. This file is stored in /usr/bds/config/Help.txt and has been set as a package "noreplace" file so that local changes will not be overwritten on BDS package updates.

  30. 890: Merge consequtive data segments on export. This has been added. There is an option in the bdsGui programs for this and bdsDataAccess has it enabled by default.

  31. 934: BdsImportData: only validate between startTime and endTime. The bdsImportData command now only performs data validation between the startTime and endTime values if set. Note that data curruption, data reordering and filename time errors are still reported and noted.

  32. 982: is downloading in "BDS" format meant to work? We have disabled the export of BDS files. This was intended to be an internal format only. If wanted we can add support for exporting in this format.

  33. TapeDigitiser data import. The block time stamps were not quiote corrrect due to a bug in the ms to us conversion (could have been out by up to 500ms).

  34. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 9. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 1.2.7 - 2010-07-16

  1. Fixed the database creation scripts.

  2. 892: database table ArrayChannels errors for "BKNI" entry: Modified bdsImportBlacknestDatabase to set up the correct channels in the ArrayChannels database table. Fixed a bug where spaces were included in ArrayChannels station names.

  3. The admin email address list and AutoDrom station and channel lsits also sufferend from having spaces included in the list components.

  4. Modified bdsImportBlacknestDatabase to ignore metadata entries for channels with the formats "GCFEKABBCD" and "GCFEKBCD". The source "EkaCD" will probably be used for this data and has been added to the inital database setup scripts. Note that for an existing database the souce "EkaCD" will need to be added.

  5. 898: start time when requested timespan includes staggered beginnings of data: There was a bug in the BdsServer where if the user selected muliple channels of channel multiplexd data begining or ending within a missing block of data the BdsServer would segment fault. This has been fixed.

  6. No API or Database changes have been made with this release.

Release 1.2.6 - 2010-06-30

  1. Added "-allowOverlap" flag to bdsImportData program.

  2. Now stores the original import file name in the DataChannels database table. This is accessible from the BDS API in the DataInfo object. The bdsDataAccess programs "dataInfo" command will return this information. There were also changes to bdsAdminGui to support this feature.

  3. 887: adminGui hangs during export of mixed-"source" data when timespan includes start of one source

  4. Modified to store the full path name of the original import file.

  5. Added code to setup stdout to be in line buffered mode for command line programs.

  6. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 8. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 1.2.5 - 2010-06-21

  1. Fixed bug where data files were not moved to ./Deleted when deleted.

  2. Modified data export to clip data to the sample level at requested times rather than just make sure the data starts abd ends at the sample sample.

  3. Disabled changing of DataChannel.importFormat field.

  4. Added support for calculating overal TapeDigitiser digitisation quality values and displaying warnings on import and export.

  5. Fixed a bug where warnings added with a time set to "unset" where not displayed on export.

  6. 881: failure to open temporary file while reading tar SEED data When importing SEED TAR files now sets the permisions on the un-tared files to allow reading.

  7. When importing GCF files, the filename time comparison now only compares the year and day.

  8. 880: very minor bug in error msg from non-existent file channel

  9. Fixed a bug that did not allow 0 length strings in comma separet lists. this apllied to -addWarning fileds and other cases.

  10. Fixed a few minor bugs.

Release 1.2.4 - 2010-06-03

  1. 828: Database: Move to transaction based database The MySql database has been changed to be a Transaction based database using the InnoDB engine. The Bds API has been updated to support Transactions and some of the BdsAdminGui's operations have been modified to use this feature. We have also updated the database to use the utf8 character set by default.

  2. 85: Data per sample access and data block interpolation

  3.  The system now has an option for "Clipping" the output data so that the start and end times of all of the channels export is the same. The data blocks of channels that are beyond this period are truncated to the appropriate sample. The truncation is performed based on the blocks time stamp and the sample rate in use.

  4. 496: WRA_AGSO won There was a bug in the WRA-AGSO file convertor where the first block was repeated and the last block lost on each channel.

  5. Modified bdsImportTapeDigitiserData to treat bdsTapeDigitiserFile processing warnings as warnings not errors.

  6. 86: Notes: Add information on TimePeriod, Network, Station, Channel andSource

  7. 685: Add GCF like variable block size data to BdsTest system

  8. 850: BdsAdminGui title to reflect connection to server

  9. 854: Add BDS Clean API call to clean logs and changes database entries

  10. 59: Error and Warning event system

  11. 665: Missing block warnings

  12. Modified dataType field to support seismic,seismicUnknown,data,unkown and empty. The bdsImportBlacknestDatabase program now sets these based on the channelType.

  13. The bdsImportBlacknestDatabase program did not import calibration info for channels without sensors. This has now been done.

  14. 837: TapeDigitiser import: Add flag to ignore sessions

  15. 701: Add support for source names in bdsDataAccess -channel options bdsImportData and bdsImportTapeDigitiserData now allow the source to be specified in the -channels flag.

  16. 700: Export sample rate information

  17. 820: "hang" for horizontal seismos of unknown orientation

  18. 777: BdsAdminGui: Improve Channel edit dialog , 522: Changing channel name loses calibrations/responses. Many improvements to the Channel editing system. The core difference is changing a Network/Station/Channel name changes all of the metadata and sesimic data in the system to match. Note that the channel information in the sesimic data files is not affected although this is not used anywhere. As we are now using a transactional database all of the changes to the channel,calibrations,instruments,responses is only commited when the "Save" button is pressed.

  19. 838: BdsAdminGui: Improve Instrument Sensor and Digitiser entry. Improved this in line with sugestions from Sheila.

  20. 77: Add support for easy import of Meta Data working at Array, Station and Channel levels

  21. 768: Instrument "split at time" when split time is before opening date/time of instrument metadata

  22. 694: Change file URL locations to just specify relative path within the data store

  23. Added support for 'M' type data records in SEED files.

  24. There have been a few parameters added to the /etc/bdsServer.conf file. It will be worth taking the new version and editing this with the desired parameters.

  25. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 7. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

The new release uses a Transactional database. By default the MySQL system stores all InnoDB database tables in a single file. If desired, the system can store each table in a separate file. To do this edit the /etc/my.conf file and add the option "innodb_file_per_table" in the "[mysqld]" section and restart the mysql daemon with the "service mysqld restart" command. If this is done before bdsDatabaseUpdate is run the new transactional database tables will be stored in separate files. See info in: https://portal.beam.ltd.uk/support/blacknest/info?cmd=infoView&id=828 for more details.

Release 1.2.3 - 2010-05-11

  1. The bdsImportScream program now uses the import validation options: "OptionReorder" and "OptionDeleteDuplicates". So any SCREAM backfilling errors should be handled as best as is possible given the faulty SCREAM system.

  2. 700: Export sample rate information. The sample rates reported in IMS,BKNAS and SEED files are now the calculated sample rates rather than the MetaData specified sample rates. This handles variable sample rate data better, such as TapeDigitiser data.

  3. 703: Save dialog, always defaults to save in ~ FIle dialog now keeps last uesed directory.

  4. 708: "dip" in SEED blockette 52 should be -90 usually for vertical seismometers. Swaped sign of dip calculation.

  5. 704: sensitivity "inf" in SEED files when calib = 0. SEED gain is set to 0.0 if the calibrationFactor is 0.0.

  6. 724: Group checkboxes dissappear on View/Append/Modify Fixed bug with Users group boxes dissappearing.

  7. 725: Admin GUI Segfault in Data Files tab

  8. 726: Channels selection View/Modify do not work with mutiple selections

  9. 728: creating array in AdminGui - added array stations without setting "type" to "array"

  10. Fixed a bug in bdsServer in array to station expander. This could cause no stations to be selected if the first station in the database was an array.

  11. 716: -duration flag doesn't work on bdsDataAccess

  12. 739: Station description rendered badly Modified all tables to display first line of multiline descriptions.

  13. 747: BKNAS1 number of channels must be < 32 BKNAS convertor restricts number of cahnnels to 31 with an error.

  14. 754: Changes tab - minor hiding issue

  15. 755: Missing control on certain forms

  16. 763: Save warning dialog has two buttons

  17. 769: bdsDataAccess reports warnings to stdout

  18. 550: Problem with multiple Pause/UnPause Fixed.

  19. 736: Restricted-length character fields Added name length checking for Network, Station, ChannelType, ChannelAux and Source fields.

  20. 784: "conjugate" pole has zero imaginary part if "return" or "tab" not hit; window focus

  21. 783: wrong "calib" in data exported over mid-block metadata change. Modified to start next segment on block following MetaData change.

  22. 781: Reconecting from Admin GUI after server restart causes bds process to fail

  23. 774: Updating list when new channel is added

  24. 705: "#" in station name where otherwise station name = array name

  25. 671: Failure to export overlapping data from EKA3566

  26. bdsImportTapeDigitiserData: There have been major changes to the bdsTapeDigitiserFile program used by bdsImportTapeDigitiserData to re-process the VELA timecode track. This now uses a Phase Locked Loop algorithm to more accurately set the data block start and end times. The system now calculates a VelaQuality based on the number of VELA codes decoded versis the number expected. This is reported as an Import Warning.

  27. 770: Filename of imported data states nearly 8 months Added a check to make sure the date range is less than 10 days in one import file.

  28. 804: Output SEED data have duplicate blocks at end

  29. 805: BDS SEED files in "Jrdseed" Now outputs channels in Station order with a single blockette 50 per station.

  30. Source names are now validated and BdsAdminGui uses comboboxes for the available source names.

  31. 810: bdsImportData with -dryRun does not issue "data already present" error Added checkif if data is present when -dryRun is used.

  32. 808: Dummy channels Added support for a channel "noMetaData" boolean. When this set on a channel it states that there is no MetaData present (Calibrations, Responses, Instruments). When set data can be imported, with a warning, however date export of the appropriate channels is not possible as an error of missing MetaData is given.

  33. 79: Add meta data validation checks.

  34. 733: Add -startTime and -endTime options to bdsImportData It is now possible to use -startTime and -endTime options with the bdsDataImport program to imports time spans of data from files.

  35. 690: default source "Main" for Responses

  36. 823: GCF files: Filename time errors when hour is not 0

  37. 456: Certain windows/dialogues incompatible with 1024x768 resolution

  38. 825: BDS server behaviour under load This has been fixed.

  39. 835: Channel DataType field This has been added.

  40. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema to version 6. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  4. Restart the BDS system. "bdsRestart"

Release 1.2.2 - 2010-03-16

  1. Minor change to BdsImportBlacknestDatabase to modify MetaData source allocation to use new EkaDig2 scheme. Set database initial source data to use "EkaDig2" rather than "Scream".

  2. Now handles MetaData for EKA:SHZ around GCF24EKA0811 better by merging MetaData either side for the source Main.

  3. Extends the EKA:SHZ:Main and EKA_BHZ_10 data to end of time.

  4. Extends the EKA:SHZ:EkaDig2 to cover all time

  5. Improved Sensor selection algorithm to share sensors for channels better.

  6. 695: preicision of longitude for WRA and YKA

  7. Extended bdsImportScream to support source names in configuration file on a per import source basis and modified default to use a source of "Main".

Release 1.2.1 - 2010-03-12

This is a minor update, fixing some small bugs found in the 1.2.0 release.

  1. BdsAdminGui: Fixed bug with data viewer where missing block data at the very start and very end was not highlighted as missing.

  2. SEED blockette MetaData is now stored in infoExtra to reduce normal data info clutter.

  3. BdsAdminGui: Added Splitter bar to DataSelect tab.

  4. BdsAdminGui: Fixed bug in DataFiles window where selected channels where not viewed correctly when a File was selected.

  5. BdsAdminGui: Changed default filename format used for downloads.

  6. BdsAdminGui: Added scroll bars to DataSelect warnings dialog.

  7. BdsImportBlacknestDatabase: Modified to ignore "SCREAM.*" format channels on import and extend MetaData endTimw's for EKA to be end of time.

  8. 657: Exporting EKA BB data with gaps: (ChannelInfo not matching actual data blocks) partially fixed.

  9. 658: wrong number of samples in IMS format data originally GCF from EKA BB array

  10. 673: SEED reader "unknown block type: R" Added support for 'R' and 'Q' SEED data blocks and type ' ' padding blocks.

  11. 668: AdminGui DataSelect output file navigator GUI error

  12. 670: "Update Select Fields" button in DataSelect window resets times

  13. 669: output data file is not completed until a warning popup is "OK"d/closed

  14. Added endTime to ASCII format MetaData.

  15. BdsAdminGui: Improved DataSelect and display to keep segementation of data rather than create multiple channels of data.

  16. 690: default source "Main" for Responses

  17. Validates user entered time in Respose,Calibration and Instrument "split" operations.

Release 1.2.0 - 2010-03-01

This is a major update to the system, fixing and changing some of the core internal structure.

  1. Changes to BDS data file format. This adds channel and segment info to the data blocks to improve the performance of dealing with segments of data when there are missing blocks or in other multi-segmented data applications. Also improved block searching performance and other minor improvements.

  2. Major changes to the DataSet class. This is responsibe for gathering the data from multiple channels of multiple segments in multiple files and providing access top the seismic data conatined. This will have fixed many issues with data access.

  3. 89: Automatically break data into segments when there are missing blocks

  4. 526: DataAccess wild card matching not supported

  5. 498: non-synchronously sampled" error when exporting (non-multiplexed) ASCII data over a file boundary

  6. 84: Improved operation when there are multiple segments of data

  7. 330: Break data into segments on MetaData changes

  8. 303: Automatically break data into segments when there are metadata changes

  9. 498: "non-synchronously sampled" error when exporting (non-multiplexed) ASCII data over a file boundary

  10. Modified the ASCII data format to support Channel Multiplexed: ASCII-CM and Sample Multiplexed: ASCII-SM formats and multiple segments. Multiple segments are implemented by adding a comment line starting with a "#" with channel, segment, time and other information.

  11. 338: failure to export EKA Red line

  12. 404: tapeDigitiser data import: Delete multiple FM deviation errors. Added InfoExtra storage in BDS data file format in in BDS API to store the extra TapeDigitiser error information. This speeds up data access but still allows full storage of the contents of a TapeDigitiser data file.

  13. 385: TapeDigitiser import fails when there is an "empty" data file

  14. 371: TapeDigitiser import failing for lack of metadata

  15. 336: TapeDigitiser data with huge numbers of errors

  16. 597: Add Calibrations name field

  17. 513: Metadata overlap error message when modifying "response" information

  18. 588: poles/zeros in output SEED files read by rdseed The system now uses base SI units of all data and uses calibrationFactors that will convert the data samples to the base SI units in question. When exporting in a particular data format unit/scaling conversion is done as appropriate.

  19. 507: BKNAS format when many poles/zeros

  20. Fixed crash in BdsServer when trying to export SEED-MINI data.

  21. Fixed problem with SEED sensitivity value.

  22. Changed SEED dip angle and gain calculations. Set Stage Number to 0.

  23. Changes units operation.

  24. Fixed problem with dataImport continuing when SEED data format errors were present.

  25. 634: Add support for SEED-METADATA file format

  26. Modifications to BdsTest system to help test the above changes.

  27. 432: Login window disappears after one failed attempt

  28. 433: TCP reset requires to have to be restarted, Access Denied error

  29. 488: DataFiles tab shows meaningless import data

  30. 537: File channel metadata editing

  31. 425: DataSelect tab multi select

  32. 427: bdsAdminGui dataselect->dataview usability issues

  33. 497: "Update" button on channels window didn't work

  34. 430: DataView channel shift, 431: DataView seems not to work when displaying multiple channels A simplified DataView tab has been implemented. This allows the viewing of multiple data channels with multiple data segments, possibly overlapping as well as channels at differeing sample rates. It has a simple autoscaling function.

  35. 647: default file name formation fails when station name includes #

  36. Numerous higher level MetaData editing functions added to "Options" menus.

  37. BdsImportBlacknestDatabase: Added support for importing Pseudo stations.

  38. Removed SensorLocations table and added depth and sensor angles to Calibrations.

  39. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

  40. Note that the BDS data file format has changed and, for simplicity at this stage of development, no compatibility with the previous file format has been provided. So all data files will need to be re-imported.

With this release there are updates to the database Schema to version 5. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:
Shutdown bdsServer: "service bdsServer stop"
Update the BDS software: "yum update"
Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"
Restart the BDS system. "bdsRestart"

Release 1.1.9 - 2010-01-26

  1. Changed default import data source to "Main" from "Master".

  2. System now sets importUserId and importTime fields in DataFiles information.

  3. Preliminary SEED data file import.

  4. BdsImportData: added "-sequential" flag. When used it will import the given set of files sequentially into a single import file. This is especially usefull for SEED-TAR format files. The files have to contain the same channels in the same order and be completely sequential in time.

  5. 570: SEED Export converter. The system can now export data in SEED 2.4 and SEED-MINI 2.4 formats.

  6. DataSelections invovling an array in combination with a set of individual stations did not work correctly. This has been fixed.

  7. Data selections within a single file that contained non-synchronously sampled data at different sample rates did not work corectly. This has been fixed.

  8. Added -formatList option to bdsImportData.

  9. 428: bdsAdminGui long processes need hourglass cursor

  10. 104: Add ability to manipulate Sensor Data information and delete sensor data files

  11. 405: New Data files window - various

  12. 523: Delete data files. System now deletes data files by moving them to a directory "Deleted" on the store in question. These will need to be actually deleted at a later date by an operator.

  13. 350: Improve data access performance by elimimanting second getInfo

  14. 27: default filename in GUI

  15. 417: Time date widget defaults to 2002

  16. 424: DataSelect tab usability issues

  17. Changed DataSelection system in bdsAdminGui. Now provides wildcard filter on each item and multi-selection restricted to database entries. Note that selection restriction for data ignores the time fields. This is for performance reasons. It would be possible to implement a cache of distinct, time based, channel information to allow selection restrictions based on actual data time periods if required.

  18. Changes to use "*" wild carding system in bdsDataAccess and bdsAdminGui.

  19. System now checks the "state" parameter on data files before serning the data.

  20. Added ability to list default file format extensions.

  21. Added BdsDataFormats manual and updated other manuals and documents.

  22. A few minor bugs have been fixed.

  23. Note that the BDS API has been updated in this release, so any client applications will need re-compiling.

With this release there are updates to the database Schema. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:
Shutdown bdsServer: "service bdsServer stop"
Update the BDS software: "yum update"
Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"
Restart the BDS system. "bdsRestart"

Release 1.1.8 - 2009-12-14

This is an interim bug release version. The BDS API has changed so that any client machines will need to be updated as well.
This release is now built on Red Hat Enterprise Linux 5.4 and has a new yum repository. The repository change will be automatic, but two yum updates are required, one to update the repository info and one to actually perform the update.

  1. 455: Window hiding issue on certain window managers

  2. 464: DataViewInfo DataViewChans inconsistent inital state

  3. 465: UserInfo tab, User Edit dialog group selection disappears

  4. 461: Dialogue window on Changes tab list

  5. 411: BdsAdminGui: Update selection lists in channel on updates

  6. 414: Database does not keep up with station/channel name change

  7. 474: Number of channels incorrectly calculated?

  8. 475: Wrong hang & vang

  9. Added "datum" import from Autodrm database in bdsImportBlacknestDatabase.

  10. Removed wild card from stations in Autodrm.

  11. 416: System puts all data files in a single directory Note that this does not update the location of any existing files imported into the BDS system.

  12. 401: Change source "Default" to "Prioritised"

  13. 460: Inconsistent list behaviour with modify

  14. 494: BDS fills /tmp

  15. 495: bdsControl exectuable missing from rpms

  16. 511: BKNAS instrument number

  17. 508: BKNAS format 1st line of Channel description

  18. 501: precision of "calib" for BKNI data Note that this bug affeced MetaData imports so values in the BDS SQL database for calibrationFactor could be wrong.

  19. 355: Add a test Array/Stations/Channels for performing automatic regression tests The first pass system has been created. This just creates BDRS files at the moment but export tests are generic.

  20. Added the "NETWORK" keyword to the BdsAutodrm. This allows the network to be set for test puposes.

  21. 492: StationLocation selection in Stations needs Network This has been added.

  22. 453: BDS AutoDRM acts as a relay The system now ignores emails from users not registered in the BDS users database. It checks the "ReturnPath" and "From" field headers for the users email address. Errors are looged into the syslog of the bdsAutodrm daemon's host.

  23. 514: Interrupted data import results in bad data status. This has been fixed.

  24. 525: DataAccess error message Error text changed.

  25. 402: Data import error/warning message improvement Warnings are now presented as a comma separated list.

  26. 403: BdsImportData: Add "-info" flag. Added -description and -addWarning flags to system. See the documentation in https://portal.beam.ltd.uk/support/blacknest/files/bds/doc/BdsImport.pdf for more information on the formats of the warning messages.

  27. 545: bdsImportTapeDigitiserData. Added ability to read jobInfo.tdim if it exists instead of jobInfo.tdi.

  28. Added import warnings when blocks are re-ordered. The warning includes a description of the block re-ordering performed in the form of an ASCII string. The format of this string is a list of block ranges and the re-ordered position. For example "0-10:30" means blocks 0 through 10 have been moved to position 30.

  29. Added the Blacknest-testing yum repository for test updates. This is not enabled by default.

Release 1.1.7 - 2009-10-02

  1. Added support for setting source names from format in bdsImportBlacknestDatabase program.

  2. Fixed a bug where the source field was not being used to validate the addition of new ChannelInstruments.

Release 1.1.6 - 2009-09-25

  1. Fixed crash on DataFile deletions.

  2. 282: multi-multiplexed data in channels 19 and 20 of BDRS_UKNET data Added support for BDRS-MM data files.

Release 1.1.5 - 2009-09-25

  1. 381: BOAP connects to localhost. This has been fixed

  2. 367: BDRS UKNET 10-Hz data import Added support for BDRS files with 10Hz sampling rate.

Release 1.1.4 - 2009-09-16

  1. 338: CalibrationUnits now set from channel name.

  2. 389: BKNAS-1 output calibration factor is written where pole-zero constant should be Now calculates the PoleZero constant for the PoleZero output in BKNAS format from the PoleZero table.

  3. 385: TapeDigitiser import fails when there is an "empty" data file Added support for zero data length TapeDigitiser files.

  4. 390: EKA BB GCF corrupt data This has been fixed.

  5. Added, first pass, DataFile and DataChannel API functions and editing ability.

Release 1.1.3 - 2009-09-08

Release 1.1.2 - 2009-09-07

Release 1.1.1 - 2009-08-17

  1. 341: WRA_AGSO alias in bdsImportData "-format" option. This has been added.

  2. 343: MaxNumSamples in bdsServer.conf. Changed to 1024 by default.

  3. Fixed a bug in calculating the number of datasamples/sample rate in a set of data.

  4. 342: "sample rate incorrect" for WRA40 and WRA_AGSO data There was a bug in the WRA-AGSO sample rate calculation routine when crossing a year boundary, This has been fixed.

  5. 340: Add -listFormats option to bdsDataAccess and use in bdsAdminGui

  6. There was a bug that caused data import to fail with the error "Unknown output format:" this has been fixed.

Release 1.1.0 - 2009-08-06

  1. 57: Added snapshotPause call. The API function takes a boolean argument, on or off to set the BdsServer into pause mode for backup puposes. See the BdsServer manual for more information.

  2. 57: Added the modeSet API call. This allows the BdsServer to be put into a read-only slave mode. See the BdsServer manual for more information.

  3. 57: Added the security group "control". The snapshotPause and modeSet calls require the user to belong to this group.

  4. 57: Adde the user "bdsBackup" for backup purposes. This user belongs to the control group but has no other abilities.

  5. 57: Added "state" field to DataFiles table to indicate the data files state. This can be set to "importing" or "ok" and can be used for data synchronisation puposes.

  6. 270: Added the "Config" tabale to the database to contain configuration parameters. One configuration parameters, "schemaVersion" has been added to this to track the BDS MySQL data base schema version. This allows for automated database schema updates on software updates. Add the command line program bdsDatabaseUpdate command to update the ruuning database's schema to the current version.

  7. 253: Change the BdsServer's database user to "bdsMain" and set the older "bds" user for read-only access. This allows external programs to read the BDS's database safely. The database grant's need to be updated with the following commands for this to work and the /etc/bdsServer.conf file needs modification.
    grant all on BDS.* to bdsMain@"%" identified by "bdsMain";
    grant all on BDS.* to bdsMain@"localhost" identified by "bdsMain";
    grant select on BDS.* to bds@"%" identified by "bds";
    grant select on BDS.* to bds@"localhost" identified by "bds";

  8. 268: Added support for re-ordering data blocks and deleting duplicate data blocks on import. The "-reorder" and "-deleteDuplicate" flags have been aded to bdsDataImport for this. Currently only the GCF format convertor supports these options, but they can be added to each or all data convertors relatively easily if required.

  9. Fixed: 278: cannot add new SensorLocation without saving-and-reentering

  10. Fixed: 264: AdminGui update bug in DataSelect window

  11. Fixed: 283: extracting "filename time" from EKA SCREAM data files

  12. Fixed: 289: AdminGui bug for a particular GCF file - "QG Vector::insert: index xx out of range"

  13. Fixed: 291: BKNAS1 format

  14. Fixed: 306: AdminGui editing channel responses

  15. Fixed: 265: No. of samples in 2nd segment of 2-segment IMS exported file

  16. Fixed: 286: sampling rate for EKA GCF data

  17. Fixed: 210: TapeDigitiser import two-tape sets

  18. 311: On importing TapeDigitiser files keep original file name The BdsImportTapeDigitiserData program now lists the actual file import in the Description field.

  19. 325: Fixed: Database description entries should allow the "' " character

  20. 336: TapeDigitiser data with huge numbers of errors We have improved the perormance of the BDictString class to help this. Howvere the number of errors produced by the tapeDigitisation process needs to be looked at.

  21. Added support for Station Aliases. These can be used to rename the pusedo stations used for extra data channels that are not associated with any physical stration, to the Array name as well as for other puposes. The Aliases are only normally used in data output formats.

  22. Added support for MetaData sources that match the data sources. This allows different MetaData parameters to be attached to different data sources.

  23. Added support for Source Priority to decide which data Source to select if a data source of "Default" is used.

  24. General bug fixing.

  25. Modified bdsImportBlacknestDatabase to suit more recent Blacknest Autodrm database schema.

With this release there are updates to the database Schema. There is an additional command "bdsDatabaseUpdate" that should be run after package update. This will Update the database schema and modify appropriate entries. So the update process is:

  1. Shutdown bdsServer: "service bdsServer stop"

  2. Update the BDS software: "yum update"

  3. Update database users: "mysql -uroot -p<root-password> < /usr/bds/bdsSql/permissionsFix.sql" (Special fix to database user names)

  4. Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"

  5. Restart the BDS system. "bdsRestart"

Release 1.0.7 - 2009-06-03

  1. BDRS data converter: Added support for reading files with corrupted first block and some other data validation improvements.

Release 1.0.6 - 2009-06-01

  1. 215,220,225,229: VELA timecode track, end of year problem. If a new tape was started when the VELA time code generator had gone beyond the end of the previous year, the year would be decoded incorrectly. This has been fixed.

  2. Added data converters for WRA-40, WRA-64 and WRA-AGSO data formats.

  3. Added "-ignoreCorruptions" option to BdsImportData program to fix data files by ignoring corrupted blocks. The data converters will scan for the next valid block when an invalid block is found on a byte by byte basis.

  4. Added "-dryRun" option to BdsImportData program to perform import test without actually importing the data.

  5. BdsImportData now as a "-warnings" flag which will list the data import warnings. The warnings are also added to the BDS MetaData for the data file so that they can be investigated at a later date.

  6. Various data import validation improvements.

Release 1.0.5 - 2009-04-02

Minor bug fixes:

Release 1.0.4 - 2009-03-27

Minor bug fixes:

Release 1.0.3 - 2009-03-23

Minor bug fixes:

Release 1.0.2 - 2009-03-16

Minor bug fixes:

Release 1.0.1 - 2009-03-10

Minor bug fixes.

Release 1.0.0 - 2009-03-04

This is the first release of the BDS software.