BDS Release Notes
Release 1.2.0 - 2010-03-01
This is a major update to the system, fixing and changing some of the core internal structure.
- Changes to BDS data file format. This adds channel and segment
info to the data blocks to improve the performance of dealing with
segments of data when there are missing blocks or in other
multi-segmented data applications. Also improved block searching
performance and other minor improvements.
- Major changes to the DataSet class. This is responsibe for
gathering the data from multiple channels of multiple segments in
multiple files and providing access top the seismic data conatined. This will have fixed many issues with data access.
- 89: Automatically break data into segments when there are missing blocks
- 526: DataAccess wild card matching not supported
- 498: non-synchronously sampled" error when exporting (non-multiplexed) ASCII data over a file boundary
- 84: Improved operation when there are multiple segments of data
- 330: Break data into segments on MetaData changes
- 303: Automatically break data into segments when there are metadata changes
- 498: "non-synchronously sampled" error when exporting (non-multiplexed) ASCII data over a file boundary
- Modified the ASCII data format to support Channel Multiplexed:
ASCII-CM and Sample Multiplexed: ASCII-SM formats and multiple
segments. Multiple segments are implemented by adding a comment line
starting with a "#" with channel, segment, time and other information.
- 338: failure to export EKA Red line
- 404: tapeDigitiser data import: Delete multiple FM deviation errors.
Added InfoExtra storage in BDS data file format in in BDS API to store
the extra TapeDigitiser error information. This speeds up data access
but still allows full storage of the contents of a TapeDigitiser data
file.
- 385: TapeDigitiser import fails when there is an "empty" data file
- 371: TapeDigitiser import failing for lack of metadata
- 336: TapeDigitiser data with huge numbers of errors
- 597: Add Calibrations name field
- 513: Metadata overlap error message when modifying "response" information
- 588: poles/zeros in output SEED files read by rdseed
The system now uses base SI units of all data and uses
calibrationFactors that will convert the data samples to the base SI
units in question. When exporting in a particular data format
unit/scaling conversion is done as appropriate.
- 507: BKNAS format when many poles/zeros
- Fixed crash in BdsServer when trying to export SEED-MINI data.
- Fixed problem with SEED sensitivity value.
- Changed SEED dip angle and gain calculations. Set Stage Number to 0.
- Changes units operation.
- Fixed problem with dataImport continuing when SEED data format errors were present.
- 634: Add support for SEED-METADATA file format
- Modifications to BdsTest system to help test the above changes.
- 432: Login window disappears after one failed attempt
- 433: TCP reset requires to have to be restarted, Access Denied error
- 488: DataFiles tab shows meaningless import data
- 537: File channel metadata editing
- 425: DataSelect tab multi select
- 427: bdsAdminGui dataselect->dataview usability issues
- 497: "Update" button on channels window didn't work
- 430: DataView channel shift, 431: DataView seems not to work when displaying multiple channels
A simplified DataView tab has been implemented. This allows the viewing
of multiple data channels with multiple data segments, possibly
overlapping as well as channels at differeing sample rates. It has a
simple autoscaling function.
- 647: default file name formation fails when station name includes #
- Numerous higher level MetaData editing functions added to "Options" menus.
- BdsImportBlacknestDatabase: Added support for importing Pseudo stations.
- Removed SensorLocations table and added depth and sensor angles to Calibrations.
- Note that the BDS API has been updated in this release, so any client applications will need re-compiling.
- Note that the BDS data file format has changed and, for
simplicity at this stage of development, no compatibility with the
previous file format has been provided. So all data files will need to
be re-imported.
With this release there are updates to the database Schema to version 5. There
is an additional command "bdsDatabaseUpdate" that should be run after
package update. This will Update the database schema and modify
appropriate entries. So the update process is:
Shutdown bdsServer: "service bdsServer stop"
Update the BDS software: "yum update"
Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"
Restart the BDS system. "bdsRestart"
Release 1.1.9 - 2010-01-26
- Changed default import data source to "Main" from "Master".
- System now sets importUserId and importTime fields in DataFiles information.
- Preliminary SEED data file import.
- BdsImportData: added "-sequential" flag. When used it will import
the given set of files sequentially into a single import file. This is
especially usefull for SEED-TAR format files. The files have to contain
the same channels in the same order and be completely sequential in
time.
- 570: SEED Export converter. The system can now export data in SEED 2.4 and SEED-MINI 2.4 formats.
- DataSelections invovling an array in combination with a set of individual stations did not work correctly. This has been fixed.
- Data selections within a single file that contained
non-synchronously sampled data at different sample rates did not work
corectly. This has been fixed.
- Added -formatList option to bdsImportData.
- 428: bdsAdminGui long processes need hourglass cursor
- 104: Add ability to manipulate Sensor Data information and delete sensor data files
- 405: New Data files window - various
- 523: Delete data files.
System now deletes data files by moving them to a directory "Deleted"
on the store in question. These will need to be actually deleted at a
later date by an operator.
- 350: Improve data access performance by elimimanting second getInfo
- 27: default filename in GUI
- 417: Time date widget defaults to 2002
- 424: DataSelect tab usability issues
- Changed
DataSelection system in bdsAdminGui. Now provides wildcard filter on
each item and multi-selection restricted to database entries. Note that
selection restriction for data ignores the time fields. This is for
performance reasons. It would be possible to implement a cache of
distinct, time based, channel information to allow selection
restrictions based on actual data time periods if required.
- Changes to use "*" wild carding system in bdsDataAccess and bdsAdminGui.
- System now checks the "state" parameter on data files before serning the data.
- Added ability to list default file format extensions.
- Added BdsDataFormats manual and updated other manuals and documents.
- A few minor bugs have been fixed.
- Note that the BDS API has been updated in this release, so any client applications will need re-compiling.
With this release there are updates to the database Schema. There
is an additional command "bdsDatabaseUpdate" that should be run after
package update. This will Update the database schema and modify
appropriate entries. So the update process is:
Shutdown bdsServer: "service bdsServer stop"
Update the BDS software: "yum update"
Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"
Restart the BDS system. "bdsRestart"
Release 1.1.8 - 2009-12-14
This is an interim bug release version. The BDS API has changed so that any client machines will need to be updated as well.
This release is now built on Red Hat Enterprise Linux 5.4 and has a new
yum repository. The repository change will be automatic, but two yum
updates are required, one to update the repository info and one to
actually perform the update.
- 455: Window hiding issue on certain window managers
- 464: DataViewInfo DataViewChans inconsistent inital state
- 465: UserInfo tab, User Edit dialog group selection disappears
- 461: Dialogue window on Changes tab list
- 411: BdsAdminGui: Update selection lists in channel on updates
- 414: Database does not keep up with station/channel name change
- 474: Number of channels incorrectly calculated?
- 475: Wrong hang & vang
- Added "datum" import from Autodrm database in bdsImportBlacknestDatabase.
- Removed wild card from stations in Autodrm.
- 416: System puts all data files in a single directory Note that this does not update the location of any existing files imported into the BDS system.
- 401: Change source "Default" to "Prioritised"
- 460: Inconsistent list behaviour with modify
- 494: BDS fills /tmp
- 495: bdsControl exectuable missing from rpms
- 511: BKNAS instrument number
- 508: BKNAS format 1st line of Channel description
- 501: precision of "calib" for BKNI data Note that this bug affeced MetaData imports so values in the BDS SQL database for calibrationFactor could be wrong.
- 355: Add a test Array/Stations/Channels for performing automatic regression tests The first pass system has been created. This just creates BDRS files at the moment but export tests are generic.
- Added the "NETWORK" keyword to the BdsAutodrm. This allows the network to be set for test puposes.
- 492: StationLocation selection in Stations needs Network This has been added.
- 453: BDS AutoDRM acts as a relay
The system now ignores emails from users not registered in the BDS
users database. It checks the "ReturnPath" and "From" field headers for
the users email address. Errors are looged into the syslog of the
bdsAutodrm daemon's host.
- 514: Interrupted data import results in bad data status. This has been fixed.
- 525: DataAccess error message Error text changed.
- 402: Data import error/warning message improvement Warnings are now presented as a comma separated list.
- 403: BdsImportData: Add "-info" flag. Added -description and -addWarning flags to system. See the documentation in https://portal.beam.ltd.uk/support/blacknest/files/bds/doc/BdsImport.pdf for more information on the formats of the warning messages.
- 545: bdsImportTapeDigitiserData. Added ability to read jobInfo.tdim if it exists instead of jobInfo.tdi.
- Added import warnings when blocks are re-ordered. The warning
includes a description of the block re-ordering performed in the form
of an ASCII string. The format of this string is a list of block ranges
and the re-ordered position. For example "0-10:30" means blocks 0
through 10 have been moved to position 30.
- Added the Blacknest-testing yum repository for test updates. This is not enabled by default.
Release 1.1.7 - 2009-10-02
- Added support for setting source names from format in bdsImportBlacknestDatabase program.
- Fixed a bug where the source field was not being used to validate the addition of new ChannelInstruments.
Release 1.1.6 - 2009-09-25
- Fixed crash on DataFile deletions.
- 282: multi-multiplexed data in channels 19 and 20 of BDRS_UKNET data Added support for BDRS-MM data files.
Release 1.1.5 - 2009-09-25
- 381: BOAP connects to localhost. This has been fixed
- 367: BDRS UKNET 10-Hz data import Added support for BDRS files with 10Hz sampling rate.
Release 1.1.4 - 2009-09-16
- 338: CalibrationUnits now set from channel name.
- 389: BKNAS-1 output calibration factor is written where pole-zero constant should be Now calculates the PoleZero constant for the PoleZero output in BKNAS format from the PoleZero table.
- 385: TapeDigitiser import fails when there is an "empty" data file Added support for zero data length TapeDigitiser files.
- 390: EKA BB GCF corrupt data This has been fixed.
- Added, first pass, DataFile and DataChannel API functions and editing ability.
Release 1.1.3 - 2009-09-08
- Removed debug print from bdsImportData with sample multiplexed file import.
Release 1.1.2 - 2009-09-07
Release 1.1.1 - 2009-08-17
- 341: WRA_AGSO alias in bdsImportData "-format" option. This has been added.
- 343: MaxNumSamples in bdsServer.conf. Changed to 1024 by default.
- Fixed a bug in calculating the number of datasamples/sample rate in a set of data.
- 342: "sample rate incorrect" for WRA40 and WRA_AGSO data There was a bug in the WRA-AGSO sample rate calculation routine when crossing a year boundary, This has been fixed.
- 340: Add -listFormats option to bdsDataAccess and use in bdsAdminGui
- There was a bug that caused data import to fail with the error "Unknown output format:" this has been fixed.
Release 1.1.0 - 2009-08-06
- 57: Added snapshotPause call. The API function takes a boolean
argument, on or off to set the BdsServer into pause mode for backup
puposes. See the BdsServer manual for more information.
- 57: Added the modeSet API call. This allows the BdsServer to be put
into a read-only slave mode. See the BdsServer manual for more
information.
- 57: Added the security group "control". The snapshotPause and modeSet
calls require the user to belong to this group.
- 57: Adde the user "bdsBackup" for backup purposes. This user belongs
to the control group but has no other abilities.
- 57: Added "state" field to DataFiles table to indicate the data files
state. This can be set to "importing" or "ok" and can be used for data
synchronisation puposes.
- 270: Added the "Config" tabale to the database to contain
configuration parameters. One configuration parameters, "schemaVersion"
has been added to this to track the BDS MySQL data base schema version.
This allows for automated database schema updates on software updates.
Add the command line program bdsDatabaseUpdate command to update the
ruuning database's schema to the current version.
- 253: Change the BdsServer's database user to "bdsMain" and set the
older "bds" user for read-only access. This allows external programs to
read the BDS's database safely. The database grant's need to be updated
with the following commands for this to work and the
/etc/bdsServer.conf file needs modification.
grant all on BDS.* to bdsMain@"%" identified by "bdsMain";
grant all on BDS.* to bdsMain@"localhost" identified by "bdsMain";
grant select on BDS.* to bds@"%" identified by "bds";
grant select on BDS.* to bds@"localhost" identified by "bds";
- 268: Added support for re-ordering data blocks and deleting
duplicate data blocks on import. The "-reorder" and "-deleteDuplicate"
flags have been aded to bdsDataImport for this. Currently only the GCF
format convertor supports these options, but they can be added to each
or all data convertors relatively easily if required.
- Fixed: 278: cannot add new SensorLocation without saving-and-reentering
- Fixed: 264: AdminGui update bug in DataSelect window
- Fixed: 283: extracting "filename time" from EKA SCREAM data files
- Fixed: 289: AdminGui bug for a particular GCF file - "QG Vector::insert: index xx out of range"
- Fixed: 291: BKNAS1 format
- Fixed: 306: AdminGui editing channel responses
- Fixed: 265: No. of samples in 2nd segment of 2-segment IMS exported file
- Fixed: 286: sampling rate for EKA GCF data
- Fixed: 210: TapeDigitiser import two-tape sets
- 311: On importing TapeDigitiser files keep original file name The BdsImportTapeDigitiserData program now lists the actual file import in the Description field.
- 325: Fixed: Database description entries should allow the "' " character
- 336: TapeDigitiser data with huge numbers of errors
We have improved the perormance of the BDictString class to help this.
Howvere the number of errors produced by the tapeDigitisation process
needs to be looked at.
- Added support for Station Aliases. These can be used to rename
the pusedo stations used for extra data channels that are not
associated with any physical stration, to the Array name as well as for
other puposes. The Aliases are only normally used in data output
formats.
- Added support for MetaData sources that match the data sources.
This allows different MetaData parameters to be attached to different
data sources.
- Added support for Source Priority to decide which data Source to select if a data source of "Default" is used.
- General bug fixing.
- Modified bdsImportBlacknestDatabase to suit more recent Blacknest Autodrm database schema.
With this release there are updates to the database Schema. There
is an additional command "bdsDatabaseUpdate" that should be run after
package update. This will Update the database schema and modify
appropriate entries. So the update process is:
- Shutdown bdsServer: "service bdsServer stop"
- Update the BDS software: "yum update"
- Update database users: "mysql -uroot -p<root-password> <
/usr/bds/bdsSql/permissionsFix.sql" (Special fix to database user names)
- Update the database schema: "bdsDatabaseUpdate -u bdsMain -p bdsMain"
- Restart the BDS system. "bdsRestart"
Release 1.0.7 - 2009-06-03
- BDRS data converter: Added support for reading files with
corrupted first block and some other data validation improvements.
Release 1.0.6 - 2009-06-01
- 215,220,225,229: VELA timecode track, end of year problem. If a
new tape was started when the VELA time code generator had gone beyond
the end of the previous year, the year would be decoded incorrectly.
This has been fixed.
- Added data converters for WRA-40, WRA-64 and WRA-AGSO data
formats.
- Added "-ignoreCorruptions" option to BdsImportData program to
fix data files by ignoring corrupted blocks. The data converters will
scan for the next valid block when an invalid block is found on a byte
by byte basis.
- Added "-dryRun" option to BdsImportData program to perform import
test without actually importing the data.
- BdsImportData now as a "-warnings" flag which will list the data
import warnings. The warnings are also added to the BDS MetaData for
the data file so that they can be investigated at a later date.
- Various data import validation improvements.
Release 1.0.5 - 2009-04-02
Minor bug fixes:
- 212:
BDS Instrument name Metadata issue. This was a bug in the BDS server
when there were multiple instruments used on a
channel at different time periods. The system was choosing the first
entry rather than the one for the time period in question.
- 217: BdsAdminGui:
Adding new stations problem
- Added the utility program bdsDataFileCreate. This program
can create test sensor data files in any of the formats supported by
the BDS system with write data convertors.
- Added the ability for bdsImportData to import sensor data files
in the BDS format.
- Made boapns daemon run as the daemon user by default.
Release 1.0.4 - 2009-03-27
Minor bug fixes:
- Bug in BdsImportData when ignoring missing data blocks. This
would cause a segment fault.
- 195: Fix to TapeDigitiser import. VELA timecode track, end of
year problem. This problem was
caused by problems in the VELA timecode track near the year end on tape
EKA 3616.There was some curruption in the VELA timecode track and then
a
missing bit of VELA time code followed by and incorrect year wrap. The
software could not cope with this unusual circumstance. We have added
some code to handle this case.
- Added Channel Aux field to BKNAS channel output lines based on
Sheilas BKNAS modification.
- Modified BdsAutodrm so that all channel names have the ".*"
postfix added so that channel types given will match all channel names
of that type irrelevent of the AuxId.
Release 1.0.3 - 2009-03-23
Minor bug fixes:
- 158:
End of day/End of year crossing time stamp errors in
formatted data output. The time stamps listed in the data output where
set to the user requested time not the actual data times from the
blocks. This has been fixed.
- 187:
Problem with bdsImportTapeDigitiserData validating meta data. This has
been fixed.
Release 1.0.2 - 2009-03-16
Minor bug fixes:
- Fixed bdsImportScream file handle leak.
- Fixed bdsImportTapedigitiserData to find supporting programs in
standard path.
- Improved Pole/Zero editor.
Release 1.0.1 - 2009-03-10
Minor bug fixes.
Release 1.0.0 - 2009-03-04
This is the first release of the BDS software.