Account
Software Interface Specification for the
Radio Astronomy FFT Processor
AstroFFT - 2.0.1
| Reference | swspec8 |
| Date | 1/3/05 |
Introduction
This is a specification for the software interfaces used in the AstroFFT processing system.FFT Control process "astrofft"
The main software process used in the AstroFFT processing system is the astrofft process. This software process is responsible for managing the FPGA hardware FFT engine, real-time post processing the FFT data, data storage and system control. It has a configuration file named /etc/astrofft.conf that sets up some basic operating parameters. The parameters in the configuration file include:| Parameter | Description |
| DataDirectory | A full path to the data directory to use for data storage |
| RemoteHosts | A list of names or IP addresses of hosts allowed to access the astroFft process. This may be set to "*" to indicate all hosts have access or not set where only programs running on the local astroFFT system can access the astroFft process. |
| NumChannels | The number of FPGA channels to process. This is set equal to the number of FPGA processing boards installed. Currently 1 or 2 boards are supported. |
| IntScale | Defines which 32bits of the 64bit integrator values are output. 0 - 63:32, 1 - 55:24, 2 - 47:16, 3 - 39:8. The default value is 1. |
| Simulate | If set to 0 the system will not produce data for missing FPGA processing engine boards. If set to 1 the system will produce simulated data for missing FPGA processing boards. |
The Astrofft process runs in superuser mode as a real-time process with all of its memory locked into physical memory.
Operation
The Astrofft process runs as a daemon on the Astro processing system. Client applications, either running locally or running on remote systems can connect to the Astrofft process and command it to perfrm processing runs.The client first sets up the various run parameters and then starts the processing run. The input analogue signals are processed and the resulting data is stored into data files on the systems hard disk. It is also possible for the data to be sent across the network conection to the client application.
On completion a message is sent to the cleint application.
The Astrofft daemon is normally started at system boot time by the script file /etc/init.d/astrofft. It can be stoped with the command: "service astrofft stop" and started with the command "service astrofft start " (On debian systems use the commands: /etc/init.d/astrofft stop and /etc/init.d/astrofft start".
AstroFFT Control Process Command and Data interfaces
The Astrofft process has a control interface accessible by means of a system socket named astrofft which has by default the number 41000 and a data interface by means of a system socket named astrofftdata which has the default number 41001. The dual socket interface allows a controlling process to control the FFT process and to sample the current data output for display to the user. The main FFT data will be stored into data and information files automatically in the data directory. The controlling process can run either on the local Astro FFT system or on a remote networked host.The protocol used on the control interface is based on ASCII commands and result codes. This allows the system to be controlled via a simple telnet connection or a fully fledged GUI control application.
Each command consists of a case sensitive string consisting of a command keyword followed by arguments separated by spaces and terminated with a line feed character. Strings are enclosed within double quotes and spaces consist of a sequence of space or tab characters.
Each command will yield a result in the following form:
<integer status> <string status>
The currently defined return status values are:
| Status Number | Status String | Description |
| 0 | ok | Command completed with no error |
| 0 | <Current state> | Information on the current state of the system. Space separated list of settings. |
| 1 | <error string> | Some undefined error occurred |
| 2 | <error string> | There was a command syntax error |
Most of the commands can only be used while the system is in the stopped state. The exceptions to this are the run, pause, setinfo, getSate, getStateLines and getParam commands. The commands and protocol provided by this interface is as follows:
| Command | Default | Description |
| setMode <mode string> | qfft | Sets the overall operational mode of the system. The modes are listed below. |
| setSampleFrequency <freq number> | 0 | Sets the sample frequency to use. The frequencies are listed below. |
| setClockMode <0,1> | 0 | Sets the clock mode to 0 - internal clock generator or 1 - external clock generator. |
| setTitle <"string"> | "" | Sets the title to place in the runs information file. |
| setProject <"string"> | "" | Sets the project the data belongs to. This defines the directory within the data file system where the data files are stored. |
| setFileBaseName <"string"> | "data" | Sets the base file name for data output. The given name will be extended with an underscore followed by a 4 digit number . The number will be automaticly set to the next available file name in the data storage directory. The base file name is stored in the "fileBaseName" parameter. The parameter "fileName" contains the extended file name actually used. |
| setFileFormat <binary,ascii> | binary | Sets the file format for the output file. Either binary or ASCII formats are supported. |
| setSockFormat <binary,ascii> | binary | Sets the data format for the output data socket. Either binary or ASCII formats are supported. |
| setAverageNumber <number> | 610 | Sets the number of FFT results to average within the FPGA FFT process. Each averaged run is sent to the host. The maximum and minimum values of this parameter have yet to be decided. The default value will be 610 which will provide an FFT result approximately once every 100 ms. If this figure is made too low the software will not be able to keep up with the data rate and this will cause data overrun errors. If the number is set to to large a value, the FPGA's internal integrator is liable to overflow. |
| setNumber <number> | 1 | Sets the total number of averaged FPGA FFT results in the run. |
| setFileAverageNumber <average number> | 0 | Sets the number of FFT operations that are averaged within the software post processing to form a data file result output vector. 0 disables this output. |
| setSockAverageNumber <number> | 1 | Sets the number of FFT operations that are averaged within the software post processing to form a socket data result output vector. 0 disables this output. |
| setMessages <1, 0> | 0 | Enables/Disables the sending of the messages on the data socket. |
| run <0,1> | 0 | Starts or stops the FFT process. 0 - stop, 1 - start. |
| pause <0,1> | 0 | Pauses the FFT process. 0 - run, 1 - pause. |
| setInfo <number> | 0 | Sets the information number that is attached to each data record. This can be used to indicate that records are from a callibration source rather than a real data source. This command can be used while the system is in the run state. |
| setPosition <type,pos1, pos2> | 0,0.0,0.0 | Sets the telescope position information fields that are attached to each data record. This command can be used while the system is in the run state. The Type field is an integer specifying the position type information. This is user defineable, except that 0 means no position information is present. The pos1 and pos2 fields are 32bit floating point values. |
| getState | Gets the current state of the FFT process. See the table following for the format of this data. | |
| getStateLines | Gets the current state of the FFT process as a number of lines of name/value pairs. | |
| getParam | <"string"> | Gets the value of the named state parameter. |
| setState <parameters> | Sets the current state of the FFT process. The data format is the same as for the getState command. Note however that some of the parameters cannot be set (such as DataSize), these will be ignored. |
The operational modes are:
| Mode String | Description |
| fft | Normal FFT processing. |
| qfft | Quadrature sampled FFT processing. |
| cfft | Combined FFT processing. |
| rfft | Real mode FFT (post processed FFT) |
| analogue | Analogue channels 1 and 2 |
The frequency settings are:
| Frequency Number | Sampling Frequency clock is 50 Mhz |
| 0 | 50 Mhz |
| 1 | 25 Mhz |
| 2 | 12.5 Mhz |
| 3 | 6.25 Mhz |
| 4 | 3.125 Mhz |
| 5 | 1.5625 Mhz |
The getState command returns the state of the astrofft process as a set of comma separated parameters in ASCII format. The parameters are in the following order:
| Position | Type | Parameter | Description |
| 0 | integer | run | Indicates it the FFT process is running. |
| 1 | integer | pause | Indicates if the FFT process is paused. |
| 2 | integer | messages | Indicates if messages sent over the data socket are enabled. |
| 3 | string | mode | Gives the current mode the system is running in. |
| 4 | integer | clockMode | The current clock mode either 0 or 1. |
| 5 | integer | sampleFrequency | The current clock frequency. |
| 6 | integer | averageNummber | Gives the number of FFT results that are averaged within the FPGA FFT process. |
| 7 | integer | number | Gives the total number of averaged FFT result vectors that are post processed. |
| 8 | integer | fileAverageNumber | Gives the number of averaged FFT result vectors that are averaged in the software post processing stage before storing in the data file. |
| 9 | integer | socketAverageNumber | Gives the number of averaged FFT result vectors that are averaged in the software post processing stage before sending over the data socket. |
| 10 | string | title | Gives the title of the run. |
| 11 | string | project | Gives the current project name. Data files will be stored in a directory of this name. |
| 12 | string | fileBaseName | Gives the base file name. |
| 13 | string | fileName | Gives the file name the data and information is stored into. This will get set after the run has been started and is based upon the fileBaseName parameter. |
| 14 | string | fileFormat | Describes the data file format. Either binary or ascii. |
| 15 | string | socketFormat | Describes the data socket format. Either binary or ascii. |
| 16 | integer | info | Gives the state of the user settable info parameter. |
Data Socket Format
The data socket interface is provided so that a controlling program can get status messages and samples of the output data sent to it for controlling purposes. It is also possible to bypass the local data storage mode of the Astrofft system and obtain all of the output data through this interface if required. The socket returns both data and messages from the output of the processing system in binary or ASCII form.If the socket interface format has been set to ASCII, the format of the returned data will consist of lines of comma separated floating point values as follows:
<channel>,<subchan>,<error>,<info>,<clips>,<amplitude>,<status>,<posType>,<pos1>,<pos2>,0,0,<data0>,<data1>,<data2>, ....
If the socket data format is binary then the output consists of 64byte header, followed by an array of double precision (8 byte) values. The data packet format is defined in the 'C' header file AstroPacket.h.
The header contains the following information in little endian format.
| Field | Type | Usage |
| length | 32bit unsigned integer | Total Number of bytes in the message including header |
| channel | 32bit unsigned integer | The channel the data is from |
| subchan | 32bit unsigned integer | The sub channel the data is from. This is normally set to 1. In analogue mode it is set to 1 or 2 to indicate the sampler channel that the data is from. |
| error | 32bit unsigned integer | The bit mask of error bits. These are only set if the Astrofft process is not configure to retry on error. |
| info | 32bit unsigned integer | User settable information field |
| clips | 32bit unsigned integer | The number of input samples that were clipped by the ATD conversion process. This is only currently supported in analogue mode. |
| status | 32bit unsigned integer | Status information. Used with message channel to indicate message type. |
| time_sec | 32bit unsigned integer | Time in seconds when data was sampled from Unix EPOC. |
| time_usec | 32bit unsigned integer | Time in micro seconds when data was sampled. |
| posType | 32bit unsigned integer | The type of position information carried in the next two fields. 0 means no position information any other value is user defined. |
| pos1 | 32bit floating point | A 32bit floating point value defining the telescopes pointing position. The actual meaning of this value is defined by the posType field. |
| pos2 | 32bit floating point | A 32bit floating point value defining the telescopes pointing position. The actual meaning of this value is defined by the posType field. |
| dummy[2] | 32bit unsigned integer | 2 x 32 bit integers, reserved for future use (pads header to an 64 byte boundary) |
| amplitude | 64bit floating point | This value indicates the RMS amplitude of the input signal. This is only currently supported in analogue mode. |
The channel value in both cases indicates the type of data within the message. The channels of data output on this socket include:
| Type Value | Data Following |
| 0 | Message channel, sends message strings such as "Run Complete" |
| 1 | Channel 1's Data array in binary or ASCII formats |
| 2 | Channel 2's Data array in binary or ASCII formats |
| Error Bit | Usage |
| 0x01 | Fpga General Fault |
| 0x02 | Fpga Data Collection buffer overflow |
| 0x04 | Fpga Data Collection timeout |
| Status Number | Usage |
| 0 | No status info |
| 1 | Run complete |
Messages are only sent if the setMessages flag is set to 1.
Data File Format
On the start of each run three files are created in a data storage directory on the AstroFFT system. If the project field is not null a directory of the projects name is used for all of data files, otherwise the files are stored in the root of the data storage directory. One of these files stores information on the processing run while the others store the actual data for each channel.The base name of these files will be as given by the setFileName command. Each files name will have an underscore followed by a 4 digit numeric extension appended. This number is obtained from the file .data which is stored in the data directory. The number in the .data file is incremented on each run.
I addition to the numeric extension the additional extension _1.dat _2.dat or .inf appended for the data and information files respectively.
The information file is in an ASCII text format with a format consisting of lines of text. Each line has a parameter name with terminating colon, followed by the settings for that parameter. The following is an example information file:
Title: NoneThe data files can be in either binary or ASCII formats. The default and recommended format is binary. The data file format consists of an array of records.
Project: proj1
FileName: data_0001
FileFormat: binary
Mode: qfft
ClockMode: 0
ClockFrequency:0
Number: 10
AverageNumber: 610
FileAverageNumber: 1
DateStarted: 0 2002-04-24T09:44:11.342Z
DateStopped: 10 2002-04-24T09:44:12.438Z
Each record has the same format as described in the Data Socket section in either binary or ASCII.
The time format is in ISO 8601 form and is in Universal time.
Errors
A number of errors can occur during a processing run. The Astrofft process will report these to the client application, in the form of a message, and to the system logger where they will be appended to the file /var/log/messages. By default, errors that can be recovered from by loseing some data, are handled by retrying the last FPGA data processing cycle.| Error | Handling | Description |
| FPGA Data Over run | Retry last FPGA average cycle | If the host system cannot keep up with the rate of data coming from the FPGA, a data over run error will occur. In this case the Astrofft process will ignore the last set of data from the FPGA and restart the FPGA. No errors will be reported to the client application but a warning will be issued to the system logger. |
| FPGA Integrator Over range | Retry last FPGA average cycle | If the incoming data stream causes an over range in the FPGA's integrator an integrator over range error will occur. In this case the Astrofft process will ignore the last set of data from the FPGA and restart the FPGA. No errors will be reported to the client application but a warning will be issued to the system logger. |
| FPGa Data Timeout | Retry last FPGA average cycle | If the software timesout awaiting the next data set from the FPGA a timeout will occur. In this case the Astrofft process will ignore the last set of data from the FPGA and restart the FPGA. No errors will be reported to the client application but a warning will be issued to the system logger. |
| FPGA board fail | Operate system without this board | If one of the FPGA boards is not present or fails, the system will continue to function with just the single board. In this case a warning will be sent to the client application and the system logger and no data will be captured for the failed channel. |
| Disk write error | Aborts current processing run with an error | If there is a failier when writing to the data files , the Astrofft process will abort the run with an error message to the Client application and system logger. Note that the run will be aborted at the start command if there is insufficient disk space available at that time. |