Software Interface Specification for the

Radio Astronomy FFT Processor

AstroFFT - 2.0.1


 
Reference swspec8
Date 1/3/05

Introduction

This is a specification for the software interfaces used in the AstroFFT processing system.

FFT Control process "astrofft"

The main software process used in the AstroFFT processing system is the astrofft process. This software process is responsible for managing the FPGA hardware FFT engine, real-time post processing the FFT data, data storage and system control. It has a configuration file named /etc/astrofft.conf that sets up some basic operating parameters. The parameters in the configuration file include:
 
 
Parameter Description
DataDirectory A full path to the data directory to use for data storage
RemoteHosts A list of names or IP addresses of hosts allowed to access the astroFft process. This may be set to "*" to indicate all hosts have access or not set where only programs running on the local astroFFT system can access the astroFft process.
NumChannels
The number of FPGA channels to process. This is set equal to the number of FPGA processing boards installed. Currently 1 or 2 boards are supported.
IntScale
Defines which 32bits of the 64bit integrator values are output. 0 - 63:32, 1 - 55:24, 2 - 47:16, 3 - 39:8. The default value is 1.
Simulate If set to 0 the system will not produce data for missing FPGA processing engine boards. If set to 1 the system will produce simulated data for missing FPGA processing boards.

The Astrofft process runs in superuser mode as a real-time process with all of its memory locked into physical memory.

Operation

The Astrofft process runs as a daemon on the Astro processing system. Client applications, either running locally or running on remote systems can connect to the Astrofft process and command it to perfrm processing runs.
The client first sets up the various run parameters and then starts the processing run. The input analogue signals are processed and the resulting data is stored into data files on the systems hard disk. It is also possible for the data to be sent across the network conection to the client application.
On completion a message is sent to the cleint application.
The Astrofft daemon is normally started at system boot time by the script file /etc/init.d/astrofft. It can be stoped with the command: "service astrofft stop" and started with the command "service astrofft start " (On debian systems use the commands: /etc/init.d/astrofft stop and /etc/init.d/astrofft start".

AstroFFT Control Process Command and Data interfaces

The Astrofft process has a control interface accessible by means of a system socket named astrofft which has by default the number 41000 and a data interface by means of a system socket named astrofftdata which has the default number 41001. The dual socket interface allows a controlling process to control the FFT process and to sample the current data output for display to the user. The main FFT data will be stored into data and information files automatically in the data directory. The controlling process can run either on the local Astro FFT system or on a remote networked host.
The protocol used on the control interface is based on ASCII commands and result codes. This allows the system to be controlled via a simple telnet connection or a fully fledged GUI control application.
Each command consists of a case sensitive string consisting of a command keyword followed by arguments separated by spaces and terminated with a line feed character. Strings are enclosed within double quotes and spaces consist of a sequence of space or tab characters.
Each command will yield a result in the following form:

<integer status> <string status>

The currently defined return status values are:
 
 
Status Number Status String Description
0 ok Command completed with no error
0 <Current state> Information on the current state of the system. Space separated list of settings.
1 <error string> Some undefined error occurred
2 <error string> There was a command syntax error

Most of the commands can only be used while the system is in the stopped state. The exceptions to this are the run, pause, setinfo, getSate, getStateLines and getParam commands. The commands and protocol provided by this interface is as follows:
 
 
Command Default Description
setMode <mode string> qfft Sets the overall operational mode of the system. The modes are listed below.
setSampleFrequency <freq number> 0 Sets the sample frequency to use. The frequencies are listed below.
setClockMode <0,1> 0 Sets the clock mode to 0 - internal clock generator or 1 - external clock generator.
setTitle <"string"> "" Sets the title to place in the runs information file.
setProject <"string"> "" Sets the project the data belongs to. This defines the directory within the data file system where the data files are stored.
setFileBaseName <"string"> "data" Sets the base file name for data output. The given name will be extended with an underscore followed by a 4 digit number . The number will be automaticly set to the next available file name in the data storage directory. The base file name is stored in the "fileBaseName" parameter. The parameter "fileName" contains the extended file name actually used.
setFileFormat <binary,ascii> binary Sets the file format for the output file. Either binary or ASCII formats are supported.
setSockFormat <binary,ascii> binary Sets the data format for the output data socket. Either binary or ASCII formats are supported.
setAverageNumber <number> 610 Sets the number of FFT results to average within the FPGA FFT process. Each averaged run is sent to the host. The maximum and minimum values of this parameter have yet to be decided. The default value will be 610 which will provide an FFT result approximately once every 100 ms. If this figure is made too low the software will not be able to keep up with the data rate and this will cause data overrun errors. If the number is set to to large a value, the FPGA's internal integrator is liable to overflow.
setNumber <number> 1 Sets the total number of averaged FPGA FFT results in the run.
setFileAverageNumber <average number> 0 Sets the number of FFT operations that are averaged within the software post processing to form a data file result output vector. 0 disables this output.
setSockAverageNumber <number> 1 Sets the number of FFT operations that are averaged within the software post processing to form a socket data result output vector. 0 disables this output.
setMessages <1, 0> 0 Enables/Disables the sending of the messages on the data socket.
run <0,1> 0 Starts or stops the FFT process. 0 - stop, 1 - start.
pause <0,1> 0 Pauses the FFT process. 0 - run, 1 - pause.
setInfo <number> 0 Sets the information number that is attached to each data record. This can be used to indicate that records are from a callibration source rather than a real data source. This command can be used while the system is in the run state.
setPosition <type,pos1, pos2>
0,0.0,0.0
Sets the telescope position information fields that are attached to each data record. This command can be used while the system is in the run state. The Type field is an integer specifying the position type information. This is user defineable, except that 0 means no position information is present.
The pos1 and pos2 fields are 32bit floating point values.
getState
Gets the current state of the FFT process. See the table following for the format of this data.
getStateLines
Gets the current state of the FFT process as a number of lines of name/value pairs.
getParam <"string"> Gets the value of the named state parameter.
setState <parameters>
Sets the current state of the FFT process. The data format is the same as for the getState command. Note however that some of the parameters cannot be set (such as DataSize), these will be ignored.

The operational modes are:
 
 
Mode String Description
fft Normal FFT processing.
qfft Quadrature sampled FFT processing.
cfft Combined FFT processing.
rfft Real mode FFT (post processed FFT)
analogue Analogue channels 1 and 2

The frequency settings are:
 
 
Frequency Number Sampling Frequency clock is 50 Mhz
0 50 Mhz
1 25 Mhz
2 12.5 Mhz
3 6.25 Mhz
4 3.125 Mhz
5 1.5625 Mhz

The getState command returns the state of the astrofft process as a set of comma separated parameters in ASCII format. The parameters are in the following order:
 
 
Position Type Parameter Description
0 integer run Indicates it the FFT process is running.
1 integer pause Indicates if the FFT process is paused.
2 integer messages Indicates if messages sent over the data socket are enabled.
3 string mode Gives the current mode the system is running in.
4 integer clockMode The current clock mode either 0 or 1.
5 integer sampleFrequency The current clock frequency.
6 integer averageNummber Gives the number of FFT results that are averaged within the FPGA FFT process.
7 integer number Gives the total number of averaged FFT result vectors that are post processed.
8 integer fileAverageNumber Gives the number of averaged FFT result vectors that are averaged in the software post processing stage before storing in the data file.
9 integer socketAverageNumber Gives the number of averaged FFT result vectors that are averaged in the software post processing stage before sending over the data socket.
10 string title Gives the title of the run.
11 string project Gives the current project name. Data files will be stored in a directory of this name.
12 string fileBaseName Gives the base file name.
13 string fileName Gives the file name the data and information is stored into. This will get set after the run has been started and is based upon the fileBaseName parameter.
14 string fileFormat Describes the data file format. Either binary or ascii.
15 string socketFormat Describes the data socket format. Either binary or ascii.
16 integer info Gives the state of the user settable info parameter.

Data Socket Format

The data socket interface is provided so that a controlling program can get status messages and samples of the output data sent to it for controlling purposes. It is also possible to bypass the local data storage mode of the Astrofft system and obtain all of the output data through this interface if required. The socket returns both data and messages from the output of the processing system in binary or ASCII form.
If the socket interface format has been set to ASCII, the format of the returned data will consist of lines of comma separated floating point values as follows:

<channel>,<subchan>,<error>,<info>,<clips>,<amplitude>,<status>,<posType>,<pos1>,<pos2>,0,0,<data0>,<data1>,<data2>, ....

If the socket data format is binary then the output consists of 64byte header, followed by an array of double precision (8 byte) values. The data packet format is defined in the 'C' header file AstroPacket.h.
The header contains the following information in little endian format.
 
 


 

Field Type Usage
length 32bit unsigned integer Total Number of bytes in the message including header
channel 32bit unsigned integer The channel the data is from
subchan 32bit unsigned integer The sub channel the data is from. This is normally set to 1. In analogue mode it is set to 1 or 2 to indicate the sampler channel that the data is from.
error 32bit unsigned integer The bit mask of error bits. These are only set if the Astrofft process is not configure to retry on error.
info 32bit unsigned integer User settable information field
clips 32bit unsigned integer The number of input samples that were clipped by the ATD conversion process. This is only currently supported in analogue mode.
status 32bit unsigned integer Status information. Used with message channel to indicate message type.
time_sec 32bit unsigned integer Time in seconds when data was sampled from Unix EPOC.
time_usec 32bit unsigned integer Time in micro seconds when data was sampled.
posType
32bit unsigned integer The type of position information carried in the next two fields. 0 means no position information any other value is user defined.
pos1
32bit floating point A 32bit floating point value defining the telescopes pointing position. The actual meaning of this value is defined by the posType field.
pos2
32bit floating point A 32bit floating point value defining the telescopes pointing position. The actual meaning of this value is defined by the posType field.
dummy[2] 32bit unsigned integer 2 x 32 bit integers, reserved for future use (pads header to an 64 byte boundary)
amplitude 64bit floating point This value indicates the RMS amplitude of the input signal. This is only currently supported in analogue mode.

The channel value in both cases indicates the type of data within the message. The channels of data output on this socket include:
 
 
Type Value Data Following
0 Message channel, sends message strings such as "Run Complete"
1 Channel 1's Data array in binary or ASCII formats
2 Channel 2's Data array in binary or ASCII formats


 

Error Bit Usage
0x01 Fpga General Fault
0x02 Fpga Data Collection buffer overflow
0x04 Fpga Data Collection timeout

 
Status Number Usage
0 No status info
1 Run complete

Messages are only sent if the setMessages flag is set to 1.

Data File Format

On the start of each run three files are created in a data storage directory on the AstroFFT system. If the project field is not null a directory of the projects name is used for all of data files, otherwise the files are stored in the root of the data storage directory. One of these files stores information on the processing run while the others store the actual data for each channel.
The base name of these files will be as given by the setFileName command. Each files name will have an underscore followed by a 4 digit numeric extension appended. This number is obtained from the file .data which is stored in the data directory. The number in the .data file is incremented on each run.
I addition to the numeric extension the additional extension _1.dat _2.dat or .inf appended for the data and information files respectively.
The information file is in an ASCII text format with a format consisting of lines of text. Each line has a parameter name with terminating colon, followed by the settings for that parameter. The following is an example information file:
Title:         None
Project:       proj1
FileName:      data_0001
FileFormat:    binary
Mode:          qfft
ClockMode:     0
ClockFrequency:0
Number:        10
AverageNumber: 610
FileAverageNumber: 1
DateStarted:   0    2002-04-24T09:44:11.342Z
DateStopped:   10   2002-04-24T09:44:12.438Z
The data files can be in either binary or ASCII formats. The default and recommended format is binary. The data file format consists of an array of records.
Each record has the same format as described in the Data Socket section in either binary or ASCII.
The time format is in ISO 8601 form and is in Universal time.

Errors

A number of errors can occur during a processing run. The Astrofft process will report these to the client application, in the form of a message, and to the system logger where they will be appended to the file /var/log/messages. By default, errors that can be recovered from by loseing some data, are handled by retrying the last FPGA data processing cycle.
 

Error Handling Description
FPGA Data Over run Retry last FPGA average cycle  If the host system cannot keep up with the rate of data coming from the FPGA, a data over run error will occur. In this case the Astrofft process will ignore the last set of data from the FPGA and restart the FPGA. No errors will be reported to the client application but a warning will be issued to the system logger.
FPGA Integrator Over range Retry last FPGA average cycle  If the incoming data stream causes an over range in the FPGA's integrator an integrator over range error will occur. In this case the Astrofft process will ignore the last set of data from the FPGA and restart the FPGA. No errors will be reported to the client application but a warning will be issued to the system logger.
FPGa Data Timeout Retry last FPGA average cycle  If the software timesout awaiting the next data set from the FPGA a timeout will occur.
In this case the Astrofft process will ignore the last set of data from the FPGA and restart the FPGA. No errors will be reported to the client application but a warning will be issued to the system logger.
FPGA board fail Operate system without this board If one of the FPGA boards is not present or fails, the system will continue to function with just the single board. In this case a warning will be sent to the client application and the system logger and no data will be captured for the failed channel.
Disk write error Aborts current processing run with an error If there is a failier when writing to the data files , the Astrofft process will abort the run with an error message to the Client application and system logger. Note that the run will be aborted at the start command if there is insufficient disk space available at that time.

Example Client Program

There is an example client program in the directory /usr/astro/astroTest. This program, written in 'C', demonstrates how to connect to and control the astrofft process.