ADM-XRC Gen 3 SDK

The Gen 3 SDK is for use on the latest Alpha Data FPGA products.

The ADB3 Bridge decouples the complexity of the PCI Express interface from the user application. The user interface is based on the Open Core Protocol (OCP) standard as defined by the OCP International Partners Organisation see http://www.ocpip.org for more details.

The ADB3 Bridge typically resides in a separate FPGA, which keeps the system usage of the user FPGA to a minimum. Only the user end of the Bridge to Target interface (MPTL) is required to be included by the user.

The ADB3 Bridge Solution provides:

• PCI Express Interface up to Gen2 x8
• Simple, high speed user interface based on OCP, designed to match the available bandwidth of the PCI Express interface
• Direct Memory Access (DMA) to user application through dedicated BARs in the PCI Express Interface
• Up to 4 High Speed DMA interfaces - all capable of host to user or user to host transfers
• Full Microsoft Windows and Linux drivers provided to make system integration easier
• ADB3 Driver and ADMXRC3 SDK provides example applications as starter designs for the users development and a library of software routines for accessing the bridge functions.

Software Overview

There are two distinct packages in the software release.

• ADB3 Device Driver - provides the software interface to the FPGA board
• ADMXRCG3 SDK - Provides the routines to initiate DMA, interrogate the hardware and allow the board to be programmed and configured.

ADB3 Device Driver

TBW

ADMXRCG3 SDK

TBW

Hardware Overview

The latest Alpha Data FPGA platform uses a PCI Express Bridge called 'ADB3' and implements a High Speed Serial Local bus (MPTL) to provide the user application with high speed data transfers to and from the Host system. Using the ADB3 device drivers and the ADM-XRC Gen 3 SDK as the interface to the user software application.

This has been accompanied by a new SDK which has been designed from scratch to utilise the higher data rates more efficiently.

The above diagram shows the main blocks in the system ( based on the ADM-XRC-6T1 FPGA board):

• The Host - PCI Express System. Either a PC or SBC system which the board has been plugged into.
• ADB3 Bridge - The Alpha Data propriatory PCI Express bridge, incorporating DMA engines and board level configuration logic. Also provides programming logic for the user application.
• The MPTL - Local Bus interface. Provides the high bandwidth connection from the Host to the user application. The MPTL includes error detection and data retry functionality to ensure maximum reliability in the user data transfers.
• XRM Interface - using Alpha Data XRM modules allows the design to connect to a variety of network/datacomms protocols, add Camera Link connectivity, A2D and D2A circuitry as well as simple difital I/O.
• On board DDR3 memory providing high speed/high capacity off FPGA data storage.
• On board monitor of FPGA temperatures and power rail voltage levels.

Local Bus (MPTL)

Unlike the older ADM-XRC2 products the latest local bus is a full duplex High Speed Serial interface. This provides a much greater available bandwidth than the parallel local bus previously used. The user application interface is based on the Open Core Protocol (OCP) standard See the OCP-IP website for more details.

There is an OCP port for each DMA channel available and one OCP port for direct read/write access to the user application.

MPTL Operation

The MPTL Interface in the User FPGA provides an OCP Master Port for direct reads and writes from the Host via BARs 2/3 and 4/5 (64 bit BARs) and a Master Port for each DMA engine in the Bridge.

Each OCP Link operates as follows:

• The Master Port outputs a command along with the address, byte enables and burst length for the transaction.

• The Slave port responds by accepting the command (and the other information).

• For write transactions:

• The Master Port outputs the data to be written along with a data valid flag.

• The Slave Port accepts the data as and when it is able to. Responding with a data accept flag for each data transfer.

• Once all the data has been transferred the Master may start the next transaction.

For Read transactions:

• The Slave Port retrieves the data that has been requested.

• The Slave Port outputs the data as and when it is available along with a data valid flag.

• The Master Port accepts the data . Responding with a data accept flag for each data transfer.

• Once all the data has been transferred the Master Port is free to start the next transaction.

All OCP ports operate independently and with multiple DMA engines the user can instigate multiple data streams into and out of the application design.

For advanced systems where the user application has a requirement for direct access from the Application to the Host an MPTL interface can be provided that has an extra Slave Port. This allows the application to make memory access requests direct to the Host System.

Open Core Protocol (OCP) Interface

The figure below shows the basic connectivity of an OCP port.

Read requests in OCP are split, the request does not hold the link until the required data has been provided. The link can carry on transferring requests. The only limit to this is how many read requests the master puts out at a time and how many requests the slave can handle at once. The slave can be designed to handle only one request at a time but this will reduce the data throughput for that port.

The figure below shows a simple write (BurstLength=1) followed by a simple read (BurstLength=2).

ADB3 OCP Interface signal list

Signal	Direction	Description
Cmd(2:0)	M2S	Defines whether the OCP command is a read or write or idle
Address(63:0)	M2S	Address for transfer
BurstLength(7:0)	M2S	Number of transfers for command (each transfer is 128bits)
Data(127:0)	M2S	Data for write commands
DataByteEn(15:0)	M2S	Byte Enables for write data
Tag(7:0)	M2S	Tag given to read so response can be decoded correctly
CmdAccept	S2M	Strobed when the slave port accepts the command
DataAccept	S2M	Strobed when the slave port accepts the data
Resp(1:0)	S2M	Indicates status of the data back to the master
Data(127:0)	S2M	Data returned for read commands
RespAccept	M2S	Response data accepted
Tag(7:0)	S2M	Tag value associated with the initiated read command

The following data flow diagram shows the OCP link when it can have up to 3 read requests in flight. Notice the delay from sending the 3rd request until the 1st requests data has been returned before the 4th request is sent.

User Application

The user application starts at the OCP interfaces of the MPTL block. These can either go to custom logic directly or through on chip or off chip memories. Sample memory interfaces are provided including blocks to provide multiport access.

For large data transfers it is simplest to set up a dual port memory and use DMA to write data to and read data from the memory. Using the second port as the application interface.

Advanced Applications

The ADB3 PCI Express Bridge design has been developed to be independent of the FPGA into which it is programmed. As a result of that it is possible to merge the bridge design into the target application.

As the MPTL has the same interface on each side the target application just connects to the bridge as it would the MPTL interface, no changes required to the target application.

Due to the structure of the system there would be some small functional compromises if this were to be carried out. Contact Alpha Data Support for more information.

Important Design Notes

PCI Express implements a timeout function on ALL read requests. If the destination device does not respond within the allocated timeframe then the BIOS terminates the transfer itself. This termination is system dependent and, in some cases, can crash the system.

To stop this occurring the ADB3 system implements its own timeout on read requests passed to the target via Direct Slave reads on BARs 2/3 and 4/5. The bridge shall terminate and respond to any read requests pending and disable any further accesses to the target until the software has acknowledged the situation and cleared the BARs error flag to continue accessing the target.

The user application designer must take this timelimit into account when designing their system. If the application takes too long to supply the required data then the system will not operate efficiently (if at all) and the user software will have to continuously monitor the BAR status register to check that the timeout has not triggered.

Due to the system timeout starting when the read request is first transmitted by the host the time available in the user application is significantly less (to take into account all intermediate stages entered while the request is transferred to the target and subsequent stages while the data is returned to the host). As these times are system dependent the ADB3's timeout is set to reduce the risk of it still causing the system timeouts to trigger.

As all Direct Slave requests, reads and writes, are handled by a single data stream then it is also important to handle all the writes as quickly as possible, if the read request is preceded by a number of writes then it will be delayed while these writes are processed.

The timeout becomes more of an issue when the user application runs at a slower clock rate (than the OCP Link) and cross clock domain logic has to be used, this introduces further delays in the transfer and reduces the margin for setting off the timeout.

Main influences on response time:

• System delays in transferring request to ADB3 - unknown as system dependent
• ADB3 processing time - deterministic but also dependent on design loading (i.e. transfer time increases as the number of requests to the application increases)
• Time taken for user application to initiate read request internally
• user application read response time
• ADB3 processing time - again deterministic but also dependent on other traffic being generated by the application
• System delays in transferring data back to host - again unknown as it is system dependent

MPTL Communication

The MPTL is the replacement for the Local Bus as used on previous Alpha Data systems. If the user application causes the MPTL to stop functioning then the ADB3 shall shut it down, return dummy data to all subsequent read requests, discard all write requests. This will continue till the user software intervenes and corrects to problem. This may require the reloading of the target application and reseting the MPTL error flags.