Introduction
This guide shows how to set up a Multi-Gigabit Transceiver (MGT) on an AMD (formerly Xilinx) Ultrascale FPGA, and how to connect it to a Xillyp2p IP Core. In this example, a GTH is configured; however almost all details remain the same for a GTY as well as later MGTs on later FPGAs (Ultrascale+ and others).
This is an implementation of the simplified application diagram shown on the guide about the IP core's ports and API:
The interface between the GTH and Xillyp2p is very simple, as shown in detail below. Nevertheless, it can be quite challenging to configure the GTH correctly and determine how to connect its ports. A complete example of a working design, including screenshots, Verilog code and constraints, is given on this guide.
This example targets the KCU105 board, which is the most popular development board for Kintex Ultrascale. The design presented here uses the board's SFP0 interface to connect with another board with the help of an optical transceiver module and an optical fiber, at a data rate of 5 Gb/s (connected as shown in the image above).
This design can be modified for other purposes and interfaces. As it uses the simplest possible GTH setup (without encoding), it's relatively easy to diagnose connectivity issues, if any arise. In addition, the Xillyp2p IP core has diagnostic output signals that can be connected to LEDs, as shown below. These can be helpful, particularly since one of the LEDs blinks for every bit error on the physical link.
For those interested in 7-series FPGAs (Kintex-7, Artix-7, Virtex-7 and Zynq-7000), there's a separate guide showing how to set up a Kintex-7 GTX, instead of the GTH used in this example.
Setting up the GTH
The procedure is illustrated with screenshots brief comments. This example was created using Vivado 2025.1, but it should work with other versions as well, possibly with only minor differences.
It's also possible to download a ready-made XCI file which reflects the configuration shown here.
In Vivado, open the IP Catalog and select UltraScale FPGAs Transceiver Wizard. Adjust the configuration according to the screenshots below. Note that the transceiver can be configured in more detail by accessing sub-menus within the Wizard; however, this is not necessary: Everything that needs to be changed is visible in the screenshots below. The Wizard's default configuration for "Start from scratch" is otherwise suitable.
The Wizard's initial window should be configured as follows:
Things to pay attention to:
- The IP's name is set to "fibergt". This is the name used in the instantiation in the Verilog example below.
- The Line Rate is set to 5 Gb/s in both directions. You may to use a different rate, depending on your equipment.
- The PLL type is chosen as CPLL for simplicity. This is a per-transceiver PLL, as opposed to Quad PLLs, which are shared among four transceivers, and offer more features but are more complex to configure.
- The "Actual Reference Clock" is the frequency of the external reference clock. On the KCU105 board, a low-jitter 156.25 MHz clock is supplied at pins P6/P5 by default, so this clock is used.
- "Encoding" remains at the default setting, Raw (no encoding), because Xillyp2p expects the transceiver to behave like a plain SERDES, without any manipulation of the data stream.
- The User Data Width and Internal Data Width are already set correctly by default for both transmitter and receiver: 32 bits. The User Data Width should match with the parallel word width specified for the Xillyp2p IP core, which it already does.
- On the Receiver side, the Termination is set to AVTT. This sets the bias voltage applied at the GTH's electrical receiver inputs. If you are not using an optical fiber link with an SFP+ module, a different setting for link coupling, termination and possibly programmable termination voltage may be preferable. These settings can dramatically affect link quality. Even if data arrives with many errors on every parallel word, adjusting these settings can sometimes make the link completely error-free.
- The ppm frequency offset is set to 200 ppm, accounting for a 100 ppm clock oscillator tolerance on each side. This is likely an exaggerated figure.
Next, we move on to the "Physical Resources" tab:
The only change required here is to set the frequency of the free-running clock to 125 MHz. This extra clock is supplied to the transceiver for its housekeeping tasks. In the example design below, one of the KCU105 board's clocks is used for this purpose, and its frequency is 125 MHz.
The bottom half of this window allows choosing which of the FPGA's GTH resources are used. These settings are overridden by placements constraints in the project's XDC file, so this section has no effect on the final result. It's also possible to delete the constraints in the XDC file and rely on these settings. Which approach to use is a matter of preference; in this example, the Wizard's settings are ignored.
Next, we move to the "Optional Features" tab:
This tab is shown here only to illustrate that there is nothing to change. In particular, receiver comma detection and alignment, as well as receiver clock corrections are features not used, as these require encoding to be enabled, which is not the case. Xillyp2p implements its own solution for synchronization, alignment and clock correction internally.
Next, we move to the last tab, "Structural Options":
Here, "Include simple receiver / transmitter user clocking network in the" are changed to Core, rather than Example design. This way, a few BUFGs are added to the core, instead of instantiating them in the design. This is minor difference.
The bottom half of this window contains numerous options for adding ports to the transceiver's interface. None of these are necessary for this example, so the defaults settings are sufficient.
That concludes the setup of the GTH. Next, a sample Verilog design is presented.
Instantiation of transceiver
The example top-level Verilog module starts as follows:
module example
(
input gth_refclk_n,
input gth_refclk_p,
input gth_rxn,
input gth_rxp,
output gth_txn,
output gth_txp,
input init_clk_p,
input init_clk_n,
output sfp_tx_enable,
output [7:0] gpio_led
);
The first six ports belong to the GTH itself: The reference clock, the reception and transmission pins.
The init_clk_p/n ports are differential inputs for the KCU105's free-running clock, which has a frequency of 125 MHz. sfp_tx_enable is explained briefly below.
Finally, there are eight LED outputs.
Before the instantiation of the GTH itself, a couple of clock buffers are added to the design:
wire gt_refclk;
wire init_clk, init_clk_i;
IBUFDS_GTE3 #(
.REFCLK_EN_TX_PATH(1'b0),
.REFCLK_HROW_CK_SEL(2'd0),
.REFCLK_ICNTL_RX(2'd0)
) ibufds_gt (
.I(gth_refclk_p),
.IB(gth_refclk_n),
.CEB(1'b0),
.O(gt_refclk),
.ODIV2()
);
IBUFDS ibufds (.O(init_clk_i), .I(init_clk_p), .IB(init_clk_n));
BUFG bufg (.O(init_clk), .I(init_clk_i));
And these wires are required for interfacing with the Xillyp2p IP core:
wire rx_clk;
wire tx_clk;
wire async_reset;
wire [31:0] in_data;
wire [31:0] out_data;
assign async_reset = 0;
async_reset is tied low in this example. This is somewhat careless, but the design works nonetheless. In a real-life design, it's recommended to use this asynchronous reset to reset the logic immediately after powerup.
Next, a small but possibly important detail:
assign sfp_tx_enable = 1; // Enable SFP+ transmitter
The sfp_tx_enable output port is connected to the optical transceiver's TX_DISABLE input through a MOSFET transistor, which negates its polarity. By default, there's a jumper installed on the board that keeps the transceiver's transmitter enabled. However, if this jumper is missing, the transceiver's transmitter will be turned off. By holding this output high, TX_DISABLE is forced low, ensuring that the optical transceiver operates normally.
Now to the instantiation of the transceiver:
fibergt fibergt_ins (
.gtwiz_userclk_tx_reset_in(async_reset),
.gtwiz_userclk_tx_srcclk_out(),
.gtwiz_userclk_tx_usrclk_out(),
.gtwiz_userclk_tx_usrclk2_out(tx_clk),
.gtwiz_userclk_tx_active_out(),
.gtwiz_userclk_rx_reset_in(async_reset),
.gtwiz_userclk_rx_srcclk_out(),
.gtwiz_userclk_rx_usrclk_out(),
.gtwiz_userclk_rx_usrclk2_out(rx_clk),
.gtwiz_userclk_rx_active_out(),
.gtwiz_reset_clk_freerun_in(init_clk),
.gtwiz_reset_all_in(async_reset),
.gtwiz_reset_tx_pll_and_datapath_in(1'b0),
.gtwiz_reset_tx_datapath_in(1'b0),
.gtwiz_reset_rx_pll_and_datapath_in(1'b0),
.gtwiz_reset_rx_datapath_in(1'b0),
.gtwiz_reset_rx_cdr_stable_out(),
.gtwiz_reset_tx_done_out(),
.gtwiz_reset_rx_done_out(),
.gtwiz_userdata_tx_in(out_data),
.gtwiz_userdata_rx_out(in_data),
.drpclk_in(init_clk),
.gthrxn_in(gth_rxn),
.gthrxp_in(gth_rxp),
.gtrefclk0_in(gt_refclk),
.gthtxn_out(gth_txn),
.gthtxp_out(gth_txp),
.gtpowergood_out(),
.rxpmaresetdone_out(),
.txpmaresetdone_out()
);
The transceiver is directly connected to the physical pins gth_rxp, gth_rxn, gth_txp and gth_txn. The reference clock, gt_refclk, is the output of the IBUFDS_GTE3 clock buffer.
The free-running clock init_clk is connected to gtwiz_reset_clk_freerun_in and drpclk_in. As mentioned above, this clock is used for the transceiver's control logic.
Note that tx_clk and rx_clk are outputs from the transceiver. In other words, the transceiver supplies these two clocks through the gtwiz_userclk_tx_usrclk2_out and gtwiz_userclk_rx_usrclk2_out ports. Together with in_data and out_data, these are the only connections between the transceiver and the Xillyp2p IP core, as shown next.
Instantiation of the Xillyp2p IP core
For this example, core A of an IP core that was generated at the IP core Factory is applied. Core B should be used on the other side of the physical link (unless the core is symmetric, a topic explained at the end of the guide about the IP Core Factory).
The configuration of the IP core is as shown in this screenshot from the web application:
The general parameters of the IP core match those of the GTH: A 32-bit parallel word and a clock frequency of 156.25 MHz. Note that this is the frequency of rx_clk and tx_clk, which happens to be the same as the reference clock. If, for example, the GTH's Line Rate is changed (without changing the reference clock and the width of the parallel word), the frequency of rx_clk and tx_clk will change accordingly. In that case, the updated frequency should be specified at the IP Core Factory.
Three streams were defined for this IP core: demo32_to_b, demo32_to_a and demo_256. A different number of streams, with different names and attributes can be configured as required for the application.
Before instantiating the IP core, a few additional wires are required. The following code segment is copied directly from the IP core's instantiation template:
wire status_link_down;
wire status_initializing;
wire status_link_partner_mismatch;
wire status_bit_error;
wire status_rev_polarity;
wire [31:0] status_debug;
wire [2:0] error_test_rate;
// Wires related to data stream "demo32_to_a"
wire user_rx_demo32_to_a_wr_en;
wire [31:0] user_rx_demo32_to_a_wr_data;
wire user_rx_demo32_to_a_full;
wire user_rx_demo32_to_a_eop;
// Wires related to data stream "demo32_to_b"
wire user_tx_demo32_to_b_rd_en;
wire [31:0] user_tx_demo32_to_b_rd_data;
wire user_tx_demo32_to_b_empty;
wire user_tx_demo32_to_b_eop;
// Wires related to data stream "demo_256"
wire user_tx_demo_256_rd_en;
wire [255:0] user_tx_demo_256_rd_data;
wire user_tx_demo_256_empty;
wire user_tx_demo_256_eop;
And now to the instantiation itself. This part too is copied from the instantiation template for core A:
xillyp2p_core_a xillyp2p_core_a_ins (
// Ports related to data stream "demo32_to_a"
// Inbound data stream:
.user_rx_demo32_to_a_wr_en(user_rx_demo32_to_a_wr_en),
.user_rx_demo32_to_a_wr_data(user_rx_demo32_to_a_wr_data),
.user_rx_demo32_to_a_full(user_rx_demo32_to_a_full),
.user_rx_demo32_to_a_eop(user_rx_demo32_to_a_eop),
// Ports related to data stream "demo32_to_b"
// Outbound data stream:
.user_tx_demo32_to_b_rd_en(user_tx_demo32_to_b_rd_en),
.user_tx_demo32_to_b_rd_data(user_tx_demo32_to_b_rd_data),
.user_tx_demo32_to_b_empty(user_tx_demo32_to_b_empty),
.user_tx_demo32_to_b_eop(user_tx_demo32_to_b_eop),
// Ports related to data stream "demo_256"
// Outbound data stream:
.user_tx_demo_256_rd_en(user_tx_demo_256_rd_en),
.user_tx_demo_256_rd_data(user_tx_demo_256_rd_data),
.user_tx_demo_256_empty(user_tx_demo_256_empty),
.user_tx_demo_256_eop(user_tx_demo_256_eop),
// General signals
.rx_clk(rx_clk),
.tx_clk(tx_clk),
.async_reset(async_reset),
.in_data(in_data),
.out_data(out_data),
.status_link_down(status_link_down),
.status_initializing(status_initializing),
.status_link_partner_mismatch(status_link_partner_mismatch),
.status_bit_error(status_bit_error),
.status_rev_polarity(status_rev_polarity),
.status_debug(status_debug),
.error_test_rate(error_test_rate)
);
// error_test_rate should always be zero unless you want to the test
// what happens when there are errors on the physical data link.
assign error_test_rate = 3'd0;
As mentioned before, the IP core connects with the GTH using four wires: rx_clk, tx_clk, in_data and out_data. The remaining connections are related to the application logic, as discussed next.
Exchanging data with the application logic
Those who have used Xillybus' IP core for PCIe or XillyUSB will find this part very familiar.
First, recall from above that this example revolves around core A, and that core B should (in general) be used on the FPGA on the opposite side of the physical link.
The application data stream named "demo32_to_a" allows the FPGA on the other side (with core B) to send data to the FPGA with core A.
The mainstream method to access this data is through a FIFO. For example, fifo_32 is a standard dual-clock FIFO with a 32 bit wide data word, defined in the Vivado project, and instantiated as follows:
fifo_32 data_in_fifo
(
.rst(async_reset),
.wr_clk(tx_clk),
.rd_clk(<connect to application logic>),
.din(user_rx_demo32_to_a_wr_data),
.wr_en(user_rx_demo32_to_a_wr_en),
.full(user_rx_demo32_to_a_full),
.rd_en(<connect to application logic>),
.dout(<connect to application logic>),
.empty(<connect to application logic>)
);
The three wires named user_rx_demo32_to_a_* are connected to the Xillyp2p IP core. The IP core uses these signals to fill the FIFO with data arriving from core B.
Note that the FIFO's "full" port is also connected. The Xillyp2p IP core respects this signal and avoid causing an overflow, by virtue of its flow control mechanism. This mechanism is optional but recommended (note that it says "Yes" on the "Flow control" column in the IP core's configuration screenshot for all three streams).
The FIFO's remaining ports are connected to the application logic, which retreives the data as needed. The application logic isn't required to fetch data at a certain rate; the only requirement is to avoid attempting to read from the FIFO when it's empty.
Also note that the FIFO's wr_clk port is connected to tx_clk, even though it receives data. All interaction with the application logic is based upon tx_clk. Don't be confused by the "rx" prefix in the other signals' names – rx_clk is used only in connection with the transceiver, and only with in_data.
As for data flowing in the opposite direction, "demo32_to_b" allows core A to send data to the FPGA on the other side (with core B). The instantiation of the FIFO used for this purpose is as follows:
fifo_32 data_out_fifo
(
.rst(async_reset),
.wr_clk(<connect to application logic>),
.rd_clk(tx_clk),
.din(<connect to application logic>),
.wr_en(<connect to application logic>),
.full(<connect to application logic>),
.rd_en(user_tx_demo32_to_b_rd_en),
.dout(user_tx_demo32_to_b_rd_data),
.empty(user_tx_demo32_to_b_empty)
);
assign user_tx_demo32_to_b_eop = 0;
The three wires named user_tx_demo32_to_b_* are connected to the Xillyp2p IP core. The IP core uses these signals to fetch data from the FIFO as soon as it becomes non-empty.
The FIFO's other ports are connected to the application logic, which fills the FIFO in the usual manner. There is no requirement on how much data the application logic writes to the FIFO, and no special action is required in order to make the data reach the other side. The only obvious requirement is to avoid writing to the FIFO when it's already full.
Since flow control is enabled for this stream, no data is fetched from the FIFO if the FIFO on the opposite side is full. As a result, the application logic on the core B side effectively controls the data flow from end to end: If the application logic on the opposite side (with core B) reads data from its FIFO more slowly than data is written to the FIFO on this side (core A), the FIFO on this side will eventually become full, forcing the application logic to stop writing. In other words, this arrangement is equivalent to a single FIFO whose write port is on one FPGA and whose read port is on the other.
Also note that user_rx_demo32_to_a_eop is unused in this design, and that user_tx_demo32_to_b_eop is tied low. These two ports are related to the ability to send an end-of-packet (EOP) alongside a data word. This feature is useful when sending data segments or packets through the stream. More about EOP can be found on the guide about Xillyp2p's ports and API.
A few comments about data exchange
For testing purposes, it may be useful to loop back the data arriving from core B back to it. In other words, to route the data received from "demo32_to_a" directly into "demo32_to_b". This can be achieved by an instantiation of one FIFO instead of two, as follows:
fifo_32 loopbackfifo
(
.rst(async_reset),
.wr_clk(tx_clk),
.rd_clk(tx_clk),
.din(user_rx_demo32_to_a_wr_data),
.wr_en(user_rx_demo32_to_a_wr_en),
.full(user_rx_demo32_to_a_full),
.rd_en(user_tx_demo32_to_b_rd_en),
.dout(user_tx_demo32_to_b_rd_data),
.empty(user_tx_demo32_to_b_empty)
);
assign user_tx_demo32_to_b_eop = 0;
Once again, note that tx_clk is used on both sides of the FIFO.
The interface with "demo_256" is handled in the same way, using a FIFO with a 256-bit–wide data word. Note that this stream uses a 256-bit-wide data word, which is by much wider than the physical link's 32-bit parallel word. Also, the maximum theoretical data rate for this stream is 256 * 156.25 MHz / 8 = 5000 MB/s, which is far above the physical link's capacity of about 606 MB/s.
Nevertheless, if the application's natural data format consists of 256-bits words, it's appropriate to use a stream with a 256-bit width, even though it provides no bandwidth advantage. This choice ensures that each word is delivered in its correct format and with proper alignment on the receiving side.
LEDs and diagnostic signals
The KCU105 board has 8 GPIO LEDs. The Xillyp2p IP core has a few outputs that can be connected to LEDs for the purpose of signaling the status of the link, as detailed on this guide.
A recommended setup is as follows:
reg [26:0] txclk_cnt, initclk_cnt;
always @(posedge init_clk)
initclk_cnt <= initclk_cnt + 1;
always @(posedge tx_clk)
txclk_cnt <= txclk_cnt + 1;
assign gpio_led[1:0] = { txclk_cnt[26], initclk_cnt[26] };
ledhelper ledhelper_ins[5:0]
(
.clk(tx_clk),
.in( {
status_link_down,
status_link_partner_mismatch,
status_initializing,
status_bit_error,
status_debug[2],
status_debug[0] } ),
.led(gpio_led[7:2])
);
For those less familiar with Verilog, the instantiation of ledhelper is duplicated six times, once (and separately) for each bit of the "in" input and its corresponding "led" output.
As apparent from above, GPIO LEDs 0 and 1 are plain heartbeat signals, which blink to indicate that init_clk and tx_clk are active.
The remaining six LEDs display the state of six status signals from the Xillyp2p IP core. The ledhelper module ensures that the LEDs remain lit or turned off long enough for the human eye to perceive events as brief as a single clock cycle. For example, status_bit_error is high during one clock cycle if an error is detected on the physical link. Without ledhelper, such short events would be invisible.
The ledhelper module is defined as follows:
module ledhelper(
input clk,
input in,
output reg led
);
// 22 bits = 4194304 counts, ~26.8 ms at 156.25 MHz clock
reg [21:0] count;
always @(posedge clk)
if (count != 0)
count <= count - 1;
else if (in != led)
begin
led <= in;
count <= ~0;
end
endmodule
This module freezes the "led" output during 222 clock cycles after each change. Hence if the input is high during one clock cycle, the LED remains lit for approximately 26.8 ms, which is long enough for the human eye to preceive. Additionally, if the input changes rapidly, the LED blinks vigorously, which the human eye correctly interprets as high activity.
This approach is suitable when 8 LEDs are available. If fewer LEDs are present, status_bit_error is usually the most useful indicator, followed by status_link_partner_mismatch, status_link_down and status_debug[2]. At least one heartbeat LED is also recommended.
XDC constraints
For completeness, the constraints in the XDC file are shown and explained. Recall that the board used is the KCU105.
First, the constraints related directly to the transceiver:
create_clock -name refclk -period 6.4 [get_ports gth_refclk_n]
set_property PACKAGE_PIN P6 [get_ports gth_refclk_p];
set_property PACKAGE_PIN P5 [get_ports gth_refclk_n];
set_property LOC GTHE3_CHANNEL_X0Y10 \
[get_cells -hier -filter { ref_name =~ GT*_CHANNEL }]
The reference clock period is set to 6.4 ns, corresponding to 156.25 MHz, consistent with the earlier discussion. Note that it is a coincidence that rx_clk and tx_clk also have the same frequency.
Then the reference clock's pins are placed explicitly, overriding the definitions made in the Wizard.
After this, the GTH is explicitly selected for the transceiver, once again overriding the selection made in the Wizard. Note that the expression that finds the GTH matches any GTH in the design; this works because there's only one GTH in the entire design. Hence if multiple GTHs are present, a more specific constraint would be required, however such constraint may not work consistently across different versions of Vivado.
Next, the constraints for the free-running clock:
create_clock -name initclk -period 8 [get_ports init_clk_n]
set_property -dict { PACKAGE_PIN F10 IOSTANDARD LVDS } \
[get_ports "init_clk_n"];
set_property -dict { PACKAGE_PIN G10 IOSTANDARD LVDS } \
[get_ports "init_clk_p"];
The free-running clock is set to 8 ns, i.e. 125 MHz. Recall that if this clock frequency is changed, the setting must also be updated in the Transceiver Wizard.
Once both clocks are defined and constrained, it's required to inform Vivado that the logic design considers init_clk, rx_clk and tx_clk as unrelated, so that all paths across these clocks' domains are false path:
set_clock_groups -asynchronous \
-group [get_clocks -include_generated_clocks -of_objects \
[get_ports init_clk_n]] \
-group [get_clocks -of_objects [ get_nets rx_clk ] ] \
-group [get_clocks -of_objects [ get_nets tx_clk ] ]
Without this, the design may fail to achieve the timing constraints.
And finally, the GPIO LEDs and sfp_tx_enable (the trivial part):
set_property PACKAGE_PIN AP8 [get_ports "gpio_led[0]"];
set_property PACKAGE_PIN H23 [get_ports "gpio_led[1]"];
set_property PACKAGE_PIN P20 [get_ports "gpio_led[2]"];
set_property PACKAGE_PIN P21 [get_ports "gpio_led[3]"];
set_property PACKAGE_PIN N22 [get_ports "gpio_led[4]"];
set_property PACKAGE_PIN M22 [get_ports "gpio_led[5]"];
set_property PACKAGE_PIN R23 [get_ports "gpio_led[6]"];
set_property PACKAGE_PIN P23 [get_ports "gpio_led[7]"];
set_property IOSTANDARD LVCMOS18 [get_ports "gpio_led[*]"]
set_false_path -to [get_ports "gpio_led[*]"]
set_property -dict {PACKAGE_PIN AL8 IOSTANDARD LVCMOS18} \
[get_ports sfp_tx_enable];
Conclusion
This guide has presented a complete example of setting up a Multi-Gigabit Transceiver (GTH) on an AMD (formerly Xilinx) UltraScale FPGA and connecting it to a Xillyp2p IP core. Starting from the Transceiver Wizard configuration in Vivado, through instantiating clocks, FIFOs, and application logic, to wiring diagnostic LEDs and defining XDC constraints, each step has been explained in detail.
This example serves both as a practical reference and as a learning tool for working with UltraScale GTH transceivers and the Xillyp2p IP core.
Recall from above that an example of setting up a transceiver for 7-series FPGAs (Kintex-7, Artix-7, Virtex-7 and Zynq-7000) is shown on a different guide.