Waveguide-coupled parallel optical transceiver technology for Tb/s-class chip-to-chip data transmission

Fuad E. Doany*a, Clint L. Schowa, Jeffrey A. Kasha, Christian Baksa, Russell Budda, Daniel M. Kuchtaa, Petar Pepeljugoskia, Frank Libscha, Roger Dangelb, Folkert Horstb, Bert J. Offreinb

aIBM – T. J. Watson Research Center, 1101 Kitchawan Road, Yorktown Heights, 10598; bIBM Research GmbH, Zurich, Switzerland

ABSTRACT

The IBM Terabus program has developed parallel optical interconnects for terabit/sec-class chip-to-chip communications through printed circuit boards with integrated optical waveguides. 16 TX + 16 RX channel transceiver “Optomodules” were assembled and fully characterized, with fiber-coupled full links operating up to 15 Gb/s, for an aggregate bi-directional data transfer rate of 240 Gb/s. Furthermore, we have demonstrated a complete link between two Optomodules through polymer waveguides on a printed circuit board, with all 32 uni-directional links operating error-free at 10Gb/s, for a 160 Gb/s bidirectional aggregate data rate. This is the fastest, widest, and most integrated multimode optical bus ever demonstrated.

Keywords: Optical interconnects, parallel optical links, optical transceivers

1. INTRODUCTION

The steadily increasing demand for higher bandwidth interconnects in high performance computing systems is progressively more difficult to meet using electrical interconnect technology. Today, optical interconnects are commonly used for the longer (>10m) high speed links and are increasingly replacing electrical links at shorter distances. Furthermore, advanced high performance chips and multi-chip modules are increasingly limited by their off-chip or off-module bandwidth. [1] Board-level optical data buses offer the potential alleviate this I/O bottleneck by providing dense, parallel high speed data links.

In the Terabus program, we are developing parallel optical interconnects for terabit/sec-class chip-to-chip and module-to-module communications through printed circuit boards with integrated optical waveguides. [2,3] The data links are enabled by parallel transceiver modules, or “Optomodules,” that incorporate key technologies including: high-speed, low-power CMOS analog amplifier circuits; efficient, high-speed 985 nm substrate illuminated/emitting photodiode (PD) and VCSEL arrays with integrated collimating lenses; and the extensive use of flip-chip packaging to minimize the module dimensions and associated packaging parasitics. The core of the module is a novel “Optochip”: a 5.25 mm x 3.25 mm single-chip CMOS optical transceiver IC with flip-chip attached 4 x 4 VCSEL and PD arrays. The Optochip is then flip-chip attached to a high-density organic package to form the Optomodule. A conventional ball grid array (BGA) solder process is then used to mount this low-cost, low-profile package on a printed circuit board with integrated optical waveguides (“Optocard”). Polymer waveguides on a 62.5-µm pitch are incorporated in the FR4-based Optocard. Optical and electrical connections are simultaneously established during the Optomodule-to-Optocard attachment.

2. COMPONENTS AND ASSEMBLY

The Terabus transceiver optical link is schematically depicted in Figure 1. The link consists of two Optomodules interconnected through the waveguides on Optocard. The fundamental component of the Optomodule is the transceiver Optochip, a CMOS IC with flip-chip attached optoelectronic devices. The Optochip is in turn flip-chip attached to the organic SLC carrier, forming the complete Optomodule. A detailed description of the components is provided below.

*doany@us.ibm.com; phone 1 914 945-2831; fax 1 914 945-4219

Invited Paper

Photonics Packaging, Integration, and Interconnects VIII, edited by Alexei L. Glebov, Ray T. Chen Proc. of SPIE Vol. 6899, 68990V, (2008) · 0277-786X/08/$18 · doi: 10.1117/12.763670

Proc. of SPIE Vol. 6899 68990V-12008 SPIE Digital Library -- Subscriber Archive Copy

0• aaa.aaaaaa a a aa a a a a .a •eaAeas aa a •.•:.: •..:Jt.u.—I.

Figure 1. A schematic of the Terabus board-level optical link. Two Optomodules are connected through an array of polymer waveguides in the Optocard.

2.1 Optochip assembly

The primary component of the Optochip is the transceiver IC, which is fabricated in IBM 130-nm CMOS technology. [4] The transceiver IC consists of sixteen independent receiver amplifier (RX) circuits and sixteen laser-diode driver (LDD) circuits arranged in two separate 4x4 blocks in the center of the IC. The TX and RX blocks are separated by 500-µm to minimize RX-TX crosstalk and to provide clearance between the two OE components in the subsequent assembly. The RX output and the LDD inputs are routed to 100 µm bond pads on the periphery of the IC. The 200-µu pitch of the bond pads is compatible with current high-volume industrial flip-chip attachment such as the IBM C-4 process. The pad configuration also allows all channels to be characterized using electrical probes at the IC level. Power and control signals for the transmitter and receiver sections of the chip are connected to additional bond pads on the edges of the chip. The transceiver chip measures 5.25 mm by 3.25 mm.

The transceiver is designed to be used with 985-nm substrate-emitting OE arrays. The sixteen-channel VCSELs and PDs are on 4x4 arrays with a pitch of 250 µm x 350 µm, identical to the IC circuit pitch. The OE arrays are designed to be flip-chip attached to the transceiver IC using bond pads in the center of the IC. Eutectic AuSn solder deposited on the central pads of the IC is used for assembly of the Optochip. The melting temperature of eutectic AuSn is 278 °C during reflow and >400 °C after reflow. This enabled the sequential flip-chip attachment of the two OE arrays to the IC. A micrograph of the assembled Optochip is shown in Figure 2.

Figure 2. Micrograph of transceiver Optochip showing the flip-chip attached VCSEL and PD arrays on the CMOS IC.

The VCSEL and PD arrays also incorporate lenses into the back surface of the GaAs and InP substrates, as seen in Fig. 2. The integrated lenses are designed collimate the VCSEL output or to focus light onto the PD. The lens pitch is 250 µm x 350 µm, same as the OE device pitch, but also includes a 62.5-µm offset between rows of OE elements which facilitated coupling of the 2-dimensional OE arrays to the 1-dimensional array of waveguides on the Optocard.

4x4 VCSELArray

4x4 PDArray

4x4 VCSELArray

4x4 PDArray

4x4 VCSELArray

4x4 PDArray

Lens Array

Optocard

SLC

CMOS IC

OE’sSLC

CMOS IC

OE’s

CMOS IC

OE’sOE’s

Optocard

Lens ArrayLens ArrayLens Array

Optocard

SLC

CMOS IC

OE’sSLC

CMOS IC

OE’s

CMOS IC

OE’sOE’s

Lens ArrayLens ArrayLens Array

Optomodule

Lens Array

Optocard

Lens ArrayLens Array

Optocard

SLC

CMOS IC

OE’sSLC

CMOS IC

OE’s

CMOS IC

OE’sOE’sSLC

CMOS IC

OE’sSLC

CMOS IC

OE’s

CMOS IC

OE’sOE’sSLC

CMOS IC

OE’s

CMOS IC

OE’sOE’sSLC

CMOS IC

OE’sOE’s

CMOS IC

OE’sOE’s

Optocard

Lens ArrayLens ArrayLens ArrayLens ArrayLens ArrayLens ArrayLens Array

Optocard

SLC

CMOS IC

OE’sSLC

CMOS IC

OE’s

CMOS IC

OE’sOE’sSLC

CMOS IC

OE’s

CMOS IC

OE’sOE’sSLC

CMOS IC

OE’sOE’s

CMOS IC

OE’sOE’s

Lens ArrayLens ArrayLens ArrayLens ArrayLens ArrayLens ArrayLens Array

Optomodule

Proc. of SPIE Vol. 6899 68990V-2

,,PP •

P P • • . .1PP P ...P P p... .. pP P ••• PP P'PP P ?P PP P P P P

•t:.=.

,'p .p .p .. p. pt. ••p.p pp.p p'pp•., $ +'v.*

• •r$'t**+4

2.2 Optomodule

The Optochip is designed to be flip-chip attached to and organic high-density carrier using the IC peripheral bond pads to form the Optomodule. The organic SLCTM carrier provides high density bond pads on the top surface for attachment to the IC (100-µm pads on a 200-µm pitch) and standard BGA pads on the bottom surface (0.5-mm pads on 1.0-mm pitch). The overall thickness of the SLC is about 0.8 mm, with the high speed signal routing provided in the top build-up layers. Differential transmission lines connect the inputs (outputs) of the IC circuits to SLC probe pads that allows all channels to be characterized using high-speed electrical probes. The low speed bias and control signals are routed to BGA pads at the bottom of the module. BGA pads are incorporated on 3 sides of the module while a keep-out region is maintained from the cutout to the edge on the fourth side to accommodate the waveguides on the Optocard.

The SLC Optomodule incorporates a cutout in the center of the module fabricated in the in the position of the OE array. The cutout enables the optical coupling from the OE array to the waveguides on the Optocard. The Optochip bond pads are located adjacent to the cutout. Eutectic PbSn solder with a melting temperature of 183ºC is deposited to the bond pads on the SLC using a solder transfer mold and used to flip-chip attach the Optochip. The assembled Optomodule is depicted in Figure 3.

Figure 3. Micrograph of Optomodule. Top view shows the CMOS IC attached at the center. Bottom view shows the cutout in the center revealing the VCSEL and PD arrays.

The SLC module is 35 mm x 35 mm, similar to typical chip carrier modules. Although these dimensions are significantly larger than required for the transceiver, it demonstrates the potential for using this SLC carrier as a multi-chip module.

The low speed bias and control signals are simultaneously routed to peripheral wire bond pads on the top surface. This provides an alternate method of powering up the entire Optomodule, enabling full testability of the Optomodule prior to attachment to the Optocard.

2.3 Optocard

The Optomodule is designed to be directly surface mounted using a conventional BGA process to a low-cost FR4 circuit board with integrated optical waveguides, providing both electrical and optical interconnects. The FR4 board incorporates a dense array of polymer waveguides of 35-µm x 35-µm core dimensions with a pitch of 62.5µm. The 62.5um offset between rows in the VCSEL and photodiode arrays shown in Fig. 2 facilitates coupling from the 2-dimentional OE arrays to the 1-dimentional waveguide array.

The Optocard incorporates BGA sites for electrical attachment of Optomodules as well as the dense waveguide array. A micrograph of the Optocard is shown in Figure 4. Forty-eight polymer waveguides extending between 2 Optomodule sites were fabricated. A photolithographic process is used to fabricate the waveguide cores using a photo-sensitive acrylate polymer material.

Top Bottom

Cutout withOEs-on-IC

IC

B G A

Top Bottom

Cutout withOEs-on-IC

IC

B G A

Proc. of SPIE Vol. 6899 68990V-3

San

"U

Figure 4. Micrograph of Optocard with integrated polymer waveguides and BGA site for Optmodule. Also shown is a cross-section of the waveguide cores and the collimating lens array.

Total-internal-reflection (TIR) mirror arrays were also fabricated in the waveguides using a laser ablation process to provide the 90º-turning from the waveguides. A collimating lens array is aligned and attached to the mirror array to facilitate optical coupling from the Optocard to the Optomodule.

3. OPTOMODULE RESULTS

The assembled transmitter and receiver Optomodules have been characterized using fiber probing for the optical outputs/inputs. The design of the individual Optomodules allows full functional measurement prior to assembly in an Optocard link. The high-speed data inputs/outputs to Optomodule were provided through electrical probes on the SLC substrate. Transmitter and receiver Optomodules were characterized using a single lensed optical fiber for optical coupling. Figure 5 shows a picture of the test setup with top and bottom views of a transmitter Optomodule under test, including the electrical and optical probing. To facilitate characterization of the Optomodules, they were mounted on a FR4 test card attached to an aluminum plate for mechanical support. The test card provides stable power and bias connections for the Optomodule through wire bond connections, and includes a milled cavity to allow back-side optical probing.

Figure 5. Optomodule test setup. Left show the top view detailing the electrical probes. Right shows the optical probe at the bottom of the Optomodule.

35µm x 35µm 62.5µm pitch35µm x 35µm 62.5µm pitch

BGA site for Optomodule

Waveguide Lens Array

BGA site for Optomodule

Waveguide Lens Array

35µm x 35µm 62.5µm pitch35µm x 35µm 62.5µm pitch

BGA site for Optomodule

Waveguide Lens Array

BGA site for Optomodule

Waveguide Lens Array

Back-side optical probing

High-speed electrical probing

Back-side optical probing

High-speed electrical probing

Proc. of SPIE Vol. 6899 68990V-4

Optical eye diagrams were measured for each of the 16 transmitters at data rates up to 20 Gb/s. All 16 channels showed clearly open eyes for the 27-1 PRBS pattern. The receiver outputs constitute complete links since one of the transmitter channels was used to provide the optical signal for receiver testing. All 16 receiver channels operated error-free for data rates up to 15 Gb/s. Eye diagrams for the transmitters and the receivers at 12.5Gb/s are shown in Figure 6. At 12.5 Gb/s, the 16Tx + 16Rx Optomodule provides a 200Gb/s bidirectional aggregate data rate.

Figure 6. Eye diagrams at 12.5Gb/s. Left shows 16 channels of TX Optomodule. Right shows 16 channels of RX Optomodule, representing a full TX-to-RX link.

The measured sensitivity of the 16 receiver channels at BER = 10-12 is shown in Figure 7. The sensitivity is better than -10 dBm (in Optical Modulation Amplitude) at 10Gb/s, with all 16 channels showing <1dB variation. The total power consumption of the Optomodule is only 2.2W.

Figure 7. Sensitivity of all 16 channels of receiver channels at 10Gb/s

TX RXTX RX

-14 -12 -10 -8 -6 -4 -2 0

-5

-6

-7

-8

-9

-10

-11

-12

OMA (dBm)

log1

0[ B

it E

rror

Rat

io ]

R1AR1BR1CR1DR2A

R2BR2C

R2DR3A

R3BR3C

R3DR4AR4BR4CR4D

Proc. of SPIE Vol. 6899 68990V-5

4. FULL LINK RESULTS

Assembly of the Optomodule to the Optocard provides simultaneous electrical and optical connections. The Optomodule is surface mounted to the Optocard using the BGA pads with eutectic PbSn solder to provide the electrical connections similar to conventional chip carriers. The optical system for coupling light from the OE devices to the waveguides is based on a two-lens optical relay system. The first lens is integrated into the backside of the OE devices while the second lens is incorporated with the waveguides and turning mirrors on the Optocard. These lenses are designed to provide nearly collimated light between the two lenses, providing greatly relaxed alignment tolerances and enhanced coupling efficiency between the Optomodule and the Optocard. The alignment tolerances realized in our prototype optical system allowed offsets up to ±35µm for the TX and up to ±70µm for the RX. The relaxed tolerances provide greater compatibility with typical board-level package assembly tooling.

Figure 8. A full link between 2 Optomodules. An infrared-sensitive camera is used to show the path of the 32 polymer waveguides.

The assembled full link between two Optomodules is also shown in Fig. 8. The optical interconnect utilizes 32 waveguides, 16 channels from Optomodule 1 to Optomodule 2 and 16 channels from Optomodule 2 to Optomodule 1. [5] The waveguide pitch is 62.5µm but is increased to 250µm at the center of the 15cm link in a simple demonstration of waveguide routing.

Figure 9. 10Gb/s eye diagrams for all 32 optical waveguide links between 2 Optomodules. Left shows 16 chanels from Optomodule 2 to Optomodule1 and right shows 16 channels from Optomodule 1 to Optomodule 2.

The Optocard full link assembly demonstrates a complete chip-to-chip optical interconnect on a printed circuit board. Figure 9 shows eye diagrams of all 32 channels at 10 Gb/s with all links operating error-free (BER < 10-12) with 3-6 dB margin. At this speed, each link consumes 13.5 mW/Gb/s. This Optocard link demonstrates the most highly integrated board-level optical data bus providing a record 160 Gb/s bidirectional transfer rate.

Optomodule 1 Optomodule 2Optomodule 1 Optomodule 2

Proc. of SPIE Vol. 6899 68990V-6

5. CONCLUSION

We have developed the requisite technologies for parallel optical interconnects for terabit/sec-class chip-to-chip and module-to-module communications through printed circuit boards with integrated optical waveguides. The data links are enabled by parallel transceiver modules, or “Optomodules,” that incorporate key technologies including: high-speed, low-power CMOS analog amplifier circuits; efficient, high-speed 985 nm substrate illuminated/emitting photodiode (PD) and VCSEL arrays with integrated collimating lenses; and the extensive use of flip-chip packaging to minimize the module dimensions and associated packaging parasitics. A novel Optochip has been demonstrated based on a single-chip CMOS optical transceiver IC with flip-chip attached VCSEL and PD arrays. The Optochip is then flip-chip attached to a high-density organic package to form the Optomodule. A conventional BGA solder process is then employed to mount this low-cost, low-profile package on a printed circuit board with integrated optical waveguides (an “Optocard”). Parallel polymer waveguides with 35-µm core dimensions on a 62.5-µm pitch are incorporated in the FR4-based Optocard. Optical and electrical connections are simultaneously established during the Optomodule-to-Optocard attachment. Sixteen-channel transceiver Optomodules were assembled and fully characterized, with transmitters operating up to 20 Gb/s while receivers (full link) operate up to 15 Gb/s, for an aggregate bi-directional data transfer rate of 240 Gb/s. Furthermore, we have demonstrated the fastest, widest, and most integrated multimode optical bus between two Optomodules, providing 32-unidirectional polymer waveguide links at 10Gb/s, for a 160 Gb/s bidirectional aggregate data rate.

Acknowledgement: This work was partially supported by DARPA under contract MDA972-03-3-0004.

REFERENCES

1 P.Pepeljugoski, et al., “Comparison of bandwidth limits for on-card electrical and optical interconnects for 100Gb/s and beyond,” SPIE Photonics West, San Jose, CA, Jan. 19–24, 2008. 2 L.Schares et al., “Terabus: Terabit/second-class card-level optical interconnect technologies,” IEEE J. Select. Topics Quantum Electron., Special Issue on Optoelectronic Packaging, Vol.12, No. 5, pp. 1032-1044, Sept./Oct. 2006. 3 F.Doany et al., “160-Gb/s Bidirectional Parallel Optical Transceiver Module for Board-Level Interconnects using a Single-Chip CMOS IC,” Proc. 57th Electron. Comp. and Technol. Conf., pp.1256-1261, Reno, NN, May 29 – June 1, 2007.4 C.Schow et al., “160-Gb/s, 16-Channel Full-Duplex, Single-Chip CMOS Optical Transceiver,” OFC 2007, Anaheim, CA, March 2007. 5 F.Doany et al., “Terabus : A 160Gb/s bidirectional board-level optical data bus,” IEEE LEOS 20th Annual meeting, Orlando, FL, Oct. 21–25, 2007

Proc. of SPIE Vol. 6899 68990V-7