Multi-fibre technology for supercomputers

When it comes to the really big challenges in science and research, they are ready: the supercomputers. They are capable of quickly processing and storing the extremely large amounts of data that accumulate at the research centre. This requires high-performance and fail-safe computer systems. The demands on the network infrastructure are therefore high. In order to meet these demands, the Jülich Supercomputing Centre of the Research Centre Jülich has implemented MPO multi-fibre technology and components of the tML-24 system platform from tde - trans data elektronik.

Based on the key competencies of physics and supercomputing, the Research Centre Jülich (FZJ) conducts interdisciplinary research in the fields of information, energy and bioeconomy. With about 5,800 employees in 2018, it is one of the largest research institutions in Europe. The FZJ is divided into ten institutes, including the Institute for Advanced Simulation (IAS). The IAS combines simulation sciences and supercomputing in one location. It is divided into eight institute divisions, including the Jülich Supercomputing Centre (JSC).

Highest demands on the IT infrastructure

The JSC has been operating the first German high-performance computing centre since 1987 and provides computing capacity of the highest performance class to scientists at FZJ, at universities, research institutions and in industry via an independent peer review process. Since September 2018, it has been operating JUWELS, one of the most powerful computers in Europe. The JSC has around 200 employees working on all aspects of supercomputing and simulation science. In cooperation with renowned hardware and software manufacturers such as IBM, Intel and ParTec, the JSC also meets the special challenges posed by the development of exaflop systems - the computers of the next supercomputer generation.

Supercomputers process huge amounts of data at high speed. The IT is therefore confronted with extreme demands. Complex simulations with countless variables and real-time applications require fast, fail-safe and high-performance computer systems and network infrastructures. Ensuring this is the task of the JSC Communication Systems Department. The department is responsible for the planning, implementation and operation of the campus-wide communication networks of the FZJ as well as the connection to external networks. It provides communication services and communication applications and assumes central tasks in the IT security process of the research centre. Various network technologies are used, which are tailored to the respective requirements.

A convincing quality

To be able to handle the high data volume, JSC decided to upgrade its network infrastructure with high-performance components. Picked for the purpose was a solutions from tde - trans data elektronik GmbH. JSC and the Dortmund-located network expert have been working together successfully for over a decade. The most important requirements for the new components were speed, high availability and reliability. Following in-depth consultations, tde supplied MPO patch cords and tML distribution technology. The high quality and reliability of tde's "Made in Germany" products, with their high packing efficiency in the rear area, and the short delivery and service times, were the deciding factors for tde. The department also wanted a plug-and-play solution that could be integrated into the IT environment without great effort.

Tde supplied tML fibre optic patch cords MPO Male/Female, tML-24 - fibre optic modules MPO, tML – rack mount enclosures 19"/1RU and, as accessories, blind plates and cable glands exactly to the planning specifications. The modular tML-24 cabling system consists of the three core components module, trunk cables and rack mount enclosures. The system components are 100 percent manufactured, pre-assembled and tested in Germany.

The modular link system can be implemented by plug-and-play very quickly and can be expanded or reduced at any time without having to replace the entire system. "In terms of plug-and-play, tde's tML systems deliver what they promise. The systems are easy to integrate. I was able to carry out the installation without any instruction", says Rainer Kofahl, Division Communication Systems of JSC.

The tde Modular Link system is characterised by high packing density and great flexibility for migrating to higher transmission rates. On a 19-inch height unit, 96 MPO connectors can be used modularly. Thanks to its patented pinout and dark fibre modules, the tde Modular Link system offers easy migration options to 400G and above. The highly compact design results in enormous space savings, which is why only one twelfth of the space is required compared to conventional cabling systems. In addition to the high transfer rates, fibre optic cables have smaller diameters and therefore lower cable volumes. The ventilation paths in the network cabinets are less blocked, which saves energy. Compared to conventional cables, the smaller cable diameters help to reduce fire loads in data centres.

Precisely manufactured components

Manufacturing a high quality MPO connector requires precision and know-how. The distribution of several fibres closely spaced at a pitch of 0.25 mm in a single ferrule not only places high demands on the production of high-precision ferrules with low tolerance, but also on the assembly of high-performance connectors. A subsequent tuning or alignment of the ferrules in the connector - common practice with other connectors to compensate for tolerances in the ferrules and thus achieve better performance - is not possible here. On the component side, this means that the holes for fibres and guide pins in particular must be of high precision. High quality and special properties of the plastic material used are required. On the manufacturing side, good performance of the MPO connectors can only be achieved by constantly optimizing and adapting the manufacturing processes. Only tightened geometric specifications for the connector surface, which exceed the IEC requirements, as well as the 100% inspection of these specifications can guarantee a constant quality and lowest attenuation losses.

Thanks to the fine selection of materials used in product design and the smoothly coordinated manufacturing processes, even complex cable constructions and sub racks (modules) from tde score points with first-class technical parameters. The tML systems are calibrated for extended temperature ranges (-40 degrees Celsius to +80 degrees Celsius). The trunk cables have - despite their slim cross-section - good longitudinal, transverse compression and tensile properties. Lastly, the system's plug connections are more resistant to lateral pull forces due to very tight manufacturing tolerances.

Ready for the future

After completing the installation, the resonance was consistently positive. The new network components fully meet the requirements of the JSC. And the research centre does not need to worry about the future either: thanks to the highly scalable solution, FZ Jülich can keep pace with the increasing requirements and, in the case of even faster supercomputers, migrate to higher transmission rates at any time.