Large-scale data transfer demo is a big win for researchers
Several leading Canadian and international organizations successfully tested large-scale data transfers at SC21, the world’s largest conference for high performance computing, networking, storage and analysis. Led by a team at the Compute Canada Federation (CCF), this test demonstrated how efficient use of high-speed networks for data transfers can be achieved by leveraging partnerships and leading-edge technology.
“Canada’s research community needs stable, dedicated, high-bandwidth connections to support data intensive science,” says Florent Parent, lead of the Networking National Team at the CCF and Advanced Research Computing site lead at the Université Laval. “Yet, the efficient use of high speed networks for large-scale data transfers over wide area networks (WAN) is still challenging. And that’s where our successful collaboration comes in.”
For this demonstration, datasets were transported over dedicated 100 Gbps networks from Université Laval, Simon Fraser University and University of Victoria to the StarLight International/National Communications Exchange Facility in Chicago and then to the final destination at the SC21 venue in St. Louis, Missouri.
“Data-intensive research — such as bioinformatics (cancer research, personalized medicine, genomics), high-energy physics, astronomy, environmental science and advanced manufacturing — relies on access to a robust digital research infrastructure to succeed,” says Florent. “High-speed networking plays an essential role to enable end-to-end data-driven workflows.”
Florent and his team partnered with Canada’s National Research and Education Network (CANARIE, RISQ and BCNET), the StarLight International/National Communications Exchange Facility Consortium and DELL to build the 100G network infrastructure to interconnect the sites to the SC21 conference. Each site was equipped with a cutting-edge data transfer node (DTN), a platform required to deliver high performance data flows required for data-intensive science.
“In the field of particle physics, we work in large international collaborations such as the ATLAS experiment at the CERN Laboratory in Geneva, Switzerland,” explains Dr. Randall Sobie, of the Institute of Particle Physics at the University of Victoria, and Chair of the Researcher Council for the Digital Research Alliance of Canada.
“The ATLAS experiment uses computing centres around the world, connected by the highest-speed research networks, to analyze our vast sample of particle collision data. Our data sample is expected to increase by a factor of ten in a few years and it is critical that our research networks, including those in Canada, can meet our critical requirements.”
The collaboration included Calcul Québec, CANARIE, Ciena®, Compute Canada, RISQ, BCNET, StarLight International/National Communications Exchange Facility, International Center for Advanced internet Research, Northwestern University, DELL, HPE, Université Laval, Simon Fraser University, University of Victoria and Triumf.
Quotes
“In collaboration with an international advanced networking community, we provide the required communication services for global data intensive sciences. During the IEEE/ACM International Conference on High Performance Computing, Networking, Storage and Analytics, we collaborated with Compute Canada, CANARIE, CERN, and other research partners to showcase demonstrations of leading-edge capabilities for large-scale global science, including capabilities of AI driven model for provisioning dynamic high-performance capacity for high energy physics.”
Joe Mambretti, International Center for Advanced Internet Research, Northwestern University