Mar 5 2018

Niagara is Canada’s Most Powerful Research Supercomputer Fuelling Canadian Innovation and Discovery

Canada’s most powerful research supercomputer, Niagara, is now available to researchers of all disciplines across the country. Located at University of Toronto and supported by the university’s high-performance computing division SciNet, the system is open to all Canadian university researchers. Niagara is part of a system of national advanced research computing infrastructure, which includes Cedar at Simon Fraser University, Arbutus at the University of Victoria, and Graham at University of Waterloo. Together, these systems provide Canadian researchers with the tools to compete globally using big data and high-performance computing systems.

Advanced research computing is the backbone of Canadian innovation. Niagara will give researchers the computing power they need to study and find solutions to some of the world’s biggest challenges. It will enable large-scale computation and simulation required for artificial intelligence, climate change research, ocean modelling, genomics, astrophysics, and other disciplines using big data research to fuel discovery.

This new system is jointly funded by the Canada Foundation for Innovation, the Government of Ontario, and the University of Toronto.

Niagara is the fastest computer system in the country and is able to run a single job across all 60,000 cores thanks to a high-performance network which interconnects all the nodes. Niagara is highly energy efficient and is configured specifically to accelerate innovation.

The cluster is an end-to-end Lenovo solution with 1,500 ultra-dense ThinkSystem SD530 compute nodes, providing more than three petaflops of processing power, supported by 12 petabytes of storage.  Mellanox EDR InfiniBand is used to create an industry-first Dragonfly+ network topology featuring adaptive routing to provide the high-speed low-latency communications necessary for large-scale full-system simulations. Burst-buffer technology from Excelero helps improve performance for data-intensive work loads. The system leverages Lenovo Ethernet for cluster management.

Niagara is named after the iconic Niagara Falls; an Ontario landmark that symbolizes incredible power, discovery, and awe.

To download a copy of the full media release, please click here.

For more information, to schedule a media tour of Niagara, or to arrange an interview, please contact Carolyn Fell – carolyn.fell[at]computeontario.ca.

“Today, we celebrate the addition of Niagara to Canada’s powerful national network of research super computers. With access to such massive computational and data storage resources, researchers in all disciplines will be positioned to advance their work and tackle pressing global challenges. This kind of venture relies on collaboration among partners: institutions, Compute Canada, funding agencies and researchers alike.”   

-Roseann O’Reilly Runte, President and CEO, Canada Foundation for Innovation

“Niagara, and the three other national systems, will serve a diverse community of over 11,000 Canadian researchers and enable discoveries in areas that are key to Canada’s social and economic prosperity such as climate change, aerospace, and advanced manufacturing. We’re honoured to collaborate with our partners and proud of the achievements we’ve accomplished together.”

– Robbin Tourangeau, Interim CEO, Compute Canada

“Advanced research computing is the backbone of Canadian innovation. We are incredibly proud of the partnerships and collaboration that have made Niagara a reality for Canadian researchers; 40% of whom are based in Ontario.”

-Nizar Ladak, President & CEO, Compute Ontario

“It is our goal to help the world’s premier research teams and institutions leverage high performance computing to solve humanity’s greatest challenges. Our close partnership with University of Toronto, Compute Ontario and Compute Canada to advance Niagara as Canada’s most powerful research computing platform is a great example of collaborative innovation we can achieve in fulfilling that goal.”

-Scott Tease, executive director of HPC and AI at Lenovo Data Center Group

“Innovation and technology are advancing at a pace never before seen in history. Niagara will help our researchers continue to do their best work and make new discoveries, which in turn will allow us to grow Ontario’s knowledge-based economy.”

– The Honourable Reza Moridi, Ontario Minister of Research, Innovation and Science.

“Niagara is a major leap forward in advanced computing power for scientists and engineers at U of T and across Canada engaged in cutting-edge big data research from aerospace and astrophysics to health research and machine learning, and increasingly the social sciences and humanities. Access to such a powerful system, believed to rank among the top 50 supercomputers in the world, is also crucial to help us train — and retain — much-needed, highly qualified personnel.”

– Dr. Vivek Goel, Vice-President, Research & Innovation, University of Toronto

Facts about Niagara

Cores

Cores are the basic processing unit of all computers, including laptops and personal computers which may have between two and four cores. Niagara has a whopping 60,000 compute cores. The unique feature of Niagara is that it is configured in such a way as to allow computation simultaneously on all 60,000 cores.

Nodes

A node is another name for server.  It is the basic building block of a cluster and runs a single operating system and contains multiple cores, 40 per node on Niagara, that all share the same memory.  Niagara has 1,500 nodes.  The previous GPC had almost 4,000 nodes and half as many cores.

Memory and Storage

Each node has 192 Gigabytes of memory (12 times as much as on a high-end laptop), for a total of 288TB memory available to use during calculations.  In terms of disk space, Niagara has 12 PB of storage available for storing data from calculations.   A GB is 109 bytes, a TB is 1012 bytes, and a Petabyte is 1015 or a million-billion bytes.  Or thought of another way, Niagara has 12,000,000 GB of storage.

Power

For its size, Niagara is incredibly power efficient. The previous University of Toronto supercomputer consumed around 1000 kW of power. Niagara will use about 650 kW and has 10 times the performance, making it 15 times more energy efficient.   This is a power savings of roughly 300 average family homes.

Network

In the data centre alone, there are over 40km of fibre optic network cables linking all of the 1500 nodes with high-speed, low-latency connections.  This high speed InfiniBand interconnect with its Dragonfly+ network is one of the key features that differentiates Niagara from other systems in Canada, in that it makes it possible to use all 60,000 cores together to solve a single problem simultaneously.

 

Flops

FLOPs or Floating Point Operations per second for Niagara is theoretically 4.61 PF (PF = Peta Flop, Peta is 1015 or million billion).

Researcher quotes

“As an initial demonstration of full system capability, Niagara is currently being employed to perform a heroic computation of the interaction between the tides raised in the oceans by the gravitational attraction of the sun and moon and ocean bottom topography at very high spatial resolution. The accurate description of this interaction is at the heart of improving the skill of the ocean component of the climate models we employ to make global warming projections.”

Prof. W. Richard Peltier, FRSC is University Professor and Professor of Physics at the University of Toronto, Director of the Centre for Global Change Science, PI of the Polar Climate Stability Network, and Scientific Director of SciNet.

“The large parallel capability of Niagara enables world-leading precision cosmological simulations incorporating neutrinos. This draws talent from across the world, focusses their research strengths, and brings visibility to the research results”

Dr.Ue-Li Pen, Professor and Director, Canadian Institute for Theoretical Astrophysics, University of Toronto; Senior Fellow, CIFAR.

“The technological solutions for biggest challenges facing humankind are being driven by a combination of scientific know-how, data science, and supercomputing. In this digital economy, research and innovation competitiveness is highly dependent upon the high-performance computing infrastructure. In this regard, the installation of Niagara, Canada’s newest computing facility, fulfills a critical need and will allow my research group to computationally design and discover new materials for promoting sustainable energy, transportation, and health in Canada and worldwide.”

Dr. Chandra Veer Singh, PhD, PEng is Erwin Edward Hart Endowed Associate Professor of Materials Science and Engineering, and Associate Chair for Research in the Department of Materials Science and Engineering at University of Toronto.

“As is recognized by many countries and increasingly in Canada, advanced computing underpins a great deal of work in science and engineering.  Consequently, access to world-class computing power is essential to enable research to address important societal issues in a timely manner. For example, Canada’s new advanced research computer, Niagara, is crucial for my research investigating next generation aircraft configurations with improved fuel efficiency and reduced environmental impact.”

 

Dr. David Zingg PhD, FCASI, AFAIAA, FCAE is University of Toronto Distinguished Professor of Computational Aerodynamics and Sustainable Aviation; Director, Centre for Research in Sustainable Aviation; Director, Centre for Computational Science and Engineering, University of Toronto Institute for Aerospace Studies