Nov 15 2018

Training from ARC experts fuels discovery of AI methods to map the cosmos

George Stein and Philippe Berger

University of Toronto researchers are applying artificial intelligence techniques to simulate the universe

When George Stein and Philippe Berger, both PhD candidates at University of Toronto, were first taught neural network computational methods, they were skeptical that it would work in their specialized field of computational cosmology to create simulations of the universe.

“We build huge cosmological experiments to survey and observe more of the universe than ever seen before,” says Stein. “To help understand these observations we need to simulate what we expect to see. We’d always taken this problem, made approximations about the gravitational evolution in our simulated universes, and run those using the biggest computer you can get time on. That’s always been the way, and you need bigger and bigger computers to solve these equations as our observations get more complex.”

Adds Berger, “Both the observational techniques [new types of telescopes] and simulations which study the large-scale structure of the Universe are becoming increasingly HPC/ARC intensive. For example, many fundamental questions in cosmology are most easily or straight-forwardly addressed through direct simulation, but it takes weeks on the world’s largest supercomputers to simulate a single galaxy. Forget the billions that this generation of galaxy surveys will observe.”

“If all you need is the end of the research, then this method is perfect. If you need any of the data in between in order to understand the process of how you got to the end – which is often the case – you need the big systems, and it takes more time. Neural networks are inherently handicapped by the fact that you need big data sets to train them against.”

 

University of Toronto’s high-performance computing division, SciNet, offers training for students across the country who are interested in bringing computational methods into their research, and do training for researchers of all fields. Applications analyst and plasma physicist Erik Spence is the instructor for the Neural Networks courses that SciNet offers. When asked about the difference between traditional ARC and AI, he says, “You need both. You definitely need your old-school calculations before you can even think about using a neural network. When I was teaching George and Phil, they had a perfect problem that fell into the framework really nicely.”

“We completely bypassed the massive computations usually needed for these type of simulations and got straight to the answer,” says Stein. “We were actually able to create the cosmological simulations 1,000-times faster.”

The benefit of this, of course, is that the simulations take less time, power and resources. In this case, Stein and Berger needed cosmological maps to used to aid telescopes that look into the far reaches of the galaxy. For some uses, the maps can be relatively approximate.

“Neural Networks are an incredibly accurate approximation method, and we have shown this is the case even for gravity which is highly non-linear. Understanding why this is the case is currently a major focus of Machine Learning.”

 

Both Stein and Spence were clear, however, that using neural network or AI methods for all aspects of research simply doesn’t work. Spence notes that, “If all you need is the end of the research, then this method is perfect. If you need any of the data in between in order to understand the process of how you got to the end – which is often the case – you need the big systems, and it takes more time. Neural networks are inherently handicapped by the fact that you need big data sets to train them against.”

“For this specific application, making of the simulations, absolutely this changes our approach,” adds Stein. “But when we get the observational data back from the telescopes, this neural network isn’t going to help us.”

“Neural Networks are an incredibly accurate approximation method, and we have shown this is the case even for gravity which is highly non-linear. Understanding why this is the case is currently a major focus of Machine Learning. Gravity is a great example to study, since the exact answer can be simulated (although expensively). This provides a solid reference to compare to the output of the Neural Network and which helps in trying to understand its inner workings,” says Berger.

Both Berger and Stein have taken several courses with SciNet and appreciate the depth that it’s brought to the way they approach their research. “When I started my PhD I barely knew how to code. I had the option to take a few SciNet courses as modules for course credit towards my program. These first covered introductory coding skills, then scientific programming in C++ and Python, and gave me a solid foundation on which I could build. I was later able to take more advanced HPC courses and the Neural Networks course that Erik taught,” says Berger.

For all of the Ontario consortia, including HPC4Health, Centre for Advanced Computing, SciNet and SHARCNET, training the next generation of researchers is foundational to the work they do in addition to maintaining some of Canada’s most-powerful computing systems. In Ontario in 2017-18, the highly technical and knowledgeable instructors and staff delivered 25,321 teaching hours to 9,201 students.

Read Berger and Stein’s research publication.