Shanbhag, his students and ICDS researchers worked relentlessly through 2019 to make the model efficient and competitive for the Department of Energy's challenge. IMAGE: iStock/@metamorworks
Interdisciplinary team wins Department of Energy Optimization Challenge
Posted on June 9, 2020UNIVERSITY PARK, Pa. — A team of Penn State researchers is part of the first round of winners for the Department of Energy’s (DOE) Grid Optimization (GO) Competition. Presented by the DOE’s Advanced Research Projects Agency-Energy (ARPA‑E), the selective competition presents challenges for the development of optimization algorithms for a crucial set of operational problems faced by the United States’ power grid.
The team, led by Uday V. Shanbhag, the Gary and Sheila Bello Chair and professor in the Harold and Inge Marcus Department of Industrial and Manufacturing Engineering (IME) in the College of Engineering, received $100,000 to advance their methods and share their technology with industry partners.
“Keeping the lights on is a necessity, and optimization plays a key role,” Shanbhag said. “Optimization theory can help provide a foundation for developing, implementing and analyzing algorithms that can be applied in settings of profound importance.”
Along with other universities and national laboratories, Shanbhag and Shiseng Cui, an IME postdoctoral scholar in Shanbhag’s lab, worked to create a solution for the enhancement of the United States’ current electricity grid, as well as for the country’s future power problems concerning renewable energy sources. Early support was also provided by former IME postdoc Jinlong Lei and former IME doctoral student Wendian Wan.
Halfway through the project period, the group started receiving assistance from the Institute of Computational and Data Sciences (ICDS) at Penn State to facilitate implementations on high performance computing (HPC) environments. The Research Innovations with Scientists and Engineering (RISE) team members serve as consultants with their extensive supercomputing experience and strong background in academic research.
“The computational research expertise provided by the ICDS RISE team contributed to the success of this talented research team in the competition,” said Jenni Evans, director of ICDS and professor of meteorology and atmospheric science. “RISE team members have a variety of research expertise, providing a strong computational foundation for research groups throughout the University. We look forward to seeing similar successes as we continue to forge new RISE partnerships.”
How they answered the challenge
The team had to design an algorithm to rapidly determine the dispatch of electricity efficiently, safely and consistently within the current grid for both traditional and renewable energy sources, otherwise known as security-constrained optimal power flow (OPF).
Challenges occur when developing algorithms that need to cope with the various possibilities that can arise when implementing the final dispatch decisions. For instance, a particular transmission line or power plant may fail or there may be a sudden and unforeseen surge in demand.
“The next generation of algorithms needs to contend with a multitude of such contingencies by being able to reliably allocate energy across the system at a moment’s notice when an incident occurs, such as a natural disaster or energy failure,” Shanbhag said.
Over time, models used in solving power grid problems have adapted to provide approximate answers, but today’s society needs far more accurate models that can contend with the fundamentally nonlinear nature of this problem. The task is not small: the power grid has more than 65 million nodes, and a solution has to be provided every 10 minutes.
Their approach focused on creating an algorithm that can properly gauge the size of the optimization problem, address its nonlinear nature and conserve computing resources so they did not overburden the system.
Shisheng Cui, formerly one of Shanbag’s graduate students, explained that the nonlinearity is challenging and is exacerbated by the size of the system; the data is not organized sequentially and cannot be easily handled for solving the problem. Additionally, Cui noted that the nature of the contest presented a unique challenge in terms of scale and complexity of the problems. Conventional solutions are incapable of addressing such problems and the team built their algorithm from scratch while leveraging existing solutions for solving smaller structured problems.
After the algorithm was designed, the team was provided sample power grid data from ARPA-E to test their algorithm and showcase its ability to find minimum-cost solutions. After the first trial ended and the team received their results, Shanbhag sought the assistance of RISE to aid in developing HPC implementations.
Justin Petucci and Danying Shao, computational scientists with the ICDS RISE team, worked to restructure the algorithm’s code to operate in a high-performance computing (HPC) environment, remove redundancies and provide overall code optimization to speed up the algorithm for an HPC environment.
“Our goal was to make this as fast as possible by allowing it to run quickly on a single node, which then allows it to run quickly across multiple nodes,” Petucci said.
After the code optimization, Petucci and Shao estimated that by leveraging parallelism and aynchronicity, the model was running up to 10 times faster than its previous rendition. This increase in speed proved to be a success.
“The assistance from Justin and Danying was crucial,” Shanbhag said. “One such instance required determining a particular switch that facilitated improved operation and addressed communication failures. This was akin to finding a needle in a whole slew of haystacks. The code had to perform in a computational environment that was hard to simulate and my team didn’t have the resources to run similar tests. Justin and Danying found a similar environment, hacked out any issues in the code and really helped in improving the robustness when the code was transitioned to the testing environment.”
What’s next?
Future competitions, beginning with challenge two, will build on the models used in
challenge one, but may include complicating factors such as solving larger network models,
optimizing power flows over both transmission and distribution systems, contending with uncertainty and discreteness, leveraging power flow control devices and increasing model detail. Challenge two will likely continue its focus on OPF and disburse fewer, but larger, awards.
“This is just the first step because these problems are incredibly challenging to solve, and this was a simpler version of the real problem,” Shanbhag said.
The ICDS RISE team members are excited for the potential to continue their involvement on this project, as Petucci explained that this kind of project is the poster child for RISE.
“RISE brought the ability to rapidly test on the ICDS Advanced CyberInfrastructure (ICDS-ACI),” Petucci said. “If they ran it on the competition system, they could only test five times a day, whereas the ICDS-ACI could run it nearly 10 times more. RISE gives another level of performance for Penn State researchers.”
Shao explained that she was honored to participate in the project and believes that it is an ideal collaboration for their team.
Shisheng said that this experience has given him the confidence to go forth and solve real-world, big data problems.
“Much of the success of this effort can be attributed to the remarkable research ecosystem at Penn State and within IME, which leads to attracting a tremendous set of faculty members and the finest students and research scholars across multiple colleges,” said Shanbhag. “Given that this work lies at the intersection of power systems operations and computational science, we are fortunate to be supported by Penn State’s Institutes of Energy and the Environment (PSIEE), led by Tom Richard, and ICDS, led by Jenni Evans. They both have been enormously supportive in the run-up to this competition. PSIEE provided early financial support for these efforts while ICDS provided crucial support through both the RISE team and their HPC resources. Lastly, the support by Gary and Sheila Bello, through the Bello chair, over the last four years has been immeasurably important in building a foundation for such initiatives.”
Based in interdisciplinary teamwork from the College of Engineering and the College of Earth and Mineral Sciences, the original team consisted of Shanbhag; Hosam Fathy, former Bryant Early Career Professor of Mechanical Engineering and now a professor at the University of Maryland at College Park; Mort Webster, professor of energy and mineral engineering and Ryan Family Faculty Fellow; Chiara Lo Prete, associate professor of energy economics; Nilanjan Ray Chaudhuri, assistant professor of electrical engineering and computer science; and Minghui Zhu, associate professor of electrical engineering and ICDS co-hired faculty member.
Lei and Wan graduated in 2019. Lei has taken a faculty position with Tongji University in Shanghai, and Wan now works with Amazon.