by Rosella (Qian-Ze) Zhu
figures by MacKenzie Mauger

Have you ever meticulously planned a holiday around some pleasant weather, according to the weather forecasts, only to find yourself caught in relentless rain throughout your vacation? The challenge of predicting weather, with its inherent unpredictability, extends beyond mere inconvenience; it’s crucial for ensuring safety in scenarios like driving in heavy rain, avoiding areas prone to wildfires, or preparing for strong winds and floods. Imagine a world where weather forecasts are as precise and personalized as the navigation app on your smartphone, and deciding whether to carry a raincoat or planning safe travel routes isn’t a morning dilemma clouded by ambiguous forecasts. This vision isn’t a distant dream– it’s rapidly becoming our reality thanks to the revolutionary impact of artificial intelligence (AI) and machine learning (ML) on meteorology, helping scientists better tackle and conquer the complexities of weather prediction.

AI, with its remarkable ability to sift through immense datasets to uncover complicated patterns, heralds a new era in meteorology. Major technology companies like Google Research, Google Deepmind, and Huawei have recently demonstrated the ability of ML-based models to outperform the traditional gold-standard methods in weather predictions, while requiring only a fraction of computational resources. From providing farmers with precise agricultural forecasts to predicting the path of deadly cyclones, AI and ML are transforming how we interact with and understand the weather (Figure 1). In this article, we’ll explore the transformative role of AI and ML in weather forecasting, delving into the underlying science, the potentially revolutionary improvement and potential applications they bring, as well as the challenges that lie ahead in our quest to predict the unpredictable.

The Traditional Way of Weather Prediction

For decades, meteorologists have relied on complex physical equations of the atmosphere to generate forecasts. In principle, if meteorologists know the current weather conditions, such as wind, pressure, and precipitation, they can use these pieces to generate a complete weather forecast; however, these equations are often inadequate, struggling to capture the intricate and ever-changing nature of our planet’s weather systems. Small mistakes in measuring the current weather or in the calculations can lead to big errors in the forecast.

Despite these great difficulties, scientists have significantly improved these traditional physics-based models by refining equations and reducing error. However, these improvements are costly, requiring expensive computational power and expert knowledge. Furthermore, these models won’t work in areas where accurate equations and robust numerical methods are not available, as in heat wave prediction or short-term rainfall forecasting.

Another main challenge with these models is managing the conflict between different timescales. Typically, people are interested in weather forecasts ranging from several hours to a few days ahead for everyday use. On the other hand, some people – in particular, environmentalists and scientists – often care about climate patterns that develop over decades. To resolve these multiple timescales effectively requires coming up with different underlying equations, or specially tailored algorithms and substantial computational resources.

Learning from the Sky: AI Joins the Forecast

AI models, specifically ML, are a type of technology that allows computers to learn from data and improve over time, much like how humans learn from experience. At its core, ML involves feeding a computer system plenty of data, which could be anything from pictures to numbers to text. This system then looks for patterns and rules within the data, usually with a function informing it of how good its current prediction aligns with the ground truth (Figure 2). Therefore, by using ML weather models, which are trained on massive amounts of weather data, we can capitalize on the rich history of weather patterns and learn to predict the future weather with greater accuracy and efficiency. Note that these models don’t just work with the data– they learn from it, identifying patterns and trends that human forecasters or traditional models might overlook.

The integration of AI and ML into meteorology marks a paradigm shift. A few months ago, studies from both Google Deepmind and Huawei demonstrated the power of AI-based models that are particularly adept at grasping complex phenomena like long-term climate predictions and extreme weather events, which traditional methods often struggle to predict accurately. More recently, Google Research and Google Deepmind announced their most up-to-date hybrid model, which combines the time-tested physics-based methods and the learning-focused techniques of ML. 

Specifically, this hybrid model has two main parts. The first part, the trunk, is called the dynamical core, which is similar to the traditional method and is built on a set of physics-based equations. This first part divides the Earth’s surface into a giant 3D grid and calculates wind, heat, and humidity for each grid point. This helps in understanding how different points interact with each other weather-wise.

The second part, called the learned physics module, fills in as the branches on the trunk. It deals with more nuanced weather aspects that the trunk can’t fully capture, like how clouds form, where and when it might rain, and how sunlight interacts with the Earth. In particular, it uses ML models to take in data about the current weather and predict what changes might happen in the next moment. 

In a nutshell, the hybrid model works by combining these two parts together in a step-by-step process. It starts with the current weather, uses the dynamical core to make some initial predictions, and then the learned physics module refines these predictions by adding more details. This process repeats iteratively, advancing the model to make predictions for the desired future timeframe.

While the traditional numerical methods spend a lot of time and computational power on the physical details, the hybrid model streamlines this, making the whole process quicker and more efficient with its AI-driven learned physics module (Figure 3). Therefore, the hybrid model excels in improving short-term weather forecasts while also providing reliable and consistent predictions for the long term, ranging from 1-15 days to climate forecasts spanning a decade (Figure 4).

Outlook

While AI has made impressive strides in weather forecasting, challenges remain. Ensuring the accuracy of AI predictions, especially in predicting rare or extreme events, and integrating these predictions into effective public warning systems, are ongoing tasks. However, the potential for further advancements is vast. With continuous advancements in AI technology and an ever-growing pool of data, it is promising that we will one day be able to forecast weather with even greater precision, finer resolution, and over longer time horizons.

As we stand at the cusp of a new era of the warming world, AI-driven weather forecasting is not just a technological marvel, but a necessity in our rapidly changing climate. The integration of AI and traditional methods is transforming the chaotic tapestry of weather into a clearer picture, promising a future where we can navigate our lives with a clearer understanding of the skies above.

---

Rosella (Qian-Ze) Zhu is a PhD student in the Applied Physics Program at Harvard University.

MacKenzie Mauger is a Ph.D. student in the Biological and Biomedical Sciences program at Harvard Medical School, where she is studying cell type-specific gene repression.

Cover image by WikiImages from pixabay.

For more information:

• For a detailed description of the hybrid model discussed here, check out the original paper.
• Check out the most recent (at the time of this writing) probabilistic weather forecasting model in this paper by Google Deepmind.
• For a collection of large weather models, check out this github repository.