Open In Colab   Open in Kaggle

Heatwaves: Assessing the Dynamic Interactions of the Atmosphere and Land#

Content creators: Sara Shamekh, Ibukun Joyce Ogwu

Content reviewers: Sloane Garelick, Grace Lindsay, Douglas Rao, Chi Zhang, Ohad Zivan

Content editors: Sloane Garelick, Zane Mitrevica, Natalie Steinemann, Ohad Zivan, Chi Zhang

Production editors: Wesley Banfield, Jenna Pearson, Chi Zhang, Ohad Zivan

Our 2023 Sponsors: NASA TOPS, Google DeepMind, and CMIP

# @title Tutorial slides
# @markdown These are the slides for the videos in all tutorials today
from IPython.display import IFrame
link_id = "wx7tu"

The atmosphere and land are entwined components of the Earth’s system, constantly exchanging energy, mass, and momentum. Their interaction contributes to a variety of physical and biological processes. Understanding of the dynamic interactions between atmosphere and land is crucial for predicting and mitigating the impacts of climate change, such as land-use changes and hazards ranging from droughts, floods, and even fluctuation in agricultural production and products (Jach et. al., 2022; Ogwu et. al. 2018; Dirmeyer et. al. 2016).

Climate change is also expected to have a significant impact on cereal production around the world. Changes in temperature, precipitation patterns, and extreme weather events can all affect crop yields, as well as the timing and quality of harvests. For example, higher temperatures can lead to reduced yields for crops like wheat and maize, while changes in rainfall patterns can result in droughts or floods that can damage crops or delay planting.

In order to better understand the relationship between climate change and cereal production, researchers have begun to explore the use of environmental monitoring data, including air temperature and soil moisture, to help identify trends and patterns in crop production. By collecting and analyzing this data over time, it may be possible to develop more accurate models and predictions of how climate change will affect cereal production in different regions of the world.

However, it is important to note that while environmental monitoring data can provide valuable insights, there are many other factors that can affect cereal production, including soil quality, pests and diseases, and agricultural practices. Therefore, any efforts to correlate cereal production with climate change must take into account a wide range of factors and be based on robust statistical analyses in order to ensure accurate and reliable results.

In this project, you will look into how specific climate variables represent and influence our changing climate. In particular,you will explore various climate variables from model data to develop a more comprehensive understanding of different drivers of heatwaves (periods during which the temperature exceeds the climatological average for a certain number of consecutive days over a region larger than a specified value). You will further use this data to understand land-atmosphere interactions, and there will also be an opportunity to relate the aforementioned climate variables to trends in cereal production.

Project Template#

Project Template

Note: The dashed boxes are socio-economic questions.

Data Exploration Notebook#

Project Setup#

# google colab installs
# !pip install condacolab
# import condacolab
# condacolab.install()
# !mamba install xarray-datatree intake intake-esm gcsfs xmip aiohttp cartopy nc-time-axis cf_xarray xarrayutils "esmf<=8.3.1" xesmf
# imports
import time

tic = time.time()

import pandas as pd
import intake
import numpy as np
import matplotlib.pyplot as plt
import xarray as xr
import xesmf as xe

from xmip.preprocessing import combined_preprocessing
from xarrayutils.plotting import shaded_line_plot

from datatree import DataTree
from xmip.postprocessing import _parse_metric

import as ccrs
import random
import pooch
import os
import tempfile
# helper functions

def pooch_load(filelocation=None,filename=None,processor=None):
    shared_location='/home/jovyan/shared/Data/Projects/Heatwaves' # this is different for each day
    if os.path.exists(os.path.join(shared_location,filename)):
        file = os.path.join(shared_location,filename)
        file = pooch.retrieve(filelocation,known_hash=None,fname=os.path.join(user_temp_cache,filename),processor=processor)

    return file
# @title Figure settings
import ipywidgets as widgets  # interactive display

%config InlineBackend.figure_format = 'retina'
# model_colors = {k:f"C{ki}" for ki, k in enumerate(source_ids)}
%matplotlib inline

CMIP6: Near Surface Temperature#

You will utilize a CMIP6 dataset to examine temperature trends and heatwaves, applying the CMIP6 loading methods intreduced in W2D1. To learn more about CMIP, including additional ways to access CMIP data, please see our CMIP Resource Bank and the CMIP website.

Specifically, in this project you will focus on near-surface temperature, which refers to the air temperature at the Earth’s surface. In this study, you will analyze data from one model and examining its historical temperature records. However, we encourage you to explore other models and investigate intermodel variability, as you learned (or will learn) during your exploration of CMIP datasets in the W2D1 tutorials.

After selecting your model, you will plot the near-surface air temperature for the entire globe.

# loading CMIP data

col = intake.open_esm_datastore(
)  # open an intake catalog containing the Pangeo CMIP cloud data

# pick our five example models
# There are many more to test out! Try executing `col.df['source_id'].unique()` to get a list of all available models
source_ids = ["MPI-ESM1-2-LR"]
# from the full `col` object, create a subset using facet search
cat =
    experiment_id=["historical"],  # add scenarios if interested in projection
    ],  # make sure that we only get models which have all of the above experiments

# convert the sub-catalog into a datatree object, by opening each dataset into an xarray.Dataset (without loading the data)
kwargs = dict(
    preprocess=combined_preprocessing,  # apply xMIP fixes to each dataset
    ),  # ensure all datasets use the same time index
        "token": "anon"
    },  # anonymous/public authentication to google cloud storage

cat.esmcat.aggregation_control.groupby_attrs = ["source_id", "experiment_id"]
dt = cat.to_datatree(**kwargs)
--> The keys in the returned dictionary of datasets are constructed as follows:
100.00% [1/1 00:04<00:00]
<xarray.DatasetView> Size: 0B
Dimensions:  ()
Data variables:
# select just a single model and experiment
tas_historical = dt["MPI-ESM1-2-LR"]["historical"].ds.tas
print("The time range is:")
The time range is:
1850-01-01T03 to 2015-01-01T00

Now it’s time to plot the data. For this initial analysis, we will focus on a specific date and time. As you may have noticed, we are using 3-hourly data, which allows us to also examine the diurnal and seasonal cycles. It would be fascinating to explore how the amplitude of the diurnal and seasonal cycles varies by region and latitude. You can explore this later!

fig, ax_present = plt.subplots(
    figsize=[12, 6], subplot_kw={"projection": ccrs.Robinson()}

# plot a timestep for July 1, 2013
tas_present = tas_historical.sel(time="2013-07-01T00").squeeze()
tas_present.plot(ax=ax_present, transform=ccrs.PlateCarree(), cmap="magma", robust=True)
ax_present.set_title("July, 1st 2013")
Text(0.5, 1.0, 'July, 1st 2013')

CMIP6: Precipitation and Soil Moisture (Optional)#

In addition to examining temperature trends, you can also load precipitation data or variables related to soil moisture. This is an optional exploration, but if you choose to do so, you can load regional precipitation data at the same time and explore how these two variables are related when analyzing regional temperature trends. This can provide insights into how changes in temperature and precipitation may be affecting the local environment.

The relationship between soil moisture, vegetation, and temperature is an active field of research. To learn more about covariability of temperature and moisture, you can have a look at Dong et al. (2022) or Humphrey et al. (2021).

World Bank Data: Cereal Production and Land Under Cereal Production#

Cereal production is a crucial component of global agriculture and food security. The World Bank collects and provides data on cereal production, which includes crops such as wheat, rice, maize, barley, oats, rye, sorghum, millet, and mixed grains. The data covers various indicators such as production quantity, area harvested, yield, and production value.

The World Bank also collects data on land under cereals production, which refers to the area of land that is being used to grow cereal crops. This information can be valuable for assessing the productivity and efficiency of cereal production systems in different regions, as well as identifying potential areas for improvement. Overall, the World Bank’s data on cereal production and land under cereals production is an important resource for policymakers, researchers, and other stakeholders who are interested in understanding global trends in agriculture and food security.

# code to retrieve and load the data
filename_cereal = 'data_cereal_land.csv'
url_cereal = ''
ds_cereal_land = pd.read_csv(pooch_load(url_cereal,filename_cereal))
Downloading data from '' to file '/tmp/data_cereal_land.csv'.
SHA256 hash of downloaded file: 0d71645aeeb9e1cca8abe179c525c496f3b2b02867119069679762c0f9f1da47
Use this value as the 'known_hash' argument of 'pooch.retrieve' to ensure that the file hasn't changed if it is downloaded again in the future.
Country Name Country Code Series Name Series Code 1972 [YR1972] 1973 [YR1973] 1974 [YR1974] 1975 [YR1975] 1976 [YR1976] 1977 [YR1977] ... 2012 [YR2012] 2013 [YR2013] 2014 [YR2014] 2015 [YR2015] 2016 [YR2016] 2017 [YR2017] 2018 [YR2018] 2019 [YR2019] 2020 [YR2020] 2021 [YR2021]
0 Afghanistan AFG Cereal production (metric tons) AG.PRD.CREL.MT 3950000 4270000 4351000 4481000 4624000 4147000 ... 6379000 6520329 6748023.28 5808288 5532695.42 4892953.97 4133051.85 5583461 6025977 4663880.79
1 Afghanistan AFG Land under cereal production (hectares) AG.LND.CREL.HA 3923100 3337000 3342000 3404000 3394000 3388000 ... 3143000 3182922 3344733 2724070 2793694 2419213 1911652 2641911 3043589 2164537
2 Albania ALB Cereal production (metric tons) AG.PRD.CREL.MT 585830 625498 646200 666500 857000 910400 ... 697400 702870 700370 695000 698430 701734 678196 666065 684023 691126.7
3 Albania ALB Land under cereal production (hectares) AG.LND.CREL.HA 331220 339400 334040 328500 350500 357000 ... 142800 142000 143149 142600 148084 145799 140110 132203 131310 134337
4 Algeria DZA Cereal production (metric tons) AG.PRD.CREL.MT 2362625 1595994 1480275 2680452 2313186 1142509 ... 5137455 4912551 3435535 3761229.6 3445227.37 3478175.14 6066252.82 5633596.78 4393336.75 2784017.29

5 rows × 54 columns

Hint for Q7: Heatwave Detection#

Question 7 asks you to detect heatwave. Below you can see a flowchart for detecting heatwaves. The flowchart includes three parameters that you need to set in adavance. These three parameters are:

  1. w-day: the window (number of days) over which you detect the extreme (95 percentile) of temperature.

  2. E (km2): the spatial extent of the heatwave.

  3. X (days): the duration of heatwave.


Hint for Q9: Correlation#

For Question 9 you need to compute the correlation between two variables. You can use Pearson’s correlation coefficient to evaluate the correlation between two variables. You can read about Pearsons correlation coefficient on Wikipedia and from Scipy python library. You are also encouraged to plot the scatter plot between two variables to visually see their correlation.

Hint for Q12: Linear Regressions for Heatwave Detection#

For Question 12, read the following article: Rousi et al. (2022)

For Question 12 you need to build the regession model. You can read abut regression models on Wikipedia and from Scipy python library.

Hint for Q13: Data-Driven Approaches for Heatwave Detection#

For Question 13, read the following articles: Li et al. (2023) and Jacques-Dumas et al. (2022)

Further Reading#

  • Dirmeyer, P. A., Gochis, D. J., & Schultz, D. M. (2016). Land-atmosphere interactions: the LoCo perspective. Bulletin of the American Meteorological Society, 97(5), 753-771.

  • Ogwu I. J., Omotesho, O. A. and Muhammad-Lawal, A., (2018) Chapter 11: Economics of Soil Fertility Management Practices in Nigeria in the book by Obayelu, A. E. ‘Food Systems Sustainability and Environmental Policies in Modern Economies’ (pp. 1-371).Hershey, PA: IGI Global. doi:10.4018/978-1-5225-3631-4

  • Jach, L., Schwitalla, T., Branch, O., Warrach-Sagi, K., and Wulfmeyer, V. (2022) Sensitivity of land–atmosphere coupling strength to changing atmospheric temperature and moisture over Europe, Earth Syst. Dynam., 13, 109–132,


This tutorial uses data from the simulations conducted as part of the CMIP6 multi-model ensemble.

For examples on how to access and analyze data, please visit the Pangeo Cloud CMIP6 Gallery

For more information on what CMIP is and how to access the data, please see this page.