site_data <- readr::read_csv("https://raw.githubusercontent.com/eco4cast/neon4cast-targets/main/NEON_Field_Site_Metadata_20220412.csv") |>
dplyr::filter(terrestrial == 1)4 Theme: Terrestrial Ecosystems
What: Net ecosystem exchange(NEE) of CO2 and evapotranspiration in terrestrial ecosystems
Where: 47 NEON sites across the U.S. and Puerto Rico.
When: Daily forecasts for at least 30-days in the future are accepted at any time. The only requirement is that submissions are predictions of the future at the time the forecast is submitted.
Why: Carbon and water cycling are fundamental for climate and water regulation services provided by ecosystems
Who: Open to any individual or team that registers
How: REGISTER your team and submit forecast.
The video below is an overview of the Terrestrial Carbon and Water Fluxes Challenge that was recorded for the 2021 Early Career Annual Meeting
We held a Q&A session on January 22, 2021. You can find a recording from that session HERE.
4.1 Overview
The exchange of water and carbon dioxide between the atmosphere and the land is akin to earth’s terrestrial ecosystems breathing rate and lung capacity. One of the best ways to monitor changes in the amount of carbon and water in an ecosystem is the eddy-covariance method. This method observes the net amount of carbon and water entering and exiting ecosystems at half-hourly timesteps, which is important because it can provide information on ecosystem processes such as photosynthesis, respiration, and transpiration, their sensitivities to ongoing climate and land use change, and greenhouse gas budgets for carbon accounting and natural climate solutions. Forecasts of carbon uptake and release along with water use can provide insights into future production of food, fiber, timber, and carbon credits. Additionally, forecasts will highlight the influence that stress and disturbance have on carbon and water cycling.
4.2 Challenge
This forecasting challenge asks teams to forecast net ecosystem exchange of carbon dioxide (NEE) and latent heat flux of evapotranspiration (LE) across 47 NEON sites with differing climates. Forecasts can be submitted the 30-minute and/or daily time step over the next 30-days. Weather forecasts from NOAA Global Ensemble Forecast System are provided to use as model drivers (if forecasting model uses meteorological inputs).
Teams are asked to submit their forecast of NEE and LE, along with uncertainty estimates. Any existing NEE and LE data may be used to build and improve the models used to generate forecasts. Other data can be used to generate forecasts.
4.3 Data: Targets
The challenge uses the following NEON data products:
DP4.00200.001: Bundled data products - eddy covariance
A file with previously released NEON data that has been processed into “targets” is provided below. The same processing will be applied to new data that are used for forecast evaluation. A processing script is available on the neon4cast-terrestrial GitHub repository.
4.3.1 Net ecosystem exchange
Definition
Net ecosystem exchange (NEE) is the net movement of carbon dioxide from the atmosphere to the ecosystem. At the 30-minute time resolution it is reported as \(\mu\)mol CO2 m-2 s-1. At the daily time resolution it is reported as g C m-2 day-1. Negative values correspond to an ecosystem absorbing CO2 from the atmosphere, positive values correspond to an ecosystem emitting CO2 to the atmosphere.
Motivation
NEE quantifies the net exchange of CO2 between the ecosystem and the atmosphere over that 30-minute or daily time period. Assessing skill at predicting 1/2 hourly - sub daily measurements provides more insight into ability to capture diel processes. The diel curve contains information on how plants and soil immediately respond to variations in meteorology.
Making daily predictions will allow us to rapidly assess skill and provide information in a timeframe pertinent to inform and implement natural resource management. It also allows for models that do not produce sub-daily estimates to participate
4.3.2 Latent heat flux
Definition
Latent heat flux is the movement of water as water vapor from the ecosystem to the atmosphere. It is reported as W m-2 (equivalent to J m-2 s-1). At the daily time resolution it is reported as mean W m-2. Positive values correspond to a transfer of water vapor from the ecosystem to the atmosphere.
Motivation
Latent heat measures the water loss from an ecosystem to the atmosphere through evapotranspiration (transpiration through plants + evaporation from surfaces).
Forecasting latent heat (evapotranspiration) can provide insights to water stress for plants and the efficiency that plants are using water relative to NEE, and to the amount of liquid water remaining in the soil for soil moisture forecasting
4.3.3 Focal sites
Information on the sites can be found here:
with full site table at the end of this page.
The distribution of sites across ecosystems types is:
| Vegetation type | Count |
|---|---|
| Evergreen Forest | 4 |
| Evergreen Forest|Shrub/Scrub | 4 |
| Deciduous Forest|Evergreen Forest|Mixed Forest|Woody Wetlands | 3 |
| Evergreen Forest|Grassland/Herbaceous|Shrub/Scrub | 3 |
| Grassland/Herbaceous | 3 |
| Cultivated Crops | 2 |
| Deciduous Forest|Evergreen Forest|Mixed Forest | 2 |
| Deciduous Forest|Evergreen Forest|Pasture/Hay | 2 |
| Deciduous Forest|Grassland/Herbaceous | 2 |
| Deciduous Forest|Mixed Forest|Woody Wetlands | 2 |
| Deciduous Forest|Pasture/Hay | 2 |
| Shrub/Scrub | 2 |
| Cultivated Crops|Deciduous Forest | 1 |
| Cultivated Crops|Deciduous Forest|Evergreen Forest|Mixed Forest | 1 |
| Cultivated Crops|Grassland/Herbaceous|Pasture/Hay | 1 |
| Deciduous Forest | 1 |
| Deciduous Forest|Evergreen Forest | 1 |
| Deciduous Forest|Woody Wetlands | 1 |
| Dwarf Scrub|Evergreen Forest|Shrub/Scrub | 1 |
| Dwarf Scrub|Shrub/Scrub | 1 |
| Emergent Herbaceous Wetlands | 1 |
| Emergent Herbaceous Wetlands|Evergreen Forest|Woody Wetlands | 1 |
| Emergent Herbaceous Wetlands|Grassland/Herbaceous | 1 |
| Evergreen Forest|Grassland/Herbaceous | 1 |
| Evergreen Forest|Shrub/Scrub|Woody Wetlands | 1 |
| Evergreen Forest|Woody Wetlands | 1 |
| Grassland/Herbaceous|Shrub/Scrub | 1 |
| Pasture/Hay|Woody Wetlands | 1 |
4.3.4 30-minute target data calculation
To create the data for evaluation (and training) for NEE and LE we extract NEE and LE that pass the turbulence quality control flags (qfqm.fluxCo2.turb.qfFinl = 0 ) provided by NEON and has flux values between -50 and 50 umol CO2 m-2 s-1.
The table with the half-hour NEE and LE has the following columns
datetime: YYYY-MM-DD HH:MM for the start of the 30-minute period in UTC
site_id: NEON site code (e.g., BART)
variable:nee(umol CO2 m-2 s-1) orleW m-2
observed: value for variable
Here is the download link and format of the terrestrial_30min target file:
readr::read_csv("https://data.ecoforecast.org/neon4cast-targets/terrestrial_30min/terrestrial_30min-targets.csv.gz", guess_max = 1e6)# A tibble: 10,461,448 × 4
datetime site_id variable observation
<dttm> <chr> <chr> <dbl>
1 2017-02-01 10:00:00 ABBY nee NA
2 2017-02-01 10:30:00 ABBY nee NA
3 2017-02-01 11:00:00 ABBY nee NA
4 2017-02-01 11:30:00 ABBY nee NA
5 2017-02-01 12:00:00 ABBY nee NA
6 2017-02-01 12:30:00 ABBY nee NA
7 2017-02-01 13:00:00 ABBY nee NA
8 2017-02-01 13:30:00 ABBY nee NA
9 2017-02-01 14:00:00 ABBY nee NA
10 2017-02-01 14:30:00 ABBY nee NA
# ℹ 10,461,438 more rowsThe code used to generate the targets from NEON data can be found here
4.3.5 Daily target data calculation
To evaluate the models that produce daily flux forecasts, we select only days with at least 24 of 48 half-hours that pass the quality control flags. For these days, we average the half-hours and convert carbon to daily units (gC/m2/day). The daily data table has the following columns.
datetime: YYYY-MM-DD (the day is determined using UTC time)
site_id: NEON site code (e.g., BART)
variable:nee(g C m-2 day-1) orle(W m-2)observation: value for variable
Here is the download link and format of the terrestrial_daily target file
readr::read_csv("https://data.ecoforecast.org/neon4cast-targets/terrestrial_daily/terrestrial_daily-targets.csv.gz", guess_max = 1e6) |>
na.omit()# A tibble: 101,600 × 4
datetime site_id variable observation
<date> <chr> <chr> <dbl>
1 2017-02-02 BART le 2.71
2 2017-02-02 BART nee 0.580
3 2017-02-03 BART le 8.47
4 2017-02-03 BART nee 0.687
5 2017-02-04 BART le 6.64
6 2017-02-04 BART nee 0.693
7 2017-02-05 BART le 3.53
8 2017-02-05 BART nee 0.912
9 2017-02-06 BART le 11.5
10 2017-02-06 BART nee 0.965
# ℹ 101,590 more rowsThe code used to generate the targets from NEON data can be found here
4.4 Timeline
Forecasts for a minimum of 30 days can be submitted daily by 6 pm ET any day. New forecasts can be submitted daily as new weather forecasts and observations (e.g., NEE) become available.
Daily submissions are allowed and encouraged as new observations and weather forecasts become available, therefore the automation of forecast generation may be ideal. There are many ways to automate scripts that are written to download observations and meteorology drivers, generate forecasts, and submit forecasts. Two tools that many have used are cron jobs (see the R package cronR) that execute tasks at user specifics times and github actions. See more in Chapter 12
Cron jobs work on unix and mac systems. An example of a script that executes a cron job using R can be found here.
4.5 Flux data latency
NEON data officially releases the flux data on their data portal and API in monthly data packages. Data for a given month is scheduled to be released around the 15th of the following month.
NEON is also processing flux data with only a 5 day delay (latency). Any data that has been processed but not included in a released monthly package is available on NEON s3 storage. The list of files that can be downloaded can found here.
Our targets file is the combination of NEON’s monthly releases and the files on the s3 bucket. As a result, flux data within 5-days of the restart of a forecast are available to inform the forecast.
The reduction of the latency from monthly to 5-days allows this theme to forecast in real-time - a major advancement for this forecasting challenge. Thank you NEON!
4.6 Submissions
The required names for forecasted variables: nee, and le.
The required time unit: date for daily forecast in YYYY-MM-DD format and date-time for 30 minute forecasts in YYYY-MM-DD HH:MM:SS format
Instructions for submitting forecasts are found in Chapter 2
4.7 Meterologic inputs for modeling
Information about forecasted meteorology that is available for you to use when generating your forecasts can be found in Chapter 9
4.8 Useful functions
Functions for validating, evaluating and submitting forecasts can be found in Chapter 10
Functions for downloading and working with the meteorology forecasts can be be found Chapter 9
4.9 Null models
Two null models are automatically generated each day - these are simple baseline models. The persistence null model uses the most recent measurement of nee or le and predicts that the values will be constant in the future. The climatology null model forecasts that the nee or lee will be equal to the historical mean of that day of year. We apply both the persistence and climatology model to the daily fluxes and the climatology to the 30 minute fluxes
Code for the daily persistence null model can be found here
Code for the daily climatology null model can be found here
Code for the 30 minute climatology null model can be found here
4.10 FAQ
Answers to frequency asks questions can be found in Chapter 12
4.11 Design team
George Burba, LI-COR Biosciences
Jamie Cleverly, Terrestrial Ecosystem Research Network (TERN)
Ankur Desai, University of Wisconsin, Madison
Mike Dietze, Boston University
Andy Fox, Joint Center for Satellite Data Assimilation
William Hammond, Oklahoma State University
Danica Lombardozzi, National Center for Atmospheric Research
Quinn Thomas, Virginia Tech
Jody Peters, University of Notre Dame
Alex Young, SUNY - College of Environmental Science & Forestry
4.12 Partners
Data used in the challenge are from the National Ecological Observatory Network (NEON)
Ameriflux is an excellent database of eddy-covariance data, including historical data for some of the four challenge sites.
Terrestrial Ecosystem Research Network (TERN) has been involved in the design of the challenge.
4.13 Site table
| siteID | site name | vegetation type | latitude | longtitude | NEON site URL |
|---|---|---|---|---|---|
| ABBY | Abby Road NEON | Evergreen Forest|Grassland/Herbaceous|Shrub/Scrub | 45.76244 | -122.33032 | https://www.neonscience.org/field-sites/abby |
| BARR | Utqiaġvik NEON | Emergent Herbaceous Wetlands | 71.28241 | -156.61936 | https://www.neonscience.org/field-sites/barr |
| BART | Bartlett Experimental Forest NEON | Deciduous Forest|Evergreen Forest|Mixed Forest | 44.06389 | -71.28737 | https://www.neonscience.org/field-sites/bart |
| BLAN | Blandy Experimental Farm NEON | Deciduous Forest|Pasture/Hay | 39.03370 | -78.04179 | https://www.neonscience.org/field-sites/blan |
| BONA | Caribou-Poker Creeks Research Watershed NEON | Deciduous Forest|Evergreen Forest|Mixed Forest|Woody Wetlands | 65.15401 | -147.50258 | https://www.neonscience.org/field-sites/bona |
| CLBJ | Lyndon B. Johnson National Grassland NEON | Deciduous Forest|Grassland/Herbaceous | 33.40123 | -97.57000 | https://www.neonscience.org/field-sites/clbj |
| CPER | Central Plains Experimental Range NEON | Grassland/Herbaceous | 40.81554 | -104.74559 | https://www.neonscience.org/field-sites/cper |
| DCFS | Dakota Coteau Field Site NEON | Grassland/Herbaceous | 47.16165 | -99.10656 | https://www.neonscience.org/field-sites/dcfs |
| DEJU | Delta Junction NEON | Evergreen Forest|Shrub/Scrub|Woody Wetlands | 63.88112 | -145.75136 | https://www.neonscience.org/field-sites/deju |
| DELA | Dead Lake NEON | Evergreen Forest|Woody Wetlands | 32.54173 | -87.80388 | https://www.neonscience.org/field-sites/dela |
| DSNY | Disney Wilderness Preserve NEON | Pasture/Hay|Woody Wetlands | 28.12505 | -81.43619 | https://www.neonscience.org/field-sites/dsny |
| GRSM | Great Smoky Mountains National Park NEON | Deciduous Forest|Evergreen Forest | 35.68896 | -83.50195 | https://www.neonscience.org/field-sites/grsm |
| GUAN | Guanica Forest NEON | Evergreen Forest | 17.96955 | -66.86870 | https://www.neonscience.org/field-sites/guan |
| HARV | Harvard Forest & Quabbin Watershed NEON | Deciduous Forest|Evergreen Forest|Mixed Forest|Woody Wetlands | 42.53691 | -72.17265 | https://www.neonscience.org/field-sites/harv |
| HEAL | Healy NEON | Dwarf Scrub|Evergreen Forest|Shrub/Scrub | 63.87580 | -149.21335 | https://www.neonscience.org/field-sites/heal |
| JERC | The Jones Center At Ichauway NEON | Cultivated Crops|Deciduous Forest|Evergreen Forest|Mixed Forest | 31.19484 | -84.46862 | https://www.neonscience.org/field-sites/jerc |
| JORN | Jornada Experimental Range NEON | Shrub/Scrub | 32.59069 | -106.84254 | https://www.neonscience.org/field-sites/jorn |
| KONA | Konza Prairie Agroecosystem NEON | Cultivated Crops | 39.11045 | -96.61293 | https://www.neonscience.org/field-sites/kona |
| KONZ | Konza Prairie Biological Station NEON | Deciduous Forest|Grassland/Herbaceous | 39.10077 | -96.56307 | https://www.neonscience.org/field-sites/konz |
| LAJA | Lajas Experimental Station NEON | Cultivated Crops|Grassland/Herbaceous|Pasture/Hay | 18.02126 | -67.07689 | https://www.neonscience.org/field-sites/laja |
| LENO | Lenoir Landing NEON | Deciduous Forest|Woody Wetlands | 31.85386 | -88.16118 | https://www.neonscience.org/field-sites/leno |
| MLBS | Mountain Lake Biological Station NEON | Deciduous Forest | 37.37831 | -80.52485 | https://www.neonscience.org/field-sites/mlbs |
| MOAB | Moab NEON | Evergreen Forest|Shrub/Scrub | 38.24828 | -109.38827 | https://www.neonscience.org/field-sites/moab |
| NIWO | Niwot Ridge NEON | Evergreen Forest|Grassland/Herbaceous | 40.05425 | -105.58237 | https://www.neonscience.org/field-sites/niwo |
| NOGP | Northern Great Plains Research Laboratory NEON | Grassland/Herbaceous | 46.76972 | -100.91535 | https://www.neonscience.org/field-sites/nogp |
| OAES | Marvin Klemme Range Research Station NEON | Grassland/Herbaceous|Shrub/Scrub | 35.41060 | -99.05878 | https://www.neonscience.org/field-sites/oaes |
| ONAQ | Onaqui NEON | Evergreen Forest|Shrub/Scrub | 40.17760 | -112.45245 | https://www.neonscience.org/field-sites/onaq |
| ORNL | Oak Ridge NEON | Deciduous Forest|Evergreen Forest|Pasture/Hay | 35.96413 | -84.28259 | https://www.neonscience.org/field-sites/ornl |
| OSBS | Ordway-Swisher Biological Station NEON | Emergent Herbaceous Wetlands|Evergreen Forest|Woody Wetlands | 29.68928 | -81.99343 | https://www.neonscience.org/field-sites/osbs |
| PUUM | Pu’u Maka’ala Natural Area Reserve NEON | Evergreen Forest | 19.55309 | -155.31731 | https://www.neonscience.org/field-sites/puum |
| RMNP | Rocky Mountains NEON | Evergreen Forest | 40.27590 | -105.54596 | https://www.neonscience.org/field-sites/rmnp |
| SCBI | Smithsonian Conservation Biology Institute NEON | Deciduous Forest|Evergreen Forest|Pasture/Hay | 38.89292 | -78.13949 | https://www.neonscience.org/field-sites/scbi |
| SERC | Smithsonian Environmental Research Center NEON | Cultivated Crops|Deciduous Forest | 38.89013 | -76.56001 | https://www.neonscience.org/field-sites/serc |
| SJER | San Joaquin Experimental Range NEON | Evergreen Forest|Grassland/Herbaceous|Shrub/Scrub | 37.10878 | -119.73228 | https://www.neonscience.org/field-sites/sjer |
| SOAP | Soaproot Saddle NEON | Evergreen Forest|Shrub/Scrub | 37.03337 | -119.26219 | https://www.neonscience.org/field-sites/soap |
| SRER | Santa Rita Experimental Range NEON | Shrub/Scrub | 31.91068 | -110.83549 | https://www.neonscience.org/field-sites/srer |
| STEI | Steigerwaldt-Chequamegon NEON | Deciduous Forest|Mixed Forest|Woody Wetlands | 45.50894 | -89.58637 | https://www.neonscience.org/field-sites/stei |
| STER | North Sterling NEON | Cultivated Crops | 40.46189 | -103.02929 | https://www.neonscience.org/field-sites/ster |
| TALL | Talladega National Forest NEON | Deciduous Forest|Evergreen Forest|Mixed Forest | 32.95047 | -87.39326 | https://www.neonscience.org/field-sites/tall |
| TEAK | Lower Teakettle NEON | Evergreen Forest|Shrub/Scrub | 37.00583 | -119.00602 | https://www.neonscience.org/field-sites/teak |
| TOOL | Toolik Field Station NEON | Dwarf Scrub|Shrub/Scrub | 68.66109 | -149.37047 | https://www.neonscience.org/field-sites/tool |
| TREE | Treehaven NEON | Deciduous Forest|Evergreen Forest|Mixed Forest|Woody Wetlands | 45.49369 | -89.58571 | https://www.neonscience.org/field-sites/tree |
| UKFS | University of Kansas Field Station NEON | Deciduous Forest|Pasture/Hay | 39.04043 | -95.19215 | https://www.neonscience.org/field-sites/ukfs |
| UNDE | University of Notre Dame Environmental Research Center NEON | Deciduous Forest|Mixed Forest|Woody Wetlands | 46.23391 | -89.53725 | https://www.neonscience.org/field-sites/unde |
| WOOD | Chase Lake National Wildlife Refuge NEON | Emergent Herbaceous Wetlands|Grassland/Herbaceous | 47.12820 | -99.24133 | https://www.neonscience.org/field-sites/wood |
| WREF | Wind River Experimental Forest NEON | Evergreen Forest | 45.82049 | -121.95191 | https://www.neonscience.org/field-sites/wref |
| YELL | Yellowstone National Park NEON | Evergreen Forest|Grassland/Herbaceous|Shrub/Scrub | 44.95348 | -110.53914 | https://www.neonscience.org/field-sites/yell |