6 Participation

6.1 Participation guidance

6.1.1 How to participate

Participation requires that one contact person from each team:

  1. Complete a REGISTRATION for each forecast theme you are participating in and each model you are contributing within a theme
  2. The contact person agrees to the participation agreement below on behalf of the team
  3. Submit forecast netCDF or csv file(s)
  4. Provide the metadata xml file documenting the forecast

One contact person should register on behalf of their team. That contact person will be asked to provide the group members’ names, emails, and affiliations so that everyone in the group can receive an invitation to join the Challenge theme Slack channel and access group resources.

Teams are allowed and encouraged to join the challenge after the start date of each Challenge theme because there are multiple deadlines to submit forecasts. However, only forecasts submitted by each submission deadline will be officially scored.

6.1.2 Teams

Teams can be individuals or groups. They can represent institutions or organizations. You will have 25 characters for a team name (e.g., “EFI Null Model”) and 10 characters for the team name ID (no spaces allowed; e.g., “EFI_Null”).

The registration includes team categories (e.g., undergraduate only, graduate only, multi-institution, etc).

If your team wants to submit multiple forecasts, please register a different team name for each model . If there are two different time steps in a challenge theme (e.g., the terrestrial carbon flux theme has a 30-minute and 1-day option), register each as separate teams.

6.1.3 Slack and GitHub Communication

Once your team is registered, everyone listed will receive an email with an invitation to join the EFI Slack group and the #neon4cast-theme channel. You will also receive details about the GitHub repository associated with the theme your team is registered for.

We strongly encourage participants to use the Challenge theme Slack channels to ask questions, discuss ideas and challenges, and share resources. Overall, we strongly encourage a collegial approach to the Challenge – this is a friendly competition to move the field forward and bring more people into the community, not a cutthroat competition to win by denying other teams useful information.

GitHub repositories for each Challenge theme will be available with helper code and an example workflow (null models). We encourage teams to contribute code to these repositories (via Pull Request) if they develop additional helper code. This is especially important if an individual or group is going to add additional data constraints to their forecast. Remember, the use of data external to NEON is allowed and encouraged so long as it is publicly available and other teams are notified about it. Also, while most anything could be used to calibrate parameters and constrain initial conditions, only other forecasts (e.g. weather) can be used as drivers/covariates during the actual forecast period.

6.1.4 Archiving models

Teams are highly encouraged to publicly archive the code they are using for their forecast models and workflows. Information about where models are archived would be included in your metadata XML.

Teams are also encouraged to use Docker or Singularity to containerize their models & workflows. EFI conventions for containerizing forecasts are still being developed, but our aim (particularly in later years of the forecast challenge) is to be able to provide shared cyberinfrastructure that makes it easier for teams to automate containerized forecasts. Containers will also facilitate Challenge themes interested in performing post-hoc analyses, such as uncertainty quantification and sensitivity analysis.

6.1.5 Computational resources

We are currently working with CyVerse for access to computational resources for teams that require resources not available through home institutions. We will update with more details as they become available.

6.2 Participation agreement

All participants agree to have their forecast posted in real-time on the NEON Ecological Forecast Challenge Dashboard and potentially published in a scientific journal. The manuscripts describing the accuracy of forecasts across teams will be coordinated by the Ecological Forecasting Initiative Research Coordination Network and authorship will be extended to members of each team with an opt-in policy.

If a publication is generated by a forecast team, we ask that the manuscript acknowledge the Ecological Forecasting Initiative Research Coordination Network and its support from the National Science Foundation (DEB-1926388).

6.3 NEON Data Use

NEON data products, software, and derivatives thereof are freely available for use when accompanied by appropriate disclaimers, acknowledgments, and data citations, defined in the NEON data use policy.

The video below explains how to download and utilize NEON data for the Challenge. The video was recorded for the 2021 Early Career Annual Meeting

6.4 Additional data options

Individuals and groups may create forecasts that use other publicly available data in addition to the NEON data, so long as other teams participating in the challenge are notified about the existence of the data via the Challenge theme’s Slack channel. Teams are encouraged to make available the code they are using to access, download, and process any additional data constraints they are using, ideally via a pull request to each Challenge Github repo.

When considering the use of data in forecasts it is important to distinguish data that are being used as drivers/covariates during each forecast from data being used to constrain model structure, parameters, initial conditions, and error distributions. While the latency of NEON data requires that some of our forecast will be (fully or partly) hindcasts, all forecasts should be run as if they are true forecasts – you cannot use any observed data as a driver/covariate or constraint during the forecast period itself as that info would not have been available at the forecast start date. For example, if you find that a particular variable is a useful covariate during the model development & calibration period (e.g. soil temperature) then you would need to find or make a forecast of that variable if you want to use it as a covariate. Teams using meteorological covariates should use the shared meteorological driver data provided by EFI (see Shared Forecast Drivers).

As an example of potentially useful external data, each NEON site has subsets of various remote sensing products that are hosted on the ORNL DAAC (ORNL DAAC subsets). These include:

MODIS collection 6: LAI, FPAR, burned area, surface reflectance, land surface temperature, vegetation indices (NDVI, EVI), modeled ET, GPP, NPP.

VIIRS collection 1: surface reflectance, vegetation indices, LAI, FPAR, land surface temperature,

SMAP: modeled NEE, GPP, Rh, SOC

Daymet: daily surface weather data

The video below demonstrates how to access meteorological covariate data for the Challenge. The video was recorded for the 2021 Early Career Annual Meeting