TY - GEN
T1 - Automated integration of continental-scale observations in near-real time for simulation and analysis of biosphere-atmosphere interactions
AU - Durden, David J.
AU - Metzger, Stefan
AU - Chu, Housen
AU - Collier, Nathan
AU - Davis, Kenneth J.
AU - Desai, Ankur R.
AU - Kumar, Jitendra
AU - Wieder, William R.
AU - Xu, Min
AU - Hoffman, Forrest M.
N1 - Publisher Copyright:
© Springer Nature Switzerland AG 2020.
PY - 2021
Y1 - 2021
N2 - The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.
AB - The National Ecological Observatory Network (NEON) is a continental-scale observatory with sites across the US collecting standardized ecological observations that will operate for multiple decades. To maximize the utility of NEON data, we envision edge computing systems that gather, calibrate, aggregate, and ingest measurements in an integrated fashion. Edge systems will employ machine learning methods to cross-calibrate, gap-fill and provision data in near-real time to the NEON Data Portal and to High Performance Computing (HPC) systems, running ensembles of Earth system models (ESMs) that assimilate the data. For the first time gridded EC data products and response functions promise to offset pervasive observational biases through evaluating, benchmarking, optimizing parameters, and training new machine learning parameterizations within ESMs all at the same model-grid scale. Leveraging open-source software for EC data analysis, we are already building software infrastructure for integration of near-real time data streams into the International Land Model Benchmarking (ILAMB) package for use by the wider research community. We will present a perspective on the design and integration of end-to-end infrastructure for data acquisition, edge computing, HPC simulation, analysis, and validation, where Artificial Intelligence (AI) approaches are used throughout the distributed workflow to improve accuracy and computational performance.
UR - http://www.scopus.com/inward/record.url?scp=85107307710&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85107307710&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-63393-6_14
DO - 10.1007/978-3-030-63393-6_14
M3 - Conference contribution
AN - SCOPUS:85107307710
SN - 9783030633929
T3 - Communications in Computer and Information Science
SP - 204
EP - 225
BT - Driving Scientific and Engineering Discoveries Through the Convergence of HPC, Big Data and AI - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020, Revised Selected Papers
A2 - Nichols, Jeffrey
A2 - Maccabe, Arthur ‘Barney’
A2 - Parete-Koon, Suzanne
A2 - Verastegui, Becky
A2 - Hernandez, Oscar
A2 - Ahearn, Theresa
PB - Springer Science and Business Media Deutschland GmbH
T2 - 17th Smoky Mountains Computational Sciences and Engineering Conference, SMC 2020
Y2 - 26 August 2020 through 28 August 2020
ER -
