ICOS improved data lifecycle

11676/pKMtoSIKHkmS7D_z7xC9GFQG (link)

ICOS improved data lifecycle.pdf

ICOS provides long term, high quality observations that follow (and cooperatively set) the global standards for the best possible quality data on the atmospheric composition for greenhouse gases (GHG), greenhouse gas exchange fluxes measured by eddy covariance and CO2 partial pressure at water surfaces. The ICOS observational data feeds into a wide area of science that covers for example plant physiology, agriculture, biology, ecology, energy & fuels, forestry, hydrology, (micro)meteorology, environmental, oceanography, geochemistry, physical geography, remote sensing, earth-, climate-, soil- science and combinations of these in multi-disciplinary projects. As ICOS is committed to provide all data and methods in an open and transparent way as free data, a dedicated system is needed to secure the long term archiving and availability of the data together with the descriptive metadata that belongs to the data and is needed to find, identify, understand and properly use the data, also in the far future, following the FAIR data principles. An added requirement is that the full data lifecycle should be completely reproducible to enable full trust in the observations and the derived data products. In this report we we define and describe the implemention of a comprehensive unified metadata flow from Thematic Centres to the Carbon Portal. The design criteria of this system were to integrate as much as possible the operational (legacy) database systems at the TCs with the data portal, thereby preserving the investments in the robust and proven QA/QC and database systems at the TCs and combining these with the benefits of a linked open data system with connected data licence check, usage tracking and dynamic machine operable data and metadata based on a versioned RDF triple store. Also we developed a connected DOI minting system, implemented the generation of data collections and a linked system for versioning of the data, all connected to the ontology driven single point of ingestion, optimised for machine to machine communication. This has been used incrementally in full operational mode over the last years and is now in place and used by all ICOS domains for all data streams, from raw data through near-real-time to final quality controlled data, and by the external users that provide elaborated products. The licence check and data usage tracking has been implemented in a completely unobtrusive way and is flexible enough to be started to interoperate with major data portals like those of FLUXNET, NEON, SOCAT and WMO WDCGG. The use of DOIs increases the exposure of the ICOS data to global and European data portals like the future EOSC portal and current OpenAIRE portal and Google Dataset Search. The ICOS data is already finding it way to many users and the growing length of the ICOS timeseries in all domains and the interoperation with the global portals this data use of ICOS data can now grow further optimally.

Harry Lankreijer , Alex Vermeulen , Benjamin Pfeil , Leo Rivier , Dario Papale , Steve Jones , Maggie Hellström , Eija Juurola , Lynn Hazan , Claudio D'Onofrio

ICOS CCBY4 Data Licence

Lankreijer, H., Vermeulen, A., Pfeil, B., Rivier, L., Papale, D., Jones, S., Hellström, M., Juurola, E., Hazan, L., D'Onofrio, C. ICOS RI, 2022. ICOS improved data lifecycle, https://hdl.handle.net/11676/pKMtoSIKHkmS7D_z7xC9GFQG

BibTex

@misc{https://hdl.handle.net/11676/pKMtoSIKHkmS7D_z7xC9GFQG,
  author={Lankreijer, Harry and Vermeulen, Alex and Pfeil, Benjamin and Rivier, Leo and Papale, Dario and Jones, Steve and Hellström, Maggie and Juurola, Eija and Hazan, Lynn and D'Onofrio, Claudio},
  title={ICOS improved data lifecycle},
  year={2022},
  url={https://hdl.handle.net/11676/pKMtoSIKHkmS7D_z7xC9GFQG},
  publisher={Carbon Portal},
  copyright={http://meta.icos-cp.eu/ontologies/cpmeta/icosLicence},
  pid={11676/pKMtoSIKHkmS7D_z7xC9GFQG}
}

RIS

TY - DATA
T1 - ICOS improved data lifecycle
ID - 11676/pKMtoSIKHkmS7D_z7xC9GFQG
PY - 2022
UR - https://hdl.handle.net/11676/pKMtoSIKHkmS7D_z7xC9GFQG
PB - Carbon Portal
AU - Lankreijer, Harry
AU - Vermeulen, Alex
AU - Pfeil, Benjamin
AU - Rivier, Leo
AU - Papale, Dario
AU - Jones, Steve
AU - Hellström, Maggie
AU - Juurola, Eija
AU - Hazan, Lynn
AU - D'Onofrio, Claudio
ER - 

4 MB (4140164 bytes)

a4a32da1220a1e4992ec3ff3ef10bd1854064fce586665177a228b6d63dc98b2

pKMtoSIKHkmS7D/z7xC9GFQGT85YZmUXeiKLbWPcmLI

Carbon Portal

2022-02-09 17:03:24

2022-02-09 17:03:21

25