Observatoire de Paris-PSL CNRS vopdc cdpp Sorbonne Université cnes Université de Paris LESIA

Home > Task 3 - Infrastructure & Interoperability

Infrastructure

Friday 15 March 2019, by Baptiste Cecconi

The MASER services are built upon international standards and community-based developments in order to ensure the project sustainability. The main pieces of MASER’s infrastructure are described here.

Interfaces

Three type of interfaces are available to reach MASER datasets.

Data Streaming Access

Direct access to data for quick visualization is available through Autoplot, a tool developed by University of Iowa. This tool is used a lot by the community. Data distribution towards Autoplot can be done through several means:

  • direct access CDF files in Autoplot, after downloading the file, or with the access URL of the file.
  • access through a preconfigured Autoplot template (.vap file), that MASER teams prepare and distribute. These templates allow to distribute multi instrument plot very easily (for instance, ground based data from Nançay/NDA/Routine and Wind/Waves space data).
  • Access through a Das2 server, developed by University of Iowa. The PADC team is now involved in the development of this server. The system optimizes the data rate on the network and is capital for remote visualization of high resolution datasets.

Metadata search interface

MASER is making use of the VESPA infrastructure to distribute data product metadata catalogues (CDF files, native format files, event catalogues, .vap files, das2 datasets...). This allows scientists to search for data of interest for their research.

Data Format

The data are distributed in their original production format as well as in one or several standard formats. This choice allows the returning users to process the data with existing tools and pipelines, and facilites access to new users through standards formats and libraries.

Standard Format

The two main distribution formats for low frequency radio data are CDF (for space physics) and FITS (for ground based observations). Preference is given to the CDF format when this choice was possible.

Other formats are also used for catalogues (VOTable, table ASCII), or for more complexe datasets (format HDF5).

When considering high resolution datasets, file based data distribution on the internet is not possible due to the data volume involved. In these cases, conversion to standard format is not recommended.

Metadata

Adopting standard formats also requires adopting metadata standards. The MASER team decided to distribute data products containing metadata compliant with the two main NASA archives:

  • heliophysics data archive: CDF ISTP (International Solar Terrestrial Program) metadata dictionary
  • planetary data archive: CDF PDS4 (Planetary Data System version 4) recommendation.

MASER data products also contain metadata extracted from the VESPA EPNcore v2 dictionary.

Hardware Infrastructure

The MASER services are relying on informatics and computing services of LESIA, PADC and USN for deployment, maintenance and evolution of interfaces and for data storage.