API of Aeroval tools

Documentation of the pyaerocom AeroVal API, for high level web processing tools.

Tools for AeroVal experiment setup

High level analysis setup for AeroVal experiment

class pyaerocom.aeroval.setupclasses.EvalRunOptions(**kwargs)

only_model_maps

If True, process only maps (skip obs evaluation)

class pyaerocom.aeroval.setupclasses.EvalSetup(proj_id: str | None = None, exp_id: str | None = None, **kwargs)

Composite class representing a whole analysis setup

This represents the level at which json I/O happens for configuration setup files.

IGNORE_JSON = ['_aux_funs']

Keys to be ignored when converting to json

property exp_id: str

experiment ID (wrapper to exp_info.exp_id)

Type:

str

static from_json(filepath: str) → EvalSetup

Load configuration from json config file

get_model_entry(model_name)

Get model entry configuration

Since the configuration files for experiments are in json format, they do not allow the storage of executable custom methods for model data reading. Instead, these can be specified in a python module that may be specified via add_methods_file and that contains a dictionary FUNS that maps the method names with the callable methods.

As a result, this means that, by default, custom read methods for individual models in model_config do not contain the callable methods but only the names. This method will take care of handling this and will return a dictionary where potential custom method strings have been converted to the corresponding callable methods.

Parameters:

model_name (str) – name of model

Returns:

Dictionary that specifies the model setup ready for the analysis

Return type:

dict

property json_filename: str

cfg_<proj_id>_<exp_id>.json

Type:

str

Type:

Savename of config file

property proj_id: str

proj ID (wrapper to proj_info.proj_id)

Type:

str

to_json(outdir: str, ignore_nan: bool = True, indent: int = 3) → None

Save configuration as JSON file

Parameters:

• outdir (str) – directory where the config json file is supposed to be stored

• ignore_nan (bool) – set NaNs to Null when writing

• indent (int) – json indentation

class pyaerocom.aeroval.setupclasses.ExperimentInfo(exp_id: str, **kwargs)

class pyaerocom.aeroval.setupclasses.ModelMapsSetup(**kwargs)

class pyaerocom.aeroval.setupclasses.OutputPaths(proj_id: str, exp_id: str, json_basedir: str | None = None, coldata_basedir: str | None = None)

Setup class for output paths of json files and co-located data

This interface generates all paths required for an experiment.

proj_id

project ID

Type:

str

exp_id

experiment ID

Type:

str

json_basedir

Type:

str

class pyaerocom.aeroval.setupclasses.ProjectInfo(proj_id: str)

class pyaerocom.aeroval.setupclasses.StatisticsSetup(**kwargs)

Setup options for statistical calculations

weighted_stats

if True, statistics are calculated using area weights, this is only relevant for gridded / gridded evaluations.

Type:

bool

annual_stats_constrained

if True, then only sites are considered that satisfy a potentially specified annual resampling constraint (see pyaerocom.colocation_auto.ColocationSetup.min_num_obs). E.g. lets say you want to calculate statistics (bias, correlation, etc.) for monthly model / obs data for a given site and year. Lets further say, that there are only 8 valid months of data, and 4 months are missing, so statistics will be calculated for that year based on 8 vs. 8 values. Now if pyaerocom.colocation_auto.ColocationSetup.min_num_obs is specified in way that requires e.g. at least 9 valid months to represent the whole year, then this station will not be considered in case annual_stats_constrained is True, else it will. Defaults to False.

Type:

bool

stats_tseries_base_freq

The statistics Time Series display in AeroVal (under Overall Evaluation) is computed in intervals of a certain frequency, which is specified via TimeSetup.main_freq (defaults to monthly). That is, monthly colocated data is used as a basis to compute the statistics for each month (e.g. if you have 10 sites, then statistics will be computed based on 10 monthly values for each month of the timeseries, 1 value for each site). stats_tseries_base_freq may be specified in case a higher resolution is supposed to be used as a basis to compute the timeseries in the resolution specified by TimeSetup.main_freq (e.g. if daily is specified here, then for the above example 310 values would be used - 31 for each site - to compute the statistics for a given month (in this case, a month with 31 days, obviously).

Type:

str, optional

Parameters:

kwargs – any of the supported attributes, e.g. StatisticsSetup(annual_stats_constrained=True)

class pyaerocom.aeroval.setupclasses.TimeSetup(**kwargs)

get_seasons()

Get list of seasons to be analysed

Returns SEASONS if add_seasons it True, else [ ‘all’] (only whole year).

Returns:

list of season strings for analysis

Return type:

list

class pyaerocom.aeroval.setupclasses.WebDisplaySetup(**kwargs)

Specification of observation datasets

class pyaerocom.aeroval.obsentry.ObsEntry(**kwargs)

Observation configuration for evaluation (dictionary)

Note

Only obs_id and obs_vars are mandatory, the rest is optional.

obs_id

ID of observation network in AeroCom database (e.g. ‘AeronetSunV3Lev2.daily’)

Type:

str

obs_vars

list of pyaerocom variable names that are supposed to be analysed (e.g. [‘od550aer’, ‘ang4487aer’])

Type:

list

obs_ts_type_read

may be specified to explicitly define the reading frequency of the observation data (so far, this does only apply to gridded obsdata such as satellites). For ungridded reading, the frequency may be specified via obs_id, where applicable (e.g. AeronetSunV3Lev2.daily). Can be specified variable specific in form of dictionary.

Type:

str or dict, optional

obs_vert_type

Aerocom vertical code encoded in the model filenames (only AeroCom 3 and later).

Type:

str, optional

obs_aux_requires

information about required datasets / variables for auxiliary variables.

Type:

dict, optional

instr_vert_loc

vertical location code of observation instrument. This is used in the aeroval interface for separating different categories of measurements such as “ground”, “space” or “airborne”.

Type:

str, optional

is_superobs

if True, this observation is a combination of several others which all have to have their own obs config entry.

Type:

bool

only_superobs

this indicates whether this configuration is only to be used as part of a superobs network, and not individually.

Type:

bool

read_opts_ungridded

dictionary that specifies reading constraints for ungridded reading (c.g. pyaerocom.io.ReadUngridded).

Type:

dict, optional

check_add_obs()

Check if this dataset is an auxiliary post dataset

check_cfg()

Check that minimum required attributes are set and okay

get_all_vars() → list

Get list of all variables associated with this entry

Returns:

DESCRIPTION.

Return type:

list

get_vert_code(var)

Get vertical code name for obs / var combination

has_var(var_name)

Check if input variable is defined in entry

Returns:

True if entry has variable available, else False

Return type:

bool

Specification of model datasets

class pyaerocom.aeroval.modelentry.ModelEntry(model_id, **kwargs)

Modeln configuration for evaluation (dictionary)

Note

Only model_id is mandatory, the rest is optional.

model_id

ID of model run in AeroCom database (e.g. ‘ECMWF_CAMS_REAN’)

Type:

str

model_ts_type_read

may be specified to explicitly define the reading frequency of the model data. Not to be confused with ts_type, which specifies the frequency used for colocation. Can be specified variable specific by providing a dictionary.

Type:

str or dict, optional

model_use_vars

dictionary that specifies mapping of model variables. Keys are observation variables, values are strings specifying the corresponding model variable to be used (e.g. model_use_vars=dict(od550aer=’od550csaer’))

Type:

dict

model_add_vars

dictionary that specifies additional model variables. Keys are observation variables, values are lists of strings specifying the corresponding model variables to be used (e.g. model_use_vars=dict(od550aer=[‘od550csaer’, ‘od550so4’]))

Type:

dict

model_rename_vars

key / value pairs specifying new variable names for model variables in the output json files (is applied after co-location).

Type:

dict

model_read_aux

may be used to specify additional computation methods of variables from models. Keys are obs variables, values are dictionaries with keys vars_required (list of required variables for computation of var and fun (method that takes list of read data objects and computes and returns var)

Type:

dict

property aux_funs_required

Boolean specifying whether this entry requires auxiliary variables

get_vars_to_process(obs_vars: list) → tuple

Get lists of obs / mod variables to be processed

Parameters:

obs_vars (list) – list of observation variables

Returns:

• list – list of observation variables (potentially extended from input list)

• list – corresponding model variables which are mapped based on content of model_add_vars and model_use_vars.

Containers for model and observation setup

Collection classes to specify a number of model entries and a number of observation entries for a given AeroVal experiment.

class pyaerocom.aeroval.collections.BaseCollection(*args, **kwargs)

FORBIDDEN_CHARS_KEYS = ['_']

Invalid chars in entry names

MAXLEN_KEYS = 25

maximum length of entry names

abstract get_entry(key) → object

Getter for eval entries

Raises:

KeyError – if input name is not in this collection

keylist(name_or_pattern: str | None = None) → list

Find model names that match input search pattern(s)

Parameters:

name_or_pattern (str, optional) – Name or pattern specifying search string.

Returns:

list of keys in collection that match input requirements. If name_or_pattern is None, all keys will be returned.

Return type:

list

Raises:

KeyError – if no matches can be found

abstract property web_iface_names: list

List of webinterface names for

class pyaerocom.aeroval.collections.ModelCollection(*args, **kwargs)

Dict-like object that represents a collection of model entries

Keys are model names, values are instances of ModelEntry. Values can also be assigned as dict and will automatically be converted into instances of ModelEntry.

Note

Entries must not necessarily be only models but may also be observations. Entries provided in this collection refer to the x-axis in the AeroVal heatmap display and must fulfill the protocol defined by ModelEntry.

get_entry(key) → object

Get model entry configuration

Since the configuration files for experiments are in json format, they do not allow the storage of executable custom methods for model data reading. Instead, these can be specified in a python module that may be specified via add_methods_file and that contains a dictionary FUNS that maps the method names with the callable methods.

As a result, this means that, by default, custom read methods for individual models in model_config do not contain the callable methods but only the names. This method will take care of handling this and will return a dictionary where potential custom method strings have been converted to the corresponding callable methods.

Parameters:

model_name (str) – name of model

Returns:

Dictionary that specifies the model setup ready for the analysis

Return type:

dict

property web_iface_names: list

List of web interface names for each obs entry

Return type:

list

class pyaerocom.aeroval.collections.ObsCollection(*args, **kwargs)

Dict-like object that represents a collection of obs entries

Keys are obs names, values are instances of ObsEntry. Values can also be assigned as dict and will automatically be converted into instances of ObsEntry.

Note

Entries must not necessarily be only observations but may also be models. Entries provided in this collection refer to the y-axis in the AeroVal heatmap display and must fulfill the protocol defined by ObsEntry.

property all_vert_types

List of unique vertical types specified in this collection

get_all_vars() → list

Get unique list of all obs variables from all entries

Returns:

list of variables specified in obs collection

Return type:

list

get_entry(key) → object

Getter for obs entries

Raises:

KeyError – if input name is not in this collection

get_web_iface_name(key)

Get webinterface name for entry

Note

Normally this is the key of the obsentry in obs_config, however, it might be specified explicitly via key web_interface_name in the corresponding value.

Parameters:

key (str) – key of entry.

Returns:

corresponding name

Return type:

str

property web_iface_names: list

List of web interface names for each obs entry

Return type:

list

Processing tools

Experiment processing engine

class pyaerocom.aeroval.experiment_processor.ExperimentProcessor(cfg: EvalSetup)

Processing engine for AeroVal experiment

By default, this class processes one configuration file, represented by EvalSetup. As such, an instance of EvalSetup represents an AeroVal experiment, comprising a list of models, a list of observations (and variables).

For each possible (or defined) model / obs / variable combination, the processing engine will perform spatial and temporal co-location and will store on co-located NetCDF file (e.g. if there are 2 models, 2 observation networks and 2 variables there will be 4 co-located NetCDF files). The co-location is done using pyaerocom.colocation_auto.Colocator.

run(model_name=None, obs_name=None, var_list=None, update_interface=True)

Create colocated data and json files for model / obs combination

Parameters:

• model_name (str or list, optional) – Name or pattern specifying model that is supposed to be analysed. Can also be a list of names or patterns to specify multiple models. If None (default), then all models are run that are part of this experiment.

• obs_name (str, or list, optional) – Like model_name, but for specification(s) of observations that are supposed to be used. If None (default) all observations are used.

• var_list (list, optional) – list variables supposed to be analysed. If None, then all variables available are used. Defaults to None. Can also be str type.

• update_interface (bool) – if true, relevant json files that determine what is displayed online are updated after the run, including the the menu.json file and also, the model info table (minfo.json) file is created and saved in exp_dir.

Returns:

list containing all colocated data objects that have been converted to json files.

Return type:

list

update_interface()

Update aeroval interface

Things done here:

• Update menu file

• Make aeroval info table json (tab informations in interface)

• update and order heatmap file

Model maps processing

class pyaerocom.aeroval.modelmaps_engine.ModelMapsEngine(cfg: EvalSetup)

Engine for processing of model maps

run(**kwargs)

Method that runs the processing based on settings in cfg

Parameters:

• *args – positional arguments.

• **kwargs – Keyword arguments.

Returns:

list of output file paths generated by the engine.

Return type:

list

Processing of super-observation entries

Super observations refer to merged observation datasets to increase the number of stations.

class pyaerocom.aeroval.superobs_engine.SuperObsEngine(cfg: EvalSetup)

Class to handle the processing of combined obs datasets

run(model_name, obs_name, var_list, try_colocate_if_missing=True)

Method that runs the processing based on settings in cfg

Parameters:

• *args – positional arguments.

• **kwargs – Keyword arguments.

Returns:

list of output file paths generated by the engine.

Return type:

list

Low-level base classes for processing engines

class pyaerocom.aeroval._processing_base.DataImporter(cfg: EvalSetup)

Class that supports reading of model and obs data based on an eval config.

Depending on a EvalSetup, reading of model and obs data may have certain constraints (e.g. freq, years, alias variable names, etc.), which are / can be specified flexibly for each model and obs entry in an analysis setup (EvalSetup). Proper handling of these reading constraints and data import settings are handled in the pyaerocom.colocation_auto.Colocator engine, therefore the reading in this class is done via the Colocator engine.

read_model_data(model_name, var_name)

Import model data

Parameters:

• model_name (str) – Name of model in cfg,

• var_name (str) – Name of variable to be read.

Returns:

data – loaded model data.

Return type:

GriddedData

read_ungridded_obsdata(obs_name, var_name)

Import ungridded observation data

Parameters:

• obs_name (str) – Name of observation network in cfg

• var_name (str) – Name of variable to be read.

Returns:

data – loaded obs data.

Return type:

UngriddedData

class pyaerocom.aeroval._processing_base.HasColocator(cfg: EvalSetup)

Config class that also has the ability to co-locate

get_colocator(model_name: str | None = None, obs_name: str | None = None) → Colocator

Instantiate colocation engine

Parameters:

• model_name (str, optional) – name of model. The default is None.

• obs_name (str, optional) – name of obs. The default is None.

Return type:

Colocator

class pyaerocom.aeroval._processing_base.HasConfig(cfg: EvalSetup)

Base class that ensures that evaluation configuration is available

cfg

AeroVal experiment setup

Type:

EvalSetup

exp_output

Manages output for an AeroVal experiment (e.g. path locations).

Type:

ExperimentOutput

class pyaerocom.aeroval._processing_base.ProcessingEngine(cfg: EvalSetup)

Abstract base for classes supposed to do one or more processing tasks

Requirement for a processing class is to inherit attrs from HasConfig and, in addition to that, to have implemented a method :fun:`run` which is running the corresponding processing task and storing all the associated output files, that are read by the frontend.

One example of an implementation is the pyaerocom.aeroval.modelmaps_engine.ModelMapsEngine.

abstract run(*args, **kwargs) → list

Method that runs the processing based on settings in cfg

Parameters:

• *args – positional arguments.

• **kwargs – Keyword arguments.

Returns:

list of output file paths generated by the engine.

Return type:

list

Helpers for processing of auxiliary variables

class pyaerocom.aeroval.aux_io_helpers.ReadAuxHandler(aux_file: str)

Helper class for import of auxiliary function objects

aux_file

path to python module containing function definitions (note: function definitions in module need to be stored in a dictionary called FUNS in the file, where keys are names of the functions and values are callable objects.)

Type:

str

Parameters:

aux_file (str) – input file containing auxiliary functions (details see Attributes section).

import_all()

Import all callable functions in module with their names

Currently, these are expected to be stored in a dictionary called FUNS which should be defined in the python module.

Returns:

function definitions.

Return type:

dict

import_module()

Import aux_file as python module

Uses importlib.import_module() for import. :returns: imported module. :rtype: module

pyaerocom.aeroval.aux_io_helpers.check_aux_info(fun, vars_required, funcs)

Make sure information is correct for computation of auxiliary variables

Parameters:

• fun (str or callable) – name of function or function used to compute auxiliary variable. If str, then arg funcs needs to be provided.

• vars_required (list) – list of required variables for computation of auxiliary variable.

• funcs (dict) – Dictionary with possible functions (values) and names (keys)

Returns:

dict containing callable function object and list of variables required.

Return type:

dict

Conversion of co-located data to json output

class pyaerocom.aeroval.coldatatojson_engine.ColdataToJsonEngine(cfg: EvalSetup)

process_coldata(coldata: ColocatedData)

Creates all json files for one ColocatedData object

Parameters:

coldata (ColocatedData) – colocated data to be processed.

Raises:

• NotImplementedError – DESCRIPTION.

• ValueError – DESCRIPTION.

• AeroValConfigError – DESCRIPTION.

Return type:

None.

run(files)

Convert colocated data files to json

Parameters:

files (list) – list of file paths pointing to colocated data objects to be processed.

Returns:

list of files that have been converted.

Return type:

list

Output management

class pyaerocom.aeroval.experiment_output.ExperimentOutput(cfg)

JSON output for experiment

clean_json_files()

Checks all existing json files and removes outdated data

This may be relevant when updating a model name or similar.

delete_experiment_data(also_coldata=True)

Delete all data associated with a certain experiment

Note

This simply deletes the experiment directory with all the json files and, if also_coldata is True, also the associated co-located data objects.

Parameters:

• base_dir (str, optional) – basic output direcory (containing subdirs of all projects)

• proj_name (str, optional) – name of project, if None, then this project is used

• exp_name (str, optional) – name experiment, if None, then this project is used

• also_coldata (bool) – if True and if output directory for colocated data is default and specific for input experiment ID, then also all associated colocated NetCDF files are deleted. Defaults to True.

property exp_dir

Experiment directory

property exp_id

Experiment ID

get_model_order_menu() → list

Order of models in menu

Note

Returns empty list if no specific order is to be used in which case the models will be alphabetically ordered

get_obs_order_menu() → list

Order of observation entries in menu

property menu_file

json file containing region specifications

property out_dirs_json: dict

json output directories (dict)

property regions_file

json file containing region specifications

property results_available

True if results are available for this experiment, else False

Type:

bool

property statistics_file

json file containing region specifications

update_interface() → None

Update web interface

Steps:

1. Check if results are available, and if so:

2. Add entry for this experiment in experiments.json

3. Create/update ranges.json file in experiment directory

4. Update menu.json against available output and evaluation setup

5. Synchronise content of heatmap json files with menu

6. Create/update file statistics.json in experiment directory

7. Copy json version of EvalSetup into experiment directory

Return type:

None

update_menu()

Update menu

The menu.json file is created based on the available json map files in the map directory of an experiment.

Parameters:

• menu_file (str) – path to json menu file

• delete_mode (bool) – if True, then no attempts are being made to find json files for the experiment specified in config.

property var_ranges_file

json file containing region specifications

class pyaerocom.aeroval.experiment_output.ProjectOutput(proj_id: str, json_basedir: str)

JSON output for project

property available_experiments

List of available experiments

property experiments_file

json file containing region specifications

property proj_dir

Project directory

reorder_experiments(exp_order=None)

Reorder experiment order in evaluation interface

Puts experiment list into order as specified by exp_order, all remaining experiments are sorted alphabetically.

Parameters:

exp_order (list, optional) – desired experiment order, if None, then alphabetical order is used.

Global settings

Global defaults

pyaerocom.aeroval.glob_defaults.statistics_defaults = {'R': {'colmap': 'RdYlGn', 'decimals': 2, 'forecast': True, 'longname': 'Correlation Coefficient', 'name': 'R', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1], 'unit': '1'}, 'R_spearman': {'colmap': 'RdYlGn', 'decimals': 2, 'longname': 'R Spearman Correlation', 'name': 'R Spearman', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1], 'unit': '1'}, 'data_mean': {'colmap': 'coolwarm', 'decimals': 2, 'longname': 'Model Mean', 'name': 'Mean-Mod', 'scale': None, 'unit': '1'}, 'fge': {'colmap': 'reverseColmap(RdYlGn)', 'decimals': 2, 'forecast': True, 'longname': 'Fractional Gross Error', 'name': 'FGE', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2], 'unit': '1'}, 'mab': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Mean Absolute Bias', 'name': 'MAB', 'scale': [0, 0.025, 0.05, 0.075, 0.1, 0.125, 0.15], 'unit': 'var'}, 'mb': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Mean Bias', 'name': 'Mean Bias', 'scale': [-0.15, -0.1, -0.05, 0, 0.05, 0.1, 0.15], 'unit': 'var'}, 'mnmb': {'colmap': 'bwr', 'decimals': 1, 'forecast': True, 'longname': 'Modified Normalized Mean Bias', 'name': 'MNMB', 'scale': [-100, -75, -50, -25, 0, 25, 50, 75, 100], 'unit': '%'}, 'nmb': {'colmap': 'bwr', 'decimals': 1, 'forecast': True, 'longname': 'Normalized Mean Bias', 'name': 'NMB', 'scale': [-100, -75, -50, -25, 0, 25, 50, 75, 100], 'unit': '%'}, 'nrms': {'colmap': 'Reds', 'decimals': 1, 'longname': 'Normalized Root Mean Square Error', 'name': 'NRMSE', 'scale': [0, 25, 50, 75, 100, 125, 150, 175, 200], 'unit': '%'}, 'num_coords_with_data': {'colmap': None, 'decimals': 0, 'longname': 'Number of Stations with data', 'name': 'Nb. Stations', 'overall_only': True, 'scale': None, 'unit': '1'}, 'num_valid': {'colmap': None, 'decimals': 0, 'longname': 'Number of Valid Observations', 'name': 'Nb. Obs', 'overall_only': True, 'scale': None, 'unit': '1'}, 'refdata_mean': {'colmap': 'coolwarm', 'decimals': 2, 'longname': 'Observation Mean', 'name': 'Mean-Obs', 'scale': None, 'unit': '1'}, 'rms': {'colmap': 'coolwarm', 'decimals': 2, 'forecast': True, 'longname': 'Root Mean Square Error', 'name': 'RMSE', 'scale': None, 'unit': '1'}}

Default information for statistical parameters

pyaerocom.aeroval.glob_defaults.statistics_trend = {'mod_trend': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Modelled Trends', 'name': 'Mod-Trends', 'scale': [-10, -7.5, -5.0, -2.5, 0, 2.5, 5.0, 7.5, 10.0], 'unit': '%/yr'}, 'obs/mod_trend': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Trends', 'name': 'Obs/Mod-Trends', 'scale': [-10.0, -7.5, -5.0, -2.5, 0, 2.5, 5.0, 7.5, 10.0], 'unit': '%/yr'}, 'obs_trend': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Observed Trends', 'name': 'Obs-Trends', 'scale': [-10.0, -7.5, -5.0, -2.5, 0, 2.5, 5.0, 7.5, 10.0], 'unit': '%/yr'}}

Default information about trend display

pyaerocom.aeroval.glob_defaults.var_ranges_defaults = {'abs550aer': {'colmap': 'coolwarm', 'scale': [0, 0.0125, 0.025, 0.0375, 0.05, 0.0625, 0.075, 0.0875, 0.1]}, 'absc550aer': {'colmap': 'coolwarm', 'scale': [0, 12.5, 25, 37.5, 50, 62.5, 75, 87.5, 100]}, 'ang4487aer': {'colmap': 'coolwarm', 'scale': [-0.2, -0.1, 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0]}, 'backscatter': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'concCecpm25': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'concCocpm25': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]}, 'concNhno3': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5]}, 'concNnh3': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]}, 'concNnh4': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0]}, 'concNno3pm10': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'concNno3pm25': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'concNtnh': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]}, 'concNtno3': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0]}, 'concco': {'colmap': 'coolwarm', 'scale': [100.0, 125.0, 150.0, 175.0, 200.0, 225.0, 250.0, 275.0, 300.0]}, 'concdust': {'colmap': 'coolwarm', 'scale': [0.0, 10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]}, 'concno2': {'colmap': 'coolwarm', 'scale': [0, 10, 20, 30, 40, 50, 60, 70, 80]}, 'conco3': {'colmap': 'coolwarm', 'scale': [0, 15, 30, 45, 60, 75, 90, 105, 120]}, 'concpm10': {'colmap': 'coolwarm', 'scale': [0, 10, 20, 30, 40, 50, 60, 70, 80]}, 'concpm25': {'colmap': 'coolwarm', 'scale': [0, 5, 10, 15, 20, 25, 30, 35, 40, 45]}, 'concso2': {'colmap': 'coolwarm', 'scale': [0, 0.75, 1.5, 2.25, 3.0, 3.75, 4.5, 5.25, 6.0, 6.75, 7.5, 8.25]}, 'concso4': {'colmap': 'coolwarm', 'scale': [0, 0.75, 1.5, 2.25, 3.0, 3.75, 4.5, 5.25, 6.0, 6.75, 7.5, 8.25]}, 'concsspm10': {'colmap': 'coolwarm', 'scale': [0, 0.75, 1.5, 2.25, 3.0, 3.75, 4.5, 5.25, 6.0, 6.75, 7.5, 8.25]}, 'concsspm25': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0]}, 'default': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'depdust': {'colmap': 'coolwarm', 'scale': [0.0, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, 100.0, 200.0, 500.0, 1000.0]}, 'depoxn': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'depoxnf': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'depoxs': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'depoxsf': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'deprdn': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'deprdnf': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'drydust': {'colmap': 'coolwarm', 'scale': [0.0, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, 100.0, 200.0, 500.0, 1000.0]}, 'dryoxn': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'dryoxs': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'dryrdn': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'extinction': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'od550aer': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'od550dust': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'od550gt1aer': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'od550lt1aer': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'prmm': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'proxydryhno3': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryhono': {'colmap': 'coolwarm', 'scale': [0.001, 0.002, 0.003, 0.004, 0.005, 0.006]}, 'proxydryn2o5': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'proxydrynh3': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}, 'proxydrynh4': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryno2': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryno3c': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'proxydryno3f': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'proxydryo3': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1, 15, 20, 25, 0.3, 40, 50]}, 'proxydryoxn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}, 'proxydryoxs': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydrypm10': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'proxydrypm25': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'proxydryrdn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}, 'proxydryso2': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryso4': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'scatc550dryaer': {'colmap': 'coolwarm', 'scale': [0, 12.5, 25, 37.5, 50, 62.5, 75, 87.5, 100]}, 'ts': {'colmap': 'coolwarm', 'scale': [265, 270, 275, 280, 285, 290, 300, 305, 310, 315, 320]}, 'vmrch4': {'colmap': 'coolwarm', 'scale': [1700, 1750, 1800, 1850, 1900, 1950, 2000, 2050, 2100, 2150, 2200]}, 'vmrco': {'colmap': 'coolwarm', 'scale': [100.0, 125.0, 150.0, 175.0, 200.0, 225.0, 250.0, 275.0, 300.0]}, 'vmrco2': {'colmap': 'coolwarm', 'scale': [400.0, 405.0, 410.0, 415.0, 420.0, 425.0, 430.0, 435.0, 440.0, 445.0, 450.0]}, 'vmrno2': {'colmap': 'coolwarm', 'scale': [0, 5, 10, 15, 20, 25, 30, 35, 40]}, 'vmro3': {'colmap': 'coolwarm', 'scale': [20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70]}, 'vmro3max': {'colmap': 'coolwarm', 'scale': [0, 7.5, 15, 22.5, 30, 37.5, 45, 52.5, 60]}, 'wetdust': {'colmap': 'coolwarm', 'scale': [0.0, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, 100.0, 200.0, 500.0, 1000.0]}, 'wetna': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxnf': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxs': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxsf': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetrdn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetrdnf': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}}

Default variable ranges for web display

Frontend variable naming conventions

class pyaerocom.aeroval.varinfo_web.VarinfoWeb(var_name: str, cmap: str | None = None, cmap_bins: list | None = None, vmin: float | None = None, vmax: float | None = None)

Additional variable information relevant for AeroVal web output

var_name

Name of variable (AeroCom name, not web display name)

Type:

str

cmap_bins

Value bins for web display

Type:

list

cmap

name of colormap for web display

Type:

str

Parameters:

• var_name (str) – Name of variable (AeroCom name, not web display name)

• cmap (str, optional) – name of colormap for web display. If None, the colormap associated with the input variable is used (via pyaerocom.variable.Variable.get_cmap()). Defaults to None.

• cmap_bins (list, optional) – Value bins for web display. If None, then they are inferred from input vmin and vmax, or, if the latter are also None, from attrs pyaerocom.variable.Variable.minimum and pyaerocom.variable.Variable.maximum. If the latter are not defined an AttributeError will be thrown on initialisation.

• vmin (float, optional) – lower end of range

• vmax (float, optional) – upper end of range

autofill_missing(vmin: float | None = None, vmax: float | None = None) → None

Autofill missing attributes related to cmap bins and cmap

Parameters:

• vmin (float, optional) – lower end of range

• vmax (float, optional) – upper end of range

Return type:

None

static from_dict(dict)

Instantiate from dictionary

Parameters:

dict (dict) – settings

Returns:

instantiated instance

Return type:

VarinfoWeb

to_dict()

Convert to dictionary

Return type:

dict

property vmax: float

Upper end of range

property vmin: float

Lower end of range

High-level utility functions

pyaerocom.aeroval.utils.compute_model_average_and_diversity(cfg, var_name, model_names=None, ts_type=None, lat_res_deg=2, lon_res_deg=3, data_id=None, avg_how=None, extract_surface=True, ignore_models=None, comment=None, model_use_vars=None)

Compute median or mean model based on input models

Note

• BETA version that will likely undergo revisions.

• Time selection currently not properly handled

Parameters:

• cfg (AerocomEvaluation) – analysis instance

• var_name (str) – name of variable

• model_names (list, optional) – list of model names. If None, all entries in input engine are used.

• ts_type (str, optional) – output freq. Defaults to monthly.

• lat_res_deg (int, optional) – output latitude resolution, defaults to 2 degrees.

• lon_res_deg (int, optional) – output longitude resolution, defaults to 3 degrees.

• data_id (str, optional) – output data_id of ensemble model.

• avg_how (str, optional) – how to compute averages (choose from mean or median), defaults to “median”.

• extract_surface (bool) – if True (and if data contains model levels), surface level is extracted

• ignore_models (list, optional) – list of models to be ignored

• comment (str, optional) – comment string added to metadata of output data objects.

• model_use_vars (dict, optional) – model variables to be used.

Returns:

• GriddedData – ensemble model for input variable computed averaged using median or mean (input avg_how). Default is median.

• GriddedData – corresponding diversity field, if avg_how is “mean”, then computed using definition from Textor et al., 2006 (ACP) DOI: 10.5194/acp-6-1777-2006. If avg_how is “median” then interquartile range is used (Q3-Q1)/Q2

• GriddedData or None – Q1 field (only output if avg_how is median)

• GriddedData or None – Q3 field (only output if avg_how is median)

• GriddedData or None – standard deviation field (only output if avg_how is mean)

pyaerocom.aeroval.utils.make_config_template(proj_id: str, exp_id: str) → EvalSetup

Make a template for an AeroVal evaluation setup

Parameters:

• proj_id (str) – ID of project.

• exp_id (str) – ID of experiment

Returns:

template evaluation setup (all defaults are set) that can be used to add model and obs entries, and modified to meet the purposes.

Return type:

EvalSetup

Helper modules

General helper functions

pyaerocom.aeroval.helpers.check_var_ranges_avail(model_data, var_name)

Check if lower and upper variable ranges are available for input variable

Parameters:

• model_data (GriddedData) – modeldata containing variable data

• var_name (str) – variable name to be checked (must be the same as model data AeroCom variable name).

Raises:

ValueError – if ranges for input variable are not defined and if input model data corresponds to a different variable than the input variable name.

Return type:

None

Helpers for coldat2json conversion

Helpers for conversion of ColocatedData to JSON files for web interface.

pyaerocom.aeroval.coldatatojson_helpers.add_profile_entry_json(profile_file: str, data: ColocatedData, profile_viz: dict, periods: list[str], seasons: list[str])

Analogous to _add_heatmap_entry_json for profile data. Every time this function is called it checks to see if the profile_file exists. If so, it reads it, if not it makes a new one. This is because one can not add to json files and so everytime we want to add entries for profile layers we must read in the old file, add the entries, and write a new file.

Parameters:

• profile_file (str) – Name of profile_file

• data (ColocatedData) – For this vertical layer

• profile_viz (dict) – Output of process_profile_data()

• periods (list[str]) – periods to compute over (years)

• seasons (list[str]) – seasons to compute over (e.g., All, DJF, etc.)

pyaerocom.aeroval.coldatatojson_helpers.get_json_mapname(obs_name, var_name_web, model_name, model_var, vert_code, period)

Get name base name of json file

pyaerocom.aeroval.coldatatojson_helpers.get_stationfile_name(station_name, obs_name, var_name_web, vert_code)

Get name of station timeseries file

pyaerocom.aeroval.coldatatojson_helpers.process_profile_data_for_regions(data: ColocatedData, region_id: str, use_country: bool, periods: list[str], seasons: list[str]) → dict

This method populates the json files in data/profiles which are use for visualization. Analogous to _process_map_and_scat for profile data. Each json file corresponds to a region or station, obs network, and variable. Inside the json, it is broken up by model. Each model has a key for “z” (the vertical dimension), “obs”, and “mod” Each “obs” and “mod” is broken up by period.

Parameters:

• data (ColocatedData) – ColocatedData object for this layer

• region_id (str) – Spatial subset to compute the mean profiles over

• station_name (str) – Station to compute mean profiles over for period

• use_country (boolean) – Passed to filter_region().

• periods (str) – Year part of the temporal range to average over

• seasons (str) – Sesonal part of the temporal range to average over

Returns:

Dictionary to write to json

Return type:

output (dict)

pyaerocom.aeroval.coldatatojson_helpers.process_profile_data_for_stations(data: ColocatedData, station_name: str, use_country: bool, periods: list[str], seasons: list[str]) → dict

This method populates the json files in data/profiles which are use for visualization. Analogous to _process_map_and_scat for profile data. Each json file corresponds to a region, obs network, and variable. Inside the json, it is broken up by model. Each model has a key for “z” (the vertical dimension), “obs”, and “mod” Each “obs” and “mod” is broken up by period.

Parameters:

• data (ColocatedData) – ColocatedData object for this layer

• region_id (str) – Spatial subset to compute the mean profiles over

• station_name (str) – Station to compute mean profiles over for period

• use_country (boolean) – Passed to filter_region().

• periods (str) – Year part of the temporal range to average over

• seasons (str) – Sesonal part of the temporal range to average over

Returns:

Dictionary to write to json

Return type:

output (dict)

pyaerocom.aeroval.coldatatojson_helpers.update_regions_json(region_defs, regions_json)

Check current regions.json for experiment and update if needed

Parameters:

• region_defs (dict) – keys are names of region (not IDs!) values define rectangular borders

• regions_json (str) – regions.json file (if it does not exist it will be created).

Returns:

current content of updated regions.json

Return type:

dict

Model maps helper functions

pyaerocom.aeroval.modelmaps_helpers.calc_contour_json(data, cmap, cmap_bins)

Convert gridded data into contours for json output

Parameters:

• data (GriddedData) – input data

• cmap (str) – colormap of output

• cmap_bins (list) – list containing the bins to which the values are mapped.

Returns:

dictionary containing contour data

Return type:

dict

pyaerocom.aeroval.modelmaps_helpers.griddeddata_to_jsondict(data, lat_res_deg=5, lon_res_deg=5)

Convert gridded data to json dictionary

Parameters:

• data (GriddedData) – input data to be converted

• lat_res_deg (int) – output latitude resolution in decimal degrees

• lon_res_deg (int) – output longitude resolution in decimal degrees

Returns:

data dictionary for json output (keys are metadata and data).

Return type:

dict