API of Aeroval tools
Documentation of the pyaerocom AeroVal API, for high level web processing tools.
Tools for AeroVal experiment setup
High level analysis setup for AeroVal experiment
- class pyaerocom.aeroval.setupclasses.EvalRunOptions(**kwargs)[source]
- only_model_maps
If True, process only maps (skip obs evaluation)
- class pyaerocom.aeroval.setupclasses.EvalSetup(proj_id: str | None = None, exp_id: str | None = None, **kwargs)[source]
Composite class representing a whole analysis setup
This represents the level at which json I/O happens for configuration setup files.
- IGNORE_JSON = ['_aux_funs']
Keys to be ignored when converting to json
- get_model_entry(model_name)[source]
Get model entry configuration
Since the configuration files for experiments are in json format, they do not allow the storage of executable custom methods for model data reading. Instead, these can be specified in a python module that may be specified via
add_methods_file
and that contains a dictionary FUNS that maps the method names with the callable methods.As a result, this means that, by default, custom read methods for individual models in
model_config
do not contain the callable methods but only the names. This method will take care of handling this and will return a dictionary where potential custom method strings have been converted to the corresponding callable methods.
- class pyaerocom.aeroval.setupclasses.OutputPaths(proj_id: str, exp_id: str, json_basedir: str | None = None, coldata_basedir: str | None = None)[source]
Setup class for output paths of json files and co-located data
This interface generates all paths required for an experiment.
- class pyaerocom.aeroval.setupclasses.StatisticsSetup(**kwargs)[source]
Setup options for statistical calculations
- weighted_stats
if True, statistics are calculated using area weights, this is only relevant for gridded / gridded evaluations.
- Type:
- annual_stats_constrained
if True, then only sites are considered that satisfy a potentially specified annual resampling constraint (see
pyaerocom.colocation_auto.ColocationSetup.min_num_obs
). E.g. lets say you want to calculate statistics (bias, correlation, etc.) for monthly model / obs data for a given site and year. Lets further say, that there are only 8 valid months of data, and 4 months are missing, so statistics will be calculated for that year based on 8 vs. 8 values. Now ifpyaerocom.colocation_auto.ColocationSetup.min_num_obs
is specified in way that requires e.g. at least 9 valid months to represent the whole year, then this station will not be considered in case annual_stats_constrained is True, else it will. Defaults to False.- Type:
- stats_tseries_base_freq
The statistics Time Series display in AeroVal (under Overall Evaluation) is computed in intervals of a certain frequency, which is specified via
TimeSetup.main_freq
(defaults to monthly). That is, monthly colocated data is used as a basis to compute the statistics for each month (e.g. if you have 10 sites, then statistics will be computed based on 10 monthly values for each month of the timeseries, 1 value for each site). stats_tseries_base_freq may be specified in case a higher resolution is supposed to be used as a basis to compute the timeseries in the resolution specified byTimeSetup.main_freq
(e.g. if daily is specified here, then for the above example 310 values would be used - 31 for each site - to compute the statistics for a given month (in this case, a month with 31 days, obviously).- Type:
str, optional
- Parameters:
kwargs – any of the supported attributes, e.g. StatisticsSetup(annual_stats_constrained=True)
Specification of observation datasets
- class pyaerocom.aeroval.obsentry.ObsEntry(**kwargs)[source]
Observation configuration for evaluation (dictionary)
Note
Only
obs_id
and obs_vars are mandatory, the rest is optional.- obs_vars
list of pyaerocom variable names that are supposed to be analysed (e.g. [‘od550aer’, ‘ang4487aer’])
- Type:
- obs_ts_type_read
may be specified to explicitly define the reading frequency of the observation data (so far, this does only apply to gridded obsdata such as satellites). For ungridded reading, the frequency may be specified via
obs_id
, where applicable (e.g. AeronetSunV3Lev2.daily). Can be specified variable specific in form of dictionary.
- obs_vert_type
Aerocom vertical code encoded in the model filenames (only AeroCom 3 and later).
- Type:
str, optional
- obs_aux_requires
information about required datasets / variables for auxiliary variables.
- Type:
dict, optional
- instr_vert_loc
vertical location code of observation instrument. This is used in the aeroval interface for separating different categories of measurements such as “ground”, “space” or “airborne”.
- Type:
str, optional
- is_superobs
if True, this observation is a combination of several others which all have to have their own obs config entry.
- Type:
- only_superobs
this indicates whether this configuration is only to be used as part of a superobs network, and not individually.
- Type:
- read_opts_ungridded
dictionary that specifies reading constraints for ungridded reading (c.g.
pyaerocom.io.ReadUngridded
).- Type:
dict
, optional
Specification of model datasets
- class pyaerocom.aeroval.modelentry.ModelEntry(model_id, **kwargs)[source]
Modeln configuration for evaluation (dictionary)
Note
Only
model_id
is mandatory, the rest is optional.- model_ts_type_read
may be specified to explicitly define the reading frequency of the model data. Not to be confused with
ts_type
, which specifies the frequency used for colocation. Can be specified variable specific by providing a dictionary.
- model_use_vars
dictionary that specifies mapping of model variables. Keys are observation variables, values are strings specifying the corresponding model variable to be used (e.g. model_use_vars=dict(od550aer=’od550csaer’))
- Type:
- model_add_vars
dictionary that specifies additional model variables. Keys are observation variables, values are lists of strings specifying the corresponding model variables to be used (e.g. model_use_vars=dict(od550aer=[‘od550csaer’, ‘od550so4’]))
- Type:
- model_rename_vars
key / value pairs specifying new variable names for model variables in the output json files (is applied after co-location).
- Type:
- model_read_aux
may be used to specify additional computation methods of variables from models. Keys are obs variables, values are dictionaries with keys vars_required (list of required variables for computation of var and fun (method that takes list of read data objects and computes and returns var)
- Type:
- property aux_funs_required
Boolean specifying whether this entry requires auxiliary variables
- get_vars_to_process(obs_vars: list) tuple [source]
Get lists of obs / mod variables to be processed
- Parameters:
obs_vars (list) – list of observation variables
- Returns:
list – list of observation variables (potentially extended from input list)
list – corresponding model variables which are mapped based on content of
model_add_vars
andmodel_use_vars
.
Containers for model and observation setup
Collection classes to specify a number of model entries and a number of observation entries for a given AeroVal experiment.
- class pyaerocom.aeroval.collections.BaseCollection(*args, **kwargs)[source]
- FORBIDDEN_CHARS_KEYS = ['_']
Invalid chars in entry names
- MAXLEN_KEYS = 25
maximum length of entry names
- abstract get_entry(key) object [source]
Getter for eval entries
- Raises:
KeyError – if input name is not in this collection
- class pyaerocom.aeroval.collections.ModelCollection(*args, **kwargs)[source]
Dict-like object that represents a collection of model entries
Keys are model names, values are instances of
ModelEntry
. Values can also be assigned as dict and will automatically be converted into instances ofModelEntry
.Note
Entries must not necessarily be only models but may also be observations. Entries provided in this collection refer to the x-axis in the AeroVal heatmap display and must fulfill the protocol defined by
ModelEntry
.- get_entry(key) object [source]
Get model entry configuration
Since the configuration files for experiments are in json format, they do not allow the storage of executable custom methods for model data reading. Instead, these can be specified in a python module that may be specified via
add_methods_file
and that contains a dictionary FUNS that maps the method names with the callable methods.As a result, this means that, by default, custom read methods for individual models in
model_config
do not contain the callable methods but only the names. This method will take care of handling this and will return a dictionary where potential custom method strings have been converted to the corresponding callable methods.
- class pyaerocom.aeroval.collections.ObsCollection(*args, **kwargs)[source]
Dict-like object that represents a collection of obs entries
Keys are obs names, values are instances of
ObsEntry
. Values can also be assigned as dict and will automatically be converted into instances ofObsEntry
.Note
Entries must not necessarily be only observations but may also be models. Entries provided in this collection refer to the y-axis in the AeroVal heatmap display and must fulfill the protocol defined by
ObsEntry
.- property all_vert_types
List of unique vertical types specified in this collection
- get_all_vars() list [source]
Get unique list of all obs variables from all entries
- Returns:
list of variables specified in obs collection
- Return type:
- get_entry(key) object [source]
Getter for obs entries
- Raises:
KeyError – if input name is not in this collection
Processing tools
Experiment processing engine
- class pyaerocom.aeroval.experiment_processor.ExperimentProcessor(cfg: EvalSetup)[source]
Processing engine for AeroVal experiment
By default, this class processes one configuration file, represented by
EvalSetup
. As such, an instance ofEvalSetup
represents an AeroVal experiment, comprising a list of models, a list of observations (and variables).For each possible (or defined) model / obs / variable combination, the processing engine will perform spatial and temporal co-location and will store on co-located NetCDF file (e.g. if there are 2 models, 2 observation networks and 2 variables there will be 4 co-located NetCDF files). The co-location is done using
pyaerocom.colocation_auto.Colocator
.- run(model_name=None, obs_name=None, var_list=None, update_interface=True)[source]
Create colocated data and json files for model / obs combination
- Parameters:
model_name (str or list, optional) – Name or pattern specifying model that is supposed to be analysed. Can also be a list of names or patterns to specify multiple models. If None (default), then all models are run that are part of this experiment.
obs_name (
str
, orlist
, optional) – Likemodel_name
, but for specification(s) of observations that are supposed to be used. If None (default) all observations are used.var_list (list, optional) – list variables supposed to be analysed. If None, then all variables available are used. Defaults to None. Can also be str type.
update_interface (bool) – if true, relevant json files that determine what is displayed online are updated after the run, including the the menu.json file and also, the model info table (minfo.json) file is created and saved in
exp_dir
.
- Returns:
list containing all colocated data objects that have been converted to json files.
- Return type:
Model maps processing
Processing of super-observation entries
Super observations refer to merged observation datasets to increase the number of stations.
Low-level base classes for processing engines
- class pyaerocom.aeroval._processing_base.DataImporter(cfg: EvalSetup)[source]
Class that supports reading of model and obs data based on an eval config.
Depending on a
EvalSetup
, reading of model and obs data may have certain constraints (e.g. freq, years, alias variable names, etc.), which are / can be specified flexibly for each model and obs entry in an analysis setup (EvalSetup
). Proper handling of these reading constraints and data import settings are handled in thepyaerocom.colocation_auto.Colocator
engine, therefore the reading in this class is done via theColocator
engine.- read_model_data(model_name, var_name)[source]
Import model data
- Parameters:
- Returns:
data – loaded model data.
- Return type:
- class pyaerocom.aeroval._processing_base.HasColocator(cfg: EvalSetup)[source]
Config class that also has the ability to co-locate
- class pyaerocom.aeroval._processing_base.HasConfig(cfg: EvalSetup)[source]
Base class that ensures that evaluation configuration is available
- exp_output
Manages output for an AeroVal experiment (e.g. path locations).
- Type:
- class pyaerocom.aeroval._processing_base.ProcessingEngine(cfg: EvalSetup)[source]
Abstract base for classes supposed to do one or more processing tasks
Requirement for a processing class is to inherit attrs from
HasConfig
and, in addition to that, to have implemented a method :fun:`run` which is running the corresponding processing task and storing all the associated output files, that are read by the frontend.One example of an implementation is the
pyaerocom.aeroval.modelmaps_engine.ModelMapsEngine
.
Helpers for processing of auxiliary variables
- class pyaerocom.aeroval.aux_io_helpers.ReadAuxHandler(aux_file: str)[source]
Helper class for import of auxiliary function objects
- aux_file
path to python module containing function definitions (note: function definitions in module need to be stored in a dictionary called FUNS in the file, where keys are names of the functions and values are callable objects.)
- Type:
- Parameters:
aux_file (str) – input file containing auxiliary functions (details see Attributes section).
- import_all()[source]
Import all callable functions in module with their names
Currently, these are expected to be stored in a dictionary called FUNS which should be defined in the python module.
- Returns:
function definitions.
- Return type:
- import_module()[source]
Import
aux_file
as python moduleUses
importlib.import_module()
for import. :returns: imported module. :rtype: module
Conversion of co-located data to json output
- class pyaerocom.aeroval.coldatatojson_engine.ColdataToJsonEngine(cfg: EvalSetup)[source]
- process_coldata(coldata: ColocatedData)[source]
Creates all json files for one ColocatedData object
- Parameters:
coldata (ColocatedData) – colocated data to be processed.
- Raises:
NotImplementedError – DESCRIPTION.
ValueError – DESCRIPTION.
AeroValConfigError – DESCRIPTION.
- Return type:
None.
Output management
- class pyaerocom.aeroval.experiment_output.ExperimentOutput(cfg)[source]
JSON output for experiment
- clean_json_files()[source]
Checks all existing json files and removes outdated data
This may be relevant when updating a model name or similar.
- delete_experiment_data(also_coldata=True)[source]
Delete all data associated with a certain experiment
Note
This simply deletes the experiment directory with all the json files and, if also_coldata is True, also the associated co-located data objects.
- Parameters:
base_dir (str, optional) – basic output direcory (containing subdirs of all projects)
proj_name (str, optional) – name of project, if None, then this project is used
exp_name (str, optional) – name experiment, if None, then this project is used
also_coldata (bool) – if True and if output directory for colocated data is default and specific for input experiment ID, then also all associated colocated NetCDF files are deleted. Defaults to True.
- property exp_dir
Experiment directory
- property exp_id
Experiment ID
Order of models in menu
Note
Returns empty list if no specific order is to be used in which case the models will be alphabetically ordered
Order of observation entries in menu
json file containing region specifications
- property regions_file
json file containing region specifications
- property results_available
True if results are available for this experiment, else False
- Type:
- property statistics_file
json file containing region specifications
- update_interface() None [source]
Update web interface
Steps:
Check if results are available, and if so:
Add entry for this experiment in experiments.json
Create/update ranges.json file in experiment directory
Update menu.json against available output and evaluation setup
Synchronise content of heatmap json files with menu
Create/update file statistics.json in experiment directory
Copy json version of EvalSetup into experiment directory
- Return type:
None
Update menu
The menu.json file is created based on the available json map files in the map directory of an experiment.
- property var_ranges_file
json file containing region specifications
- class pyaerocom.aeroval.experiment_output.ProjectOutput(proj_id: str, json_basedir: str)[source]
JSON output for project
- property available_experiments
List of available experiments
- property experiments_file
json file containing region specifications
- property proj_dir
Project directory
- reorder_experiments(exp_order=None)[source]
Reorder experiment order in evaluation interface
Puts experiment list into order as specified by exp_order, all remaining experiments are sorted alphabetically.
- Parameters:
exp_order (list, optional) – desired experiment order, if None, then alphabetical order is used.
Global settings
Global defaults
- pyaerocom.aeroval.glob_defaults.statistics_defaults = {'R': {'colmap': 'RdYlGn', 'decimals': 2, 'forecast': True, 'longname': 'Correlation Coefficient', 'name': 'R', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1], 'unit': '1'}, 'R_spearman': {'colmap': 'RdYlGn', 'decimals': 2, 'longname': 'R Spearman Correlation', 'name': 'R Spearman', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1], 'unit': '1'}, 'data_mean': {'colmap': 'coolwarm', 'decimals': 2, 'longname': 'Model Mean', 'name': 'Mean-Mod', 'scale': None, 'unit': '1'}, 'fge': {'colmap': 'reverseColmap(RdYlGn)', 'decimals': 2, 'forecast': True, 'longname': 'Fractional Gross Error', 'name': 'FGE', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2], 'unit': '1'}, 'mab': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Mean Absolute Bias', 'name': 'MAB', 'scale': [0, 0.025, 0.05, 0.075, 0.1, 0.125, 0.15], 'unit': 'var'}, 'mb': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Mean Bias', 'name': 'Mean Bias', 'scale': [-0.15, -0.1, -0.05, 0, 0.05, 0.1, 0.15], 'unit': 'var'}, 'mnmb': {'colmap': 'bwr', 'decimals': 1, 'forecast': True, 'longname': 'Modified Normalized Mean Bias', 'name': 'MNMB', 'scale': [-100, -75, -50, -25, 0, 25, 50, 75, 100], 'unit': '%'}, 'nmb': {'colmap': 'bwr', 'decimals': 1, 'forecast': True, 'longname': 'Normalized Mean Bias', 'name': 'NMB', 'scale': [-100, -75, -50, -25, 0, 25, 50, 75, 100], 'unit': '%'}, 'nrms': {'colmap': 'Reds', 'decimals': 1, 'longname': 'Normalized Root Mean Square Error', 'name': 'NRMSE', 'scale': [0, 25, 50, 75, 100, 125, 150, 175, 200], 'unit': '%'}, 'num_coords_with_data': {'colmap': None, 'decimals': 0, 'longname': 'Number of Stations with data', 'name': 'Nb. Stations', 'overall_only': True, 'scale': None, 'unit': '1'}, 'num_valid': {'colmap': None, 'decimals': 0, 'longname': 'Number of Valid Observations', 'name': 'Nb. Obs', 'overall_only': True, 'scale': None, 'unit': '1'}, 'refdata_mean': {'colmap': 'coolwarm', 'decimals': 2, 'longname': 'Observation Mean', 'name': 'Mean-Obs', 'scale': None, 'unit': '1'}, 'rms': {'colmap': 'coolwarm', 'decimals': 2, 'forecast': True, 'longname': 'Root Mean Square Error', 'name': 'RMSE', 'scale': None, 'unit': '1'}}
Default information for statistical parameters
- pyaerocom.aeroval.glob_defaults.statistics_trend = {'mod_trend': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Modelled Trends', 'name': 'Mod-Trends', 'scale': [-10, -7.5, -5.0, -2.5, 0, 2.5, 5.0, 7.5, 10.0], 'unit': '%/yr'}, 'obs/mod_trend': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Trends', 'name': 'Obs/Mod-Trends', 'scale': [-10.0, -7.5, -5.0, -2.5, 0, 2.5, 5.0, 7.5, 10.0], 'unit': '%/yr'}, 'obs_trend': {'colmap': 'bwr', 'decimals': 1, 'longname': 'Observed Trends', 'name': 'Obs-Trends', 'scale': [-10.0, -7.5, -5.0, -2.5, 0, 2.5, 5.0, 7.5, 10.0], 'unit': '%/yr'}}
Default information about trend display
- pyaerocom.aeroval.glob_defaults.var_ranges_defaults = {'abs550aer': {'colmap': 'coolwarm', 'scale': [0, 0.0125, 0.025, 0.0375, 0.05, 0.0625, 0.075, 0.0875, 0.1]}, 'absc550aer': {'colmap': 'coolwarm', 'scale': [0, 12.5, 25, 37.5, 50, 62.5, 75, 87.5, 100]}, 'ang4487aer': {'colmap': 'coolwarm', 'scale': [-0.2, -0.1, 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0]}, 'backscatter': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'concCecpm25': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'concCocpm25': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]}, 'concNhno3': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5]}, 'concNnh3': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]}, 'concNnh4': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0]}, 'concNno3pm10': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'concNno3pm25': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'concNtnh': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0]}, 'concNtno3': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1.0]}, 'concco': {'colmap': 'coolwarm', 'scale': [100.0, 125.0, 150.0, 175.0, 200.0, 225.0, 250.0, 275.0, 300.0]}, 'concdust': {'colmap': 'coolwarm', 'scale': [0.0, 10.0, 20.0, 30.0, 40.0, 50.0, 60.0, 70.0, 80.0, 90.0, 100.0]}, 'concno2': {'colmap': 'coolwarm', 'scale': [0, 10, 20, 30, 40, 50, 60, 70, 80]}, 'conco3': {'colmap': 'coolwarm', 'scale': [0, 15, 30, 45, 60, 75, 90, 105, 120]}, 'concpm10': {'colmap': 'coolwarm', 'scale': [0, 10, 20, 30, 40, 50, 60, 70, 80]}, 'concpm25': {'colmap': 'coolwarm', 'scale': [0, 5, 10, 15, 20, 25, 30, 35, 40, 45]}, 'concso2': {'colmap': 'coolwarm', 'scale': [0, 0.75, 1.5, 2.25, 3.0, 3.75, 4.5, 5.25, 6.0, 6.75, 7.5, 8.25]}, 'concso4': {'colmap': 'coolwarm', 'scale': [0, 0.75, 1.5, 2.25, 3.0, 3.75, 4.5, 5.25, 6.0, 6.75, 7.5, 8.25]}, 'concsspm10': {'colmap': 'coolwarm', 'scale': [0, 0.75, 1.5, 2.25, 3.0, 3.75, 4.5, 5.25, 6.0, 6.75, 7.5, 8.25]}, 'concsspm25': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0]}, 'default': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'depdust': {'colmap': 'coolwarm', 'scale': [0.0, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, 100.0, 200.0, 500.0, 1000.0]}, 'depoxn': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'depoxnf': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'depoxs': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'depoxsf': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'deprdn': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'deprdnf': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'drydust': {'colmap': 'coolwarm', 'scale': [0.0, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, 100.0, 200.0, 500.0, 1000.0]}, 'dryoxn': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'dryoxs': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'dryrdn': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'extinction': {'colmap': 'coolwarm', 'scale': [0, 0.125, 0.25, 0.375, 0.5, 0.625, 0.75, 0.875, 1]}, 'od550aer': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'od550dust': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'od550gt1aer': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'od550lt1aer': {'colmap': 'coolwarm', 'scale': [0.0, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 2.0]}, 'prmm': {'colmap': 'coolwarm', 'scale': [0, 1.25, 2.5, 3.75, 5, 6.25, 7.5, 8.75, 10]}, 'proxydryhno3': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryhono': {'colmap': 'coolwarm', 'scale': [0.001, 0.002, 0.003, 0.004, 0.005, 0.006]}, 'proxydryn2o5': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'proxydrynh3': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}, 'proxydrynh4': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryno2': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryno3c': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'proxydryno3f': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'proxydryo3': {'colmap': 'coolwarm', 'scale': [0, 0.5, 1, 15, 20, 25, 0.3, 40, 50]}, 'proxydryoxn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}, 'proxydryoxs': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydrypm10': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'proxydrypm25': {'colmap': 'coolwarm', 'scale': [0, 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2]}, 'proxydryrdn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}, 'proxydryso2': {'colmap': 'coolwarm', 'scale': [0, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4]}, 'proxydryso4': {'colmap': 'coolwarm', 'scale': [0.01, 0.02, 0.03, 0.04, 0.05]}, 'scatc550dryaer': {'colmap': 'coolwarm', 'scale': [0, 12.5, 25, 37.5, 50, 62.5, 75, 87.5, 100]}, 'ts': {'colmap': 'coolwarm', 'scale': [265, 270, 275, 280, 285, 290, 300, 305, 310, 315, 320]}, 'vmrch4': {'colmap': 'coolwarm', 'scale': [1700, 1750, 1800, 1850, 1900, 1950, 2000, 2050, 2100, 2150, 2200]}, 'vmrco': {'colmap': 'coolwarm', 'scale': [100.0, 125.0, 150.0, 175.0, 200.0, 225.0, 250.0, 275.0, 300.0]}, 'vmrco2': {'colmap': 'coolwarm', 'scale': [400.0, 405.0, 410.0, 415.0, 420.0, 425.0, 430.0, 435.0, 440.0, 445.0, 450.0]}, 'vmrno2': {'colmap': 'coolwarm', 'scale': [0, 5, 10, 15, 20, 25, 30, 35, 40]}, 'vmro3': {'colmap': 'coolwarm', 'scale': [20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70]}, 'vmro3max': {'colmap': 'coolwarm', 'scale': [0, 7.5, 15, 22.5, 30, 37.5, 45, 52.5, 60]}, 'wetdust': {'colmap': 'coolwarm', 'scale': [0.0, 0.2, 0.5, 1.0, 2.0, 5.0, 10.0, 20.0, 50.0, 100.0, 200.0, 500.0, 1000.0]}, 'wetna': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxnf': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxs': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetoxsf': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetrdn': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}, 'wetrdnf': {'colmap': 'coolwarm', 'scale': [0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.7, 1, 1.5]}}
Default variable ranges for web display
Frontend variable naming conventions
- class pyaerocom.aeroval.varinfo_web.VarinfoWeb(var_name: str, cmap: str | None = None, cmap_bins: list | None = None, vmin: float | None = None, vmax: float | None = None)[source]
Additional variable information relevant for AeroVal web output
- Parameters:
var_name (str) – Name of variable (AeroCom name, not web display name)
cmap (str, optional) – name of colormap for web display. If None, the colormap associated with the input variable is used (via
pyaerocom.variable.Variable.get_cmap()
). Defaults to None.cmap_bins (list, optional) – Value bins for web display. If None, then they are inferred from input vmin and vmax, or, if the latter are also None, from attrs
pyaerocom.variable.Variable.minimum
andpyaerocom.variable.Variable.maximum
. If the latter are not defined an AttributeError will be thrown on initialisation.vmin (float, optional) – lower end of range
vmax (float, optional) – upper end of range
- autofill_missing(vmin: float | None = None, vmax: float | None = None) None [source]
Autofill missing attributes related to cmap bins and cmap
High-level utility functions
- pyaerocom.aeroval.utils.compute_model_average_and_diversity(cfg, var_name, model_names=None, ts_type=None, lat_res_deg=2, lon_res_deg=3, data_id=None, avg_how=None, extract_surface=True, ignore_models=None, comment=None, model_use_vars=None)[source]
Compute median or mean model based on input models
Note
BETA version that will likely undergo revisions.
Time selection currently not properly handled
- Parameters:
cfg (AerocomEvaluation) – analysis instance
var_name (str) – name of variable
model_names (list, optional) – list of model names. If None, all entries in input engine are used.
ts_type (str, optional) – output freq. Defaults to monthly.
lat_res_deg (int, optional) – output latitude resolution, defaults to 2 degrees.
lon_res_deg (int, optional) – output longitude resolution, defaults to 3 degrees.
data_id (str, optional) – output data_id of ensemble model.
avg_how (str, optional) – how to compute averages (choose from mean or median), defaults to “median”.
extract_surface (bool) – if True (and if data contains model levels), surface level is extracted
ignore_models (list, optional) – list of models to be ignored
comment (str, optional) – comment string added to metadata of output data objects.
model_use_vars (dict, optional) – model variables to be used.
- Returns:
GriddedData – ensemble model for input variable computed averaged using median or mean (input avg_how). Default is median.
GriddedData – corresponding diversity field, if avg_how is “mean”, then computed using definition from Textor et al., 2006 (ACP) DOI: 10.5194/acp-6-1777-2006. If avg_how is “median” then interquartile range is used (Q3-Q1)/Q2
GriddedData or None – Q1 field (only output if avg_how is median)
GriddedData or None – Q3 field (only output if avg_how is median)
GriddedData or None – standard deviation field (only output if avg_how is mean)
Helper modules
General helper functions
- pyaerocom.aeroval.helpers.check_var_ranges_avail(model_data, var_name)[source]
Check if lower and upper variable ranges are available for input variable
- Parameters:
model_data (GriddedData) – modeldata containing variable data
var_name (str) – variable name to be checked (must be the same as model data AeroCom variable name).
- Raises:
ValueError – if ranges for input variable are not defined and if input model data corresponds to a different variable than the input variable name.
- Return type:
None
Helpers for coldat2json conversion
Helpers for conversion of ColocatedData to JSON files for web interface.
- pyaerocom.aeroval.coldatatojson_helpers.add_profile_entry_json(profile_file: str, data: ColocatedData, profile_viz: dict, periods: list[str], seasons: list[str])[source]
Analogous to _add_heatmap_entry_json for profile data. Every time this function is called it checks to see if the profile_file exists. If so, it reads it, if not it makes a new one. This is because one can not add to json files and so everytime we want to add entries for profile layers we must read in the old file, add the entries, and write a new file.
- pyaerocom.aeroval.coldatatojson_helpers.get_json_mapname(obs_name, var_name_web, model_name, model_var, vert_code, period)[source]
Get name base name of json file
- pyaerocom.aeroval.coldatatojson_helpers.get_stationfile_name(station_name, obs_name, var_name_web, vert_code)[source]
Get name of station timeseries file
- pyaerocom.aeroval.coldatatojson_helpers.process_profile_data_for_regions(data: ColocatedData, region_id: str, use_country: bool, periods: list[str], seasons: list[str]) dict [source]
This method populates the json files in data/profiles which are use for visualization. Analogous to _process_map_and_scat for profile data. Each json file corresponds to a region or station, obs network, and variable. Inside the json, it is broken up by model. Each model has a key for “z” (the vertical dimension), “obs”, and “mod” Each “obs” and “mod” is broken up by period.
- Parameters:
data (ColocatedData) – ColocatedData object for this layer
region_id (str) – Spatial subset to compute the mean profiles over
station_name (str) – Station to compute mean profiles over for period
use_country (boolean) – Passed to filter_region().
periods (str) – Year part of the temporal range to average over
seasons (str) – Sesonal part of the temporal range to average over
- Returns:
Dictionary to write to json
- Return type:
output (dict)
- pyaerocom.aeroval.coldatatojson_helpers.process_profile_data_for_stations(data: ColocatedData, station_name: str, use_country: bool, periods: list[str], seasons: list[str]) dict [source]
This method populates the json files in data/profiles which are use for visualization. Analogous to _process_map_and_scat for profile data. Each json file corresponds to a region, obs network, and variable. Inside the json, it is broken up by model. Each model has a key for “z” (the vertical dimension), “obs”, and “mod” Each “obs” and “mod” is broken up by period.
- Parameters:
data (ColocatedData) – ColocatedData object for this layer
region_id (str) – Spatial subset to compute the mean profiles over
station_name (str) – Station to compute mean profiles over for period
use_country (boolean) – Passed to filter_region().
periods (str) – Year part of the temporal range to average over
seasons (str) – Sesonal part of the temporal range to average over
- Returns:
Dictionary to write to json
- Return type:
output (dict)
Model maps helper functions
- pyaerocom.aeroval.modelmaps_helpers.calc_contour_json(data, cmap, cmap_bins)[source]
Convert gridded data into contours for json output
- Parameters:
data (GriddedData) – input data
cmap (str) – colormap of output
cmap_bins (list) – list containing the bins to which the values are mapped.
- Returns:
dictionary containing contour data
- Return type:
- pyaerocom.aeroval.modelmaps_helpers.griddeddata_to_jsondict(data, lat_res_deg=5, lon_res_deg=5)[source]
Convert gridded data to json dictionary
- Parameters:
data (GriddedData) – input data to be converted
lat_res_deg (int) – output latitude resolution in decimal degrees
lon_res_deg (int) – output longitude resolution in decimal degrees
- Returns:
data dictionary for json output (keys are metadata and data).
- Return type: