sysvar package

Submodules

sysvar.api module

sysvar.api.add_weights_to_dataframe(df: DataFrame, systematic: str, MC_production: str, prefix: str, weightname: str, overwrite: bool = False, Nvar: int = 0)

Add weight columns to a DataFrame.

This function augments df in-place by adding a central weight column whose name is constructed from prefix and weightname, and filling it using the configured correction for the given systematic and MC_production. If Nvar > 0, it also adds Nvar variation columns (named “{column_name}_var_{j}”) and fills them from variations (toys) of the central values of the corrections. A dedicated Variator object is used internally to generate the toys.

Parameters:

• df (DataFrame) – pandas DataFrame to be augmented (modified in-place).

• systematic (str) – Name of the systematic/correction to apply.

• MC_production (str) – MC production tag used to locate the correction.

• prefix (str) – Prefix used when building the weight column name.

• weightname (str) – Base name of the weight column to add.

• overwrite (bool, optional) – If True, overwrite an existing column with the same name. Defaults to False.

• Nvar (int, optional) – Number of variation columns to add. Must be a non-negative integer. If 0, only the central value column is added. Defaults to 0.

Returns:

The DataFrame df is modified in-place.

Return type:

None

Raises:

ValueError – If Nvar is negative.

sysvar.api.calculate_covariance_matrix(df: DataFrame, settings: Dict, syst_effect: str | Dict, binning: Dict, channels: List, input_cov: np.ndarray = None, save_cov: bool = False)

Calculate the covariance matrix for a given dataset.

This function computes a covariance matrix based on the input data, configuration settings, and systematic effects. It provides support for pre-defined systematics or custom-defined ones and allows the user to specify binning and channels. Optionally, it can save the covariance matrix to a file.

Parameters:

• df (DataFrame) – The input data to calculate the covariance matrix from.

• settings (Dict) – Configuration settings, same as for the EigenDecomposer.

• syst_effect (str | Dict) – The name of the systematic effect to consider for the covariance matrix. For systematics from YAML files the name is enough. If this is a custom systematic then a dictionary with for the custom systematic is expected similarly to the dictionary necessary for the custom correction object in the eigendecomposition.

• binning (Dict) – Binning information for the covariance matrix. Keys should be the variable names present in the df and values lists of bin edges.

• channels (List) – List of channels to consider for the covariance matrix.

• save_cov (bool, optional) – If True, saves the covariance matrix. The path should be read from the settings dictionary. Defaults to False.

Returns:

the covariance matrix from the covariance matrix calculator

sysvar.api.eigendecompose(df: DataFrame, settings: Dict, syst_effect: str, criterion: str = 'max_differences', prc: float = 0.0001, max_variations: int | None = None, save_variations: bool = False, save_channel_covariance_matrices: bool = False, verbose: bool = True, seed: int = 8311311)

Performs eigendecomposition on the input DataFrame based on specified settings, systematic effect, and criterion, and returns the resulting EigenDecomposer object.

This function initializes an EigenDecomposer instance using the provided DataFrame, settings, and systematic effect. It then applies a precision level to the decomposition and identifies important eigendimension indices based on the specified criterion. Optionally, it saves template variations.

Parameters:

• df (DataFrame) – The input data to be decomposed.

• settings (Dict) – Configuration settings for the decomposition.

• syst_effect (str) – The systematic effect to consider.

• criterion (str, optional) – Criterion for selecting important eigendimensions. Defaults to “max_differences”.

• prc (float, optional) – Precision level for the decomposition. Defaults to 1e-4.

• max_variations (int | None, optional) – Maximum number of variations to consider taking into account the precision criterion. If None, all variations up to the precision are considered. Defaults to None.

• save_variations (bool, optional) – If True, saves template variations. Defaults to False.

• save_channel_covariance_matrices (bool, optional) – If True, saves covariance matrices per channel. Defaults to False.

• verbose (bool, optional) – If True, prints additional information during execution. Defaults to True.

• seed (int, optional) – Random seed for reproducibility. Defaults to 8311311.

Returns:

An instance of the EigenDecomposer class containing the decomposition results.

Return type:

EigenDecomposer

sysvar.api.plot_analysis_corr_matrix(eigendecomposer_obj: EigenDecomposer, save: bool = False, filename: None | str = None) → tuple[Figure, Axes]

Plot the correlation matrix of an eigendecomposition analysis.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object containing correlation data.

• save (bool, optional) – If True, saves the plot to file. Defaults to False.

• filename (str, optional) – Output file name if saving. Defaults to None.

Returns:

None

sysvar.api.plot_correction_cov_and_corr(eigendecomposer_obj: EigenDecomposer, save: bool = False, filename: None | str = None) → tuple[Figure, Axes]

Plot correction covariance and correlation matrices.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object containing correction data.

• save (bool, optional) – If True, saves the plot. Defaults to False.

• filename (str, optional) – Output file name if saving. Defaults to None.

Returns:

None

sysvar.api.plot_correction_errors(eigendecomposer_obj: EigenDecomposer, save: bool = False, filename: None | str = None) → tuple[Figure, Axes]

Plot correction error comparisons.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object containing correction information.

• save (bool, optional) – If True, saves the plot. Defaults to False.

• filename (str, optional) – Output file name if saving. Defaults to None.

Returns:

None

sysvar.api.plot_correction_variations_in_grid(eigendecomposer_obj: EigenDecomposer, nbins=21, save: bool = False, filename: None | str = None) → tuple[Figure, Axes]

Plot correction variations in a grid layout.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object containing correction variations.

• nbins (int, optional) – Number of bins to use in the grid plot. Defaults to 21.

• save (bool, optional) – If True, saves the plots. Defaults to False.

• filename (str, optional) – Output file name if saving. Defaults to None.

Returns:

None

sysvar.api.plot_cov_diff(eigendecomposer_obj: EigenDecomposer, save: bool = False, filename: None | str = None) → tuple[Figure, Axes]

Plot the normalized covariance difference between original and eigendecomposed covariance matrix for an initial truncation guess.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object containing covariance data.

• save (bool, optional) – If True, saves the plot. Defaults to False.

• filename (str, optional) – Output file name if saving. Defaults to None.

Returns:

None

sysvar.api.plot_templates_relative_variations_in_grid(eigendecomposer_obj: EigenDecomposer, save: bool = False, filename: None | str = None) → List[tuple[Figure, Axes]]

Plot relative template variations in a grid layout.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object containing templates.

• save (bool, optional) – If True, saves the plots. Defaults to False.

• filename (str, optional) – Output file name if saving. Defaults to None.

Returns:

None

sysvar.api.plot_up_and_down_variations(eigendecomposer_obj: EigenDecomposer, save: bool = False, filename: None | str = None) → List[tuple[Figure, Axes]]

Plot up/down variations for each template in the decomposition.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object containing templates.

• save (bool, optional) – If True, saves the plots. Defaults to False.

• filename (str, optional) – Output file name if saving. Defaults to None.

Returns:

None

sysvar.api.register_saving_info(eigendecomposer_obj: EigenDecomposer, saving_info: Dict)

Register saving information in the eigendecomposer object.

Parameters:

• eigendecomposer_obj (EigenDecomposer) – The decomposition object to update.

• saving_info (Dict) – Dictionary containing saving parameters.

Returns:

None

sysvar.api.save_existing_eigenvariations(df: DataFrame, settings: Dict, syst_effect: str, verbose=True)

Save existing eigenvariations for a specified systematic effect.

This function complements the nominal-template saving step: nominal templates should already be present in the configured output file before calling this function.

The saver will read variations from df and write eigenvariation histograms into the same ROOT file structure expected by cabinetry/pyhf so that the model can later be built including these systematic eigenvariations. Instead of using the nominal weights for the histogram filling, the nominal weight for the syst_effect is replaced by the variations present in df. The number of variations that should be present in df is read from the settings dictionary in the systematics part.

Parameters:

• df (DataFrame) – The dataset to extract variations from.

• settings (Dict) – Configuration settings for eigenvariation saving.

• syst_effect (str) – The systematic effect to save variations for.

• verbose (bool, optional) – If True, enables verbose logging. Defaults to True.

Returns:

None

sysvar.api.save_nominal_templates(df: DataFrame, settings: Dict, data=None)

Save nominal templates for an MC dataset.

Write nominal templates for a Monte Carlo (MC) dataset (and optional experimental data) to the output file configured in settings. Only ROOT (.root) files are currently supported. Channels, templates, signal extraction variables, binning, and other required configuration are read from the settings dictionary. The produced file structure is compatible with cabinetry and can be used to build a pyhf model including systematic uncertainties.

The function creates or recreates the configured output file on disk and therefore will overwrite any existing file at that location. Because the file is recreated, eigenvariation histograms saved before calling this function would be lost; call this function before saving eigenvariations for systematics.

Parameters:

• df (DataFrame) – MC dataset containing template information used to build the nominal templates (e.g., event records or pre-binned contents). This object is read but not modified by the function.

• settings (Dict) – Configuration dictionary. Must include the output filename (currently a .root path) and the definitions for channels, templates, signal-extraction variables and binning required to produce the histograms.

• data (optional) – Experimental (observed) dataset to be histogrammed and included in the output using the same channels, variables and binning as the MC templates. If None, no observed-data histograms are written.

Returns:

The handler object used to save the nominal

templates. This object can be returned for later use inspect saved templates).

Return type:

ChannelTemplateHandler

sysvar.channel_template_handler module

class sysvar.channel_template_handler.ChannelTemplateHandler(df: DataFrame, settings: dict, verbose: bool = True)

Bases: SavableAttributesObject

property Nbins: int

create_templates()

property decay_modes: list

property enumerated_iterator: product

property iterator: product

property nominal_hist: ndarray

save_nominal_templates(filepath=None, data=None)

property syst_effect: str

property template_names: list

property templates

Creates templates based on the provided DataFrame and settings.

Returns:

A list of created templates.

Return type:

list

Example

>>> templates = self.create_templates()
>>> len(templates)
5

sysvar.corrections module

class sysvar.corrections.BaseCorrection(uncertainties: 'dict' = <factory>)

Bases: ABC, SavableAttributesObject

property N: int

add_uncertainty(unc_name, unc_values, unc_obj: Uncertainty, explicit_cov_matrix: ndarray | None = None) → None

Add an uncertainty to the Correction.

Parameters:

unc (Uncertainty) – The uncertainty to be added.

Raises:

UncertaintyWithSameNameExists – If uncertainty with the same name has already been added to the variator.

abstract build_queries() → list

plot_error_comparison(save: bool = False, filename: str = '') → tuple[Figure, Axes]

plot_uncertainty(unc_name: str | None = None, save: bool = False, filename: str = '')

populate_uncertainties()

Populate the uncertainties attribute with uncertainty objects based on the provided information.

This method adds uncertainties either from a provided covariance matrix (explicitly correlated uncertainty), or from a dictionary of uncertainty types and values specified in the info attribute.

If a covariance matrix (cov_matrix) is available, it creates and adds a single ExplicitlyCorrelatedUncertainty. Otherwise, it looks up available uncertainty types, validates them, and populates the uncertainties accordingly.

Raises:

UnknownUncertaintyType – If an unsupported uncertainty type is found in the input data.

Notes

• The uncertainty type definitions are expected to be returned from get_uncertainty_types().

• Each uncertainty must have a unique name.

property total_error: ndarray

uncertainties: dict

abstract property visual_labels

class sysvar.corrections.BaseCorrectionFromYaml(uncertainties: 'dict' = <factory>, systematic: 'str' = None, MC_production: 'str' = None)

Bases: BaseCorrection

MC_production: str = None

add_extra_cuts(queries: str, prefix: str) → str

build_table_path(suffix: None | str = None) → str

Builds the path for a table file based on the given suffix.

Parameters:

suffix (str) – The suffix to be added to the table name.

Returns:

The full path of the table file.

Return type:

str

Raises:

ValueError – If the table extension is unknown.

systematic: str = None

property table_dir: str

Returns the directory path where the tables are stored.

Returns:

The directory path where the tables are stored.

Return type:

str

property table_ext: str

Returns the table extension by reading it from the config file.

Returns:

The table extension.

Return type:

str

property table_key: str

Returns the table key by reading it from the config file.

Returns:

The table key.

Return type:

str

property table_name: str

Returns the table name by reading it from the config file.

Returns:

The table name.

Return type:

str

class sysvar.corrections.Correction1D(uncertainties: 'dict' = <factory>, systematic: 'str' = None, MC_production: 'str' = None, dependant_variable: 'str | None' = None, central_values: 'Iterable' = None, lower_bounds: 'Iterable' = None, upper_bounds: 'Iterable' = None)

Bases: BaseCorrectionFromYaml

build_queries(prefix: str | None = None) → list

central_values: Iterable = None

dependant_variable: str | None = None

lower_bounds: Iterable = None

read_corrections() → ndarray

Reads correction values either directly from the config or from a table file.

This method checks whether the ‘corrections’ entry in the config (self.info) is a list/array or a dictionary. If it is a list or NumPy-compatible array, the correction values are loaded directly. If it is a dictionary, the method builds the table path using self.build_table_path(), reads the table, and extracts the correction values using the key specified in self.table_key.

Returns:

An array of correction values.

Return type:

np.ndarray

Raises:

• KeyError – If ‘corrections’ or self.table_key is missing from the config.

• FileNotFoundError – If the table file does not exist at the constructed path.

• ValueError – If the table does not contain the specified key.

upper_bounds: Iterable = None

property value_edges: ndarray

property value_mids: ndarray

property visual_labels: List[str]

class sysvar.corrections.Correction2D(uncertainties: 'dict' = <factory>, systematic: 'str' = None, MC_production: 'str' = None)

Bases: BaseCorrectionFromYaml

build_queries(prefix: str | None = None) → list

property central_values_table: DataFrame

property iterator

populate_uncertainties()

Populate the uncertainties attribute with uncertainty objects based on the provided information.

This method adds uncertainties either from a provided covariance matrix (explicitly correlated uncertainty), or from a dictionary of uncertainty types and values specified in the info attribute.

If a covariance matrix (cov_matrix) is available, it creates and adds a single ExplicitlyCorrelatedUncertainty. Otherwise, it looks up available uncertainty types, validates them, and populates the uncertainties accordingly.

Raises:

UnknownUncertaintyType – If an unsupported uncertainty type is found in the input data.

Notes

• The uncertainty type definitions are expected to be returned from get_uncertainty_types().

• Each uncertainty must have a unique name.

property stat_error_table: DataFrame

property sys_error_table: DataFrame

uncertainties: dict

property visual_labels: List[str]

class sysvar.corrections.Correction2DCategorical(uncertainties: 'dict' = <factory>, systematic: 'str' = None, MC_production: 'str' = None, categorical_variable: 'str | None' = None, continuus_variable: 'str | None' = None, central_values: 'Iterable' = None, continuus_edges: 'Iterable' = None, categorical_values: 'Iterable' = None, categorical_label: 'str' = None)

Bases: BaseCorrectionFromYaml

categorical_label: str = None

categorical_values: Iterable = None

categorical_variable: str | None = None

central_values: Iterable = None

continuus_edges: Iterable = None

continuus_variable: str | None = None

property iterator

property queries

property strings: List[str]

class sysvar.corrections.CorrectionBF(uncertainties: 'dict' = <factory>, systematic: 'str' = None, MC_production: 'str' = None, dependant_variable: 'str | None' = None, central_values: 'Iterable' = None, visual_labels: 'Iterable' = None)

Bases: BaseCorrectionFromYaml

build_queries(prefix: str | None = None) → list

central_values: Iterable = None

dependant_variable: str | None = None

get_uncertainty_correlation()

populate_uncertainties(error_amplitudes: list)

Overrides the method of the base class method as the error amplitutes are calculated dynamically from the calculate_scaling_ratios method

uncertainties: dict

visual_labels: Iterable = None

class sysvar.corrections.CorrectionPID(uncertainties: 'dict' = <factory>, systematic: 'str' = None, MC_production: 'str' = None)

Bases: BaseCorrectionFromYaml

build_queries(prefix: str | None = None, extra_cut: str | None = None) → List[str]

static check_valid_rate(rate)

get_table(rate)

property iterator

property queries

property true_pdg: list

uncertainties: dict

property visual_labels: List[str]

class sysvar.corrections.CorrectionTableFinder(particle_species, online_cut, base_table_path, variable, MC_production)

Bases: object

Factory method class to get correction tables for kaons, pions, electrons and muons

classmethod Kfake_electrons(external_info, MC_production)

classmethod Kfake_muons(external_info, MC_production)

add_mcPDG_to_table(table)

build_hid_table_name(table_id, charge)

build_table_name(table_ids: str) → list

Builds the efficiency and fake rate tables path names

Args: table_ids: efficiency or fake table id

Returns: list with the efficiency or fake rate table file names

classmethod electrons(external_info, MC_production)

get_cut_type() → str

Reads the yaml configuration file and extracts the cut type that have been applied in the online reconstuction

Args: species: Particle species, should be K+ or pi+

Returns: the cut type that has been applied online. Binary or global

get_cut_value() → str

Reads the yaml configuration file and extracts the cut type that have been applied in the online reconstuction

Args: species: Particle species, should be K+ or pi+

Returns: the cut type that has been applied online. Binary or global

get_lid_queries()

get_table(table_names)

classmethod kaons(external_info, MC_production)

static make_pidvar_compatible(MC_production: str, table: DataFrame, max_uncertainty: float | None = 100.0)

Convert the pandas dataframes obtained Hadron ID CSV tables via into a format consistent with the format of the lepton ID tables which PIDvar understands.

In particular, convert the charge column from 1/-1 integer entries to +/- string entries and calculate the theta_min/theta_max columns.

Parameters:

• table – Pandas dataframe obtained from pandas.read_csv on HID table

• inplace – Whether to modify the existing dataframe in place. Otherwise, a copy of the existing dataframe will be returned.

• max_uncertainty – Drop rows in HID tables where any of the data-MC uncertainties (sys/stat up/down) exceed this value. Rationale: The HID tables contain rows with nonsense uncertainties > 10⁸, so it is meant to remove those entries. Therefore, the exact value is not important. Set to None to disable dropping any columns.

Returns:

Modified dataframe that can be used by PIDvar.

classmethod muons(external_info, MC_production)

property particle_species_settings: dict

classmethod pifake_electrons(external_info, MC_production)

classmethod pifake_muons(external_info, MC_production)

classmethod pions(external_info, MC_production)

production_table_ids()

Updates hadron table IDs based on the MC production type.

Raises:

ValueError – If an unsupported MC production type is provided.

class sysvar.corrections.CustomCorrection(uncertainties: 'dict' = <factory>, info: 'InitVar[dict]' = None, dependant_variable: 'str | None' = None, central_values: 'Iterable' = None, query_targets: 'Iterable' = None)

Bases: BaseCorrection

build_queries(prefix: str | None = None) → list

central_values: Iterable = None

dependant_variable: str | None = None

info: InitVar[dict] = None

query_targets: Iterable = None

uncertainties: dict

property value_edges: ndarray

property value_mids: ndarray

property visual_labels: List[str]

exception sysvar.corrections.MissingInformationError

Bases: Exception

exception sysvar.corrections.NotValidRateType

Bases: Exception

exception sysvar.corrections.UncertaintyWithSameNameExists

Bases: Exception

exception sysvar.corrections.UnkownUncertaintyType

Bases: Exception

sysvar.corrections.create_correction_object(syst_effect: str | dict | None, MC_prod: str) → BaseCorrection

Retrieves amd creates the appropriate correction object based on the systematic effect and MC production type.

Parameters:

• syst_effect (str) – The systematic effect identifier.

• MC_prod (str) – The Monte Carlo production type identifier.

Returns:

The appropriate correction object based on the provided systematic effect and MC production type.

Return type:

BaseCorrection

Raises:

NotImplementedError – If the correction type specified in the configuration is not implemented.

Example

>>> correction = get_correction_object("syst1", "MC1")
>>> isinstance(correction, BaseCorrection)
True

sysvar.covariance_calculator module

class sysvar.covariance_calculator.CovarianceCalculator(df: DataFrame, settings: dict, syst_effect: str, binning: dict, channels: int | List[str] = None, cov_input: np.ndarray | None = None, verbose: bool = False)

Bases: EigenDecomposer

property cov: ndarray

save_covariance()

sysvar.eigendecomposer module

class sysvar.eigendecomposer.EigenDecomposer(df: DataFrame, settings: dict, syst_effect: str | dict | None, verbose: bool = True, seed: int = 8311311)

Bases: ChannelTemplateHandler

property combined_variations: ndarray

property corr: ndarray

property cov: ndarray

property eigen_decomposition: tuple

property eigen_values: ndarray

property eigen_variations: ndarray

property eigen_vectors: ndarray

find_important_eigendimension_indices(method: str = 'max_differences')

property max_differences: list

property max_variations: int | None

plot_corr_matrix(save: bool = False, filename: str = '')

plot_cov_diff(save: bool = False, filename: str = '')

property precision: float

property prefices: str

save_channel_covariance_matrices(filepath=None)

save_one_sigma_variations(filepath=None)

save_template_variations(filepath=None)

save_toys(filepath=None)

property syst_weight: str

static var2cov(mat) → ndarray

vary_templates()

class sysvar.eigendecomposer.ExistingEigenVariationsSaver(df: DataFrame, settings: dict, verbose: bool = True)

Bases: ChannelTemplateHandler

property Nvar: int

property prefices: str | list | None

save_existing_eigenvariations(filepath=None, verbose: bool = True)

property weight: str

sysvar.fit_setup module

sysvar.fit_setup.save_existing_eigenvariations(df: DataFrame, analysis: str, systematic: str) → None

sysvar.saver module

exception sysvar.saver.MissingFilenameError

Bases: Exception

exception sysvar.saver.MissingMandatorySavingInfo

Bases: Exception

class sysvar.saver.PlotSaver(object_to_save, filename, saving_info)

Bases: Saver

Saves plot objects (e.g., figures) with additional handling for file extensions and logging.

Inherits from the Saver class and provides specific functionality for saving plot objects.

Parameters:

• object_to_save – The plot object (e.g., Matplotlib figure) to be saved.

• filename (str) – The name of the file to save the plot to.

• saving_info (dict) – Dictionary containing information on directories, namespaces, and file extensions.

saving_info

Contains both mandatory and optional fields related to saving.

Type:

dict

add_default_extensions()

Ensures default file extensions (png and pdf) are added if none are provided.

static get_key_description(key: str) → str

Provides a description of the specified saving info key.

Parameters:

key (str) – The key for which the description is requested.

Returns:

The description of the key.

Return type:

str

Example

>>> PlotSaver.get_key_description('top_dir')
'The top directory that your objects should be saved in'

log_missing_optional_field(key: str)

Logs a warning if an optional field is missing.

Parameters:

key (str) – The missing optional field key.

raise_missing_mandatory_field(key: str)

Raises an error if a mandatory field is missing.

Parameters:

key (str) – The missing mandatory field key.

Raises:

MissingMandatorySavingInfo – If the mandatory field is not found in saving_info.

save()

Saves the plot object in the specified directory with the given extensions.

Ensures default extensions are added, then loops over the extensions to save the figure.

Logs the location where the figure is saved.

class sysvar.saver.Saver(object_to_save, filename, saving_info)

Bases: ABC

Abstract base class for saving objects with mandatory and optional saving information.

Parameters:

• object_to_save – The object to be saved.

• filename (str) – The name of the file to save the object to.

• saving_info (dict) – Dictionary containing information about saving directories, namespaces, and other options.

saving_info

Stores both mandatory and optional fields related to saving.

Type:

dict

object_to_save

The object that needs to be saved.

save_dir

The directory path where the file will be saved.

Type:

str

filename

The formatted filename for saving the object.

Type:

str

check_if_dir_exists()

Checks if the directory exists; if not, creates the directory.

abstract log_missing_optional_field(key)

Abstract method for logging missing optional fields.

property mandatory_keys: list

List of mandatory keys required for saving.

property optional_keys: list

List of optional keys that can be provided for saving.

populate_mandatory_fields(saving_info: dict)

Populates the mandatory fields from the saving information.

Parameters:

saving_info (dict) – The dictionary containing saving information.

Raises:

The specific method raise_missing_mandatory_field if a mandatory field is missing. –

populate_optional_fields(saving_info: dict)

Populates the optional fields from the saving information.

Parameters:

saving_info (dict) – The dictionary containing saving information.

Logs missing optional fields using the log_missing_optional_field method.

abstract raise_missing_mandatory_field(key)

Abstract method for raising errors when mandatory fields are missing.

abstract save()

Abstract method to be implemented by subclasses for saving the object.

sysvar.templates module

exception sysvar.templates.NotADictError

Bases: Exception

exception sysvar.templates.NotCorrectVariableError

Bases: Exception

exception sysvar.templates.NotIncreasingBinning

Bases: Exception

class sysvar.templates.Template(df: DataFrame, binning: dict, total_weight: str, syst_weight: None | str = None, prefices: None | str | list = None, correction: None | BaseCorrection = None, variator: None | Variator = None, verbose: bool = True)

Bases: ABC, SavableAttributesObject

property Nbins: int

property Nvar: int

add_variations()

collect_weights(index)

Collects weights based on the provided index, handling different variations.

Parameters:

index – Specifies the type of weight to collect. Can be None, a string such as “MC”, “up”, “down”, or variations like “up0”, “up1”, …, “up8”, “down0”, “down1”, …, “down8”, or an integer.

Returns:

An array of weights based on the specified index.

Return type:

numpy.ndarray

Raises:

NotImplementedError – If the provided index is not supported.

Notes

• If index is None, returns the nominal total weight.

• If index is “MC”, returns the square of the total weight.

• For “up” and “down” variations, computes the weights with specific adjustments.

• Handles both string and dictionary types for self.syst_weight.

Example

>>> self.collect_weights(None)
array([...])

>>> self.collect_weights("MC")
array([...])

>>> self.collect_weights("up1")
array([...])

property corr_matrix: ndarray

property correction: BaseCorrection

property cov_matrix: ndarray

drop_unecessary_columns()

property eigen_decomposition: tuple

property eigen_values: ndarray

property eigen_variations: ndarray

property eigen_vectors: ndarray

abstract make_hist(index: None | int = None) → np.ndarray

plot_relative_variations_in_grid(title: str = '', save: bool = False, filename: str = '') → tuple[Figure, Axes]

plot_systematic_overview(save: bool = False, filename: str = '')

plot_up_and_down_variations(title: str = '', save: bool = False, filename: str = '') → tuple[Figure, Axes]

property prefices: str | list

property syst_weight: str

property variator: Variator

class sysvar.templates.Template1D(df: DataFrame, binning: dict, total_weight: str, syst_weight: str = None, prefices: str | list = None, correction: None | BaseCorrection = None, variator: None | Variator = None, verbose: bool = True)

Bases: Template

make_hist(index: None | int | str = None) → np.ndarray

class sysvar.templates.TemplateND(df: DataFrame, binning: dict, total_weight: str, syst_weight: None | str = None, prefices: str | list = None, correction: None | BaseCorrection = None, variator: None | Variator = None, verbose: bool = True)

Bases: Template

make_hist(index: None | int | str = None) → np.ndarray

sysvar.uncertainties module

exception sysvar.uncertainties.EmptyArrayError

Bases: Exception

class sysvar.uncertainties.ExplicitlyCorrelatedUncertainty(name: str, errors: np.ndarray, string_boundaries: List[str], explicit_cov_matrix: np.ndarray | None = None)

Bases: Uncertainty

build_correlation_matrix()

Build the correlation matrix of the explicitly correlated uncertainty.

Returns:

The correlation matrix of the uncertainty.

Return type:

np.ndarray

build_covariance() → ndarray

Build the covariance matrix of the explicitly correlated uncertainty.

Returns:

The covariance matrix of the uncertainty.

Return type:

np.ndarray

class sysvar.uncertainties.FullyCorrelatedUncertainty(name: str, errors: Iterable, string_boundaries: List)

Bases: Uncertainty

Represents a fully correlated uncertainty with a name and error values.

This class inherits from the Uncertainty base class and implements a fully correlated uncertainty with a correlation matrix of ones.

Parameters:

• name (str) – The name of the uncertainty.

• errors (Iterable) – An iterable containing the error values.

name

The name of the uncertainty.

Type:

str

errors

An array containing the error values.

Type:

np.ndarray

cov_matrix

The covariance matrix of the uncertainty.

Type:

np.ndarray

corr_matrix

The correlation matrix of the uncertainty.

Type:

np.ndarray

build_covariance() → ndarray

Build the covariance matrix of the fully correlated uncertainty.

Returns:

The covariance matrix of the uncertainty.

Return type:

np.ndarray

class sysvar.uncertainties.FullyCorrelatedUncertaintyInParts(name: str, errors: Iterable, string_boundaries: List, part_dimensions)

Bases: Uncertainty

Represents a fully correlated uncertainty only in parts with a name and error values.

This class inherits from the Uncertainty base class and implements a fully correlated uncertainty with a correlation matrix that is full of ones in some regions and completely uncorrelated in other regions

Parameters:

• name (str) – The name of the uncertainty.

• errors (Iterable) – An iterable containing the error values.

• dimensions (part =) – dimensions of correlated/uncorrelated parts

name

The name of the uncertainty.

Type:

str

errors

An array containing the error values.

Type:

np.ndarray

cov_matrix

The covariance matrix of the uncertainty.

Type:

np.ndarray

corr_matrix

The correlation matrix of the uncertainty.

Type:

np.ndarray

build_correlation_matrix()

build_covariance() → ndarray

Build the covariance matrix of the fully correlated uncertainty.

Returns:

The covariance matrix of the uncertainty.

Return type:

np.ndarray

exception sysvar.uncertainties.MultiDimArrayError

Bases: Exception

exception sysvar.uncertainties.NotAnArrayError

Bases: Exception

class sysvar.uncertainties.Uncertainty(name: str, errors: ndarray, string_boundaries: List)

Bases: ABC, SavableAttributesObject

Abstract base class for representing uncertainties with a name and error values.

Parameters:

• name (str) – The name of the uncertainty.

• errors (np.ndarray) – An array containing the error values.

name

The name of the uncertainty.

Type:

str

errors

An array containing the error values.

Type:

np.ndarray

cov_matrix

The covariance matrix of the uncertainty.

Type:

np.ndarray

abstract build_covariance() → ndarray

Abstract method to build the covariance matrix of the uncertainty.

Returns:

The covariance matrix of the uncertainty.

Return type:

np.ndarray

class sysvar.uncertainties.UncorrelatedUncertainty(name: str, errors: Iterable, string_boundaries: List)

Bases: Uncertainty

Represents an uncorrelated uncertainty with a name and error values.

This class inherits from the Uncertainty base class and implements an uncorrelated uncertainty with a correlation matrix of identity.

Parameters:

• name (str) – The name of the uncertainty.

• errors (Iterable) – An iterable containing the error values.

name

The name of the uncertainty.

Type:

str

errors

An array containing the error values.

Type:

np.ndarray

cov_matrix

The covariance matrix of the uncertainty.

Type:

np.ndarray

corr_matrix

The correlation matrix of the uncertainty.

Type:

np.ndarray

build_covariance() → ndarray

Build the covariance matrix of the uncorrelated uncertainty.

Returns:

The covariance matrix of the uncertainty.

Return type:

np.ndarray

exception sysvar.uncertainties.ValueError

Bases: Exception

sysvar.uncertainties.get_uncertainty_types()

sysvar.utils module

class sysvar.utils.SavableAttributesObject

Bases: object

A class for managing objects with savable attributes.

This class provides a mechanism to register and store saving information for various attributes.

saving_info

A dictionary to store saving-related information.

Type:

dict

register_saving_info(saving_info: dict)

Registers the saving information for the object’s attributes.

Parameters:

saving_info (dict) – A dictionary containing information about how attributes should be saved.

Example

>>> obj = SavableAttributesObject()
>>> obj.register_saving_info({'attribute_name': 'save_path'})

sysvar.utils.corr2cov(corr, var)

Calculates the covariance matrix from a given correlation matrix and a variance vector.

Parameters:

• corr (np.ndarray) – Correlation matrix of shape (n,n).

• var (np.ndarray) – Variance vector of shape (n,).

Returns:

out – Covariance matrix. Shape is (n,n).

Return type:

np.ndarray

sysvar.utils.cov2corr(covariance)

Compute the correlation matrix from the given covariance matrix.

Parameters:

covariance (numpy.ndarray) – The covariance matrix.

Returns:

The correlation matrix.

Return type:

numpy.ndarray

sysvar.utils.load_covariance_matrix(config: dict, key_matrix: None | str = None, key_path: str = 'cov_matrix_path') → ndarray | None

Load a covariance matrix from a configuration dictionary.

This function attempts to load a covariance matrix either directly from the configuration using a specified key or from a file path (supporting .npy or CSV files).

Parameters:

• config (dict) – Configuration dictionary that may contain the covariance matrix or a path to the matrix file.

• key_matrix (Optional[str], optional) – Key in the config dictionary where the matrix might be directly stored. Defaults to None.

• key_path (str, optional) – Key in the config dictionary where the file path to the covariance matrix is specified. Defaults to “cov_matrix_path”.

Returns:

The loaded covariance matrix as a NumPy array if found, otherwise None.

Return type:

Optional[np.ndarray]

sysvar.utils.read_yaml(cfg_name: str, deeper_dir: str = '')

Reads a yaml file file from the configuration directory of the repository and and returns a dictionary with the yaml data

Args: cfg_name: Name of the yaml file which is located in the configs directory deeper_dir: Specify the directory within the configs directory.

Returns: Dictionary read from the yaml file

sysvar.variations module

exception sysvar.variations.MissingSavingInfo

Bases: Exception

class sysvar.variations.Variator(correction: BaseCorrection, Nvar: int = 20, seed: int = 8311311)

Bases: ABC, SavableAttributesObject

Abstract base class for generating variations on a correction. Then the variator will create a big covariance matrix of all the uncertainties. Through the get_variations_from_uncertainty method, one can examine the effect of the variations coming from a specific uncertainty.

Parameters:

• correction (BaseCorrection) – The correction object.

• Nvar (int) – The number of variations to be generated. Defaults to 20.

correction

The correction object.

Type:

BaseCorrection

uncertainties

A dictionary to store uncertainties associated with the correction.

Type:

dict

property corr_matrix: ndarray

property cov_matrix: ndarray

Build the total covariance matrix from all added uncertainties. Here all the covariance matrices from all uncertainties are summed up to a total covariance matrix Return the covariance matrix from one uncertainty type of only one is present

Returns:

The total covariance matrix.

Return type:

np.ndarray

generate_variations(Nvar: int, covariance: ndarray, seed: int = 8311311) → ndarray

Generate variations based on a covariance matrix. This is using a standard multivariate normal.

Parameters:

• Nvar (int) – The number of variations to generate.

• covariance (np.ndarray) – The covariance matrix.

Returns:

An array of variations.

Return type:

np.ndarray

get_correction_variations(seed: int = 8311311) → ndarray

Get variations on the correction. This adds the generated variations from the total covariance matrix to the nominal values of the correction.

Parameters:

Nvar (int) – The number of variations to generate.

Returns:

An array of correction variations.

Return type:

np.ndarray

get_variations_from_uncertainty(Nvar: int, name: str, seed: int = 8311311) → ndarray

Helper function to inspect variations coming from a single source of uncertainty. Creates new variations but likely this is okay as we’re interested in examining the sources qualitatevely.

Parameters:

• Nvar (int) – The number of variations to generate.

• name (str) – The name of the uncertainty.

Returns:

An array of variations from the specified uncertainty.

Return type:

np.ndarray

plot_cov_and_corr(save: bool = False, filename: str = '') → tuple[Figure, Axes]

plot_gaussian_variations(save: bool = False, filename: str = '')

plot_relative_variations_in_grid(nbins: int = 41, save: bool = False, filename: str = '') → tuple[Figure, Axes]

property relative_variations: ndarray

property variations: ndarray

sysvar.visualize module

class sysvar.visualize.CorrectionVisualizer(instance: BaseCorrection)

Bases: Visualizer

annotate_matrix_plot(ax: Axes)

plot_corr_matrix()

plot_cov_matrix()

plot_error_comparison(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

class sysvar.visualize.EigenDecomposerVisualizer(instance: EigenDecomposer)

Bases: Visualizer

annotate_matrix_plot(ax: Axes)

plot_corr_matrix(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_cov_diff(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_cov_matrix()

plot_eigenvalues(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

exception sysvar.visualize.NoMatrixError

Bases: Exception

class sysvar.visualize.TemplateVisualizer(instance: Template)

Bases: Visualizer

annotate_matrix_plot(ax: Axes)

plot_corr_matrix(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_cov_matrix(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_eigenvalues(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_nominal_template(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_relative_variations_in_grid(fig: plt.Figure | None = None, ax: plt.Axes | None = None, title: str = '', nbins: int = 11, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_systematic_overview(save: bool = False, filename: str = '')

plot_up_and_down_variations(fig: plt.Figure | None = None, ax: List[plt.Axes] | None = None, title: str = '', save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_variations(Nvar: int = 5, save: bool = False, filename: str = '')

class sysvar.visualize.UncertaintyVisualizer(instance: Uncertainty)

Bases: Visualizer

annotate_matrix_plot(fig: Figure, ax: Axes)

plot_corr_matrix(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_cov_matrix(ax: plt.Axes | None = None, save: bool = False, filename: str = '')

class sysvar.visualize.VariatorVisualizer(instance: Variator)

Bases: Visualizer

annotate_matrix_plot(fig: Figure, ax: Axes)

plot_corr_matrix(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_cov_matrix(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

plot_gaussian_variations(save: bool = False, filename: str = '')

Plot Gaussian variations of the corrections.

Args:

Returns:

None

plot_relative_variations(value_edges: Iterable, Nvar: int = 5, save: bool = False, filename: str = '') → tuple[Figure, Axes]

Plots the relative variations of the templates. The Nvar argument specifies the number of variatios that will be plotted. Defaults to 5.

Parameters:

Nvar (int, optional) – The number of variations to visualize.

Returns:

A tuple containing the figure and axis objects.

Return type:

Tuple[plt.Figure, Axis]

plot_relative_variations_in_grid(fig: plt.Figure | None = None, ax: plt.Axes | None = None, nbins: int = 41, save: bool = False, filename: str = '') → tuple[plt.Figure, plt.Axes]

property strings: list

class sysvar.visualize.Visualizer(instance: BaseCorrection)

Bases: ABC

abstract annotate_matrix_plot()

available_plots()

create_single_figure()

abstract plot_corr_matrix()

plot_cov_and_corr(fig: plt.Figure | None = None, ax: plt.Axes | None = None, save: bool = False, filename: str = '')

abstract plot_cov_matrix()

static plot_variation_on_axis(ax: plt.Axes, x: np.ndarray, variation: np.ndarray, index: None | int = None, plot_func: str = 'step', save: bool = False, filename: str = '')

Plots a variation on a given axis. The absence of a value for the index arguments shows that this is a nominal template. The function creates the correct labels, colors and linestyle based on the value of the index.

Parameters:

• ax (plt.Axes) – The axis to plot on.

• x (np.ndarray) – The x values for the plot. Should be the bin mid values as the method makes use of the matplotlib’s steps method

• variation (np.ndarray) – The variation values to plot.

• index (int, None) – The index of the variation (None for nominal).

• plot_func (str) – name of the matplotlib.pyplot function to use for the plot

Returns:

None

save_figure(figure, filename: str)

sysvar.visualize.get_latex_symbol(key)

Module contents