DataManagement

class pomato.data.DataManagement(options, wdir)

The DataManagement class provides processed data to all other modules in POMATO

This is done by managing the data read-in, processing and validation of input data and facilitation of results.

Parameters:

wdirpathlib.Path

Working directory, proliferating from POMATO main module.

optionsdict

The options from POMATO main method persist in the DataManagement.

Attributes:

wdirpathlib.Path

Working directory

optionsdict

The options from POMATO main method persist in the DataManagement.

data_structuredict

data_structure contains the structure of the input data including the name i.e. table like nodes and its attributes e.g. zones, lat/lon etc

model_structuredict

Similarity data_structure, model_structure defines the data necessary to run the market model and its attributes. This is used to verify the processed input data in this module.

data_attributesdict

All input and model data is available as an attribute of this method. data_attributes is used to initialize them and to track if this data is loaded from file as indicated by the dict{attr, bool}.

resultsdict(str, Results)

The Results module allows to attach and process results from the market model alongside the DataManagement module. This ensures consitency of the data and results. The dict results can store multiple results.

Methods

load_data(filepath)	Load Data from dataset at filepath.
validate_inputdata()	Validate input data and compare it to the predefined data structure.
validate_modeldata()	Validate the processed input data to be conform with predefined model data structure.
process_input()	Input Processing to bring input data is the desired pomato format.
load_model_structure()	Load model structure as part of init.
save_data(filepath)	Write Data to excel file.
return_results([title])	Interface method to allow access to results from Results.
process_inflows()	Process inflows to (hydro-) storages.
process_results(result_folder, grid)	Initialize Results with results_folder and the own instance.
set_default_net_position(net_position)	Add default net position.

load_data(filepath)

Load Data from dataset at filepath.

Currently .xls(x) and .mat work and filepath with or without ext. After the raw data is read it is processed in the process_input method of this module.

Parameters:

filepath: pathlib.Path

Filepath .mat or .xlsx file. There are some path arithmetics to catch errors but not a lot.

load_model_structure()

Load model structure as part of init.

process_inflows()

Process inflows to (hydro-) storages.

If no raw data create an all zero timeseries for all electric storage (plant_type es) power plants

process_input()

Input Processing to bring input data is the desired pomato format.

This is done in a separate module InputProcessing and should result in a dataset as defined in the model_structure attribute. The instance of InputProcessing does not persist as it only processes the data one.

The processed data gets validated based on the predefined input and model data structures.

process_results(result_folder, grid)

Initialize Results with results_folder and the own instance.

return_results(title=None)

Interface method to allow access to results from Results.

save_data(filepath)

Write Data to excel file.

This methods provides a way to save a working data set, which then can be imported without processing.

Parameters:

filepath: pathlib.Path

Filepath to an .xls or .xlsx file, should include file extension.

set_default_net_position(net_position)

Add default net position.

validate_inputdata()

Validate input data and compare it to the predefined data structure.

This method validates the input data and checks whether it fits the predefined structure. This makes it easy to find errors and nan values in the data.

Input data is generally defined by:

• table/name/data: e.g. nodes, the general data type which always has an index.

• attributes: e.g. zone of a node, attributes of the data.

• type: the type can be int/float etc or the index of another table. E.g. zone is of type zone.index, so the index of another table.

• optional: bool to indicate whether a nan/missing value is an error. E.g. the attribute name is optional, as some nodes might not have a specific name assigned and are solely identified by the index.

Therefore, this method check for all data if its required attributes are available and reports missing ones in missing_data. Additionally it checks the values for each attribute to be of the defined type and if it refers to another data’s index, checks for consistency. Invalid values are removed in this process and reported. This can indicate a problem with the input data, but also wanted (in most cases).

validate_modeldata()

Validate the processed input data to be conform with predefined model data structure.

Analogues to the validate_inputdata() method, this methods checks the consistency of the input data. In contrast to validate_inputdata() this method compares the processed data to the predefined model structure and adds default values or empty tables to allow different data sets run in POMATO.

For example: the IEEE118 case study does not contain heat demand or generation, also are there no timeseries to cover adjacent regions via net export parameters. So the input data can cover a subjet of possible data set but the model data has to be consistent.