pomato.tools¶

Collection of tools potentially used by multiple components of POMATO.

This collection is characterized by a certain degree of generality and they cannot be attributed to a specified component of pomato.

Classes

JuliaDaemon(logger, wdir, package_dir, ...)

Class to communicate with a julia daemon process.

Functions

`add_default_options`(input_options)	Takes the loaded option dict and adds missing values from default options.
`copytree`(src, dst[, symlinks, ignore])	Copy folder from src to dst.
`create_folder_structure`(base_path[, logger])	Create folder structure to run POMATO.
`default_options`()	Returns the default options of POMATO.
`newest_file_folder`(folder[, keyword, ...])	Return newest (n) folders/files from a folder.

class pomato.tools.JuliaDaemon(logger, wdir, package_dir, julia_module, solver)¶

Class to communicate with a julia daemon process.

The RedundancyRemoval and MarketModel processes are written in Julia. This class’s purpose is to communicate with a daemon process in julia that runs these processes on demand and allows to execute them multiple times without restarting the julia process, which would require lengthy precompile everytime instead of one lengthy precompile.

This is implemented by two daemon processes, one in python the other in julia, or more specifically a threaded julia daemon is initialized in python. This allows start-up of the julia process while other pomato related processes are done. The communication is done through a json file in the data_temp/julia_files folder.

Attributes:

daemon_filepathlib.Path: Path to json file for process management.
julia_daemonThread with julia subprocess.: Threaded subprocess of the julia daemon process.
julia_modulestr: Defines whether RedundancyRemoval or MarketModel is initialized/used.
julia_daemon_pathpathlib.Path: Description
solvedbool: Indicator if julia process has successfully concluded.
wdirpathlib.Path: Workingdirectory, should be pomato root directory.

default_daemon_file()¶

Return default daemon file

Returns:

filedict: daemon file as dictionary.

disable_multi_threading()¶: Disable multithreading.

halt_until_ready()¶

Halt python main thread until julia is initialized.

Julias startup time can be fairly long, therefore when a julia process is started it has to be ready before a model can be run. This method halts the main thread until julia is ready.

halt_while_processing()¶

Halt python main thread, while julia is processing.

Sometimes its better for the user to wait until julia is done.

property is_alive¶: Check if the julia process is alive.

join()¶: Exit the julia daemon and join python threads

read_daemon_file()¶: Read daemon file from disk

run(args=None)¶

Run julia process. Writes commands to daemon file, initiating start of a process. Halts while it is active and set attribute solved as true.

Parameters:

argsdict, optional: Dictionary with pairs of values to change in the daemon file.

start_julia_daemon()¶: Stat julia daemon

write_daemon_file(file)¶: Write (updated) file to disk

class pomato.tools.TimeSinceLastLog¶

A handler class which stores custom logrecord attribute for last log.

emit(record)¶

Do whatever it takes to actually log the specified logging record.

This version is intended to be implemented by subclasses and so raises a NotImplementedError.

pomato.tools.add_default_options(input_options)¶

Takes the loaded option dict and adds missing values from default options.

Uses function that are a result from https://stackoverflow.com/a/14692747.

Parameters:

input_optionsdict: Optionfile from user input.

pomato.tools.add_default_values_to_dict(value_dict, default_dict)¶

Combines values from user dict with default values from default_dict.

Parameters:

value_dictdict: Dict with values.
default_dictdict: Dict containing default values that are added if not present in value_dict.

pomato.tools.copytree(src, dst, symlinks=False, ignore=None)¶

Copy folder from src to dst.

Utilizes the shutil.copytree function. Is based on https://stackoverflow.com/a/12514470

Parameters:

srcstr/pathlib.Path: Path to folder that is copied.
dststr/pathlib.Path: Destination path of the copied folder.
symlinksbool, optional: copytree option
ignoreNone, optional: copytree option

pomato.tools.create_folder_structure(base_path, logger=None)¶

Create folder structure to run POMATO.

Since the repository does not contain the empty folders for temporary data, this function checks whether these exist and creates them if necessary. This is only valid if the process is run from the pomato root folder, here checked by looking if the pomato package folder exists.

Parameters:

base_pathpathlib.Path: Pomato root folder.
loggerlogger, optional: If a logger is supplied the status messages will be logged there.

pomato.tools.default_options()¶: Returns the default options of POMATO.

pomato.tools.newest_file_folder(folder, keyword='', number_of_elm=1)¶

Return newest (n) folders/files from a folder.

Platform sensitive function to return the last file generated in a specified folder.

Parameters:

folderpathlib.Path: Folder to look for files in.
keywordstring, optional: A supplied string reduces the number of possibilities and makes it more robust.
number_of_filesint, optional: Specify the number of files or folders returned.

pomato.tools.print_timestep(start_time, logger, message='')¶

Print seconds passed from reference timestep.

Parameters:

start_timedatetime.datetime: Reference timestep.
loggerlogging.Logger: Logger to print the message.
messagestring, optional: Append Message.

pomato.tools.reduce_df_size(df)¶: Reduce size of DataFrame by assigning adequate data types.

pomato.tools.remove_duplicate_words_string(words)¶: Removes duplicate words from string.

pomato.tools.remove_empty_subdicts(old_dict)¶

Removing all empty subdicts.

Based on https://stackoverflow.com/a/33529384. A dictionary is empty when values are empty string, None, {} or [].

pomato.tools.remove_unsupported_chars(line)¶: Remove string from julia log, that are not supported by python logging.