DeepEdit and PET Interface Module (DE-PET)

Introduction

The DE-PET interface allows the creation, editing, execution and visualization of PET cases by using the visual tools of DeepEdit. The DE-PET interface was developed in Java so, like DeepEdit, it is a cross-platform interface that does not depend on a particular operating system. In general terms, the DE-PET interface allows you to:

Prepare a PET case from a DeepEdit Schematic.
Make modifications from the DeepEdit interface to an existing PET case.
Create the files required by the PET for its execution.
Create visual tabular reports, graphs, and documents from PET simulation results.

Interface structure

The interface that manages the creation and editing of PET input cases was developed as an implementation of the DeepEdit interfaces for editing "edil" dispatch software. The results display interface was developed as an implementation of the "vsi" interface. The fundamental diagram of the DeepEdit modules is shown in the figure below.

Figure 124: DeepEdit Modular Diagram: Packages for the DE-PET Interface

The components of the DE-PET interface are as follows:

Figure 125: Structural components of the DE-PET interface

The different components of DeepEdit and PET, as well as the interface components, are described in the table below.

Component	Description	Specific functions DE-PET
DeepEdit	Competitive electricity market analysis tool.	The interface allows editing schematics, modifying the main components of the power system, hydro system and market. It also allows creating, editing and visualizing PET constraints through its constraint builder interface.
PET	Power system planning tool.	Executes long-term planning simulations. It creates and solves the problem using mixed-integer programming. PET reads data from a text file (called "input.txt" although its name is user-configurable) and writes the results to a series of csv files.
DE-PET Manager	A series of graphical tools that facilitate interaction with PET input data. These tools are integrated with the DeepEdit interface.	Visualize, configure, and validate data for the PET case.
DE-Solver	A set of libraries developed in Java that allow interaction between the DeepEdit graphical and data interface and the PET input files (intermediate spreadsheet and input files). These tools are integrated with the DeepEdit interface.	Allows the creation and validation of data from the input.xls spreadsheet. This Java module also allows the validation and creation of text files in the format required by the PET software.
DE-vsi	Series of graphical tools that facilitate the interaction with the PET output files (results). These tools are integrated with the DeepEdit interface.	It allows configuring, reading and displaying graphical and tabular reports from PET output files.

The files that are part of the DE-PET interface can be classified into 3 groups:

Input: these are the files that contain case study specific data.
Configuration: These are files that configure the behavior and default values of the tool.
Interface: These are communication files auto generated by the different modules. These files are generated and read by different modules, so they should not be manually modified by the user.

The table below describes the files shown in Figure 125:

File	Type	Description	DE-PET specific functions
Schematic.sim	Input	DeepEdit file containing the schematic information. Its name and location are user configurable.	Initializes the cases. Once the PET case is created, DeepEdit saves the Schematic information in the Input.xlsm file.
Input.xlsm	Input	This file is also called "Intermediate" spreadsheet because it contains all the information of the PET case, in a "friendly" tabular format. Its name and location is user configurable.	Input data that change over time and data that are specific to the PET (e.g. firm power data). It also allows visualizing all the sets and tables that make up the PET case data.
PET_Model.txt	Configuration	GAMS nomenclature definition file used by PET for the definition of all Sets and Parameters. This file must not be modified by the user (only changes per DE-PET version)	Defines nomenclature and structure of the file created by DeepEdit.
PET-DE_mapping.xml	Configuration	“Map" file of the location of the Sets and Parameters in the intermediate template. This file must not be modified by user (only changes per DE-PET version).	Defines the location of the SETs and Parameters defined in the PET_Model.txt file in the intermediate spreadsheet PET_input.xlsm.
PET_input.txt	Interface	PET input GAMS formatted text file. It is automatically generated in its entirety by DeepEdit from the information in the intermediate sheet Input.xlsm. Its name and location is user configurable.	PET input data (case) file.
gamsconfig.txt	Configuration	File with GAMS configuration parameters to be used by PET. It is automatically generated in its entirety by DeepEdit from the information in the intermediate Input.xlsm spreadsheet.	Defines the GAMS configuration parameters to be used by PET.
report.txt	Configuration	File defining the reports (output files) to be printed by PET at the end of the simulations. This configuration file is automatically generated in its entirety by DeepEdit from the information in the intermediate Input.xlsm spreadsheet.	It defines the reports to be delivered by PET simulations.
PET_result.csv	Interface	Set of PET output files. The file name and location is user configurable.	These files constitute the input data for the PET result visualization tools in DeepEdit.
PET-vsi_mapping.xml	Configuration	File defining the visualizations (graph types, axes, titles, etc) for each of the PET output files.	Configures the display of PET results through the DeepEdit interface.

Workflow

The figure below shows the typical workflow for the creation, execution, and visualization of cases in PET using the DeepEdit interface.

Figure 126: Proposed workflow for PET case creation, execution and visualization

The case editing procedure, which may represent the most frequent use of the DE-PET interface, is proposed through the combined use of both MS EXCEL and DeepEdit interfaces. The proposal is based on using the benefits of both programs. On one side, Excel through its sophisticated set of tabular editing tools allows massive data entry over time. For example, some long-term entries such as nodal demands, inflow flows, fuel projection, scheduled maintenance, etc. On the other hand, DeepEdit's visual tools allow for easier and safer structural changes. For example, the addition, editing or removal of power system elements such as generators, lines or busbars, or modification or definition of mathematical relations (any of the PET constraints) or topological relations of the transmission or hydrological system, etc. In this case, DeepEdit facilitates the complex modifications that these changes require on the multiple data set matrices and tables in the intermediate Input.xlsm spreadsheet to maintain the consistency of the planning case.

Usage tutorial

Creating a new case

To create a new case in PET, using the DE-PET interface, it is always suggested to do it from a schematic. Although the interface allows you to prepare the empty "input.xlsm" data entry form, this operation is not recommended to start the creation of a new case. The suggested procedure is to create (first) the power system (generators, lines, topology, hydro system) in the DeepEdit interface and then create the PET case (next step). The schematics can be saved temporarily or permanently in text files. Figure 127 shows the menu for opening DeepEdit schematics.

Figure 127: Opening an existing schematic file

It should be noted that schematics can be stored in 2 types of files: text (usually with the extension ".sim") and Excel. Moreover, the intermediate Input.xlsm spreadsheet contains a schematic (on a sheet called "schematic") that can be used normally as a DeepEdit schematic or to initialize a new PET blank case.

Once a unilinear is created or loaded, the New Case Creation Wizard is executed.

Figure 128: Executing the Wizard for creating a new case

The Wizard window facilitates the various steps required for case creation.

The first step defines the new case. A schematic can store different objects of type "Simulation". These objects store simulation configurations of various types (including PET cases). A new case can be created from an existing Simulation. It can be seen from the list of components in the example in Figure 129 that there are no existing Simulation components, so the "Create Case from Existing Simulation" option should provide a blank list.

Figure 129: Wizard case creation: Creating a new case

The second step allows defining the fundamental data of the case:

Name: This field will be used both for naming the Input.xlsm file to be created and for the PET "Case" field.
Destination: The Input.xlsm intermediate spreadsheet will be stored in this folder. By default it is stored in DEEPEDIT_FOLDER\database\
Reference: (optional) User can define a reference case (Input.xlsm template). This case will inherit the values of some SET and Parameter (configured in the configuration file PET-DE_mapping.xml).

Figure 130: Wizard case creation: Fundamental case information

The third step defines the configuration of the simulation. That is, the most important parameters of the simulation:

Periods: Usually the years of the study. This is the PET Set "a".
LDCs: Usually the divisions of the year (e.g., months, quarters, semesters, weeks). This is the PET Set "p".
Blocks: The block divisions of the duration curves (usually representing hours of the day or segments of the duration curve). This is PET Set "b".
Hydro Samples: Hydrologies. In case of being deterministic place one entry with any name (eg. "deterministic").
Initial conditions: These are the sub-sets that define the start/end of the simulations and the beginning of the hydrological period.

It is important to note that the procedure to fill these Sets should be as follows:

Enter the size of the set of values.
Call the Sets editing window […]
Enter the values and click OK. This window does not have "Cancel". The values are entered by pressing "enter" in each field. Closing the window will only close the editor, the values entered will persist.

Note: Going backwards will reset all the information entered so far. The user will have to enter the values again.

Figure 131: Wizard case creation: Configuration of simulations

The fourth step corresponds to the optional configuration of PET type constraints. It also allows defining the future cost function (FCF), both its dimensions and values (the values are optional, they can be entered later from the intermediate Input.xlsm spreadsheet).

A 10-span FCF called "MyFCF" is exemplified in Figure 132.

All PET constraints can be constructed from the Wizard. The Edit button allows editing the constraints using DeepEdit's Constraint Builder.

Generic constraints will be ignored.

Figure 132: Wizard case creation: Creation of PET and FCF constraints

The advanced configuration of the simulations refers to the modification of Sets and Parameters inherited during the creation of the case from a reference (step 2).

Figure 133: Wizard case creation: Advanced configuration

The last step corresponds to the completion confirmation. The DE-PET interface will perform the following steps and print a confirmation of execution in the "PET case" tab of the Output section:

Create an intermediate Excel spreadsheet with the case name in the location noted in step 2.
Register a Simulation object in the current schematic.
Enter the value of all the Sets and Parameters defined in the Wizard.
Save the current schematic in the Excel file.

Figure 134: Wizard case creation: Completion

Opening an existing case

Existing cases must (always) be opened from the Input.xlsm spreadsheet.

Figure 135: Opening an existing PET case from DeepEdit

The Input.xlsm spreadsheet stores the schematic information required by DeepEdit to display the unilinear.

Include constraints

One of the main functions of the DE-PET interface is the assistance to create/edit/remove PET constraints to the different cases. Making modifications is easy through DeepEdit's Constraint Builder. The Constraint Builder can be invoked from an existing constraint (double click on the "Constraint" object) or when creating a new constraint.

Figure 136: Constraint builder: PET "rea" constraint (annual energy generated)

The use is intuitive:

Name the constraint. In the example "ERNC2025".
Select the category: All PET constraints are supported by the interface. Please refer to the PET manual for more details. In the example, the category corresponds to the "rea" constraints of PET (set of annual energy constraints of generator sets). The type of constraint is, in the case of PET constraints, controlled by its category.

Note: Changing the category of a constraint is not allowed. User must enter the constraint again.

Update excel from schematic

There are 2 options for editing an existing PET case:

DeepEdit: Edition to the schematic. Using any of the DeepEdit graphical tools (see previous sections).
Editing the Input.xlsm spreadsheet: Direct modification of the input file. However, the DE-PET interface can modify several input values from schematic.

Changes made to the schematic entered through DeepEdit GUI are directly updated in the input.xlsm spreadsheets by clicking on ”File/PET Study Manager/Save”.

Figure 137: Save changes in schematic in Input.xlsm spreadsheet

Note: Click on File/Save would synchronize from schematic to Excel.

It is easy to identify fields that are controlled by DeepEdit or that are directly entered in the intermediate spreadsheet:

All parameters available in DeepEdit will be sent to the Input.xlsm spreadsheet.
The columns in the spreadsheet sheets, identified with blue background, are controlled by DeepEdit.
Columns of another color (usually with PET's identifying colors - green background with red font) are entered directly into the Input.xlsm spreadsheet.

Figure 138: Input.xlsm spreadsheet: Fields updateable from DeepEdit (in blue)

Updating the schematic from excel

The DE-PET interface allows updates to the schematic from the Input.xlsm spreadsheet. This is done through the synchronization tool in File/PET Manager.

Only parameters can be updated (Sets are excluded).
Only the information in the DeepEdit controllable columns (in blue in the Input.xlsm spreadsheet) can be read and entered into the Schematic.
Likewise, only the editable cells in the Input.xlsm spreadsheet. For example, changing the bar of a generator can only be done in the schematic (via the DeepEdit interface).

Figure 139: Update values in Schematic from Spreadsheet Input.xlsm

Create PET files

Following the workflow suggested in Figure 126, the step that follows the data entry of the case in the Input.xlsm spreadsheet (from Excel) corresponds to the validation and creation of PET input files. That is, the case can be validated and a text file in GAMS format ready for PET execution can be generated.

Figure 140: Creating PET case files

The DE-PET interface will ask for the path (file name and location) of the text file with the PET input data. As shown in the component schematic in Figure 126, the interface also creates the solver and report configuration files in the same location as the input file.

Note: If there are validation errors, the PET input file creation process will be interrupted. The user must correct validation errors to continue.

Read results

The DE-PET interface implements the DeepEdit reporting specification "Visual Solution Importer" (VSI). This interface defines how DeepEdit presents results from output files from other dispatching software. In this case, a solution was implemented to display the following PET files:

Name in DeepEdit	Title (graph or table)	PET csv file	Report description
Offer-Demand	Total supply and demand	PET_Sal_DdaPInstCMg_new.csv	Total supply and demand
Installed Capacity	Annual Installed Capacity by Technology	PET_Sal_GenPInst.csv	Installed capacity
Expected Generation	Annual Expected Capacity by Technology	PET_Sal_GenEsp.csv	Annual Expected Generation
Dispatch	Dispatches by technologies	PET_Sal_Despacho.csv	Dispatches by technologies
Flow	Chronological Flows	PET_Sal_Flujos.csv	Chronological Flows
Load	Systemic demand	PET_Sal_Dem.csv	Systemic demand
Load Rate	Annual demand growth rate	PET_Sal_Dem.csv	Annual demand growth rate
Generation Profile	Generation Profiles	PET_Sal_Despacho.csv	Generation Profiles

To read the results of PET simulations, the DE-PET interface implements a result reading Wizard that facilitates the creation of the visual report. The Wizard is executed from the shortcut in the File/PET Solution Manager menu.

Figure 141: Executing Wizard for visualization of results

The Wizard helps the user to define the following entries:

Type of solution: In this case obviously the solution comes from the PET software.
Path: In the case of PET, it is sufficient to define the directory (complete path to the folder) where the csv output files are located.

Figure 142: Wizard PET results: PET output csv files folder selection

Graphical reports

DeepEdit VSI reports always follow the following guidelines:

The report opens in an additional tab in the "Database View" section. The name of the tab will be pet-vsi_Results.
The report view is divided into 2 panes:
- Left: Graph
- Right: Tabular
The tabular section header allows the user to:
- Change the report type (performs a read of all files. This operation may take a few minutes, depending on the size of the csv file).
- Select the value of the active filters.
- Sort the values alphabetically in each of the columns.
The menu of the graph section allows the user to:
- Change the chart type: Area, Line, Column, Column Stacked, Pie.
- Change the chronology of the results: Chronological (as presented in the PET report) and duration (or cumulative probability) curves from highest to lowest.
The tabular section presents the following fields (columns):
- Domain: These are the values of the X-axis. If the axis is composed of several fields, the value of each cell will correspond to the concatenation of the values. For example, the "Offer Demand" report of the csv file "PET_Sal_Dda-PInstCMg_new.csv", is configured to present on the X axis the values of the Year and Period Sets (ie. PET’s SETs “a” and “p”), separated by space: See column "Domain" in Figure 141.
- Series: These are the values of the fields that determine the series.
- Range: Value (necessarily numerical) that each of the series has for the points of the domain indicated.
The graphic section is always coordinated with the tabular section. This means that report selection and filters in the table header will always be reflected in the graph.
The user can select rows and make a copy (by typing "ctrl+c"), which will create a copy of the selected values to the clipboard. The values passed to the OS clipboard have a configuration that allows them to be pasted to Excel in columns and rows.

Figure 143: VSI results interface: PET execution "Offer Demand" Report

The report configuration is defined in the file "PET-vsi_mapping.xml" and is fully configurable by the user (advanced). The current version of the DE-PET interface is distributed with the following graphical configuration:

Csv file	Primary Axis	Secondary Axis	X-axis
	Chart type	Field	Series	Chart type	Field	Series	Field(s)	Filter(s)
PET_Sal_Dda-PInstCMg_new.csv	AreaStack	DemMax, PInst	-	Line	CMgProm-PondxDda	-	Year, Period	System
PET_Sal_GenPInst.csv	AreaStack	PInst	tec				a
PET_Sal_GenEsp.csv	AreaStack	Pgen	tec				a
PET_Sal_Despacho.csv	AreaStack	PGen	Name	-	-	-	Block	Year, Period, Hydrology
PET_Sal_Flujos.csv	Line	Flow	Hydrology			Year, Period, Block	Circuit NodeA, NodeB
PET_Sal_Dem.csv	Line	Demand	zg				b	a,p
PET_Sal_Dem.csv	Line	Rate	zg				a
PET_Sal_Despacho.csv	Line	PGen	Period	-	-	-	Block	Year, Hydrology, Name

The names of the output fields in the above table (csv file column names) are assigned by PET.

The user can select different values of the filters from the header of the tabular section. In the case of the "Dispatch" report, Figure 141 shows the 3 fields configured as filters of the report: Year (the first one is shown: "2015), period (the first one is shown: "trim1") and hydrology (the first one is shown: "HSeca").

Figure 144: VSI results interface: PET execution "Dispatch" Report

Execution summary

The pet-vsi implementation allows the display of the simulation summary file " PET_Sal_Resumen.csv". DeepEdit displays the contents of the file by means of its own zfileviewer text editor/viewer.

Figure 145: VSI results interface: Selection of the run summary

Figure 146: VSI results interface: Run summary in DeepEdit

Loading a PET dispatch solution

The DE-PET interface implements a module that allows loading a PET dispatch solution into an existing schematic.

Figure 147: VSI results interface: Import PET dispatch solution

The implementation of the importer follows the vsi specification, so its operation follows the VSI usage guidelines (see Visual Solution Import (VSI)). To load the results from PET, the following fields are required:

Folder: Select the complete path to the folder with the PET results.
Year: Year of the simulation (PET set value "a").
Stage: Period or stage of the simulation (PET set value "p").
Block: Block of the period (PET set value "b").

Wiki

deepedit35 / DeepEdit and PET Interface Module (DE-PET)