Importing csv timeseries into MODSIM using custom coding

Issue #86 resolved
SJ Kim created an issue

Hello,

I am trying to import csv data into MODSIM non-storage node and MODSIM maximum link capacity (haven’t attempted setting timeseries data as maximum link capacity). Some of the import functions (setData, setDataTable) in MODSIM has been discontinued but are not marked in the manual, which makes it hard to select the correct function to import timeseries data.

I have tried the suggested line of code in Issue#69: Dim m_TSTable As DataTable = m_StephenNode.m.adaInflowsM.DataTable()
However it gives an error in compilation.

How can I set imported timeseries data into a Modsim non-storage node or link? Which functions should I be using? Please provide the code to import timeseries data into a non-storage node and as a maximum link capacity.

The attached are:

  1. trial code to import "lake_sd.csv" file’s 1_lake_inflow timeseries into Modsim node "node1".
  2. example.csv (csv containing timeseries)
  3. example_bitbucket.xy (importing Modsim model)

Thank you

'Import required libraries
Imports Csu.Modsim.ModsimIO
Imports Csu.Modsim.ModsimModel
Imports Csu.Modsim.NetworkUtils
Imports System.Data
Imports System.Collections    
Imports System.Data.OleDb
Imports System.IO

Module Module1

    Dim myModel As New Model
    Sub Main(ByVal CmdArgs() As String)
        Dim FileName As String = CmdArgs(0)
        AddHandler myModel.Init, AddressOf OnInitialize
        AddHandler myModel.IterBottom, AddressOf OnIterationBottom
        AddHandler myModel.IterTop, AddressOf OnIterationTop
        AddHandler myModel.Converged, AddressOf OnIterationConverge
        AddHandler myModel.End, AddressOf OnFinished
        AddHandler myModel.OnMessage, AddressOf OnMessage
        AddHandler myModel.OnModsimError, AddressOf OnMessage

        XYFileReader.Read(myModel, FileName)
        'Enable Hydropower Add-on
        'myModel.hydro.IterativeTechnique = IterativeSolutionTechnique.SuccApprox
        'myodel.hydro.SetHydroLinkCosts(10000)
        Modsim.RunSolver(myModel)
    End Sub

    Function getcsvFiles() As DataTable
        'The purpose of this function is to read and import WATFLOOD/HecHMS reservoir inflow data
        'and specify inflow into MODSIM link / FlowThru deman

        Dim fileLoc As String
        Dim fl As FileInfo
        Dim CnStr As String
        Dim tbl As DataTable
        Dim qry As String

        'Dim cnsl_win as System.Console

        fileLoc = "C:\Users\user\Desktop\MODSIM\2019May_newTrial\"
        fl = New FileInfo(fileLoc)
        CnStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & fl.DirectoryName & "\;Extended Properties='text;HDR=Yes;FMT=TabDelimited';"

        'make OleDbConnection to lake_sd.csv file
        'select dates + stphn_resIn, wusk_resIn & klsy_resIn

        qry = "SELECT [time], [1_lake_inflow] FROM [example.csv]" 'need bracket bc OleDb doesn't like digit in front

        Using con As OleDbConnection = New OleDbConnection(CnStr)
            'Using cmd As OleDbCommand = New OleDbCommand(String.Format("SELECT * FROM [{0}]", fl.Name), con)
            Using cmd As OleDbCommand = New OleDbCommand(qry, con)
                con.Open()
                'Using reader As OleDbDataReader = cmd.ExecuteReader()

                'End Using
                Using adp As OleDbDataAdapter = New OleDbDataAdapter(cmd)
                    tbl = New DataTable()
                    adp.Fill(tbl)
                End Using
                con.Close()
            End Using
        End Using
        'cnsl_win = New System.Console()
        'cnsl_win.WriteLine(tbl.ToString())
        Return tbl
    End Function

    Private Sub OnInitialize()
        Dim myTime as TimeManager
        Dim m_node As node
        Dim csvTable As DataTable
        'Dim dataStartDate_val As System.DateTime
        'Dim d, dIni, dEnd As Date
        'Dim maxTSteps, tStep,TStepDays,FlowVal as Integer

        m_node = New Node()
        csvTable = New DataTable()

        m_node = myModel.FindNode("NodeA")

        'Option 1
        'csvTable = getcsvFiles()
        'Dim m_TSTable as DataTable = m_node.m.adaInflowsM.DataTable()
        'm_TSTable = getcsvFiles()

        'Option 2
        'Dim nodeTS as TimeSeries
        'nodeTS = New TimeSeries(0) 'Non-storage TS
        'nodeTS = getcsvFiles()
        'dataStartDate_val = #1/1/2009 00:00:00#
        'dataStartDate_val = myTime.dataStartDate
        'm_node.m.adaInflowsM = nodeTS
        'm_node.m.adaInflowsM.setDataL(csvTable, dataStartDate_val)


    End Sub

    Private Sub OnIterationTop()

    End Sub

    Private Sub OnMessage(ByVal message As String)
    End Sub

    Private Sub OnIterationBottom()

    End Sub
    Private Sub OnIterationConverge()

    End Sub
    Private Sub OnFinished()

    End Sub

End Module

Comments (4)

  1. SJ Kim reporter

    Hello,

    I am trying to import csv data into MODSIM non-storage node and MODSIM maximum link capacity (haven’t attempted setting timeseries data as maximum link capacity). Some of the import functions (setData, setDataTable) in MODSIM has been discontinued but are not marked in the manual, which makes it hard to select the correct function to import timeseries data.

    I have tried the suggested line of code in Issue#69: Dim m_TSTable As DataTable = m_StephenNode.m.adaInflowsM.DataTable()
    However it gives an error in compilation.

    How can I set imported timeseries data into a Modsim non-storage node or link? Which functions should I be using? Please provide the code to import timeseries data into a non-storage node and as a maximum link capacity.

    The attached are:

    1. trial code to import "lake_sd.csv" file’s 1_lake_inflow timeseries into Modsim node "node1".
    2. example.csv (csv containing timeseries)
    3. example_bitbucket.xy (importing Modsim model)

    Thank you

    'Import required libraries
    Imports Csu.Modsim.ModsimIO
    Imports Csu.Modsim.ModsimModel
    Imports Csu.Modsim.NetworkUtils
    Imports System.Data
    Imports System.Collections    
    Imports System.Data.OleDb
    Imports System.IO
    
    Module Module1
    
        Dim myModel As New Model
        Sub Main(ByVal CmdArgs() As String)
            Dim FileName As String = CmdArgs(0)
            AddHandler myModel.Init, AddressOf OnInitialize
            AddHandler myModel.IterBottom, AddressOf OnIterationBottom
            AddHandler myModel.IterTop, AddressOf OnIterationTop
            AddHandler myModel.Converged, AddressOf OnIterationConverge
            AddHandler myModel.End, AddressOf OnFinished
            AddHandler myModel.OnMessage, AddressOf OnMessage
            AddHandler myModel.OnModsimError, AddressOf OnMessage
    
            XYFileReader.Read(myModel, FileName)
            'Enable Hydropower Add-on
            'myModel.hydro.IterativeTechnique = IterativeSolutionTechnique.SuccApprox
            'myodel.hydro.SetHydroLinkCosts(10000)
            Modsim.RunSolver(myModel)
        End Sub
    
        Function getcsvFiles() As DataTable
            'The purpose of this function is to read and import WATFLOOD/HecHMS reservoir inflow data
            'and specify inflow into MODSIM link / FlowThru deman
    
            Dim fileLoc As String
            Dim fl As FileInfo
            Dim CnStr As String
            Dim tbl As DataTable
            Dim qry As String
    
            'Dim cnsl_win as System.Console
    
            fileLoc = "C:\Users\user\Desktop\MODSIM\2019May_newTrial\"
            fl = New FileInfo(fileLoc)
            CnStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & fl.DirectoryName & "\;Extended Properties='text;HDR=Yes;FMT=TabDelimited';"
    
            'make OleDbConnection to lake_sd.csv file
            'select dates + stphn_resIn, wusk_resIn & klsy_resIn
    
            qry = "SELECT [time], [1_lake_inflow] FROM [example.csv]" 'need bracket bc OleDb doesn't like digit in front
    
            Using con As OleDbConnection = New OleDbConnection(CnStr)
                'Using cmd As OleDbCommand = New OleDbCommand(String.Format("SELECT * FROM [{0}]", fl.Name), con)
                Using cmd As OleDbCommand = New OleDbCommand(qry, con)
                    con.Open()
                    'Using reader As OleDbDataReader = cmd.ExecuteReader()
    
                    'End Using
                    Using adp As OleDbDataAdapter = New OleDbDataAdapter(cmd)
                        tbl = New DataTable()
                        adp.Fill(tbl)
                    End Using
                    con.Close()
                End Using
            End Using
            'cnsl_win = New System.Console()
            'cnsl_win.WriteLine(tbl.ToString())
            Return tbl
        End Function
    
        Private Sub OnInitialize()
            Dim myTime as TimeManager
            Dim m_node As node
            Dim csvTable As DataTable
            'Dim dataStartDate_val As System.DateTime
            'Dim d, dIni, dEnd As Date
            'Dim maxTSteps, tStep,TStepDays,FlowVal as Integer
    
            m_node = New Node()
            csvTable = New DataTable()
    
            m_node = myModel.FindNode("NodeA")
    
            'Option 1
            'csvTable = getcsvFiles()
            'Dim m_TSTable as DataTable = m_node.m.adaInflowsM.DataTable()
            'm_TSTable = getcsvFiles()
    
            'Option 2
            'Dim nodeTS as TimeSeries
            'nodeTS = New TimeSeries(0) 'Non-storage TS
            'nodeTS = getcsvFiles()
            'dataStartDate_val = #1/1/2009 00:00:00#
            'dataStartDate_val = myTime.dataStartDate
            'm_node.m.adaInflowsM = nodeTS
            'm_node.m.adaInflowsM.setDataL(csvTable, dataStartDate_val)
    
    
        End Sub
    
        Private Sub OnIterationTop()
    
        End Sub
    
        Private Sub OnMessage(ByVal message As String)
        End Sub
    
        Private Sub OnIterationBottom()
    
        End Sub
        Private Sub OnIterationConverge()
    
        End Sub
        Private Sub OnFinished()
    
        End Sub
    
    End Module
    
  2. Enrique T

    For link capacity time series simply set the table that you are reading from the csv.

    Link l = myModel.FindLink(LinkName);
    l.m.maxVariable.dataTable = getcsvFiles().Copy();
    

    This assumes that the time series is already initialized, i.e., there are values in the XY. If you need to initialize the time series you can do something like

    TimeSeries m_TS = new TimeSeries(TimeSeriesType.VariableCapacity);
    m_TS.dataTable = getcsvFiles().Copy();
    m_TS.units = _modsimModel.GetDefaultUnits(TimeSeriesType.VariableCapacity);
    m_TS.VariesByYear = true;
    

    For other timeseries use the following

    Variable Object Description
    adaTargetsM Reservoir Storage target time series. This time series is imported using the End Date.
    adaInflowsM Non-Storage Inflow time series
    adaEvaporationsM Reservoir Net evaporation rate time series (units of depth per time step).
    This time series is a decimal type, so it is imported without the accuracy factor.
    adaDemandsM Demand Water demand time series
    maxVariable Link Link capacity time series
    adaMeasured Link Measured flow time series
    adaForecastsM Reservoir Forecasted inflow to the reservoir
    adaGeneratingHrsM Reservoir Hydropower generating hours time series (units of hours)
    adaInfiltrationsM Demand fraction of the demand that infiltrates to the ground time series

  3. Log in to comment