Symptom: calling mjsonrpc's
hs_read_arraybuffer can sometimes result in “duplicated” data being returned - you will be told that there are twice as many datapoints as there should be for a variable, and you will get the data in the order T0,T1…TN,T0,T1…TN.
The problem appears for data coming from the “System” event (e.g. links defined in /History/Links/System), and I think it only affects MySQL history systems.
The main function to study is
The underlying cause is that
fSchema (populated by
MysqlHistory::read_table_and_event_names) contains schemas where the event_name is
system AND where the event_name is
System (note the different capitalisation). The first of these is created directly in
MysqlHistory::read_table_and_event_names , has a time_from of 0, and sets the event_name to be the same as the table_name; the latter is created in
ReadMysqlTableNames, has a time_from that depends on when the schema was last changed, and sets the event_name to be the “real” midas event name.
The problem is that
SchemaHistoryBase::hs_read_buffer reads data from BOTH of these schemas! Reading from multiple schemas normally makes sense, as if the schema changed during your period of interest, you still want all the data. But here the schemas have overlapping validity periods (and in my case we read the full lot of data twice).
I don't know enough about the MySQL history system to suggest the correct resolution. Are both versions of the schema required (the one based on table name and the one based on event name)? If not, then you could probably remove the call to
ReadMysqlTableNames? If both are required, then perhaps some extra logic needs to be added to
hs_read_buffer so that if a variable matches multiple schemas, we don't re-read data for periods we've already read? Or you could do a "de-duplication" at the end of
hs_read_buffer if that’s easier?