Performance issue with Find Multiple Items

[Olivier Jourdan]

After upgrading from 1.3.1.84 to 1.4.5.91, I saw a huge decrease in performance using the Find Multiple Items function.

Here is my use case and the behavior I saw:

• Measurement data are stored in an “in memory” SQLite database as a JSON object.
• I display the lastest 1/5/15 min of data querying records from the db and finding user-selected data using the Find Multiple Items function.

For a specific configuration on a specific computer CPU load with the 1.3.1.84 toolkit version is around 43 %

After upgrading to v1.4.5.91 CPU load increase 60+ percent

With the 2 versions, I removed all plots displayed and the CPU load is identical. It seems to indicate the performance issue is related to the “read” part.

Are there any changes between these 2 versions that could explain this performance difference?

[James Powell]

Would it be possible for you to supply your test case? I usually suspect a performance loss as being a parsing VI that is accidently not inlined, but I can’t see anything obvious.

[James Powell]

Used Execution Trace Toolkit. Found a couple of VIs that could be inlined for performance, but neither was inlined in 1.3.1, so that can’t be it.

[James Powell]

BTW, I usually use the SQLite JSON1 functions to do this kind of stuff. Using JSON_extract(col,path) in the SQL. Doing the JSON search inside the db and only returning what you want to display is higher performance, especially if you apply data-reduction in the SQL (doing averages over groups and returning only enough points to plot). More work to set up, but the more work you can do in the db the quicker things will be.

[Olivier Jourdan]

That’s part of a large project, and I don’t have a specific VI test for this. I need to build one. I’ll attach it once done but don’t expect it too early…

Thanks for the database tip. The code was done last year and I remember making lots of performance tests but don't remember why I ended up with that implementation. That being said, I didn't implement everything I'd love to have, and I'd be happy to share some thoughts with you as soon as I start to work on these features again. I'm pretty sure I'm not using SQLite capabilities as its best.

‌

[James Powell]

I mostly want an example of your JSON, so I can run it through Find Multiple Items and see what code paths it follows.

[Olivier Jourdan]

JSON string:

{"406":0.000000,"407":0.000000,"AU":10.000000,"Coef":0.000000,"Com_CoolAux_Fan":0.000000,"Com_Cool_Fan":0.000000,"Com_Cool_Pump":0.000000,"Com_Elec_Imax":0.000000,"Com_Elec_Umin":0.000000,"Com_H2_P":0.000000,"Com_Wdg":1.200000,"Computed tag in Computed Tag":304.531670,"Delta_P_Air":0.000000,"Delta_P_Cool":0.000000,"Delta_P_H2":0.000000,"Delta_T_Cool":0.005435,"Function":0.000000,"I_Air_Cp":27.063136,"I_Elec_Chrg":366.669335,"I_Elec_Chrg (1)":229.175807,"I_Elec_Chrg (1) Med":90.227972,"I_Elec_Chrg (5)":214.928079,"I_Elec_Chrg (5) Med":549.186755,"I_Elec_PAC":231.288697,"MbInput_405":1.000000,"Number of cells":1.000000,"Number of cells + 2":3.000000,"Output":0.000000,"P_Air_Ave":3.500000,"P_Air_Cp":4.000000,"P_Air_Hum_Wet":4.000000,"P_Air_In":3.500000,"P_Air_Int":3.500000,"P_Air_Out":3.500000,"P_Air_Src":4.000000,"P_Atm":3.500000,"P_Cool_In":3.500000,"P_Cool_Out":3.500000,"P_Elec":96.551511,"P_H2_In":3.500000,"P_H2_Out":3.500000,"P_H2_Prop":5.166545,"P_H2_Syst":5.529353,"Q_Air_Syst":165.000000,"Q_CoolAux_In":150.000000,"Q_Cool_In":150.000000,"Q_H2_Syst":0.995986,"RH_Atm":100.000000,"RH_H2_In":54.826272,"RH_Hum_Out":57.501686,"St_Air":6.624550,"St_H2":1.365726,"Stack Use":1.000000,"T_Air_Cp":-241.836866,"T_Air_In":-241.832359,"T_Air_Int":-241.825333,"T_Air_Out":-241.813932,"T_Atm":120.000000,"T_CoolAux_In":-241.812607,"T_CoolAux_Out":-241.799615,"T_Cool_In":-241.791263,"T_Cool_Out":-241.785828,"UCell_0000":0.615120,"UCell_0001":0.670196,"UCell_0002":0.600780,"UCell_0003":0.657126,"UCell_0004":0.694978,"UCell_0005":0.699264,"UCell_0006":0.654073,"UCell_0007":0.620587,"UCell_0008":0.696928,"UCell_0009":0.630456,"UCell_0010":0.644162,"UCell_0011":0.660957,"UCell_0012":0.657532,"UCell_0013":0.667833,"UCell_0014":0.683812,"UCell_0015":0.659447,"UCell_0016":0.686071,"UCell_0017":0.692846,"UCell_0018":0.676537,"UCell_0019":0.638368,"UCell_0020":0.694736,"UCell_0021":0.670636,"UCell_0022":0.657033,"UCell_0023":0.677902,"UCell_0024":0.611770,"UCell_0025":0.649215,"UCell_0026":0.643180,"UCell_0027":0.633837,"UCell_0028":0.607736,"UCell_0029":0.678091,"UCell_Ave":0.657707,"UCell_Ave by calculator":0.642658,"UCell_Dev":28.509048,"UCell_Max":0.699264,"UCell_Min":0.600780,"U_Air_Cp":313.246378,"U_Elec_Chrg":263.320385}

Path

$.[I_Elec_Chrg,I_Elec_Chrg (1),I_Elec_Chrg (5),I_Elec_Chrg (1) Med,I_Elec_Chrg (5) Med,P_H2_Syst]

‌

[James Powell]

How do you convert your JSON Numbers into LabVIEW numbers?

[Olivier Jourdan]

Here is the code I’m using for to get data from the database

‌

[James Powell]

Haven’t been able to identify what might have caused this, but a couple of workaround suggestions:

1. Try using “Find Object Items by Name.vi” instead (avoids the path-parsing code used by “Find Multiple Items.vi”).
2. Try using “Get all Object Items.vi” and searching on the names explicitly

If (1) fixes the CPU issue let me know, as that may indicate teh problem is in teh path-parsing code, rather than the actual JSON-searching bit.

[Olivier Jourdan]

Thanks for the update. I’ll keep you posted as far as I can make the tests.

[Olivier Jourdan]

Hi @James Powell

@James Powell

,

‌

(1) is doing really well. I’m going from nearly 100% to less than 60% of CPU load! Just replacing the JSON function!

Thank you for the suggestion.

That will make more than a workaround, but do you know why performances changed on that specific function from version from 1.3.1.84 to 1.4.5.91?

[James Powell]

Unfortunately a lot changed between those two versions, but the comparison between the two functions narrows down where it could be (possibly in the code that parses the Path).