add support for maps / set
Issue #74
resolved
It would be cool if Maps and Sets could be handled as if they were Arrays
Comments (29)
-
repo owner
-
repo owner
Think I have a working method (saved in JDP Common Utilities repo):
-
Related discussion on the forum:
https://lavag.org/topic/22009-mapset-compositiondecomposition/
Is this still on the roadmap?
-
repo owner
Is this still on the roadmap?
Yes, but low priority, as my stuff is 95% in LabVIEW 2017-18.
-
got it, no worries, thanks!
-
Would it potentially make sense to leverage the new fancy “LVClass Serializer” class as a mechanism to inject custom code for handling Map/Set serialization?
Just a curiosity, thanks
-
@James Powell we’ve become addicted to maps/sets, they’ve been a fantastic addition to LabVIEW. We’d love to be able to use them with JSONtext.
LAVA LogMAN’s latest LabVIEW Composition library has added support for map/set composition/decomposition.
https://github.com/LogMANOriginal/LabVIEW-Composition
This library was written in LabVIEW 2017.
Just like your snippet from above, the LabVIEW composition library uses flatten/unflatten from string techniques. Like you’ve commented, this is a pretty serious performance hit.
NI R&D is tracking a feature ticket for adding variant composition/decomposition support natively with variants. However, it sounds like this would likely not arrive until LV2022 or LV2023. This would also then require JSONtext to release on this newer version… which I recognize that you tend to try to lag JSONtext by a few years to widen the userbase.
From my perspective, I would like to see JSONtext add map/set support using the flatten/unflatten from string solution for now. As a user, I would accept the performance penalty. The string methods could then be swapped with the variant comp/decomp methods when NI gives them to us and JSONtext is able to upgrade LabVIEW version.
I’m happy to collaborate and perform all of this work and then push a merge request if that makes your life easier. Happy to help.
Let me know your thoughts, thanks!
-
Also note that it is possible to determine if a variant is a map or set using the variant TypeDescriptor. We would have to use this since the LV2017 “Get Type Information.vi” does not contain map/set, which is inconvenient
-
repo owner
We would have to use this since the LV2017 “Get Type Information.vi” does not contain map/set, which is inconvenient
We can work around this by having a “Fixed Point..” case (where Map and Set will fall) and in that cast to a U8 and have a case structure on 45,46,47.
-
repo owner
Potential problem: User likely to provide an empty map as Type: we need to be able to get default values of the data elements.
-
We can work around this by having a “Fixed Point..” case (where Map and Set will fall) and in that cast to a U8 and have a case structure on 45,46,47.
Yes! agreed. That would work very well
-
Correct me if I’m wrong here, variant/string flattening stuff is still a bit of a mystery to me, but does this not work?
Basically, I pull the TypeDescriptor out of the “variant to flattened string” function, and then pass this into the OpenG Toolkit “Get Default Data From TD”
-
Oof, we’d have to modify the “Get Default Data from TD” VI a bit. Under the hood, it doesn’t support LV Classes, or, ironically, maps/sets. We’d have to add that.
-
Well, in semi-good news, this problem is already solved for us in LV2020:
It’s probably best to just use these VIs from the NI data parsing palette if possible. Ugh.Is there a mechanism that would allow us to keep JSONtext is LV2017, but conditionally call these data parsing VIs for LV2020 or newer?
-
repo owner
Let me think about this a bit. All options are problematic.
-
repo owner
This will work, to call the 2019+ methods, without breaking in 2017:
-
Yes! That would certainly work in source code. However, the tricky part is that this would not cause the “Get Map Collection Information.vi” to be automatically collected by the application builder when you build an executable. You’d have to force-include the VI, which is painful.
The only way I can think to solve this would be to use your “LVClass Serializer” to provide a hook into the Map/Set code, and then provide a LV2019+ class override which uses the “Get Map Collection Information.vi”. But this has cons to it as well.
So close, yet so far
-
repo owner
Implemented, and posted a beta here: https://forums.ni.com/t5/JDP-Science-Tools/BETA-version-of-JSONtext-1-6/m-p/4153244#M42
-
James, I just downloaded. Fantastic! Thanks so much for adding this.
Also, very creative solution to getting the empty map type. Kudos
-
Ok, so, I get an error with the following snippet:
-
ok, yeah, it looks like I get an error whenever I use a cluster for the map data
-
repo owner
I think I know what the problems is, though it may be tricky to solve…
-
repo owner
-
James, thanks for the patch, this is looking great!
I found another error code, but this one might be unrelated.
I can’t seem to have a map with a variant inside of it. Moreover, JSONtext rips off variant names. Is this intended?
-
repo owner
Variants and JSON are problematic. Conversion of JSON to your empty Variant can’t properly happen, because of the lack of a type to convert to. Converting a Variant to JSON does work, but in that case there are two possible names, that of the Variant, and that of the data in the Variant (the unnamed 2 constant in your example). I chose to use the name of the data, rather than the name of the Variant, but don’t know if that is a good choice. I actually, have never used a Variant with JSONtext, I always use subJSON instead, up until the point that I convert to teh actual real type.
-
repo owner
Now, what I could do with is a custom error message to explain this, rather than that unhelpful Error 74.
-
Got it.
Looks great. Thanks again James
-
repo owner
- changed status to resolved
Marking done, as I think it is ready to release.
-
Thanks again James, we really appreciate this one. I’ll buy you a beer at NI week
- Log in to comment
Problems:
Route forward could be to use Flattening techniques (like OpenG), Checked that the Set flattened format is just an Array, and the Map format seems to be two Arrays, so that seems doable. Problematic performance though, as Flattening techniques are typically about an order of magnitude slower than Variant-based ones.