add support for maps / set

Create issue
Issue #74 resolved
Antoine Chalons created an issue

It would be cool if Maps and Sets could be handled as if they were Arrays

Comments (29)

  1. James Powell repo owner

    Problems:

    1. Maps/Sets don’t exist in 2017
    2. Even in 2020, support for Maps/Sets in Variants seems inadequate to convert to JSON (I checked)

    Route forward could be to use Flattening techniques (like OpenG), Checked that the Set flattened format is just an Array, and the Map format seems to be two Arrays, so that seems doable. Problematic performance though, as Flattening techniques are typically about an order of magnitude slower than Variant-based ones.

  2. James Powell repo owner

    Is this still on the roadmap?

    Yes, but low priority, as my stuff is 95% in LabVIEW 2017-18.

  3. Brent Justice

    Would it potentially make sense to leverage the new fancy “LVClass Serializer” class as a mechanism to inject custom code for handling Map/Set serialization?

    Just a curiosity, thanks

  4. Brent Justice

    @James Powell we’ve become addicted to maps/sets, they’ve been a fantastic addition to LabVIEW. We’d love to be able to use them with JSONtext.

    LAVA LogMAN’s latest LabVIEW Composition library has added support for map/set composition/decomposition.

    https://github.com/LogMANOriginal/LabVIEW-Composition

    This library was written in LabVIEW 2017.

    Just like your snippet from above, the LabVIEW composition library uses flatten/unflatten from string techniques. Like you’ve commented, this is a pretty serious performance hit.

    NI R&D is tracking a feature ticket for adding variant composition/decomposition support natively with variants. However, it sounds like this would likely not arrive until LV2022 or LV2023. This would also then require JSONtext to release on this newer version… which I recognize that you tend to try to lag JSONtext by a few years to widen the userbase.

    From my perspective, I would like to see JSONtext add map/set support using the flatten/unflatten from string solution for now. As a user, I would accept the performance penalty. The string methods could then be swapped with the variant comp/decomp methods when NI gives them to us and JSONtext is able to upgrade LabVIEW version.

    I’m happy to collaborate and perform all of this work and then push a merge request if that makes your life easier. Happy to help.

    Let me know your thoughts, thanks!

  5. Brent Justice

    Also note that it is possible to determine if a variant is a map or set using the variant TypeDescriptor. We would have to use this since the LV2017 “Get Type Information.vi” does not contain map/set, which is inconvenient

  6. James Powell repo owner

    We would have to use this since the LV2017 “Get Type Information.vi” does not contain map/set, which is inconvenient

    We can work around this by having a “Fixed Point..” case (where Map and Set will fall) and in that cast to a U8 and have a case structure on 45,46,47.

  7. James Powell repo owner

    Potential problem: User likely to provide an empty map as Type: we need to be able to get default values of the data elements.

  8. Brent Justice

    We can work around this by having a “Fixed Point..” case (where Map and Set will fall) and in that cast to a U8 and have a case structure on 45,46,47.

    Yes! agreed. That would work very well

  9. Brent Justice

    Correct me if I’m wrong here, variant/string flattening stuff is still a bit of a mystery to me, but does this not work?

    Basically, I pull the TypeDescriptor out of the “variant to flattened string” function, and then pass this into the OpenG Toolkit “Get Default Data From TD”

  10. Brent Justice

    Oof, we’d have to modify the “Get Default Data from TD” VI a bit. Under the hood, it doesn’t support LV Classes, or, ironically, maps/sets. We’d have to add that.

  11. Brent Justice

    Well, in semi-good news, this problem is already solved for us in LV2020:

    It’s probably best to just use these VIs from the NI data parsing palette if possible. Ugh.

    Is there a mechanism that would allow us to keep JSONtext is LV2017, but conditionally call these data parsing VIs for LV2020 or newer?

  12. Brent Justice

    Yes! That would certainly work in source code. However, the tricky part is that this would not cause the “Get Map Collection Information.vi” to be automatically collected by the application builder when you build an executable. You’d have to force-include the VI, which is painful.

    The only way I can think to solve this would be to use your “LVClass Serializer” to provide a hook into the Map/Set code, and then provide a LV2019+ class override which uses the “Get Map Collection Information.vi”. But this has cons to it as well.

    So close, yet so far

  13. Brent Justice

    James, I just downloaded. Fantastic! Thanks so much for adding this.

    Also, very creative solution to getting the empty map type. Kudos

  14. Brent Justice

    James, thanks for the patch, this is looking great!

    I found another error code, but this one might be unrelated.

    I can’t seem to have a map with a variant inside of it. Moreover, JSONtext rips off variant names. Is this intended?

  15. James Powell repo owner

    Variants and JSON are problematic. Conversion of JSON to your empty Variant can’t properly happen, because of the lack of a type to convert to. Converting a Variant to JSON does work, but in that case there are two possible names, that of the Variant, and that of the data in the Variant (the unnamed 2 constant in your example). I chose to use the name of the data, rather than the name of the Variant, but don’t know if that is a good choice. I actually, have never used a Variant with JSONtext, I always use subJSON instead, up until the point that I convert to teh actual real type.

  16. James Powell repo owner

    Now, what I could do with is a custom error message to explain this, rather than that unhelpful Error 74.

  17. Log in to comment