Visual treats zero as blank/null leading to totally wrong results

Issue #67 closed
Former user created an issue

While analyzing hours of sunshine I realized that calculations are wrong for both the visual and the tooltips.

0 (zero) hours of sunshine will not be taken into account. Minimum is wrong, mean is wrong etc.

In general, calculations are only correct, if no value equals zero.

Official response

  • Daniel Marsh-Patrick repo owner

    Root cause:

    • If a Category field is supplied, the type-checking we use when we map values from the data view is not correctly handling zero values in all cases and sometimes casting them as null.
    • This type-checking and handling is inconsistent for cases when Category is not supplied (i.e. cannot be reproduced without a Category field).
    • The Category case should be corrected to be consistent with the Non-Category case.

Comments (21)

  1. Daniel Marsh-Patrick repo owner

    Hi there, and thanks for letting me know. I'll take a look as soon as I can.

    Out of interest, are you able to share your workbook (privately by email is fine if you don't want to attach here - d.s.marsh@gmail.com)? It'll be good to investigate your particular situation in context and to see the data you're working with. Either that or a link to the dataset (if publicly available) would be very useful.

  2. Daniel Marsh-Patrick repo owner

    I've had a look at this, and while I don't have the reporter's dataset, I've mocked up some fictional data for February. I've duplicated this with zeroes for some days, and another copy with the zeroes replaced with null/blank. I have not been able to reproduce the issue:

    Issue 57 - Comparison.png

    On the left is the raw data, violin plot, measures and a copy of the violin plot tooltip display for the dataset containing zeroes. The right is a duplicate of this but for the dataset containing the null values.

    In each case, the plot is behaving as I would expect for the data I'm using. I would re-iterate my request from the reporter for their report or dataset, or reproduction steps so that I can investigate further. I'll give this a week or so and close if there's no further follow-up, as I cannot reproduce for my assumptions of the reported scenario.

  3. FrankT

    Daniel, maybe it has to do with culture awareness.I'd raised that issue, but found no way to get an updated version.

    example.png

    Minimum should be 0.0 Mean should be about 1.0 (as shown in line chart)

  4. Daniel Marsh-Patrick repo owner

    Hi Frank - thanks for passing on the detail. The next version (1.2.0) will contain the culture fixes - they are resolved in the code but not yet published. The publish process takes 2-3 weeks end-to-end, so I have a couple more changes to finish implementing before I send everything off to MS.

    If you want, I can make a separate, development version of the visual available to you for testing, so you can confirm if the issue occurs for you in there? It would be good to try and head this one off sooner rather than later if I can (particularly given the long publish time).

    This version of the visual should not be used for production reports, as it would have a different identifier from the published one and would not update automatically if any subsequent updates are published to the marketplace (Power BI will always load the version from the marketplace for a known ID, even if you're uploading a different version). Let me know if this is something you'd like to try and I can arrange.

  5. FrankT

    Thanks Daniel, the link worked fine, but the calculation problem persists ☹️ while the decimal sign is localized now.

    example.jpg

    example2.jpg

    data.jpg

    Hopefully, you can fix that.

  6. Daniel Marsh-Patrick repo owner

    Glad the locale is working but concerned you're still having issues. Is it possible for me to get a copy of the .pbix you're using please?

    I'd like to check how the visual is handling/aggregating your Sampling field, as I would expect the #samples tooltip to read less if it were truly ignoring zero values (20 rather than 31, and I'd expect 33 based on the number of rows in your data). If you can't send the .pbix, could you please advise which field you're using in Sampling or do a show data on the visual and add a screengrab of that to the ticket?

    If you haven't done so, it might also be worth trying to add an index field to your data and adding that to the Sampling field (this should be unique for every row otherwise Power BI doesn't give us the unique values for the Measure field and will send through aggregated ones).

  7. Daniel Marsh-Patrick repo owner

    I've just manually entered the data from the above table (for January) and plotted using the v1.1.0 visual (the one in the marketplace) and am still not able to reproduce:

    67 - Repro for Jan.png

    It would definitely be good if it's possible to get a copy of the .pbix to see if there's anything specific to your setup that we might be missing in the visual when handling values. I can also do some additional debugging in-place and look at the Power BI internals for this specific case.

    One thing that stands out, although it might be due to regional differences - the Wert field is left-aligned in the data view, which in the NZ locale implies the field is textual rather than numeric (I see that Bereinigt is right-aligned, although that might be serving the purpose of textually compensating for the local issues present in the current release) but would be good to check off. I've tried manually adding my own but can't get the visual to plot (which is also what I expect in this case).

  8. FrankT

    "Wert" (value) field is all text because some of the unpivoted attributes are of type text. However, "Bereinigt" (adjusted) - which is used in the plots - is numbers only.

    I had switched off the Y-axis, otherwise it would have been clear that something is wrong. Check the .pbix yourself. It's certainly about dots and commas.

    e.g. for January 2018 Temperatur/Temperature: Line Chart: 5,952 Violin Chart: 5.952 Niederschlagshöhe/Rainfall: Line Chart: 2,300 Violin Chart: 3.565

    ViolinPlotPBIX

    Hopefully, you will find a solution. BTW, might it be possible to localize/translate the Tooltip measure names as well ?

  9. Daniel Marsh-Patrick repo owner

    Thanks for the .pbix, Frank - it has been very helpful. I've only had time to look at the internals, but I wanted to let you know I've received your workbook and it definitely confirms that something is wrong:

    67 - Repro Internals.png

    The left is the sorted array we have in the final view model for the plot. On the right, is the data we get from Power BI to start our work, so we're definitely getting the zeroes from Power BI. I'll do some investigation into why this is getting lost but it should be a straightforward fix once I get some time to go into this. Are you okay if I pass you another dev version of the visual to confirm things are okay from your side once I've done my own testing? It would be good to know that your case is working correctly from your side of things.


    Regarding the tooltip name localisation, I'd be happy to do it, but introducing this is a bigger (albeit necessary) change in the code so will need to address in a later release, particularly given the criticality of the issue we're currently looking at. If you're able to raise another issue with the translations you'd like, and the desired localised values (as well as confirming the Power BI locale you're using - presumably Deutsch?) I will implement this for you and pop a credit for your work on the wiki (or happy to keep you anonymous if you'd prefer that).

  10. Daniel Marsh-Patrick repo owner

    Root cause:

    • If a Category field is supplied, the type-checking we use when we map values from the data view is not correctly handling zero values in all cases and sometimes casting them as null.
    • This type-checking and handling is inconsistent for cases when Category is not supplied (i.e. cannot be reproduced without a Category field).
    • The Category case should be corrected to be consistent with the Non-Category case.
  11. FrankT

    I'm waiting for your updated dev version. Asap I will raise another issue regarding translations and provide the German locale.

  12. Daniel Marsh-Patrick repo owner

    Hi Frank - I've uploaded a new version of the visual to the repo for testing. This contains what I hope is the correct fix for your issue.

    Please have a look and let me know how you get on - if all's good, and pending personal commitments I'll be looking to get this sent off to MS next week (still working on one more issue and need to update the documentation before I do so).

  13. Daniel Marsh-Patrick repo owner

    Thanks, Frank - glad to know it's working. I'm going to be sending the latest version of the visual off for approval shortly.

  14. Daniel Marsh-Patrick repo owner

    Yup - as per previous comment it'll take up to 2 weeks once the AppSource listing is updated. MS do their own testing before they fully release the visual. It is very confusing for all of us...

  15. Daniel Marsh-Patrick repo owner

    I will close the issue when I've observed the update of the visual in my reports.

  16. Log in to comment