Wrong colors drawn sometimes

[Robert Leach]

Attaching 2 examples. This may be 2 issues: drawing black when 2 or more data cells map to the same pixel and drawing red or green sometimes when data is missing. The screen recording shows a black bar that goes away upon zoom using John Matese's data during a debug session where black was 0.0 and it was closely flanked by 2 shades of sky blue. The other is also John's data showing red being drawn instead of gray for missing data.

[Robert Leach]

I tested the "drawing black" when there are colors present issue in TreeView2, as was discussed in our last meeting.

First off, we're doing better than TV2 when it comes to large data files, because by default, when you try to open a 6kx6x matrix in treeview2, the default of 500 megs that the app is launched with is not enough. Nor was 1000M (memory error), 2000M (hung app), 5000M (hung) and I gave up. Also, I got all sorts of errors from TV2 about numbers ending in 'e' which we handle just fine. So I think we are doing better in that regard.

It's hard to tell if the pixel averaging of data cells assigned to the same pixel is working better or worse because the color settings are different (tv2 includes a contrast value), but one thing is for sure: TV2 is managing the averaging better when it comes to missing data. TV2 ignores missing data in the calculation of the average color/value. Note this comparison screen-cap I took. I tried to make TV2's overview the same dimensions as ours with the same data (John Matese's 11.cdt file):

Note how all that red in our version is the "missing data" in John's file. By contrast, TV2's gray color for missing data completely vanishes at this zoom level. I suspect that when Alok averages the data for a spot, he's just counti9ng the number of cells with data, adding their values up, and dividing by that total "with data".

[Robert Leach]

It's also interesting to note that TV2's default red/green are switched compared to ours.

[Christopher Keil]

I just decided to link whatever I can find that could contribute to the issue. Maybe this way we can get a hold of what is happening.

https://stackoverflow.com/questions/25473302/wrong-colors-when-painting-in-graphics2d-at-16bit-color-depth

[Robert Leach]

I have a suspicion that color averaging across numerous cells drawn to the same pixel has some complication with contrast or else the way averaging is done is just not a good strategy. I just performed a test. I took a screen-cap of a colorful zoomed-in portion of a large matrix. I zoomed out and too a screen cap of that section. I then took the screen cap of the zoomed-in colorful shot and adjusted its size using Preview. I ended up with a colorful (though admittedly still somewhat subtle) image compared to what treeview draws:

Treeview:

Preview:

I think that we should look into the averaging and make sure that nothing is happening to the contrast and that the averaging is accurate.

Note, TV2 has this same issue:

so it's not anything that we did.

[Christopher Keil]

Another issue caused by color averaging made perfectly visible by a binary data set.

Look at the defined values and what appears when all pixels are drawn normally (Interactive matrix). When the values are compressed into single pixels these pixels receive the average value from all their source pixels during that process. This average value is then used to derive a color.

This is terribly misleading and will happen to both matrices every time there's not enough space for all tiles. I will try to find a solution that uses scaling of the BufferedImage class.

https://stackoverflow.com/questions/4216123/how-to-scale-a-bufferedimage

[Robert Leach]

Is that a complete re-work of the way the matrix is drawn? Our image currently contains no "compression" effects, which I think is an asset. Would scaling a buffered image produce a compression effect, where the dots would be fuzzy at all?

[Christopher Keil]

The screenshot above is the current running code. I will try to do some stuff with the matrix graphics on some local branch.

[Robert Leach]

Sorry, I meant to ask if switching to a method involving scaling a buffered image would be a rework of the way the matrix is drawn.

[Robert Leach]

In other words, would we have to rewrite all the zooming, and other navigation functions? And what side affects could there be to doing this?

[Christopher Keil]

I am not sure. Since we have always been using the working legacy code I haven't ever gauged the impact this kind of rewrite would have. I can experiment a bit around and bring it up in Thursdays discussion.

[Christopher Keil]

Going to aggressively tackle this as the next step. This will come down to the guts of the matrix drawing code. The averaging of values is what leads to these issues and somehow we need to find out a proper scaling algorithm instead.

[Christopher Keil]

The missing data drawn as some extreme color was actually a separate (and subtle) issue that was the result of code mix up from TV2 and TV3. Another issue should be opened specifically for averaging because that still occurs. But the original issue here has been resolved.

[Anastasia Baryshnikova]

Checked in version 20151118, issue fixed.

[Robert Leach]

Removing milestone: Color settings (automated comment)