1. Brian Lewis Avatar Brian Lewis
  2. Untitled project
  3. pineapples
  4. Issues

Migrate all charts to echarts

Issue #827 resolved
Brian Lewis repo owner created an issue

Standardise on echarts as the charting tool of choice

‌

https://echarts.apache.org/en/index.html

‌

This issue is intended as a parent issue to put general comments and notes about this process, as a resource. Specific issues can be created for subtasks.

Official response

Comments (8)

  1. Brian Lewis reporter

    Resizing strategy:

    echarts has an effective resize() method, but this requires that

    • the <div> on which the echart is created has a deterministic height ( ie not just sized by its contents
    • any change in the height is detected

    The best way to get the deterministic height is to:

    • start with some <div> or element that has a fixed height; then
    • use angularjs-material flex layout options layout=”column” layout-fill to get the size to pass down the chain to the echart

    Then the echart component inherits from SizeableChartController which watches for changes of the div.

    Ultimately, I think we could start with <body> fitted to the window, then the entire page structure flexed down from there using layout layout-fill flex from angularjs-material. This will then carry over to angular10/ angular-material

    ‌

    ‌

  3. Brian Lewis reporter

    Some thoughts on architecture and what we need to do to cover off on everything we need:

    At the lowest level, an echart will contain an instance of the component EChart.Component

    This does very little - in particular - it does not attempt to draw a chart.

    It functions only to:

    • instantiate the echart object,
    • provide a callback (onChartInit) so that the containing component can get the echart
    • manage resizing of the echart (see Resizing Strategy above)
    • dispose of the echart when the component is destoyed

    So, via the callback, the parent of the Echart.Component holds the instance of its underlying echart object. This is where the chart construction takes place.

    I foresee at this level, a number of powerful and flexible components aimed at different circumstances. Let’s call these Supercharts.

    Currently we have our first superchart: EchartRowCol, which takes the place of the plottable-based rowColChart.

    This is the go-to component for drawing 2-dimensional data; and it can render 8 formats:

    • bar, percentage bar, stacked bar
    • column, percentage column, stacked column
    • line (not stacked), area (stacked)

    i propose we build a Map component superchart; and we need something for pies and donuts.

    In the echart world, a Map such as we would currently draw with the google maps api becomes just a particular form of scatter chart, where X and Y are lat and long, and leaflet provides the underlying map imagery and the co-ordinate frame for lat/long.

    Echarts also provides cloropleth mapping ( colored regions) - we could build a superchart around that.

    Pies and Donuts could possibly be a separate superchart; or managed in an enhancement to EchartRowCol.

    Data

    Currently, for historic reasons and compatibility with stored procs and the original RowColChart, EchartRowCol expects data in a particular ‘rowcol’ format: each row looks like

    { R: <the row identifier>, C: <column identifier>, Num: value }

    In the more recent dashboards that use RowColChart, data starts off in a Crossfilter group; and this crossfilter data is massaged into RowCol format to give to the chart.

    However, the internal formats that echarts wants for its datasets are different again - there a a couple of formats supported, but probably best for general purposes is:

    { 
  dimensions: ['School', 'Boys','Girls'] ,   // array of ‘dimensions’; ie field names
  source: [
    ['School A',6,5],
    ['School B', 4, 3]
  ]
}

    So we need ways to get different data shapes into this format ( an echarts Dataset), currently we can accept only ‘RowCol’ data.

    We need to be able to transform:

    • a crossfilter group
    • a key value collection derived from Crossfilter.group.all()
    • nested data??
    • plottable legacy models built on a data collection and accessor functions - these accessors can be used to generate the echarts Dataset?
    • We also need to rationalise various one-off formats used in Dashboards - see e.g. schoolDashboard, and plottableBarChart, PlottableColumnChart .

    So my thinking is to build a Transformer class, that will take incoming data, identify what it is, and work it into echarts format. As noted above, this is currently happening in two steps in new dashboard by

    group.all() => RowCol data => echarts Dataset

    If we can effectively transform crossfilter objects, we can substitute our echarts-based components for our wrappers around dcjs-based components.

    EchartRowCol can then effectively handle a significant part of our charting needs - especially if it adds pie and donut capability.

    Otherwise, we have our other supercharts - map, cloropleth etc - to choose from.

    Superchart design

    The trade-off here is : how much flexibility to build in to a component like RowColChart via bindings?

    Taken to extremes, you could end up creating a parallel API to not much advantage.

    So my preferred approach is:

    • have a relatively concise set of bindings exposed on each superchart ( ie EchartRowCol) . As well as providing the data (so that a data change can force a redraw via binding architecture) most options will be about construction of the chart.
    • expose from the supercharts a callback onChartRender(option) that is called passing back the echarts option immediately before this is submitted to echarts (ie echarts.setOption(option); Then the consumer of the superchart can do whatever it likes to the chart using the echarts option API.

    As an example, the exam charts are now drawn by EchartRowCol, but use the onChartRender callback to do some specific non-standard things - specific colors, order of the series, width of the label area …. See

    https://bitbucket.org/softwords/pineapples/commits/232c2c6ddbd8e8c667a95dc88e685a417e232fa4#chg-Pineapples.Client/src/app/ts/Features/Exam/Dashboard/ResultsByBenchmark.Component.ts

    where onChatRender is implemented.

    Supercharts will also provide a callback for formatting tooltips; and a clickhandler callback.

    There are many events raised by echarts, not just mouse events, but events related to actions as well. Supercharts may consume these in internally if they need to. But if the consumer of the superchart wants to handle an event in some special way, I think, rather than having an & binding for every possible event on every superchart; its more expedient for the superchart to simply relay onChartInit(echart)

    to its client, which can then set up its on bindings, in its own scope.

    Steps in building an echart

    There are a number of steps in the sequence of building an echart:

    1. Data transformation - as discussed, we need to get an echarts Dataset from whatever input data we are given. Call this end-product the transformed data.
    2. Data processing - some modifications we may want to make to the visual appearance of the chart may be best (or only) handled by manipulating the data. e.g. sort a bar or column chart from lowest to high values. So there may be a pipeline of data processes required to get the ‘transformed data’ in to the chart-ready data.
    3. Chart initialisation. A template for the chart options
    4. Series initialisation. The series are created as required ( possibly determined by bindings)
    5. Chart customisations. other options determined by bindings may be sequentially applied to the chart.

    Note that when reacting to binding changes, where you need to start in this sequence will depend on what bindings have changed. For example, data transformation is only needed if the dataset has changed. Data processing will be required to respond to specific changes (currently flipping, change to a percent based format, sort changes).

    I’d like this flow of steps to be clear in the internal construction of echart components. For example, I’m considering a formal “pipeline” of data processes that is defined so that each step in the pipeline consumes a Dataset produced by its precursor, and outputs and Dataset. The final result of this pipeline is the chart-ready data. So Step 1) above is the output of a Data Transformer as described above (IN: some data OUT: echarts Dataset). Step 2) the is application of the pipeline (IN: transformed Data OUT: chart-ready data)

    work-in-progress at this stage….

    ‌

    ‌

    ‌

    ‌

    ‌

    ‌

  4. Brian Lewis reporter

    @Ghislain Hachey here’s a handy tip for debugging echarts you are creating in pineapples code:

    ‌

    First notice that any of the examples at https://echarts.apache.org/examples/en/index.html

    open an interactive javascript editor. You just need to set the value of a variable called option in this window, and that option will get generated into a chart.

    ‌

    Now if you are creating an option object in your pineapples typescript, use the chrome debugger to

    console.log(option)

    to get your option object into the console window.

    Following this tip: https://superuser.com/questions/777213/copy-json-from-console-log-in-developer-tool-to-clipboard

    you can get this option object on to the clipboard.

    • Right-click the object and select "Store as global variable"

    2 - The console will print the new variable's name, for example:

    //temp1

    3 - Type:

    copy(temp1)

    The object is now available in your clipboard.

    Now you can paste that object into the echarts example JS, to set option to this value.

    Run it to see the result. You can now tweak the option in the debugger to fix any problems, then go back to your typescript code to implement these fixes.

    ‌

  5. Brian Lewis reporter

    @ghachey excellent progress on migration in the issue 827 brnach which I have merged.

    If you look at PlottableBarChart, PlottableLineChart, PlottableAreaChart you’ll see how the migration is done by a minor repackaging of the incoming data - to make a ‘widgetPkg’ and a change in the template to use RowColChart.

    If you want to “get your fingers in the clay” could you do the same for PlottablePieChart and PlottableHorizontalBarChart?

    Then we have the dc-based components to look at…..

    ‌

  7. Ghislain Hachey

    @Brian Lewis Sure. Need to get a few things out of the way first and will look and this and give it a go.

  9. Log in to comment
Assignee
Type
enhancement
Priority
major
Status
resolved
Votes
0
Watchers
1
Jira: the preferred issue tracker for Bitbucket. Join the team!