mjsonrpc calls should return valid utf8 encoded data

[dd1]

mjsonrpc returns data in json format, most web browsers automatically decode it into a javascript object and complain (throw an exception) if there is any non-utf8 strings. most midas web pages malfunction when this happens. Worst case is the “messages” page where we read data from midas.log and sometimes this is binary data (written by cm_msg() or by the user or by midas.log file corruption/truncation).

mjsonrpc server has 3 general cases:

• non-utf8 strings returned from ODB - bug 215, almost resolved
• internally generated data (i.e. “get alarms”, etc) - need to audit this code, but think it is all good.
• data read from files (midas.log, etc) - biggest problem. not clear how to sanitize non-utf8 (or even non-text) data for safe decoding and display in browser.

It would be good to have this sorted out for the midas-2020-07 release.

K.O.

‌

[dd1]

mjsonrpc methods that can return non-utf8 data:

• js_cm_msg_facilities - names from from filesystem “ls”
• js_cm_msg_retrieve - calls cm_msg_retrieve2(), reads data from file
• js_hs_xxx - maybe okey? data comes from ODB
• js_el_xxx - reads data from files
• jrpc - returns arbitrary string from user c++ code
• js_seq_list_files - names come from filesystem “ls”

K.O.

‌

[dd1]

mjsonrpc - all calls to MakeJSON() and MakeString() are now protected with ss_repair_utf8(). Perhaps this protection should be inside MakeJSON() and MakeString(), but right now mjson.{h,cxx} does not care about utf8, and it is not clear if the JSON standard requires utf8 or not. K.O.

[dd1]

fixed as of https://bitbucket.org/tmidas/midas/commits/ea7b55d9c9d4553072e9f69e15c97762c2772c91