- edited description
PGM JSON Output Issues
Found several issues with JSON output. I'm using this sample RPG:
H AlwNull(*UsrCtl)
dcl-ds inputDS qualified;
in1 varchar(5:2);
in2 varchar(5:2);
end-ds;
dcl-pr Main extpgm;
inCount int(10);
input likeds(inputDS) dim(20);
outCount int(10);
outputA char(10) dim(20);
last char(20);
end-pr;
dcl-pi Main;
inCount int(10);
input likeds(inputDS) dim(20);
outCount int(10);
outputA char(10) dim(20);
last char(20);
end-pi;
dcl-s i int(10);
outCount = inCount;
for i = 1 to inCount;
outputA(i) = 'test' + %char(i);
endfor;
last = '"quoted" text';
outCount = i - 1;
return;
Sample call:
{"pgm":[
{"name":"DRTEST01", "lib":"DROESSNER"},
{"s": {"name":"inCount", "type":"10i0", "value":1, "by":"in"}},
{"ds": [{"name":"inputDS","dim":20, "by": "in"},
{"s":[
{"name":"in1", "type":"5av2", "value":"i1"},
{"name":"in2", "type":"5av2", "value":"i2"}
]}
]},
{"s": {"name":"outCount", "type":"10i0", "value":1, "setlen":"outputA"}},
{"s": {"name":"outputA", "type":"10a", "dim":20, "dou": "outCount"}},
{"s": {"name":"last", "type":"20a", "value":"ll"}}
]}
Sample output:
{"script":[{"pgm":["DRTEST01","DROESSNER",{"inputDS":[[]]},{"outCount":1},{"outputA":["test1","","","","","","","","","","","","","","","","","","",""]},{"last":""quoted" text"}]}]}
Noticing the following issues from this example:
- inputDS specifies "by": "in", but the output still passes it back
- outputA array should only be the size of outCount
- Quotes are not escaped in strings so the output JSON is invalid
Let me know if you need me to provide any additional info.
Comments (18)
-
reporter -
Account Deleted You are proposing a new feature for simple data ("s"). The functions dealing with "dou": "outCount" are only available with structures ("ds").
"dou": "outCount" ... not available (only "ds")
No "dou": "outCount" feature for simple data data ("s").
"setlen":"outputA" ... not available (only "ds").
No "setlen":"outputA" feature for simple data data ("s").
You are misusing "setlen" (length, not count)
Even if "setlen" supported for "s" (not supported). You are also misusing "setlen":"outputA", as value result of "outCount" is length 20 * 10 = 200. Aka, "setlen" meant for "length" of a "ds" to help with offset math of many MI and system API interfaces, not "count" of a ds.
More, maybe not "s" ...
Simple dimension data "s", dim(20), usually is considered one-array-simple-type. I am not sure we even want to attempt sub setting the output based on "count".
For example, most times simple "s" array data will be inside of a "ds" structure. This would get messy if we allow "chunks" of the "s" elements to disappear inside a "ds".
Mmmm ... i don't know about this idea. In fact, may be a bad idea for simple types.
work around, use "ds" (not "s")
To control "output count" of a simple data "s", simply surround with a "ds". That is, is no difference between "s":dim(20) and a "ds":dim(20) with one data element.
bash-4.3$ ./test1000_sql400json32 ../json/j0161_pgm_danny01-ds input(5000000): {"pgm":[ {"name":"DANNY01", "lib":"DB2JSON"}, {"s": {"name":"inCount", "type":"10i0", "value":1, "by":"in"}}, {"ds": [{"name":"inputDS","dim":20, "by": "in"}, {"s":[ {"name":"in1", "type":"5av2", "value":"i1"}, {"name":"in2", "type":"5av2", "value":"i2"} ]} ]}, {"s": {"name":"outCount", "type":"10i0", "value":1}}, {"ds": [ {"name":"workAround", "dim":20, "dou": "outCount"}, {"s": {"name":"outputA", "type":"10a"}} ]}, {"s": {"name":"last", "type":"20a", "value":"ll"}} ]} output(138): {"script":[{"pgm":["DANNY01","DB2JSON", {"inputDS":[[]]}, {"outCount":1}, {"workAround":[[{"outputA":"test1"}]]}, {"last":""quoted" text"}]}]} result: success (0)
Note: "setlen":"outputA" is removed from "name":"outCount" because you are not using "length" feature correctly. Aka, "outCount" result is 20 * 10 = 200 (offset length), not a "count" of elements.
-
Account Deleted Again ... You are proposing a new feature for simple data ("s"). The functions dealing with "dou": "outCount" and "setlen":"outputA" are only available with structures ("ds").
Mmmm ... i don't know about this idea. How bad do you want it???
-
reporter It would be nice to be able to have the ability to specify a counter for simple data since we are dealing with legacy code where data is passed in an array as described. I think going back and adding the proposed workaround is not an option due to the scope of the work, but if you are against adding this functionality we can also just add a post processing procedure to strip out the additional data before returning to the client.
Having it built in would be preferred, but is not necessary as there are different ways to work around it.
-
Account Deleted Quotes are not escaped in strings so the output JSON is invalid
Ok, I added escape to json string output (i think).
Yips Super Driver - test driver - 1.1.1-sg8 - toolkit added escape for json strings returned (Danny R)
bash-4.3$ ./test1000_sql400json32 ../json/j0161_pgm_danny01-ds input(5000000): {"pgm":[ {"name":"DANNY01", "lib":"DB2JSON"}, {"s": {"name":"inCount", "type":"10i0", "value":3, "by":"in"}}, {"ds": [{"name":"inputDS","dim":20, "by": "in"}, {"s":[ {"name":"in1", "type":"5av2", "value":"i1"}, {"name":"in2", "type":"5av2", "value":"i2"} ]} ]}, {"s": {"name":"outCount", "type":"10i0", "value":1}}, {"ds": [{"name":"workAround", "dim":20, "dou": "outCount"}, {"s": {"name":"outputA", "type":"10a"}} ]}, {"s": {"name":"last", "type":"20a", "value":"ll"}} ]} output(184): {"script":[{"pgm":["DANNY01","DB2JSON", {"inputDS":[[]]}, {"outCount":3}, {"workAround":[[{"outputA":"test1"}],[{"outputA":"test2"}],[{"outputA":"test3"}]]}, {"last":"\"quoted\" text"}]}]} result: success (0)
This needs more testing in your environment.
-
Account Deleted ... would be nice to be able to have the ability to specify a counter for simple data since we are dealing with legacy code where data is passed in an array as described.
Delay. I want to think about effects of change to replicate advance "counting" functions of "ds" into "s". Will let you know.
-
Account Deleted ... would be nice to be able to have the ability to specify a counter for simple data
Ok, added dou and dob to simple data ("s").
Yips Super Driver - tests driver - 1.1.1-sg9 - toolkit add output dou and dob “s” data array (Danny R)
option 1) "s" dou - do until other data counter field (request)
This will terminate data "s" array output at other field limit.
bash-4.3$ ./test1000_sql400json32 ../json/j0162_pgm_danny01-ds_dou input(5000000): {"pgm":[ {"name":"DANNY01", "lib":"DB2JSON"}, {"s": {"name":"inCount", "type":"10i0", "value":3, "by":"in"}}, {"ds": [{"name":"inputDS","dim":20, "by": "in"}, {"s":[ {"name":"in1", "type":"5av2", "value":"i1"}, {"name":"in2", "type":"5av2", "value":"i2"} ]} ]}, {"s": {"name":"outCount", "type":"10i0", "value":1}}, {"s": {"name":"outputA", "type":"10a", "dim":20, "dou": "outCount"}}, {"s": {"name":"last", "type":"20a", "value":"ll"}} ]} output(139): {"script":[{"pgm":["DANNY01","DB2JSON",{"inputDS":[[]]}, {"outCount":3}, {"outputA":["test1","test2","test3"]}, {"last":"\"quoted\" text"}]}]}
BTW -- Again, "dou": "outCount", you are using "setlen" wrong, so change your test for outCount limit.
option 2) "s" dob - do until blank/zero
This will terminate data "s" array output at first zero/blank array element.
bash-4.3$ ./test1000_sql400json32 ../json/j0163_pgm_danny01-ds_dob input(5000000): {"pgm":[ {"name":"DANNY01", "lib":"DB2JSON"}, {"s": {"name":"inCount", "type":"10i0", "value":3, "by":"in"}}, {"ds": [{"name":"inputDS","dim":20, "by": "in"}, {"s":[ {"name":"in1", "type":"5av2", "value":"i1"}, {"name":"in2", "type":"5av2", "value":"i2"} ]} ]}, {"s": {"name":"outCount", "type":"10i0", "value":1}}, {"s": {"name":"outputA", "type":"10a", "dim":20, "dob": "on"}}, {"s": {"name":"last", "type":"20a", "value":"ll"}} ]} output(139): {"script":[{"pgm":["DANNY01","DB2JSON", {"inputDS":[[]]},{"outCount":3}, {"outputA":["test1","test2","test3"]}, {"last":"\"quoted\" text"}]}]}
This needs testing in your environment.
-
Account Deleted update for dob zero (not Danny requested)
Yips Super Driver - tests driver - 1.1.1-sg10 - toolkit fix dob "s" array zero terminate output
-
reporter Appreciate the quick fixes on this again. I will load the new build today and do some more testing.
-
Looks like the "dou" on the simple data works for me in sg10 using the sample rpg/call (no setlen on the counter) when the count > 0. (Note "dob":"on" is needed when the count = 0).
Still getting inputDS returned when "by":"in" is set.
-
Account Deleted Looks like the "dou" on the simple data works for me in sg10
Cool. Hopefully continues on more tests (should work).
when the count > 0. (Note "dob":"on" is needed when the count = 0).
Mmmm ... "dou":"label" should work for associated data parm with count == 0. Aka, you should not have to add "dob":"on".
Basically for readers.
1) "dou":"label" - limits data array elements to 'count' value in associated "label" count parm (0 should work)
2) "dob":"on" - lazy man way to stop array output when reaching first array element blank or zero (nothing to see past blank or zero in this array).
I will need to look at this count == 0 issue. Thanks.
-
Account Deleted "dou:"count" == 0 issue. Thanks.
Ok. I have a new version with this problem fixed for simple array data limitcount.
Yips Super Driver - test driver - 1.1.2-sg3 - toolkit “dou”:”label” fix count equal zero (Danny R)
bash-4.3$ ./test1000_sql400json32 ../json/j0166_pgm_danny01-ds_dou input(5000000): {"pgm":[ {"name":"DANNY01", "lib":"DB2JSON"}, {"s": {"name":"inCount", "type":"10i0", "value":0, "by":"in"}}, {"ds": [{"name":"inputDS","dim":20, "by": "in"}, {"s":[ {"name":"in1", "type":"5av2", "value":"i1"}, {"name":"in2", "type":"5av2", "value":"i2"} ]} ]}, {"s": {"name":"outCount", "type":"10i0", "value":0}}, {"s": {"name":"outputA", "type":"10a", "dim":20, "dou": "outCount"}}, {"s": {"name":"last", "type":"20a", "value":"ll"}} ]} output(116): {"script":[{"pgm":["DANNY01","DB2JSON", {"inputDS":[[]]}, {"outCount":0}, {"outputA":[]}, {"last":"\"quoted\" text"}]}]} result: success (0)
BTW -- You do not have to be concerned about the other php async work in this driver. That work has nothing to do with your current activities. (However, if you are a node toolkit user/author, we should replace current 'bad' toolkit for a 'good' async model based on db2sock work SQL400JsonAsync w/callback).
-
Account Deleted {"inputDS":[[]]} ... still getting inputDS returned when "by":"in" is set.
Mmm ... bit more complex to make "go away". I will look into issue.
-
@rangercairns
2) "dob":"on" ...
Yeah I realized it was the lazy way and would rather just use "dou":"label" so I'm glad it is fixed now in the latest.
-
Account Deleted {"inputDS":[[]]} ... still getting inputDS returned when "by":"in" is set.
Ok. Changes to remove empty inputDS.
- Yips Super Driver - tests driver - 1.1.2-sg4 - toolkit remove extra “inputDS” when “by”:”in” (Danny R)
$ ./test1000_sql400json32 ../json/j0162_pgm_danny01-ds_dou input(5000000): {"pgm":[ {"name":"DANNY01", "lib":"DB2JSON"}, {"s": {"name":"inCount", "type":"10i0", "value":3, "by":"in"}}, {"ds": [{"name":"inputDS","dim":20, "by": "in"}, {"s":[ {"name":"in1", "type":"5av2", "value":"i1"}, {"name":"in2", "type":"5av2", "value":"i2"} ]} ]}, {"s": {"name":"outCount", "type":"10i0", "value":1}}, {"s": {"name":"outputA", "type":"10a", "dim":20, "dou": "outCount"}}, {"s": {"name":"last", "type":"20a", "value":"ll"}} ]} output(122): {"script":[{"pgm":["DANNY01","DB2JSON", {"outCount":3}, {"outputA":["test1","test2","test3"]}, {"last":"\"quoted\" text"}]}]} result: success (0)
-
Account Deleted Have you tried this yet? I would Ike to remove this issue.
-
I retested this in sg5. The inputDS no longer is returned in the output. Unfortunately I cannot mark this as resolved.
-
reporter - changed status to resolved
All items reported are fixed. Marking as resolved.
- Log in to comment