Script output truncated in monit status

[Styx]

Hello,

I have a script that sends a long output (>500 characters) and it’s truncated most of the time (not always the same length is truncated) in the monit status.

To reproduce this easily, I did a simple test where I wrote a script that just outputs a string of 1000 “1” character and using monit to run that script and check the return code..

When running monit status , the output captured only shows 202 characters out of the 1000 expected :

root@OPNsense-primary:/usr/local/mycustomscripts # ./myTestScript.sh
1111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
root@OPNsense-primary:/usr/local/mycustomscripts # monit status my_test
Monit 5.32.0 uptime: 18d 16h 1m

Program 'my_test'
  status                       OK
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  last exit value              3
  last output                  111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
  data collected               Thu, 05 May 2022 10:28:20

This is monit 5.32 on FreeBSD 13.0

[Lutz Mader]

Hello Styx,
try to increase the default limit in the monitrc file used by Monit.

set limits {
    programOutput:     1024 B,    # check program's output truncate limit
    fileContentBuffer: 1024 B,    # limit for file content test
}

Some additional information are available in https://www.mmonit.com/monit/documentation/monit.html#LIMITS

A suggestion only,
Lutz

p.s.

You can add a line break/new line after 80 or 100 character and you will see some more output in multiple lines.

‌

[Tildeslash]

The buffer size for the program output can be tweaked using the “set limits”: https://mmonit.com/monit/documentation/monit.html#LIMITS. The default value is 512B (the whole string van be compared using the content test (https://mmonit.com/monit/documentation/monit.html#PROGRAM-OUTPUT-CONTENT-TEST).

‌

The limitation you see is related only to the “monit status” command terminal output, which is limited to first 200 characters to prevent the terminal being unreadable due to huge program output.

[Tildeslash]

not bug

[Styx]

p.s.

You can add a line break/new line after 80 or 100 character and you will see some more output in multiple lines.

Hmmm, I guess I was tricked by the multi-lines output that potentially displays more output. That made this look like a bug to me (because sometimes I would get an output with ~500 characters, sometimes with ~700. So I did not think it could be a config parameter that controlled that (as it looked to me it was not always the same number of characters).

Thank you for pointing me to the documentation

But now, I am not sure to understand how that limit exactly works ? if with multi-line output you can exceed the limit, so does that mean that the limit is a per line (until it finds a line break) limit ?

‌

[Tildeslash]

The “programOutput“ limit doesn’t control the monit status terminal width limitation. The output is printed to the terminal by the _formatStatus function which you can see in https://bitbucket.org/tildeslash/monit/src/master/src/http/cervlet.c. The Monit HTTP GUI will display the whole program output, but the terminal output limits the line width to 200 characters (there’s no option to increase this limit currently, nor the terminfo is used to get the terminal width).

[Lutz Mader]

Hello Tildeslash,
sorry but you are wrong, sorry!

Maybe I am wrong, but I get something like this (for my test program with monit status statusinfo).

Program 'StatusInfo'
  status                       OK
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  last exit value              0
  last output                  1
                               2 12345678901234567890123456789012345678901234567890
                               3 12345678901234567890123456789012345678901234567890
                               4 12345678901234567890123456789012345678901234567890
                               5 12345678901234567890123456789012345678901234567890
                               6 12345678901234567890123456789012345678901234567890
                               7 12345678901234567890123456789012345678901234567890
                               8 12345678901234567890123456789012345678901234567890
                               9 12345678901234567890123456789012345678901234567890
                               A 12345678901234567890123456789012345678901234567890
                               B 12345678901234567890123456789012345678901234567890
                               C
  data collected               Thu, 05 May 2022 19:47:14

I like this additional output, this is very helpful and is available via the Web GUI also.

With regards,
Lutz

[Tildeslash]

Lutz, as mentioned in the previous post, the _formatStatus function cuts each line after 200 characters. Multiple lines are supported.

No line in your example has more then 200 characters, so you see full output.

Styx’s example is cut as the whole output in his example is stored in one very long line. If there would be multiple lines, each of them will show up to 200 characters

[Styx]

I have to agree with Lutz, below the kind of output I get through monit from a script I used to check on VPN connections (I redacted some parts):

Program 'ipsec_status_connections'
  status                       OK
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  last exit value              3
  last output                  Security Associations (3 up, 0 connecting):
                               XXXXXXXXXXXXXX[137690]: ESTABLISHED 13 seconds ago, XXX.XXX.X.XXX[XXXXXX.XXXXXXX.XXX]...XXX.XX.XXX.XX[C=XX, ST=XX, L=XXXXXXX, O=XXXX XXX, E=[e-mail], CN=XXXXXXXXXXXXXXXXXXX]
                               XXXXXXXXXXXXXX{267311}:  INSTALLED, TUNNEL, reqid 2, ESP in UDP SPIs: XXXXXXXX_X XXXXXXXX_X
                               XXXXXXXXXXXXXX{267311}:   XX.XX.X.XX/XX XX.XX.X.X/XX === XXX.XXX.XX.XX/XX
                               XXXXXXXXXXXXXX{267312}:  INSTALLED, TUNNEL, reqid 2, ESP in UDP SPIs: XXXXXXXX_X XXXXXXXX_X
                               XXXXXXXXXXXXXX{267312}:   XX.XX.X.XX/XX XX.XX.X.X/XX === XXX.XXX.XX.XX/XX
                               XXXXXXXXXXXXXX[137691]: ESTABLISHED 3 seconds ago, XXX.XXX.X.XXX[XXXXXX.XXXXXXX.XXX]...XXX.XX.XXX.XX[C=XX, ST=XX, L=XXXXXXX, O=XXXX XXX, E=[e-mail], CN=XXXXXXXXXXXXXXXXXXX]
                               XXXXXXXXXXXXXX{26731
  data collected               Thu, 05 May 2022 10:16:18

That’s 1293 character in the “last output”. And you can see that it’s truncated, the last line only contains 21 character as it gets truncated.

The real output from the script contains 5 or 6 more lines after that.

[Tildeslash]

@Styx as explained, the “monit status” shows only first 200 characters of each program output line. Your latest example didn’t exceed 200 character on any line, but it probably did hit the “set limits { programOutput: ...} limit (the hard limit on program output).

[Styx]

Ok make sense. Thank you !

[Lutz Mader]

Hello Tildeslash,
you are right. The 200 characters limit is for each line.

but the terminal output limits the line width to 200 characters

I try to limit the line length to max 60 (sometimes 80) characters only and limit the number of lines to 9 in my scripts. This is a useful limit for me and the tools I used.

Hello Styx,
like suggested, the output should be useful and easy to read. And you should use ASCII characters only.

With regards,
Lutz

‌

p.s.

The slides show a nice sample, see https://mmonit.com/monit/#documentation, page “Output from check program“.

In the service details page, up to 1kB is displayed and if output goes over several lines, a mono-spaced font is used to retain formating. Here for example, we display the output of running top and sensors (press right-arrow).

‌

[Styx]

Thx for the advice, i’ll add some output formatting in my script to make sure it does not get truncated in a weird way.