revid-cli: memory leak in MTS mode
Issue #184
resolved
This happens on master but was first observed as of commit 6dccc5c4d5074e7a8093e378566a098b70ff2b2f. It may well predate that commit though.
Note 59.5% memory usage was reached after just a few hours of running:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2222 pi 20 0 862408 224912 4236 S 3.9 59.5 5:35.66 revid-cli
This speaks to the need for a performance test to detect such regression, a separate issue.
Comments (20)
-
-
reporter
There appears to be no plateauing or slowdown, etc. Memory increases monotonically until it exhausts all available memory and revid-cli crashes.
It is readily reproducible (within minutes).
-
wow okay, that’s bad, I’ll have a look.
What are your MTSRBSize and MTSRBElementSize vars set to (if being set)? -
reporter
MTSRBElementSize is 200000 and MTSRBSize is the default.
FYI, I’ve been repro’ing sending to a standalone vidgind and simultaneously live streaming via HLS.
-
Okay I’m going to do some testing to try and narrow down what the cause is. If you wanted to try something, set MTSRBSize to 100. I’m interested to see how differences in the ring buffer sizes affect the behaviour.
-
reporter
I’ve just restated revid with MTSRBSize = 100 and will track memory usage.
-
reporter
FYI, reducing the MTSRBSize does not appear to curb the memory leak.
-
Really? I’ve had revid-cli running on a pi zero for a couple of hours… for the past hour mem usage has stayed at 16.2%. This is with MTSRBSize=100 and MTSRBElementSize=300000. I’m officially confused if you’re still seeing the same behaviour. The only difference I think there would be between the zero I’m using and yours is that you’re using the latest version of raspbian ?
-
I’m using go1.13 fyi
-
What do you have MinFrames set to (if set)? and what bitrate (if set)? I will try to replicate your conditions.
-
reporter
pi@PI000012:~ $ go version
go version go1.13 linux/armpi@PI000012:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)" -
I’m using v9 (stretch)
-
reporter
MinFrames and bitrate are not set. Everything is defaults except for MTSRBSize and MTSRBElementSize. The latter is still 200000. I will bump that up now.
-
reporter
Please move to Buster (v10), which has been out for a while. I’m configuring all of our new cameras to use it.
-
Yeah okay - I’ve defaulted MinFrames and Bitrate now too. I’ll see how that goes.
-
Do you have a stretch image handy ? You could try that with same vars and see how it goes. I’ll have to download buster (it’ll take a while on my home conn)
EDIT: shouldn’t take too long - I’m going with buster lite -
reporter
No, I don’t have any Stretch images any more. Do you just want to pick up PI00012 and play with it?
-
I reckon I’ll just update the pi I’ve got to buster doing this then if that doesn’t give me any answers I’ll swing by and pick up PI00012
-
@Alan Noble do you think we are okay to resolve this one ?
-
reporter
- changed status to resolved
Resolved in commit 6b5911633af245cbefc8412d3d7b272c0ef0e40e
- Log in to comment
Is there a particular sign that indicates that it is a memory leak? I’ve observed this phenomenon for a while; memory usage does quickly increase, but the rate of increase slows and usage plateaus.