java.lang.NullPointerException when killing processes
Issue #5
new
We frequently see a null pointer exception when trying to kill processes. We use a process to kill regex that looks like this:
^.*?(dbus-daemon|firefox|postgres|python|Xvfb).*$
The exception is:
02-Nov-2017 10:18:01 Running pre-build action: Run Predator
02-Nov-2017 10:18:01 [PredatorPreBuildAction] Starting predator plugin, pre build agent cleaner
02-Nov-2017 10:18:01 [PredatorPreBuildAction] Running these commands:
02-Nov-2017 10:18:01 [PredatorPreBuildAction] Exception when trying to kill processes
02-Nov-2017 10:18:01 java.lang.NullPointerException
02-Nov-2017 10:18:01 at com.jezhumble.javasysmon.ProcessInfo.stringFormat(ProcessInfo.java:184)
02-Nov-2017 10:18:01 at com.jezhumble.javasysmon.ProcessInfo.stringFormat(ProcessInfo.java:179)
02-Nov-2017 10:18:01 at com.jezhumble.javasysmon.ProcessInfo.toString(ProcessInfo.java:160)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.plugin.predator.executor.PredatorExecutor.killProcesses(PredatorExecutor.java:240)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.plugin.predator.executor.PredatorExecutor.killProcesses(PredatorExecutor.java:214)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.plugin.predator.executor.PredatorPreBuildAction.call(PredatorPreBuildAction.java:130)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.runPreBuildActions(PrepareBuildTask.java:125)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:98)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:185)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl$1.call(BuildAgentControllerImpl.java:120)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl$1.call(BuildAgentControllerImpl.java:113)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:185)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:113)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:126)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:47)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
02-Nov-2017 10:18:01 at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
02-Nov-2017 10:18:01 at java.lang.Thread.run(Thread.java:745)
02-Nov-2017 10:18:01 [PredatorPreBuildAction] Initial free space: 301.83392 GB
02-Nov-2017 10:18:01 [PredatorPreBuildAction] Skipping the removal of build working directory
02-Nov-2017 10:18:01 [PredatorPreBuildAction] Skipping the removal of maven snapshots
Comments (5)
-
reporter
-
Account Deactivated
This looks to be caused by a 3rd party library Predator uses; see https://github.com/jezhumble/javasysmon/issues/39 for what's likely happening, specifically the second post in that thread.
-
reporter
I tried reproducing using the same method used at https://github.com/jezhumble/javasysmon/issues/39.
[grahamsi@localhost ~]$ java -cp "./javasysmon-0.3.5.jar" com.jezhumble.javasysmon.JavaSysMon OS name: Linux Uptime: 281 days 20 hours Current PID: 20064 Number of CPUs: 28 CPU frequency: 2400 MHz RAM total: 257675Mb free: 121857Mb SWAP total: 4095Mb free: 80Mb Sampling CPU usage... CPU Usage: 0.049053967 pid name ppid user total res time command ================================================================================ 1 systemd 0 root 188Mb 3Mb 00:37:38 /usr/lib/systemd/system 2 kthreadd 0 root 0Mb 0Mb 00:00:09 3 ksoftirqd 2 root 0Mb 0Mb 00:21:27 8 migration 2 root 0Mb 0Mb 00:02:42 9 rcu_bh 2 root 0Mb 0Mb 00:00:00 10 rcu_sched 2 root 0Mb 0Mb 01:54:51 11 watchdog 2 root 0Mb 0Mb 00:00:30 12 watchdog 2 root 0Mb 0Mb 00:00:35 13 migration 2 root 0Mb 0Mb 00:02:28 14 ksoftirqd 2 root 0Mb 0Mb 00:01:02 17 watchdog 2 root 0Mb 0Mb 00:00:29 18 migration 2 root 0Mb 0Mb 00:01:31 19 ksoftirqd 2 root 0Mb 0Mb 00:00:43 22 watchdog 2 root 0Mb 0Mb 00:00:26 23 migration 2 root 0Mb 0Mb 00:00:42 24 ksoftirqd 2 root 0Mb 0Mb 00:00:32 27 watchdog 2 root 0Mb 0Mb 00:00:24 28 migration 2 root 0Mb 0Mb 00:00:43 29 ksoftirqd 2 root 0Mb 0Mb 00:00:33 32 watchdog 2 root 0Mb 0Mb 00:00:25 33 migration 2 root 0Mb 0Mb 00:00:38 34 ksoftirqd 2 root 0Mb 0Mb 00:00:31 37 watchdog 2 root 0Mb 0Mb 00:00:21 38 migration 2 root 0Mb 0Mb 00:00:31 39 ksoftirqd 2 root 0Mb 0Mb 00:00:30 42 watchdog 2 root 0Mb 0Mb 00:00:24 43 migration 2 root 0Mb 0Mb 00:00:31 44 ksoftirqd 2 root 0Mb 0Mb 00:00:28 47 watchdog 2 root 0Mb 0Mb 00:00:26 48 migration 2 root 0Mb 0Mb 00:00:26 49 ksoftirqd 2 root 0Mb 0Mb 00:00:28 52 watchdog 2 root 0Mb 0Mb 00:00:24 53 migration 2 root 0Mb 0Mb 00:00:25 54 ksoftirqd 2 root 0Mb 0Mb 00:00:27 57 watchdog 2 root 0Mb 0Mb 00:00:23 58 migration 2 root 0Mb 0Mb 00:00:25 59 ksoftirqd 2 root 0Mb 0Mb 00:00:25 62 watchdog 2 root 0Mb 0Mb 00:01:17 63 migration 2 root 0Mb 0Mb 00:00:15 64 ksoftirqd 2 root 0Mb 0Mb 00:00:25 67 watchdog 2 root 0Mb 0Mb 00:00:17 68 migration 2 root 0Mb 0Mb 00:00:07 69 ksoftirqd 2 root 0Mb 0Mb 00:00:20 72 watchdog 2 root 0Mb 0Mb 00:00:23 73 migration 2 root 0Mb 0Mb 00:00:26 74 ksoftirqd 2 root 0Mb 0Mb 00:00:23 77 watchdog 2 root 0Mb 0Mb 00:00:48 78 migration 2 root 0Mb 0Mb 00:01:02 79 ksoftirqd 2 root 0Mb 0Mb 00:01:13 83 watchdog 2 root 0Mb 0Mb 00:00:40 84 migration 2 root 0Mb 0Mb 00:02:45 85 ksoftirqd 2 root 0Mb 0Mb 00:01:10 88 watchdog 2 root 0Mb 0Mb 00:00:33 89 migration 2 root 0Mb 0Mb 00:01:40 90 ksoftirqd 2 root 0Mb 0Mb 00:00:58 93 watchdog 2 root 0Mb 0Mb 00:00:29 94 migration 2 root 0Mb 0Mb 00:00:55 95 ksoftirqd 2 root 0Mb 0Mb 00:00:43 98 watchdog 2 root 0Mb 0Mb 00:00:27 99 migration 2 root 0Mb 0Mb 00:01:01 100 ksoftirqd 2 root 0Mb 0Mb 00:00:46 103 watchdog 2 root 0Mb 0Mb 00:00:30 104 migration 2 root 0Mb 0Mb 00:00:50 105 ksoftirqd 2 root 0Mb 0Mb 00:00:46 108 watchdog 2 root 0Mb 0Mb 00:00:28 109 migration 2 root 0Mb 0Mb 00:00:45 110 ksoftirqd 2 root 0Mb 0Mb 00:00:43 113 watchdog 2 root 0Mb 0Mb 00:00:17 114 migration 2 root 0Mb 0Mb 00:00:54 115 ksoftirqd 2 root 0Mb 0Mb 00:00:39 118 watchdog 2 root 0Mb 0Mb 00:00:26 119 migration 2 root 0Mb 0Mb 00:00:33 120 ksoftirqd 2 root 0Mb 0Mb 00:00:39 123 watchdog 2 root 0Mb 0Mb 00:00:20 124 migration 2 root 0Mb 0Mb 00:00:41 125 ksoftirqd 2 root 0Mb 0Mb 00:00:52 128 watchdog 2 root 0Mb 0Mb 00:00:29 129 migration 2 root 0Mb 0Mb 00:00:31 130 ksoftirqd 2 root 0Mb 0Mb 00:00:37 133 watchdog 2 root 0Mb 0Mb 00:00:28 134 migration 2 root 0Mb 0Mb 00:00:32 135 ksoftirqd 2 root 0Mb 0Mb 00:00:35 138 watchdog 2 root 0Mb 0Mb 00:00:17 139 migration 2 root 0Mb 0Mb 00:00:38 140 ksoftirqd 2 root 0Mb 0Mb 00:00:36 143 watchdog 2 root 0Mb 0Mb 00:00:24 144 migration 2 root 0Mb 0Mb 00:00:29 145 ksoftirqd 2 root 0Mb 0Mb 00:00:33 150 khelper 2 root 0Mb 0Mb 00:00:00 151 kdevtmpfs 2 root 0Mb 0Mb 00:00:00 152 netns 2 root 0Mb 0Mb 00:00:00 153 khungtaskd 2 root 0Mb 0Mb 00:00:51 154 writeback 2 root 0Mb 0Mb 00:00:00 155 kintegrity 2 root 0Mb 0Mb 00:00:00 156 bioset 2 root 0Mb 0Mb 00:00:00 157 kblockd 2 root 0Mb 0Mb 00:00:00 159 md 2 root 0Mb 0Mb 00:00:00 168 kswapd0 2 root 0Mb 0Mb 00:29:52 169 kswapd1 2 root 0Mb 0Mb 00:25:52 170 ksmd 2 root 0Mb 0Mb 00:00:00 171 fsnotify_m 2 root 0Mb 0Mb 00:00:12 172 crypto 2 root 0Mb 0Mb 00:00:00 180 kthrotld 2 root 0Mb 0Mb 00:00:00 184 kmpath_rda 2 root 0Mb 0Mb 00:00:00 189 kpsmoused 2 root 0Mb 0Mb 00:00:00 190 ipv6_addrc 2 root 0Mb 0Mb 00:00:00 210 deferwq 2 root 0Mb 0Mb 00:00:00 266 kauditd 2 root 0Mb 0Mb 00:00:00 376 kworker 2 root 0Mb 0Mb 00:00:01 Exception in thread "main" java.lang.NullPointerException at com.jezhumble.javasysmon.ProcessInfo.stringFormat(ProcessInfo.java:184) at com.jezhumble.javasysmon.ProcessInfo.stringFormat(ProcessInfo.java:179) at com.jezhumble.javasysmon.ProcessInfo.toString(ProcessInfo.java:160) at com.jezhumble.javasysmon.JavaSysMon.main(JavaSysMon.java:98)The process that apparently causes the exception (having the next highest PID) is this one:
[grahamsi@localhost ~]$ ps aux | grep 384 zookeeper 384 0.1 0.2 37548996 630776 ? Sl Nov02 2:23 java -Dzookeeper.log.dir=/var/log/zookeeper/3.4.9/log -Dzookeeper.root.logger=INFO,CONSOLE -cp /opt/third/zookeeper/3.4.9/bin/../build/classes:/opt/third/zookeeper/3.4.9/bin/../build/lib/*.jar:/opt/third/zookeeper/3.4.9/bin/../lib/slf4j-log4j12-1.6.1.jar:/opt/third/zookeeper/3.4.9/bin/../lib/slf4j-api-1.6.1.jar:/opt/third/zookeeper/3.4.9/bin/../lib/netty-3.10.5.Final.jar:/opt/third/zookeeper/3.4.9/bin/../lib/log4j-1.2.16.jar:/opt/third/zookeeper/3.4.9/bin/../lib/jline-0.9.94.jar:/opt/third/zookeeper/3.4.9/bin/../zookeeper-3.4.9.jar:/opt/third/zookeeper/3.4.9/bin/../src/java/lib/*.jar:/opt/third/zookeeper/3.4.9/bin/../conf: -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=false org.apache.zookeeper.server.quorum.QuorumPeerMain /opt/third/zookeeper/3.4.9/bin/../conf/zoo.cfgI don't notice anything at all unusual about that process.
-
reporter
Running the same test on another bunch of machines I see this error: https://github.com/jezhumble/javasysmon/issues/36, I notice that he javasysmon library appears to be orphaned. There are no commits since 2014. I suppose the chance of getting any of these issues fixed upstream will be very slim. Perhaps there are some similar libs that might do a better job?
-
Account Deactivated
Unfortunately it's unlikely a rewrite of the plugin is happening any time soon. You're free to submit a PR to our fork of JavaSysMon which we are using in this plugin: https://github.com/atlassian/javasysmon or a PR to the original repo and I'll merge it.
- Log in to comment
This is seen using Bamboo v6.1.1 and Predator Plugin v2.0.12.