kuka controller misbehaving

[Former user]

I have noticed that if the Kuka robot has a collision or a sustained contact (e.g. pressing down on a part), its controller subsequently misbehaves. The joints no longer go to their commanded angles. (There might be other causes as well).

It MIGHT be that this is due to integral-error wind-up (with a very slow extinction). Or could it be something else?

Also, trajectory goals often return the state "ABORTED" to the trajectory action client. Is this related?

Regardless, can the robot controller robustness be improved? It presents lots of problems.

[Rud Merriam]

Also noted the controller is confused after a sustained collision. It was pushing down on a box and then wouldn't move properly after. This was using the RQT joint positioning to test some limits and actions.

If you watch the MoveIt launch in the terminal you might see additional error indications on the abort. I've seen it abort because the final movement was not within tolerance limits. Don't know what the tolerances are by default. I set greater tolerances and the errors went away. Even when it aborted it was where I wanted it to be.

[Tudor Donca]

The goal tolerance limits are defined in the osrf_gear/vendor/iiwa_control/config/ariac_control.yaml to be 0.1 for each joint.

I have noticed the same problem with trajectory executions, and after a few tests, it seems that iiwa_joint_3 is the cause, usually ending up about +/- 0.15 from the goal value. Then it very slowly converges to the desired value. iiwa_joint_5 rarely also has problems converging, but all of the even numbered joints end up almost perfectly in the goal value.

[Tudor Donca]

I also noticed a possible typo in the aric_control.yaml file at lines 29-30 where iiwa_joint_2 is missing.

[Joey Gannon]

I've also seen weird issues with the odd-numbered joints. I started the system, let it settle for a full minute, and with the arm still straight up and not moving, I'd expect the joint velocities to be close to zero. Instead, this is the output of rostopic echo /ariac/joint_states | grep velocity:

velocity: [0.23048123440235208, 0.0029024705005993184, -0.7251514868212177, 0.029188904041764795, 1.6042213541580184, 0.24981515557240902, -4.200614455790818, 4.952220246703924e-05, 0.0]
velocity: [0.23048123440284196, 0.0029024704968727295, -0.7251514868211342, 0.02918890402840093, 1.60422135414941, 0.2498151554871007, -4.200614455758224, 4.952218264369258e-05, 0.0]
velocity: [0.23048123440332824, 0.002902470493145631, -0.7251514868210543, 0.02918890401503673, 1.6042213541408126, 0.24981515540175775, -4.200614455725614, 4.952216284020659e-05, 0.0]
velocity: [0.23048123440381402, 0.002902470489417438, -0.7251514868209658, 0.029188904001669375, 1.6042213541322008, 0.2498151553164439, -4.200614455693022, 4.9522143077598676e-05, 0.0]
velocity: [0.2304812344043047, 0.0029024704856908464, -0.7251514868208894, 0.02918890398831508, 1.6042213541235963, 0.24981515523113493, -4.200614455660424, 4.952212322613014e-05, 0.0]
velocity: [0.2304812344047786, 0.0029024704819877634, -0.7251514868208006, 0.029188903974949547, 1.604221354115007, 0.24981515514579603, -4.200614455627868, 4.9522103446230055e-05, 0.0]
velocity: [0.2304812344052734, 0.002902470478246487, -0.725151486820722, 0.02918890396157764, 1.6042213541064019, 0.2498151550604519, -4.20061445559525, 4.952208357361898e-05, 0.0]
velocity: [0.23048123440575025, 0.0029024704745462724, -0.7251514868206288, 0.02918890394822544, 1.6042213540977903, 0.24981515497511408, -4.200614455562657, 4.9522063837033646e-05, 0.0]
velocity: [0.23048123440624046, 0.002902470470805763, -0.7251514868205485, 0.029188903934847882, 1.604221354089192, 0.2498151548898015, -4.200614455530066, 4.952204394638828e-05, 0.0]
velocity: [0.23048123440672416, 0.002902470467076807, -0.7251514868204638, 0.02918890392148636, 1.6042213540805983, 0.24981515480444258, -4.200614455497473, 4.952202416290684e-05, 0.0]

The same behavior happens if you move the arm to a new position and let it settle there. Maybe that's meant to be part of the challenge though? I'm fine with working around it if that's the case.

[Deanna Hood]

Thanks to the various participants for pitching in on this. Reproducible examples to get the robot into this state will assist with our evaluation of the issue.

[Deanna Hood]

There are a few different points that have been brought up in this thread. I have a change prepared that I believe addresses them. It has not been released yet. If anyone is building from source, please test it out with the iiwa_controller_improvements branch.

(1) Trajectories get aborted.

As pointed out by @rmerriam, the trajectory being aborted is not always an issue, because actually the trajectory is not cancelled even if it is reported as aborted, so you can still end up at the right pose. As @doncat mentioned, the PID on the iiwa is a bit slow to converge for some joints, so just relaxing the tolerances should be enough to prevent this from being reported. You might still get some false positives, but if you relax it too much it's hard to distinguish between them and a collision.

(2) Controller behaviour changes after sustained collision.

I was able to reproduce what looks like the same issue without a collision, just with the arm's default state (all 0s). Trying to control only joint 7 with the joint trajectory controller in rqt, the joint gets stuck and doesn't respond to a change in desired value. Limiting the effort applied to that joint prevents it from getting stuck, in that situation at least.

I imagine that this is what was meant by the original poster/ @rmerriam with "joints no longer go to their commanded angles". I have only seen this happen with joint 7, so that's the only one that's been changed (to prevent unnecessary movement slowdown). Did anyone see other joints get stuck like this? If it's still an issue we can work on the other joints.

(3) Joints 3 and 5 are slow to converge.

Their parameters have been updated so they should be a bit faster now.

(4) Nonzero joint velocities.

@joeygannon Aside from the issue with joint 7 I haven't come across a situation where the nonzero joint velocity is causing an issue for the simulation or the arm, so I will let this be for now unless it's causing a specific issue.

[Deanna Hood]

@doncat @joeygannon @rmerriam we have released the change into the "pre-release" repository so it can be tested from binaries.

Please see https://bitbucket.org/osrf/ariac/wiki/2018/updates#markdown-header-upcoming-changes for instructions and report back if you have tested it.

[Deanna Hood]

After reports the issue can still occur, another iteration has been released into the pre-release repos.

You can check you got the update with:

dpkg -s ariac2 | grep Version

The version should start with "2.1.5-pre2" instead of pre1.

Looking for feedback on that version ASAP so it can be released in time for qual2 submissions.

[Deanna Hood]

Based on feedback received, 2.1.5~pre1 has been released as ariac2.1.6 (2.1.5~pre2 was originally released as ariac2.1.5, but subsequently we rolled back to 2.1.5~pre1 in ariac2.1.6).

[Deanna Hood]

Given the observed behaviour in qual2 submissions, the arm controller looks to be sufficient for the Finals.