Graph pruning after IndeterminantLinearSystemException

[Jeffrey Delmerico]

Hi GTSAM team,

This isn't a bug per-se, but I didn't get a response from an email to the info list ([e-mail]), so I am reposting here.

I'm a researcher with the Robotics and Perception Group at the University of Zurich, and I've been working with some code that uses gtsam as a backend for a visual odometry frontend (SVO). This was originally some work that came out of a collaboration between a former colleague from my lab (Christian Forster) and yours (with Luca and Frank).

I'm running into a problem where I frequently encounter an IndeterminantLinearSystemException during an iSAM update. The nearby variable listed in the exception is always a visual landmark, and although I know that doesn't necessarily mean that this is the problematic variable, there are no missing priors or any other obvious oversight in the process of building the graph, so my conclusion is that certain landmarks are underdetermined. Despite having multiple observations of these particular landmarks, and some checks to ensure that there is sufficient parallax between the observations, the system always becomes ill posed in some datasets.

I'm looking for a solution to this from the perspective of a VIO system where the success of a particular intermediate optimization is not critical. In other words, it would be ok to resolve the ill-posed system issue after an underconstrained variable is discovered, and then just try to optimize again the next time the backend is called.

Is it possible to remove the offending variable from the graph? We have many visual features and the loss of one poorly constrained one should be fine. I have attempted this by passing in the indices of the observations of this landmark as "removeFactorIndices" in the call to isam2::update (with no other arguments, as in the removeVariables test in tests/testGaussianISAM2.cpp), but this also produces the same exception.

Otherwise, can you recommend a principled way to resolve this issue? I have attempted to do some introspection as recommended by the documentation for the exception, but it's not clear how to output the Jacobian and Hessian through the C++ API when I am just interfacing with an instance of the iSAM2 class (trying to use the MATLAB API is not feasible due to the complexity of the software).

Any help will be greatly appreciated, and I'll be happy to provide more information if you like.

Thanks, Jeff

[Frank Dellaert]

There is some functionality to remove variables in iSAM, but it's off code and I saw several issues posted on it that have not been resolved. I have no time to resolve them now, but I would love to see them resolved :-) :-)

In terms of avoiding adding unconstrained landmarks, the key is to do a very careful screening of how good a landmark is before adding it to the global optimization (for at least a few frames). Doing this well is a key component of any visual SLAM pipeline, and one of the reasons visual SLAM is still hard.

[Jeffrey Delmerico]

Thank you very much for the insight. If I make any headway on the variable removal side, I will try to contribute it to the codebase, but more likely I need to dig deeper into the landmark screening that Christian set up. It has worked in the past for the results in the SVO+GTSAM papers, but on the datasets I am working with it is apparently insufficient. Thanks.

[Frank Dellaert]

Not a bug, and no activity in a long time. Likely Jeffrey found different way around it.

[Jeffrey Delmerico]

Yes, I came to the conclusion that there are fundamental properties of the frontend (SVO) that make its pairing with GTSAM problematic. When it works it is very efficient, but because SVO is so lightweight, it's easy for erroneous measurements to find their way into the backend and cause this type of crash. Closing the issue.