Nqueens using an unreasonable amount of memory

[Joachim Jansen]

Running Nqueens over and over stacks up memory quite fast.

The attached file does modelexpansion for N=50 100 times. These runs are completely separate, so in theory no intermediate results should be kept by IDP.

[Broes De Cat]

One problem is in lua itself, which seems to call garbage collection too lazily. Calling collectgarbage() manually (after each loop e.g.), considerably improves the performance. (but this does not completely solve it)

[Bart Bogaerts]

We can adapt lua, by setting the garbage-collector pause option to a better value

We can fix this by for example calling collectgarbage("setpause",120) at every startup of lua. By default this value is 200, meaning that lua performs garbage collect every time the used memory is doubled.

see http://www.lua.org/manual/5.1/manual.html#2.10 and http://www.lua.org/manual/5.1/manual.html#pdf-collectgarbage

[Broes De Cat]

Expect resolved if collectgarbage() is added to the lua code. (otherwise reopen)

[Joachim Jansen]

Compared vs commit a991986

Without collectgarbage(): new memory usage is 1.5 times as much as the old one With collectgarbage(): new memory usage is more than twice as much

Note: adding the lines in the attached file nqueens2.idp has little impact on the memory usage

[Joachim Jansen]

Changing the contents of the main method in nqueens2.idp to runTimedExperiment(1,30,1,5,queens1,"tmp",createGrid)

Yields even more obvious differences in the old commit versus the current version.

[Broes De Cat]

One reason is certainly that grounders are currently not deleted (was disabled because of lazy grounding interference), this might be the main cause of the problem. (one of the deletes in grounding.hpp)

[Joachim Jansen]

Cleaned up testfile.

The test file runs entirely with less than 4GB RAM memory as of commit https://bitbucket.org/krr/idp/commits/21165571d6e4126f0b14d64920147a8a283c3eca

This is still too much though. N=100 (on itself) is solved with <100MB RAM memory.

[Broes De Cat]

Remaining problems: Several objects are not deleted (soon enough): GroundingInference is not deleted intern_voc (created by groundinginference) is not deleted pre-interpreted predicate objects (e.g. equality) currently store a list of all interpretations they have generated. These are destroyed when the predicate objects are destroyed, but they are part of the std vocabulary, which is only destroyed when idp exits.

==> More deleted and shared_ptr for the predinters