Reaching number of file open limit

[Jean-Baptiste Ruffio]

Can't read many files in the dataset structure.

It looks like the process_file function creates a lot of file handles that are not closed by the hdulist.close() statement. Are maybe something else?

This looks relevant: https://mail.scipy.org/pipermail/astropy/2013-June/002609.html

[Jason Wang]

Please provide some way to reproduce this bug. How many files are you reading in?

[Jean-Baptiste Ruffio]

Actually Kate had the issue with MAGAO. Never had it with GPI. She is apparently reaching the 4000 files limit set up on our computer.

[Jason Wang]

It sounds like the easy fix would be to increase the file limit on your computer. Otherwise, it sounds like the next thing to try would be applying the fix in the link to the MagAO.py code that reads in the files.

[Jean-Baptiste Ruffio]

Yeah that's what she did. But I feel like that's something that should not happen. Probably not urgent to fix though.

[Jason Wang]

Ok, well all code to read in files is in MagAO.py for this specific case, so I will assign this issue for Kate to look into.