I've written the value float('nan') to a spreadsheet, and it doesn't seem like NaN is represented correctly. The cell comes back as 0 after opening the resulting spreadsheet. Should NaN be represented something like this in Excel?
#!NUM is an error code, NaN is technically a number – the equivalent of NULL – that can be used in arrays. So the two are not the same. Excel will just see an empty cell because it does not have an NaN concept. This might lead to #!NUM errors but that's not openpyxl's business.