OK take a look at what I did in b61eccce5e45651ccdb25. im not sure about the existing_server_default=None thing, if there's some problem there that would be a separate issue, I added a9426a4ec4f14aa12c17f to test that also.
Yeah, I think that does the trick. I've tested this on a live mysql db with the use case which discovered this issue.
Oop. There is one buglet when attempting to drop the server_default along with other column changes. Doing an
op.alter_column(..., server_default=None, nullable=True, ...) results in AttributeError: 'NoneType' object has no attribute '_compiler_dispatch'.