If the signature on the registration request has expired it MUST mark the registration as invalid and demand that the RP MUST re-register
The spec currently has a gap about how the OP is to signal to the RP the fact that its registration has expired (assuming that was the intent of “demand”).
If we assume std RFC 6749 behaviour this would mean the
client_id is no longer valid. But with a invalid client_id the OP / AS is not allowed to redirect back to the RP.
One possible solution is to define a special error code and let the redirection proceed. The significant downside of that is the OP will need to store expired registrations, potentially indefinitely.
Perhaps the simple solution is to not have any explicit signalling from OP to RP at all, but let the RP figure out the time when its registration is going to expire. This will simplify the implementation of the OP.