[tnozaki-openssl] CVE-2023-3446 Excessive time spent checking DH keys and parameters

[Takehiko NOZAKI]

see https://www.openssl.org/news/secadv/20230719.txt

[Takehiko NOZAKI]

done <<cset:22d1d0cafc6197e52484ccf7c4d969cc12373706>>

[Takehiko NOZAKI]

BUGFIX: Issue #348 - CVE-2023-3446 Excessive time spent checking DH keys and parameters

Cherry-picked from: https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=8780a896543a654e757db1b9396383f9d8095528

Original commit message: Fix DH_check() excessive time with over sized modulus

The DH_check() function checks numerous aspects of the key or parameters that have been supplied. Some of those checks use the supplied modulus value even if it is excessively large.

There is already a maximum DH modulus size (10,000 bits) over which OpenSSL will not generate or derive keys. DH_check() will however still perform various tests for validity on such a large modulus. We introduce a new maximum (32,768) over which DH_check() will just fail.

An application that calls DH_check() and supplies a key or parameters obtained from an untrusted source could be vulnerable to a Denial of Service attack.

The function DH_check() is itself called by a number of other OpenSSL functions. An application calling any of those other functions may similarly be affected. The other functions affected by this are DH_check_ex() and EVP_PKEY_param_check().

CVE-2023-3446

Reviewed-by: Paul Dale [e-mail] Reviewed-by: Tom Cosgrove [e-mail] Reviewed-by: Bernd Edlinger [e-mail] Reviewed-by: Tomas Mraz [e-mail] (Merged from https://github.com/openssl/openssl/pull/21452)

→ <<cset 22d1d0cafc61>>