[tnozaki-openssl] CVE-2023-3817 Excessive time spent checking DH q parameter value

[Takehiko NOZAKI]

see https://www.openssl.org/news/secadv/20230731.txt

[Takehiko NOZAKI]

done <<cset:4ec5cd4a717a183cc7b3c9c9c325828696be1668>>

[Takehiko NOZAKI]

BUGFIX: Issue #349 - CVE-2023-3817 Excessive time spent checking DH q parameter value

Cherry-picked from: https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=91ddeba0f2269b017dc06c46c993a788974b1aa5

Original commit message: DH_check(): Do not try checking q properties if it is obviously invalid

If |q| >= |p| then the q value is obviously wrong as q is supposed to be a prime divisor of p-1.

We check if p is overly large so this added test implies that q is not large either when performing subsequent tests using that q value.

Otherwise if it is too large these additional checks of the q value such as the primality test can then trigger DoS by doing overly long computations.

Fixes CVE-2023-3817

Reviewed-by: Paul Dale [e-mail] Reviewed-by: Matt Caswell [e-mail] (Merged from https://github.com/openssl/openssl/pull/21551)

→ <<cset 4ec5cd4a717a>>