Skip to content

align failed handshake behavior - #402

Merged
kares merged 3 commits into
masterfrom
ssl-align-alert2
Nov 4, 2026
Merged

kares merged 3 commits into
masterfrom
ssl-align-alert2

Conversation

@kares

@kares kares commented Nov 4, 2026

Copy link
Copy Markdown
Member

expected to address #399

primary fix is the same as in #400 but we have some extra work to e.g. align the reported verify-result (with MRI)

toabctl and others added 3 commits November 4, 2026 10:58
When BCJSSE's unwrap() fails during handshake (e.g. the trust manager
rejects the server cert), the fatal alert is queued for output. unwrap()
does not throw: saves the exception (ProvSSLEngine.deferredException)
and asks for NEED_WRAP.

The next wrap() throws the saved exception and writes nothing.

doShutdown() stopped at that first exception, so no alert was sent;
a server whose peer kept the TCP socket open then blocked in accept
indefinitely.

regression since 0.19.2 - reduced shutdown to one doShutdown() call
rejected peer certificate raised a bare "certificate verify failed" on
SunJSSE and a synthetic "Socket closed" on BCJSSE - in both cases
`SSLSocket#verify_result` stayed 0;
while MRI reports the verify error in the message and in verify_result

trust manager now throws VerifyCertificateException; which carries the
verify error, connect/accept find it in the cause chain of the JSSE
exception and set the message and verify_result.

BCJSSE holds trust failure back until the next wrap(); when unwrap hits
end of stream while the engine still asks for NEED_WRAP, doHandshake now
runs that wrap, so real failure is raised instead of "Socket closed".
@kares kares added this to the 0.19.5 milestone Nov 4, 2026
@kares kares linked an issue Nov 4, 2026 that may be closed by this pull request
@kares
kares merged commit 9b09a7f into master Nov 4, 2026
236 of 237 checks passed
@kares
kares deleted the ssl-align-alert2 branch November 4, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Server accept hangs when the client rejects the certificate (since 0.19.2)

2 participants