Skip to content

HDDS-17462. Send structured errors from the OM gRPC server instead of exception class names - #11763

Open
peterxcli wants to merge 5 commits into
apache:masterfrom
peterxcli:HDDS-17462
Open

peterxcli wants to merge 5 commits into
apache:masterfrom
peterxcli:HDDS-17462

Conversation

@peterxcli

@peterxcli peterxcli commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

The OM gRPC server reports a failed submitRequest as Status.INTERNAL with the description class: message, and GrpcOmTransport rebuilds the exception from that text: it splits it at the first :, and the class is then created by reflection. This is fragile, it lets the server pick the class the client loads, and it keeps nothing but the message. In particular, the suggested leader in OMNotLeaderException never reaches the retry policy.

With this change:

  • A new OMGrpcErrorProto message in OmClientProtocol.proto carries the message and a oneof with a variant for each OM error the client handles: NotLeader (with the suggested leader node ID), LeaderNotReady, ReadIndexFailed and ReadFailed. No variant means any other error. The OM gRPC server sends it in a binary gRPC trailer.
  • OMGrpcError, its Java form, is the one place that maps an OM exception to the error on the server, and the variant back to the OM exception on the client. There is no text parsing and no reflection.
  • unwrapException gives the retry policy a ServiceException with the OM exception as its cause. The get*Exception helpers in OMFailoverProxyProviderBase now accept such a cause as well as a RemoteException, and the four of them share one implementation.
  • The retry policy finds the suggested leader by node ID. It used to compare the address too, but the OM sends InetAddress#toString (host/ip) while the proxy has host:port, so it never matched.

HDDS-17448 (#11760) already gives the retry policy the shape it recognizes, by parsing the description. This change keeps that behavior and replaces the parsing.

Compatibility: the description keeps its old format, so older gRPC clients still work. A new client talking to an older OM gets no trailer, treats the error as a plain failure and fails over to the next OM. The Hadoop RPC client is unchanged.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-17462

How was this patch tested?

  • TestOMGrpcError: every variant maps back to its exception type and message, including the suggested leader; any other exception becomes INTERNAL_ERROR; the description keeps the old format.
  • TestGrpcOzoneManagerServer: the OM gRPC server sends the error with the suggested leader.
  • TestOMFailoverProxyProvider: a not-leader error goes to the suggested leader, not just to the next OM.
  • TestS3GrpcOmTransport: over gRPC, the client goes from a not-leader OM straight to the suggested leader.

The fake OM servers in the gRPC tests now build their errors the same way as the OM gRPC server.

Copilot AI balanced review requested due to automatic review settings October 25, 2026 17:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@github-actions github-actions Bot added the om label Oct 25, 2026
@peterxcli
peterxcli marked this pull request as ready for review October 26, 2026 00:57

optional string message = 1;
// Not set for any other error.
oneof error {

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this allow us to add more error type and structural info in future

# Conflicts:
#	hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/protocolPB/GrpcOmTransport.java
#	hadoop-ozone/common/src/test/java/org/apache/hadoop/ozone/om/protocolPB/TestS3GrpcOmTransport.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants