Skip to content

HDDS-17169. Add per-command OM client latency metrics - #11738

Open
rich7420 wants to merge 3 commits into
apache:masterfrom
rich7420:HDDS-17169
Open

rich7420 wants to merge 3 commits into
apache:masterfrom
rich7420:HDDS-17169

Conversation

@rich7420

@rich7420 rich7420 commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Adds per-command OM client latency metrics to help diagnose slow S3 Gateway requests. Each measured OmTransport.submitRequest contributes one latency sample in nanoseconds, including failures, network time, and internal retries.

Reuses PerformanceMetrics with configurable percentiles and a process-wide source shared by RpcClient instances. The last client releases the source. Also closes leaked test clients before cluster shutdown. This PR covers the OM portion of HDDS-17169.

What is the link to the Apache JIRA

HDDS-17169

How was this patch tested?

Revalidated on JDK 21 after merging master at 885f83dd16 into 27fe1e62d0:

  • TestOmClientMetrics (12), TestOzoneManagerProtocolClientSideTranslatorPB (21), and TestRpcClient (182): all passed, with no skipped tests.
  • The 42-module reactor package build passed with TestS3ClientMetrics (3) and the relocated TestOzoneFileSystemPrefixParser (1). S3 coverage includes PUT/GET/HEAD/LIST, multipart upload, error responses, and /prom//jmx output.
  • checkstyle.sh passed. rat.sh passed on a clean export of the committed source tree.
  • Fork CI for 27fe1e62d0 passed after one rerun of the ozone job. The first attempt failed during scanner teardown with RDBMetrics still registered; this did not recur on rerun.

Copilot AI balanced review requested due to automatic review settings October 24, 2026 09:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@github-actions github-actions Bot added client s3 S3 Gateway labels Oct 24, 2026
Release cached filesystems and explicitly owned clients before cluster shutdown. Use tracked clients for tenant tests and the configured EC filesystem for the SequenceFile reader.
@rich7420
rich7420 marked this pull request as ready for review October 24, 2026 14:12

@smengcl smengcl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Thanks @rich7420 for the patch.

@adoroszlai
adoroszlai marked this pull request as draft October 27, 2026 07:14
@rich7420
rich7420 marked this pull request as ready for review October 27, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

client s3 S3 Gateway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants