Skip to content

Serialize XML request payloads without pretty-printing - #3965

Open
Madrigal wants to merge 5 commits into
mainfrom
fix-compact-xml-serialize
Open

Madrigal wants to merge 5 commits into
mainfrom
fix-compact-xml-serialize

Conversation

@Madrigal

@Madrigal Madrigal commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Issue #, if available: #3891

Description of changes:

XML request payloads (S3, S3 CRT, S3 Control, CloudFront, Route 53) were serialized with XmlDocument::ConvertToString(), which uses tinyxml2's default pretty-printing. That adds indentation and newlines to every element. For a CompleteMultipartUpload with 9,331 parts the body is 1,090,743 bytes (1.04 MiB). Some S3-compatible services with a 1 MiB request/XML limit, such as Ceph RGW, reject it even though the upload is under the 10,000-part maximum.

Changes:

  • Add XmlDocument::ConvertToCompactString(), which prints the same document without whitespace between elements. Whitespace inside text content is unchanged.
  • ConvertToString() is unchanged and still pretty-prints, since it is public API and may be used outside the SDK, e.g. to save a document to a file. Both methods' doc comments now point to each other.
  • Switch the codegen templates (XmlRequestSource.vm, S3 PutBucketNotificationConfigurationRequest.vm) to ConvertToCompactString(), and regenerate the affected clients. That's 190 files, each a one-line change: payloadDoc.ConvertToString() → payloadDoc.ConvertToCompactString().
  • Add unit tests for compact output: exact bytes, nested elements, escaping, newlines and spaces in text surviving a round trip through the parser, and empty output for a moved-from document.

Result: the 9,331-part CompleteMultipartUpload body shrinks to 829,472 bytes (0.79 MiB), and 10,000 parts comes to 0.85 MiB.

This covers S3Client and S3CrtClient::CompleteMultipartUpload, and Transfer Manager v1. Automatic multipart uploads through S3CrtClient::PutObject and the S3 Transfer Manager v2 build this body inside aws-c-s3, which hard-codes the same whitespace. That needs a separate fix in awslabs/aws-c-s3.

This also replaces the approach in #3892 (a ClientConfiguration flag), which was closed in review in favor of not threading serializer options through requests.

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change. The API change is additive only. Request bodies change byte-for-byte but stay
    semantically identical XML.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Madrigal
Madrigal marked this pull request as ready for review October 22, 2026 17:58
@Madrigal
Madrigal requested a review from wty-Bryant October 23, 2026 16:50

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant