Repository navigation
WIP [Rust SDK] Add the Cargo workspace, Gradle integration and model crate - #40508
prodriguezdefino wants to merge 1 commit into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #40508 +/- ##
=============================================
+ Coverage 56.16% 79.91% +23.74%
=============================================
Files 1124 488 -636
Lines 178336 82562 -95774
Branches 1489 0 -1489
=============================================
- Hits 100170 65981 -34189
+ Misses 75645 16581 -59064
+ Partials 2521 0 -2521
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Hey @prodriguezdefino, I would like to help this PR as much as possible since I didn't know Beam is getting a Rust SDK. I am new to beam and I am excited to see Rust based work here. I tested this branch locally on Windows with One small note: building on Windows requires Happy to help test upcoming PRs on Windows or contribute to the downstream crates as the SDK progresses. |
50d94ea to
fbfacb2
Compare
|
Thanks for testing this out on Windows @CH-Abhinav, great to hear In Beam, host dependencies are normally set up via Once the RFC document (it's on the dev@ distro if you're curious about it) gets reviewed and hopefully, eventually approved, I planned to start working on having the initial PRs land. This draft is more about making sure checks are in place for once/if things get in motion. It would be great to have your help making sure things work smoothly on Windows! |
|
/gemini review |
There was a problem hiding this comment.
📋 Review Summary
This PR cleanly scaffolds the new Apache Beam Rust SDK workspace (sdks/rust), Gradle integration (:sdks:rust), release version tooling, and the apache-beam-model crate for compiling portability Protobuf and gRPC definitions. The structure is well-organized and follows Beam's repository conventions.
🔍 General Feedback
- Environment Setup (
local-env-setup.sh): Whileprotocwas added to the macOS setup path, it is currently missing from the Linux setup path (dev-support/docker/pkglistdoes not includeprotobuf-compiler). - Release Script (
release/src/main/scripts/set_version.sh): Thegit add 'sdks/rust/*Cargo.toml'pathspec only matches single-depth subdirectories in Git, which misses crate manifests nested deeper such assdks/rust/beam/model/Cargo.toml.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
First change in the series that adds the Apache Beam Rust SDK. This change adds: - The Cargo workspace at sdks/rust (rustfmt and clippy configuration, workspace lints, dev profiles) with a single member. - apache-beam-model: protobuf and gRPC bindings generated at build time from the model protos in this repository (model/pipeline, model/fn-execution, model/job-management). - Gradle integration (:sdks:rust), so the SDK builds like the rest of the repository: build, check (rustfmt + clippy -D warnings), test, fmt and clean. build.gradle applies one script per build area from sdks/rust/gradle/; the tasks call cargo, which handles incremental builds. Gradle passes the Beam version from gradle.properties to cargo; no Beam version is hard-coded in the Rust sources. - Release tooling: release/src/main/scripts/set_version.sh bumps the Cargo manifests through sdks/rust/scripts/set_version.sh. - local-env-setup.sh installs Rust and protoc (protobuf-compiler in dev-support/docker/pkglist on Linux and in the Docker build env). - README.md and docs/development.md.
fbfacb2 to
0443786
Compare
|
/gemini review |
There was a problem hiding this comment.
📋 Review Summary
This PR cleanly sets up the initial Cargo workspace, Gradle build integration (:sdks:rust), version-bumping scripts, environment setup, and the apache-beam-model crate with generated Protobuf/gRPC bindings. All feedback from the previous review (adding protobuf-compiler to dev-support/docker/pkglist for Linux setups and resolving the broken link in README.md) has been addressed.
🔍 General Feedback
- The Gradle-to-Cargo task delegation (
build,check,test,fmt,clean) and version propagation fromgradle.propertiesintegrate smoothly with existing Beam build and release conventions. - Protobuf compilation in
beam/model/build.rsproperly registerscargo:rerun-if-changeddirectives for all model proto inputs.
💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.
First change in the series that adds the Apache Beam Rust SDK.
This change adds: