Repository navigation
Add extra volumes, volume mounts and service account per component - #1701
josephtate wants to merge 1 commit into
Conversation
Add extra_volumes, extra_volume_mounts and service_account_name to the api, content and worker sections of the Pulp CR. Fixes pulp#1587 Assisted By: Hermes Agent (claude-sonnet-5-5)
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: josephtate The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @josephtate. Thanks for your PR. I'm waiting for a pulp member to verify that this patch is reasonable to test. If it is, they should reply with Tip We noticed you've done this a few times! Consider joining the org to skip this step and gain Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Fixes #1587
Summary
Adds three optional fields to the
api,contentandworkersections of the Pulp CR:extra_volumes[]corev1.Volumeextra_volume_mounts[]corev1.VolumeMountservice_account_namestringUnset fields leave the generated Deployments unchanged (covered by tests), so existing installs do not roll on upgrade.
Use case
We run Pulp on EKS with metadata/package signing. Today the operator cannot express two things we need on the worker pods, so we scale the operator-managed worker Deployment to 0 and maintain a hand-written copy:
defaultMode: 0755), and secrets from a CSI secret store;This PR lets us drop the hand-written Deployment. It is also the Azure Key Vault CSI case from the issue.
Syntax: typed fields instead of the issue's YAML strings
The issue proposes AWX-style strings (
extra_volumes: |). This PR uses typedcorev1fields instead, likeenv_vars,tolerationsandaffinityalready do in this CRD:kubectl explaindocuments them, so mistakes fail atkubectl applyinstead of at reconcile time;csi,secret,projected, ...) works without extra code and tracks the k8s dependency.The cost is a different syntax from the issue's literal example. The same use cases work.
Discarded approaches
Notes for reviewers
webare unchanged.mount_trusted_caon vanilla Kubernetes with trust-manager) could be considered redundant now:env_varsplus these mounts can also deliver a CA bundle. I think it is still warranted for the trust-manager and OpenShift integration it provides. That integration is described indocs/trust-manager-integration.md, which is not linked from the docs site nav.Review of this patch
The first version of this patch was reviewed by an independent, read-only Claude Code session (claude-sonnet-5-5), prompted to be adversarial and to assume shortcuts had been taken. I checked each finding against the code before acting on it. The patch is a single commit that I amended, so this section describes what changed after review.
What I changed, and why
image_pull_secretswere dropped when a custom ServiceAccount was set.spec.image_pull_secretsto the ServiceAccount it manages (controllers/repo_manager/rbac.go), and pods get their pull secrets through it. The first version only changedServiceAccountNamein the pod spec. A user-supplied ServiceAccount replaces the managed one instead of adding to it, so pods lost their pull secrets and a private registry would have failed withImagePullBackOff.service_account_nameis set, the operator copiesspec.image_pull_secretsinto the pod spec. Pods using the managed ServiceAccount are unchanged. A generator test covers both cases.The docs and sample used a path the operator already uses.
/var/lib/pulp/scripts. Whensigning_secretis set, the operator mounts each signing script under that path (settings.SigningScriptPath), and the path sits inside thefile-storagemount at/var/lib/pulp./opt/pulp-scripts.The docs did not say which pods use the fields. The docs page now states that jobs and the
webpods do not use them.The first tests could not catch a broken implementation.
setVolumeMountson an empty struct. Deleting the line insetVolumesthat appends the extra volumes, or reverting the ServiceAccount line inDeploy, would not have failed them.Deploypath for api, content and worker (controllers/extra_volumes_test.go). I reverted the ServiceAccount, volume, mount and pull-secret lines one at a time, and each revert fails 3 to 6 tests.controllers/repo_manager/extra_volumes_test.go) that check CRD validation against the generated CRD. There were none before.What I did not change, and why
Pattern/MaxLengthtoservice_account_namesa_annotationsalready covers the common IRSA case, so a per-component ServiceAccount is redundantsa_annotationsannotates the single managed ServiceAccount for all components. Per-componentservice_account_nameis for when components need different identities (for example only the worker reads AWS Secrets Manager).Volume, because the generated pod spec shares nested pointers (such asdefaultMode) with the CRenv_vars,tolerationsandaffinityhandling shares them the same way.corev1.Volumefor three components causes this. Avoiding it means an untyped field such asx-kubernetes-preserve-unknown-fields, which loses the validation that is the reason for choosing typed fields.mount_trusted_ca, LDAP, telemetry orsigning_secretsigning_secretpath needs a Secret in the cluster, which the unit tests do not set up. This is untested.Tests
controllers/extra_volumes_test.go) run the realDeploypath for api, content and worker.controllers/repo_manager/extra_volumes_test.go) check CRD validation: valid input is accepted and preserved, and wrong types and missing required keys are rejected.KUBEBUILDER_ASSETS=... go test ./...passes;make bundleandoperator-sdk bundle validatepass;.ci/scripts/validate_commit_message.pypasses.Live cluster test
Ran the operator built from this branch against a throwaway kind cluster (Kubernetes v1.33.1,
quay.io/pulp/pulp-minimal:stable) with a Pulp CR that setservice_account_name,extra_volumesandextra_volume_mountsonworker(a Secret withdefaultMode: 0755, mounted read-only) and anemptyDironcontent, plusimage_pull_secrets.system:serviceaccount:<ns>:pulp-worker. The api pod's token identified the operator-managed ServiceAccount.contentmount was writable. The api and content pods did not have the worker's mount.service_account_namerolled the worker to a new pod with the new identity. Removing all three fields rolled it back to the operator-managed ServiceAccount, with no extra volume, no mount and no pod-level pull secrets.service_account_namethat does not exist is not caught by the operator. The ReplicaSet reportsFailedCreate: error looking up service accountand the existing pods keep running.Not covered by the live test: a CSI volume (no CSI driver in the cluster; the CSI schema is covered by the envtest specs) and a real IAM-bound ServiceAccount.
AI disclosure
Per the AI policy: the code, tests, docs and this description were written with Hermes Agent running claude-sonnet-5-5 (Anthropic). The adversarial review above ran in Claude Code (claude-sonnet-5-5). The commit carries an
Assisted By:label. I reviewed the code and this description and can explain the change.