Repository navigation
template: Update builtin templates from CentOS 5.x to Debian 13 - #14493
Open
weizhouapache wants to merge 26 commits into
Open
weizhouapache wants to merge 26 commits into
weizhouapache wants to merge 26 commits into
Conversation
- templates.sql: use the patched Debian 13 images (20261017-2634) from http://10.0.34.8/templates/debian-13-genericcloud-amd64/ for the KVM, XenServer (vhd.bz2) and VMware (ova) templates, with their SHA-512 checksums - scripts/util/build-acs-image.sh: new script to patch, shrink and convert a cloud image (qcow2) to VHD, OVA and XenServer VHD for CloudStack, and to create SHA512SUMS - schema-21to22.sql, schema-2214to30.sql: remove the inserts of the old CentOS 5 builtin templates (ids 4, 7 and 5)
…n 12 (64-bit) templates.sql cannot reference the 'Debian GNU/Linux 12 (64-bit)' guest OS because it is only added by a later upgrade script with an auto-generated id. Update the guest_os_id of the built-in Debian 13 templates (KVM, XenServer, VMware) in the 4.23.0.0 to 4.24.0.0 upgrade, looking the guest OS up by name. Also rename the VMware built-in template to debian13-x86_64-vmware to match the KVM and XenServer names, and drop the 4.3 upgrade statement that forced the XenServer built-in template back to guest OS 142.
get_suitable_test_template now returns the shared public test template (tiny-kvm on KVM), registered by the admin only when it is missing, and falls back to the built-in template. It was already doing this for XenServer. The hypervisor argument is optional and is detected from the hosts when not given. Tests that only need a small guest now use it instead of the built-in template, and test_ipv4_routing, test_snapshots and test_restore_vm no longer register their own copy of the tiny template. Tests that depend on the built-in template are not changed, for example test_deploy_vm_root_resize (needs a template of at least 1 GiB), test_templates, test_sharedfs_lifecycle and test_vm_autoscaling.
…again TestSnapshotStandaloneBackup takes a snapshot, expunges the VM and then deletes its template, to test the snapshot without its template. It needs its own template, so it cannot use the shared public test template.
The Debian 13 cloud image has no NFS client, so mounting an NFS share in the guest fails with "NFS: mount program didn't pass remote address". The shared file system smoke test mounts the share in the guest. Install nfs-common with the other guest packages.
Moving the smoke tests to the shared tiny-kvm template broke the tests that need more from the guest. Run these on the Debian 13 built-in template, taken with get_template on every hypervisor. - test_attach_multiple_volumes, test_volumes: they detach data disks from a running VM, which the tiny-kvm guest never releases. - test_backup_recovery_nas, test_backup_recovery_kboss: they create a VM from a backup and expect the files written in the guest to be in it. - test_primary_storage: it puts a primary storage with running VMs in maintenance, which needs a guest that shuts down on request. - test_volumes: use the 512 MiB big offering, as the Debian 13 guest does not boot with the 128 MiB tiny or the 256 MiB small offering. - test_vm_autoscaling: the scale up condition needs at least 1 percent CPU, and an idle Debian 13 VM uses about 0.4 percent. Start a CPU loop from the template user data.
… templates The images were rebuilt with the nfs-common package installed. The URLs are unchanged, so the KVM, XenServer and VMware templates get the checksums of the new files.
…n templates The images are built from the Debian 13 genericcloud image of 20261021-2638 with the nfs-common package installed. Update the URLs and the SHA-512 checksums of the KVM, XenServer and VMware templates.
The tests write a file in the guest and take a live backup a few seconds later. The backup is not quiesced, so a file that was not flushed yet can be missing from the backup. The guest root filesystem commits its journal about every 5 seconds, which is also the wait before the backup. Run sync after each touch, so that the file is on the disk before the backup starts.
boring-cyborg
Bot
added
component:integration-test
component:marvin
Python
Member
Author
|
@blueorangutan package |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #14493 +/- ##
============================================
+ Coverage 20.39% 20.76% +0.36%
- Complexity 21283 21907 +624
============================================
Files 6481 6496 +15
Lines 588040 591603 +3563
Branches 72021 72584 +563
============================================
+ Hits 119957 122870 +2913
- Misses 455052 455391 +339
- Partials 13031 13342 +311
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19708 |
Member
Author
|
@blueorangutan test matrix |
|
@weizhouapache a [SL] Trillian-Jenkins matrix job (ol8 mgmt + kvm-ol8, ubuntu24 mgmt + kvm-ubuntu24, ol8 mgmt + vmware-70u3, ol9 mgmt + xcpng82) has been kicked to run smoke tests |
|
[SF] Trillian Build Failed (tid-17238) |
|
[SF] Trillian Build Failed (tid-17240) |
|
[SF] Trillian Build Failed (tid-17239) |
|
[SF] Trillian Build Failed (tid-17237) |
|
[SF] Trillian test result (tid-133)
|
|
[SF] Trillian test result (tid-135)
|
|
[SF] Trillian test result (tid-132)
|
|
[SF] Trillian test result (tid-137)
|
The usage text is printed from the header comment, so the line range it prints is moved down by the length of the license.
…d.cloudstack.org The images are the same 20261021-2638 files with the same checksums, now served from download.cloudstack.org instead of the build server.
An error answer, like the timeout when the secondary storage VM disconnects during a download, has no install path. The download callback wrote it to the template_store_ref entry anyway, so the install path became null and the next download of the template went to a directory named "null". Only update the install path when the answer has one.
Member
Author
|
@blueorangutan package |
|
Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 19742 |
weizhouapache
force-pushed
the
24-builtin-debian13-templates
branch
from
October 26, 2026 10:59
2a518d7 to
6031bab
Compare
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 19746 |
The sshd drop-in only sets PermitRootLogin yes now, and it is named 40-root-login.conf. PasswordAuthentication is left to the default of the image instead of being forced to yes.
|
[SF] Trillian test result (tid-138)
|
…are template The guest kernel has no driver for the default e1000 NIC, so set the nicAdapter (Vmxnet3) and rootDiskController (pvscsi) details of the template instead of deploying it as is, which would prevent resizing the root disk.
…he tiny template on VMware VirtualMachine.create adds a cloud-init userdata to the VMs deployed from a built-in template, to allow ssh login with the root password, as the template disables it, so the tests can reach the VMs on any template. get_suitable_test_template returns the tiny template on VMware too, and test_update_security_group uses it.
|
[SF] Trillian test result (tid-139)
|
|
[SF] Trillian test result (tid-144)
|
|
[SF] Trillian test result (tid-145)
|
…-2643) Download the images from 10.0.34.8 and update the checksums.
14 tasks
init.sh runs before dbus is started on a slow boot, e.g. on XenServer, so hostnamectl fails and the hostname stays 'systemvm'. The agent then registers the system VM with that name and its agent state is not shown.
The check for a built-in template listed the template with the filter 'all', which fails for an ISO (incorrect format) and for a non-admin user. Use the filter 'executable' and consider the template as not built-in on any error.
JoaoJandre
reviewed
Oct 28, 2026
JoaoJandre
left a comment
JoaoJandre
left a comment
Contributor
There was a problem hiding this comment.
The Java and SQLs LGTM. I did not review the bash script.
…mages The XenServer/XCP-ng guest agent is needed to attach volumes to a running VM (PV drivers are not detected without it). The packages are downloaded from the GitHub release on the build host. On Debian the deb provides the xenstore tools, so xenstore-utils and libxenstore4 are no longer installed.
Contributor
|
Hi @weizhouapache, I have reviewed PR #14493 and focused on the changes related to built-in Debian 13 templates and Marvin's I checked the helper's behavior, its hypervisor-detection logic, and the updated integration tests, including the VM deployment planner and shared-network tests. I also ran three isolated tests for the helper, and all three passed. These tests do not cover a real CloudStack environment. I noticed that some test cases explicitly pass the target host or cluster's hypervisor, while others rely on automatic hypervisor detection. I haven't confirmed this as a bug yet. Would you recommend investigating this behavior further, or is there another specific area of the PR you'd like me to focus on? |
cloud-init applies set-passwords and ssh (the public keys of the instance) once per instance, so the password and the SSH key pair reset in CloudStack are not applied when the VM is started again. Set both modules to always in cloud.cfg. The ssh module deletes and regenerates the host keys whenever it runs, so set ssh_deletekeys to false to keep them across the boots.
… are detected XenServer and XCP-ng refuse to hot-attach or hot-detach a volume until the guest tools report in, which is about 45 seconds after the start of a Debian 13 guest. The tests attach a volume right after the VM is Running or restarted, so retry for up to 3 minutes when the error is about the PV drivers.
The Debian 13 built-in template does not boot with the 128 MiB of the tiny offering, so the volume cannot be attached to the VM.
Use 512 MiB and 1024 MiB service offerings, as the guest does not have enough memory with 256 MiB when the dynamic scaling is enabled. Check the open-vm-tools service too, as the guest does not have the vmware-tools service, so all the tests were skipped on VMware.
The images are rebuilt with the changes of build-acs-image.sh: set-passwords and ssh run on every boot, and xe-guest-utilities is installed.
weizhouapache
force-pushed
the
24-builtin-debian13-templates
branch
from
October 29, 2026 13:37
6031bab to
c46250d
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR replaces the built-in CentOS 5.x templates with Debian 13 (trixie) templates, and moves the smoke tests to them. More discussion on #13797
Templates
setup/db/templates.sql: the KVM (qcow2), XenServer (vhd.bz2) and VMware (ova) built-in templates are the Debian 13genericcloudamd64 images (20261021-2638) patched for CloudStack, downloaded fromdownload.cloudstack.org, with their SHA-512 checksums.scripts/util/build-acs-image.sh: new script that patches, shrinks and converts a cloud image (qcow2) to VHD, OVA and XenServer VHD for CloudStack, and createsSHA512SUMS. It installsnfs-commonin the Debian and Ubuntu images, because the guest needs an NFS client to mount a shared file system.schema-21to22.sqlandschema-2214to30.sql.Debian GNU/Linux 12 (64-bit), looked up by name because the guest OS is only added by a later upgrade script with an auto-generated id. The VMware template is renameddebian13-x86_64-vmwareto match the KVM and XenServer names. The 4.3 upgrade statement that forced the XenServer built-in template back to guest OS 142 is dropped.Fix for the template download
template_store_refentry. Before, the install path became null and the next download of the template went to a directory namednull, which failed.Smoke tests
get_suitable_test_templatereturns the shared public test template (tiny-kvmon KVM), registered by the admin only when it is missing, and falls back to the built-in template. Tests that only need a small guest use it.get_templateon every hypervisor:test_attach_multiple_volumes,test_volumes(with the 512 MiB offering),test_backup_recovery_nas,test_backup_recovery_kboss,test_primary_storage,test_vm_autoscaling.test_snapshots: the standalone backup class registers its own template again, because it deletes the template.test_backup_recovery_nas:syncthe guest after writing the marker files, so that they are in the backup.Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
DOWNLOAD_ERRORand kept its install path (template/tmpl/<account id>/<template id>). After the next sync the template was downloaded to that path, and nonulldirectory was created. Before the fix the install path was cleared.BaseImageStoreDriverImplTest(the install path is kept for an error answer without one and set when the answer has one).How did you try to break this feature and the system with this change?