Move partitioning into a separate engine

README.md

@@ -72,6 +72,21 @@ the `db/structure.sql`, this mode creates files in the `db/schema_migrations` di
 PostgreSQL supports declarative table partitioning. The partition manager automates the
 management of partitions without manual operations or extensions on the PostgreSQL server.
 
+Table partitioning is an optional feature which has to be explicitly required.
+There are multiple options to do this:
+
+1. Add the `require` above the application class definition in `config/application.rb`:
+
+   ```ruby
+   require 'code0/zero_track/features/database_partitioning'
+   ```
+
+2. Use the `require` option in the `Gemfile`
+
+   ```ruby
+   gem 'code0-zero_track', require: %w[code0/zero_track/features/database_partitioning]
+   ```
+
 #### Configuration
 
 ```ruby
@@ -82,31 +97,37 @@ config.zero_track.db_partitioning.base_ar_class = 'ActiveRecord::Base' # default
 - `dynamic_partition_schema`: The PostgreSQL schema where dynamic partitions are stored.
 - `base_ar_class`: The ActiveRecord base class used for the internal partitioning models.
 
+#### Schema and Views Setup
+
+The partition manager depends on a few PostgreSQL views.
+
+The migrations to create these views can be installed as following:
+
+```bash
+$ bin/rails code0_zero_track_database_partitioning:install:migrations
+$ bin/rails db:migrate
+```
+
 #### Migration Helpers
 
-Include the migration helpers by inheriting from `Code0::ZeroTrack::Database::Migration[1.0]` (or the
-appropriate version). The following methods become available:
+Include the migration helpers by inheriting from `Code0::ZeroTrack::Database::Migration[1.1]` (or the
+appropriate version). The following method becomes available:
 
 `create_partition_by_date_table(table_name, partition_column:, primary_key: nil, **options, &block)` creates a table
 partitioned by range on the given column. By default, it adds a `bigserial` id column and sets up a composite
 primary key of `(id, partition_column)`. If `primary_key` is provided (e.g. `primary_key: %i[date other_column]`),
 the id column is omitted and the given columns are used as the primary key instead.
 
-`create_dynamic_partition_schema` / `drop_dynamic_partition_schema` creates or drops the schema
-used for storing dynamic partitions.
-
-`create_partitioning_views` / `drop_partitioning_views` creates or drops the PostgreSQL views
-(`postgres_partitioned_tables`, `postgres_partitions`, `postgres_detached_partitions`) that the
-partition manager uses to inspect existing partitions.
+> The schema/view helpers `create_dynamic_partition_schema`, `drop_dynamic_partition_schema`,
+> `create_partitioning_views` and `drop_partitioning_views` have been removed in migration version 1.1.
+> The schema and views no longer have to be created in your own migration with these helpers and instead
+> are created by migrations from the gem.
 
 Example migration:
 
 ```ruby
-class CreatePartitionedEvents < Code0::ZeroTrack::Database::Migration[1.0]
+class CreatePartitionedEvents < Code0::ZeroTrack::Database::Migration[1.1]
   def change
-    create_dynamic_partition_schema
-    create_partitioning_views
-
     create_partition_by_date_table :events, partition_column: :created_at do |t|
       t.text :name, null: false
       t.timestamps_with_timezone null: false
@@ -115,8 +136,6 @@ class CreatePartitionedEvents < Code0::ZeroTrack::Database::Migration[1.0]
 end
 ```
 
-The schema and views only need to be created once before creating the first partitioned table.
-
 Tables don't necessarily have to be created with the provided helper. The partition manager will
 work as long as the model is correctly configured.
 

db/migrate/database_partitioning/20261024153522_create_partitioning_schema.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+class CreatePartitioningSchema < Code0::ZeroTrack::Database::Migration[1.1]
+  def up
+    schema = quote_table_name(Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema)
+    execute "CREATE SCHEMA #{schema}"
+  end
+
+  def down
+    schema = quote_table_name(Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema)
+    execute "DROP SCHEMA #{schema}"
+  end
+end

db/migrate/database_partitioning/20261024153725_create_partitioning_views.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+class CreatePartitioningViews < Code0::ZeroTrack::Database::Migration[1.1]
+  def up
+    dynamic_schema = Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema
+
+    execute <<~SQL.squish
+      CREATE OR REPLACE VIEW postgres_partitioned_tables AS
+      SELECT c.oid::regclass::text AS identifier,
+             c.oid,
+             n.nspname AS schema,
+             c.relname AS name,
+             CASE p.partstrat
+               WHEN 'l' THEN 'list'
+               WHEN 'r' THEN 'range'
+               WHEN 'h' THEN 'hash'
+             END AS strategy,
+             pg_get_partkeydef(c.oid) AS partition_key
+      FROM pg_partitioned_table p
+      JOIN pg_class c ON c.oid = p.partrelid
+      JOIN pg_namespace n ON n.oid = c.relnamespace
+      WHERE n.nspname = current_schema();
+    SQL
+
+    execute <<~SQL.squish
+      CREATE OR REPLACE VIEW postgres_partitions AS
+      SELECT c.oid::regclass::text AS identifier,
+             c.oid,
+             n.nspname AS schema,
+             c.relname AS name,
+             i.inhparent::regclass::text AS parent_identifier,
+             pg_get_expr(c.relpartbound, c.oid) AS condition,
+             obj_description(c.oid) AS comment,
+             i.inhrelid IS NOT NULL AS attached
+      FROM pg_class c
+      LEFT JOIN pg_inherits i ON c.oid = i.inhrelid
+      JOIN pg_namespace n ON n.oid = c.relnamespace
+      WHERE c.relispartition
+        AND c.relkind = 'r'
+        AND n.nspname IN (current_schema(), #{quote(dynamic_schema)});
+    SQL
+
+    execute <<~SQL.squish
+      CREATE OR REPLACE VIEW postgres_detached_partitions AS
+      SELECT c.oid::regclass::text AS identifier,
+             c.oid,
+             n.nspname AS schema,
+             c.relname AS name,
+             obj_description(c.oid)::jsonb ->> 'table' AS parent_identifier,
+             (obj_description(c.oid)::jsonb ->> 'detached_at')::timestamptz AS detached_at
+      FROM pg_class c
+      JOIN pg_namespace n ON n.oid = c.relnamespace
+      WHERE c.relkind = 'r'
+        AND n.nspname = #{quote(dynamic_schema)}
+        AND NOT EXISTS (
+          SELECT 1 FROM pg_inherits WHERE inhrelid = c.oid
+        )
+        AND obj_description(c.oid)::jsonb ? 'table'
+        AND obj_description(c.oid)::jsonb ? 'detached_at';
+    SQL
+  end
+
+  def down
+    execute 'DROP VIEW IF EXISTS postgres_detached_partitions'
+    execute 'DROP VIEW IF EXISTS postgres_partitions'
+    execute 'DROP VIEW IF EXISTS postgres_partitioned_tables'
+  end
+end

lib/code0/zero_track.rb

@@ -5,9 +5,16 @@
 
 require 'zeitwerk'
 loader = Zeitwerk::Loader.new
-loader.tag = File.basename(__FILE__, '.rb')
+loader.tag = 'code0-zero_track'
 loader.inflector = Zeitwerk::GemInflector.new(__FILE__)
 loader.push_dir(File.expand_path(File.join(__dir__, '..')))
+
+# ignore feature entrypoints
+loader.ignore("#{__dir__}/zero_track/features")
+
+# database partitioning is a separate feature which needs to be explicitly required
+loader.ignore("#{__dir__}/zero_track/database/partitioning")
+
 loader.setup
 
 module Code0

lib/code0/zero_track/database/migration.rb

@@ -12,6 +12,16 @@ class V1_0 < ::ActiveRecord::Migration[7.1]
           include Database::MigrationHelpers::RemoveColumnEnhancements
           include Database::MigrationHelpers::TableEnhancements
           include Database::MigrationHelpers::TablePartitioning
+          include Database::MigrationHelpers::TablePartitioning::V1_0
+        end
+
+        class V1_1 < ::ActiveRecord::Migration[7.1]
+          include Database::MigrationHelpers::AddColumnEnhancements
+          include Database::MigrationHelpers::ConstraintHelpers
+          include Database::MigrationHelpers::IndexHelpers
+          include Database::MigrationHelpers::RemoveColumnEnhancements
+          include Database::MigrationHelpers::TableEnhancements
+          include Database::MigrationHelpers::TablePartitioning
         end
         # rubocop:enable Naming/ClassAndModuleCamelCase
 

lib/code0/zero_track/database/migration_helpers/table_partitioning.rb

@@ -27,20 +27,21 @@ def create_partition_by_date_table(table_name, partition_column:, primary_key: n
             end
           end
 
-          def create_dynamic_partition_schema
-            schema = quote_table_name(Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema)
-            execute "CREATE SCHEMA #{schema}"
-          end
+          module V1_0 # rubocop:disable Naming/ClassAndModuleCamelCase
+            def create_dynamic_partition_schema
+              schema = quote_table_name(Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema)
+              execute "CREATE SCHEMA #{schema}"
+            end
 
-          def drop_dynamic_partition_schema
-            schema = quote_table_name(Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema)
-            execute "DROP SCHEMA #{schema}"
-          end
+            def drop_dynamic_partition_schema
+              schema = quote_table_name(Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema)
+              execute "DROP SCHEMA #{schema}"
+            end
 
-          def create_partitioning_views
-            dynamic_schema = Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema
+            def create_partitioning_views
+              dynamic_schema = Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema
 
-            execute <<-SQL.squish
+              execute <<-SQL.squish
               CREATE OR REPLACE VIEW postgres_partitioned_tables AS
               SELECT c.oid::regclass::text AS identifier,
                      c.oid,
@@ -56,9 +57,9 @@ def create_partitioning_views
               JOIN pg_class c ON c.oid = p.partrelid
               JOIN pg_namespace n ON n.oid = c.relnamespace
               WHERE n.nspname = current_schema();
-            SQL
+              SQL
 
-            execute <<-SQL.squish
+              execute <<-SQL.squish
               CREATE OR REPLACE VIEW postgres_partitions AS
               SELECT c.oid::regclass::text AS identifier,
                      c.oid,
@@ -74,9 +75,9 @@ def create_partitioning_views
               WHERE c.relispartition
                 AND c.relkind = 'r'
                 AND n.nspname IN (current_schema(), #{quote(dynamic_schema)});
-            SQL
+              SQL
 
-            execute <<-SQL.squish
+              execute <<-SQL.squish
               CREATE OR REPLACE VIEW postgres_detached_partitions AS
               SELECT c.oid::regclass::text AS identifier,
                      c.oid,
@@ -93,13 +94,14 @@ def create_partitioning_views
                 )
                 AND obj_description(c.oid)::jsonb ? 'table'
                 AND obj_description(c.oid)::jsonb ? 'detached_at';
-            SQL
-          end
+              SQL
+            end
 
-          def drop_partitioning_views
-            execute 'DROP VIEW IF EXISTS postgres_detached_partitions'
-            execute 'DROP VIEW IF EXISTS postgres_partitions'
-            execute 'DROP VIEW IF EXISTS postgres_partitioned_tables'
+            def drop_partitioning_views
+              execute 'DROP VIEW IF EXISTS postgres_detached_partitions'
+              execute 'DROP VIEW IF EXISTS postgres_partitions'
+              execute 'DROP VIEW IF EXISTS postgres_partitioned_tables'
+            end
           end
         end
       end

lib/code0/zero_track/database/partitioning/engine.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+module Code0
+  module ZeroTrack
+    module Database
+      module Partitioning
+        class Engine < Rails::Engine
+          isolate_namespace Partitioning
+
+          config.zero_track.db_partitioning = ActiveSupport::OrderedOptions.new
+          config.zero_track.db_partitioning.dynamic_partition_schema = 'partitions_dynamic'
+          config.zero_track.db_partitioning.base_ar_class = 'ActiveRecord::Base'
+
+          initializer 'code0-zero_track.database_partitioning.paths' do
+            paths['db/migrate'] = 'db/migrate/database_partitioning'
+          end
+
+          initializer 'code0-zero_track.database_partitioning.register_schema_cleaner' do
+            config.zero_track.active_record.schema_cleaner_callbacks << SchemaCleaner.new
+          end
+        end
+      end
+    end
+  end
+end

lib/code0/zero_track/database/partitioning/schema_cleaner.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+module Code0
+  module ZeroTrack
+    module Database
+      module Partitioning
+        class SchemaCleaner
+          def clean!(structure)
+            schema = Regexp.escape(dynamic_partition_schema)
+
+            # Remove CREATE TABLE <schema>.<partition> (...);
+            structure.gsub!(/^CREATE TABLE #{schema}\.\S+\s*\(.*?\);\n/m, '')
+
+            # Remove ALTER TABLE ... ATTACH PARTITION <schema>.<partition> ...;
+            structure.gsub!(/^ALTER TABLE .+ ATTACH PARTITION #{schema}\.\S+.*?;\n/, '')
+
+            # Remove ALTER TABLE ONLY <schema>.<partition> ...;
+            structure.gsub!(/^ALTER TABLE ONLY #{schema}\.\S+\n.*?;\n/m, '')
+
+            # Remove CREATE [UNIQUE] INDEX ... ON <schema>.<partition> ...;
+            structure.gsub!(/^CREATE (?:UNIQUE )?INDEX \S+ ON #{schema}\.\S+.*?;\n/m, '')
+
+            # Remove ALTER INDEX ... ATTACH PARTITION <schema>.<partition>;
+            structure.gsub!(/^ALTER INDEX \S+ ATTACH PARTITION #{schema}\.\S+;\n/, '')
+
+            # Remove COMMENT ON TABLE <schema>.<partition> IS '<comment>';
+            structure.gsub!(/^COMMENT ON TABLE #{schema}\.\S+ IS .*;\n/, '')
+          end
+
+          private
+
+          def dynamic_partition_schema
+            Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema
+          end
+        end
+      end
+    end
+  end
+end

lib/code0/zero_track/database/schema_cleaner.rb

@@ -38,9 +38,9 @@ def clean(io)
             'CREATE EXTENSION IF NOT EXISTS \1;'
           )
 
-          # Remove dynamic partition objects that are managed automatically at runtime.
-          # These would cause schema drift on every partition rotation if left in the dump.
-          remove_dynamic_partitions!(structure)
+          Rails.application.config.zero_track.active_record.schema_cleaner_callbacks.each do |cleaner|
+            cleaner.clean!(structure)
+          end
 
           structure.gsub!(/\n{3,}/, "\n\n")
 
@@ -49,34 +49,6 @@ def clean(io)
 
           nil
         end
-
-        private
-
-        def dynamic_partition_schema
-          Rails.application.config.zero_track.db_partitioning.dynamic_partition_schema
-        end
-
-        def remove_dynamic_partitions!(structure)
-          schema = Regexp.escape(dynamic_partition_schema)
-
-          # Remove CREATE TABLE <schema>.<partition> (...);
-          structure.gsub!(/^CREATE TABLE #{schema}\.\S+\s*\(.*?\);\n/m, '')
-
-          # Remove ALTER TABLE ... ATTACH PARTITION <schema>.<partition> ...;
-          structure.gsub!(/^ALTER TABLE .+ ATTACH PARTITION #{schema}\.\S+.*?;\n/, '')
-
-          # Remove ALTER TABLE ONLY <schema>.<partition> ...;
-          structure.gsub!(/^ALTER TABLE ONLY #{schema}\.\S+\n.*?;\n/m, '')
-
-          # Remove CREATE [UNIQUE] INDEX ... ON <schema>.<partition> ...;
-          structure.gsub!(/^CREATE (?:UNIQUE )?INDEX \S+ ON #{schema}\.\S+.*?;\n/m, '')
-
-          # Remove ALTER INDEX ... ATTACH PARTITION <schema>.<partition>;
-          structure.gsub!(/^ALTER INDEX \S+ ATTACH PARTITION #{schema}\.\S+;\n/, '')
-
-          # Remove COMMENT ON TABLE <schema>.<partition> IS '<comment>';
-          structure.gsub!(/^COMMENT ON TABLE #{schema}\.\S+ IS .*;\n/, '')
-        end
       end
     end
   end