Repository navigation
Conversation
The previous commit rewired the provider to Doobie and removed the Mapper entity from Boot.scala's ToSchemify list, but missed deleting the entity class itself - it lived in its own file, separate from the provider file that got deleted. No remaining references; full suite still green.
Replace the Lift Mapper tax-residence entity with a Doobie-backed provider (forty-fifth table off Lift Mapper). mCustomerId is a MappedLongForeignKey pointing at mappedcustomer.id (the customer's internal BIGINT primary key, not their UUID customerId) - the Doobie provider resolves customerId back to the UUID via a MappedCustomer lookup by id, falling back to the raw long id as a string if the customer row is missing, matching the Mapper entity's own getter exactly. Two indexes carried over: a plain index on mcustomerid (from the foreign-key field) and a UNIQUE INDEX on (mcustomerid, mdomain, mtaxnumber), confirmed against a booted instance's information_schema.indexes - caught a first attempt at the migration script that used ALTER TABLE ... ADD CONSTRAINT ... UNIQUE, which H2 backs with an auto-suffixed index name rather than the literal constraint name; switched to CREATE UNIQUE INDEX to match Schemifier's actual output, per MigratedTablesExistTest. deletion.DeleteCustomerCascade's cascade delete moves to raw SQL against the new table.
Replace the Lift Mapper customer-link entity with a Doobie-backed provider (forty-sixth table off Lift Mapper). Unique index on customerlinkid; plain indexes on customerid and othercustomerid, confirmed against a booted instance's information_schema.indexes. No test exercised this table's provider or the connector methods wired to it (bank-to-bank customer relationships, e.g. spouse/parent at another bank), so a characterization test (CustomerLinkProviderTest) covering full CRUD plus bulkDelete was added and confirmed green against the pristine Mapper entity before the migration, then again against the Doobie provider.
Replace the Lift Mapper counterparty-limit entity with a Doobie-backed provider (forty-seventh table off Lift Mapper). Two unique indexes: one on counterpartylimitid, one on the composite (bankid, accountid, viewid, counterpartyid) - at most one limit per tuple - confirmed against a booted instance's information_schema.indexes. Amount fields are NUMERIC(16,10); count fields default to -1 and amount fields to 0 at the application layer on create, matching the Mapper fields' own defaultValue overrides. toJValue moves onto the new CounterpartyLimitRow case class verbatim, since CounterpartyLimitTrait extends JsonAble. MigrationOfCounterpartyLimitFieldType, a historical migration, switches to the tableExistsByName overload used by the other historical migrations in this series.
Replace the Lift Mapper customer-account-link entity with a Doobie-backed provider (forty-eighth table off Lift Mapper). Two unique indexes: one on customeraccountlinkid, one on the composite (accountid, customerid) - a customer has at most one link per account - confirmed against a booted instance's information_schema.indexes. createAgentAccountLink builds its own AgentAccountLinkTraitCommons from the CustomerAccountLinkTrait result fields (documented in the entity as "customer and agent share the same model"), so the row type only needs to implement CustomerAccountLinkTrait. The endpoint test covers create/read/update/delete, but not getOrCreateCustomerAccountLink, the unfiltered getCustomerAccountLinks, or bulkDeleteCustomerAccountLinks, so a characterization test (CustomerAccountLinkProviderTest) covering those three was added and confirmed green against the pristine Mapper entity before the migration, then again against the Doobie provider. Two cascade-delete call sites move to the new provider: deletion.DeleteBankCascade's account-scoped lookup (filters by accountId only, no bankId) and deletion.DeleteCustomerCascade's cascade delete. A third direct reference in LocalMappedConnector.getBankAccountsForUser also moves over.
Replace the Lift Mapper user-customer-link entity with a Doobie-backed provider (forty-ninth table off Lift Mapper). Two unique indexes: one on musercustomerlinkid, one on the composite (muserid, mcustomerid) - a user has at most one link per customer - confirmed against a booted instance's information_schema.indexes. mdateinserted is a separate column from the createdat/updatedat pair the CreatedUpdated mixin also adds; the trait's dateInserted getter reads the former. getOCreateUserCustomerLink preserves the Mapper version's find-then-insert-with-retry-on-conflict shape exactly, including the scala.util.Try wrapping around the insert: ConcurrentDuplicateCreationTest scenario L races 8 concurrent calls for the same (userId, customerId) and asserts no exception and exactly one row, relying on the unique index to reject the losing insert so it can be caught and retried as a re-fetch. Confirmed green against the pristine Mapper entity first, then again after the migration. Two direct callers by name (not through the DI seam) move to the new object: deletion.DeleteCustomerCascade's cascade delete and MappedCustomerProvider.getCustomersByUserId.
Replace the Lift Mapper CRM-event entity with a Doobie-backed provider (fiftieth table off Lift Mapper). Three indexes: UNIQUE on mcrmeventid, plain on mbankid, plus a plain index on muserid that the entity's own dbIndexes never declared - Lift auto-indexes MappedLongForeignKey columns, confirmed against a booted instance's information_schema.indexes. mUserId stores ResourceUser's internal BIGINT primary key; the row's user accessor resolves it back to a live ResourceUser the same way the Mapper entity's getter did, including throwing if the id doesn't resolve. The sandbox importer (LocalMappedConnectorDataImport) built a not-yet-saved MappedCrmEvent via Mapper's create/.validate/Saveable pattern; it now builds a CrmEventCreateParams (a transient CrmEventTrait implementation whose user accessor throws, matching what the Mapper version would have thrown for the same reason - the importer never sets mUserId) and saves it through DoobieCrmEventProvider.createEvent via a new SaveableCrmEvent, mirroring the SaveableAtm pattern already used for the Atm table. MappedCrmEventProviderTest, which called the Mapper entity directly, was rewritten to go through DoobieCrmEventProvider's create/get methods; confirmed green against the pristine Mapper entity before the migration under its original form. SandboxDataLoadingTest's "should create CRM Events ok" scenario exercises the sandbox-import path end-to-end.
Table 51/140 in the Lift Mapper to Doobie strangler migration. Replaces MappedUserRefreshes with DoobieUserRefreshesProvider, backed by a Flyway migration matching the probed schema (muserid varchar(44), createdat/updatedat timestamps, unique index on muserid). This table drives the refresh_user.interval login flow that periodically forces a fresh credential check. AuthUserTest exercises this table's row-count assertions extensively across its login scenarios; rewrote its direct MappedUserRefreshes calls (findAll().length, bulkDelete_!!()) to the new provider's count()/bulkDelete() helpers, added specifically to support those assertions. All 5 AuthUserTest scenarios pass unchanged against the new backing store, along with the full suite (3638 tests, 0 failures).
Table 52/140 in the Lift Mapper to Doobie strangler migration. MappedThing was never wired into Boot.scala's ToSchemify list, so its table was never created by Schemifier, and Thing.thingProvider had no caller anywhere in the codebase - a leftover example/tutorial entity with zero live callers and zero test coverage. There is no data to migrate and no table to create a Flyway script for, so the correct move is deleting the scaffold rather than building a Doobie provider nothing would ever invoke. Removes the code.examplething package entirely and its exemption entry in MappedClassNameTest. Full suite passes unchanged (3638 tests, 0 failures).
Table 53/140 in the Lift Mapper to Doobie strangler migration. Replaces MappedPayeeLookupProvider with DoobiePayeeLookupProvider, backed by a Flyway migration matching the probed schema (unique index on lookupid, plain index on expiresat). This is the short-lived lookup cache behind the mobile-wallet payee-lookup flow in Http4s700 - createPayeeLookup writes a row with a ttl, later reads filter out expired rows in application code without deleting them, preserved exactly as before. Covered by the existing Http4s700RoutesTest suite; no new characterization test needed. Full suite passes unchanged (3638 tests, 0 failures).
Table 54/140 in the Lift Mapper to Doobie strangler migration. MetricsArchiveRun's concrete Mapper type leaked into two cross-file signatures - RunCompleted(run: MetricsArchiveRun) in MetricsArchiveScheduler and metricsArchiveRunToJson(r: MetricsArchiveRun) in JSONFactory7.0.0 - so this introduces MetricsArchiveRunTrait with plain getters and widens both call sites to it, backed by MetricsArchiveRunRow. The plain-object API (recordRun, lastRun, lastSuccessfulRun, pruneToMostRecent, maxRowsToKeep, count, bulkDelete_!!) keeps its original names so only field-level access (PascalCase MappedField.get -> lowercase trait getter) needed to change at call sites, in both production code and MetricsArchiveSchedulerTest. Flyway migration matches the probed schema: unique index on runid, plain index on startedat. Full suite passes unchanged (3638 tests, 0 failures).
Table 55/140 in the Lift Mapper to Doobie strangler migration. Introduces OpenCorridorFeeAccrualTrait so the plain-object API (accrue, unswept, find, markSwept) can keep returning domain values instead of the Mapper entity. The one behavioural translation is the sweep's per-row mutation - Lift called `accrual.FeeSettlementId(id).saveMe()` on each row in place; the Doobie replacement is `OpenCorridorFeeAccrual.markSwept(trId, id)`, an explicit UPDATE keyed on the unique transaction_request_id, called once per accrual from OpenCorridorFees.sweep exactly as before. Flyway migration matches the probed schema: unique index on transaction_request_id (idempotent accrual per covered promise), plain indexes on debtor_bank_id and fee_settlement_id. Covered by the existing Http4s700RoutesTest fee-sweep scenario; no new characterization test needed. Full suite passes unchanged (3638 tests, 0 failures).
Table 56/140 in the Lift Mapper to Doobie strangler migration. Replaces MappedUtilityPaymentCallbackProvider with DoobieUtilityPaymentCallbackProvider behind the existing UtilityPaymentCallbackProvider trait, so UtilityCallbackDispatcher and the injector wiring in UtilityPaymentCallbacks are unaffected. This is the one-shot callback registry for UTILITY transaction-requests that supply a callback_url - distinct from the standing account-event webhook system. Flyway migration matches the probed schema: unique index on callbackid, plain index on transactionrequestid. Covered by the existing Http4s700RoutesTest scenarios for createUtilityVendResult and createTransactionRequestUtility. Full suite passes (3638 tests, 0 failures) on a clean, uncontaminated run; an earlier sharded run's single failure in ConcurrentBackoffCounterSelfHealTest (an unrelated in-memory timing test with no dependency on this table) turned out to be a flake introduced by running an isolated retest concurrently against the same target/ directory - confirmed by both an isolated rerun and this clean full-suite rerun passing.
Table 57/140 in the Lift Mapper to Doobie strangler migration. MappedWebUiPropsProvider is referenced directly by name (no DI injector indirection) from roughly fifteen call sites across AuthUser, I18NUtil, Glossary, APIUtil, and several APIMethods/Http4s version files, all going through the provider-interface methods (getAll/getByName/createOrUpdate/delete/getWebUiPropsValue) rather than raw Mapper fields - so the object keeps its name and only its backing implementation moves to Doobie, avoiding a large-radius rename. createOrUpdate preserves the original's find-by-name-then-upsert shape, including the quirk that a newly created row's webUiPropsId is always a freshly generated UUID - the caller's own webUiPropsId field, if any, is ignored on create, matching Lift's MappedUUID default-value behaviour where .create() never had that field set explicitly. Flyway migration matches the probed schema: unique indexes on both webuipropsid and name. Covered by three dedicated WebUiPropsTest suites (v3.1.0/v5.1.0/v6.0.0, 37 scenarios). Full suite passes (3638 tests, 0 failures).
Table 58/140 in the Lift Mapper to Doobie strangler migration. MappedUserScope had zero callers anywhere in the codebase - no endpoint, no test, nothing reaching UserScope.userScope.vend. Its sibling table Scope/MappedScope (plural, backing the actual v4.0.0 scopes endpoints) is unrelated and stays untouched. There is no data to migrate for a table nothing ever writes to or reads from, so the scaffold is deleted rather than given a Doobie provider nothing would invoke. Removes the code.scope.UserScope and code.scope.MappedUserScopeProvider files entirely, their Boot.scala ToSchemify entry, and the MappedClassNameTest exemption. Full suite passes unchanged (3638 tests, 0 failures).
Table 59/140 in the Lift Mapper to Doobie strangler migration. Backs the v6.0.0 management/groups CRUD endpoints (a named bundle of roles that can be granted to a user's entitlement in one shot, scoped to a bank or system-wide). Had zero existing test coverage - no endpoint test ever exercised group creation, retrieval, update, or deletion - so this adds GroupTest.scala (13 scenarios covering create/get/list/update/delete, role-gating for bank-scoped vs system-level groups, and 404s) as a characterization suite, confirmed green against the pristine Mapper implementation before migrating, then confirmed green again against DoobieGroupProvider. Flyway migration matches the probed schema: two plain (non-unique) indexes on groupid and bankid, matching the entity's own Index(...)/Index(...) declaration. Full suite passes (3651 tests, 0 failures).
Table 60/140 in the Lift Mapper to Doobie strangler migration. Replaces MappedOrganisationProvider with DoobieOrganisationProvider behind the existing OrganisationProvider trait, so the injector wiring in Organisations and every call site through Organisations.organisation.vend is unaffected. updateOrganisation preserves the original's partial-update semantics: each Option field overwrites only when present, and lastupdate is stamped on every successful update. Note that website and logoUrl use orElse against the existing value rather than getOrElse, matching Lift's behaviour where a None in the request left the stored column untouched rather than blanking it. Flyway migration matches the probed schema: one unique index on organisationid. Covered by the existing Http4s700RoutesTest organisation CRUD scenarios. Full suite passes (3651 tests, 0 failures).
The three internal id-mapping tables (account, customer, transaction)
each declared their unique indexes as:
UniqueIndex(<id>) :: UniqueIndex(<id>, <plainTextReference>)
Neither constrains the reference column. The id column is a MappedUUID
- a fresh random value on every insert, so it never collides - and the
composite is strictly implied by the single-column unique index on that
same id, so it can never reject a row the first index would have
allowed. It is dead weight: no constraint value, and no lookup value
either, since its leading column is already covered.
The consequence is that getOrCreate*Id, a SELECT-then-INSERT with
nothing underneath it, lets two concurrent calls for the SAME reference
both miss the SELECT, both INSERT, and both succeed - minting two
different OBP ids for one underlying bank reference. A later read
(LIMIT 1, no ORDER BY) then returns an arbitrary one of them, so data
written under one id is invisible under the other. This sits on the hot
path: Helper.convertToId runs it for every inbound message on the
RabbitMQ, gRPC, REST and stored-procedure connectors.
The providers already carry a "unique-index violation from a concurrent
insert - re-fetch the committed row" retry branch, written for exactly
the constraint that was never created. This makes that branch live
rather than dead code; no provider changes are needed.
V057 collapses any duplicates the missing constraint already permitted
(keeping the lowest id per reference - the earliest row, the one most
likely to have downstream data keyed to it), drops the redundant
composite indexes, and creates the single-column unique indexes. Rows
with a NULL reference are left alone: unique indexes permit multiple
NULLs, so they cannot violate the new constraint.
Only db/migration/h2 exists today and flyway.enabled defaults to false,
so this changes the test schema now and becomes the template for the
other vendor folders later. V057 carries a caveat for that port: on
PostgreSQL a constraint violation aborts the surrounding transaction,
so the retry branch only works if the INSERT is not sharing one with
later statements.
ConcurrentIdMappingRaceTest pins the invariant. It asserts the
constraint directly rather than racing: a first draft fired N
concurrent getOrCreateAccountId calls and passed with the fix reverted,
because the first insert always won before the others reached their
SELECT. A test that passes without the fix guards nothing.
The "completes immediately (no regression)" scenario failed twice in full-suite runs under load with "openFuturesCount(ok)=1: 1 was not equal to 0", while passing every time in isolation. It is a genuine race in the test, not an environment artefact. futureWithLimits returns the ORIGINAL future rather than one derived from its own onComplete callback, so Await.result can return while that callback - the one that runs decrementOnce - is still queued on the ExecutionContext. Asserting the counter on the first read after the await therefore samples a value that is correct but not yet settled. Polls with `eventually` instead. The sibling scenario deliberately keeps its Thread.sleep: there the counter is already 0 and what is being guarded against is the completion callback wrongly taking it to -1, so `eventually` would pass on its first poll, potentially before that callback has run at all, and would not exercise the hazard. Verified with three consecutive green runs. Also documents the target/classes trap that surfaced while verifying a Flyway migration in this area: Flyway loads from the classpath, and maven never deletes a script you removed from src, so stashing the .sql and re-running gives a false green off the stale copy.
Table 61/140 in the Lift Mapper to Doobie strangler migration.
The entity type leaked out of its package: AttributeDefinition appears
in Connector.scala's method signatures, LocalMappedConnector's
overrides, NewStyle's AttributeDocumentation wrapper and
JSONFactory4.0.0. The Doobie row case class therefore keeps the name
rather than being called DoobieAttributeDefinition, so none of those
signatures change and this stays a swap of the storage layer instead of
a rename rippling through the connector API. The four generated
connectors (RabbitMQ, gRPC, REST, stored-procedure) do not override
these three methods, so the trait defaults in Connector.scala are the
only definition sites.
Four call sites outside the provider queried the Mapper companion
directly - DeleteProductCascade and the already-migrated Doobie
account/transaction/transaction-request attribute providers, which were
still reaching into AttributeDefinition.findAll(By(...)). They now use
findAllByBankIdAndCategory / findAllByBankIdsAndCategory /
deleteByAttributeDefinitionId on the new companion, which reproduce the
same By / ByList shapes in SQL.
Preserves a Mapper quirk deliberately: canBeSeenOnViews is stored as a
";"-joined string and read back with a bare split, so an empty column
yields List("") rather than Nil. Callers filter that list by
membership, where the empty-string element is inert, but normalising it
would be a behaviour change smuggled in with a storage swap.
Flyway migration matches the probed schema: one unique index on
(bankid, name, category), the natural key the provider's
find-then-update keys off. Covered by the eight existing attribute
suites. Full suite passes (3653 tests, 0 failures).
Table 62/140 in the Lift Mapper to Doobie strangler migration. jobscheduler is a lock table rather than a job-history log: a row exists only while a job holds the lock and is deleted when the job finishes, so in healthy operation it is empty and any row present is either a running job or a stale lock left by a JVM that died before its finally block. The concrete type leaks into JSONFactory7.0.0's createSchedulerJobsJsonV700 signature, so the Doobie row case class keeps the name JobScheduler and that signature is untouched; only the field reads inside it change from Mapper accessors (r.JobId.get) to trait ones (r.jobId). createdAt is carried on the row because both schedulers and the v7 diagnostics endpoint use it to distinguish a running job from an abandoned lock. The Mapper query syntax at the call sites is replaced with named finders - findAllByApiInstanceId / findAllByName / findAllCreatedOnOrBefore / findByName / findByJobId - and create/delete with createJob / delete. Preserves a pre-existing bug in DataBaseCleanerScheduler verbatim: its boot-time cleanup matches Name against apiInstanceId, which never hits, because lock rows store Name=jobName and ApiInstanceId=apiInstanceId. MetricsArchiveScheduler carries a comment describing this exact mismatch and was fixed to key on ApiInstanceId; this scheduler was not. Left as-is with a comment - correcting it would change self-heal-on-redeploy behaviour under cover of a storage swap. Flyway migration matches the probed schema: one unique index on jobid. Covered by MetricsArchiveSchedulerTest and Http4s700RoutesTest's scheduler-job-lock scenarios. Full suite passes (3653 tests, 0 failures).
Table 63/140 in the Lift Mapper to Doobie strangler migration. The concrete type is what BankAccountBalanceProviderTrait is written in, so the Doobie row case class keeps the name and the provider signatures are untouched. The Connector trait is already stated in terms of the obp-commons BankAccountBalanceTrait, so nothing leaks that far and no connector changes. The Flyway script reproduces a surprising fact rather than tidying it: this table has no primary key and no indexes at all. The entity is a KeyedMapper[String, _] declaring BalanceId_ as its primaryKeyField, but Schemifier emitted a bare CREATE TABLE with none of that - information_schema.indexes returns nothing for it on a booted instance. Adding the declared primary key would be a schema change beyond a storage swap, and on an existing database it could fail outright if duplicate balanceid_ values were already let in by its absence. balanceamount is stored in the smallest currency unit, so reading it back needs the account's currency, which lives on mappedbankaccount. Mapper resolved that with a per-row lookup in a val evaluated at row construction - an N+1 - falling back to "EUR" when the account was missing. The same first-match-or-EUR resolution is now a correlated subquery inside the one SELECT: same answer, without the extra round trips. createOrUpdateBankAccountBalance keeps both of its Empty paths: an unknown account is Empty (the account is what supplies the currency), and on the update branch an unknown balanceId is Empty rather than an insert - it is create-or-update on a supplied id, not upsert-by-id. Covered by BankAccountBalanceTest plus the v2.1.0/v4.0.0 TransactionRequests suites. Full suite passes (3653 tests, 0 failures).
Table 64/140 in the Lift Mapper to Doobie strangler migration. The concrete type is used directly by LocalMappedConnector's twelve endpoint-tag methods, so the Doobie row case class keeps the name and those signatures are untouched; only the Mapper query syntax inside them becomes named finders. Both the provider trait and the Connector trait are already stated in terms of the obp-commons EndpointTagT, so nothing leaks further. A tag is system-level when bankId is absent and bank-level otherwise. Mapper wrote the system-level case as a literal null into the column and read it back as None for null-or-empty; that null/empty equivalence is preserved on the read side, and inserts now bind None. Preserves a pre-existing bug in getBankLevelEndpointTag verbatim: it repeats By(TagName, tagName) where By(BankId, bankId) was clearly meant, so a bank-level lookup has always resolved exactly like a system-level one, ignoring the bank entirely. Left as-is with a comment - correcting it would change which tag callers get back, under cover of a storage swap. Flyway migration matches the probed schema: one unique index on endpointtagid, and deliberately nothing on (operationid, tagname), so duplicate tags for one endpoint remain possible exactly as before. Covered by EndpointTagTest. Full suite passes (3653 tests, 0 failures).
Table 65/140 in the Lift Mapper to Doobie strangler migration.
Every external caller already went through MappedApiProductsProvider
and ApiProductTrait, so nothing outside the package sees the entity
type; the Doobie row case class keeps the name anyway to leave the
trait and provider signatures untouched.
Had no direct test coverage - the only suite naming ApiProduct
exercised the separate api-product-ATTRIBUTE table - so this adds
ApiProductsProviderTest, written against the Lift implementation and
confirmed green there before the rewrite, then green again after. It
pins the parts a storage swap can silently break: createOrUpdate is
keyed on (bankId, apiProductCode) and updates in place, preserving the
generated apiProductId the caller never supplies; tags are normalised
on the way in (trim, lower-case, strip embedded pipes, de-duplicate,
drop empties); and the tag filter matches a whole tag rather than a
substring, so filtering by "beta" must not return a product tagged only
"beta-2".
That last one is the reason for the storage format: tags live
pipe-delimited WITH leading and trailing pipes ("|featured|beta|") so
LIKE '%|beta|%' cannot match inside a longer tag. Kept verbatim.
Flyway migration matches the probed schema: unique index on
(bankid, apiproductcode) plus a plain index on bankid. Full suite
passes (3660 tests, 0 failures).
Table 66/140 in the Lift Mapper to Doobie strangler migration. Per-bank AMQP broker coordinates: where OBP-API publishes messages destined for a bank's own infrastructure. Every caller already went through the three companion methods (findByBankId / upsert / deleteByBankId), so replacing the storage behind them touches nothing in Http4s700, OpenCorridorPublisher, OpenCorridorSettlement or OpenCorridorFees. Two details kept deliberately. password stays on the row - it is needed to connect - but remains write-only by contract: AmqpBankBrokerJsonV700 has no password field, so no endpoint echoes it, and the migration comment records that. And upsert stays keyed on bank_id, which the unique index enforces as one broker per bank; that index is what makes the find-then-insert-or-update well-defined. The Mapper class also overrode save to stamp UpdatedAt on every write; the Doobie upsert writes updated_at explicitly on both branches, so that behaviour survives without the override. Flyway migration matches the probed schema: one unique index on bank_id. Covered by the Http4s700RoutesTest amqp-broker scenarios. Full suite passes (3660 tests, 0 failures).
Table 67/140 in the Lift Mapper to Doobie strangler migration. The name is kept from the Lift entity because DeleteProductCascade and three historical MigrationOf* scripts refer to it directly, and the row type is what ProductFeeTrait callers already see through the provider. The `type` field is stored in a column called TYPE_C, not TYPE: Schemifier appends "_c" to column names that collide with SQL reserved words. The Flyway script reproduces that name exactly - calling it TYPE would create a column the code never reads. Both declared indexes are plain Index, not UniqueIndex, so productfeeid is NOT unique at the database level even though the provider treats it as an identifier. Reproduced as-is rather than tightened; adding uniqueness would be a schema decision beyond a storage swap, and could fail outright on data its absence has already allowed. createOrUpdateProductFee keeps its asymmetry: a supplied productFeeId means update-that-row-or-Empty, no id means insert with a generated one - and the update branch rewrites bankId and productCode too, so a fee can be moved between products by id. The three MigrationOf* scripts move to the String-based DbFunction.tableExistsByName overload now the Mapper object is gone. Flyway migration matches the probed schema, including DECIMAL(34,2) for amount. Covered by ProductFeeTest. Full suite passes (3660 tests, 0 failures).
Table 68/140 in the Lift Mapper to Doobie strangler migration. The transactional outbox: business events commit their outbound messages as rows in the same DB transaction, and the relay publishes them afterwards with at-least-once redelivery. This is the table that closes the atomicity gap between a DB commit and a broker publish, so its mutation semantics matter more than most. Unlike the read-mostly tables migrated so far, this one is written from several places as the row moves PENDING -> DELIVERED or STICKY. The Mapper form was field-mutation-then-saveMe scattered across the relay; that becomes named intent-revealing updates - recordAttempt, markDelivered, markSticky, resetForRetry - which keeps each call site's meaning visible and, crucially, makes the updated_at stamp unmissable. updated_at is load-bearing rather than bookkeeping: the relay's exponential backoff is computed from it (row.updatedAt + backoff <= now), so a write that forgot to stamp it would make that row retry immediately, forever. Mapper guaranteed the stamp with a `save` override; every update helper here writes it explicitly instead, and the migration comment records why. Flyway migration matches the probed schema: three plain indexes (status, subject_id, outbox_type), none unique. Covered by the Http4s700RoutesTest operator-endpoint scenarios. Full suite passes (3660 tests, 0 failures).
Table 69/140 in the Lift Mapper to Doobie strangler migration. The OIDC token set obtained for an AuthUser at login, keyed to that user by authuserprimarykey - AuthUser's internal row id, not a user_id UUID. Every caller already went through the provider, so replacing the storage behind it touches nothing in AuthUser. Two facts reproduced rather than tidied. The table has NO index on authuserprimarykey even though every read filters on it: the entity overrode dbIndexes with nothing but super.dbIndexes, so Schemifier created only the primary key. And rows accumulate rather than being replaced - createToken always inserts and the read takes the newest by createdat - so a user's token history is retained. Both left as-is; adding an index or switching to upsert would be decisions beyond a storage swap. MigrationOfOpnIDConnectToken moves to the String-based DbFunction.tableExistsByName overload now the Mapper object is gone, and the MappedClassNameTest exemption for the entity is removed. Exercised through the AuthUser login path. Full suite passes (3660 tests, 0 failures).
Table 70/140 in the Lift Mapper to Doobie strangler migration. Had no direct coverage, so this adds UserAgreementProviderTest, written against the Lift implementation first. That baseline run earned its keep: it failed, and the failure was the test being wrong rather than the code. getLastUserAgreement does NOT return the newest row when a user re-accepts the same agreement type on the same day. The date column is MappedDate - day precision, no time - so both rows tie, and Lift broke the tie with a stable sort over rows in insertion order, which means the OLDEST of the tied rows wins despite the method's name. So re-accepting terms today keeps reporting yesterday's... in fact today's earlier... text. The Doobie query reproduces that exactly with ORDER BY date_c DESC, id ASC. Without the explicit id tie-break SQL would be free to return either tied row, which would have been a real behaviour drift hidden behind a green suite. The test pins the quirk with a comment saying plainly that it is a defect being preserved, not endorsed; fixing it needs its own change and a decision about existing rows. Two more details carried over: agreementhash is DERIVED (a SHA-256 of the text that Mapper recomputed in a beforeSave hook, so a caller could never store a hash disagreeing with the text) and is computed on insert here for the same reason; and the `date` field lives in a column called DATE_C, since Schemifier appends "_c" to names colliding with SQL reserved words - the same treatment ProductFee's `type` gets. LiftUsers' batched multi-user lookup moves from findAll(ByList(...)) to findAllByUserIds. Full suite passes (3663 tests, 0 failures).
Table 71/140 in the Lift Mapper to Doobie strangler migration. Every caller already went through the provider, so replacing the storage behind it is confined to this package plus one accessor change at the claim endpoint. Three things carried over deliberately rather than tidied. secretKey is the sole credential on an invitation link: getUserInvitationBySecretLink resolves an invitation from it alone, with no bank scoping. So the Doobie insert generates it with the same CSPRNG rather than accepting one from a caller. The column has neither an index nor a uniqueness constraint despite every claim filtering on it - reproduced as-is, since changing that is a schema decision, but worth knowing it is the shape of the thing. createdAt is not bookkeeping: the claim endpoint expires an invitation 24 hours after it, so the row carries it. Dropping it as a CreatedUpdated artefact was my first instinct and the compiler caught it - the field is load-bearing and is now commented as such. scramble, the personal-data erasure path, replaces each field with a random string OF THE ORIGINAL FIELD'S LENGTH. That is what the Mapper version did and it is preserved: a fixed-width scramble would change how much a retained row still reveals. Flyway migration matches the probed schema: one unique index on userinvitationid. Covered by UserInvitationApiTest, AgentDelegationTest and Http4s700RoutesTest. Full suite passes (3663 tests, 0 failures).
getTopUsers and getTopConsumers called createQueriesByHttpParamsFuture directly on the raw request params instead of going through APIMetrics.applyMetricsFromDateDefault the way every other metrics-reading endpoint does. With no from_date, APIUtil.getFromDate substitutes the epoch, which makes MappedMetrics.determineMetricsCacheTTL classify the query as "only stable data" and pick the 24-hour TTL - so the default, no-parameter call an operator dashboard would make froze for a day while traffic kept arriving, and the first miss of that day scanned the whole metric table since 1970. Corrected the two ResourceDoc descriptions to match (they claimed "defaults to one year ago" / "the current date", which was never the actual range).
createAccountJSON's Links.Self does list.head.AccountId unconditionally. getAccount builds that list by filtering the caller's own private accounts down to the requested accountId, which is legitimately empty for an id that does not exist (or belongs to someone else) - the same shape a real TPP integration hits on a typo or a stale id. That empty list reached list.head and threw NoSuchElementException, answering 500 instead of the 404 UK Open Banking's spec calls for. Found by extending the endpoint auth/crash sweep to cover Berlin Group and UK Open Banking (previously OBP-standard only) - FailureSweepTest calls every endpoint with a nonexistent id and asserts none of them 5xx.
…n Banking
EndpointCatalog.all was Http4s700.allResourceDocs - the OBP-standard
aggregation only. AuthSweepTest, SuccessSweepTest and FailureSweepTest all
read their coverage from it, so every Berlin Group and UK Open Banking
endpoint was silently outside the anonymous-401/crash sweep: a doc in
those standards missing AuthenticatedUserIsRequired with empty roles would
let anonymous callers reach account data and nothing would catch it.
Switching to ResourceDocRegistry.allStaticResourceDocs (the same
cross-standard union APIUtil.getAllResourceDocs already exposes) needed
three follow-on fixes, all specific to a catalog that now spans multiple
independent route trees rather than one:
- EndpointCatalog.concretePath hard-coded "/obp/" + apiShortVersion.
Berlin Group and UK Open Banking routes match on Root / urlPrefix /
apiShortVersion with no "/obp" segment at all (see e.g.
Http4sBGv13AIS.bgV13Prefix) - urlPrefix is "obp" for the OBP standard by
construction (ApiVersion.setUrlPrefix patches it to the configured
apiPathZero at boot), so using implementedInApiVersion.urlPrefix
uniformly reproduces the old OBP behaviour while giving BG/UK their own
real prefix instead of a path that 404s before reaching any route.
- AuthSweepTest.messageOf only read the top-level "message" field. Berlin
Group requests get a PSD2-mandated {"tppMessages": [{"text": ...}]}
envelope instead (ErrorResponseConverter.toBgErrorBody) - the endpoint
was correctly answering 401, the sweep just could not see the message
text to compare it against. Fall back to tppMessages[0].text, which
carries the identical string the OBP envelope would have.
- SweepCoverageTest's "deduplicated by (url, verb)" check assumed one
route shape maps to one operation, true for OBP but not for Berlin
Group: several SCA sub-steps (e.g. updatePsuAuthentication /
selectPsuAuthenticationMethod / transactionAuthorisation) legitimately
share one URL and verb, disambiguated by request body rather than path.
Replaced with a check on operationId uniqueness, which is the union's
actual by-construction guarantee and still catches a genuine duplicate
(e.g. two ResourceDoc objects registered under the same operation id).
Two categories of endpoint answer non-2xx to SuccessSweepTest's
fully-entitled-but-consentless caller and are documented in
expectedNon2xx rather than treated as failures: Berlin Group AIS and UK
Open Banking account-read endpoints both require an established,
standard-tagged consent regardless of role, which the sweep's generic
fixture (grants every role, creates no consent) does not provide. The
403 in both cases is the endpoint correctly refusing, not a defect - the
anonymous case is what AuthSweepTest already covers independently.
…ie stores
develop's 73 commits here are four features and one large teardown.
The features: maker/checker for runtime-supplied code (a Dynamic Change Request holds a
proposed create/update/delete of a dynamic resource doc, connector method, message doc or ABAC
rule until a second person approves its payload hash, and the runtime executes only rows whose
body hash equals the approved one); API Product Subscriptions (one Consumer holds one product
for a period, with a status machine and the Scope and RateLimiting rows it created); Dynamic
Glossary Items, which an operator adds at runtime and which may shadow a static item; and the
on-behalf-of attribution framework, which renames accountableUserId to onBehalfOfUserId, pulls
the resolver into Users.onBehalfOfUserIdOf, and states in UserReference.scala, per column,
whether a write by a consent user is recorded against the consent user or the human it acts for.
The teardown: the 12 OBPAPIx_y_z aggregator objects and the VersionedOBPApis trait are gone,
along with the APIMethodsXYZ.scala shims. Version enumeration is now explicit in
Http4sResourceDocAggregation.allVersions, and the Lift ResourceDoc text those files carried as
comments was exported first to scripts/resource_doc_baseline/*.json, which is what the parity
audit now reads.
All four features were written against Lift Mapper entities and created their tables through
Schemifier. This branch has neither: ToSchemify.models is empty, so a Mapper column here would
compile and then not exist. The resolution carries the behaviour across instead of restoring the
entities - DynamicChangeRequest, DynamicGlossaryItem, ApiProductSubscription, its Scope join
table and its Attribute table are case classes over Doobie stores, MakerChecker's 33 Mapper call
sites read through the Doobie providers, and the five tables plus the approvedhash/isactive
columns on the four target types and dynamicentity.authmode come from a new
db.changelog-develop-merge-2.yaml.
Three of develop's entries in the new attribution policy name entities this branch does not
have, and one names a column it does not have:
- 16 of the 65 classes UserReference.scala names are Lift Mapper entities this branch already
moved to Doobie. They are only ever read to build the log line in Users.attributionOf, but
AgentDelegationTest resolves each one, so they are repointed at the class that actually owns
the write here, with the real column names beside them.
- PemUsageLastUser and UserScopeUser are dropped: 6bfcb4b deleted the PemUsage scaffold (its
table was never created in any environment) and ac72b13 deleted MappedUserScope (no
callers). A policy entry for a table that does not exist is not a policy.
- abacrule already carries isactive from the baseline, unlike the three dynamic-code tables, so
its maker/checker changeset adds only approvedhash. Liquibase aborted the boot on the
duplicate column, which reads as a test-infrastructure failure rather than a schema one.
Two defects were introduced while resolving conflicts and are fixed here rather than left for CI:
The attribution policy was not actually applied to dynamic data. MapppedDynamicDataProvider
defines ownerOf and never calls it - save, update, get, getAll and existsData all passed the raw
caller id - and MapppedDynamicEntityProvider.createOrUpdate stored dynamicEntity.userId
unresolved. Taking our side of those conflicts kept the Doobie method bodies and dropped the
wrapping develop had put around them, so a consent user's rows and definitions were owned by the
agent rather than by the human it acts for. That is the whole point of the feature, and it is
silent: nothing fails, the row is simply attributed to the wrong user.
Four resource-doc examples published a dangling $ref to Object. A field of an erased type takes
its published type from its example value, so ProvenanceJsonV700.is_active and three of
RateLimiterLimitJsonV700's windows, left as None by develop's examples, had no type to recover.
DynamicChangeRequestJsonV700.current_payload is a separate case: it is a bare JValue and is
genuinely JNothing for a CREATE, which json4s omits from the body entirely - that now publishes
as {"type":"object"}, which is what the null branch already answered for the same field, while
every other declared type with an empty example still fails. buildSwaggerSchema's own failures
named neither the class nor the field, so translateEntity now adds that; it is how the offending
field was found at all.
Alongside: the four table-count assertions move from 147 to 152 for the five new tables, and
run_tests_parallel.sh's per-shard hard kill becomes SHARD_TIMEOUT_SECONDS, defaulting to 2400s
on Postgres. That kill exists to stop a JVM whose Pekko threads hang after the tests finish, not
to bound how long tests may take, and at 1200s three of the four Postgres shards were killed
mid-run - each still emitting test output at the moment it died. A killed shard reports zero
failures while dropping ~800 tests from the audit line, so the run reads as a pass unless the
per-shard durations are checked.
Suite: 4233 tests, 0 failures on H2 and the same 4233 on Postgres, where the three shards now
finish in 20:43, 21:58 and 22:39 - which is what the old 20-minute kill was cutting off.
… their roles ResourceDocMatcher picked the first doc whose template matched, which is registration order. Two docs can share (verb, version, segment count) and both match a request when one spells a segment as an all-caps wildcard: v4's catch-all /banks/BANK_ID/accounts/ACCOUNT_ID/GRANT_VIEW_ID/transaction-request-types/TRANSACTION_REQUEST_TYPE/transaction-requests also matches .../FREE_FORM/..., and it is registered in the object body while the per-type alias docs are added later by initBatch9AliasResourceDocs(). The catch-all declares no roles, so ResourceDocMiddleware - which reads roles from the matched doc only - ran no role check at all, and the FREE_FORM doc's Some(List(canCreateAnyTransactionRequest)) was never enforced. A user holding only the owner view could create a FREE_FORM transaction request and get 201; Lift returned 403, and Lift had no wildcard doc for this URL at all - all nine were literal per-type entries, so this was introduced by the migration. Two changes, both at the mechanism rather than at the call site: findResourceDoc now picks the match with the most literal segments instead of the first one. sortBy is stable, so docs of equal specificity keep their registration order and nothing else moves. literalAllCapsSegments gains the seven transaction-request types that were missing from it - BULK, HOLD, MOBILE_WALLET, UTILITY, CARDANO, ETH_SEND_TRANSACTION, ETH_SEND_RAW_TRANSACTION. Specificity ordering cannot separate two wildcards, and MOBILE_WALLET was shadowing UTILITY and OPEN_CORRIDOR_PROMISE in v7 (HOLD and CARDANO shadowing the two ETH_* docs in v6) the same way, which attached the wrong operationId to those requests - so api_disabled_endpoints, the JSON schema and auth-type allow-lists, and the metric rows all keyed on the wallet endpoint. Its own doc comment already said to add a value here when a path uses a new all-caps literal. TRANSACTION_REQUEST_TYPE stays a wildcard: it is the only one of the eighteen that is a placeholder rather than a type.
… not the human createTransactionRequestImpl210 is the only writer of on_behalf_of_user_id. develop resolved it with Users.attributionOf(userId, UserReference.TransactionRequest); porting the method to the Doobie store kept the Doobie body and dropped the attribution block, leaving onBehalfOfUserId = callContext.flatMap(cc => cc.onBehalfOfUser.or(cc.consenter).map(_.userId)) CallContext.onBehalfOfUser is consentCreator.or(consenter).or(user) - a plain field fallback. It does not consult the consent chain; the resolver that does lives in the String sibling (ApiSession.onBehalfOfUserId -> Users.onBehalfOfUserIdOf), whose own doc describes three steps. The expression above did step 1 and then jumped to step 3, so for a consent user whose CallContext carries only `user` - the ordinary case, and the one AgentDelegationTest asserts is resolved through ResourceUser.createdByConsentId - the row was written with userId == onBehalfOfUserId == the agent's id. The request succeeds and the attribution WARN is never logged, so nothing surfaces: the payment is simply owned by the wrong user. Restores the attributionOf call, keeping develop's precedence rule that a consentCreator or consenter populated by a middleware still wins over the database chain. (.or(cc.consenter) in the old expression was dead in any case - consenter is already inside onBehalfOfUser's chain.) The other five attributionOf call sites survived the port; this was the one that did not. UserReference.TransactionRequest still declared policy UseOnBehalfOfUserId with no callers left.
…same id deleteAtm issued DELETE FROM mappedatm WHERE matmid = ?, keyed on the ATM id alone. The unique index mappedatm_mbankid_matmid is on the pair, so two banks may legitimately hold the same atm_id, and every other statement in this store - getAtmFromProvider, the createOrUpdateAtm re-read, updateColumn - filters on both columns. deleteAtm was the only one that did not, though atm.bankId.value was in hand. The Lift code this replaced was also bank-unscoped, but it deleted through find(By(mAtmId, ...)), which returns at most one row - so the blast radius was one wrong row rather than all of them. Widening that to a set-based DELETE is what the migration introduced. Note deleteAtmAttributesByAtmId, called immediately after this in Http4s510, is unscoped for the same reason and atmattribute does carry a bankid column - but its signature comes from the Connector trait, so fixing it changes a connector method signature and is left out of this commit deliberately.
…mn, a dead tryo MetricsArchiveRun.pruneToMostRecent built DELETE ... WHERE id < (SELECT MIN(id) FROM (...)) with no alias on the derived table. PostgreSQL required one until version 16, so the statement is a syntax error on any PG 15 or older deployment. Nothing catches it: CI pins postgres:16-alpine and H2 never required an alias. The consequence is worse than a failed prune, because MetricsArchiveScheduler calls recordRun again from inside its own catch - the second throw is unguarded and escapes runOnce, so the tick ends in an exception and the table this class documents as self-capping grows without bound. buildTopConsumersQuery reused buildFilterConditions, whose consumerId fragment is the unqualified `consumerid`. That query selects FROM metric, consumer and both tables carry the column, so ?consumer_id=X on /management/metrics/top-consumers fails with an ambiguous column reference. getTopConsumersByConsumerId avoided the reuse for exactly this reason and says so in a comment. buildFilterConditions already took resolvedUserIdExpr so a caller could qualify userid; consumerIdExpr does the same for this one, and buildTopApisQuery - which selects FROM metric alone - keeps the bare default. getAllAggregateMetricsBox and getTopConsumersFuture wrapped an already-evaluated val in tryo. The query ran on the line above, so tryo had nothing to catch and a failure threw straight past it, bypassing the caller's unboxFullOrFail and its OBP error code. The query now runs inside the tryo, which is how the other four memoized methods in this file were already written.
Every other version reaches the chain through gate(), which resolves to HttpRoutes.empty when APIUtil.versionIsAllowed is false. The three Berlin Group trees were spliced in raw. Neither BG wrapper re-checks the prop, and ResourceDocMiddleware deliberately does not re-check version-level enablement per request - it says so at isEndpointEnabled, because the check is meant to happen once at startup in exactly this place. So Berlin Group had no version gate at any layer. The asymmetry is what makes it a problem rather than an omission: with api_disabled_versions=BGv1.3 set, GET /obp/v7.0.0/api/versions reports BG as inactive while /berlin-group/v1.3/accounts/ACCOUNT_ID/transactions keeps returning consent-scoped account data, and UKv3.1 disabled by the same prop does go dark.
…eware skipped them ResourceDocMatcher indexes on (verb, apiVersion, segment count). elasticSearchWarehouse's template was "/search/warehouse" - two segments - while its route is `prefixPath / "search" / "warehouse" / queryString`, three. A real call therefore looked up a key the doc was not filed under and found nothing, and there is no prefix or longest-prefix fallback. elasticSearchMetrics had the same mismatch. With no doc, ResourceDocMiddleware takes its unmatched branch, which calls resolveCallerWithoutRateLimiting instead of anonymousAccess and never runs validateOnly. Four things silently did not happen for these endpoints: the Consumer rate limiter never counted the call, so this Elasticsearch proxy could be driven unthrottled and any per-endpoint limit row for it was inert; api_disabled_endpoints could not disable them; auth-type and JSON-schema validation never ran; and the metric row carried an empty operationId. Authentication and the canSearchWarehouse / canSearchMetrics checks are inline in the handlers, so this was a control bypass rather than data exposure. The templates gain a third segment. SEARCH_QUERY is outside literalAllCapsSegments so it stays a wildcard. That diverges from the Lift baseline, which is the documented placeholder-rename case: both are recorded in parity_allowlist.json with the digests allowlist_helper.py computed.
resetDatabaseForTestClass states that every table is listed explicitly, since no entity is a Lift Mapper any more and there is no model loop to clear them. The five tables db.changelog-develop-merge-2.yaml creates were not in the list, so that statement was false for them and their rows survived into later test classes - the suite runs forkMode=once against one h2:mem database with DB_CLOSE_DELAY=10, so the leak spans the whole JVM. Nothing failed because of it: the suites that write these tables use UUID-suffixed ids, and the one fixed glossary title is deleted by every scenario that creates it. That is luck rather than isolation, and check_test_isolation.py would not catch the next table added without it - it checks setPropsValues placement, not table coverage.
…em were bare
check_nullable_column_reads.py exists to prove that no store reads a nullable column into a
non-nullable Scala type, because doobie's Get throws NonNullableColumnRead on a SQL NULL and
fails the whole query rather than the one row - one legacy row turns a listing endpoint into a
500. It printed "OK: every nullable column is read through Option" while structurally unable to
see most of the code it was pointed at. Two independent blind spots:
The scan matched only `type XxxRow = (...)` alias declarations. About a third of the stores are
written that way; the rest write the row type inline at the `.query[(...)]` call site or use a
case class, and neither was ever examined. Adding those two shapes - plus comment stripping,
without which this script's own scaladoc example (a `case class TopApi` beside a
`SELECT ... FROM metric`) matched itself and reported three violations against a util class that
runs no queries - takes the count from 0 to 597 across 47 stores.
read_ddl broke out of a createTable block the moment it saw `tableName:`. That works only
because Liquibase's writer emits keys alphabetically and so puts tableName after columns; a
hand-written changelog puts it first, and the scan then registered the table with no columns at
all. Empty is worse than absent here: find_violations skips any column not in the map, so every
column of those tables was silently exempt. Six tables from the develop merge were in that
state, addColumn changesets were never read, and flow-style `- column: {name: x}` entries were
not parsed. The block is now read by indentation, in either key order and either style.
The 597 reads are fixed the way the script's own docstring prescribes - bound as Option and
collapsed to what Mapper's reader returned: String to null, Boolean to false, numerics to the
field's default, Timestamp to a null Date. Three shapes appear across the stores and each keeps
its callers unchanged: a tuple consumed as `r._N`, a `case (...)` destructuring (binders renamed
and given a collapsing val), and a case-class row (fields become `xRaw: Option[T]` with a
defaulted `def x: T` accessor). Where a row class is constructed by name outside its store -
DoobieInvestigationQueries.CustomerRow is built by Http4s600 - the class keeps its shape and the
collapse happens at the query instead.
Spot-checked rather than taken on faith: customerlink declares every column except its primary
key nullable, and the store read eight of them bare at six query sites.
The attribute providers bind the free-text columns that come straight from the request body - name and value - as bare String. Doobie's Put for a non-nullable type calls unsafeSetNonNullable, which throws on a null, and the create branch is not inside tryo, so the exception escapes the Future and the caller gets a 500 with no OBP frame in the trace. A null does reach the bind. The JSON case classes declare value as a plain String, both of this repo's custom String deserializers explicitly exclude JNull, and json4s 4.1.1's default ExtractionNullStrategy is Keep - so extraction yields null rather than raising, and no upstream validation rejects it: the endpoint validates `type` and passes `value` through untouched. Lift's MappedString did not override setFilter, so it stored SQL NULL and returned 201. Binding as Option restores that. The same wrapping lands on the name filter in the attribute WHERE clauses, which is the behaviour Lift's By(field, null) had: it renders `= NULL` and matches nothing, rather than throwing at bind time. Ten providers, 48 binds.
AuthSweepTest keeps a signed-off list of endpoints whose observed authentication differs from what their ResourceDoc declares, and a scenario that fails when an entry stops being needed - an exemption nobody removes is read by the next person as still true. createTransactionRequestFreeForm was on that list for answering 400 rather than the 403 its doc declares. The written reason - that the endpoint does no upfront role check and leaves the decision to the connector - described the symptom rather than the cause: the doc the middleware matched was v4's wildcard catch-all, which declares no roles at all, so no role check ran for any transaction-request type. With the matcher now preferring the most specific template, the per-type doc wins and its canCreateAnyTransactionRequest is enforced. The sweep caught this on its own, which is the whole point of that scenario.
|
develop's 6b1b49f put the AGPL header at the top of 1119 files. This branch has changed the imports directly under the package line in most of them, so merging that commit as it stands turns a comment-only change into a conflict in every one of those files. Applying the same patch here first makes the two sides agree on the top of each file: 916 files took develop's diff unchanged, and 118 whose leading context had moved got the identical 27 lines prepended. The 85 the commit touches that this branch has deleted are left deleted. No header is added to files that exist only on this branch - develop itself carries several hundred Scala files without one, so it is a one-off change there rather than an enforced rule.
develop adds 496 commits, written against Lift Mapper and Schemifier, which this branch no longer has. Their features are carried across as Doobie stores, with the schema they need declared in Liquibase: - db.changelog-develop-merge-3.yaml: the 21 new tables (files, assets, retention, tenants, domain APIs, platform apps, consumer account access, Dynamic Entity webhooks, account creation, group memberships, self-service banks), the new columns (Dynamic Entity record metadata, metric forwarded_for/domain_api_url, signing basket consumer, user auth context update bank, counterparty on-behalf-of creator, dynamic resource doc lang) and indexes, taken from the schema upstream's Schemifier built at the merge base and at db74dd4. - db.changelog-dynamic-entity-space.yaml: system level Dynamic Entity rows move from a NULL bank id to SYS, and the indexes the space-scoped ones replace are dropped. - The Dynamic Entity, record, access and resource doc stores address a row by (bank or SYS, entity, id); records keep who created and last updated them. - Data migrations upstream wrote with Mapper (role renames, Organisation to Tenant, self-service banks, currency upper-casing) read and write the rows through JDBC; MakerChecker re-hashes dynamic code with its language once. - Organisation is replaced by Tenant, as upstream did. - The dynamic code dependency scan stays ungated: getDependentMethods and getDynamicCodeDependentMethods accept upstream's `force` argument but always scan, so the security validation never sees an empty list.
Takes develop's v7.0.0 customers-at-one-bank attribute filter. The new test imports the json4s extraction implicit that this branch's Scala 3 build needs.
…credentials for the credential rule
…nario call With the brace on the next line the body is a separate statement of the Feature, so the suite aborted while loading.
…ules and code rows back on
…NULL reply or error can be settled, and one failing row no longer stops a relay pass
…e columns it changes
…uthor's code, and the dry-run output is deleted at once
…defines, so restricted calls inside closures and local objects are rejected
…en row-level access is switched on
… classes of Java snippets, and does not reject the wrapper's own calls
… per-line tag, and lines of the template report as 0
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Migrates OBP-API to Scala 3, replacing the two frameworks that blocked it. 287 commits on top of
3df73fe11; 1027 files, +66009/-33154.Opened for review of the whole line. It does not merge cleanly onto
developyet — the branchis 26 commits behind and conflicts in 12 files, all of them where
developextended a Lift Mapperentity this branch had already moved to Doobie. That merge is in progress separately and will be
pushed here; the conflicts are listed at the end so reviewers know what is coming rather than
discovering it from a red merge box.
What changed
Scala 3. Lift Mapper cannot compile under Scala 3 — the compiler crashes on the
object X extends class Xshape every Mapper entity uses (reduced to a 5-line case). Both of itsconsumers had to go first.
Lift Web → http4s, complete.
net.liftweb.httpno longer appears in any.scalasource. Thereis no Lift fallback in the request chain: an unmatched
/obp/*path returns a JSON 404 fromnotFoundCatchAll. API versions are unchanged by this — a framework migration happens in placeinside the existing version file, and a version bump still means a changed API signature.
Lift Mapper → Doobie, complete.
ToSchemify.modelsis nowNil; Schemifier creates nothing.Flyway → Liquibase. Flyway needed one hand-written script set per vendor: 118 for H2, 118 for
Postgres, and nothing for the three other drivers its
vendorFolderwould have booted againstsilently, with no tables. One changelog now describes each change once and Liquibase emits the
dialect. The baseline is generated from a Postgres database the Flyway scripts built, not
hand-written, and is regenerated with
scripts/GenerateChangelog.javaplus a normaliser ratherthan edited.
Defects found and fixed along the way
Migrating the data layer surfaced behaviour that lived in Mapper's field types rather than in the
entities, and which a column-by-column port drops silently:
MappedBooleanreads back asfalsewhateverdefaultValuedeclares, a NULLMappedLongas the declared default — read as a hardcoded-1, six call-limit columns turned"the configured limit" into "no limit", and that value is what the rate limiter enforces from
MappedEmaillowercases and trims on every set and validates on save;authuserlost all of itMappedMetrics, reachable from read-only roles, giving a boolean-blindoracle over the whole database
Security fixes on top: a locked account could still authenticate through OIDC and Keycloak (both
read
v_oidc_usersover JDBC and never callverify-credentials);consents.sca.enabled=falseaccepted any SCA answer in production; the dynamic-code dependency validation inspected an empty
list because its scan was gated on an unrelated diagnostic prop; and
compileScalaCodenow refusesto run when no SecurityManager can be installed (JEP 486) unless the operator says so explicitly.
Testing
3876 scenarios, 0 failures, on both H2 and Postgres. Postgres is not optional here: H2 tolerates
things Postgres does not, and the schema is generated from the changelog at boot on both.
Conflicts with current develop
developaddedcreatedByUserId/updatedByUserId/ a method-body hash to DynamicResourceDoc,DynamicMessageDoc and ConnectorMethod, and one new Mapper entity (
ChatEmailDigestState). Thisbranch had already moved the first three to Doobie, so the resolution ports the new fields into the
Doobie stores and the changelog rather than restoring the Mapper entities;
ChatEmailDigestStateneeds the same treatment, since with
models = Nilits table would otherwise never be created.