Compare commits

...
Author SHA1 Message Date
zhangwenjian 2c50317a98 fix🐛: stop the duplicate check refusing what the index would accept
The check grouped by (app_code, path, action) and refused whatever
appeared more than once. GROUP BY treats two NULLs as the same value; a
unique index treats them as different ones and allows both. So a
database holding rows with a null path or action was refused for
duplicates the index it was blocking would have accepted - and the
migration stopped, on a database with nothing wrong with it.

Both columns are nullable: neither carries a not-null tag, so gorm built
them that way. Measured on MySQL 8.0, PostgreSQL 15 and SQLite: two rows
with both columns null are one group to GROUP BY, and the unique index
builds over them without complaint. Standard SQL, not a dialect quirk.

They are now excluded from the check rather than grouped. Each column
needs its own exclusion and has its own test: one null column is enough
to make the index accept the pair, so removing either condition alone
lets that half through - which is what the two subtests are for, and
each fails only for its own half.

The message could not name the rows either. MySQL's CONCAT returns NULL
when any argument is, and scanning that into a string fails with
"converting NULL to string is unsupported" - so the check reported a
driver error instead of the duplicates it exists to report. SQLite and
PostgreSQL treat a null argument as empty and say nothing, which is why
this never surfaced in the tests: they run on SQLite, and this
repository has no MySQL in CI. The comment says so, so that a
postgres-only test file is not mistaken for cover.

No COALESCE was added to paper over that. It would have had nothing left
to guard once the nulls are excluded, and it would make a future
regression quieter: someone dropping the exclusions would get a report
naming rows that are not duplicates, which reads as a real answer,
rather than a scan error that reads as a broken query.

Leaving path and action nullable is deliberate. Tightening them is a
migration of its own - existing null rows have to be given values, and
what those values should be belongs to whoever owns the data, not to a
migration whose job is adding an index.
2026-09-08 20:17:50 +08:00
zhangwenjian 28350a15bb fix🐛: repair the row a retry reuses instead of walking past it
The idempotency check added in the previous commit stopped a retry
inserting a second copy, and introduced a quieter failure in its place:
a retry that found an existing sys_menu row skipped everything after the
insert. Those are the sys_menu_api_rule bindings and the materialized
path, and neither is written by the statement that writes the menu -
paths is a separate UPDATE, and on MySQL an earlier DDL has already
committed the transaction that was supposed to hold them together.

So an install interrupted between those steps left a menu that exists,
sits outside the tree with an empty path, and is bound to no API. What
core's contract.md says about such a menu is that it is invisible to
every role and its apis are authorized for no one - while the installer
reports success.

Reusing now repairs. paths is compared before it is written, so a row
that is already right is not touched. Bindings are inserted with WHERE
NOT EXISTS rather than deleted and rebuilt: an administrator can bind an
api to a menu from the menu screen, and delete-then-rebuild would take
that with it on the next retry - the same accident as sys_role.go's
Association.Delete, pointing the other way.

Confirmed as a defect before it was fixed, by building the half-written
state and watching the assertions fail:

    dir.Paths = "", want "/0/1"
    binding count for list = 0, want 1

Four paths through the repair, each with a degradation that reds its own
test and leaves the others green: missing bindings only, missing paths
only, both, and neither. The fourth asserts no UPDATE is issued for a
row already correct.

A fifth covers what the repair must not do. Rebuilding bindings instead
of inserting them leaves every other test green while silently deleting
a binding this code did not create; that one now fails with "a retry
silently deleted a binding it does not own".

Bindings an older version of a manifest created and a newer one no
longer lists are left alone. Removing them is a delete, and a delete
needs the same certainty about ownership that uninstall does - this
function cannot tell a stale binding from one somebody added by hand.
2026-09-08 19:51:22 +08:00
zhangwenjian c6d3ea5f81 fix🐛: stop a retried seed from inserting a second copy
seedApis and seedMenuTree were bare tx.Create calls. A migration that
failed partway and was run again re-inserted everything it had already
written - which is not hypothetical: the demo site collected eighteen
duplicate sys_menu rows this way, and three duplicate menus were visible
in its sidebar.

Both now look for a live row already holding the natural key and reuse
it. Only live rows count: a row an earlier soft-delete retired does not
stand in the way of a fresh insert under the same key, which is also
what the unique indexes allow.

The app_code half of each key has a test of its own. Without it the
lookups still passed every existing test while quietly letting one
application adopt another's rows - and an uninstall would then delete
rows the other application believed were its own, on both sides without
an error. Removing app_code from either lookup now fails with
"has 1 row(s) ... want 2 - one per app".
2026-09-08 19:26:42 +08:00
zhangwenjian 7fadb4b585 feat✨: give the seeded rows a natural key to be found by
Seeding needs something to look for before it inserts, or a retry writes
a second copy of everything it already wrote. sys_api already had one in
(app_code, path, action). sys_menu had nothing usable: menu_name is
pascalCase(appCode) + pascalCase(code), which is not injective -
"list-all", "listAll" and "list_all" all become "ListAll" - so the
original code cannot be recovered from it. Hence a new column.

seed_code is nullable, against this repository's habit of NOT NULL
DEFAULT '' for a new column, and deliberately. Every row that predates
it has no meaningful value, and under a unique index an empty string
collides with every other empty string while NULL collides with nothing.
The convention exists because deleted_at's nullability broke a unique
index; here nullability is what makes one possible.

The unique index on sys_api cannot simply be created: a live database is
known to hold historical duplicates - the demo site had eighteen. The
migration looks first and refuses while naming the offending rows,
rather than letting CREATE UNIQUE INDEX fail with a constraint error
that names none. Same shape as 1786700003000's refuseOnDuplicates.

Run against SQLite, MySQL 8.0 and PostgreSQL 15, including the CONCAT
duplicate check, which had only ever been executed by SQLite's driver.
2026-09-08 19:26:42 +08:00
zhangwenjian 691df82016 feat✨: add the sys_app registry and the casbin grant ledger
sys_app is one row per installed application. It is physically deleted
on uninstall rather than following the millisecond soft-delete marker
the other sys_ tables use: an installed-app registry has no "deleted by
accident, needs recovering" case, and a physical delete is what lets the
same code be installed again afterwards.

status is installing/installed/failed rather than a boolean, because an
install spanning several migration files is not atomic on MySQL - DDL
commits implicitly, so a run can stop in the middle. failed_version and
last_error are diagnostic snapshots for a person to read; nothing may
decide anything from them, and the field comments say so. Where to
resume is answered by sys_migration, which cannot drift from what was
actually applied.

sys_app_casbin_grant records which casbin_rule rows an install created,
keyed by casbin_rule's own natural key. That table is not extended
instead: gorm-adapter's SavePolicy truncates and reloads it from an
in-memory model, which would drop any column added here without a word.

Built against SQLite, MySQL 8.0 and PostgreSQL 15.
2026-09-08 19:26:40 +08:00
zhangwenjian ea348fa9d1 build📦: pin go-admin-core v2.8.0
v2.8.0 adds sdk/contract/app - an application's manifest, and the one
comparator for its version - which the installer in this batch is built
on. Nothing here uses it yet; this is the dependency arriving.

Checked that the release is consumable rather than only tagged: a
program built against the published module registers a manifest, reads
it back, and gets -1 from Compare("1.9.0", "1.10.0"), which is the
multi-digit case a string comparison would order backwards.

25 packages pass and checksilent is clean on the new version.
2026-09-08 18:35:13 +08:00
wenjianzhang 925c6772a6 Merge pull request #921 from go-admin-team/fix/920-schema-readiness
Fail readiness while the database is behind the migrations
2026-09-08 17:18:34 +08:00
wenjianzhang 0c60e44aee Merge pull request #922 from go-admin-team/fix/919-drop-index-postgres
Drop the index with SQL this dialect can parse
2026-09-08 17:18:08 +08:00
zhangwenjian a716086295 test✅: fail these tests when the database does not answer
Every Create and Raw dropped its error. Most of them would have failed
an assertion further down anyway, with a message describing the wrong
problem - but one of them would not.

The last count in the index test reads how many indexes survived and
expects zero. An unchecked query that fails leaves the variable at zero,
and zero is what success looks like: a test that cannot reach the
database reports that the indexes were dropped.

Demonstrated rather than assumed, by breaking that one query both ways:

    with the check:    FAIL  counting the indexes after: relation
                             "pg_indexes_nope" does not exist
    without it:        PASS

Raised by Copilot on #922, as a consistency point with the SQLite tests
in this package. It is that as well, but the reason it is worth doing is
the row above.
2026-09-08 17:04:31 +08:00
zhangwenjian f8a5066a40 test✅: run the migration against PostgreSQL in CI
The rest of this package's tests run on SQLite, where dropping an index
through the migrator works. That is why a migration which failed on
every PostgreSQL database it was pointed at had a green suite: the
defect cannot occur on the backend being tested.

A test alone would not have helped either - without a service it skips,
and a test that never runs is the same as no test. So the workflow gains
a postgres service and the DSN, and the helper refuses to skip when CI
is set: a workflow that drops the service or renames the variable fails
rather than going quietly green, which is the shape of the original
defect.

Counter-proved by putting Migrator().DropIndex back, which reproduces
the statement verbatim:

    DROP INDEX CURRENT_SCHEMA()."idx_sd_pg_user_deleted_at"
    ERROR: syntax error at or near "CURRENT_SCHEMA" (SQLSTATE 42601)

The conversion test also checks the timestamp survives as a marker,
since a conversion that dropped it would bring deleted rows back live
while still passing a column-type assertion.
2026-09-08 16:57:17 +08:00
zhangwenjian f978967ef1 fix🐛: drop the index with SQL this dialect can parse
The soft-delete conversion dropped the indexes on deleted_at through
gorm's Migrator().DropIndex. Its PostgreSQL driver resolves a schema for
the statement and falls back to an expression when it cannot:

    currentSchema, _ := m.CurrentSchema(stmt, stmt.Table)
    m.DB.Exec("DROP INDEX ?.?", currentSchema, clause.Column{Name: name})

DROP INDEX takes an identifier in that position, so what reached the
server was

    DROP INDEX CURRENT_SCHEMA()."idx_sys_api_deleted_at"
    ERROR: syntax error at or near "CURRENT_SCHEMA" (SQLSTATE 42601)

The schema is unresolvable for every call this migration makes, because
it passes a table name as a string rather than a model. So it failed on
every PostgreSQL database rather than intermittently, and stopped the
whole conversion at the first table.

What that looked like from outside is go-admin#919: an upgrade that
could not complete, and a login rejecting a correct password, because
deleted_at was still a timestamptz while the current query compares it
to 0. Neither symptom names a migration.

Written per dialect, for the same reason addBigIntColumn and
renameColumn already are. MySQL and SQL Server name the table and have
no IF EXISTS for it; PostgreSQL and SQLite name the index alone.

Verified by running the shipped migration against PostgreSQL 15 and
MySQL 8.0 in containers, and SQLite through this package's tests. The
SQL Server form is from its documentation and has not been run - there
is no SQL Server here to run it against, and saying so is better than
implying four dialects were checked.
2026-09-08 16:57:15 +08:00
zhangwenjian d6e2c02fda fix🐛: fail readiness while the database is behind the migrations
The process started, both probes passed, and the first sign that the
schema did not match was a login failing with a driver-level encoding
error - go-admin#919, where an operator upgraded the binary and
restarted the API without running migrate. Nothing between those two
events had an opinion about the schema.

Readiness is where this belongs. Liveness asks "restart me", and a
process whose database is on the wrong schema comes back to the same
schema. Readiness asks "send me requests", and the answer is no. A
rolling update then stalls at the deploy - new instances never become
ready, the old ones keep serving - rather than at somebody's login, and
running migrate clears it without a restart because the check is
evaluated per request.

Any tenant database being behind fails the check, not only the one being
served: migrations are applied to every database in one run, so one
behind means that run did not finish, and serving the rest would let a
half-applied deploy look like a partial success.

A missing sys_migration table is nothing applied rather than an error.
That is a first deploy, where every migration is pending and the
operator can act on being told so.

The one test that matters is the one that cannot be written normally.
The registry is filled by init() in packages cmd/api does not import, so
a test that imported them to look at it would pass whatever the real
binary links - and a binary that links none of them gives a check that
reports every database current, forever, with every other test here
still green. TestTheServingBinaryLinksTheMigrationRegistry asks the
build instead, with a negative control so that a query matching
everything fails rather than passes.

Closes #920.
2026-09-08 15:42:14 +08:00
zhangwenjian e98b65cf90 feat✨: let the host add a readiness check this package cannot make
Whether the schema matches what the binary expects is answered by the
migration registry, which lives under cmd/. common/ has never imported
cmd/, and starting with this would put the shared layer behind the
command layer for one check.

Register instead, from where both are already in scope. A duplicate name
panics rather than appending: two checks under one name make the failing
one impossible to identify from the response body, and registering the
same one twice is a wiring mistake better heard at start-up than never.

The registered check is run through the same guard as the built-in ones,
so one that panics fails its check rather than taking down the probe
that asked.
2026-09-08 15:42:14 +08:00
zhangwenjian bc5411c30c feat✨: report what the migration registry holds without a database
Status answers what is registered, what is applied, and what is applied
while nothing registers it - and needs a database to do it. A readiness
check needs only the first half, and it already holds the databases it
is asking about.

Without this it would have to call SetDb to reuse Status, writing this
package's shared state from a request path, for a question that does not
depend on any database at all.
2026-09-08 15:42:11 +08:00
wenjianzhang 27f23121f0 Merge pull request #918 from go-admin-team/fix/911-queue-shutdown
Drain the queue on the way out instead of abandoning it
2026-09-07 17:39:15 +08:00
22 changed files with 2305 additions and 13 deletions
+19
View File
@@ -33,8 +33,27 @@ jobs:
--health-timeout 3s
--health-retries 10
postgres:
image: postgres:15-alpine
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: goadmin_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 3s
--health-retries 10
env:
GO_ADMIN_TEST_REDIS_ADDR: 127.0.0.1:6379
# The soft-delete conversion drops an index, and gorm's PostgreSQL driver
# produced unparseable SQL for that - on SQLite, where the rest of these
# tests run, the same code works. The suite reported success for a
# migration that failed on every PostgreSQL database it was pointed at.
# See go-admin#919.
GO_ADMIN_TEST_POSTGRES_DSN: "host=127.0.0.1 port=5432 user=postgres password=postgres dbname=goadmin_test sslmode=disable"
steps:
+89
View File
@@ -0,0 +1,89 @@
package models
import (
"time"
"go-admin/common/models"
)
// SysApp is the sys_app row model: one row per installed application (PRD
// 008 F2). It deliberately does not embed models.ModelTime - see the design
// doc (docs-prd/008-应用清单与安装器/数据库变更.md) §1.1 for why an
// installed-app registry does not need the millisecond soft-delete marker
// every other sys_* table follows. Uninstalling an app deletes its row
// outright; a later reinstall creates a fresh one.
type SysApp struct {
models.Model // Id int, primary key, autoincrement
// AppCode is the app.Manifest.Code / migration.ForApp / seed.SeedMenus
// identity, already lower-cased by migration.NormalizeAppCode before
// anything reaches this table. Unique: row existence alone answers G2
// ("is app X installed").
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;uniqueIndex:uk_sys_app_app_code;comment:app code"`
Name string `json:"name" gorm:"size:128;not null;comment:display name, from Manifest.Name"`
// Version is the version this row currently reflects - attempted or
// confirmed, disambiguated by Status. It does not drive which
// migrations run next; sys_migration's per-version rows do that (see
// design doc §1.5's resume flow). This field is descriptive, refreshed
// from the manifest on every install/upgrade/resume attempt.
Version string `json:"version" gorm:"size:32;not null;comment:version this row currently reflects, see Status"`
Description string `json:"description" gorm:"size:255;not null;default:'';comment:from Manifest.Description"`
Author string `json:"author" gorm:"size:128;not null;default:'';comment:from Manifest.Author"`
// Requires is a comma-separated list of app codes this app declared as
// dependencies (Manifest.Requires). Stored as plain VARCHAR CSV, not
// JSON - see design doc §1.3 for why. F8 (P1) is what validates and
// orders on this; this batch only stores what the manifest declared.
Requires string `json:"requires" gorm:"size:255;not null;default:'';comment:declared dependency app codes, comma separated"`
// Pricing/License are reserved passthrough fields (PRD 003; PRD 008
// open question 1). This batch stores whatever the manifest carries and
// does not interpret either one.
Pricing string `json:"pricing" gorm:"size:64;not null;default:'';comment:reserved, not interpreted by this batch"`
License string `json:"license" gorm:"size:64;not null;default:'';comment:reserved, not interpreted by this batch"`
// Status: 1=installing 2=installed 3=failed. Three states, not a
// single "1=installed", because a partial, stuck install has to be an
// observable row rather than "the row doesn't exist yet" - see design
// doc §1.5 for why cross-migration-file atomicity is not available on
// MySQL (implicit commit on DDL).
Status int `json:"status" gorm:"size:4;not null;default:1;comment:1=installing 2=installed 3=failed"`
// FailedVersion and LastError are DIAGNOSTIC TEXT ONLY - what a human
// looking at this row is told about the last failure, nothing more. No
// code anywhere may read either one to decide what to do next.
//
// The question "where should a resume pick up" has exactly one
// authoritative answer, and it is not these two columns: subtract
// sys_migration's applied rows for this app_code from what the app's
// own compiled-in code has registered (migration.Snapshot()/ForApp -
// the same set F7's `migrate status` already walks). That answer can
// never go stale, because it is not stored anywhere to go stale - it is
// recomputed from sys_migration every time it is asked. FailedVersion
// is a snapshot of what that computation returned at the moment of
// failure, kept only so an operator does not have to go find the
// process's logs; if it and a fresh recomputation from sys_migration
// ever disagree, sys_migration is right and this column is stale, by
// definition, and nothing should ever notice or care except a human
// reading the row.
FailedVersion string `json:"failedVersion" gorm:"size:64;not null;default:'';comment:diagnostic snapshot only, not a judgment basis; meaningful only when status=3"`
LastError string `json:"lastError" gorm:"size:255;not null;default:'';comment:diagnostic text only, not a judgment basis; meaningful only when status=3"`
// InstalledAt is when this app first reached status=installed - set
// once, never moved by a later upgrade (see design doc §1.4). Nullable,
// unlike every other column here: a row can exist before it has a
// value (a fresh install starts at status=installing). This is not the
// deleted_at problem 1786700003000_soft_delete_marker.go fixed - that
// column sat inside a unique index, where NULL <> NULL let two live
// rows coexist under the same key. InstalledAt is in no index at all,
// so nullability here opens no such hole.
InstalledAt *time.Time `json:"installedAt" gorm:"comment:first successful install time; null until status first reaches installed"`
UpdatedAt time.Time `json:"updatedAt" gorm:"comment:last updated time"`
models.ControlBy // CreateBy/UpdateBy: which operator triggered the attempt
}
func (*SysApp) TableName() string {
return "sys_app"
}
+43
View File
@@ -0,0 +1,43 @@
package models
import "time"
// SysAppCasbinGrant is a ledger of casbin_rule rows an app install created,
// keyed by the exact natural key casbin_rule itself is unique on. It exists
// because casbin_rule is not a table this project owns (see design doc
// docs-prd/008-应用清单与安装器/数据库变更.md §2.2): we cannot add an
// app_code column to it without that column being silently zeroed the first
// time anything calls the gorm-adapter's SavePolicy/SavePolicyCtx. Recording
// the natural key here, instead of a foreign key into casbin_rule, is also
// what survives SysRole.Update's RemoveFilteredPolicy+re-add cycle for a
// role's policies (app/admin/service/sys_role.go): that cycle replaces the
// underlying row (a new auto-increment ID) but reproduces the same
// (ptype,v0,v1,v2) tuple from the same sys_menu/sys_api data, so a
// natural-key match here still finds it. What it does not survive is the
// role being renamed, or the tuple being rebuilt from a completely different
// source (a future SavePolicy call from outside this seeder) - in both cases
// the match legitimately fails, and business rule 3 says the uninstaller
// should report and skip, not delete something else that happens to look
// the same.
type SysAppCasbinGrant struct {
Id int `json:"id" gorm:"primaryKey;autoIncrement"`
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;index:idx_sys_app_casbin_grant_app_code;comment:app code that created this grant"`
// Column widths mirror gorm-adapter's own CasbinRule struct exactly, so
// a value that fits into casbin_rule always fits here, and the unique
// index below matches the one createTable() puts on casbin_rule itself.
Ptype string `json:"ptype" gorm:"size:100;not null;uniqueIndex:uk_sys_app_casbin_grant_rule;comment:casbin ptype, 'p' today"`
V0 string `json:"v0" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:role_key at grant time"`
V1 string `json:"v1" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:api path"`
V2 string `json:"v2" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:http method"`
V3 string `json:"v3" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
V4 string `json:"v4" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
V5 string `json:"v5" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
CreatedAt time.Time `json:"createdAt" gorm:"comment:when this grant was recorded"`
}
func (*SysAppCasbinGrant) TableName() string {
return "sys_app_casbin_grant"
}
+13
View File
@@ -32,6 +32,19 @@ type SysMenu struct {
// AutoMigrate adding this column to an existing table leaves every
// pre-existing row reading back as "" rather than NULL.
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;default:'';index:idx_sys_menu_app_code;comment:AppCode"`
// SeedCode is the raw seed.MenuSpec.Code this row was created from, kept
// so seedMenuTree can ask "did I already write this node" without
// relying on MenuName's PascalCase concatenation, which is not
// injective (see design doc §1.6). Nullable, unlike AppCode: every row
// seed.SeedMenus writes sets a real value, but every pre-existing row -
// the host's own hand-placed menus, and every app-seeded row written
// before this column existed - has none, and there is no way to
// backfill one that means anything. NULL is what lets an unbounded
// number of those coexist under the same app_code without tripping the
// unique index below: the database never treats two NULLs as equal, so
// only rows that do carry a real code participate in the uniqueness
// check at all.
SeedCode *string `json:"seedCode" gorm:"size:64;uniqueIndex:uk_sys_menu_app_seed_code_del;comment:raw MenuSpec.Code, null for rows not written through SeedMenus"`
models.ControlBy
models.ModelTime
}
+152 -8
View File
@@ -90,6 +90,23 @@ func (adminSeeder) SeedMenus(tx *gorm.DB, appCode string, menus []seed.MenuSpec,
// application's ids in the module cache. Never accepting a caller-chosen id
// here removes the collision this Seeder has no way to detect instead of
// trying to detect it after the fact.
//
// The natural key is (app_code, path, action) - the same three columns
// 1786700002000_remove_refresh_token_api.go already used to identify a
// single API by hand, and the ones 1786700008000_seed_natural_keys.go put a
// unique index on. Before inserting, this looks for a live row (deleted_at
// = 0, applied automatically by the soft-delete plugin on every query
// against models.SysApi) already holding that key and reuses it instead of
// inserting a second one - see the design doc §1.6: a migration retried
// after a partial failure previously re-ran this as a bare tx.Create and
// produced duplicate rows on the demo site.
//
// Unlike seedMenuTree's reuse branch, this one has nothing left to repair
// after finding an existing row: models.SysApi carries no association
// (nothing like SysMenu's many2many SysApi field) and this function writes
// nothing beyond the row itself - no second statement comparable to
// seedMenuTree's paths UPDATE follows tx.Create below. An interrupted retry
// can therefore only ever find this row complete or not find it at all.
func seedApis(tx *gorm.DB, appCode string, apis []seed.ApiSpec) (map[string]models.SysApi, error) {
seen := make(map[string]bool, len(apis))
rows := make(map[string]models.SysApi, len(apis))
@@ -102,6 +119,19 @@ func seedApis(tx *gorm.DB, appCode string, apis []seed.ApiSpec) (map[string]mode
}
seen[a.Code] = true
var existing models.SysApi
err := tx.Where("app_code = ? AND path = ? AND action = ?", appCode, a.Path, a.Method).
First(&existing).Error
switch {
case err == nil:
rows[a.Code] = existing
continue
case errors.Is(err, gorm.ErrRecordNotFound):
// Not seen yet; fall through to insert it.
default:
return nil, fmt.Errorf("api %q: checking for an existing row: %w", a.Code, err)
}
row := models.SysApi{
Handle: a.Handle,
Title: a.Title,
@@ -153,6 +183,12 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
continue
}
// Resolved before the idempotency check below, whether or not
// this spec's own row turns out to already exist: repairing an
// existing-but-incomplete row's paths needs the parent's
// already-resolved Paths exactly as much as creating a fresh
// row does (see repairExistingMenu), so both have to wait for
// it the same way.
var parentRow models.SysMenu
if s.Parent != "" {
parent, ok := created[s.Parent]
@@ -165,6 +201,32 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
parentRow = parent
}
// Idempotency check: does this node already have a row, from
// an earlier, possibly-interrupted attempt? The natural key is
// (app_code, seed_code) - menu_name's PascalCase concatenation
// is not injective and cannot be used for this (see menuName's
// doc comment and the design doc §1.6). Only a live row counts;
// the soft-delete plugin scopes deleted_at = 0 automatically on
// every query against models.SysMenu.
var existing models.SysMenu
err := tx.Where("app_code = ? AND seed_code = ?", appCode, s.Code).First(&existing).Error
switch {
case err == nil:
row, err := repairExistingMenu(tx, existing, s, parentRow, apiRows)
if err != nil {
return nil, fmt.Errorf("%q: repairing an existing row: %w", s.Code, err)
}
created[s.Code] = row
ids = append(ids, row.MenuId)
progressed = true
continue
case errors.Is(err, gorm.ErrRecordNotFound):
// Not written yet; fall through to create it below.
default:
return nil, fmt.Errorf("%q: checking for an existing row: %w", s.Code, err)
}
seedCode := s.Code
row := models.SysMenu{
MenuName: menuName(appCode, s.Code),
Title: s.Title,
@@ -179,9 +241,10 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
// 1786700001000_demo_menu.go seeds its own menu with. A
// freshly installed application's menu should not need an
// administrator to first find and unhide it.
Visible: "0",
IsFrame: "1",
AppCode: appCode,
Visible: "0",
IsFrame: "1",
AppCode: appCode,
SeedCode: &seedCode,
}
for _, code := range s.ApiCodes {
api, ok := apiRows[code]
@@ -205,11 +268,7 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
// two-step create-then-update 1786700001000_demo_menu.go's
// hand-assigned ids let it do in one literal, sequenced here
// instead.
if s.Parent == "" {
row.Paths = "/0/" + strconv.Itoa(row.MenuId)
} else {
row.Paths = parentRow.Paths + "/" + strconv.Itoa(row.MenuId)
}
row.Paths = expectedPaths(row.MenuId, s.Parent, parentRow)
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", row.MenuId).
Update("paths", row.Paths).Error; err != nil {
return nil, fmt.Errorf("%q: writing paths: %w", s.Code, err)
@@ -226,6 +285,91 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
return ids, nil
}
// expectedPaths is the materialized path a fresh insert of menuID under
// parent (or at the root, if parent is "") computes - factored out so
// repairExistingMenu can ask the same question about a row it did not just
// create.
func expectedPaths(menuID int, parent string, parentRow models.SysMenu) string {
if parent == "" {
return "/0/" + strconv.Itoa(menuID)
}
return parentRow.Paths + "/" + strconv.Itoa(menuID)
}
// repairExistingMenu brings a row seedMenuTree's idempotency check found up
// to what a fresh insert of the same spec would have produced.
//
// A row can be found and still be incomplete: tx.Create's own association
// write (the sys_menu_api_rule bindings from row.SysApi) and the paths
// UPDATE that follows it are each their own statement, and design doc §1.5
// establishes that nothing after the first DDL in a migration function can
// be rolled back together - a process interrupted between the row insert
// and either of those two steps leaves exactly this row: present, findable
// by its natural key, but missing what makes it a working menu entry. A
// retry that only checked "does the row exist" and stopped there would
// report success while the sys_menu_api_rule binding stays missing (the
// api is granted to no one) or paths stays empty (a materialized-path
// break that orphans the rest of the subtree from the root) - as silent as
// the duplicate-row defect the idempotency check itself was written to
// close.
//
// Both checks are read-before-write, so a row that is already complete -
// the ordinary case on every retry after the first successful one - causes
// no writes at all: existing.Paths already equals what expectedPaths
// computes, and the sys_menu_api_rule INSERT is itself guarded by
// WHERE NOT EXISTS, the same idempotent-insert shape grantToAdminRole
// already uses for sys_role_menu/casbin_rule. Never DELETEs an existing
// binding to rebuild it - that is the FullSaveAssociations mistake
// sys_role.go's SysRole.Update makes for sys_role_menu/casbin_rule
// (app/admin/service/sys_role.go:148-153), the exact pattern this design
// went out of its way to avoid for the tables that do use it.
//
// Insert-only cuts both ways, deliberately. A binding an administrator
// added by hand through the menu management UI, for an api never in
// s.ApiCodes at all, is never touched by this loop and survives every
// later retry (TestSeedMenusPreservesAHandAddedBinding is the reproduction
// case for the opposite mistake: delete-then-reinsert wipes it silently,
// the same shape as sys_role_menu/casbin_rule getting zeroed by a role
// edit, just with this code as the actor instead of the victim). The
// converse case - a MenuSpec that used to list an ApiCode and no longer
// does - is not handled here either, and that half is intentional rather
// than an oversight: this loop only ever adds rows for codes the *current*
// call's ApiCodes names, so a binding for a code an earlier version
// granted and the current one dropped is left in place, stale. Reconciling
// that is deleting something, which needs the same certainty about
// ownership uninstall's design (see design doc §5) already requires -
// this function has no way to tell "stale, from an older version of this
// same app" apart from "hand-added, for a reason", and business rule 3
// ("uninstall deletes only what it can attribute with certainty") applies
// here just as much as it does there. Reconciling stale seed-driven
// bindings, if it is ever wanted, belongs in the upgrade path with that
// same ownership check - not silently inside every retry of every install.
func repairExistingMenu(tx *gorm.DB, existing models.SysMenu, s seed.MenuSpec, parentRow models.SysMenu, apiRows map[string]models.SysApi) (models.SysMenu, error) {
want := expectedPaths(existing.MenuId, s.Parent, parentRow)
if existing.Paths != want {
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", existing.MenuId).
Update("paths", want).Error; err != nil {
return models.SysMenu{}, fmt.Errorf("repairing paths: %w", err)
}
existing.Paths = want
}
for _, code := range s.ApiCodes {
api, ok := apiRows[code]
if !ok {
return models.SysMenu{}, fmt.Errorf("ApiCodes references %q, which is not an ApiSpec.Code in this call", code)
}
if err := tx.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) SELECT ?, ? WHERE NOT EXISTS (SELECT 1 FROM sys_menu_api_rule WHERE sys_menu_menu_id = ? AND sys_api_id = ?)",
existing.MenuId, api.Id, existing.MenuId, api.Id,
).Error; err != nil {
return models.SysMenu{}, fmt.Errorf("binding %q: %w", code, err)
}
}
return existing, nil
}
// validateMenuSpec rejects the malformed input tools/checksilent's
// menu-sort-overflow and Kind-adjacent checks would catch for an in-tree
// seed but cannot for a third-party application's - see menuSortRange's doc
+552
View File
@@ -1,13 +1,17 @@
package service
import (
"context"
"errors"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
gormlogger "gorm.io/gorm/logger"
contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/seed"
@@ -289,3 +293,551 @@ func TestSeedMenusWithNothingRegisteredWritesNothing(t *testing.T) {
}
}
}
// newSeedTestDB's AutoMigrate builds a unique index on seed_code alone,
// because SysMenu.SeedCode is the only field in the struct carrying the
// uk_sys_menu_app_seed_code_del tag - app_code already carries a different,
// non-unique index name of its own, and the embedded ModelTime's
// DeletedAt (aliased from go-admin-core) cannot be given a third one. The
// real migration (cmd/migrate/migration/version/1786700008000_seed_natural_keys.go)
// never lets AutoMigrate touch this table for exactly that reason: it
// builds the composite (app_code, seed_code, deleted_at) index by hand
// instead. Reproduce that by hand here too, so a test that seeds two rows
// sharing a seed_code under different deleted_at values sees what a real
// install would, not gorm's narrower default.
func useCompositeSeedCodeIndex(t *testing.T, db *gorm.DB) {
t.Helper()
if db.Migrator().HasIndex(&models.SysMenu{}, "uk_sys_menu_app_seed_code_del") {
if err := db.Migrator().DropIndex(&models.SysMenu{}, "uk_sys_menu_app_seed_code_del"); err != nil {
t.Fatalf("drop the single-column seed_code index: %v", err)
}
}
if err := db.Exec(
"CREATE UNIQUE INDEX uk_sys_menu_app_seed_code_del ON sys_menu (app_code, seed_code, deleted_at)",
).Error; err != nil {
t.Fatalf("create the composite seed_code index: %v", err)
}
}
// A retried migration - one that failed partway through and is run again,
// or simply run twice by mistake - must not create a second sys_api or
// sys_menu row for the same (appCode, natural key). This is the defect the
// demo site hit in production: duplicate sys_menu/casbin_rule rows from a
// bare tx.Create on a natural key nothing was checking.
func TestSeedMenusIsIdempotentAcrossARetry(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{
{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10},
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: "Orders", Sort: 1, ApiCodes: []string{"list"}},
}
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"},
}
run := func() {
t.Helper()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
}
run()
firstMenuIDs := allMenuIDs(t, db, "order")
firstApiIDs := allApiIDs(t, db, "order")
run() // the retry
if got := allMenuIDs(t, db, "order"); !sameIDs(got, firstMenuIDs) {
t.Errorf("sys_menu ids after retry = %v, want unchanged %v (a second call inserted new rows)", got, firstMenuIDs)
}
if got := allApiIDs(t, db, "order"); !sameIDs(got, firstApiIDs) {
t.Errorf("sys_api ids after retry = %v, want unchanged %v (a second call inserted new rows)", got, firstApiIDs)
}
assertRowCount(t, db, "sys_api", 1)
assertRowCount(t, db, "sys_menu", 2)
assertRowCount(t, db, "sys_menu_api_rule", 1)
assertRowCount(t, db, "sys_role_menu", 2)
assertRowCount(t, db, "casbin_rule", 1)
}
// Only a live row counts as "already written". A row a prior, unrelated
// soft-delete already retired must not be reused - seedApis/seedMenuTree
// have to insert a fresh one under the same natural key, the same way the
// unique indexes 1786700008000_seed_natural_keys.go builds only bind live
// rows.
func TestSeedMenusOnlyReusesLiveRows(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10}}
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
// Soft-delete both rows this first call wrote, as if an operator (or an
// earlier uninstall) had retired them, independently of this migration
// ever running again.
if err := db.Exec("UPDATE sys_menu SET deleted_at = 1").Error; err != nil {
t.Fatalf("soft-delete sys_menu: %v", err)
}
if err := db.Exec("UPDATE sys_api SET deleted_at = 1").Error; err != nil {
t.Fatalf("soft-delete sys_api: %v", err)
}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus after soft-delete: %v", err)
}
// Two rows total: the soft-deleted original, plus a fresh one - not the
// dead row resurrected in place, and not left with zero live rows.
assertRowCount(t, db, "sys_menu", 2)
assertRowCount(t, db, "sys_api", 2)
var liveMenus, liveApis int64
db.Model(&models.SysMenu{}).Where("app_code = ?", "order").Count(&liveMenus)
db.Model(&models.SysApi{}).Where("app_code = ?", "order").Count(&liveApis)
if liveMenus != 1 {
t.Errorf("live sys_menu rows = %d, want 1", liveMenus)
}
if liveApis != 1 {
t.Errorf("live sys_api rows = %d, want 1", liveApis)
}
}
// app_code is part of the natural key, not a descriptive column alongside
// it. Two applications that happen to register an identical (path, action)
// or seed_code must each get their own row - reusing one app's row for
// another's install would make an uninstall of the first delete a row the
// second considers its own.
func TestSeedMenusScopesTheNaturalKeyByAppCode(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{{Code: "dir", Kind: contractmodels.Directory, Title: "Dir", Sort: 10}}
apis := []seed.ApiSpec{{Code: "list", Title: "Shared endpoint", Path: "/api/v1/shared", Method: "GET"}}
for _, appCode := range []string{"order", "billing"} {
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, appCode, menus, apis)
}); err != nil {
t.Fatalf("SeedMenus(%q): %v", appCode, err)
}
}
var apiRows []models.SysApi
if err := db.Where("path = ? AND action = ?", "/api/v1/shared", "GET").
Order("app_code").Find(&apiRows).Error; err != nil {
t.Fatalf("read sys_api: %v", err)
}
if len(apiRows) != 2 {
t.Fatalf("sys_api has %d row(s) for the shared (path, action), want 2 - one per app", len(apiRows))
}
if apiRows[0].AppCode != "billing" || apiRows[1].AppCode != "order" {
t.Errorf("sys_api app_codes = [%s %s], want [billing order]", apiRows[0].AppCode, apiRows[1].AppCode)
}
var menuRows []models.SysMenu
if err := db.Where("seed_code = ?", "dir").Order("app_code").Find(&menuRows).Error; err != nil {
t.Fatalf("read sys_menu: %v", err)
}
if len(menuRows) != 2 {
t.Fatalf("sys_menu has %d row(s) for the shared seed_code, want 2 - one per app", len(menuRows))
}
if menuRows[0].AppCode != "billing" || menuRows[1].AppCode != "order" {
t.Errorf("sys_menu app_codes = [%s %s], want [billing order]", menuRows[0].AppCode, menuRows[1].AppCode)
}
}
func allMenuIDs(t *testing.T, db *gorm.DB, appCode string) []int {
t.Helper()
var rows []models.SysMenu
if err := db.Where("app_code = ?", appCode).Order("menu_id").Find(&rows).Error; err != nil {
t.Fatalf("read sys_menu: %v", err)
}
ids := make([]int, len(rows))
for i, r := range rows {
ids[i] = r.MenuId
}
return ids
}
func allApiIDs(t *testing.T, db *gorm.DB, appCode string) []int {
t.Helper()
var rows []models.SysApi
if err := db.Where("app_code = ?", appCode).Order("id").Find(&rows).Error; err != nil {
t.Fatalf("read sys_api: %v", err)
}
ids := make([]int, len(rows))
for i, r := range rows {
ids[i] = r.Id
}
return ids
}
func sameIDs(a, b []int) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
func assertRowCount(t *testing.T, db *gorm.DB, table string, want int64) {
t.Helper()
var n int64
if err := db.Table(table).Count(&n).Error; err != nil {
t.Fatalf("count %s: %v", table, err)
}
if n != want {
t.Errorf("%s has %d row(s), want %d", table, n, want)
}
}
// A retried migration does not just risk inserting a second copy of a row
// it already wrote (that gap is closed above) - the reuse path itself has
// to leave the row in the same state a fresh insert would have. Before
// this defect was fixed, the reuse branch (seed.go's "case err == nil")
// stopped at reusing the row's id and skipped everything a fresh insert
// does afterwards: the sys_menu_api_rule binding gorm's association save
// writes as part of Create, and the paths UPDATE that follows Create as a
// separate statement. A row a prior attempt inserted but did not finish -
// exactly the shape design doc §1.5 says a non-transactional retry can
// leave behind - would be "found" and then left broken forever, with the
// migration reporting success.
//
// existingHalfWrittenMenu inserts a sys_menu row the way seedMenuTree's own
// tx.Create leaves one when interrupted immediately afterwards: the row
// exists with its natural key, but paths was never computed and no
// sys_menu_api_rule binding was ever written for it - Create's association
// save and the paths UPDATE are each a separate statement from the row
// insert itself.
func existingHalfWrittenMenu(t *testing.T, db *gorm.DB, appCode, seedCode string, parentID int) models.SysMenu {
t.Helper()
code := seedCode
row := models.SysMenu{
MenuName: menuName(appCode, seedCode),
AppCode: appCode,
SeedCode: &code,
ParentId: parentID,
Visible: "0",
IsFrame: "1",
// Paths deliberately left "" - never computed, the same as a row
// whose Create succeeded but whose follow-up paths UPDATE never ran.
}
if err := db.Create(&row).Error; err != nil {
t.Fatalf("seed half-written menu %q: %v", seedCode, err)
}
return row
}
func bindingCount(t *testing.T, db *gorm.DB, menuID, apiID int) int64 {
t.Helper()
var n int64
if err := db.Table("sys_menu_api_rule").
Where("sys_menu_menu_id = ? AND sys_api_id = ?", menuID, apiID).Count(&n).Error; err != nil {
t.Fatalf("count sys_menu_api_rule: %v", err)
}
return n
}
// TestSeedMenusRepairsAnIncompleteExistingRow is the reproduction case:
// both paths and the api binding are missing on the row seedMenuTree finds
// through its idempotency check, the shape a real interrupted retry leaves
// behind. Run against the unfixed reuse branch, this must fail - that is
// what proves the defect is real rather than a three-way guess.
func TestSeedMenusRepairsAnIncompleteExistingRow(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
dir := existingHalfWrittenMenu(t, db, "order", "dir", 0)
list := existingHalfWrittenMenu(t, db, "order", "list", dir.MenuId)
menus := []seed.MenuSpec{
{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10},
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: "Orders", Sort: 1, ApiCodes: []string{"list"}},
}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
wantDirPaths := "/0/" + strconv.Itoa(dir.MenuId)
wantListPaths := wantDirPaths + "/" + strconv.Itoa(list.MenuId)
var gotDir, gotList models.SysMenu
if err := db.First(&gotDir, dir.MenuId).Error; err != nil {
t.Fatalf("read dir: %v", err)
}
if err := db.First(&gotList, list.MenuId).Error; err != nil {
t.Fatalf("read list: %v", err)
}
if gotDir.Paths != wantDirPaths {
t.Errorf("dir.Paths = %q, want %q - a retried install left a root menu with no materialized path", gotDir.Paths, wantDirPaths)
}
if gotList.Paths != wantListPaths {
t.Errorf("list.Paths = %q, want %q - a retried install left the seeded subtree with a broken materialized path", gotList.Paths, wantListPaths)
}
if n := bindingCount(t, db, list.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("sys_menu_api_rule binding count for list = %d, want 1 - a retried install left the menu with its api granted to no one", n)
}
}
// soloMenuSpec is a single, parent-less menu with one api binding - the
// smallest shape that can exhibit "paths wrong" and "binding missing"
// independently of each other, used by the three tests below to isolate
// one repair path at a time from TestSeedMenusRepairsAnIncompleteExistingRow's
// combined (both broken) case.
func soloMenuSpec() []seed.MenuSpec {
return []seed.MenuSpec{{Code: "solo", Kind: contractmodels.Menu, Title: "Solo", Sort: 1, ApiCodes: []string{"list"}}}
}
// Only the binding is missing; paths is already correct. The repair must
// add the binding and must not touch the already-correct paths value.
func TestSeedMenusRepairsOnlyAMissingBinding(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
solo := existingHalfWrittenMenu(t, db, "order", "solo", 0)
wantPaths := "/0/" + strconv.Itoa(solo.MenuId)
if err := db.Model(&models.SysMenu{}).Where("menu_id = ?", solo.MenuId).
Update("paths", wantPaths).Error; err != nil {
t.Fatalf("set paths: %v", err)
}
// The binding is deliberately left unwritten.
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", soloMenuSpec(), apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
var got models.SysMenu
if err := db.First(&got, solo.MenuId).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
if got.Paths != wantPaths {
t.Errorf("paths changed from %q to %q; repairing a missing binding must not touch an already-correct path", wantPaths, got.Paths)
}
if n := bindingCount(t, db, solo.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("binding count = %d, want 1", n)
}
}
// Only paths is missing; the binding already exists (as if Create's own
// association write had succeeded but the paths UPDATE that follows it
// never ran). The repair must fix paths and must not duplicate the
// already-correct binding.
func TestSeedMenusRepairsOnlyMissingPaths(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
solo := existingHalfWrittenMenu(t, db, "order", "solo", 0)
if err := db.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) VALUES (?, ?)",
solo.MenuId, apiRows["list"].Id,
).Error; err != nil {
t.Fatalf("seed binding: %v", err)
}
// solo.Paths is deliberately left "" by existingHalfWrittenMenu.
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", soloMenuSpec(), apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
wantPaths := "/0/" + strconv.Itoa(solo.MenuId)
var got models.SysMenu
if err := db.First(&got, solo.MenuId).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
if got.Paths != wantPaths {
t.Errorf("paths = %q, want %q", got.Paths, wantPaths)
}
if n := bindingCount(t, db, solo.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("binding count = %d, want 1 - repairing paths must not duplicate an already-correct binding", n)
}
}
// capturingLogger records every SQL statement gorm actually executes, so a
// test can assert that a fully-consistent retry performs no write at all -
// not just that its net effect happens to be zero rows changed. Mirrors
// common/actions/crud_shim_test.go's logger of the same name and shape;
// duplicated locally rather than exported and shared, matching how small
// gorm-facing test doubles are kept next to the test that needs them
// elsewhere in this repository.
type capturingLogger struct {
gormlogger.Interface
mu sync.Mutex
stmts []string
}
func (l *capturingLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
sql, _ := fc()
l.mu.Lock()
l.stmts = append(l.stmts, sql)
l.mu.Unlock()
}
func (l *capturingLogger) all() string {
l.mu.Lock()
defer l.mu.Unlock()
return strings.Join(l.stmts, "\n")
}
// Both paths and the binding are already correct - the ordinary shape of
// every retry after the first one succeeds in full. Repairing an
// already-consistent row must not touch it: paths is read-before-write and
// so must not be UPDATEd at all (asserted directly, by statement, since the
// code gates that call behind a value comparison); the binding's own
// insert is guarded by WHERE NOT EXISTS the same way grantToAdminRole's
// already are, so its row count staying put is the meaningful claim - the
// guarded statement itself may still be sent, the same way it already is
// for sys_role_menu/casbin_rule.
func TestSeedMenusFullyConsistentRowCausesNoPathsUpdate(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
menus := soloMenuSpec()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (first): %v", err)
}
var apiRows []models.SysApi
db.Where("app_code = ?", "order").Find(&apiRows)
var soloRow models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "solo").First(&soloRow).Error; err != nil {
t.Fatalf("read solo after first call: %v", err)
}
if soloRow.Paths == "" {
t.Fatalf("solo.Paths is empty after the first call; the fixture itself is broken, not what this test means to check")
}
wantBindings := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id)
if wantBindings != 1 {
t.Fatalf("binding count after the first call = %d, want 1; the fixture itself is broken", wantBindings)
}
capturing := &capturingLogger{Interface: gormlogger.Default.LogMode(gormlogger.Info)}
captured := db.Session(&gorm.Session{Logger: capturing})
if err := captured.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (retry): %v", err)
}
all := strings.ToUpper(capturing.all())
if strings.Contains(all, "UPDATE") && strings.Contains(all, "SYS_MENU") && strings.Contains(all, "PATHS") {
t.Errorf("a fully consistent retry executed a paths UPDATE against sys_menu:\n%s", capturing.all())
}
if got := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id); got != 1 {
t.Errorf("binding count after the retry = %d, want 1 (unchanged)", got)
}
}
// An administrator can bind a menu to an additional api by hand through
// the menu management UI - a sys_menu_api_rule row for an api never in
// s.ApiCodes at all. A retried SeedMenus call must not touch it: deleting
// every binding for the menu and reinserting only what s.ApiCodes lists
// would wipe it out silently, the same shape as sys_role_menu/casbin_rule
// getting zeroed by SysRole.Update's FullSaveAssociations save - just with
// this code as the actor instead of the victim this time.
func TestSeedMenusPreservesAHandAddedBinding(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
menus := soloMenuSpec()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (first): %v", err)
}
var soloRow models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "solo").First(&soloRow).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
// An api this call's ApiSpec list never mentions - standing in for one
// belonging to some other feature entirely, bound to this menu by an
// administrator, not by any SeedMenus call.
handAdded := models.SysApi{Path: "/api/v1/order/export", Action: "GET", Type: "SYS", AppCode: "order"}
if err := db.Create(&handAdded).Error; err != nil {
t.Fatalf("seed the hand-added api: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) VALUES (?, ?)",
soloRow.MenuId, handAdded.Id,
).Error; err != nil {
t.Fatalf("seed the hand-added binding: %v", err)
}
// A retry with the exact same specs - solo's ApiCodes still names only
// "list".
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (retry): %v", err)
}
if n := bindingCount(t, db, soloRow.MenuId, handAdded.Id); n != 1 {
t.Errorf("hand-added binding count = %d, want 1 - a retry silently deleted a binding it does not own", n)
}
var apiRows []models.SysApi
db.Where("app_code = ? AND path = ?", "order", "/api/v1/order").Find(&apiRows)
if len(apiRows) != 1 {
t.Fatalf("seeded api not found as expected: %+v", apiRows)
}
if n := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id); n != 1 {
t.Errorf("the seed's own binding count = %d, want 1 - it must survive the retry too", n)
}
}
+117
View File
@@ -0,0 +1,117 @@
package api
import (
"context"
"fmt"
"sort"
"strings"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"gorm.io/gorm"
"go-admin/cmd/migrate/migration"
"go-admin/common/health"
commonmodels "go-admin/common/models"
)
// schemaCheckName is what a failing schema reports itself as in /ready's body.
const schemaCheckName = "schema"
// registerSchemaCheck adds the pending-migration check to readiness.
//
// Readiness rather than a refusal to start, and rather than a log line alone.
// The two probes answer different questions: liveness is "restart me", and a
// process whose database is on the wrong schema comes back to the same schema,
// so restarting is not the answer. Readiness is "send me requests", and with a
// schema the binary does not match the answer is no.
//
// Issue #919 is what the absence of this looked like: the process started,
// both probes passed, and the first sign of trouble was a login failing with a
// driver-level encoding error. Refusing to start would have been the wrong fix
// - a process that exits tells an operator less than one that runs and says
// why, and under an orchestrator it crash-loops - while a log line alone is
// not something an orchestrator can act on.
func registerSchemaCheck() {
health.Register(schemaCheckName, schemaCheck)
}
// schemaCheck fails while any tenant database is behind the migrations this
// binary registers.
//
// Any one of them, rather than only the tenant being served: migrations are
// applied to every database in one run, so one database behind means that run
// did not finish. Serving the rest would let a half-applied deploy look like a
// partial success.
//
// Evaluated per request rather than decided at start-up, so that running
// migrate clears it without a restart.
func schemaCheck(ctx context.Context) error {
registered := migration.RegisteredVersions()
if len(registered) == 0 {
// Nothing registered means nothing can be pending, which is the honest
// answer for a tree with no migrations. It is also what a broken build
// would produce - the registry is filled by init() in packages the
// binary has to link - so cmd/api's dependency test asserts the real
// binary links them.
return nil
}
behind := make([]string, 0, 2)
for name, db := range sdk.Runtime.GetAllDb() {
applied, err := appliedVersions(ctx, db)
if err != nil {
return fmt.Errorf("reading applied migrations for %q: %w", name, err)
}
if pending := pendingVersions(registered, applied); len(pending) > 0 {
behind = append(behind, fmt.Sprintf("%s is %d behind, first pending %s",
name, len(pending), pending[0]))
}
}
if len(behind) == 0 {
return nil
}
sort.Strings(behind)
return fmt.Errorf("%s; run `go-admin migrate -c <config>` and see `go-admin migrate status`",
strings.Join(behind, "; "))
}
// appliedVersions reads what sys_migration records for one database.
//
// A missing table is not an error: a database that has never been migrated has
// applied nothing, which is exactly what the caller needs to hear, and is the
// state a first deploy is in.
func appliedVersions(ctx context.Context, db *gorm.DB) (map[string]bool, error) {
if db == nil {
return nil, fmt.Errorf("no database")
}
db = db.WithContext(ctx)
if !db.Migrator().HasTable(&commonmodels.Migration{}) {
return map[string]bool{}, nil
}
var rows []commonmodels.Migration
if err := db.Select("version").Find(&rows).Error; err != nil {
return nil, err
}
out := make(map[string]bool, len(rows))
for _, r := range rows {
out[r.Version] = true
}
return out, nil
}
// pendingVersions returns the registered versions applied does not contain.
//
// Split out and taking both sides as arguments because the registry is
// process-wide and filled by init() in packages cmd/api does not import: a
// test in this package cannot arrange it, so the arranging part is the part
// that is not tested here.
func pendingVersions(registered []string, applied map[string]bool) []string {
out := make([]string, 0)
for _, v := range registered {
if !applied[v] {
out = append(out, v)
}
}
sort.Strings(out)
return out
}
+124
View File
@@ -0,0 +1,124 @@
package api
import (
"context"
"os/exec"
"strings"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
commonmodels "go-admin/common/models"
)
func TestPendingVersionsReportsOnlyWhatIsNotApplied(t *testing.T) {
registered := []string{"1000_a", "2000_b", "3000_c"}
applied := map[string]bool{"1000_a": true, "3000_c": true}
got := pendingVersions(registered, applied)
if len(got) != 1 || got[0] != "2000_b" {
t.Errorf("pending = %v, want [2000_b]", got)
}
}
func TestPendingVersionsIsEmptyWhenTheDatabaseIsCurrent(t *testing.T) {
registered := []string{"1000_a", "2000_b"}
applied := map[string]bool{"1000_a": true, "2000_b": true}
if got := pendingVersions(registered, applied); len(got) != 0 {
t.Errorf("pending = %v, want none", got)
}
}
// A row recorded that this binary no longer registers is not pending. It is
// the orphan `migrate status` already reports, and readiness has nothing to
// say about it: the schema is ahead, not behind, and requests will be served
// correctly.
func TestPendingVersionsIgnoresAppliedRowsNothingRegisters(t *testing.T) {
registered := []string{"1000_a"}
applied := map[string]bool{"1000_a": true, "9999_gone": true}
if got := pendingVersions(registered, applied); len(got) != 0 {
t.Errorf("pending = %v, want none - an orphaned row is not a pending migration", got)
}
}
func memoryDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file::memory:?cache=shared"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { db.Migrator().DropTable(&commonmodels.Migration{}) })
return db
}
// A first deploy has no sys_migration table. That is "nothing applied", not an
// error: reporting it as one would make the check fail for a reason the
// operator cannot act on, on the one deployment where every migration really
// is pending.
func TestAppliedVersionsTreatsAMissingTableAsNothingApplied(t *testing.T) {
db := memoryDB(t)
db.Migrator().DropTable(&commonmodels.Migration{})
got, err := appliedVersions(context.Background(), db)
if err != nil {
t.Fatalf("appliedVersions: %v", err)
}
if len(got) != 0 {
t.Errorf("applied = %v, want empty", got)
}
}
func TestAppliedVersionsReadsWhatTheTableHolds(t *testing.T) {
db := memoryDB(t)
if err := db.AutoMigrate(&commonmodels.Migration{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
db.Create(&commonmodels.Migration{Version: "1000_a"})
db.Create(&commonmodels.Migration{Version: "2000_b"})
got, err := appliedVersions(context.Background(), db)
if err != nil {
t.Fatalf("appliedVersions: %v", err)
}
if !got["1000_a"] || !got["2000_b"] || len(got) != 2 {
t.Errorf("applied = %v, want the two rows written", got)
}
}
// The check is only worth anything if the registry it reads is populated in
// the binary that serves requests, and it is filled by init() in packages
// cmd/api does not import - cmd/migrate blank-imports them, and cmd wires both
// subcommands into one binary.
//
// This cannot be asserted from an ordinary test: importing the version package
// to look at the registry would put it in the test binary's dependency graph
// and pass whatever the real binary links. So ask the build instead.
//
// Without this, dropping those blank imports leaves a check that reports
// "nothing pending" for every database forever, and every test above still
// passes.
func TestTheServingBinaryLinksTheMigrationRegistry(t *testing.T) {
out, err := exec.Command("go", "list", "-deps", "go-admin").Output()
if err != nil {
t.Skipf("go list unavailable: %v", err)
}
deps := string(out)
const versions = "go-admin/cmd/migrate/migration/version"
if !strings.Contains(deps, versions+"\n") {
t.Errorf("the main package does not link %s, so the schema check would "+
"read an empty registry and report every database as current", versions)
}
// Negative control: a package the binary genuinely must not link, so that a
// `deps` that somehow contained everything would fail here rather than pass
// the assertion above for the wrong reason.
const notLinked = "go-admin/tools/checksilent"
if strings.Contains(deps, notLinked+"\n") {
t.Errorf("%s is in the binary's dependency closure, so this test cannot "+
"tell a real link from a query that matches anything", notLinked)
}
}
+5
View File
@@ -82,6 +82,11 @@ func setup() {
// can call the API, which is only true once the socket is accepting.
sdk.Runtime.SetPhase(runtime.AfterListen, startCronJobs)
// Registered before the configuration is read, because it registers a
// callback rather than reading anything: the check runs per request and
// asks the databases that exist then.
registerSchemaCheck()
//1. 读取配置
bootstrap.SetupConfig(
file.NewSource(file.WithPath(configYml)),
+22
View File
@@ -185,6 +185,28 @@ func (e *Migration) mergedEntries() map[string]versionEntry {
return out
}
// RegisteredVersions returns every migration version this binary registers,
// sorted, without touching a database.
//
// Status answers a richer question - what is registered, what is applied, and
// what is applied while nothing registers it - and needs a database to do it.
// This is the half that can be asked of the process alone, which is what a
// readiness check needs: the check holds the databases it is asking about, and
// reusing Status would mean calling SetDb from a request handler, writing this
// package's shared state from a request path.
func (e *Migration) RegisteredVersions() []string {
all := e.mergedEntries()
out := make([]string, 0, len(all))
for k := range all {
out = append(out, k)
}
sort.Strings(out)
return out
}
// RegisteredVersions reports what the process-wide registry holds.
func RegisteredVersions() []string { return Migrate.RegisteredVersions() }
// StatusEntry is one row of migrate status.
type StatusEntry struct {
AppCode string
@@ -237,13 +237,53 @@ func dropIndexesOn(db *gorm.DB, table, column string) error {
if !m.HasIndex(table, name) {
continue
}
if err := m.DropIndex(table, name); err != nil {
if err := db.Exec(dropIndex(db, table, name)).Error; err != nil {
return fmt.Errorf("dropping index %s: %w", name, err)
}
}
return nil
}
// dropIndex spells DROP INDEX for one dialect, rather than going through
// Migrator().DropIndex.
//
// The migrator cannot be used here on PostgreSQL. Its driver resolves a schema
// for the statement and falls back to an expression when it cannot:
//
// currentSchema, _ := m.CurrentSchema(stmt, stmt.Table) // CURRENT_SCHEMA()
// m.DB.Exec("DROP INDEX ?.?", currentSchema, clause.Column{Name: name})
//
// DROP INDEX takes an identifier in that position, not an expression, so the
// statement does not parse. The schema is unresolvable for every call made
// here, because this passes a table name as a string rather than a model - so
// it failed on every PostgreSQL database rather than intermittently, and took
// the whole conversion with it. Reported as go-admin#919, where the visible
// symptom was a login rejecting a correct password: the migration had stopped
// here, leaving deleted_at a timestamptz that the current query compares to 0.
//
// Written per dialect for the same reason addBigIntColumn and renameColumn
// already are.
//
// MySQL and SQL Server name the table in the statement and have no IF EXISTS
// for it; PostgreSQL and SQLite name the index alone, in its own namespace.
// The caller has already checked HasIndex, so IF EXISTS is only there to make
// the two that support it say nothing rather than fail on a race with another
// migrator.
//
// Verified against PostgreSQL 15, MySQL 8.0 and SQLite. The SQL Server form is
// from its documentation and has not been run - this repository has no SQL
// Server to run it against.
func dropIndex(db *gorm.DB, table, index string) string {
switch db.Dialector.Name() {
case "mysql":
return fmt.Sprintf("DROP INDEX `%s` ON `%s`", index, table)
case "sqlserver":
return fmt.Sprintf("DROP INDEX [%s] ON [%s]", index, table)
default:
return fmt.Sprintf(`DROP INDEX IF EXISTS "%s"`, index)
}
}
// indexNamesFor asks the database which indexes cover column.
func indexNamesFor(db *gorm.DB, table, column string) ([]string, error) {
indexes, err := db.Migrator().GetIndexes(table)
@@ -0,0 +1,156 @@
package version
import (
"os"
"testing"
"time"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
// postgresDSNEnv points these tests at a database. They are skipped without
// it, so a developer with no PostgreSQL running still gets a green run.
//
// The whole file exists because the rest of this package's tests run on
// SQLite, where the defect they cover cannot happen: dropping an index through
// gorm's migrator works there and produces unparseable SQL on PostgreSQL. A
// suite that only ever exercised SQLite reported success for a migration that
// failed on every PostgreSQL database it was pointed at - go-admin#919.
const postgresDSNEnv = "GO_ADMIN_TEST_POSTGRES_DSN"
func postgresDB(t *testing.T) *gorm.DB {
t.Helper()
dsn := os.Getenv(postgresDSNEnv)
if dsn == "" {
// Skipping locally is the point; skipping in CI is the failure this
// file exists to prevent. A workflow that renamed the variable or
// dropped the service would otherwise go green while these tests
// quietly did nothing - the same shape as the defect they cover.
if os.Getenv("CI") != "" {
t.Fatalf("%s is not set while CI is: the PostgreSQL migration tests must not skip here", postgresDSNEnv)
}
t.Skipf("%s is not set; skipping the PostgreSQL migration tests", postgresDSNEnv)
}
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatalf("connecting to %s: %v", postgresDSNEnv, err)
}
return db
}
// pgOldUser is the pre-migration shape: a nullable timestamp with an index on
// it, which is what makes dropping the column require dropping the index.
type pgOldUser struct {
UserId int64 `gorm:"column:user_id;primaryKey;autoIncrement"`
Username string
DeletedAt *time.Time `gorm:"index"`
}
func (pgOldUser) TableName() string { return "sd_pg_user" }
// The conversion completes on PostgreSQL.
//
// It did not. dropIndexesOn went through Migrator().DropIndex, whose
// PostgreSQL driver falls back to an expression when it cannot resolve a
// schema - which is every call made here, because the migration passes a table
// name as a string:
//
// DROP INDEX CURRENT_SCHEMA()."idx_sd_pg_user_deleted_at"
//
// DROP INDEX takes an identifier there, so it failed to parse and took the
// whole conversion with it. Every PostgreSQL deployment stopped at this
// migration, and the visible symptom was a login rejecting a correct password
// because deleted_at was still a timestamptz being compared to 0.
func TestConversionCompletesOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&pgOldUser{}) })
db.Migrator().DropTable(&pgOldUser{})
if err := db.AutoMigrate(&pgOldUser{}); err != nil {
t.Fatalf("building the old shape: %v", err)
}
deleted := time.Now().Add(-time.Hour)
// Checked rather than fired and forgotten: a failed insert leaves the
// assertions below reading an empty table, and "no rows" is a shape some
// of them cannot tell from success.
if err := db.Create(&pgOldUser{Username: "gone", DeletedAt: &deleted}).Error; err != nil {
t.Fatalf("seeding the deleted row: %v", err)
}
if err := db.Create(&pgOldUser{Username: "live"}).Error; err != nil {
t.Fatalf("seeding the live row: %v", err)
}
if err := convertDeletedAt(db, "sd_pg_user"); err != nil {
t.Fatalf("convertDeletedAt: %v", err)
}
var dataType string
if err := db.Raw(`SELECT data_type FROM information_schema.columns
WHERE table_name = 'sd_pg_user' AND column_name = 'deleted_at'`).Scan(&dataType).Error; err != nil {
t.Fatalf("reading the column type: %v", err)
}
if dataType != "bigint" {
t.Errorf("deleted_at is %q after the conversion, want bigint", dataType)
}
// The marker has to carry the timestamp across, or a row that was deleted
// comes back live.
var markers []int64
if err := db.Raw(`SELECT deleted_at FROM sd_pg_user ORDER BY user_id`).Scan(&markers).Error; err != nil {
t.Fatalf("reading the markers: %v", err)
}
if len(markers) != 2 {
t.Fatalf("read %d rows, want 2", len(markers))
}
if markers[0] == 0 {
t.Error("the deleted row came back live")
}
if markers[1] != 0 {
t.Errorf("the live row is marked deleted at %d", markers[1])
}
}
// The index on deleted_at is gone afterwards, which is the step that failed.
//
// Asserted separately from the conversion because the conversion can succeed
// on a table with no index at all, and this migration exists for tables that
// have one.
func TestTheIndexOnDeletedAtIsDroppedOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&pgOldUser{}) })
db.Migrator().DropTable(&pgOldUser{})
if err := db.AutoMigrate(&pgOldUser{}); err != nil {
t.Fatalf("building the old shape: %v", err)
}
var before int64
if err := db.Raw(`SELECT count(*) FROM pg_indexes
WHERE tablename = 'sd_pg_user' AND indexdef LIKE '%deleted_at%'`).Scan(&before).Error; err != nil {
t.Fatalf("counting the indexes before: %v", err)
}
if before == 0 {
t.Fatal("the old shape has no index on deleted_at, so this test asserts nothing")
}
if err := dropIndexesOn(db, "sd_pg_user", "deleted_at"); err != nil {
t.Fatalf("dropIndexesOn: %v", err)
}
// This one is why the errors are checked at all rather than as a matter of
// habit: a query that fails leaves after at zero, and zero is what success
// looks like. An unchecked error here is a test that passes when it cannot
// reach the database.
var after int64
if err := db.Raw(`SELECT count(*) FROM pg_indexes
WHERE tablename = 'sd_pg_user' AND indexdef LIKE '%deleted_at%'`).Scan(&after).Error; err != nil {
t.Fatalf("counting the indexes after: %v", err)
}
if after != 0 {
t.Errorf("%d index(es) on deleted_at survived", after)
}
}
@@ -0,0 +1,42 @@
package version
import (
"runtime"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
)
// Create sys_app (PRD 008 F2) and sys_app_casbin_grant (F4/F6's casbin
// attribution ledger - see the design doc's (docs-prd/008-应用清单与安装器/
// 数据库变更.md) §2.2/§3 for why casbin_rule itself is not touched:
// gorm-adapter's SavePolicyCtx truncates and reloads that table from its
// in-memory model, and any column this migration added to it would be
// silently zeroed the first time anything calls SavePolicy.
//
// Ordered after 1786700003000 (the soft-delete conversion), so importing
// cmd/migrate/migration/models is banned here - see
// schema_coverage_test.go's TestPostConversionMigrationsAvoidFrozenSeedModels.
// Both new tables are AutoMigrate'd from their runtime model shape under
// app/admin/models directly, which is also why neither one is added to
// 1786700003000's frozen softDeleteTables list: neither embeds
// common.ModelTime in the first place (see design doc §1.1).
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700007000AppRegistryTables)
}
func _1786700007000AppRegistryTables(db *gorm.DB, version string) error {
return db.Transaction(func(tx *gorm.DB) error {
if err := tx.Migrator().AutoMigrate(
new(adminmodels.SysApp),
new(adminmodels.SysAppCasbinGrant),
); err != nil {
return err
}
return tx.Create(&common.Migration{Version: version}).Error
})
}
@@ -0,0 +1,62 @@
package version
import (
"testing"
common "go-admin/common/models"
adminmodels "go-admin/app/admin/models"
)
// postgresDB is defined in 1786700003000_soft_delete_marker_postgres_test.go
// and shared across this package's PostgreSQL-only tests.
//
// This migration is plain AutoMigrate on two brand-new tables, unlike
// 1786700003000's DROP INDEX (go-admin#919's actual defect), so there is no
// dialect-specific SQL here for AutoMigrate itself to get wrong on
// PostgreSQL specifically. What is worth a real PostgreSQL run is
// 1786700008000's CONCAT()-based duplicate check next door - PostgreSQL has
// had CONCAT() since 9.1, but it was never verified against a real server
// until this file, only inferred from documentation - and the same
// AutoMigrate call this test makes, so a schema/character-set mistake
// AutoMigrate might make silently on a dialect nobody ran it against here
// has somewhere to surface.
func TestAppRegistryTablesAreCreatedOnPostgres(t *testing.T) {
db := postgresDB(t)
const version = "1786700007000-pg"
cleanup := func() {
db.Migrator().DropTable(&adminmodels.SysAppCasbinGrant{}, &adminmodels.SysApp{})
// Only this test's own row, not the whole shared sys_migration
// table: postgresDB points at a real, persistent database (unlike
// the SQLite tests' fresh in-memory one per run), so a version left
// behind by a previous run of this same binary collides with the
// wrapper's own INSERT the next time this test runs.
db.Exec("DELETE FROM sys_migration WHERE version = ?", version)
}
t.Cleanup(cleanup)
cleanup()
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("automigrate sys_migration: %v", err)
}
if err := _1786700007000AppRegistryTables(db, version); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order", Version: "v1"}).Error; err != nil {
t.Fatalf("insert sys_app: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "dup", Version: "v1"}).Error; err == nil {
t.Fatal("a second sys_app row with the same app_code was accepted on PostgreSQL")
}
grant := adminmodels.SysAppCasbinGrant{AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET"}
if err := db.Create(&grant).Error; err != nil {
t.Fatalf("insert sys_app_casbin_grant: %v", err)
}
dup := grant
dup.Id = 0
if err := db.Create(&dup).Error; err == nil {
t.Fatal("a second sys_app_casbin_grant row with the same natural key was accepted on PostgreSQL")
}
}
@@ -0,0 +1,130 @@
package version
import (
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
common "go-admin/common/models"
)
func openAppRegistryDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("automigrate sys_migration: %v", err)
}
return db
}
// The migration has to build both tables and record itself as applied -
// F2/F6's acceptance case is a row landing in either one, and neither is
// possible if the table it belongs to was never created.
func TestAppRegistryTablesAreCreated(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasTable(&adminmodels.SysApp{}) {
t.Fatal("sys_app was not created")
}
if !db.Migrator().HasTable(&adminmodels.SysAppCasbinGrant{}) {
t.Fatal("sys_app_casbin_grant was not created")
}
// A row that exercises every column, not just HasTable/HasColumn -
// AutoMigrate can build a column with the wrong type and still report
// that it exists.
if err := db.Create(&adminmodels.SysApp{
AppCode: "order", Name: "Order", Version: "v1", Description: "d", Author: "a",
Requires: "payment", Pricing: "free", License: "MIT", Status: 1,
}).Error; err != nil {
t.Fatalf("insert sys_app: %v", err)
}
if err := db.Create(&adminmodels.SysAppCasbinGrant{
AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET",
}).Error; err != nil {
t.Fatalf("insert sys_app_casbin_grant: %v", err)
}
var applied common.Migration
if err := db.Where("version = ?", "1786700007000").First(&applied).Error; err != nil {
t.Fatalf("sys_migration was not recorded: %v", err)
}
}
// Running it twice must be safe: DDL does not roll back on MySQL, so an
// operator whose first attempt failed partway through has nothing to do but
// run it again. This calls AutoMigrate directly rather than the wrapper,
// which also inserts a sys_migration row that a second call would collide
// on - a collision Migrate.run() itself prevents by never calling a
// function twice for the same recorded version, so it is not this
// migration's job to tolerate.
func TestAppRegistryTablesAutoMigrateIsRepeatable(t *testing.T) {
db := openAppRegistryDB(t)
for i := 0; i < 3; i++ {
if err := db.Migrator().AutoMigrate(
new(adminmodels.SysApp),
new(adminmodels.SysAppCasbinGrant),
); err != nil {
t.Fatalf("automigrate %d: %v", i, err)
}
}
}
// sys_app.app_code is the unique key G2 ("is app X installed") answers with
// - a second row for the same app code must be rejected, not tolerated.
func TestSysAppAppCodeIsUnique(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order", Version: "v1"}).Error; err != nil {
t.Fatalf("first insert: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order dup", Version: "v1"}).Error; err == nil {
t.Fatal("a second sys_app row with the same app_code was accepted")
}
}
// sys_app_casbin_grant's unique index mirrors casbin_rule's own natural key
// (ptype,v0..v5) exactly - see design doc §3. A duplicate grant for the
// same rule must be rejected the same way gorm-adapter's own unique index
// on casbin_rule would reject it.
func TestSysAppCasbinGrantNaturalKeyIsUnique(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
grant := adminmodels.SysAppCasbinGrant{AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET"}
if err := db.Create(&grant).Error; err != nil {
t.Fatalf("first insert: %v", err)
}
dup := grant
dup.Id = 0
if err := db.Create(&dup).Error; err == nil {
t.Fatal("a second sys_app_casbin_grant row with the same natural key was accepted")
}
// A grant for a different app, but the identical casbin natural key, is
// exactly the collision two applications granting the same api/role
// pair would produce - the natural key has to be the one thing that
// rejects it, app_code is descriptive only and not part of the index.
other := grant
other.Id = 0
other.AppCode = "another-app"
if err := db.Create(&other).Error; err == nil {
t.Fatal("a duplicate natural key under a different app_code was accepted")
}
}
@@ -0,0 +1,145 @@
package version
import (
"fmt"
"runtime"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
)
// Give seed.SeedMenus's two write paths (seedApis, seedMenuTree in
// app/admin/service/seed.go) a real natural key to check before inserting,
// so a retried, partially-failed migration (see the design doc
// docs-prd/008-应用清单与安装器/数据库变更.md §1.5/§1.6) does not insert the
// same row twice. This has already happened in production once (duplicate
// sys_menu/casbin_rule rows on the demo site), not a theoretical risk.
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700008000SeedNaturalKeys)
}
func _1786700008000SeedNaturalKeys(db *gorm.DB, version string) error {
if err := seedNaturalKeys(db); err != nil {
return err
}
return db.Create(&common.Migration{Version: version}).Error
}
// seedNaturalKeys is split out from the wrapper above so tests can call it
// against a database that only has sys_menu/sys_api, without also standing
// up sys_migration - and so it can be called more than once in the same
// test to prove the re-run tolerance the doc comment above promises: DDL
// does not roll back on MySQL, so an operator whose first attempt failed
// partway through has nothing to do but run the whole migration again.
func seedNaturalKeys(db *gorm.DB) error {
m := db.Migrator()
// sys_menu.seed_code is a brand-new column: every existing row becomes
// NULL, and NULL never collides in the unique index built below, so
// this needs no pre-check.
if !m.HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
if err := m.AddColumn(&adminmodels.SysMenu{}, "SeedCode"); err != nil {
return err
}
}
if !m.HasIndex(&adminmodels.SysMenu{}, "uk_sys_menu_app_seed_code_del") {
if err := db.Exec(
"CREATE UNIQUE INDEX uk_sys_menu_app_seed_code_del ON sys_menu (app_code, seed_code, deleted_at)",
).Error; err != nil {
return err
}
}
// sys_api reuses existing, already-populated columns, which the demo
// site has already proven can hold duplicates. Refuse rather than let
// CREATE UNIQUE INDEX fail on an operator with no idea which rows to
// reconcile - same shape as 1786700003000_soft_delete_marker.go's
// refuseOnDuplicates.
if err := refuseOnDuplicateApis(db); err != nil {
return err
}
if !m.HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
if err := db.Exec(
"CREATE UNIQUE INDEX uk_sys_api_app_path_action_del ON sys_api (app_code, path, action, deleted_at)",
).Error; err != nil {
return err
}
}
return nil
}
// refuseOnDuplicateApis reports the (app_code, path, action) values that
// would make the unique index impossible, rather than the index failing to
// build and saying only that it did. Only live rows count: a soft-deleted
// duplicate does not block the index it will never occupy a slot in.
//
// sys_api.path/action (app/admin/models/sys_api.go) carry no NOT NULL
// constraint, and that stays true here on purpose: tightening it is an
// independent, backward-incompatible change of its own - existing NULL
// rows in a real database would need reconciling or backfilling before
// ALTER TABLE ... NOT NULL could even run, which is a decision for
// whoever owns that data, not something this migration should force as a
// side effect of adding an unrelated index. So this function has to
// tolerate NULL path/action rather than assume they cannot occur - see the
// query below for how it does that without either crashing on them
// (MySQL's CONCAT) or wrongly flagging them (GROUP BY's NULL-equals-NULL).
//
// The two are independent bugs that happened to share one root cause, and
// SQLite's own test suite for this file would have caught neither on its
// own: MySQL's CONCAT() returns NULL if any argument is NULL, which turned
// a duplicate check against a NULL-holding library into "converting NULL
// to string is unsupported" instead of a report - but SQLite's (and
// PostgreSQL's) CONCAT() treats a NULL argument as an empty string
// instead, so the exact same query never errors there no matter how it is
// called. A suite that only ever ran on SQLite would report success for
// both defects; only a real MySQL server surfaces the first one at all -
// this migration's PostgreSQL-only sibling test file
// (1786700008000_seed_natural_keys_postgres_test.go) rules out one more
// dialect, but MySQL specifically has to be checked by hand, since this
// repository's test suite has no MySQL service to run against in CI.
func refuseOnDuplicateApis(db *gorm.DB) error {
var dupes []string
if err := db.Raw(
// This has to agree with what the unique index it guards actually
// enforces, not just with what looks like a duplicate at a glance.
// Two different SQL rules collide on a NULL: GROUP BY treats two
// NULLs as equal, so a naive query flags every pair of rows that
// share a NULL path or action - even a pair with only one of the
// two NULL, since GROUP BY's equality still holds on whichever
// column both rows leave NULL - but a UNIQUE INDEX treats every
// NULL as distinct from every other value, including another
// NULL, so the index itself accepts every one of those pairs
// without complaint. Excluding any row missing either column from
// consideration entirely is what makes the two agree: a row
// missing path, or missing action, or missing both, can never
// violate the index no matter how many other rows are also
// missing the same one, so none of them belong in this count.
//
// No COALESCE: with both columns excluded whenever either is
// NULL, CONCAT here never receives a NULL argument for path or
// action - app_code cannot be NULL at all (see its own NOT NULL
// tag) - so there is nothing left for COALESCE to guard against,
// and leaving it out is deliberate rather than an oversight. A
// future regression that removed the two IS NOT NULL conditions
// above would fail loudly on MySQL (the same Scan error this
// query used to produce) instead of quietly reporting a made-up
// "duplicate" whose path and action both print as empty - the
// failure this function exists to prevent in the first place.
`SELECT CONCAT(app_code, '|', path, '|', action) FROM sys_api
WHERE deleted_at = 0 AND path IS NOT NULL AND action IS NOT NULL
GROUP BY app_code, path, action HAVING COUNT(*) > 1`,
).Scan(&dupes).Error; err != nil {
return fmt.Errorf("checking sys_api for duplicates: %w", err)
}
if len(dupes) > 0 {
return fmt.Errorf(
"sys_api already holds duplicate (app_code,path,action) %v; reconcile them before this migration can add its unique index",
dupes)
}
return nil
}
@@ -0,0 +1,105 @@
package version
import (
"testing"
)
// postgresDB is defined in 1786700003000_soft_delete_marker_postgres_test.go.
//
// This file exists because refuseOnDuplicateApis's duplicate check is
// spelled with CONCAT(), a function this migration's design assumed
// PostgreSQL has carried since 9.1 but that nothing had run against a real
// PostgreSQL server before this test - only against the pure-Go SQLite
// driver, which happens to bundle a SQLite new enough to have grown its own
// CONCAT() only recently. A dialect where that assumption were wrong would
// otherwise only be discovered the first time an operator's install hit a
// genuine sys_api duplicate on PostgreSQL in production.
func TestSeedNaturalKeysRefusesDuplicateApisOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
for i := 0; i < 2; i++ {
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed duplicate %d: %v", i, err)
}
}
err := seedNaturalKeys(db)
if err == nil {
t.Fatal("PostgreSQL accepted sys_api rows that already hold a duplicate (app_code, path, action)")
}
if !contains(err.Error(), "order") || !contains(err.Error(), "/api/v1/order") {
t.Errorf("the error does not name the offending row: %v", err)
}
if db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was built despite the migration refusing")
}
}
// GROUP BY treats two NULLs as equal for grouping; a UNIQUE INDEX treats
// every NULL as distinct from every other value, including another NULL.
// Both are standard SQL, not a SQLite/PostgreSQL/MySQL difference - this
// file exists to confirm that on a real server rather than assume it, the
// same reason TestSeedNaturalKeysRefusesDuplicateApisOnPostgres above
// exists for CONCAT(). See TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAccept
// in the SQLite-backed test file for the full account of why this matters:
// a naive duplicate check that does not exclude NULL path/action refuses
// an install the unique index itself would accept without complaint.
func TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAcceptOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
for i := 0; i < 2; i++ {
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, NULL, 0)",
).Error; err != nil {
t.Fatalf("seed NULL row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts on PostgreSQL: %v", err)
}
if !db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built on PostgreSQL even though seedNaturalKeys reported success")
}
}
// The success path, on the same server: both columns and both unique
// indexes have to actually build on PostgreSQL, not merely fail to error
// out on SQLite. Mirrors TestSeedNaturalKeysIsRepeatable's SQLite coverage.
func TestSeedNaturalKeysBuildsOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
for i := 0; i < 2; i++ {
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate %d: %v", i, err)
}
}
if !db.Migrator().HasColumn(&oldSeedMenu{}, "seed_code") {
t.Error("sys_menu.seed_code was not added on PostgreSQL")
}
if !db.Migrator().HasIndex(&oldSeedMenu{}, "uk_sys_menu_app_seed_code_del") {
t.Error("the sys_menu unique index was not built on PostgreSQL")
}
if !db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the sys_api unique index was not built on PostgreSQL")
}
}
@@ -0,0 +1,305 @@
package version
import (
"testing"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
common "go-admin/common/models"
)
// oldSeedMenu/oldSeedApi are the shape of sys_menu/sys_api immediately
// before this migration: post-1786700003000 (deleted_at is the NOT NULL
// millisecond marker) and post-1786700006000 (app_code exists), but before
// seed_code or either unique index. They stand in for the real runtime
// models, which by the time this file is read already carry the columns
// this migration adds - the same relationship oldUser bears to sys_user in
// 1786700003000_soft_delete_marker_test.go.
type oldSeedMenu struct {
MenuId int `gorm:"column:menu_id;primaryKey;autoIncrement"`
AppCode string `gorm:"column:app_code;type:varchar(64);not null;default:''"`
DeletedAt int64 `gorm:"column:deleted_at;not null;default:0"`
}
func (oldSeedMenu) TableName() string { return "sys_menu" }
type oldSeedApi struct {
Id int `gorm:"column:id;primaryKey;autoIncrement"`
AppCode string `gorm:"column:app_code;type:varchar(64);not null;default:''"`
Path string `gorm:"column:path;type:varchar(128)"`
Action string `gorm:"column:action;type:varchar(16)"`
DeletedAt int64 `gorm:"column:deleted_at;not null;default:0"`
}
func (oldSeedApi) TableName() string { return "sys_api" }
func openSeedNaturalKeysDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}, &common.Migration{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
return db
}
// The host's own hand-placed menus, and every app-seeded row written
// before this column existed, have no seed_code at all - an unbounded
// number of those must coexist under the same app_code without tripping
// the new unique index (design doc §1.6: "NULL never treated as equal to
// NULL").
func TestSeedNaturalKeysToleratesManyPreExistingMenusWithNoSeedCode(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 3; i++ {
if err := db.Create(&oldSeedMenu{AppCode: ""}).Error; err != nil {
t.Fatalf("seed pre-existing menu %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
t.Fatal("sys_menu.seed_code was not added")
}
}
// The point of adding seed_code at all: a second row with the same
// (app_code, seed_code) while both are live is what seedMenuTree's
// idempotency check depends on the database to reject if the Go-level
// check above it is ever bypassed or raced.
func TestSeedNaturalKeysMenuUniqueIndexBindsLiveRowsOnly(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error; err != nil {
t.Fatalf("seed: %v", err)
}
t.Run("a second live row with the same natural key is rejected", func(t *testing.T) {
err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error
if err == nil {
t.Fatal("a duplicate (app_code, seed_code) was accepted while both rows were live")
}
})
t.Run("the key is free again once the row is soft-deleted", func(t *testing.T) {
if err := db.Exec("UPDATE sys_menu SET deleted_at = ? WHERE seed_code = 'dir'", time.Now().UnixMilli()).Error; err != nil {
t.Fatalf("soft-delete: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error; err != nil {
t.Errorf("the key stayed taken after its row was soft-deleted: %v", err)
}
})
}
// The demo site has already proven sys_api can hold historical duplicates;
// the migration has to name them and refuse, not let CREATE UNIQUE INDEX
// fail on an operator with no idea which rows to reconcile.
func TestSeedNaturalKeysRefusesDuplicateApis(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed duplicate %d: %v", i, err)
}
}
err := seedNaturalKeys(db)
if err == nil {
t.Fatal("the migration accepted sys_api rows that already hold a duplicate (app_code, path, action)")
}
if !contains(err.Error(), "order") || !contains(err.Error(), "/api/v1/order") {
t.Errorf("the error does not name the offending row: %v", err)
}
if db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was built despite the migration refusing")
}
// sys_menu's column and index are independent of sys_api's outcome and
// should already be in place - a partial failure here still leaves a
// record of what succeeded, same as any other non-transactional DDL
// migration in this package.
if !db.Migrator().HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
t.Error("sys_menu.seed_code was not added even though only the sys_api step failed")
}
}
// Only live rows count towards the duplicate check: a row a prior,
// unrelated soft-delete already retired does not block the index it will
// never occupy a slot in.
func TestSeedNaturalKeysIgnoresSoftDeletedApiDuplicates(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed live row: %v", err)
}
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET", DeletedAt: time.Now().UnixMilli()}).Error; err != nil {
t.Fatalf("seed soft-deleted row: %v", err)
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built")
}
}
// The point of the sys_api index, mirroring
// TestSeedNaturalKeysMenuUniqueIndexBindsLiveRowsOnly above: a second live
// row is rejected, and the key is free again once the row is
// soft-deleted.
func TestSeedNaturalKeysApiUniqueIndexBindsLiveRowsOnly(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
t.Run("a second live row with the same natural key is rejected", func(t *testing.T) {
err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', 'GET', 0)",
).Error
if err == nil {
t.Fatal("a duplicate (app_code, path, action) was accepted while both rows were live")
}
})
t.Run("the key is free again once the row is soft-deleted", func(t *testing.T) {
if err := db.Exec(
"UPDATE sys_api SET deleted_at = ? WHERE path = '/api/v1/order'", time.Now().UnixMilli(),
).Error; err != nil {
t.Fatalf("soft-delete: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', 'GET', 0)",
).Error; err != nil {
t.Errorf("the key stayed taken after its row was soft-deleted: %v", err)
}
})
}
// Running it twice must be safe: DDL does not roll back on MySQL, so an
// operator whose first attempt failed partway through (say, sys_menu's step
// succeeded and sys_api's refused) has nothing to do but run the whole
// migration again once the duplicates are reconciled.
func TestSeedNaturalKeysIsRepeatable(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
for i := 0; i < 3; i++ {
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate %d: %v", i, err)
}
}
}
// The wrapper's contract with Migrate.run(): the version is only recorded
// once the whole thing - both columns, both indexes - succeeded.
func TestSeedNaturalKeysWrapperRecordsTheVersion(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := _1786700008000SeedNaturalKeys(db, "1786700008000"); err != nil {
t.Fatalf("migrate: %v", err)
}
var applied common.Migration
if err := db.Where("version = ?", "1786700008000").First(&applied).Error; err != nil {
t.Fatalf("sys_migration was not recorded: %v", err)
}
}
// GROUP BY treats two NULLs as equal for grouping purposes; a UNIQUE INDEX
// treats every NULL as distinct from every other value, including another
// NULL - both are standard SQL semantics, not a quirk of one dialect (see
// the postgres-only test file next to this one for the same check against
// a real server). A duplicate check that groups on the raw columns without
// accounting for that difference refuses an install the index itself would
// accept without complaint, on data there is nothing to "reconcile" -
// worse than the index simply failing to build, because it stops a library
// that has nothing wrong with it.
//
// sys_api.path/action carry no NOT NULL constraint - see the design doc's
// note on this migration for why that stays true in this batch, changing
// it is an independent, backward-incompatible migration of its own - so
// this state is reachable in a real database even though seedApis's own
// Create call, which always writes the Go zero value "" rather than NULL,
// never produces it itself. Inserted via raw SQL for exactly that reason:
// models.SysApi's Path/Action are plain (non-pointer) Go strings, which
// cannot represent NULL through a normal Create call.
func TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAccept(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, NULL, 0)",
).Error; err != nil {
t.Fatalf("seed NULL row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built even though seedNaturalKeys reported success")
}
}
// The case above has both path and action NULL on every row, which both
// of the query's two NULL-exclusion conditions independently catch - it
// cannot tell "only path IS NOT NULL is doing anything here" apart from
// "both conditions are doing something". A row missing only one of the
// two is exactly as real (an api registered with a path but no method,
// or vice versa) and exercises only one condition at a time: two rows
// sharing a real path but both NULL in action, or two rows sharing a real
// action but both NULL in path. GROUP BY treats each pair's shared NULL
// the same way it treats a shared (NULL, NULL) - as equal - and the
// unique index accepts both pairs for the same reason it accepts the
// (NULL, NULL) case, so neither belongs in the count either.
func TestSeedNaturalKeysDoesNotFlagPartiallyNullRows(t *testing.T) {
cases := []struct {
name string
insert string // two rows, sharing a value in exactly one of path/action
}{
{
name: "path is null, action repeats",
insert: "INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, 'GET', 0)",
},
{
name: "action is null, path repeats",
insert: "INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', NULL, 0)",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Exec(tc.insert).Error; err != nil {
t.Fatalf("seed row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built even though seedNaturalKeys reported success")
}
})
}
}
+55 -1
View File
@@ -41,6 +41,7 @@ import (
"encoding/hex"
"errors"
"fmt"
"sync"
"sync/atomic"
"time"
@@ -68,6 +69,55 @@ type Check struct {
Err string `json:"error,omitempty"`
}
// extra are checks a host registers that this package cannot make itself.
//
// The direction is why this exists. Whether the schema matches what the binary
// expects is answered by the migration registry, which lives under cmd/ - and
// common/ has never imported cmd/. Rather than start, the host registers the
// check from where both are already in scope.
var (
extraMu sync.RWMutex
extra []namedCheck
)
type namedCheck struct {
name string
fn func(context.Context) error
}
// Register adds a check to what Ready asks.
//
// It panics on a duplicate name rather than replacing or appending: two checks
// under one name make the failing one impossible to identify from the response,
// and registering the same one twice is a wiring mistake worth hearing about at
// start-up rather than never.
func Register(name string, fn func(context.Context) error) {
if name == "" {
panic("health: a registered check needs a name")
}
if fn == nil {
panic("health: check " + name + " is nil")
}
extraMu.Lock()
defer extraMu.Unlock()
for _, c := range extra {
if c.name == name {
panic("health: check " + name + " is already registered")
}
}
extra = append(extra, namedCheck{name: name, fn: fn})
}
// registered returns the checks a host has added, copied so that Ready is not
// iterating the slice while another goroutine appends to it.
func registered() []namedCheck {
extraMu.RLock()
defer extraMu.RUnlock()
out := make([]namedCheck, len(extra))
copy(out, extra)
return out
}
// Ready asks every dependency this process cannot serve a request without.
//
// The queue is deliberately absent. Nothing on AdapterQueue answers "are you
@@ -75,10 +125,14 @@ type Check struct {
// a queue that is down degrades logging rather than stopping requests - which
// is a reason to alert, not a reason to leave the load balancer pool.
func Ready(ctx context.Context) []Check {
return []Check{
checks := []Check{
safely("database", func() error { return pingDB(ctx) }),
safely("cache", probeCache),
}
for _, c := range registered() {
checks = append(checks, safely(c.name, func() error { return c.fn(ctx) }))
}
return checks
}
// safely turns a panic into a failed check.
+125
View File
@@ -0,0 +1,125 @@
package health
import (
"context"
"errors"
"testing"
)
// isolate empties the registered checks and puts them back, so one test in
// this package cannot decide what the next one sees.
func isolate(t *testing.T) {
t.Helper()
extraMu.Lock()
previous := extra
extra = nil
extraMu.Unlock()
t.Cleanup(func() {
extraMu.Lock()
extra = previous
extraMu.Unlock()
})
}
func findCheck(checks []Check, name string) (Check, bool) {
for _, c := range checks {
if c.Name == name {
return c, true
}
}
return Check{}, false
}
// A registered check has to reach Ready's answer, or the host has wired
// something that never gets asked.
func TestARegisteredCheckIsAsked(t *testing.T) {
isolate(t)
Register("schema", func(context.Context) error { return errors.New("two behind") })
checks := Ready(context.Background())
c, ok := findCheck(checks, "schema")
if !ok {
t.Fatal("Ready did not ask the registered check")
}
if c.OK {
t.Error("the check returned an error and was still reported OK")
}
if c.Err != "two behind" {
t.Errorf("Err = %q, want the check's own message", c.Err)
}
if Healthy(checks) {
t.Error("Healthy said yes while a registered check was failing")
}
}
// The context Ready is given has to reach the check: it carries the probe's
// deadline, and a check that ignores it can hold the handler past it.
func TestTheRegisteredCheckIsGivenReadysContext(t *testing.T) {
isolate(t)
type key struct{}
Register("ctx", func(ctx context.Context) error {
if ctx.Value(key{}) != "carried" {
return errors.New("the check was handed a different context")
}
return nil
})
checks := Ready(context.WithValue(context.Background(), key{}, "carried"))
c, ok := findCheck(checks, "ctx")
if !ok {
t.Fatal("the registered check was not asked")
}
if !c.OK {
t.Errorf("check failed: %s", c.Err)
}
}
// A check that panics must not take the process down through the probe, the
// same guarantee the built-in checks have.
func TestARegisteredCheckThatPanicsFailsRatherThanCrashes(t *testing.T) {
isolate(t)
Register("boom", func(context.Context) error { panic("registry unreachable") })
checks := Ready(context.Background())
c, ok := findCheck(checks, "boom")
if !ok {
t.Fatal("the registered check was not asked")
}
if c.OK {
t.Error("a panicking check was reported OK")
}
}
func TestRegisteringTheSameNameTwicePanics(t *testing.T) {
isolate(t)
Register("dup", func(context.Context) error { return nil })
defer func() {
if recover() == nil {
t.Error("registering a duplicate name did not panic; two checks under " +
"one name make the failing one impossible to identify")
}
}()
Register("dup", func(context.Context) error { return nil })
}
func TestRegisterRefusesAnEmptyNameOrNilCheck(t *testing.T) {
isolate(t)
for _, tc := range []struct {
name string
fn func(context.Context) error
why string
}{
{"", func(context.Context) error { return nil }, "empty name"},
{"nilfn", nil, "nil function"},
} {
func() {
defer func() {
if recover() == nil {
t.Errorf("%s did not panic", tc.why)
}
}()
Register(tc.name, tc.fn)
}()
}
}
+1 -1
View File
@@ -11,7 +11,7 @@ require (
github.com/casbin/casbin/v3 v3.8.1
github.com/gin-gonic/gin v1.12.0
github.com/glebarez/sqlite v1.11.0
github.com/go-admin-team/go-admin-core/v2 v2.7.0
github.com/go-admin-team/go-admin-core/v2 v2.8.0
github.com/google/uuid v1.6.0
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible
github.com/mssola/user_agent v0.6.0
+2 -2
View File
@@ -145,8 +145,8 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-admin-team/go-admin-core/v2 v2.7.0 h1:1qV0/5iFBvkE3BRtm4ip0v0QYG9Fgx4UtOTd8zkQT9c=
github.com/go-admin-team/go-admin-core/v2 v2.7.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-admin-team/go-admin-core/v2 v2.8.0 h1:ZTw5Z/UT1/7OltbGPEaEVerRk4z3koB6O8nDbb84tPM=
github.com/go-admin-team/go-admin-core/v2 v2.8.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o=