Compare commits

..
Author SHA1 Message Date
zhangwenjian 2c50317a98 fix🐛: stop the duplicate check refusing what the index would accept
The check grouped by (app_code, path, action) and refused whatever
appeared more than once. GROUP BY treats two NULLs as the same value; a
unique index treats them as different ones and allows both. So a
database holding rows with a null path or action was refused for
duplicates the index it was blocking would have accepted - and the
migration stopped, on a database with nothing wrong with it.

Both columns are nullable: neither carries a not-null tag, so gorm built
them that way. Measured on MySQL 8.0, PostgreSQL 15 and SQLite: two rows
with both columns null are one group to GROUP BY, and the unique index
builds over them without complaint. Standard SQL, not a dialect quirk.

They are now excluded from the check rather than grouped. Each column
needs its own exclusion and has its own test: one null column is enough
to make the index accept the pair, so removing either condition alone
lets that half through - which is what the two subtests are for, and
each fails only for its own half.

The message could not name the rows either. MySQL's CONCAT returns NULL
when any argument is, and scanning that into a string fails with
"converting NULL to string is unsupported" - so the check reported a
driver error instead of the duplicates it exists to report. SQLite and
PostgreSQL treat a null argument as empty and say nothing, which is why
this never surfaced in the tests: they run on SQLite, and this
repository has no MySQL in CI. The comment says so, so that a
postgres-only test file is not mistaken for cover.

No COALESCE was added to paper over that. It would have had nothing left
to guard once the nulls are excluded, and it would make a future
regression quieter: someone dropping the exclusions would get a report
naming rows that are not duplicates, which reads as a real answer,
rather than a scan error that reads as a broken query.

Leaving path and action nullable is deliberate. Tightening them is a
migration of its own - existing null rows have to be given values, and
what those values should be belongs to whoever owns the data, not to a
migration whose job is adding an index.
2026-09-08 20:17:50 +08:00
zhangwenjian 28350a15bb fix🐛: repair the row a retry reuses instead of walking past it
The idempotency check added in the previous commit stopped a retry
inserting a second copy, and introduced a quieter failure in its place:
a retry that found an existing sys_menu row skipped everything after the
insert. Those are the sys_menu_api_rule bindings and the materialized
path, and neither is written by the statement that writes the menu -
paths is a separate UPDATE, and on MySQL an earlier DDL has already
committed the transaction that was supposed to hold them together.

So an install interrupted between those steps left a menu that exists,
sits outside the tree with an empty path, and is bound to no API. What
core's contract.md says about such a menu is that it is invisible to
every role and its apis are authorized for no one - while the installer
reports success.

Reusing now repairs. paths is compared before it is written, so a row
that is already right is not touched. Bindings are inserted with WHERE
NOT EXISTS rather than deleted and rebuilt: an administrator can bind an
api to a menu from the menu screen, and delete-then-rebuild would take
that with it on the next retry - the same accident as sys_role.go's
Association.Delete, pointing the other way.

Confirmed as a defect before it was fixed, by building the half-written
state and watching the assertions fail:

    dir.Paths = "", want "/0/1"
    binding count for list = 0, want 1

Four paths through the repair, each with a degradation that reds its own
test and leaves the others green: missing bindings only, missing paths
only, both, and neither. The fourth asserts no UPDATE is issued for a
row already correct.

A fifth covers what the repair must not do. Rebuilding bindings instead
of inserting them leaves every other test green while silently deleting
a binding this code did not create; that one now fails with "a retry
silently deleted a binding it does not own".

Bindings an older version of a manifest created and a newer one no
longer lists are left alone. Removing them is a delete, and a delete
needs the same certainty about ownership that uninstall does - this
function cannot tell a stale binding from one somebody added by hand.
2026-09-08 19:51:22 +08:00
zhangwenjian c6d3ea5f81 fix🐛: stop a retried seed from inserting a second copy
seedApis and seedMenuTree were bare tx.Create calls. A migration that
failed partway and was run again re-inserted everything it had already
written - which is not hypothetical: the demo site collected eighteen
duplicate sys_menu rows this way, and three duplicate menus were visible
in its sidebar.

Both now look for a live row already holding the natural key and reuse
it. Only live rows count: a row an earlier soft-delete retired does not
stand in the way of a fresh insert under the same key, which is also
what the unique indexes allow.

The app_code half of each key has a test of its own. Without it the
lookups still passed every existing test while quietly letting one
application adopt another's rows - and an uninstall would then delete
rows the other application believed were its own, on both sides without
an error. Removing app_code from either lookup now fails with
"has 1 row(s) ... want 2 - one per app".
2026-09-08 19:26:42 +08:00
zhangwenjian 7fadb4b585 feat✨: give the seeded rows a natural key to be found by
Seeding needs something to look for before it inserts, or a retry writes
a second copy of everything it already wrote. sys_api already had one in
(app_code, path, action). sys_menu had nothing usable: menu_name is
pascalCase(appCode) + pascalCase(code), which is not injective -
"list-all", "listAll" and "list_all" all become "ListAll" - so the
original code cannot be recovered from it. Hence a new column.

seed_code is nullable, against this repository's habit of NOT NULL
DEFAULT '' for a new column, and deliberately. Every row that predates
it has no meaningful value, and under a unique index an empty string
collides with every other empty string while NULL collides with nothing.
The convention exists because deleted_at's nullability broke a unique
index; here nullability is what makes one possible.

The unique index on sys_api cannot simply be created: a live database is
known to hold historical duplicates - the demo site had eighteen. The
migration looks first and refuses while naming the offending rows,
rather than letting CREATE UNIQUE INDEX fail with a constraint error
that names none. Same shape as 1786700003000's refuseOnDuplicates.

Run against SQLite, MySQL 8.0 and PostgreSQL 15, including the CONCAT
duplicate check, which had only ever been executed by SQLite's driver.
2026-09-08 19:26:42 +08:00
zhangwenjian 691df82016 feat✨: add the sys_app registry and the casbin grant ledger
sys_app is one row per installed application. It is physically deleted
on uninstall rather than following the millisecond soft-delete marker
the other sys_ tables use: an installed-app registry has no "deleted by
accident, needs recovering" case, and a physical delete is what lets the
same code be installed again afterwards.

status is installing/installed/failed rather than a boolean, because an
install spanning several migration files is not atomic on MySQL - DDL
commits implicitly, so a run can stop in the middle. failed_version and
last_error are diagnostic snapshots for a person to read; nothing may
decide anything from them, and the field comments say so. Where to
resume is answered by sys_migration, which cannot drift from what was
actually applied.

sys_app_casbin_grant records which casbin_rule rows an install created,
keyed by casbin_rule's own natural key. That table is not extended
instead: gorm-adapter's SavePolicy truncates and reloads it from an
in-memory model, which would drop any column added here without a word.

Built against SQLite, MySQL 8.0 and PostgreSQL 15.
2026-09-08 19:26:40 +08:00
zhangwenjian ea348fa9d1 build📦: pin go-admin-core v2.8.0
v2.8.0 adds sdk/contract/app - an application's manifest, and the one
comparator for its version - which the installer in this batch is built
on. Nothing here uses it yet; this is the dependency arriving.

Checked that the release is consumable rather than only tagged: a
program built against the published module registers a manifest, reads
it back, and gets -1 from Compare("1.9.0", "1.10.0"), which is the
multi-digit case a string comparison would order backwards.

25 packages pass and checksilent is clean on the new version.
2026-09-08 18:35:13 +08:00
wenjianzhang 925c6772a6 Merge pull request #921 from go-admin-team/fix/920-schema-readiness
Fail readiness while the database is behind the migrations
2026-09-08 17:18:34 +08:00
wenjianzhang 0c60e44aee Merge pull request #922 from go-admin-team/fix/919-drop-index-postgres
Drop the index with SQL this dialect can parse
2026-09-08 17:18:08 +08:00
zhangwenjian a716086295 test✅: fail these tests when the database does not answer
Every Create and Raw dropped its error. Most of them would have failed
an assertion further down anyway, with a message describing the wrong
problem - but one of them would not.

The last count in the index test reads how many indexes survived and
expects zero. An unchecked query that fails leaves the variable at zero,
and zero is what success looks like: a test that cannot reach the
database reports that the indexes were dropped.

Demonstrated rather than assumed, by breaking that one query both ways:

    with the check:    FAIL  counting the indexes after: relation
                             "pg_indexes_nope" does not exist
    without it:        PASS

Raised by Copilot on #922, as a consistency point with the SQLite tests
in this package. It is that as well, but the reason it is worth doing is
the row above.
2026-09-08 17:04:31 +08:00
zhangwenjian f8a5066a40 test✅: run the migration against PostgreSQL in CI
The rest of this package's tests run on SQLite, where dropping an index
through the migrator works. That is why a migration which failed on
every PostgreSQL database it was pointed at had a green suite: the
defect cannot occur on the backend being tested.

A test alone would not have helped either - without a service it skips,
and a test that never runs is the same as no test. So the workflow gains
a postgres service and the DSN, and the helper refuses to skip when CI
is set: a workflow that drops the service or renames the variable fails
rather than going quietly green, which is the shape of the original
defect.

Counter-proved by putting Migrator().DropIndex back, which reproduces
the statement verbatim:

    DROP INDEX CURRENT_SCHEMA()."idx_sd_pg_user_deleted_at"
    ERROR: syntax error at or near "CURRENT_SCHEMA" (SQLSTATE 42601)

The conversion test also checks the timestamp survives as a marker,
since a conversion that dropped it would bring deleted rows back live
while still passing a column-type assertion.
2026-09-08 16:57:17 +08:00
zhangwenjian f978967ef1 fix🐛: drop the index with SQL this dialect can parse
The soft-delete conversion dropped the indexes on deleted_at through
gorm's Migrator().DropIndex. Its PostgreSQL driver resolves a schema for
the statement and falls back to an expression when it cannot:

    currentSchema, _ := m.CurrentSchema(stmt, stmt.Table)
    m.DB.Exec("DROP INDEX ?.?", currentSchema, clause.Column{Name: name})

DROP INDEX takes an identifier in that position, so what reached the
server was

    DROP INDEX CURRENT_SCHEMA()."idx_sys_api_deleted_at"
    ERROR: syntax error at or near "CURRENT_SCHEMA" (SQLSTATE 42601)

The schema is unresolvable for every call this migration makes, because
it passes a table name as a string rather than a model. So it failed on
every PostgreSQL database rather than intermittently, and stopped the
whole conversion at the first table.

What that looked like from outside is go-admin#919: an upgrade that
could not complete, and a login rejecting a correct password, because
deleted_at was still a timestamptz while the current query compares it
to 0. Neither symptom names a migration.

Written per dialect, for the same reason addBigIntColumn and
renameColumn already are. MySQL and SQL Server name the table and have
no IF EXISTS for it; PostgreSQL and SQLite name the index alone.

Verified by running the shipped migration against PostgreSQL 15 and
MySQL 8.0 in containers, and SQLite through this package's tests. The
SQL Server form is from its documentation and has not been run - there
is no SQL Server here to run it against, and saying so is better than
implying four dialects were checked.
2026-09-08 16:57:15 +08:00
zhangwenjian d6e2c02fda fix🐛: fail readiness while the database is behind the migrations
The process started, both probes passed, and the first sign that the
schema did not match was a login failing with a driver-level encoding
error - go-admin#919, where an operator upgraded the binary and
restarted the API without running migrate. Nothing between those two
events had an opinion about the schema.

Readiness is where this belongs. Liveness asks "restart me", and a
process whose database is on the wrong schema comes back to the same
schema. Readiness asks "send me requests", and the answer is no. A
rolling update then stalls at the deploy - new instances never become
ready, the old ones keep serving - rather than at somebody's login, and
running migrate clears it without a restart because the check is
evaluated per request.

Any tenant database being behind fails the check, not only the one being
served: migrations are applied to every database in one run, so one
behind means that run did not finish, and serving the rest would let a
half-applied deploy look like a partial success.

A missing sys_migration table is nothing applied rather than an error.
That is a first deploy, where every migration is pending and the
operator can act on being told so.

The one test that matters is the one that cannot be written normally.
The registry is filled by init() in packages cmd/api does not import, so
a test that imported them to look at it would pass whatever the real
binary links - and a binary that links none of them gives a check that
reports every database current, forever, with every other test here
still green. TestTheServingBinaryLinksTheMigrationRegistry asks the
build instead, with a negative control so that a query matching
everything fails rather than passes.

Closes #920.
2026-09-08 15:42:14 +08:00
zhangwenjian e98b65cf90 feat✨: let the host add a readiness check this package cannot make
Whether the schema matches what the binary expects is answered by the
migration registry, which lives under cmd/. common/ has never imported
cmd/, and starting with this would put the shared layer behind the
command layer for one check.

Register instead, from where both are already in scope. A duplicate name
panics rather than appending: two checks under one name make the failing
one impossible to identify from the response body, and registering the
same one twice is a wiring mistake better heard at start-up than never.

The registered check is run through the same guard as the built-in ones,
so one that panics fails its check rather than taking down the probe
that asked.
2026-09-08 15:42:14 +08:00
zhangwenjian bc5411c30c feat✨: report what the migration registry holds without a database
Status answers what is registered, what is applied, and what is applied
while nothing registers it - and needs a database to do it. A readiness
check needs only the first half, and it already holds the databases it
is asking about.

Without this it would have to call SetDb to reuse Status, writing this
package's shared state from a request path, for a question that does not
depend on any database at all.
2026-09-08 15:42:11 +08:00
wenjianzhang 27f23121f0 Merge pull request #918 from go-admin-team/fix/911-queue-shutdown
Drain the queue on the way out instead of abandoning it
2026-09-07 17:39:15 +08:00
zhangwenjian 047b23872c fix🐛: let a drain that finishes on the deadline count as finished
The wait was one select over done and ctx.Done(). Both can be ready when
it runs, select picks at random among ready cases, and so a queue that
drained in the same instant the budget expired was reported as an
overrun about half the times it landed there - often enough to be read
as noise, and pointing at the wrong thing when it was not. core's own
RunShutdown re-checks for this reason; this did not.

The tie-break is now a function taking channels rather than a queue,
which is what lets a test hand it a closed done and an expired ctx
together. That state is the whole of the bug and cannot be arrived at
reliably from the outside; over 1000 iterations the single-select
version fails, and the second look does not.

The test for giving up on the deadline read the call counter straight
after shutdownQueue returned, while Shutdown runs on a goroutine nobody
joins. It passed because the goroutine is scheduled promptly, not
because anything ordered the two. The fake now signals that Shutdown has
been entered and the test waits for it.

Both raised by Copilot on #918.

The first attempt at the tie-break test was wrong and is not what
landed: it asserted that an immediately-returning Shutdown always counts
as drained under an already-expired context, which is not true and
should not be - if the goroutine has not run, nothing has drained. That
test failed, correctly. What is being claimed is narrower: when both are
ready, done wins.
2026-09-07 17:21:03 +08:00
zhangwenjian 84bd87dcc9 fix🐛: drain the queue on the way out instead of abandoning it
Nothing stopped the queue when the process exited. core v2.7.0 made the
drain work - Memory.Shutdown closes the queue and waits for every
consumer to finish what it holds, and the legacy adapter cancels its
context and closes the underlying queue - but no call site ever reached
it. The only Shutdown() in this repository applies to the previous
adapter during a reload, so the installed one was simply left. The login
log, the operation log and the API sync all publish through it, so a
rolling restart dropped whatever had not been consumed, on the path
where the process exits 0 and reports "Server exiting".

Setup now registers a BeforeExit callback that shuts down the adapter
this package installed.

Three things it has to get right, each with a test.

It reads `installed` when it runs, not when it registers. A reload
replaces the adapter, and the one from start-up is a queue nobody has
published to since.

It never goes through sdk.Runtime.GetQueueAdapter. That accessor never
returns nil - with no queue section configured it wraps the runtime's
own fallback - so it would look like it worked while closing a queue
this package neither built nor started. That is the same trap setupQueue
already had to drop an `if q != nil` for.

It registers once. Setup is re-run on every configuration change, and a
callback per reload would leave the shutdown phase holding a row of
identical entries, each eligible to be named as the one that overran the
budget.

That last one needed a seam. shutdownQueue takes the adapter on its
first run, so the second and third callbacks find nothing and return -
three registrations produce exactly the same observable result as one,
and a test going through the effect passes either way. It did: the
counter-proof for "register on every reload" came back green until the
registration was counted at the seam instead.

The wait is bounded here rather than left to the phase. Shutdown takes
no context, so a consumer that never finishes would hold the process
until SIGKILL; the callback gives up and says what is being lost, which
the phase's generic overrun message cannot.

Ordering falls out of the phase rather than being arranged: callbacks
run in reverse registration order, this one registers during setup and
the job scheduler's registers on AfterListen, so the schedulers stop
before the queue drains. Verified against core v2.7.0 rather than read
off the source.

Closes #911.
2026-09-07 17:10:13 +08:00
wenjianzhang 0e7a13aeba Merge pull request #917 from go-admin-team/fix/914-gen-write-guard
Register the generator's writing endpoints only in a development mode
2026-09-07 15:32:13 +08:00
zhangwenjian 85d50da494 fix🐛: tell the start-up warning's reader to restart, not only to reconfigure
The warning said to set application.mode and stopped there. Following
that on a running process does not close anything: buildRouter has one
call site, in run(), and route registration is on no phase and no reload
callback, so a configuration reload moves the mode and leaves the routes
exactly where they were.

The reader is then worse off than before they acted. The mode now says
prod, GenWriteRoutesEnabled agrees, and the endpoints are still served -
so the one thing they could check to confirm the fix reports success
while the exposure is untouched, until something restarts the process.

A test pins the gap rather than the prose: build under dev, move the
mode to prod, and the routes are still in the engine. It fails if
registration ever becomes dynamic, which is the change that would make
the new sentence wrong.

That test degrades differently from the others - making it fail means
rewriting registration, not weakening it - so what was checked instead
is that it cannot go vacuous. Both of its premises are guarded: with the
gate always refusing it reports building under dev without the writing
routes, and with the gate always allowing it reports the predicate still
allowing prod. Neither failure can be mistaken for the assertion passing.

Raised by Copilot on #917.
2026-09-07 15:20:36 +08:00
zhangwenjian 1d9def4314 test✅: restore the mode before the route helper returns
registeredRoutes set config.ApplicationConfig.Mode and gave it back with
t.Cleanup, which runs at the end of the test rather than at the end of
the helper. Everything the caller did after the call therefore ran under
the mode the helper had been asked about, not one the caller chose.

Nothing was wrong yet: the one caller that reads the mode afterwards
sets it itself, and the two cleanups happen to unwind in an order that
leaves the right value. Both of those are accidents, and neither is
visible at the call site.

A defer inside the helper makes the borrowing end where it starts. The
doc comment said the mode was put back before returning while the code
did not, so that is now true rather than aspirational.

The counter-proof is the reason this has a test of its own: with
t.Cleanup back in place TestRegisteredRoutesRestoresTheModeBeforeReturning
fails and nothing else does, which is what a leak this quiet looks like
when something is actually watching for it.

Raised by Copilot on #917.
2026-09-07 15:14:39 +08:00
zhangwenjian ed9bbd01e2 feat✨: warn at start-up when the generator can write to this host
The gate in the previous commit is decided by application.mode, and the
shipped configuration says dev. So the deployment most likely to be
serving the writing endpoints is the one that changed nothing, and that
is also the one least likely to go looking for them. A gate whose
default is open needs to say so.

Nothing is said in demo mode. The routes are registered there, but
DemoEvn refuses all three by name, so a warning would describe an
exposure that is not present.

The decision is split from the logging so it can be tested. Three
counter-proofs: warning in demo as well fails mode=demo; a warning that
never fires fails mode=dev, which is what shows the line can be reached
at all; and one that always fires fails every mode but dev.
2026-09-07 15:07:07 +08:00
zhangwenjian 523d6a3649 fix🐛: register the generator's writing endpoints only in a development mode
Three of the code generator's endpoints do not read. /gen/toproject
writes seven Go and Vue source files onto the host, one of them under
the path gen.frontpath names; /gen/apitofile writes a migration;
/gen/todb inserts menus and APIs. All three are GET, and all three are
listed in CasbinExclude - which AuthCheckRole skips - so Enforce never
runs for them. Any account that can log in could call them, on every
deployment.

They are now registered only where application.mode is dev or demo. dev
is the shipped default and is where the generator is meant to be used.
demo keeps them because demo mode already has a better answer than a
404: DemoEvn refuses these three by name and explains itself, which is
what the demo is for. test and prod get nothing, and so does a process
whose mode was never set.

This does not make the endpoints safe where they exist; it stops them
existing where nobody should be calling them. A host left on the shipped
dev is still open, which is why the next commit says so at start-up.

CasbinExclude is left alone on purpose. Taking the three off that list
would make them require a permission no existing deployment has granted,
so every non-admin user would start getting 403 from a tool that worked
yesterday. That is a migration, not a guard, and it belongs with a
release that can carry one.

Four counter-proofs, each red on the test that names the behaviour and
green everywhere else: a gate that always allows fails test/prod/unset
only; a gate that always refuses fails dev/demo and takes
TestEveryRouteDemoModeRefusesStillExists with it; moving a read-only
route inside the gate fails the reading test; and spelling the condition
at the registration site instead of calling the predicate fails the
agreement test, which is what keeps that test from being a tautology.
2026-09-07 15:05:24 +08:00
zhangwenjian 6326962862 style🎨: gofmt gen_router.go
Two pre-existing deviations: a space before the comma in
sysNoCheckRoleRouter's parameter list, and no newline at end of file.
Separated from the change that follows so its diff is only the change.
2026-09-07 15:02:33 +08:00
wenjianzhang cd7c8375c0 Merge pull request #916 from go-admin-team/docs/replicas-constraint
Name the job scheduler as the other reason for one replica, and stop the manifests redeploying the demo
2026-09-07 14:52:53 +08:00
zhangwenjian 63bcc912ef ci👷: stop the k8s manifests from redeploying the demo
The comment at the top of this workflow says documentation-only changes
skip it, because a push to master pushes an image, runs the migrations
and restarts the demo container. The ignore list did not cover
scripts/k8s, so editing a manifest that the deploy never reads bought
the site an outage.

The pattern is scripts/k8s/** rather than scripts/** because
scripts/Dockerfile is a build input - go.yml builds the release image
from it on a tag.

This workflow file stays outside the list on purpose. paths-ignore skips
only when every changed path matches, so a change that edits the deploy
still runs it, which is the point.
2026-09-07 14:14:39 +08:00
zhangwenjian adcdd2edcd docs📝: name the job scheduler as the other reason for one replica
The comment on replicas gave one obstacle to raising it, the shared log
volume, which reads as the only one. Someone who moves the log path off
that volume would conclude the way is clear.

The scheduler in app/jobs is the second, and it is the one that does not
announce itself. Its handle on a job lives in sys_job.entry_id, one
column shared by every process, and startup zeroes the whole column
before writing its own ids. A second pod therefore erases the first
pod's, and both pods run the full enabled list. Stopping a job from the
UI then removes an entry from whichever process is asked, by an id that
belongs to another one, and answers 200.

See #915.
2026-09-07 13:32:43 +08:00
wenjianzhang 29406f839e Merge pull request #913 from go-admin-team/fix/demo-mode-guard
fix🐛: demo 模式放行了注册成 GET 的写接口
2026-09-07 08:11:14 +08:00
zhangwenjian a43133ab7b fix🐛: stop demo mode serving the writes that are registered as GET
DemoEvn decided by HTTP method: GET and OPTIONS through, everything else
refused. Three of the code generator's routes are registered as GET and write
anyway - two emit Go source files onto the server's filesystem, and the third
inserts menus, APIs and casbin rules into the database. They sit in a group
whose own name says it does no role check, and a demo deployment lets anybody
log in. So on the demo host any visitor could write to the machine and to the
database, and the one that writes menus had in fact been used: three generated
SysCasbinRule entries is how this was noticed.

The guard now also looks at the matched route. The method cannot answer the
question - whether a request changes anything is not something the verb reports
truthfully here - so the three are named, as gin route patterns, which is what
Context.FullPath returns and how CasbinExclude already spells them.

Changing them to POST would be the better shape and is not this change. sys_api
records an endpoint by method and path and the casbin policy follows it, so
flipping the verb needs a migration and a policy resync; until both land, every
existing deployment would start answering 403 to a role that could use the
generator the day before.

The read-only half stays reachable: preview, the table tree, and the two
database listings. A demo host that cannot demonstrate the generator is as
broken as one that lets visitors write to it - refusing too much is the same
defect facing the other way, and there is a test for that direction too.

Half of the general hole is closed and the other half is written down. The
closed half is a test beside the route registrations: it builds the generator's
routes, enumerates them, and fails if any entry in the guard has stopped being
a real route, so renaming one turns the list red instead of quietly making it
match nothing. It lives there because common/ may not import app/ - which is
also why the guard cannot check its own list from where it is. The open half is
that no static check can tell a handler that writes from one that reads, so the
next GET that writes has to be added by hand. The comment says that rather than
leaving the impression the class is covered.

application.demomsg was configuration nothing read. The message was hard-coded
in the middleware, and the demo host's configured string happened to be
identical, so the setting looked like it worked and never had. It is read now,
with the old string kept verbatim as the fallback, so a deployment that never
set it is answered exactly as before.

This covers demo mode only. On a deployment that is not a demo those three
routes remain in CasbinExclude and stay reachable by any authenticated user
whatever their role; that is a separate decision and is not touched here.
2026-09-07 08:03:40 +08:00
wenjianzhang 7002cd4065 Merge pull request #912 from go-admin-team/feat/007-drain-window
feat✨: 关闭时先排空再停止接收——让 /ready 的 503 真的能被采到
2026-09-07 08:01:59 +08:00
32 changed files with 3376 additions and 40 deletions
+16 -1
View File
@@ -1,12 +1,25 @@
name: Build
# Documentation-only changes skip this workflow entirely.
# Documentation-only changes, and changes confined to the Kubernetes
# manifests, skip this workflow entirely.
#
# A push to master here does not just build - it pushes an image, runs the
# migrations and restarts the demo container, so the site takes a short outage.
# Paying that for a README edit is waste at best; at worst a deploy fails for a
# reason unrelated to anything in the change. Code coverage is unaffected,
# because go.yml still builds every push and pull request.
#
# scripts/k8s holds deploy.yml, storage.yml and prerun.sh, and the deploy below
# reads none of them - it is an ssh into one host that runs docker, building the
# Dockerfile at the repository root. Those manifests are for people deploying to
# a cluster of their own. The pattern is scripts/k8s/** rather than scripts/**
# because scripts/Dockerfile is a build input: go.yml builds the release image
# from it on a tag.
#
# A file outside these patterns still runs the workflow even when the rest of
# the change is ignorable: paths-ignore skips only when every changed path
# matches. Editing this file is one such case, on purpose - a deploy script
# that is never exercised by the change that broke it is worse than an outage.
on:
push:
branches: [ master ]
@@ -15,6 +28,7 @@ on:
- 'docs/**'
- 'LICENSE*'
- '.github/ISSUE_TEMPLATE/**'
- 'scripts/k8s/**'
pull_request:
branches: [ master ]
paths-ignore:
@@ -22,6 +36,7 @@ on:
- 'docs/**'
- 'LICENSE*'
- '.github/ISSUE_TEMPLATE/**'
- 'scripts/k8s/**'
# One deploy at a time. Two merges seconds apart raced here: both runs did
# docker rm -f then docker run, the second removed the container the first had
+19
View File
@@ -33,8 +33,27 @@ jobs:
--health-timeout 3s
--health-retries 10
postgres:
image: postgres:15-alpine
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: goadmin_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 3s
--health-retries 10
env:
GO_ADMIN_TEST_REDIS_ADDR: 127.0.0.1:6379
# The soft-delete conversion drops an index, and gorm's PostgreSQL driver
# produced unparseable SQL for that - on SQLite, where the rest of these
# tests run, the same code works. The suite reported success for a
# migration that failed on every PostgreSQL database it was pointed at.
# See go-admin#919.
GO_ADMIN_TEST_POSTGRES_DSN: "host=127.0.0.1 port=5432 user=postgres password=postgres dbname=goadmin_test sslmode=disable"
steps:
+89
View File
@@ -0,0 +1,89 @@
package models
import (
"time"
"go-admin/common/models"
)
// SysApp is the sys_app row model: one row per installed application (PRD
// 008 F2). It deliberately does not embed models.ModelTime - see the design
// doc (docs-prd/008-应用清单与安装器/数据库变更.md) §1.1 for why an
// installed-app registry does not need the millisecond soft-delete marker
// every other sys_* table follows. Uninstalling an app deletes its row
// outright; a later reinstall creates a fresh one.
type SysApp struct {
models.Model // Id int, primary key, autoincrement
// AppCode is the app.Manifest.Code / migration.ForApp / seed.SeedMenus
// identity, already lower-cased by migration.NormalizeAppCode before
// anything reaches this table. Unique: row existence alone answers G2
// ("is app X installed").
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;uniqueIndex:uk_sys_app_app_code;comment:app code"`
Name string `json:"name" gorm:"size:128;not null;comment:display name, from Manifest.Name"`
// Version is the version this row currently reflects - attempted or
// confirmed, disambiguated by Status. It does not drive which
// migrations run next; sys_migration's per-version rows do that (see
// design doc §1.5's resume flow). This field is descriptive, refreshed
// from the manifest on every install/upgrade/resume attempt.
Version string `json:"version" gorm:"size:32;not null;comment:version this row currently reflects, see Status"`
Description string `json:"description" gorm:"size:255;not null;default:'';comment:from Manifest.Description"`
Author string `json:"author" gorm:"size:128;not null;default:'';comment:from Manifest.Author"`
// Requires is a comma-separated list of app codes this app declared as
// dependencies (Manifest.Requires). Stored as plain VARCHAR CSV, not
// JSON - see design doc §1.3 for why. F8 (P1) is what validates and
// orders on this; this batch only stores what the manifest declared.
Requires string `json:"requires" gorm:"size:255;not null;default:'';comment:declared dependency app codes, comma separated"`
// Pricing/License are reserved passthrough fields (PRD 003; PRD 008
// open question 1). This batch stores whatever the manifest carries and
// does not interpret either one.
Pricing string `json:"pricing" gorm:"size:64;not null;default:'';comment:reserved, not interpreted by this batch"`
License string `json:"license" gorm:"size:64;not null;default:'';comment:reserved, not interpreted by this batch"`
// Status: 1=installing 2=installed 3=failed. Three states, not a
// single "1=installed", because a partial, stuck install has to be an
// observable row rather than "the row doesn't exist yet" - see design
// doc §1.5 for why cross-migration-file atomicity is not available on
// MySQL (implicit commit on DDL).
Status int `json:"status" gorm:"size:4;not null;default:1;comment:1=installing 2=installed 3=failed"`
// FailedVersion and LastError are DIAGNOSTIC TEXT ONLY - what a human
// looking at this row is told about the last failure, nothing more. No
// code anywhere may read either one to decide what to do next.
//
// The question "where should a resume pick up" has exactly one
// authoritative answer, and it is not these two columns: subtract
// sys_migration's applied rows for this app_code from what the app's
// own compiled-in code has registered (migration.Snapshot()/ForApp -
// the same set F7's `migrate status` already walks). That answer can
// never go stale, because it is not stored anywhere to go stale - it is
// recomputed from sys_migration every time it is asked. FailedVersion
// is a snapshot of what that computation returned at the moment of
// failure, kept only so an operator does not have to go find the
// process's logs; if it and a fresh recomputation from sys_migration
// ever disagree, sys_migration is right and this column is stale, by
// definition, and nothing should ever notice or care except a human
// reading the row.
FailedVersion string `json:"failedVersion" gorm:"size:64;not null;default:'';comment:diagnostic snapshot only, not a judgment basis; meaningful only when status=3"`
LastError string `json:"lastError" gorm:"size:255;not null;default:'';comment:diagnostic text only, not a judgment basis; meaningful only when status=3"`
// InstalledAt is when this app first reached status=installed - set
// once, never moved by a later upgrade (see design doc §1.4). Nullable,
// unlike every other column here: a row can exist before it has a
// value (a fresh install starts at status=installing). This is not the
// deleted_at problem 1786700003000_soft_delete_marker.go fixed - that
// column sat inside a unique index, where NULL <> NULL let two live
// rows coexist under the same key. InstalledAt is in no index at all,
// so nullability here opens no such hole.
InstalledAt *time.Time `json:"installedAt" gorm:"comment:first successful install time; null until status first reaches installed"`
UpdatedAt time.Time `json:"updatedAt" gorm:"comment:last updated time"`
models.ControlBy // CreateBy/UpdateBy: which operator triggered the attempt
}
func (*SysApp) TableName() string {
return "sys_app"
}
+43
View File
@@ -0,0 +1,43 @@
package models
import "time"
// SysAppCasbinGrant is a ledger of casbin_rule rows an app install created,
// keyed by the exact natural key casbin_rule itself is unique on. It exists
// because casbin_rule is not a table this project owns (see design doc
// docs-prd/008-应用清单与安装器/数据库变更.md §2.2): we cannot add an
// app_code column to it without that column being silently zeroed the first
// time anything calls the gorm-adapter's SavePolicy/SavePolicyCtx. Recording
// the natural key here, instead of a foreign key into casbin_rule, is also
// what survives SysRole.Update's RemoveFilteredPolicy+re-add cycle for a
// role's policies (app/admin/service/sys_role.go): that cycle replaces the
// underlying row (a new auto-increment ID) but reproduces the same
// (ptype,v0,v1,v2) tuple from the same sys_menu/sys_api data, so a
// natural-key match here still finds it. What it does not survive is the
// role being renamed, or the tuple being rebuilt from a completely different
// source (a future SavePolicy call from outside this seeder) - in both cases
// the match legitimately fails, and business rule 3 says the uninstaller
// should report and skip, not delete something else that happens to look
// the same.
type SysAppCasbinGrant struct {
Id int `json:"id" gorm:"primaryKey;autoIncrement"`
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;index:idx_sys_app_casbin_grant_app_code;comment:app code that created this grant"`
// Column widths mirror gorm-adapter's own CasbinRule struct exactly, so
// a value that fits into casbin_rule always fits here, and the unique
// index below matches the one createTable() puts on casbin_rule itself.
Ptype string `json:"ptype" gorm:"size:100;not null;uniqueIndex:uk_sys_app_casbin_grant_rule;comment:casbin ptype, 'p' today"`
V0 string `json:"v0" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:role_key at grant time"`
V1 string `json:"v1" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:api path"`
V2 string `json:"v2" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:http method"`
V3 string `json:"v3" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
V4 string `json:"v4" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
V5 string `json:"v5" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
CreatedAt time.Time `json:"createdAt" gorm:"comment:when this grant was recorded"`
}
func (*SysAppCasbinGrant) TableName() string {
return "sys_app_casbin_grant"
}
+13
View File
@@ -32,6 +32,19 @@ type SysMenu struct {
// AutoMigrate adding this column to an existing table leaves every
// pre-existing row reading back as "" rather than NULL.
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;default:'';index:idx_sys_menu_app_code;comment:AppCode"`
// SeedCode is the raw seed.MenuSpec.Code this row was created from, kept
// so seedMenuTree can ask "did I already write this node" without
// relying on MenuName's PascalCase concatenation, which is not
// injective (see design doc §1.6). Nullable, unlike AppCode: every row
// seed.SeedMenus writes sets a real value, but every pre-existing row -
// the host's own hand-placed menus, and every app-seeded row written
// before this column existed - has none, and there is no way to
// backfill one that means anything. NULL is what lets an unbounded
// number of those coexist under the same app_code without tripping the
// unique index below: the database never treats two NULLs as equal, so
// only rows that do carry a real code participate in the uniqueness
// check at all.
SeedCode *string `json:"seedCode" gorm:"size:64;uniqueIndex:uk_sys_menu_app_seed_code_del;comment:raw MenuSpec.Code, null for rows not written through SeedMenus"`
models.ControlBy
models.ModelTime
}
+152 -8
View File
@@ -90,6 +90,23 @@ func (adminSeeder) SeedMenus(tx *gorm.DB, appCode string, menus []seed.MenuSpec,
// application's ids in the module cache. Never accepting a caller-chosen id
// here removes the collision this Seeder has no way to detect instead of
// trying to detect it after the fact.
//
// The natural key is (app_code, path, action) - the same three columns
// 1786700002000_remove_refresh_token_api.go already used to identify a
// single API by hand, and the ones 1786700008000_seed_natural_keys.go put a
// unique index on. Before inserting, this looks for a live row (deleted_at
// = 0, applied automatically by the soft-delete plugin on every query
// against models.SysApi) already holding that key and reuses it instead of
// inserting a second one - see the design doc §1.6: a migration retried
// after a partial failure previously re-ran this as a bare tx.Create and
// produced duplicate rows on the demo site.
//
// Unlike seedMenuTree's reuse branch, this one has nothing left to repair
// after finding an existing row: models.SysApi carries no association
// (nothing like SysMenu's many2many SysApi field) and this function writes
// nothing beyond the row itself - no second statement comparable to
// seedMenuTree's paths UPDATE follows tx.Create below. An interrupted retry
// can therefore only ever find this row complete or not find it at all.
func seedApis(tx *gorm.DB, appCode string, apis []seed.ApiSpec) (map[string]models.SysApi, error) {
seen := make(map[string]bool, len(apis))
rows := make(map[string]models.SysApi, len(apis))
@@ -102,6 +119,19 @@ func seedApis(tx *gorm.DB, appCode string, apis []seed.ApiSpec) (map[string]mode
}
seen[a.Code] = true
var existing models.SysApi
err := tx.Where("app_code = ? AND path = ? AND action = ?", appCode, a.Path, a.Method).
First(&existing).Error
switch {
case err == nil:
rows[a.Code] = existing
continue
case errors.Is(err, gorm.ErrRecordNotFound):
// Not seen yet; fall through to insert it.
default:
return nil, fmt.Errorf("api %q: checking for an existing row: %w", a.Code, err)
}
row := models.SysApi{
Handle: a.Handle,
Title: a.Title,
@@ -153,6 +183,12 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
continue
}
// Resolved before the idempotency check below, whether or not
// this spec's own row turns out to already exist: repairing an
// existing-but-incomplete row's paths needs the parent's
// already-resolved Paths exactly as much as creating a fresh
// row does (see repairExistingMenu), so both have to wait for
// it the same way.
var parentRow models.SysMenu
if s.Parent != "" {
parent, ok := created[s.Parent]
@@ -165,6 +201,32 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
parentRow = parent
}
// Idempotency check: does this node already have a row, from
// an earlier, possibly-interrupted attempt? The natural key is
// (app_code, seed_code) - menu_name's PascalCase concatenation
// is not injective and cannot be used for this (see menuName's
// doc comment and the design doc §1.6). Only a live row counts;
// the soft-delete plugin scopes deleted_at = 0 automatically on
// every query against models.SysMenu.
var existing models.SysMenu
err := tx.Where("app_code = ? AND seed_code = ?", appCode, s.Code).First(&existing).Error
switch {
case err == nil:
row, err := repairExistingMenu(tx, existing, s, parentRow, apiRows)
if err != nil {
return nil, fmt.Errorf("%q: repairing an existing row: %w", s.Code, err)
}
created[s.Code] = row
ids = append(ids, row.MenuId)
progressed = true
continue
case errors.Is(err, gorm.ErrRecordNotFound):
// Not written yet; fall through to create it below.
default:
return nil, fmt.Errorf("%q: checking for an existing row: %w", s.Code, err)
}
seedCode := s.Code
row := models.SysMenu{
MenuName: menuName(appCode, s.Code),
Title: s.Title,
@@ -179,9 +241,10 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
// 1786700001000_demo_menu.go seeds its own menu with. A
// freshly installed application's menu should not need an
// administrator to first find and unhide it.
Visible: "0",
IsFrame: "1",
AppCode: appCode,
Visible: "0",
IsFrame: "1",
AppCode: appCode,
SeedCode: &seedCode,
}
for _, code := range s.ApiCodes {
api, ok := apiRows[code]
@@ -205,11 +268,7 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
// two-step create-then-update 1786700001000_demo_menu.go's
// hand-assigned ids let it do in one literal, sequenced here
// instead.
if s.Parent == "" {
row.Paths = "/0/" + strconv.Itoa(row.MenuId)
} else {
row.Paths = parentRow.Paths + "/" + strconv.Itoa(row.MenuId)
}
row.Paths = expectedPaths(row.MenuId, s.Parent, parentRow)
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", row.MenuId).
Update("paths", row.Paths).Error; err != nil {
return nil, fmt.Errorf("%q: writing paths: %w", s.Code, err)
@@ -226,6 +285,91 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
return ids, nil
}
// expectedPaths is the materialized path a fresh insert of menuID under
// parent (or at the root, if parent is "") computes - factored out so
// repairExistingMenu can ask the same question about a row it did not just
// create.
func expectedPaths(menuID int, parent string, parentRow models.SysMenu) string {
if parent == "" {
return "/0/" + strconv.Itoa(menuID)
}
return parentRow.Paths + "/" + strconv.Itoa(menuID)
}
// repairExistingMenu brings a row seedMenuTree's idempotency check found up
// to what a fresh insert of the same spec would have produced.
//
// A row can be found and still be incomplete: tx.Create's own association
// write (the sys_menu_api_rule bindings from row.SysApi) and the paths
// UPDATE that follows it are each their own statement, and design doc §1.5
// establishes that nothing after the first DDL in a migration function can
// be rolled back together - a process interrupted between the row insert
// and either of those two steps leaves exactly this row: present, findable
// by its natural key, but missing what makes it a working menu entry. A
// retry that only checked "does the row exist" and stopped there would
// report success while the sys_menu_api_rule binding stays missing (the
// api is granted to no one) or paths stays empty (a materialized-path
// break that orphans the rest of the subtree from the root) - as silent as
// the duplicate-row defect the idempotency check itself was written to
// close.
//
// Both checks are read-before-write, so a row that is already complete -
// the ordinary case on every retry after the first successful one - causes
// no writes at all: existing.Paths already equals what expectedPaths
// computes, and the sys_menu_api_rule INSERT is itself guarded by
// WHERE NOT EXISTS, the same idempotent-insert shape grantToAdminRole
// already uses for sys_role_menu/casbin_rule. Never DELETEs an existing
// binding to rebuild it - that is the FullSaveAssociations mistake
// sys_role.go's SysRole.Update makes for sys_role_menu/casbin_rule
// (app/admin/service/sys_role.go:148-153), the exact pattern this design
// went out of its way to avoid for the tables that do use it.
//
// Insert-only cuts both ways, deliberately. A binding an administrator
// added by hand through the menu management UI, for an api never in
// s.ApiCodes at all, is never touched by this loop and survives every
// later retry (TestSeedMenusPreservesAHandAddedBinding is the reproduction
// case for the opposite mistake: delete-then-reinsert wipes it silently,
// the same shape as sys_role_menu/casbin_rule getting zeroed by a role
// edit, just with this code as the actor instead of the victim). The
// converse case - a MenuSpec that used to list an ApiCode and no longer
// does - is not handled here either, and that half is intentional rather
// than an oversight: this loop only ever adds rows for codes the *current*
// call's ApiCodes names, so a binding for a code an earlier version
// granted and the current one dropped is left in place, stale. Reconciling
// that is deleting something, which needs the same certainty about
// ownership uninstall's design (see design doc §5) already requires -
// this function has no way to tell "stale, from an older version of this
// same app" apart from "hand-added, for a reason", and business rule 3
// ("uninstall deletes only what it can attribute with certainty") applies
// here just as much as it does there. Reconciling stale seed-driven
// bindings, if it is ever wanted, belongs in the upgrade path with that
// same ownership check - not silently inside every retry of every install.
func repairExistingMenu(tx *gorm.DB, existing models.SysMenu, s seed.MenuSpec, parentRow models.SysMenu, apiRows map[string]models.SysApi) (models.SysMenu, error) {
want := expectedPaths(existing.MenuId, s.Parent, parentRow)
if existing.Paths != want {
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", existing.MenuId).
Update("paths", want).Error; err != nil {
return models.SysMenu{}, fmt.Errorf("repairing paths: %w", err)
}
existing.Paths = want
}
for _, code := range s.ApiCodes {
api, ok := apiRows[code]
if !ok {
return models.SysMenu{}, fmt.Errorf("ApiCodes references %q, which is not an ApiSpec.Code in this call", code)
}
if err := tx.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) SELECT ?, ? WHERE NOT EXISTS (SELECT 1 FROM sys_menu_api_rule WHERE sys_menu_menu_id = ? AND sys_api_id = ?)",
existing.MenuId, api.Id, existing.MenuId, api.Id,
).Error; err != nil {
return models.SysMenu{}, fmt.Errorf("binding %q: %w", code, err)
}
}
return existing, nil
}
// validateMenuSpec rejects the malformed input tools/checksilent's
// menu-sort-overflow and Kind-adjacent checks would catch for an in-tree
// seed but cannot for a third-party application's - see menuSortRange's doc
+552
View File
@@ -1,13 +1,17 @@
package service
import (
"context"
"errors"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
gormlogger "gorm.io/gorm/logger"
contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/seed"
@@ -289,3 +293,551 @@ func TestSeedMenusWithNothingRegisteredWritesNothing(t *testing.T) {
}
}
}
// newSeedTestDB's AutoMigrate builds a unique index on seed_code alone,
// because SysMenu.SeedCode is the only field in the struct carrying the
// uk_sys_menu_app_seed_code_del tag - app_code already carries a different,
// non-unique index name of its own, and the embedded ModelTime's
// DeletedAt (aliased from go-admin-core) cannot be given a third one. The
// real migration (cmd/migrate/migration/version/1786700008000_seed_natural_keys.go)
// never lets AutoMigrate touch this table for exactly that reason: it
// builds the composite (app_code, seed_code, deleted_at) index by hand
// instead. Reproduce that by hand here too, so a test that seeds two rows
// sharing a seed_code under different deleted_at values sees what a real
// install would, not gorm's narrower default.
func useCompositeSeedCodeIndex(t *testing.T, db *gorm.DB) {
t.Helper()
if db.Migrator().HasIndex(&models.SysMenu{}, "uk_sys_menu_app_seed_code_del") {
if err := db.Migrator().DropIndex(&models.SysMenu{}, "uk_sys_menu_app_seed_code_del"); err != nil {
t.Fatalf("drop the single-column seed_code index: %v", err)
}
}
if err := db.Exec(
"CREATE UNIQUE INDEX uk_sys_menu_app_seed_code_del ON sys_menu (app_code, seed_code, deleted_at)",
).Error; err != nil {
t.Fatalf("create the composite seed_code index: %v", err)
}
}
// A retried migration - one that failed partway through and is run again,
// or simply run twice by mistake - must not create a second sys_api or
// sys_menu row for the same (appCode, natural key). This is the defect the
// demo site hit in production: duplicate sys_menu/casbin_rule rows from a
// bare tx.Create on a natural key nothing was checking.
func TestSeedMenusIsIdempotentAcrossARetry(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{
{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10},
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: "Orders", Sort: 1, ApiCodes: []string{"list"}},
}
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"},
}
run := func() {
t.Helper()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
}
run()
firstMenuIDs := allMenuIDs(t, db, "order")
firstApiIDs := allApiIDs(t, db, "order")
run() // the retry
if got := allMenuIDs(t, db, "order"); !sameIDs(got, firstMenuIDs) {
t.Errorf("sys_menu ids after retry = %v, want unchanged %v (a second call inserted new rows)", got, firstMenuIDs)
}
if got := allApiIDs(t, db, "order"); !sameIDs(got, firstApiIDs) {
t.Errorf("sys_api ids after retry = %v, want unchanged %v (a second call inserted new rows)", got, firstApiIDs)
}
assertRowCount(t, db, "sys_api", 1)
assertRowCount(t, db, "sys_menu", 2)
assertRowCount(t, db, "sys_menu_api_rule", 1)
assertRowCount(t, db, "sys_role_menu", 2)
assertRowCount(t, db, "casbin_rule", 1)
}
// Only a live row counts as "already written". A row a prior, unrelated
// soft-delete already retired must not be reused - seedApis/seedMenuTree
// have to insert a fresh one under the same natural key, the same way the
// unique indexes 1786700008000_seed_natural_keys.go builds only bind live
// rows.
func TestSeedMenusOnlyReusesLiveRows(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10}}
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
// Soft-delete both rows this first call wrote, as if an operator (or an
// earlier uninstall) had retired them, independently of this migration
// ever running again.
if err := db.Exec("UPDATE sys_menu SET deleted_at = 1").Error; err != nil {
t.Fatalf("soft-delete sys_menu: %v", err)
}
if err := db.Exec("UPDATE sys_api SET deleted_at = 1").Error; err != nil {
t.Fatalf("soft-delete sys_api: %v", err)
}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus after soft-delete: %v", err)
}
// Two rows total: the soft-deleted original, plus a fresh one - not the
// dead row resurrected in place, and not left with zero live rows.
assertRowCount(t, db, "sys_menu", 2)
assertRowCount(t, db, "sys_api", 2)
var liveMenus, liveApis int64
db.Model(&models.SysMenu{}).Where("app_code = ?", "order").Count(&liveMenus)
db.Model(&models.SysApi{}).Where("app_code = ?", "order").Count(&liveApis)
if liveMenus != 1 {
t.Errorf("live sys_menu rows = %d, want 1", liveMenus)
}
if liveApis != 1 {
t.Errorf("live sys_api rows = %d, want 1", liveApis)
}
}
// app_code is part of the natural key, not a descriptive column alongside
// it. Two applications that happen to register an identical (path, action)
// or seed_code must each get their own row - reusing one app's row for
// another's install would make an uninstall of the first delete a row the
// second considers its own.
func TestSeedMenusScopesTheNaturalKeyByAppCode(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{{Code: "dir", Kind: contractmodels.Directory, Title: "Dir", Sort: 10}}
apis := []seed.ApiSpec{{Code: "list", Title: "Shared endpoint", Path: "/api/v1/shared", Method: "GET"}}
for _, appCode := range []string{"order", "billing"} {
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, appCode, menus, apis)
}); err != nil {
t.Fatalf("SeedMenus(%q): %v", appCode, err)
}
}
var apiRows []models.SysApi
if err := db.Where("path = ? AND action = ?", "/api/v1/shared", "GET").
Order("app_code").Find(&apiRows).Error; err != nil {
t.Fatalf("read sys_api: %v", err)
}
if len(apiRows) != 2 {
t.Fatalf("sys_api has %d row(s) for the shared (path, action), want 2 - one per app", len(apiRows))
}
if apiRows[0].AppCode != "billing" || apiRows[1].AppCode != "order" {
t.Errorf("sys_api app_codes = [%s %s], want [billing order]", apiRows[0].AppCode, apiRows[1].AppCode)
}
var menuRows []models.SysMenu
if err := db.Where("seed_code = ?", "dir").Order("app_code").Find(&menuRows).Error; err != nil {
t.Fatalf("read sys_menu: %v", err)
}
if len(menuRows) != 2 {
t.Fatalf("sys_menu has %d row(s) for the shared seed_code, want 2 - one per app", len(menuRows))
}
if menuRows[0].AppCode != "billing" || menuRows[1].AppCode != "order" {
t.Errorf("sys_menu app_codes = [%s %s], want [billing order]", menuRows[0].AppCode, menuRows[1].AppCode)
}
}
func allMenuIDs(t *testing.T, db *gorm.DB, appCode string) []int {
t.Helper()
var rows []models.SysMenu
if err := db.Where("app_code = ?", appCode).Order("menu_id").Find(&rows).Error; err != nil {
t.Fatalf("read sys_menu: %v", err)
}
ids := make([]int, len(rows))
for i, r := range rows {
ids[i] = r.MenuId
}
return ids
}
func allApiIDs(t *testing.T, db *gorm.DB, appCode string) []int {
t.Helper()
var rows []models.SysApi
if err := db.Where("app_code = ?", appCode).Order("id").Find(&rows).Error; err != nil {
t.Fatalf("read sys_api: %v", err)
}
ids := make([]int, len(rows))
for i, r := range rows {
ids[i] = r.Id
}
return ids
}
func sameIDs(a, b []int) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
func assertRowCount(t *testing.T, db *gorm.DB, table string, want int64) {
t.Helper()
var n int64
if err := db.Table(table).Count(&n).Error; err != nil {
t.Fatalf("count %s: %v", table, err)
}
if n != want {
t.Errorf("%s has %d row(s), want %d", table, n, want)
}
}
// A retried migration does not just risk inserting a second copy of a row
// it already wrote (that gap is closed above) - the reuse path itself has
// to leave the row in the same state a fresh insert would have. Before
// this defect was fixed, the reuse branch (seed.go's "case err == nil")
// stopped at reusing the row's id and skipped everything a fresh insert
// does afterwards: the sys_menu_api_rule binding gorm's association save
// writes as part of Create, and the paths UPDATE that follows Create as a
// separate statement. A row a prior attempt inserted but did not finish -
// exactly the shape design doc §1.5 says a non-transactional retry can
// leave behind - would be "found" and then left broken forever, with the
// migration reporting success.
//
// existingHalfWrittenMenu inserts a sys_menu row the way seedMenuTree's own
// tx.Create leaves one when interrupted immediately afterwards: the row
// exists with its natural key, but paths was never computed and no
// sys_menu_api_rule binding was ever written for it - Create's association
// save and the paths UPDATE are each a separate statement from the row
// insert itself.
func existingHalfWrittenMenu(t *testing.T, db *gorm.DB, appCode, seedCode string, parentID int) models.SysMenu {
t.Helper()
code := seedCode
row := models.SysMenu{
MenuName: menuName(appCode, seedCode),
AppCode: appCode,
SeedCode: &code,
ParentId: parentID,
Visible: "0",
IsFrame: "1",
// Paths deliberately left "" - never computed, the same as a row
// whose Create succeeded but whose follow-up paths UPDATE never ran.
}
if err := db.Create(&row).Error; err != nil {
t.Fatalf("seed half-written menu %q: %v", seedCode, err)
}
return row
}
func bindingCount(t *testing.T, db *gorm.DB, menuID, apiID int) int64 {
t.Helper()
var n int64
if err := db.Table("sys_menu_api_rule").
Where("sys_menu_menu_id = ? AND sys_api_id = ?", menuID, apiID).Count(&n).Error; err != nil {
t.Fatalf("count sys_menu_api_rule: %v", err)
}
return n
}
// TestSeedMenusRepairsAnIncompleteExistingRow is the reproduction case:
// both paths and the api binding are missing on the row seedMenuTree finds
// through its idempotency check, the shape a real interrupted retry leaves
// behind. Run against the unfixed reuse branch, this must fail - that is
// what proves the defect is real rather than a three-way guess.
func TestSeedMenusRepairsAnIncompleteExistingRow(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
dir := existingHalfWrittenMenu(t, db, "order", "dir", 0)
list := existingHalfWrittenMenu(t, db, "order", "list", dir.MenuId)
menus := []seed.MenuSpec{
{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10},
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: "Orders", Sort: 1, ApiCodes: []string{"list"}},
}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
wantDirPaths := "/0/" + strconv.Itoa(dir.MenuId)
wantListPaths := wantDirPaths + "/" + strconv.Itoa(list.MenuId)
var gotDir, gotList models.SysMenu
if err := db.First(&gotDir, dir.MenuId).Error; err != nil {
t.Fatalf("read dir: %v", err)
}
if err := db.First(&gotList, list.MenuId).Error; err != nil {
t.Fatalf("read list: %v", err)
}
if gotDir.Paths != wantDirPaths {
t.Errorf("dir.Paths = %q, want %q - a retried install left a root menu with no materialized path", gotDir.Paths, wantDirPaths)
}
if gotList.Paths != wantListPaths {
t.Errorf("list.Paths = %q, want %q - a retried install left the seeded subtree with a broken materialized path", gotList.Paths, wantListPaths)
}
if n := bindingCount(t, db, list.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("sys_menu_api_rule binding count for list = %d, want 1 - a retried install left the menu with its api granted to no one", n)
}
}
// soloMenuSpec is a single, parent-less menu with one api binding - the
// smallest shape that can exhibit "paths wrong" and "binding missing"
// independently of each other, used by the three tests below to isolate
// one repair path at a time from TestSeedMenusRepairsAnIncompleteExistingRow's
// combined (both broken) case.
func soloMenuSpec() []seed.MenuSpec {
return []seed.MenuSpec{{Code: "solo", Kind: contractmodels.Menu, Title: "Solo", Sort: 1, ApiCodes: []string{"list"}}}
}
// Only the binding is missing; paths is already correct. The repair must
// add the binding and must not touch the already-correct paths value.
func TestSeedMenusRepairsOnlyAMissingBinding(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
solo := existingHalfWrittenMenu(t, db, "order", "solo", 0)
wantPaths := "/0/" + strconv.Itoa(solo.MenuId)
if err := db.Model(&models.SysMenu{}).Where("menu_id = ?", solo.MenuId).
Update("paths", wantPaths).Error; err != nil {
t.Fatalf("set paths: %v", err)
}
// The binding is deliberately left unwritten.
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", soloMenuSpec(), apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
var got models.SysMenu
if err := db.First(&got, solo.MenuId).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
if got.Paths != wantPaths {
t.Errorf("paths changed from %q to %q; repairing a missing binding must not touch an already-correct path", wantPaths, got.Paths)
}
if n := bindingCount(t, db, solo.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("binding count = %d, want 1", n)
}
}
// Only paths is missing; the binding already exists (as if Create's own
// association write had succeeded but the paths UPDATE that follows it
// never ran). The repair must fix paths and must not duplicate the
// already-correct binding.
func TestSeedMenusRepairsOnlyMissingPaths(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
solo := existingHalfWrittenMenu(t, db, "order", "solo", 0)
if err := db.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) VALUES (?, ?)",
solo.MenuId, apiRows["list"].Id,
).Error; err != nil {
t.Fatalf("seed binding: %v", err)
}
// solo.Paths is deliberately left "" by existingHalfWrittenMenu.
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", soloMenuSpec(), apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
wantPaths := "/0/" + strconv.Itoa(solo.MenuId)
var got models.SysMenu
if err := db.First(&got, solo.MenuId).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
if got.Paths != wantPaths {
t.Errorf("paths = %q, want %q", got.Paths, wantPaths)
}
if n := bindingCount(t, db, solo.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("binding count = %d, want 1 - repairing paths must not duplicate an already-correct binding", n)
}
}
// capturingLogger records every SQL statement gorm actually executes, so a
// test can assert that a fully-consistent retry performs no write at all -
// not just that its net effect happens to be zero rows changed. Mirrors
// common/actions/crud_shim_test.go's logger of the same name and shape;
// duplicated locally rather than exported and shared, matching how small
// gorm-facing test doubles are kept next to the test that needs them
// elsewhere in this repository.
type capturingLogger struct {
gormlogger.Interface
mu sync.Mutex
stmts []string
}
func (l *capturingLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
sql, _ := fc()
l.mu.Lock()
l.stmts = append(l.stmts, sql)
l.mu.Unlock()
}
func (l *capturingLogger) all() string {
l.mu.Lock()
defer l.mu.Unlock()
return strings.Join(l.stmts, "\n")
}
// Both paths and the binding are already correct - the ordinary shape of
// every retry after the first one succeeds in full. Repairing an
// already-consistent row must not touch it: paths is read-before-write and
// so must not be UPDATEd at all (asserted directly, by statement, since the
// code gates that call behind a value comparison); the binding's own
// insert is guarded by WHERE NOT EXISTS the same way grantToAdminRole's
// already are, so its row count staying put is the meaningful claim - the
// guarded statement itself may still be sent, the same way it already is
// for sys_role_menu/casbin_rule.
func TestSeedMenusFullyConsistentRowCausesNoPathsUpdate(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
menus := soloMenuSpec()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (first): %v", err)
}
var apiRows []models.SysApi
db.Where("app_code = ?", "order").Find(&apiRows)
var soloRow models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "solo").First(&soloRow).Error; err != nil {
t.Fatalf("read solo after first call: %v", err)
}
if soloRow.Paths == "" {
t.Fatalf("solo.Paths is empty after the first call; the fixture itself is broken, not what this test means to check")
}
wantBindings := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id)
if wantBindings != 1 {
t.Fatalf("binding count after the first call = %d, want 1; the fixture itself is broken", wantBindings)
}
capturing := &capturingLogger{Interface: gormlogger.Default.LogMode(gormlogger.Info)}
captured := db.Session(&gorm.Session{Logger: capturing})
if err := captured.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (retry): %v", err)
}
all := strings.ToUpper(capturing.all())
if strings.Contains(all, "UPDATE") && strings.Contains(all, "SYS_MENU") && strings.Contains(all, "PATHS") {
t.Errorf("a fully consistent retry executed a paths UPDATE against sys_menu:\n%s", capturing.all())
}
if got := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id); got != 1 {
t.Errorf("binding count after the retry = %d, want 1 (unchanged)", got)
}
}
// An administrator can bind a menu to an additional api by hand through
// the menu management UI - a sys_menu_api_rule row for an api never in
// s.ApiCodes at all. A retried SeedMenus call must not touch it: deleting
// every binding for the menu and reinserting only what s.ApiCodes lists
// would wipe it out silently, the same shape as sys_role_menu/casbin_rule
// getting zeroed by SysRole.Update's FullSaveAssociations save - just with
// this code as the actor instead of the victim this time.
func TestSeedMenusPreservesAHandAddedBinding(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
menus := soloMenuSpec()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (first): %v", err)
}
var soloRow models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "solo").First(&soloRow).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
// An api this call's ApiSpec list never mentions - standing in for one
// belonging to some other feature entirely, bound to this menu by an
// administrator, not by any SeedMenus call.
handAdded := models.SysApi{Path: "/api/v1/order/export", Action: "GET", Type: "SYS", AppCode: "order"}
if err := db.Create(&handAdded).Error; err != nil {
t.Fatalf("seed the hand-added api: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) VALUES (?, ?)",
soloRow.MenuId, handAdded.Id,
).Error; err != nil {
t.Fatalf("seed the hand-added binding: %v", err)
}
// A retry with the exact same specs - solo's ApiCodes still names only
// "list".
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (retry): %v", err)
}
if n := bindingCount(t, db, soloRow.MenuId, handAdded.Id); n != 1 {
t.Errorf("hand-added binding count = %d, want 1 - a retry silently deleted a binding it does not own", n)
}
var apiRows []models.SysApi
db.Where("app_code = ? AND path = ?", "order", "/api/v1/order").Find(&apiRows)
if len(apiRows) != 1 {
t.Fatalf("seeded api not found as expected: %+v", apiRows)
}
if n := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id); n != 1 {
t.Errorf("the seed's own binding count = %d, want 1 - it must survive the retry too", n)
}
}
+91
View File
@@ -0,0 +1,91 @@
package router
import (
"testing"
"github.com/gin-gonic/gin"
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"go-admin/common/middleware"
)
// registeredRoutes builds the generator's routes on an engine of its own and
// reports the patterns they were registered under.
//
// The mode has to be given rather than inherited, because it now decides what
// gets registered: a test that leaves it at the zero value would be asking
// about a mode no deployment runs in, and would pass whether or not the gate
// works.
//
// It is put back before this returns, not at the end of the test. t.Cleanup
// would leave the mode set for everything the caller does afterwards, so a
// caller that went on to assert something mode-dependent would be reading a
// value this helper left behind rather than one it chose. A caller that does
// want the mode set has to set it, which is visible where it happens.
//
// The JWT middleware is a zero value. MiddlewareFunc only closes over the
// receiver and is never called here - no request is served, the engine is
// asked what it has - so nothing dereferences it.
func registeredRoutes(t *testing.T, mode string) map[string]bool {
t.Helper()
gin.SetMode(gin.TestMode)
previous := config.ApplicationConfig.Mode
defer func() { config.ApplicationConfig.Mode = previous }()
config.ApplicationConfig.Mode = mode
r := gin.New()
v1 := r.Group("/api/v1")
sysNoCheckRoleRouter(v1, &jwt.GinJWTMiddleware{})
registerDBRouter(v1, &jwt.GinJWTMiddleware{})
out := map[string]bool{}
for _, route := range r.Routes() {
out[route.Path] = true
}
return out
}
// The list of routes demo mode refuses lives in common/middleware, which may
// not import app/ and therefore cannot see whether any of them is still a
// route. This is the half that can be checked, and it is checked here because
// this is where the routes are declared: rename one, and the entry over there
// stops matching anything, demo mode silently starts serving it again, and
// nothing else would say so.
func TestEveryRouteDemoModeRefusesStillExists(t *testing.T) {
routes := registeredRoutes(t, "demo")
for _, guarded := range middleware.DemoWriteRoutes() {
if !routes[guarded] {
t.Errorf("demo mode refuses %q, but no route is registered under that pattern - "+
"either it was renamed, or it moved to another file; the guard now matches nothing",
guarded)
}
}
}
// The other direction, and the one the demo host cares about: the generator's
// read-only routes have to stay reachable, or a demo deployment cannot show
// the feature at all. Refusing too much is as much of a defect as refusing too
// little.
func TestTheGeneratorsReadOnlyRoutesAreNotRefused(t *testing.T) {
refused := map[string]bool{}
for _, guarded := range middleware.DemoWriteRoutes() {
refused[guarded] = true
}
for _, readOnly := range []string{
"/api/v1/gen/preview/:tableId",
"/api/v1/gen/tabletree",
"/api/v1/db/tables/page",
"/api/v1/db/columns/page",
} {
if !registeredRoutes(t, "demo")[readOnly] {
t.Fatalf("%s is not registered, so this test is asserting against nothing", readOnly)
}
if refused[readOnly] {
t.Errorf("demo mode refuses %s, which only reads - the demo host needs it to "+
"demonstrate the generator", readOnly)
}
}
}
+41 -5
View File
@@ -3,15 +3,49 @@ package router
import (
"github.com/gin-gonic/gin"
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"go-admin/app/admin/apis"
"go-admin/app/other/apis/tools"
)
// GenWriteRoutesEnabled reports whether the code generator's writing endpoints
// are registered in this process.
//
// Three of the generator's endpoints do not read. /gen/toproject writes seven
// Go and Vue source files onto this host, one of them under the path
// gen.frontpath names; /gen/apitofile writes a migration; /gen/todb inserts
// menus and APIs. All three are GET, all three are listed in CasbinExclude,
// and AuthCheckRole skips what is on that list - so Enforce never runs for
// them and any account that can log in may call them. That is a bargain a
// workstation can make and a deployment cannot.
//
// dev is the shipped default and is where the generator is meant to be used.
// demo keeps them because demo mode already has a better answer than a 404:
// DemoEvn refuses these three by name and explains itself, which is the thing
// the demo exists to show. test and prod get nothing.
//
// The mode is read once, while the routes are being built. Changing
// application.mode in a running process adds and removes nothing - a
// configuration reload rebuilds neither the engine nor its routes.
//
// core has constants for dev, test and prod but none for demo, which this
// repository spells as a literal in common/middleware/demo.go. Both are
// literals here so that the two read as one set rather than two conventions.
func GenWriteRoutesEnabled() bool {
switch config.ApplicationConfig.Mode {
case "dev", "demo":
return true
default:
return false
}
}
func init() {
routerCheckRole = append(routerCheckRole, sysNoCheckRoleRouter, registerDBRouter, registerSysTableRouter)
}
func sysNoCheckRoleRouter(v1 *gin.RouterGroup ,authMiddleware *jwt.GinJWTMiddleware) {
func sysNoCheckRoleRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddleware) {
r1 := v1.Group("")
{
sys := apis.System{}
@@ -22,9 +56,11 @@ func sysNoCheckRoleRouter(v1 *gin.RouterGroup ,authMiddleware *jwt.GinJWTMiddlew
{
gen := tools.Gen{}
r.GET("/gen/preview/:tableId", gen.Preview)
r.GET("/gen/toproject/:tableId", gen.GenCode)
r.GET("/gen/apitofile/:tableId", gen.GenApiToFile)
r.GET("/gen/todb/:tableId", gen.GenMenuAndApi)
if GenWriteRoutesEnabled() {
r.GET("/gen/toproject/:tableId", gen.GenCode)
r.GET("/gen/apitofile/:tableId", gen.GenApiToFile)
r.GET("/gen/todb/:tableId", gen.GenMenuAndApi)
}
sysTable := tools.SysTable{}
r.GET("/gen/tabletree", sysTable.GetSysTablesTree)
}
@@ -53,4 +89,4 @@ func registerSysTableRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddl
tablesInfo.GET("", sysTable.GetSysTablesInfo)
}
}
}
}
+151
View File
@@ -0,0 +1,151 @@
package router
import (
"testing"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
// genWritingRoutes are the three that do not read. They write Go and Vue
// source onto the host, a migration, and rows in sys_menu.
var genWritingRoutes = []string{
"/api/v1/gen/toproject/:tableId",
"/api/v1/gen/apitofile/:tableId",
"/api/v1/gen/todb/:tableId",
}
// genReadingRoutes are the rest of the generator's surface. Gating the three
// above must not cost any of these: a deployment that cannot list its tables
// or preview a template has lost the feature, not secured it.
var genReadingRoutes = []string{
"/api/v1/gen/preview/:tableId",
"/api/v1/gen/tabletree",
"/api/v1/db/tables/page",
"/api/v1/db/columns/page",
}
// The endpoints that write are registered where the mode says development and
// nowhere else.
//
// They are in CasbinExclude, so Enforce never runs for them and any account
// that can log in may call them. dev is the shipped default and is where the
// generator is meant to be used; demo keeps them because DemoEvn refuses these
// three by name and saying so is the thing the demo is for. Everything else,
// including the empty mode a process gets when nothing set one, is refused by
// not existing.
func TestGeneratorWritingRoutesExistOnlyWhereTheModeAllowsIt(t *testing.T) {
for _, tc := range []struct {
mode string
expected bool
why string
}{
{"dev", true, "the shipped default, and where the generator is used"},
{"demo", true, "registered so demo mode can refuse them by name"},
{"test", false, "a deployment, however much it is called a test"},
{"prod", false, "a deployment"},
{"", false, "no mode configured is not a reason to trust the caller"},
} {
t.Run("mode="+tc.mode, func(t *testing.T) {
routes := registeredRoutes(t, tc.mode)
for _, writing := range genWritingRoutes {
if got := routes[writing]; got != tc.expected {
t.Errorf("mode %q: %s registered = %v, want %v (%s)",
tc.mode, writing, got, tc.expected, tc.why)
}
}
})
}
}
// The other direction. Refusing too much is as much of a defect as refusing
// too little, and the read-only half of the generator is what a demo shows.
func TestGeneratorReadingRoutesExistInEveryMode(t *testing.T) {
for _, mode := range []string{"dev", "demo", "test", "prod", ""} {
t.Run("mode="+mode, func(t *testing.T) {
routes := registeredRoutes(t, mode)
for _, reading := range genReadingRoutes {
if !routes[reading] {
t.Errorf("mode %q: %s is not registered - the gate took a route that only reads",
mode, reading)
}
}
})
}
}
// GenWriteRoutesEnabled is what cmd/api reads to decide whether to warn at
// start-up. If it and the registration ever disagree, the log says one thing
// and the engine does another, so they are checked against each other rather
// than each against a list.
func TestGenWriteRoutesEnabledAgreesWithWhatWasRegistered(t *testing.T) {
for _, mode := range []string{"dev", "demo", "test", "prod", ""} {
t.Run("mode="+mode, func(t *testing.T) {
routes := registeredRoutes(t, mode)
// registeredRoutes puts the mode back before it returns, so ask
// the predicate under a mode set here - about the same value the
// engine was just built under.
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = mode
claimed := GenWriteRoutesEnabled()
actual := routes["/api/v1/gen/todb/:tableId"]
if claimed != actual {
t.Errorf("mode %q: GenWriteRoutesEnabled() = %v but the route was registered = %v",
mode, claimed, actual)
}
})
}
}
// The helper restores the mode before it returns, so nothing it was asked
// about leaks into what the caller does next.
//
// Worth a test of its own because the failure is silent: a helper that left
// the mode set would make every assertion after the call read a value the
// caller did not choose, and each of those assertions would still pass for as
// long as the leaked value happened to be the right one.
func TestRegisteredRoutesRestoresTheModeBeforeReturning(t *testing.T) {
const sentinel = "not-a-mode"
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = sentinel
registeredRoutes(t, "prod")
if got := config.ApplicationConfig.Mode; got != sentinel {
t.Errorf("mode after the helper returned = %q, want %q - it was left set to what "+
"the helper was asked about", got, sentinel)
}
}
// Changing the mode after the routes were built unregisters nothing.
//
// buildRouter has one call site, in run(), and route registration is not on
// any phase or reload callback - so a configuration reload moves
// config.ApplicationConfig.Mode without moving the routes. From that moment
// GenWriteRoutesEnabled answers about a mode the engine was not built under.
//
// That gap is why the start-up warning tells the reader to restart rather than
// only to change the mode. This pins it: if registration ever becomes dynamic,
// this test fails and the message it justifies has to be revisited.
func TestChangingTheModeDoesNotUnregisterWhatWasAlreadyBuilt(t *testing.T) {
built := registeredRoutes(t, "dev")
if !built["/api/v1/gen/todb/:tableId"] {
t.Fatal("built under dev without the writing routes, so this test asserts nothing")
}
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = "prod"
if GenWriteRoutesEnabled() {
t.Fatal("the predicate still allows prod, so the disagreement below is not the one meant")
}
if !built["/api/v1/gen/todb/:tableId"] {
t.Error("the route left the engine when the mode changed - registration has become " +
"dynamic, and the start-up warning's advice to restart is now wrong")
}
}
+38
View File
@@ -0,0 +1,38 @@
package api
import (
"testing"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
// The warning fires exactly where the generator's writing endpoints are served
// and nothing else refuses them.
//
// dev is the case the warning exists for: it is the shipped default, so it is
// the mode a deployment that changed nothing is running in. demo serves the
// routes too, but DemoEvn refuses all three by name, so warning there would
// describe an exposure that is not there.
func TestGeneratorWriteRoutesWarningFiresWhereTheExposureIs(t *testing.T) {
for _, tc := range []struct {
mode string
want bool
why string
}{
{"dev", true, "shipped default, endpoints served and not refused"},
{"demo", false, "served, but DemoEvn refuses all three"},
{"test", false, "not served"},
{"prod", false, "not served"},
{"", false, "not served"},
} {
t.Run("mode="+tc.mode, func(t *testing.T) {
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = tc.mode
if got := generatorWriteRoutesNeedWarning(); got != tc.want {
t.Errorf("mode %q: warning = %v, want %v (%s)", tc.mode, got, tc.want, tc.why)
}
})
}
}
+117
View File
@@ -0,0 +1,117 @@
package api
import (
"context"
"fmt"
"sort"
"strings"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"gorm.io/gorm"
"go-admin/cmd/migrate/migration"
"go-admin/common/health"
commonmodels "go-admin/common/models"
)
// schemaCheckName is what a failing schema reports itself as in /ready's body.
const schemaCheckName = "schema"
// registerSchemaCheck adds the pending-migration check to readiness.
//
// Readiness rather than a refusal to start, and rather than a log line alone.
// The two probes answer different questions: liveness is "restart me", and a
// process whose database is on the wrong schema comes back to the same schema,
// so restarting is not the answer. Readiness is "send me requests", and with a
// schema the binary does not match the answer is no.
//
// Issue #919 is what the absence of this looked like: the process started,
// both probes passed, and the first sign of trouble was a login failing with a
// driver-level encoding error. Refusing to start would have been the wrong fix
// - a process that exits tells an operator less than one that runs and says
// why, and under an orchestrator it crash-loops - while a log line alone is
// not something an orchestrator can act on.
func registerSchemaCheck() {
health.Register(schemaCheckName, schemaCheck)
}
// schemaCheck fails while any tenant database is behind the migrations this
// binary registers.
//
// Any one of them, rather than only the tenant being served: migrations are
// applied to every database in one run, so one database behind means that run
// did not finish. Serving the rest would let a half-applied deploy look like a
// partial success.
//
// Evaluated per request rather than decided at start-up, so that running
// migrate clears it without a restart.
func schemaCheck(ctx context.Context) error {
registered := migration.RegisteredVersions()
if len(registered) == 0 {
// Nothing registered means nothing can be pending, which is the honest
// answer for a tree with no migrations. It is also what a broken build
// would produce - the registry is filled by init() in packages the
// binary has to link - so cmd/api's dependency test asserts the real
// binary links them.
return nil
}
behind := make([]string, 0, 2)
for name, db := range sdk.Runtime.GetAllDb() {
applied, err := appliedVersions(ctx, db)
if err != nil {
return fmt.Errorf("reading applied migrations for %q: %w", name, err)
}
if pending := pendingVersions(registered, applied); len(pending) > 0 {
behind = append(behind, fmt.Sprintf("%s is %d behind, first pending %s",
name, len(pending), pending[0]))
}
}
if len(behind) == 0 {
return nil
}
sort.Strings(behind)
return fmt.Errorf("%s; run `go-admin migrate -c <config>` and see `go-admin migrate status`",
strings.Join(behind, "; "))
}
// appliedVersions reads what sys_migration records for one database.
//
// A missing table is not an error: a database that has never been migrated has
// applied nothing, which is exactly what the caller needs to hear, and is the
// state a first deploy is in.
func appliedVersions(ctx context.Context, db *gorm.DB) (map[string]bool, error) {
if db == nil {
return nil, fmt.Errorf("no database")
}
db = db.WithContext(ctx)
if !db.Migrator().HasTable(&commonmodels.Migration{}) {
return map[string]bool{}, nil
}
var rows []commonmodels.Migration
if err := db.Select("version").Find(&rows).Error; err != nil {
return nil, err
}
out := make(map[string]bool, len(rows))
for _, r := range rows {
out[r.Version] = true
}
return out, nil
}
// pendingVersions returns the registered versions applied does not contain.
//
// Split out and taking both sides as arguments because the registry is
// process-wide and filled by init() in packages cmd/api does not import: a
// test in this package cannot arrange it, so the arranging part is the part
// that is not tested here.
func pendingVersions(registered []string, applied map[string]bool) []string {
out := make([]string, 0)
for _, v := range registered {
if !applied[v] {
out = append(out, v)
}
}
sort.Strings(out)
return out
}
+124
View File
@@ -0,0 +1,124 @@
package api
import (
"context"
"os/exec"
"strings"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
commonmodels "go-admin/common/models"
)
func TestPendingVersionsReportsOnlyWhatIsNotApplied(t *testing.T) {
registered := []string{"1000_a", "2000_b", "3000_c"}
applied := map[string]bool{"1000_a": true, "3000_c": true}
got := pendingVersions(registered, applied)
if len(got) != 1 || got[0] != "2000_b" {
t.Errorf("pending = %v, want [2000_b]", got)
}
}
func TestPendingVersionsIsEmptyWhenTheDatabaseIsCurrent(t *testing.T) {
registered := []string{"1000_a", "2000_b"}
applied := map[string]bool{"1000_a": true, "2000_b": true}
if got := pendingVersions(registered, applied); len(got) != 0 {
t.Errorf("pending = %v, want none", got)
}
}
// A row recorded that this binary no longer registers is not pending. It is
// the orphan `migrate status` already reports, and readiness has nothing to
// say about it: the schema is ahead, not behind, and requests will be served
// correctly.
func TestPendingVersionsIgnoresAppliedRowsNothingRegisters(t *testing.T) {
registered := []string{"1000_a"}
applied := map[string]bool{"1000_a": true, "9999_gone": true}
if got := pendingVersions(registered, applied); len(got) != 0 {
t.Errorf("pending = %v, want none - an orphaned row is not a pending migration", got)
}
}
func memoryDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file::memory:?cache=shared"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { db.Migrator().DropTable(&commonmodels.Migration{}) })
return db
}
// A first deploy has no sys_migration table. That is "nothing applied", not an
// error: reporting it as one would make the check fail for a reason the
// operator cannot act on, on the one deployment where every migration really
// is pending.
func TestAppliedVersionsTreatsAMissingTableAsNothingApplied(t *testing.T) {
db := memoryDB(t)
db.Migrator().DropTable(&commonmodels.Migration{})
got, err := appliedVersions(context.Background(), db)
if err != nil {
t.Fatalf("appliedVersions: %v", err)
}
if len(got) != 0 {
t.Errorf("applied = %v, want empty", got)
}
}
func TestAppliedVersionsReadsWhatTheTableHolds(t *testing.T) {
db := memoryDB(t)
if err := db.AutoMigrate(&commonmodels.Migration{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
db.Create(&commonmodels.Migration{Version: "1000_a"})
db.Create(&commonmodels.Migration{Version: "2000_b"})
got, err := appliedVersions(context.Background(), db)
if err != nil {
t.Fatalf("appliedVersions: %v", err)
}
if !got["1000_a"] || !got["2000_b"] || len(got) != 2 {
t.Errorf("applied = %v, want the two rows written", got)
}
}
// The check is only worth anything if the registry it reads is populated in
// the binary that serves requests, and it is filled by init() in packages
// cmd/api does not import - cmd/migrate blank-imports them, and cmd wires both
// subcommands into one binary.
//
// This cannot be asserted from an ordinary test: importing the version package
// to look at the registry would put it in the test binary's dependency graph
// and pass whatever the real binary links. So ask the build instead.
//
// Without this, dropping those blank imports leaves a check that reports
// "nothing pending" for every database forever, and every test above still
// passes.
func TestTheServingBinaryLinksTheMigrationRegistry(t *testing.T) {
out, err := exec.Command("go", "list", "-deps", "go-admin").Output()
if err != nil {
t.Skipf("go list unavailable: %v", err)
}
deps := string(out)
const versions = "go-admin/cmd/migrate/migration/version"
if !strings.Contains(deps, versions+"\n") {
t.Errorf("the main package does not link %s, so the schema check would "+
"read an empty registry and report every database as current", versions)
}
// Negative control: a package the binary genuinely must not link, so that a
// `deps` that somehow contained everything would fail here rather than pass
// the assertion above for the wrong reason.
const notLinked = "go-admin/tools/checksilent"
if strings.Contains(deps, notLinked+"\n") {
t.Errorf("%s is in the binary's dependency closure, so this test cannot "+
"tell a real link from a query that matches anything", notLinked)
}
}
+39
View File
@@ -82,6 +82,11 @@ func setup() {
// can call the API, which is only true once the socket is accepting.
sdk.Runtime.SetPhase(runtime.AfterListen, startCronJobs)
// Registered before the configuration is read, because it registers a
// callback rather than reading anything: the check runs per request and
// asks the databases that exist then.
registerSchemaCheck()
//1. 读取配置
bootstrap.SetupConfig(
file.NewSource(file.WithPath(configYml)),
@@ -177,6 +182,7 @@ func run() error {
gin.SetMode(gin.ReleaseMode)
}
buildRouter()
reportGeneratorWriteRoutes()
srv := &http.Server{
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
@@ -356,6 +362,39 @@ const (
kubernetesGraceSeconds = 30
)
// reportGeneratorWriteRoutes says whether this process serves the code
// generator's writing endpoints, and to whom.
//
// The endpoints are gated on the mode, and the shipped configuration says dev -
// so the deployment most likely to be exposed is the one that changed nothing,
// and the one least likely to go looking. Silence there would leave the gate
// technically correct and practically useless.
//
// Nothing is said in demo mode. The routes are registered, but DemoEvn refuses
// all three by name, so a warning would describe an exposure that is not there.
func reportGeneratorWriteRoutes() {
if !generatorWriteRoutesNeedWarning() {
return
}
log.Warnf("the code generator's writing endpoints are served in mode %q: "+
"/api/v1/gen/{toproject,apitofile,todb} write Go and Vue source onto this host and rows "+
"into this database, and they are in CasbinExclude, so any account that can log in may "+
"call them. Set application.mode to prod or test on anything that is not a workstation, "+
"then restart: these routes were registered at start-up and a configuration reload does "+
"not rebuild them.",
config.ApplicationConfig.Mode)
}
// generatorWriteRoutesNeedWarning reports whether there is an exposure to warn
// about: the endpoints are served, and nothing else is refusing them.
//
// Split from the logging so the decision can be tested. A warning nobody can
// make fire is indistinguishable from no warning at all, and this one exists
// precisely for the case nobody is looking at.
func generatorWriteRoutesNeedWarning() bool {
return otherrouter.GenWriteRoutesEnabled() && config.ApplicationConfig.Mode != "demo"
}
// reportShutdownBudget states what a shutdown will spend and whether it fits.
//
// The sum is taken from the resolved values, not from the configuration file:
+22
View File
@@ -185,6 +185,28 @@ func (e *Migration) mergedEntries() map[string]versionEntry {
return out
}
// RegisteredVersions returns every migration version this binary registers,
// sorted, without touching a database.
//
// Status answers a richer question - what is registered, what is applied, and
// what is applied while nothing registers it - and needs a database to do it.
// This is the half that can be asked of the process alone, which is what a
// readiness check needs: the check holds the databases it is asking about, and
// reusing Status would mean calling SetDb from a request handler, writing this
// package's shared state from a request path.
func (e *Migration) RegisteredVersions() []string {
all := e.mergedEntries()
out := make([]string, 0, len(all))
for k := range all {
out = append(out, k)
}
sort.Strings(out)
return out
}
// RegisteredVersions reports what the process-wide registry holds.
func RegisteredVersions() []string { return Migrate.RegisteredVersions() }
// StatusEntry is one row of migrate status.
type StatusEntry struct {
AppCode string
@@ -237,13 +237,53 @@ func dropIndexesOn(db *gorm.DB, table, column string) error {
if !m.HasIndex(table, name) {
continue
}
if err := m.DropIndex(table, name); err != nil {
if err := db.Exec(dropIndex(db, table, name)).Error; err != nil {
return fmt.Errorf("dropping index %s: %w", name, err)
}
}
return nil
}
// dropIndex spells DROP INDEX for one dialect, rather than going through
// Migrator().DropIndex.
//
// The migrator cannot be used here on PostgreSQL. Its driver resolves a schema
// for the statement and falls back to an expression when it cannot:
//
// currentSchema, _ := m.CurrentSchema(stmt, stmt.Table) // CURRENT_SCHEMA()
// m.DB.Exec("DROP INDEX ?.?", currentSchema, clause.Column{Name: name})
//
// DROP INDEX takes an identifier in that position, not an expression, so the
// statement does not parse. The schema is unresolvable for every call made
// here, because this passes a table name as a string rather than a model - so
// it failed on every PostgreSQL database rather than intermittently, and took
// the whole conversion with it. Reported as go-admin#919, where the visible
// symptom was a login rejecting a correct password: the migration had stopped
// here, leaving deleted_at a timestamptz that the current query compares to 0.
//
// Written per dialect for the same reason addBigIntColumn and renameColumn
// already are.
//
// MySQL and SQL Server name the table in the statement and have no IF EXISTS
// for it; PostgreSQL and SQLite name the index alone, in its own namespace.
// The caller has already checked HasIndex, so IF EXISTS is only there to make
// the two that support it say nothing rather than fail on a race with another
// migrator.
//
// Verified against PostgreSQL 15, MySQL 8.0 and SQLite. The SQL Server form is
// from its documentation and has not been run - this repository has no SQL
// Server to run it against.
func dropIndex(db *gorm.DB, table, index string) string {
switch db.Dialector.Name() {
case "mysql":
return fmt.Sprintf("DROP INDEX `%s` ON `%s`", index, table)
case "sqlserver":
return fmt.Sprintf("DROP INDEX [%s] ON [%s]", index, table)
default:
return fmt.Sprintf(`DROP INDEX IF EXISTS "%s"`, index)
}
}
// indexNamesFor asks the database which indexes cover column.
func indexNamesFor(db *gorm.DB, table, column string) ([]string, error) {
indexes, err := db.Migrator().GetIndexes(table)
@@ -0,0 +1,156 @@
package version
import (
"os"
"testing"
"time"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
// postgresDSNEnv points these tests at a database. They are skipped without
// it, so a developer with no PostgreSQL running still gets a green run.
//
// The whole file exists because the rest of this package's tests run on
// SQLite, where the defect they cover cannot happen: dropping an index through
// gorm's migrator works there and produces unparseable SQL on PostgreSQL. A
// suite that only ever exercised SQLite reported success for a migration that
// failed on every PostgreSQL database it was pointed at - go-admin#919.
const postgresDSNEnv = "GO_ADMIN_TEST_POSTGRES_DSN"
func postgresDB(t *testing.T) *gorm.DB {
t.Helper()
dsn := os.Getenv(postgresDSNEnv)
if dsn == "" {
// Skipping locally is the point; skipping in CI is the failure this
// file exists to prevent. A workflow that renamed the variable or
// dropped the service would otherwise go green while these tests
// quietly did nothing - the same shape as the defect they cover.
if os.Getenv("CI") != "" {
t.Fatalf("%s is not set while CI is: the PostgreSQL migration tests must not skip here", postgresDSNEnv)
}
t.Skipf("%s is not set; skipping the PostgreSQL migration tests", postgresDSNEnv)
}
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatalf("connecting to %s: %v", postgresDSNEnv, err)
}
return db
}
// pgOldUser is the pre-migration shape: a nullable timestamp with an index on
// it, which is what makes dropping the column require dropping the index.
type pgOldUser struct {
UserId int64 `gorm:"column:user_id;primaryKey;autoIncrement"`
Username string
DeletedAt *time.Time `gorm:"index"`
}
func (pgOldUser) TableName() string { return "sd_pg_user" }
// The conversion completes on PostgreSQL.
//
// It did not. dropIndexesOn went through Migrator().DropIndex, whose
// PostgreSQL driver falls back to an expression when it cannot resolve a
// schema - which is every call made here, because the migration passes a table
// name as a string:
//
// DROP INDEX CURRENT_SCHEMA()."idx_sd_pg_user_deleted_at"
//
// DROP INDEX takes an identifier there, so it failed to parse and took the
// whole conversion with it. Every PostgreSQL deployment stopped at this
// migration, and the visible symptom was a login rejecting a correct password
// because deleted_at was still a timestamptz being compared to 0.
func TestConversionCompletesOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&pgOldUser{}) })
db.Migrator().DropTable(&pgOldUser{})
if err := db.AutoMigrate(&pgOldUser{}); err != nil {
t.Fatalf("building the old shape: %v", err)
}
deleted := time.Now().Add(-time.Hour)
// Checked rather than fired and forgotten: a failed insert leaves the
// assertions below reading an empty table, and "no rows" is a shape some
// of them cannot tell from success.
if err := db.Create(&pgOldUser{Username: "gone", DeletedAt: &deleted}).Error; err != nil {
t.Fatalf("seeding the deleted row: %v", err)
}
if err := db.Create(&pgOldUser{Username: "live"}).Error; err != nil {
t.Fatalf("seeding the live row: %v", err)
}
if err := convertDeletedAt(db, "sd_pg_user"); err != nil {
t.Fatalf("convertDeletedAt: %v", err)
}
var dataType string
if err := db.Raw(`SELECT data_type FROM information_schema.columns
WHERE table_name = 'sd_pg_user' AND column_name = 'deleted_at'`).Scan(&dataType).Error; err != nil {
t.Fatalf("reading the column type: %v", err)
}
if dataType != "bigint" {
t.Errorf("deleted_at is %q after the conversion, want bigint", dataType)
}
// The marker has to carry the timestamp across, or a row that was deleted
// comes back live.
var markers []int64
if err := db.Raw(`SELECT deleted_at FROM sd_pg_user ORDER BY user_id`).Scan(&markers).Error; err != nil {
t.Fatalf("reading the markers: %v", err)
}
if len(markers) != 2 {
t.Fatalf("read %d rows, want 2", len(markers))
}
if markers[0] == 0 {
t.Error("the deleted row came back live")
}
if markers[1] != 0 {
t.Errorf("the live row is marked deleted at %d", markers[1])
}
}
// The index on deleted_at is gone afterwards, which is the step that failed.
//
// Asserted separately from the conversion because the conversion can succeed
// on a table with no index at all, and this migration exists for tables that
// have one.
func TestTheIndexOnDeletedAtIsDroppedOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&pgOldUser{}) })
db.Migrator().DropTable(&pgOldUser{})
if err := db.AutoMigrate(&pgOldUser{}); err != nil {
t.Fatalf("building the old shape: %v", err)
}
var before int64
if err := db.Raw(`SELECT count(*) FROM pg_indexes
WHERE tablename = 'sd_pg_user' AND indexdef LIKE '%deleted_at%'`).Scan(&before).Error; err != nil {
t.Fatalf("counting the indexes before: %v", err)
}
if before == 0 {
t.Fatal("the old shape has no index on deleted_at, so this test asserts nothing")
}
if err := dropIndexesOn(db, "sd_pg_user", "deleted_at"); err != nil {
t.Fatalf("dropIndexesOn: %v", err)
}
// This one is why the errors are checked at all rather than as a matter of
// habit: a query that fails leaves after at zero, and zero is what success
// looks like. An unchecked error here is a test that passes when it cannot
// reach the database.
var after int64
if err := db.Raw(`SELECT count(*) FROM pg_indexes
WHERE tablename = 'sd_pg_user' AND indexdef LIKE '%deleted_at%'`).Scan(&after).Error; err != nil {
t.Fatalf("counting the indexes after: %v", err)
}
if after != 0 {
t.Errorf("%d index(es) on deleted_at survived", after)
}
}
@@ -0,0 +1,42 @@
package version
import (
"runtime"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
)
// Create sys_app (PRD 008 F2) and sys_app_casbin_grant (F4/F6's casbin
// attribution ledger - see the design doc's (docs-prd/008-应用清单与安装器/
// 数据库变更.md) §2.2/§3 for why casbin_rule itself is not touched:
// gorm-adapter's SavePolicyCtx truncates and reloads that table from its
// in-memory model, and any column this migration added to it would be
// silently zeroed the first time anything calls SavePolicy.
//
// Ordered after 1786700003000 (the soft-delete conversion), so importing
// cmd/migrate/migration/models is banned here - see
// schema_coverage_test.go's TestPostConversionMigrationsAvoidFrozenSeedModels.
// Both new tables are AutoMigrate'd from their runtime model shape under
// app/admin/models directly, which is also why neither one is added to
// 1786700003000's frozen softDeleteTables list: neither embeds
// common.ModelTime in the first place (see design doc §1.1).
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700007000AppRegistryTables)
}
func _1786700007000AppRegistryTables(db *gorm.DB, version string) error {
return db.Transaction(func(tx *gorm.DB) error {
if err := tx.Migrator().AutoMigrate(
new(adminmodels.SysApp),
new(adminmodels.SysAppCasbinGrant),
); err != nil {
return err
}
return tx.Create(&common.Migration{Version: version}).Error
})
}
@@ -0,0 +1,62 @@
package version
import (
"testing"
common "go-admin/common/models"
adminmodels "go-admin/app/admin/models"
)
// postgresDB is defined in 1786700003000_soft_delete_marker_postgres_test.go
// and shared across this package's PostgreSQL-only tests.
//
// This migration is plain AutoMigrate on two brand-new tables, unlike
// 1786700003000's DROP INDEX (go-admin#919's actual defect), so there is no
// dialect-specific SQL here for AutoMigrate itself to get wrong on
// PostgreSQL specifically. What is worth a real PostgreSQL run is
// 1786700008000's CONCAT()-based duplicate check next door - PostgreSQL has
// had CONCAT() since 9.1, but it was never verified against a real server
// until this file, only inferred from documentation - and the same
// AutoMigrate call this test makes, so a schema/character-set mistake
// AutoMigrate might make silently on a dialect nobody ran it against here
// has somewhere to surface.
func TestAppRegistryTablesAreCreatedOnPostgres(t *testing.T) {
db := postgresDB(t)
const version = "1786700007000-pg"
cleanup := func() {
db.Migrator().DropTable(&adminmodels.SysAppCasbinGrant{}, &adminmodels.SysApp{})
// Only this test's own row, not the whole shared sys_migration
// table: postgresDB points at a real, persistent database (unlike
// the SQLite tests' fresh in-memory one per run), so a version left
// behind by a previous run of this same binary collides with the
// wrapper's own INSERT the next time this test runs.
db.Exec("DELETE FROM sys_migration WHERE version = ?", version)
}
t.Cleanup(cleanup)
cleanup()
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("automigrate sys_migration: %v", err)
}
if err := _1786700007000AppRegistryTables(db, version); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order", Version: "v1"}).Error; err != nil {
t.Fatalf("insert sys_app: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "dup", Version: "v1"}).Error; err == nil {
t.Fatal("a second sys_app row with the same app_code was accepted on PostgreSQL")
}
grant := adminmodels.SysAppCasbinGrant{AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET"}
if err := db.Create(&grant).Error; err != nil {
t.Fatalf("insert sys_app_casbin_grant: %v", err)
}
dup := grant
dup.Id = 0
if err := db.Create(&dup).Error; err == nil {
t.Fatal("a second sys_app_casbin_grant row with the same natural key was accepted on PostgreSQL")
}
}
@@ -0,0 +1,130 @@
package version
import (
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
common "go-admin/common/models"
)
func openAppRegistryDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("automigrate sys_migration: %v", err)
}
return db
}
// The migration has to build both tables and record itself as applied -
// F2/F6's acceptance case is a row landing in either one, and neither is
// possible if the table it belongs to was never created.
func TestAppRegistryTablesAreCreated(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasTable(&adminmodels.SysApp{}) {
t.Fatal("sys_app was not created")
}
if !db.Migrator().HasTable(&adminmodels.SysAppCasbinGrant{}) {
t.Fatal("sys_app_casbin_grant was not created")
}
// A row that exercises every column, not just HasTable/HasColumn -
// AutoMigrate can build a column with the wrong type and still report
// that it exists.
if err := db.Create(&adminmodels.SysApp{
AppCode: "order", Name: "Order", Version: "v1", Description: "d", Author: "a",
Requires: "payment", Pricing: "free", License: "MIT", Status: 1,
}).Error; err != nil {
t.Fatalf("insert sys_app: %v", err)
}
if err := db.Create(&adminmodels.SysAppCasbinGrant{
AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET",
}).Error; err != nil {
t.Fatalf("insert sys_app_casbin_grant: %v", err)
}
var applied common.Migration
if err := db.Where("version = ?", "1786700007000").First(&applied).Error; err != nil {
t.Fatalf("sys_migration was not recorded: %v", err)
}
}
// Running it twice must be safe: DDL does not roll back on MySQL, so an
// operator whose first attempt failed partway through has nothing to do but
// run it again. This calls AutoMigrate directly rather than the wrapper,
// which also inserts a sys_migration row that a second call would collide
// on - a collision Migrate.run() itself prevents by never calling a
// function twice for the same recorded version, so it is not this
// migration's job to tolerate.
func TestAppRegistryTablesAutoMigrateIsRepeatable(t *testing.T) {
db := openAppRegistryDB(t)
for i := 0; i < 3; i++ {
if err := db.Migrator().AutoMigrate(
new(adminmodels.SysApp),
new(adminmodels.SysAppCasbinGrant),
); err != nil {
t.Fatalf("automigrate %d: %v", i, err)
}
}
}
// sys_app.app_code is the unique key G2 ("is app X installed") answers with
// - a second row for the same app code must be rejected, not tolerated.
func TestSysAppAppCodeIsUnique(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order", Version: "v1"}).Error; err != nil {
t.Fatalf("first insert: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order dup", Version: "v1"}).Error; err == nil {
t.Fatal("a second sys_app row with the same app_code was accepted")
}
}
// sys_app_casbin_grant's unique index mirrors casbin_rule's own natural key
// (ptype,v0..v5) exactly - see design doc §3. A duplicate grant for the
// same rule must be rejected the same way gorm-adapter's own unique index
// on casbin_rule would reject it.
func TestSysAppCasbinGrantNaturalKeyIsUnique(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
grant := adminmodels.SysAppCasbinGrant{AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET"}
if err := db.Create(&grant).Error; err != nil {
t.Fatalf("first insert: %v", err)
}
dup := grant
dup.Id = 0
if err := db.Create(&dup).Error; err == nil {
t.Fatal("a second sys_app_casbin_grant row with the same natural key was accepted")
}
// A grant for a different app, but the identical casbin natural key, is
// exactly the collision two applications granting the same api/role
// pair would produce - the natural key has to be the one thing that
// rejects it, app_code is descriptive only and not part of the index.
other := grant
other.Id = 0
other.AppCode = "another-app"
if err := db.Create(&other).Error; err == nil {
t.Fatal("a duplicate natural key under a different app_code was accepted")
}
}
@@ -0,0 +1,145 @@
package version
import (
"fmt"
"runtime"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
)
// Give seed.SeedMenus's two write paths (seedApis, seedMenuTree in
// app/admin/service/seed.go) a real natural key to check before inserting,
// so a retried, partially-failed migration (see the design doc
// docs-prd/008-应用清单与安装器/数据库变更.md §1.5/§1.6) does not insert the
// same row twice. This has already happened in production once (duplicate
// sys_menu/casbin_rule rows on the demo site), not a theoretical risk.
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700008000SeedNaturalKeys)
}
func _1786700008000SeedNaturalKeys(db *gorm.DB, version string) error {
if err := seedNaturalKeys(db); err != nil {
return err
}
return db.Create(&common.Migration{Version: version}).Error
}
// seedNaturalKeys is split out from the wrapper above so tests can call it
// against a database that only has sys_menu/sys_api, without also standing
// up sys_migration - and so it can be called more than once in the same
// test to prove the re-run tolerance the doc comment above promises: DDL
// does not roll back on MySQL, so an operator whose first attempt failed
// partway through has nothing to do but run the whole migration again.
func seedNaturalKeys(db *gorm.DB) error {
m := db.Migrator()
// sys_menu.seed_code is a brand-new column: every existing row becomes
// NULL, and NULL never collides in the unique index built below, so
// this needs no pre-check.
if !m.HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
if err := m.AddColumn(&adminmodels.SysMenu{}, "SeedCode"); err != nil {
return err
}
}
if !m.HasIndex(&adminmodels.SysMenu{}, "uk_sys_menu_app_seed_code_del") {
if err := db.Exec(
"CREATE UNIQUE INDEX uk_sys_menu_app_seed_code_del ON sys_menu (app_code, seed_code, deleted_at)",
).Error; err != nil {
return err
}
}
// sys_api reuses existing, already-populated columns, which the demo
// site has already proven can hold duplicates. Refuse rather than let
// CREATE UNIQUE INDEX fail on an operator with no idea which rows to
// reconcile - same shape as 1786700003000_soft_delete_marker.go's
// refuseOnDuplicates.
if err := refuseOnDuplicateApis(db); err != nil {
return err
}
if !m.HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
if err := db.Exec(
"CREATE UNIQUE INDEX uk_sys_api_app_path_action_del ON sys_api (app_code, path, action, deleted_at)",
).Error; err != nil {
return err
}
}
return nil
}
// refuseOnDuplicateApis reports the (app_code, path, action) values that
// would make the unique index impossible, rather than the index failing to
// build and saying only that it did. Only live rows count: a soft-deleted
// duplicate does not block the index it will never occupy a slot in.
//
// sys_api.path/action (app/admin/models/sys_api.go) carry no NOT NULL
// constraint, and that stays true here on purpose: tightening it is an
// independent, backward-incompatible change of its own - existing NULL
// rows in a real database would need reconciling or backfilling before
// ALTER TABLE ... NOT NULL could even run, which is a decision for
// whoever owns that data, not something this migration should force as a
// side effect of adding an unrelated index. So this function has to
// tolerate NULL path/action rather than assume they cannot occur - see the
// query below for how it does that without either crashing on them
// (MySQL's CONCAT) or wrongly flagging them (GROUP BY's NULL-equals-NULL).
//
// The two are independent bugs that happened to share one root cause, and
// SQLite's own test suite for this file would have caught neither on its
// own: MySQL's CONCAT() returns NULL if any argument is NULL, which turned
// a duplicate check against a NULL-holding library into "converting NULL
// to string is unsupported" instead of a report - but SQLite's (and
// PostgreSQL's) CONCAT() treats a NULL argument as an empty string
// instead, so the exact same query never errors there no matter how it is
// called. A suite that only ever ran on SQLite would report success for
// both defects; only a real MySQL server surfaces the first one at all -
// this migration's PostgreSQL-only sibling test file
// (1786700008000_seed_natural_keys_postgres_test.go) rules out one more
// dialect, but MySQL specifically has to be checked by hand, since this
// repository's test suite has no MySQL service to run against in CI.
func refuseOnDuplicateApis(db *gorm.DB) error {
var dupes []string
if err := db.Raw(
// This has to agree with what the unique index it guards actually
// enforces, not just with what looks like a duplicate at a glance.
// Two different SQL rules collide on a NULL: GROUP BY treats two
// NULLs as equal, so a naive query flags every pair of rows that
// share a NULL path or action - even a pair with only one of the
// two NULL, since GROUP BY's equality still holds on whichever
// column both rows leave NULL - but a UNIQUE INDEX treats every
// NULL as distinct from every other value, including another
// NULL, so the index itself accepts every one of those pairs
// without complaint. Excluding any row missing either column from
// consideration entirely is what makes the two agree: a row
// missing path, or missing action, or missing both, can never
// violate the index no matter how many other rows are also
// missing the same one, so none of them belong in this count.
//
// No COALESCE: with both columns excluded whenever either is
// NULL, CONCAT here never receives a NULL argument for path or
// action - app_code cannot be NULL at all (see its own NOT NULL
// tag) - so there is nothing left for COALESCE to guard against,
// and leaving it out is deliberate rather than an oversight. A
// future regression that removed the two IS NOT NULL conditions
// above would fail loudly on MySQL (the same Scan error this
// query used to produce) instead of quietly reporting a made-up
// "duplicate" whose path and action both print as empty - the
// failure this function exists to prevent in the first place.
`SELECT CONCAT(app_code, '|', path, '|', action) FROM sys_api
WHERE deleted_at = 0 AND path IS NOT NULL AND action IS NOT NULL
GROUP BY app_code, path, action HAVING COUNT(*) > 1`,
).Scan(&dupes).Error; err != nil {
return fmt.Errorf("checking sys_api for duplicates: %w", err)
}
if len(dupes) > 0 {
return fmt.Errorf(
"sys_api already holds duplicate (app_code,path,action) %v; reconcile them before this migration can add its unique index",
dupes)
}
return nil
}
@@ -0,0 +1,105 @@
package version
import (
"testing"
)
// postgresDB is defined in 1786700003000_soft_delete_marker_postgres_test.go.
//
// This file exists because refuseOnDuplicateApis's duplicate check is
// spelled with CONCAT(), a function this migration's design assumed
// PostgreSQL has carried since 9.1 but that nothing had run against a real
// PostgreSQL server before this test - only against the pure-Go SQLite
// driver, which happens to bundle a SQLite new enough to have grown its own
// CONCAT() only recently. A dialect where that assumption were wrong would
// otherwise only be discovered the first time an operator's install hit a
// genuine sys_api duplicate on PostgreSQL in production.
func TestSeedNaturalKeysRefusesDuplicateApisOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
for i := 0; i < 2; i++ {
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed duplicate %d: %v", i, err)
}
}
err := seedNaturalKeys(db)
if err == nil {
t.Fatal("PostgreSQL accepted sys_api rows that already hold a duplicate (app_code, path, action)")
}
if !contains(err.Error(), "order") || !contains(err.Error(), "/api/v1/order") {
t.Errorf("the error does not name the offending row: %v", err)
}
if db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was built despite the migration refusing")
}
}
// GROUP BY treats two NULLs as equal for grouping; a UNIQUE INDEX treats
// every NULL as distinct from every other value, including another NULL.
// Both are standard SQL, not a SQLite/PostgreSQL/MySQL difference - this
// file exists to confirm that on a real server rather than assume it, the
// same reason TestSeedNaturalKeysRefusesDuplicateApisOnPostgres above
// exists for CONCAT(). See TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAccept
// in the SQLite-backed test file for the full account of why this matters:
// a naive duplicate check that does not exclude NULL path/action refuses
// an install the unique index itself would accept without complaint.
func TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAcceptOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
for i := 0; i < 2; i++ {
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, NULL, 0)",
).Error; err != nil {
t.Fatalf("seed NULL row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts on PostgreSQL: %v", err)
}
if !db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built on PostgreSQL even though seedNaturalKeys reported success")
}
}
// The success path, on the same server: both columns and both unique
// indexes have to actually build on PostgreSQL, not merely fail to error
// out on SQLite. Mirrors TestSeedNaturalKeysIsRepeatable's SQLite coverage.
func TestSeedNaturalKeysBuildsOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
for i := 0; i < 2; i++ {
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate %d: %v", i, err)
}
}
if !db.Migrator().HasColumn(&oldSeedMenu{}, "seed_code") {
t.Error("sys_menu.seed_code was not added on PostgreSQL")
}
if !db.Migrator().HasIndex(&oldSeedMenu{}, "uk_sys_menu_app_seed_code_del") {
t.Error("the sys_menu unique index was not built on PostgreSQL")
}
if !db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the sys_api unique index was not built on PostgreSQL")
}
}
@@ -0,0 +1,305 @@
package version
import (
"testing"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
common "go-admin/common/models"
)
// oldSeedMenu/oldSeedApi are the shape of sys_menu/sys_api immediately
// before this migration: post-1786700003000 (deleted_at is the NOT NULL
// millisecond marker) and post-1786700006000 (app_code exists), but before
// seed_code or either unique index. They stand in for the real runtime
// models, which by the time this file is read already carry the columns
// this migration adds - the same relationship oldUser bears to sys_user in
// 1786700003000_soft_delete_marker_test.go.
type oldSeedMenu struct {
MenuId int `gorm:"column:menu_id;primaryKey;autoIncrement"`
AppCode string `gorm:"column:app_code;type:varchar(64);not null;default:''"`
DeletedAt int64 `gorm:"column:deleted_at;not null;default:0"`
}
func (oldSeedMenu) TableName() string { return "sys_menu" }
type oldSeedApi struct {
Id int `gorm:"column:id;primaryKey;autoIncrement"`
AppCode string `gorm:"column:app_code;type:varchar(64);not null;default:''"`
Path string `gorm:"column:path;type:varchar(128)"`
Action string `gorm:"column:action;type:varchar(16)"`
DeletedAt int64 `gorm:"column:deleted_at;not null;default:0"`
}
func (oldSeedApi) TableName() string { return "sys_api" }
func openSeedNaturalKeysDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}, &common.Migration{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
return db
}
// The host's own hand-placed menus, and every app-seeded row written
// before this column existed, have no seed_code at all - an unbounded
// number of those must coexist under the same app_code without tripping
// the new unique index (design doc §1.6: "NULL never treated as equal to
// NULL").
func TestSeedNaturalKeysToleratesManyPreExistingMenusWithNoSeedCode(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 3; i++ {
if err := db.Create(&oldSeedMenu{AppCode: ""}).Error; err != nil {
t.Fatalf("seed pre-existing menu %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
t.Fatal("sys_menu.seed_code was not added")
}
}
// The point of adding seed_code at all: a second row with the same
// (app_code, seed_code) while both are live is what seedMenuTree's
// idempotency check depends on the database to reject if the Go-level
// check above it is ever bypassed or raced.
func TestSeedNaturalKeysMenuUniqueIndexBindsLiveRowsOnly(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error; err != nil {
t.Fatalf("seed: %v", err)
}
t.Run("a second live row with the same natural key is rejected", func(t *testing.T) {
err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error
if err == nil {
t.Fatal("a duplicate (app_code, seed_code) was accepted while both rows were live")
}
})
t.Run("the key is free again once the row is soft-deleted", func(t *testing.T) {
if err := db.Exec("UPDATE sys_menu SET deleted_at = ? WHERE seed_code = 'dir'", time.Now().UnixMilli()).Error; err != nil {
t.Fatalf("soft-delete: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error; err != nil {
t.Errorf("the key stayed taken after its row was soft-deleted: %v", err)
}
})
}
// The demo site has already proven sys_api can hold historical duplicates;
// the migration has to name them and refuse, not let CREATE UNIQUE INDEX
// fail on an operator with no idea which rows to reconcile.
func TestSeedNaturalKeysRefusesDuplicateApis(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed duplicate %d: %v", i, err)
}
}
err := seedNaturalKeys(db)
if err == nil {
t.Fatal("the migration accepted sys_api rows that already hold a duplicate (app_code, path, action)")
}
if !contains(err.Error(), "order") || !contains(err.Error(), "/api/v1/order") {
t.Errorf("the error does not name the offending row: %v", err)
}
if db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was built despite the migration refusing")
}
// sys_menu's column and index are independent of sys_api's outcome and
// should already be in place - a partial failure here still leaves a
// record of what succeeded, same as any other non-transactional DDL
// migration in this package.
if !db.Migrator().HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
t.Error("sys_menu.seed_code was not added even though only the sys_api step failed")
}
}
// Only live rows count towards the duplicate check: a row a prior,
// unrelated soft-delete already retired does not block the index it will
// never occupy a slot in.
func TestSeedNaturalKeysIgnoresSoftDeletedApiDuplicates(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed live row: %v", err)
}
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET", DeletedAt: time.Now().UnixMilli()}).Error; err != nil {
t.Fatalf("seed soft-deleted row: %v", err)
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built")
}
}
// The point of the sys_api index, mirroring
// TestSeedNaturalKeysMenuUniqueIndexBindsLiveRowsOnly above: a second live
// row is rejected, and the key is free again once the row is
// soft-deleted.
func TestSeedNaturalKeysApiUniqueIndexBindsLiveRowsOnly(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
t.Run("a second live row with the same natural key is rejected", func(t *testing.T) {
err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', 'GET', 0)",
).Error
if err == nil {
t.Fatal("a duplicate (app_code, path, action) was accepted while both rows were live")
}
})
t.Run("the key is free again once the row is soft-deleted", func(t *testing.T) {
if err := db.Exec(
"UPDATE sys_api SET deleted_at = ? WHERE path = '/api/v1/order'", time.Now().UnixMilli(),
).Error; err != nil {
t.Fatalf("soft-delete: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', 'GET', 0)",
).Error; err != nil {
t.Errorf("the key stayed taken after its row was soft-deleted: %v", err)
}
})
}
// Running it twice must be safe: DDL does not roll back on MySQL, so an
// operator whose first attempt failed partway through (say, sys_menu's step
// succeeded and sys_api's refused) has nothing to do but run the whole
// migration again once the duplicates are reconciled.
func TestSeedNaturalKeysIsRepeatable(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
for i := 0; i < 3; i++ {
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate %d: %v", i, err)
}
}
}
// The wrapper's contract with Migrate.run(): the version is only recorded
// once the whole thing - both columns, both indexes - succeeded.
func TestSeedNaturalKeysWrapperRecordsTheVersion(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := _1786700008000SeedNaturalKeys(db, "1786700008000"); err != nil {
t.Fatalf("migrate: %v", err)
}
var applied common.Migration
if err := db.Where("version = ?", "1786700008000").First(&applied).Error; err != nil {
t.Fatalf("sys_migration was not recorded: %v", err)
}
}
// GROUP BY treats two NULLs as equal for grouping purposes; a UNIQUE INDEX
// treats every NULL as distinct from every other value, including another
// NULL - both are standard SQL semantics, not a quirk of one dialect (see
// the postgres-only test file next to this one for the same check against
// a real server). A duplicate check that groups on the raw columns without
// accounting for that difference refuses an install the index itself would
// accept without complaint, on data there is nothing to "reconcile" -
// worse than the index simply failing to build, because it stops a library
// that has nothing wrong with it.
//
// sys_api.path/action carry no NOT NULL constraint - see the design doc's
// note on this migration for why that stays true in this batch, changing
// it is an independent, backward-incompatible migration of its own - so
// this state is reachable in a real database even though seedApis's own
// Create call, which always writes the Go zero value "" rather than NULL,
// never produces it itself. Inserted via raw SQL for exactly that reason:
// models.SysApi's Path/Action are plain (non-pointer) Go strings, which
// cannot represent NULL through a normal Create call.
func TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAccept(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, NULL, 0)",
).Error; err != nil {
t.Fatalf("seed NULL row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built even though seedNaturalKeys reported success")
}
}
// The case above has both path and action NULL on every row, which both
// of the query's two NULL-exclusion conditions independently catch - it
// cannot tell "only path IS NOT NULL is doing anything here" apart from
// "both conditions are doing something". A row missing only one of the
// two is exactly as real (an api registered with a path but no method,
// or vice versa) and exercises only one condition at a time: two rows
// sharing a real path but both NULL in action, or two rows sharing a real
// action but both NULL in path. GROUP BY treats each pair's shared NULL
// the same way it treats a shared (NULL, NULL) - as equal - and the
// unique index accepts both pairs for the same reason it accepts the
// (NULL, NULL) case, so neither belongs in the count either.
func TestSeedNaturalKeysDoesNotFlagPartiallyNullRows(t *testing.T) {
cases := []struct {
name string
insert string // two rows, sharing a value in exactly one of path/action
}{
{
name: "path is null, action repeats",
insert: "INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, 'GET', 0)",
},
{
name: "action is null, path repeats",
insert: "INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', NULL, 0)",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Exec(tc.insert).Error; err != nil {
t.Fatalf("seed row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built even though seedNaturalKeys reported success")
}
})
}
}
+55 -1
View File
@@ -41,6 +41,7 @@ import (
"encoding/hex"
"errors"
"fmt"
"sync"
"sync/atomic"
"time"
@@ -68,6 +69,55 @@ type Check struct {
Err string `json:"error,omitempty"`
}
// extra are checks a host registers that this package cannot make itself.
//
// The direction is why this exists. Whether the schema matches what the binary
// expects is answered by the migration registry, which lives under cmd/ - and
// common/ has never imported cmd/. Rather than start, the host registers the
// check from where both are already in scope.
var (
extraMu sync.RWMutex
extra []namedCheck
)
type namedCheck struct {
name string
fn func(context.Context) error
}
// Register adds a check to what Ready asks.
//
// It panics on a duplicate name rather than replacing or appending: two checks
// under one name make the failing one impossible to identify from the response,
// and registering the same one twice is a wiring mistake worth hearing about at
// start-up rather than never.
func Register(name string, fn func(context.Context) error) {
if name == "" {
panic("health: a registered check needs a name")
}
if fn == nil {
panic("health: check " + name + " is nil")
}
extraMu.Lock()
defer extraMu.Unlock()
for _, c := range extra {
if c.name == name {
panic("health: check " + name + " is already registered")
}
}
extra = append(extra, namedCheck{name: name, fn: fn})
}
// registered returns the checks a host has added, copied so that Ready is not
// iterating the slice while another goroutine appends to it.
func registered() []namedCheck {
extraMu.RLock()
defer extraMu.RUnlock()
out := make([]namedCheck, len(extra))
copy(out, extra)
return out
}
// Ready asks every dependency this process cannot serve a request without.
//
// The queue is deliberately absent. Nothing on AdapterQueue answers "are you
@@ -75,10 +125,14 @@ type Check struct {
// a queue that is down degrades logging rather than stopping requests - which
// is a reason to alert, not a reason to leave the load balancer pool.
func Ready(ctx context.Context) []Check {
return []Check{
checks := []Check{
safely("database", func() error { return pingDB(ctx) }),
safely("cache", probeCache),
}
for _, c := range registered() {
checks = append(checks, safely(c.name, func() error { return c.fn(ctx) }))
}
return checks
}
// safely turns a panic into a failed check.
+125
View File
@@ -0,0 +1,125 @@
package health
import (
"context"
"errors"
"testing"
)
// isolate empties the registered checks and puts them back, so one test in
// this package cannot decide what the next one sees.
func isolate(t *testing.T) {
t.Helper()
extraMu.Lock()
previous := extra
extra = nil
extraMu.Unlock()
t.Cleanup(func() {
extraMu.Lock()
extra = previous
extraMu.Unlock()
})
}
func findCheck(checks []Check, name string) (Check, bool) {
for _, c := range checks {
if c.Name == name {
return c, true
}
}
return Check{}, false
}
// A registered check has to reach Ready's answer, or the host has wired
// something that never gets asked.
func TestARegisteredCheckIsAsked(t *testing.T) {
isolate(t)
Register("schema", func(context.Context) error { return errors.New("two behind") })
checks := Ready(context.Background())
c, ok := findCheck(checks, "schema")
if !ok {
t.Fatal("Ready did not ask the registered check")
}
if c.OK {
t.Error("the check returned an error and was still reported OK")
}
if c.Err != "two behind" {
t.Errorf("Err = %q, want the check's own message", c.Err)
}
if Healthy(checks) {
t.Error("Healthy said yes while a registered check was failing")
}
}
// The context Ready is given has to reach the check: it carries the probe's
// deadline, and a check that ignores it can hold the handler past it.
func TestTheRegisteredCheckIsGivenReadysContext(t *testing.T) {
isolate(t)
type key struct{}
Register("ctx", func(ctx context.Context) error {
if ctx.Value(key{}) != "carried" {
return errors.New("the check was handed a different context")
}
return nil
})
checks := Ready(context.WithValue(context.Background(), key{}, "carried"))
c, ok := findCheck(checks, "ctx")
if !ok {
t.Fatal("the registered check was not asked")
}
if !c.OK {
t.Errorf("check failed: %s", c.Err)
}
}
// A check that panics must not take the process down through the probe, the
// same guarantee the built-in checks have.
func TestARegisteredCheckThatPanicsFailsRatherThanCrashes(t *testing.T) {
isolate(t)
Register("boom", func(context.Context) error { panic("registry unreachable") })
checks := Ready(context.Background())
c, ok := findCheck(checks, "boom")
if !ok {
t.Fatal("the registered check was not asked")
}
if c.OK {
t.Error("a panicking check was reported OK")
}
}
func TestRegisteringTheSameNameTwicePanics(t *testing.T) {
isolate(t)
Register("dup", func(context.Context) error { return nil })
defer func() {
if recover() == nil {
t.Error("registering a duplicate name did not panic; two checks under " +
"one name make the failing one impossible to identify")
}
}()
Register("dup", func(context.Context) error { return nil })
}
func TestRegisterRefusesAnEmptyNameOrNilCheck(t *testing.T) {
isolate(t)
for _, tc := range []struct {
name string
fn func(context.Context) error
why string
}{
{"", func(context.Context) error { return nil }, "empty name"},
{"nilfn", nil, "nil function"},
} {
func() {
defer func() {
if recover() == nil {
t.Errorf("%s did not panic", tc.why)
}
}()
Register(tc.name, tc.fn)
}()
}
}
+91 -17
View File
@@ -1,29 +1,103 @@
package middleware
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"net/http"
)
// defaultDemoMsg is what a refused request is told when nothing is configured.
//
// It is the string this middleware used to carry hard-coded, kept verbatim so
// that a deployment which never set application.demomsg is answered exactly as
// it was before.
const defaultDemoMsg = "谢谢您的参与,但为了大家更好的体验,所以本次提交就算了吧!\U0001F600\U0001F600\U0001F600"
// demoWriteRoutes are routes that change something despite being registered as
// GET, so the method alone does not say whether they are safe to serve.
//
// All three belong to the code generator: two write Go source files onto the
// server's filesystem and the third inserts menus, APIs and casbin rules into
// the database. They are registered under a group whose own name says it does
// no role check, and a demo deployment lets anybody log in - so on a demo host
// they were reachable by any visitor, and the menus one had in fact been used.
//
// Spelled as gin route patterns, which is what Context.FullPath returns, so a
// path parameter matches whatever value it is given.
//
// This list cannot be checked from here: common/ may not import app/, so this
// package cannot see which routes exist. What keeps it honest is a test beside
// the routes themselves - see app/other/router - which registers them and
// fails if any entry here has stopped being a real route.
//
// It also does not close the general hole. Nothing stops the next GET handler
// that writes something from being added without an entry here, and no static
// check can tell a handler that writes from one that reads. Demo mode refuses
// the routes it has been told about; that is the whole of the guarantee.
var demoWriteRoutes = map[string]bool{
"/api/v1/gen/toproject/:tableId": true,
"/api/v1/gen/apitofile/:tableId": true,
"/api/v1/gen/todb/:tableId": true,
}
// DemoWriteRoutes returns the routes demo mode refuses despite their method.
//
// Exported only so the test that lives beside the route registrations can
// check every one of them still exists; nothing else should need it.
func DemoWriteRoutes() []string {
out := make([]string, 0, len(demoWriteRoutes))
for route := range demoWriteRoutes {
out = append(out, route)
}
return out
}
// demoAllows reports whether demo mode lets a request through.
//
// route is the matched gin route pattern and uri the raw request target; the
// two are different things and both are needed. The route is what identifies a
// handler regardless of the values in its path parameters, and it is empty for
// a request that matched nothing - which is why the login and logout checks
// still read the raw target, as they always did.
func demoAllows(method, route, uri string) bool {
if demoWriteRoutes[route] {
return false
}
return method == http.MethodGet ||
method == http.MethodOptions ||
uri == "/api/v1/login" ||
uri == "/api/v1/logout"
}
// demoMessage is the answer a refused request gets.
//
// application.demomsg has been in the configuration all along and nothing read
// it: the message was hard-coded here, and the demo host's configured string
// happened to be identical, so the setting looked like it worked. An empty
// value falls back rather than answering with nothing.
func demoMessage() string {
if msg := config.ApplicationConfig.DemoMsg; msg != "" {
return msg
}
return defaultDemoMsg
}
// DemoEvn refuses anything that would change state while mode is demo.
func DemoEvn() gin.HandlerFunc {
return func(c *gin.Context) {
method := c.Request.Method
if config.ApplicationConfig.Mode == "demo" {
if method == "GET" ||
method == "OPTIONS" ||
c.Request.RequestURI == "/api/v1/login" ||
c.Request.RequestURI == "/api/v1/logout" {
c.Next()
} else {
c.JSON(http.StatusOK, gin.H{
"code": 500,
"msg": "谢谢您的参与,但为了大家更好的体验,所以本次提交就算了吧!\U0001F600\U0001F600\U0001F600",
})
c.Abort()
return
}
if config.ApplicationConfig.Mode != "demo" {
c.Next()
return
}
c.Next()
if demoAllows(c.Request.Method, c.FullPath(), c.Request.RequestURI) {
c.Next()
return
}
c.JSON(http.StatusOK, gin.H{
"code": 500,
"msg": demoMessage(),
})
c.Abort()
}
}
+144
View File
@@ -0,0 +1,144 @@
package middleware
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
// demoMode puts the process in demo mode for one test and puts it back.
func demoMode(t *testing.T, mode, msg string) {
t.Helper()
previousMode, previousMsg := config.ApplicationConfig.Mode, config.ApplicationConfig.DemoMsg
t.Cleanup(func() {
config.ApplicationConfig.Mode = previousMode
config.ApplicationConfig.DemoMsg = previousMsg
})
config.ApplicationConfig.Mode, config.ApplicationConfig.DemoMsg = mode, msg
}
// The method is not enough on its own. Three of the generator's routes are
// registered as GET and write anyway - two of them onto the server's
// filesystem, one into the database - so a guard that reads only the method
// serves them to anybody who can log in, which on a demo host is everybody.
func TestDemoRefusesTheWritesThatAreServedOverGET(t *testing.T) {
const login = "/api/v1/login"
for _, tc := range []struct {
name string
method string
route, uri string
wantThrough bool
}{
{"a plain read", http.MethodGet, "/api/v1/dept", "/api/v1/dept", true},
{"a write, by method", http.MethodPost, "/api/v1/dept", "/api/v1/dept", false},
{"login is how a visitor gets in", http.MethodPost, login, login, true},
{"logout", http.MethodPost, "/api/v1/logout", "/api/v1/logout", true},
{"preflight", http.MethodOptions, "/api/v1/dept", "/api/v1/dept", true},
// A request that matched no route has an empty pattern, and the guard
// still has to refuse it by method - this is what a POST to a path
// that does not exist looks like from in here.
{"a write to nothing at all", http.MethodPost, "", "/api/v1/__probe__", false},
// The three this change is about.
{"generator writes the database", http.MethodGet,
"/api/v1/gen/todb/:tableId", "/api/v1/gen/todb/3", false},
{"generator writes source files", http.MethodGet,
"/api/v1/gen/toproject/:tableId", "/api/v1/gen/toproject/3", false},
{"generator writes an api file", http.MethodGet,
"/api/v1/gen/apitofile/:tableId", "/api/v1/gen/apitofile/3", false},
// The read-only half of the generator has to keep working, or the demo
// host cannot demonstrate the feature at all. Refusing too much is as
// much of a defect as refusing too little.
{"generator preview stays available", http.MethodGet,
"/api/v1/gen/preview/:tableId", "/api/v1/gen/preview/3", true},
{"generator table tree stays available", http.MethodGet,
"/api/v1/gen/tabletree", "/api/v1/gen/tabletree", true},
{"table list stays available", http.MethodGet,
"/api/v1/db/tables/page", "/api/v1/db/tables/page", true},
{"column list stays available", http.MethodGet,
"/api/v1/db/columns/page", "/api/v1/db/columns/page", true},
} {
t.Run(tc.name, func(t *testing.T) {
if got := demoAllows(tc.method, tc.route, tc.uri); got != tc.wantThrough {
t.Errorf("demoAllows(%s %s) = %v, want %v", tc.method, tc.route, got, tc.wantThrough)
}
})
}
}
// Everything above is about demo mode only. A deployment that is not a demo
// runs the generator for real, and a guard that reached it there would have
// taken the feature away from every production install.
func TestOutsideDemoModeNothingIsRefused(t *testing.T) {
demoMode(t, "prod", "")
gin.SetMode(gin.TestMode)
for _, route := range append(DemoWriteRoutes(), "/api/v1/dept") {
t.Run(route, func(t *testing.T) {
r := gin.New()
r.Use(DemoEvn())
r.GET(route, func(c *gin.Context) { c.String(http.StatusOK, "served") })
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, requestFor(route), nil))
if w.Body.String() != "served" {
t.Errorf("answered %q; outside demo mode the handler must run", w.Body.String())
}
})
}
}
// The refusal has to come back as the demo message rather than a 403 or a 404:
// the front end shows it to the visitor, and the point of a demo host is that
// being turned away is explained.
func TestDemoRefusalCarriesTheConfiguredMessage(t *testing.T) {
gin.SetMode(gin.TestMode)
const route = "/api/v1/gen/todb/:tableId"
for _, tc := range []struct {
name, configured, want string
}{
{"configured", "come back tomorrow", "come back tomorrow"},
// A deployment that never set application.demomsg keeps the answer it
// already had; an empty setting must not become an empty message.
{"not configured", "", defaultDemoMsg},
} {
t.Run(tc.name, func(t *testing.T) {
demoMode(t, "demo", tc.configured)
r := gin.New()
r.Use(DemoEvn())
r.GET(route, func(c *gin.Context) { c.String(http.StatusOK, "served") })
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, requestFor(route), nil))
if w.Code != http.StatusOK {
t.Errorf("answered %d, want 200 so the front end reads the body", w.Code)
}
if body := w.Body.String(); !strings.Contains(body, tc.want) {
t.Errorf("body %q does not carry %q", body, tc.want)
}
if strings.Contains(w.Body.String(), "served") {
t.Error("the handler ran; the request was supposed to be refused")
}
})
}
}
// requestFor turns a route pattern into a request target by giving every path
// parameter a value.
func requestFor(route string) string {
segments := strings.Split(route, "/")
for i, segment := range segments {
if strings.HasPrefix(segment, ":") {
segments[i] = "1"
}
}
return strings.Join(segments, "/")
}
+118
View File
@@ -8,10 +8,12 @@
package storage
import (
"context"
"log"
"sync"
"github.com/go-admin-team/go-admin-core/v2/captcha"
corelog "github.com/go-admin-team/go-admin-core/v2/logger"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
@@ -21,6 +23,7 @@ func Setup() {
setupCache()
setupCaptcha()
setupQueue()
registerQueueDrain()
}
func setupCache() {
@@ -41,8 +44,123 @@ var (
// shut it down, and counted so a consumer can tell one from the next.
installed interface{ Shutdown() }
installedGen uint64
// drainRegistered records that the BeforeExit callback is on the runtime,
// so that a reload does not add another one.
drainRegistered bool
)
// setShutdown is sdk.Runtime.SetShutdown, indirected so that registering can
// be observed.
//
// It has to be: the runtime does not report how many callbacks a phase holds,
// and shutdownQueue takes the adapter on its first run, so every registration
// after the first returns immediately and changes nothing anybody can see. A
// reload adding one callback per round would therefore be invisible from the
// outside - which is exactly how it would survive.
var setShutdown = func(f func(context.Context)) { sdk.Runtime.SetShutdown(f) }
// registerQueueDrain puts shutdownQueue on the BeforeExit phase, once.
//
// Setup is one of the callbacks bootstrap.SetupConfig re-runs on every
// configuration change, so registering from it without a guard would leave one
// callback per reload - each shutting down the same adapter, each reported
// separately when the budget runs out.
//
// A flag under the existing mutex rather than a sync.Once: the tests in this
// package already save and restore installed and installedGen to keep one test
// from deciding what the next one sees, and a sync.Once cannot be put back.
func registerQueueDrain() {
queueMu.Lock()
first := !drainRegistered
drainRegistered = true
queueMu.Unlock()
if first {
setShutdown(shutdownQueue)
}
}
// drainedInTime shuts q down and reports whether it finished before ctx expired.
func drainedInTime(ctx context.Context, q interface{ Shutdown() }) bool {
done := make(chan struct{})
go func() {
defer close(done)
q.Shutdown()
}()
return finishedBeforeDeadline(ctx, done)
}
// finishedBeforeDeadline waits for done or for ctx, and resolves a tie in
// favour of done.
//
// The tie is the reason this is a function of its own rather than one select
// inline. Both channels can be ready when the select runs, select picks at
// random among ready cases, and so a single look reports an overrun for a
// drain that completed - about half the times it lands there, which is exactly
// often enough to be dismissed as noise. core's own RunShutdown re-checks for
// this reason.
//
// Taking channels rather than a queue is what makes it testable: a closed done
// and an expired ctx can be handed in together, which is the state a race
// would otherwise have to be caught in.
func finishedBeforeDeadline(ctx context.Context, done <-chan struct{}) bool {
select {
case <-done:
return true
case <-ctx.Done():
}
select {
case <-done:
return true
default:
return false
}
}
// shutdownQueue drains the queue this package installed, on the way out.
//
// Nothing used to. core's Memory.Shutdown closes the queue and waits for every
// consumer to finish what it is holding, and the legacy adapter cancels its
// context and closes the underlying queue - but neither ran at exit, so the
// process left with the login log, the operation log and the API sync still
// buffered, and left reporting success.
//
// The adapter is read here rather than captured at registration because a
// reload replaces it. Registration happens once per process; this runs against
// whatever is current when the signal arrives.
//
// Only an adapter this package installed. sdk.Runtime.GetQueueAdapter never
// returns nil - with no queue section configured the runtime wraps its own
// fallback queue - so going through that accessor would shut down a queue this
// package neither built nor started.
//
// The adapter is taken, not read: after this the package owns nothing, so a
// reload arriving mid-shutdown builds a new one instead of being handed a
// closed one to shut down again. Both implementations tolerate a second
// Shutdown, so this is about who owns it rather than about a crash.
func shutdownQueue(ctx context.Context) {
queueMu.Lock()
q := installed
installed = nil
queueMu.Unlock()
if q == nil {
return
}
if drainedInTime(ctx, q) {
corelog.Info("queue: drained")
return
}
// The wait is what the budget bounds, not the work: Shutdown takes no
// context and is still running in that goroutine. Saying so here names what
// is being lost, which the generic overrun message cannot.
corelog.Warnf("queue: the shutdown budget ran out while the queue was still draining - " +
"whatever it had not delivered goes with the process. Raise extend.shutdown.cleanup " +
"if this recurs.")
}
// QueueGeneration reports how many times this package has installed a queue
// adapter. It changes every time setupQueue builds a new one, which is on
// every configuration reload, and stays 0 for as long as the configuration has
+333
View File
@@ -0,0 +1,333 @@
package storage
import (
"context"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
)
// countingQueue stands in for an installed adapter. Only Shutdown is exercised
// - the drain callback never publishes or consumes - so the rest of
// AdapterQueue is deliberately absent: `installed` is typed on Shutdown alone,
// and widening the fake would only invite it to be used for something else.
type countingQueue struct {
calls atomic.Int32
block chan struct{}
// started is closed on the way into Shutdown, so a test can wait for the
// call rather than assume the goroutine that makes it was scheduled. The
// caller returns on its own deadline while Shutdown is still running, so
// reading calls straight after that return is a race with the increment.
startOnce sync.Once
started chan struct{}
}
func newCountingQueue() *countingQueue {
return &countingQueue{started: make(chan struct{})}
}
func (q *countingQueue) Shutdown() {
q.startOnce.Do(func() { close(q.started) })
q.calls.Add(1)
if q.block != nil {
<-q.block
}
}
// waitStarted blocks until Shutdown has been entered, or fails the test.
func (q *countingQueue) waitStarted(t *testing.T) {
t.Helper()
select {
case <-q.started:
case <-time.After(5 * time.Second):
t.Fatal("Shutdown was never called")
}
}
// isolate gives the test its own runtime and its own view of what this package
// has installed, and puts the process-wide state back afterwards.
//
// Same isolation as TestSetupBumpsTheQueueGenerationOnEveryReload, plus
// drainRegistered: it is what stops a reload registering a second callback, so
// leaving it set would make every later test in this binary see a package that
// has already registered.
func isolate(t *testing.T) {
t.Helper()
prevQ, prevC := config.QueueConfig, config.CacheConfig
prevRuntime := sdk.Runtime
queueMu.Lock()
prevInstalled, prevGen, prevRegistered := installed, installedGen, drainRegistered
queueMu.Unlock()
t.Cleanup(func() {
config.QueueConfig, config.CacheConfig = prevQ, prevC
sdk.Runtime = prevRuntime
queueMu.Lock()
installed, installedGen, drainRegistered = prevInstalled, prevGen, prevRegistered
queueMu.Unlock()
})
sdk.Runtime = runtime.NewConfig()
queueMu.Lock()
installed, drainRegistered = nil, false
queueMu.Unlock()
}
// setInstalled puts a fake where setupQueue would have left the real adapter.
//
// Legitimate because the callback reads `installed` when it runs rather than
// capturing it at registration - that is the property that lets a reload
// replace the adapter and still have the right one drained.
func setInstalled(q interface{ Shutdown() }) {
queueMu.Lock()
installed = q
queueMu.Unlock()
}
func currentInstalled() interface{ Shutdown() } {
queueMu.Lock()
defer queueMu.Unlock()
return installed
}
// Issue #911: nothing shut the queue down at exit, so whatever was buffered
// went with the process.
//
// This is the half that matters most - a callback is on BeforeExit and it
// reaches the adapter this package installed. It says nothing about how many
// times the callback was registered; see the test below for that.
func TestSetupPutsTheQueueDrainOnBeforeExit(t *testing.T) {
isolate(t)
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
Setup()
Setup()
Setup()
q := newCountingQueue()
setInstalled(q)
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
t.Fatalf("RunShutdown: %v", err)
}
if got := q.calls.Load(); got != 1 {
t.Errorf("Shutdown called %d times, want 1 - 0 means nothing registered the drain", got)
}
}
// Setup is re-run on every configuration change, so registering from it has to
// be guarded: a callback per reload would leave the shutdown phase holding a
// row of identical entries, each timed and each eligible to be named as the one
// that overran the budget.
//
// Counted at the seam rather than through the effect. The test above cannot see
// this - shutdownQueue takes the adapter on its first run, so the second and
// third callbacks find nothing and return, and three registrations produce
// exactly the same observable result as one. That is a good property of the
// callback and a blind spot for any test that goes through it.
func TestSetupRegistersTheDrainOncePerProcessHoweverManyReloads(t *testing.T) {
isolate(t)
previous := setShutdown
t.Cleanup(func() { setShutdown = previous })
registrations := 0
setShutdown = func(func(context.Context)) { registrations++ }
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
Setup()
Setup()
Setup()
if registrations != 1 {
t.Errorf("three reloads registered the drain %d times, want 1", registrations)
}
}
// The drain reaches the adapter that is current when the signal arrives, not
// one captured while wiring up. A reload replaces the adapter, and draining the
// one that was installed at start-up would drain something nobody has published
// to since.
func TestTheDrainRunsAgainstTheAdapterInstalledLast(t *testing.T) {
isolate(t)
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
Setup()
first, second := newCountingQueue(), newCountingQueue()
setInstalled(first)
setInstalled(second)
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
t.Fatalf("RunShutdown: %v", err)
}
if first.calls.Load() != 0 {
t.Error("the adapter that was replaced was shut down; the callback captured it instead of " +
"reading it when it ran")
}
if second.calls.Load() != 1 {
t.Errorf("the current adapter was shut down %d times, want 1", second.calls.Load())
}
}
// Nothing installed is the shipped default: settings.yml has no queue section,
// so setupQueue returns early and the runtime's own fallback queue is what
// callers get. Shutting that down would close a queue this package neither
// built nor started.
func TestTheDrainDoesNothingWhenThisPackageInstalledNothing(t *testing.T) {
isolate(t)
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{}
Setup()
if currentInstalled() != nil {
t.Fatal("an empty queue configuration installed an adapter, so this test asserts nothing")
}
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
t.Fatalf("RunShutdown: %v", err)
}
}
// The budget bounds the wait, not the work. A consumer that never finishes must
// not hold the process past its grace period - SIGKILL would arrive mid-write
// instead of at a point of the process's choosing.
func TestTheDrainStopsWaitingWhenTheBudgetIsGone(t *testing.T) {
isolate(t)
blocked := newCountingQueue()
blocked.block = make(chan struct{})
t.Cleanup(func() { close(blocked.block) })
setInstalled(blocked)
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
returned := make(chan struct{})
go func() {
defer close(returned)
shutdownQueue(ctx)
}()
select {
case <-returned:
case <-time.After(5 * time.Second):
t.Fatal("shutdownQueue did not return after its context expired - it waits on a Shutdown " +
"that takes no context, so the wait has to be bounded here")
}
// Waited for rather than read straight after the return: Shutdown runs on a
// goroutine that the caller does not join, so the increment is not ordered
// against the caller giving up on its deadline.
blocked.waitStarted(t)
if blocked.calls.Load() != 1 {
t.Errorf("Shutdown called %d times, want 1 - the drain has to be attempted even when it "+
"cannot be waited out", blocked.calls.Load())
}
}
// After the drain this package owns nothing. A reload arriving mid-shutdown
// then builds a new adapter rather than being handed a closed one as its
// `previous` to shut down again.
func TestTheDrainGivesUpOwnershipOfTheAdapter(t *testing.T) {
isolate(t)
setInstalled(newCountingQueue())
shutdownQueue(context.Background())
if got := currentInstalled(); got != nil {
t.Errorf("installed is %T after the drain, want nil", got)
}
}
// runtimeQueue is a full AdapterQueue, so it can be handed to the runtime
// rather than only to this package's own record of what it installed.
type runtimeQueue struct {
countingQueue
}
func (q *runtimeQueue) String() string { return "runtime-fake" }
func (q *runtimeQueue) Append(corestorage.Messager) error { return nil }
func (q *runtimeQueue) Register(string, corestorage.ConsumerFunc) {}
func (q *runtimeQueue) Run() {}
// The drain must not reach a queue this package did not install.
//
// sdk.Runtime.GetQueueAdapter never returns nil: with no queue section
// configured it wraps the runtime's own fallback, and the wrapper's Shutdown
// forwards. Reaching for the accessor would therefore look like it worked and
// would close a queue this package neither built nor started - the same shape
// as the `if q := GetQueueAdapter(); q != nil` that setupQueue already had to
// drop.
//
// The previous test's empty-configuration case cannot see this: it only checks
// that RunShutdown returns, which it would either way.
func TestTheDrainNeverReachesTheRuntimesOwnQueue(t *testing.T) {
isolate(t)
onTheRuntime := &runtimeQueue{countingQueue: *newCountingQueue()}
sdk.Runtime.SetQueueAdapter(onTheRuntime)
if currentInstalled() != nil {
t.Fatal("this package installed something, so the distinction under test is not set up")
}
if sdk.Runtime.GetQueueAdapter() == nil {
t.Fatal("the accessor returned nil, so it is no longer the trap this guards")
}
shutdownQueue(context.Background())
if got := onTheRuntime.calls.Load(); got != 0 {
t.Errorf("the runtime's queue was shut down %d times - the drain went through "+
"GetQueueAdapter instead of the adapter this package installed", got)
}
}
// A drain that finishes in the same instant the budget expires counts as
// finished.
//
// Both channels are ready when the select runs, and select picks at random
// among ready cases, so a single look reports an overrun for a drain that
// completed - roughly half the times it lands here. The repetition is what
// makes that visible: one iteration passes either way.
func TestATieBetweenTheDeadlineAndTheDrainGoesToTheDrain(t *testing.T) {
expired, cancel := context.WithCancel(context.Background())
cancel()
<-expired.Done()
done := make(chan struct{})
close(done)
for i := 0; i < 1000; i++ {
if !finishedBeforeDeadline(expired, done) {
t.Fatalf("iteration %d of 1000: both the deadline and the drain were ready and the "+
"deadline won - a drain that completed is being reported as an overrun", i)
}
}
}
// The other side of it. A drain that really has not finished has to be
// reported, or the warning never fires and the tie-break above has quietly
// turned into "always say it drained".
func TestADrainThatHasNotFinishedIsReportedAsAnOverrun(t *testing.T) {
expired, cancel := context.WithCancel(context.Background())
cancel()
<-expired.Done()
stillRunning := make(chan struct{}) // never closed
if finishedBeforeDeadline(expired, stillRunning) {
t.Error("an unfinished drain was reported as having finished in time")
}
}
+1 -1
View File
@@ -11,7 +11,7 @@ require (
github.com/casbin/casbin/v3 v3.8.1
github.com/gin-gonic/gin v1.12.0
github.com/glebarez/sqlite v1.11.0
github.com/go-admin-team/go-admin-core/v2 v2.7.0
github.com/go-admin-team/go-admin-core/v2 v2.8.0
github.com/google/uuid v1.6.0
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible
github.com/mssola/user_agent v0.6.0
+2 -2
View File
@@ -145,8 +145,8 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-admin-team/go-admin-core/v2 v2.7.0 h1:1qV0/5iFBvkE3BRtm4ip0v0QYG9Fgx4UtOTd8zkQT9c=
github.com/go-admin-team/go-admin-core/v2 v2.7.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-admin-team/go-admin-core/v2 v2.8.0 h1:ZTw5Z/UT1/7OltbGPEaEVerRk4z3koB6O8nDbb84tPM=
github.com/go-admin-team/go-admin-core/v2 v2.8.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o=
+14 -4
View File
@@ -23,10 +23,20 @@ metadata:
version: v1
spec:
# One replica, and the drain window below buys nothing at one replica: there
# is nowhere to send the traffic this pod stops taking. Raising it needs one
# more change than the number - the volume below is shared by every replica,
# and the log path in settings.yml lives on it, so a second pod would append
# to the same rotating file.
# is nowhere to send the traffic this pod stops taking. Raising it needs two
# changes that are not this number:
#
# The volume below is shared by every replica, and the log path in
# settings.yml lives on it, so a second pod would append to the same
# rotating file.
#
# The job scheduler is per process while its handle on a job is one shared
# column. Startup runs `UPDATE sys_job SET entry_id = 0 WHERE entry_id > 0`
# across the whole table (app/jobs/jobbase.go), so a second pod erases the
# first pod's ids and writes its own, and every pod registers the whole
# enabled list in its own scheduler. Neither symptom logs anything: an
# enabled job fires once per pod, and stopping one from the UI removes an
# entry from the wrong process and still answers 200. See #915.
replicas: 1
selector:
matchLabels: