Compare commits

..
Author SHA1 Message Date
zhangwenjian d01cdc040f fix🐛: accept the import table list from a JSON body as well
The generator's import reads its comma-separated table list with
c.Request.FormValue("tables"), which on a request declaring itself as JSON reads
the URL query and nothing else. go-admin-ui v3.2.0 began sending that list in
the body, so the handler saw an empty string, asked information_schema for a
table named "", and every import failed with "table name cannot be empty!" —
on a fresh installation that is the first thing the generator is asked to do.

tablesToImport reads the query first and falls back to the body, so a front end
sending either works against this server. It also drops blank entries:
splitting "" yields one empty name rather than nothing at all, which is why the
old code reached a database query at all before failing.

The front end sends the list in the query again on its side; this half is what
lets an installation already running v3.2.0 recover without changing it.
2026-09-16 17:57:51 +08:00
zhangwenjian 92b9af17b7 refactor🎨: name the empty-table-name message once
The string was spelled out at each site that raises it, and once more in the
test file that asserts on it. A test holding its own copy cannot tell the
difference between the handler answering something else and the message having
been reworded: it goes on asserting a string the server no longer sends, and
goes on passing.

The three copies in app/other/models/tools are left alone; they are raised from
a different layer and nothing asserts on them.
2026-09-16 17:57:37 +08:00
zhangwenjian 898e1b023a refactor🎨: share the generator tests' engine and response decoding
newEngine takes the method, path and handler, so a second test file does not
have to restate the sqlite connection, the driver override and its cleanup, the
CustomError middleware and the two context keys. serveJSON does the same for
running one request and decoding the envelope.

Nothing about what is asserted changes; newColumnListEngine and columnListMsg
keep their names and their callers.
2026-09-16 17:57:28 +08:00
wenjianzhang a90c67473e Merge pull request #924 from jackwalkerlabs/fix/job-stop-timeout-890
fix🐛: report job stop timeouts as errors
2026-09-16 08:02:47 +08:00
wenjianzhang 1a84b8a892 Merge pull request #930 from Tuoxie423/docs/remove-dead-contributor-links
docs📝: 删除 README 中失效的贡献者链接
2026-09-16 08:02:43 +08:00
wenjianzhang 656d14cd54 Merge pull request #934 from go-admin-team/fix/928-929-seed-repair
Reseeding a menu tree repairs what it finds, and finds what it wrote
2026-09-14 14:52:33 +08:00
zhangwenjian 9dd271ecab fix🐛: claim the rows an application wrote before seed_code existed
1786700008000 added sys_menu.seed_code and left it NULL on every row that was
already there. That is right for the host's own hand-placed menus: there is
nothing to derive one from.

An application's rows are in that population too, and for those it is
derivable - menu_name is what identified them before the column existed. The
natural-key lookup missed them, so a reseed inserted a second copy beside each
one, and the new unique index could not object, because NULL never collides on
MySQL, PostgreSQL or SQLite and is filtered out of the index on SQL Server.

Claimed when the application is seeded rather than by a backfill migration.
The value is only derivable where the spec's own Code is in hand: menuName
concatenates two pascalCase strings and does not reverse, so a migration
looking at menu_name alone would be guessing. For the same reason more than
one match is refused and named rather than picked from - attaching an
application's menu to whichever row the database returned first is the failure
this is meant to prevent, not a smaller version of it.

The match is scoped to the application's own app_code, so a row belonging to
another application, or to the host, is not claimed.

An adopted row then goes through the ordinary repair, so it comes out carrying
what the spec says rather than what it held from before.

Three degradations turn the new assertions red: not adopting at all, picking a
row when there is more than one, and dropping the app_code from the match.
2026-09-13 20:34:04 +08:00
zhangwenjian 4973ee030d fix🐛: bring an existing seeded menu up to what the spec says
repairExistingMenu reconciled a row's paths and its api bindings and left
every other column as an earlier run had written it. That cost three different
things, and none of them announced itself.

A menu whose parent was removed and seeded again kept parent_id pointing at
the dead row while its paths named the new one. The tree is built from
parent_id - SysMenu.GetPage walks down from ParentId == 0 - so the menu was
gone from the sidebar, with the migration reporting success.

A menu somebody added by hand under a seeded one kept the old prefix when its
ancestor moved. It is in no spec, so nothing else would ever rewrite it;
SysMenu.Update already does this cascade for the same column when a menu is
moved through the UI.

An application that renamed a menu, or moved its component, in a new version
had the change ignored: the row was found by its natural key and returned
untouched.

The row a spec describes now has one definition, and both the insert and the
repair use it - they cannot drift into disagreeing about what a spec decides.
The repair writes every column on that list.

Visible and IsFrame are deliberately not on it. They are seeding defaults the
application never expressed, so an administrator who hid a seeded menu keeps
it hidden; there is a test that hides one and reseeds.

The cascade matches the row itself or a row strictly underneath it, rather
than `paths LIKE old || '%'`, which also catches /0/1/20 when old is /0/1/2.
An empty old path takes the single-row branch instead: there is no subtree
under one, and the LIKE would have matched the whole table.

Five degradations turn the new assertions red: not writing the spec columns at
all, leaving ParentId off the list, putting Visible on it, not cascading, and
cascading on the loose prefix. The last one did not, at first - the decoy rows
were built against the path of the menu whose parent moved rather than the
path that actually gets rewritten, so the prefix they collided with was never
the one passed to the query.
2026-09-13 20:33:47 +08:00
wenjianzhang 137bb3ad33 Merge pull request #931 from go-admin-team/feat/008-wire-example
008 layer three: status reads sys_app, dependencies are checked, and the example gets installed for real
2026-09-12 21:17:33 +08:00
zhangwenjian 03d587db6a ci👷: give the end-to-end install a make target
It was an inlined `go test` in the workflow, and the only thing in the build
that runs it. Every other gate there goes through make - make test, make
build, make checksilent - and `make test` is `go test ./...` in this module,
which cannot reach test/e2e-apporder because that is a module of its own.

So the one check that exercises installing an application was the one check a
developer had no command for, and the only place it could turn red was after
pushing.
2026-09-11 08:23:42 +08:00
zhangwenjian 1f56b956d2 test✅: build the end-to-end binary once, and assert from one list
Three tests each called newEnv, and newEnv built the binary, so the same
binary was linked three times - about 17 seconds of the run, measured. A
binary is read-only and there is nothing to isolate between tests; each test
still gets its own directory and its own database. The package now builds it
on the first test that needs one and removes it in TestMain. The suite goes
from 39 seconds to 9.

The three assertion blocks listed the same six queries, two or three times
each, differing only in the counts expected - so renaming a table meant
finding three places. They now share one list, with a flag for the uninstall's
"all of them at zero". The reinstall check gets stronger on the way past: it
was three of the six and is now all six.

The per-call sql.Open in count and exec stays. It looks like waste and is not:
the binary under test writes the same file, and a connection held open across
a run of it is a second writer for nothing. The shared part is factored out;
the opening is still per call, and the comment now says why.
2026-09-11 08:21:36 +08:00
zhangwenjian 37aece9791 test✅: share one sys_app fixture, and count through the helper that checks
Two tests in this package each wrote out what an installed sys_app row looks
like, field for field, and a third inlined the same Create with a different
status. One appRow(t, db, code, status) now covers all three, so a new NOT
NULL column on SysApp is one edit rather than three.

One assertion counted with a bare db.Model(...).Count(&n) and dropped the
error that call returns. A failing query leaves n at zero, which is exactly
what that assertion wanted to see - so the test would have passed on a broken
query. The package already had a count helper that fails on the error, and
this now uses it.

Also a cycle reached from outside itself. The existing case walks straight
into its own cycle from the first code, so the path trimming had nothing to do
and replacing it with the untrimmed path left the test green - the trimming
was never covered. With a requiring b, b requiring c and c requiring b, the
untrimmed report names a as part of a cycle it is not in, and the test goes
red.
2026-09-11 08:21:35 +08:00
zhangwenjian 309b400bc0 refactor♻️: take one registry snapshot, and one route into sys_app
Cleanup from a review pass over this branch. No behaviour changes except the
two noted below.

runInstall took app.Snapshot() twice, once inside manifestFor and once for the
cycle check. Snapshot is a deep copy of the registry, and worse than the
copying, the two calls could in principle disagree - the set the cycle check
validated was not provably the set the manifest came from. One snapshot,
passed to both.

appSummary converted a display code back to a stored one with
NormalizeAppCode, which is not that inverse: it leaves "core" as "core", so
the framework needed a branch of its own to stay out of the listing. AppFilter
is the documented inverse and maps it to the empty string, which is not a code
any row is filed under - so the branch goes, and the function now matches
filterAppsByApp twenty lines below it, which was already using AppFilter.

That branch only half-covered what it guarded: a sys_app row carrying an empty
or reserved app_code was still merged into the framework's group by
groupByApp, with only its summary suppressed. loadApps now drops such rows,
which is the one place that settles it for every reader of the map.

requiresInstalled built two parallel slices with a tuple assignment repeated in
three branches; it now picks a reason and appends once. Its last arm was a
catch-all on "not installed", so a status constant added later would have been
described as "did not finish" - a sentence that would be wrong for whatever
reason the constant was added. Unrecognised values now say so. It also takes
the normalised code the caller already has rather than computing it a third
time.

refuseOnDependencyCycle sorted each manifest's Requires before walking them.
Requires is a slice and already has a fixed order, so the sort bought no
determinism - that comes from the sorted outer loop, which walks a map - and
only made a reported cycle harder to line up against the manifest that caused
it. The filtering pass that went with it is covered by the registration check
underneath. The cycle path is trimmed with slices.Index, which also removes a
fallback return that the grey/path invariant made unreachable.
2026-09-11 08:20:56 +08:00
zhangwenjian 1b9868b72b feat✨: refuse an install whose dependencies are not installed
An application's manifest can name others it needs. Until now the list was
stored and never read.

It is checked, not satisfied. Installing the dependencies too would make
"install this application" mean "and everything it happens to name, and
everything those name" - a blast radius the operator did not ask for and
cannot see beforehand. What they get is the list and the order to do it in.

A dependency whose own install failed, or never finished, is not a dependency
that is there. The message says which, because the two send you to different
places: one to install it, the other to look at why it did not take.

The check runs before anything is written, so a refusal cannot cost the
operator the row that told them what they had.

Separately, a cycle anywhere in the registered manifests is refused, whether
or not the application being installed is in it. A cycle between two others is
still an authoring mistake, and the day somebody installs into it - with an
error naming two applications they did not ask for - is the worse time to find
out. The error is the cycle rather than the walk that reached it, and the
walk's order is sorted, so the same set of manifests always reports the same
one. Requires naming an application that is not registered is not a cycle; it
is the database's answer to give, at the time it matters.

Six degradations turn the new assertions red: accepting any dependency,
accepting a row regardless of its status, returning no cycle, not trimming the
reported path to the cycle itself, and running the check after the row has
already been written - the last of which was rebuilt after the first attempt
at it deleted the check rather than moving it, and so went red on the wrong
assertion.
2026-09-10 21:30:29 +08:00
zhangwenjian 25344aa572 feat✨: let migrate status say what sys_app knows
The migration rows answer "did this run". They cannot answer "is this
application installed", and the difference is not academic: an install that
stopped partway leaves every migration reading applied and a row saying the
install never finished. Until now nothing printed that row.

    [order]  1.0.0 failed at order-1793800000000
      applied   order-1793800000000  2026-09-10 21:02:07

The application list is the union of the two sources rather than either one.
Reading it from sys_app alone would drop an application whose migrations ran
under plain `migrate`, which records no row; reading it from the migration
rows alone drops one whose code has been taken out of the binary, which is
when somebody most wants to see it named - that one now gets a group of its
own, empty, saying why.

A database from before sys_app existed prints exactly what it printed before.
`migrate status` has to keep working on a database that has not been migrated
at all, which is when it is most wanted.

Four degradations turn the new assertions red: dropping the sys_app-only
applications from the listing, printing no summary, not narrowing sys_app by
--app, and reporting an unfinished install as an installed one.
2026-09-10 21:30:03 +08:00
拖鞋423 ea9d27cf6d docs📝: 删除 README 中失效的贡献者链接 2026-09-09 20:28:35 +08:00
zhangwenjian 0bee8ec46c ci👷: run the end-to-end install on every push
The separate module is invisible to `go test ./...` in the main module, which
is the point, and also means nothing would ever run it. This step does.
2026-09-09 16:40:15 +08:00
zhangwenjian 3a5afeb518 test✅: install and uninstall the example application end to end
Everything under `migrate install` was covered with an injected engine and a
hand-built schema, which is where the shapes belong. What none of it could
catch is the wiring: whether an application's init() reaches both registries,
whether the installer finds a manifest through app.Snapshot, whether the
seeder writes what the uninstaller goes looking for, and whether the command
exits non-zero when a migration fails - which a deployment reads to decide
whether to start the new version.

This builds a go-admin binary with the example application linked in, migrates
a real database with it, and drives the whole sequence: framework migrations
only, install, install again, put a row in the application's own table,
uninstall, reinstall.

It lives in its own module. A tagged import in the main module would still be
resolved by `go mod tidy`, which considers every build tag and would go
looking for github.com/go-admin-team/example-app-order on the network - a
repository that does not exist, because the example is a directory inside this
one. That was checked rather than assumed: tidy fails there with "Repository
not found". A build tag of `ignore` is skipped by tidy but cannot be turned on
either, because the standard library uses it for files that are not meant to
build at all. A separate module with replace directives is invisible to the
main module's tidy, its build, its tests and checksilent, and needs no
go.work.

Three degradations turn it red: the example application not registering a
manifest, the uninstall not clearing sys_migration - where the reinstall then
seeds nothing and the assertion reads "menus = 0, want 4" - and the seeder not
recording its grants, where the uninstall then leaves every policy behind.
2026-09-09 16:40:15 +08:00
zhangwenjian fc8ba4d615 feat✨: let the example application say what it is
app-order registered its migrations and its menus and nothing else, so
`migrate install order` answered that no application in the binary registers a
manifest. Which was true, and made the installer untestable against the one
application this repository ships.

The manifest goes in the migration package rather than one of its own because
that is the package a host has to import for the application to exist at all -
its migrations register from there too. A second package would be a second
thing to remember to import, and forgetting it would leave an application
whose migrations run and which no installer can name.

Its Version is not the migration version and the two move independently:
adding a migration file without renaming the application is normal, and so is
a release that changes no schema. The migration versions decide what runs;
this decides what the installed row says.

go-admin-core moves to v2.8.0, which is where contract/app lives.
2026-09-09 16:40:14 +08:00
wenjianzhang dc20062e5b Merge pull request #927 from go-admin-team/fix/sqlserver-null-unique
The seed natural-key indexes cannot be built on SQL Server
2026-09-09 16:19:26 +08:00
zhangwenjian ff430c509b ci👷: run the migration tests against SQL Server too
The fourth registered driver, and the one that disagrees with the other three
about NULL. A suite that never pointed at it reported success for a migration
no SQL Server database could apply - the same shape as the PostgreSQL gap that
put the postgres service here, one driver further along.

SQL Server has no equivalent of POSTGRES_DB, so the database the DSN names is
created in a step before the tests. The test helper fails rather than skips
when CI is set and the variable is not, so dropping the service or renaming
the variable cannot quietly go green.
2026-09-09 13:49:20 +08:00
zhangwenjian d43d7a46dd fix🐛: make the seed natural-key indexes buildable on SQL Server
1786700008000 could not be applied to any SQL Server database. Not an old one
with awkward data - any of them, including an empty one:

    Msg 1505 ... duplicate key ... The duplicate key value is (, <NULL>, 0).

MySQL, PostgreSQL and SQLite treat two NULLs in a unique index as different
values, so any number of rows missing a seed_code coexist under
uk_sys_menu_app_seed_code_del. SQL Server treats them as equal and permits
exactly one. 1786700001000 seeds five menus and none of them has a seed_code,
so the second one already collides with the first. sys_api's index has the
same shape over two nullable columns, path and action.

On SQL Server the index is now filtered to the rows that carry a value, which
is what the other three engines do by not comparing their NULLs. The filter is
not added elsewhere: MySQL has no filtered index at all, and on PostgreSQL and
SQLite it would only restate what those engines already do.

Nothing that has applied this migration is affected, and no SQL Server
database can have.

Verified against SQL Server 2022. The migration completes; the filtered index
still rejects a second (order, dir) and still lets another app reuse "dir",
so filtering removed the NULL rows from the index rather than the index's
teeth. Two degradations turn that red: dropping the filter, and naming only
path in sys_api's - the second one needed a fixture row with a path and no
action, because rows missing both are excluded either way and the first
attempt at that degradation came out green.

There is also a control test asserting the unfiltered statement still fails on
this engine, so the first test is passing because of the fix rather than
because SQL Server turned out not to mind.
2026-09-09 13:49:20 +08:00
wenjianzhang 72c496ab93 Merge pull request #926 from go-admin-team/feat/008-installer
008 layer two: the installer and the uninstaller
2026-09-09 13:43:08 +08:00
zhangwenjian b228152308 docs📝: drop a reference to a document this repository does not carry
Two comments added in this branch cite docs-prd/008-.../数据库变更.md by path.
That directory is not tracked here, so the citation reads as a file the reader
can open and cannot. The reasoning it pointed at is short enough to state in
place.

Three comments from the previous batch cite the same path and are left alone;
they belong to a different change.
2026-09-09 13:28:45 +08:00
zhangwenjian 006756ea40 test✅: cover the uninstall's chunk boundary and its empty id lists
Copilot could not review this branch - the account is over its review quota -
so these are what a second pass over the uninstaller turned up. No defect: the
three cases were uncovered rather than wrong.

findOrphanPolicies batches its OR chain because a driver runs out of
placeholders long before an application runs out of endpoints, and nothing
exercised the boundary. 205 paths across three batches, the last one short,
plus one policy no key names as a control. Taking one fewer per batch,
advancing one too far, and stopping after the first batch each turn it red.

An application with apis and no menus, and one with menus and no apis, are
both normal - endpoints another service calls, or a section with no endpoints
of its own - and each leaves one of the two id lists the uninstall reads
empty.

That last pair also corrected a comment. The guard in front of the join-table
delete was described as being there because an empty IN list is a syntax
error. It is in raw SQL, but GORM renders IN with an empty slice as a
condition that matches nothing, and removing the guard leaves the new test
green. It stays as a statement of intent, and now says so.
2026-09-09 12:56:33 +08:00
zhangwenjian aa539c061f feat✨: uninstall one application's menus, apis and grants
`migrate uninstall <code>` removes what an application's install wrote and
leaves the application's own tables alone. Removing an order module is not the
same decision as destroying the orders, and nothing here can tell an operator
who is done with it from one who will reinstall tomorrow.

One transaction, and this one really is one: every statement is DML or a
SELECT, so unlike an install there is no DDL to commit it out from under
itself. Child rows go first, while the ids that identify them can still be
read from their parents, and the api paths are read before the rows carrying
them are deleted.

The two join tables need no ledger and get none. menu_id is a surrogate key,
so a sys_role_menu or sys_menu_api_rule row can only have come from a menu
this application wrote - there is no "looks like it but is not". A column on
sys_role_menu would have been worse than unnecessary: SysRole.Update deletes a
role's rows and writes them back through GORM's many2many, which does not
carry extra columns, so the column would be blanked the first time anybody
edited a role, silently. There is a test that performs that edit and then
uninstalls.

casbin_rule is the opposite case, because its key is business text somebody
else may have written for their own reasons. Policies are removed one at a
time, by exact tuple, and only the ones the ledger says this install created.
A tuple the ledger names that is no longer there is reported, not treated as a
failure - the uninstall wanted it gone and it is gone. Then, with everything
the ledger could speak for already dealt with, a read-only pass lists the
policies still naming this application's paths: those are grants somebody made
by hand, they are about to point at APIs that no longer exist, and they are
not this command's to delete. The two lists stay separate because they mean
different things - one is something of ours that had already gone, the other
is somebody else's grant now pointing at nothing - and merged into one "could
not remove" list neither would be actionable.

sys_migration's rows for the application go too. Without that a reinstall
finds every version applied, runs no migration, seeds nothing, and reports
success. It is the easiest step to leave out, because a migration record does
not look like the application's data.

A sys_app row is not required. `migrate` with no subcommand applies every
registered migration, an application's included, so an application can have
all of its rows without ever having gone through the installer - and that is
the case where nothing else can clean up after it.

Eleven degradations were applied one at a time, each red on the assertion it
was aimed at: skipping either join table, deleting sys_role_menu without its
filter, skipping sys_migration, deleting sys_migration without its filter,
matching policies by path instead of by ledger tuple, dropping the orphan
pass, treating a missing policy as a failure, leaving the ledger behind, soft
deleting sys_menu instead of removing it, and running the whole thing outside
a transaction.
2026-09-09 12:35:35 +08:00
zhangwenjian 74b0ee8776 fix🐛: stop sys_menu declaring an index stricter than the real one
uk_sys_menu_app_seed_code_del covers (app_code, seed_code, deleted_at) and is
created by 1786700008000 with explicit SQL. The struct tag named the same
index on SeedCode alone, and a named uniqueIndex tag collects only the fields
carrying that name - so AutoMigrate on this model would build a unique index
on seed_code by itself: stricter than the real one, and forbidding two
applications from both having a "dir" node, which the composite key exists to
allow.

Worse than being stricter, it would win. The migration only creates its index
when HasIndex says the name is free, so a schema built by AutoMigrate first
keeps the wrong index and the migration steps over it without a word.

The tag cannot express the real index: deleted_at comes from the ModelTime
embed shared by every table, which no single model can add a tag to. So the
tag goes and the migration is the only thing that creates it.

No database is affected. The initial table migration AutoMigrates a frozen
snapshot of this model that has neither app_code nor seed_code, and nothing
else in the repository AutoMigrates the live one - which is why this stayed
invisible until a test built the schema from the live model and seeded two
applications, and got a unique-constraint failure on a seed code they are
supposed to be able to share.
2026-09-09 12:35:17 +08:00
zhangwenjian 412413c12f feat✨: record which casbin policies an install created
sys_app_casbin_grant has existed since the registry tables were added and
nothing ever wrote to it. An uninstaller reading it would have found it empty,
deleted no policy at all, and reported every one of them as an unattributable
leftover - which is what "report and skip" looks like when the ledger was
simply never written, and is indistinguishable from it working.

grantToAdminRole now writes an entry for each policy it creates. The entry
carries the tuple casbin_rule is unique on rather than a foreign key into it,
because casbin_rule is not this project's table: the gorm adapter's SavePolicy
truncates it and writes it back from memory, and SysRole.Update replaces a
role's policy rows wholesale. Both rebuild the same tuple from the same
sys_menu/sys_api data, so a match on the tuple survives what a row id does
not.

Only policies this install actually created are recorded - the insert is
conditional and its RowsAffected says which. A policy that was already there
was granted by somebody else and is not this app's to take away.

The two ways that can be wrong are not equally bad, which is what settles it.
Under-recording leaves a policy behind and the uninstall says so, because a
policy naming this app's own path with no ledger entry is exactly what it
reports as an orphan. Over-recording deletes somebody's authorization,
silently. Between a visible leftover and an invisible deletion, take the
leftover.

The ledger insert is itself conditional, for a case the obvious retry test
does not reach: on a plain re-run the policy still exists, so the insert is
skipped before the ledger is touched. It is reached when the policy row was
removed while its entry stayed, and a plain insert would then abort the whole
seed on the ledger's unique index. There is a test for that specific shape,
and replacing the insert with a plain one turns it red - which the plain retry
test does not.

Ordering: the ledger table is created by a framework migration, and version
strings sort bare digits ahead of any app-prefixed one, so it exists before
any application's seed runs. Nothing in the framework's own migrations calls
SeedMenus.
2026-09-09 12:27:38 +08:00
zhangwenjian 35d213f339 feat✨: install one application from its manifest
`migrate install <code>` brings one application up to the version its manifest
declares: it runs that application's outstanding migrations and records what
it did in sys_app.

It goes under migrate rather than under the existing `app` command, which
already means "generate the skeleton of a new application" - a directory that
does not exist yet, not an application already compiled into this binary.
Installing one is running its migrations, which is what this command is, so
--domain, resolveDB and the guard that refuses a mistyped code instead of
reporting a successful no-op are all already here.

Three phases, each committing on its own, and they are not one transaction.
An application's versions are separate migration files, and on MySQL a DDL
statement commits the transaction around it - destroying an outer transaction
and every savepoint taken from it. So this does not promise that a
half-installed application cannot happen. It promises one is visible when it
does: phase A writes "installing" before anything that can fail, phase B runs
the migrations, phase C turns that into "installed" or into "failed" with the
version it stopped on.

What is left to apply comes from sys_migration, never from sys_app. sys_app
is a derived view - a summary, and the answer to "which version does this app
think it is at". If it were the authority, an operator who deleted
sys_migration rows by hand would be told an application is installed while its
schema is not, which is worse than not knowing. So "already installed, nothing
to do" needs all three: nothing outstanding, recorded as installed, and the
same version. A row stuck at "installing" - what it reads as after the process
was killed partway - is not installed, and retrying is just running the
command again.

An upgrade is in place and keeps the first install's time; a downgrade is
refused, and refused before phase A writes anything, so a refusal cannot cost
the operator the row that told them what they had. An unparseable recorded
version is refused the same way, while it is still readable.

The report ends by saying the code is not running yet. That is not a
pleasantry: Go links at build time and Vite resolves its import globs at build
time, so installing an application writes its menus, its APIs and its
permissions and cannot make one line of its code run - and the menus appearing
is exactly what makes an operator believe otherwise.

Ten degradations were applied one at a time to check the tests name the
behaviour rather than the shape: deciding the no-op from sys_app alone,
always writing installed_at, allowing the downgrade, keeping the previous
attempt's diagnostics on a row that now says installed, treating "installing"
as installed, truncating last_error by bytes so a Chinese message is cut
mid-rune, not recording the failure at all, skipping code normalization, and
writing phase A before either the downgrade or the version-parse refusal.
Each went red on the assertion it was aimed at. An eleventh was discarded
rather than counted: it failed in the first install's setup, not on the claim.
2026-09-09 12:22:16 +08:00
zhangwenjian 9c68bc25a5 refactor♻️: report a failed migration instead of ending the process
run() called log.Fatalf on the first migration that failed, which ended the
process from inside the migration engine. Nothing above it could record what
happened - an installer needs to write down which version an attempt stopped
on - and no test could exercise a failing migration at all without taking the
test binary with it, which is why the one test that covers a failed migration
drove the registered function directly and left the scheduler uncovered.

run(), Migrate() and MigrateApp() now return an error, and the exit moved to
the command layer where the exit code is the command's business.

Two of those errors say more than "it failed". A migration that fails comes
back as a *VersionFailure naming the version, because an installer records
that as a diagnostic snapshot - the authoritative answer to where a retry
resumes is always recomputed from sys_migration, never read back, and asking
the database what is still pending answers a different question that merely
has the same answer most of the time. An app code nothing registered under is
now an error rather than a log line, so an installer asking for one app by
name cannot be told that installing an app that does not exist succeeded; the
command layer still rejects a typo before any database work.

exitOnError is what makes the command exit non-zero, and it covers more than
it replaces. Every path out of migrateModel used to return without an exit
code: an unreachable tenant database or a failed AutoMigrate printed a line
and exited 0, so a caller that migrates before starting a server - the deploy
workflow does exactly that - carried on onto a schema that had not been
brought forward. A failing migration function was the only failure reported,
and only as a side effect of the log.Fatalf this commit removes.

Each of these was checked by degrading it and watching the named assertion
go red: returning nil instead of the failure, naming the first version rather
than the one that failed, accepting an unregistered app code, and not exiting.

One gap is left open deliberately. Go allows a call whose only result is an
error to stand as a statement, so `migration.Migrate.Migrate()` still compiles
while dropping what it returns - `go build` passed while migrateModel was
doing exactly that during this change. Both call sites now return the value,
which the compiler does check, but nothing guards against the statement form
coming back. A checksilent rule was considered and dropped: that tool parses
without type information, so it could only match the method name, and a guard
that fires on any type with a Migrate method is noise.
2026-09-09 12:14:12 +08:00
wenjianzhang 9c5d9d16a7 Merge pull request #925 from go-admin-team/fix/deploy-image-bloat
Stop the demo host filling up with old images
2026-09-09 10:54:08 +08:00
zhangwenjian 46c10f999a ci👷: reclaim before the pull, not only after a healthy deploy
Cleaning up after a successful deployment never runs on the host that needs it.
The pull is the first thing in this script that needs space and it is where a
full disk stops it, so the run ends before reaching any cleanup - and so does
the next run, and the one after that. That is not hypothetical: a deployment
failed on the pull with no space left on the device, and rerunning the workflow
unchanged failed at the same place. The disk had to be cleared by hand before a
deployment could go through.

The pipeline is now a function called twice, before the pull and after the
health check, so the window is bounded on both sides.

Verified against a real docker daemon, in the function form rather than the
inlined one: with five images newer than the running one, so position alone no
longer protects it, it leaves three and does not select the live one; removing
the id exclusion from the same function does select it. With NAME pointing at a
container that does not exist it selects nothing - as it also does without the
explicit guard, which is there because grep -v on an empty id reads like the
opposite of what it does, not because it changes the outcome.
2026-09-09 08:11:26 +08:00
zhangwenjian d12f40c9a0 ci👷: remove this repository's old images after a healthy deploy
Every deployment pulls an image tagged with its commit and nothing removed the
previous one, so they only accumulated. 68 had built up when a deployment failed
on a pull with no space left on the device. That is the harmless place to fail -
the site kept serving the image it already had - but no later run would have
recovered on its own.

Three are kept so a release can be re-run by tag by hand. Only this repository's
images are listed, because the host runs other services. The image the new
container is on is excluded by id rather than by position, and rmi is called
without -f so an image a container still holds is refused rather than taken from
it.

Verified against a real docker daemon: with five images newer than the running
one, so position alone no longer protects it, the pipeline leaves three and does
not select the live one. Removing the id exclusion from the same pipeline does
select it, so that guard is load-bearing rather than decorative.
2026-09-09 07:45:51 +08:00
zhangwenjian cd363fce3d perf👌: stop shipping a C toolchain in the runtime image
gcc and g++ were 273MB of a 381MB image, and nothing in the container ever
invoked them: the binary is compiled and statically linked before the image is
built and arrives as a COPY, and Go is not installed here either.

The layer cost more than its size. apk resolves against an index that moves, so
its digest differed on every build and no two images shared it - a host that
keeps one image per deployed commit paid the full 273MB each time rather than
storing it once.

libc6-compat is kept although nothing measured needs it: a container built
without it resolves a hostname and opens a database connection exactly as one
built with it, but it costs half a megabyte and covers a ./main that was linked
dynamically, which this Dockerfile cannot check.

Verified by building this Dockerfile and running the result under the check the
deploy script uses - captcha answering 200 and the log reporting the datastore
connected. A control built from the current recipe passes the same check and
carries a 273MB apk layer this one does not; a third build with a deliberately
truncated binary fails the check, so it distinguishes a serving process from a
dead one.
2026-09-09 07:45:40 +08:00
Jack Walker 709cebd4a7 fix🐛: return an error when stopping a job times out
Fixes #890
2026-09-08 19:26:49 -04:00
wenjianzhang ba5ef9f79c Merge pull request #923 from go-admin-team/feat/008-host-schema
008: the application registry, its natural keys, and an idempotent seed
2026-09-08 20:35:28 +08:00
zhangwenjian 2c50317a98 fix🐛: stop the duplicate check refusing what the index would accept
The check grouped by (app_code, path, action) and refused whatever
appeared more than once. GROUP BY treats two NULLs as the same value; a
unique index treats them as different ones and allows both. So a
database holding rows with a null path or action was refused for
duplicates the index it was blocking would have accepted - and the
migration stopped, on a database with nothing wrong with it.

Both columns are nullable: neither carries a not-null tag, so gorm built
them that way. Measured on MySQL 8.0, PostgreSQL 15 and SQLite: two rows
with both columns null are one group to GROUP BY, and the unique index
builds over them without complaint. Standard SQL, not a dialect quirk.

They are now excluded from the check rather than grouped. Each column
needs its own exclusion and has its own test: one null column is enough
to make the index accept the pair, so removing either condition alone
lets that half through - which is what the two subtests are for, and
each fails only for its own half.

The message could not name the rows either. MySQL's CONCAT returns NULL
when any argument is, and scanning that into a string fails with
"converting NULL to string is unsupported" - so the check reported a
driver error instead of the duplicates it exists to report. SQLite and
PostgreSQL treat a null argument as empty and say nothing, which is why
this never surfaced in the tests: they run on SQLite, and this
repository has no MySQL in CI. The comment says so, so that a
postgres-only test file is not mistaken for cover.

No COALESCE was added to paper over that. It would have had nothing left
to guard once the nulls are excluded, and it would make a future
regression quieter: someone dropping the exclusions would get a report
naming rows that are not duplicates, which reads as a real answer,
rather than a scan error that reads as a broken query.

Leaving path and action nullable is deliberate. Tightening them is a
migration of its own - existing null rows have to be given values, and
what those values should be belongs to whoever owns the data, not to a
migration whose job is adding an index.
2026-09-08 20:17:50 +08:00
zhangwenjian 28350a15bb fix🐛: repair the row a retry reuses instead of walking past it
The idempotency check added in the previous commit stopped a retry
inserting a second copy, and introduced a quieter failure in its place:
a retry that found an existing sys_menu row skipped everything after the
insert. Those are the sys_menu_api_rule bindings and the materialized
path, and neither is written by the statement that writes the menu -
paths is a separate UPDATE, and on MySQL an earlier DDL has already
committed the transaction that was supposed to hold them together.

So an install interrupted between those steps left a menu that exists,
sits outside the tree with an empty path, and is bound to no API. What
core's contract.md says about such a menu is that it is invisible to
every role and its apis are authorized for no one - while the installer
reports success.

Reusing now repairs. paths is compared before it is written, so a row
that is already right is not touched. Bindings are inserted with WHERE
NOT EXISTS rather than deleted and rebuilt: an administrator can bind an
api to a menu from the menu screen, and delete-then-rebuild would take
that with it on the next retry - the same accident as sys_role.go's
Association.Delete, pointing the other way.

Confirmed as a defect before it was fixed, by building the half-written
state and watching the assertions fail:

    dir.Paths = "", want "/0/1"
    binding count for list = 0, want 1

Four paths through the repair, each with a degradation that reds its own
test and leaves the others green: missing bindings only, missing paths
only, both, and neither. The fourth asserts no UPDATE is issued for a
row already correct.

A fifth covers what the repair must not do. Rebuilding bindings instead
of inserting them leaves every other test green while silently deleting
a binding this code did not create; that one now fails with "a retry
silently deleted a binding it does not own".

Bindings an older version of a manifest created and a newer one no
longer lists are left alone. Removing them is a delete, and a delete
needs the same certainty about ownership that uninstall does - this
function cannot tell a stale binding from one somebody added by hand.
2026-09-08 19:51:22 +08:00
zhangwenjian c6d3ea5f81 fix🐛: stop a retried seed from inserting a second copy
seedApis and seedMenuTree were bare tx.Create calls. A migration that
failed partway and was run again re-inserted everything it had already
written - which is not hypothetical: the demo site collected eighteen
duplicate sys_menu rows this way, and three duplicate menus were visible
in its sidebar.

Both now look for a live row already holding the natural key and reuse
it. Only live rows count: a row an earlier soft-delete retired does not
stand in the way of a fresh insert under the same key, which is also
what the unique indexes allow.

The app_code half of each key has a test of its own. Without it the
lookups still passed every existing test while quietly letting one
application adopt another's rows - and an uninstall would then delete
rows the other application believed were its own, on both sides without
an error. Removing app_code from either lookup now fails with
"has 1 row(s) ... want 2 - one per app".
2026-09-08 19:26:42 +08:00
zhangwenjian 7fadb4b585 feat✨: give the seeded rows a natural key to be found by
Seeding needs something to look for before it inserts, or a retry writes
a second copy of everything it already wrote. sys_api already had one in
(app_code, path, action). sys_menu had nothing usable: menu_name is
pascalCase(appCode) + pascalCase(code), which is not injective -
"list-all", "listAll" and "list_all" all become "ListAll" - so the
original code cannot be recovered from it. Hence a new column.

seed_code is nullable, against this repository's habit of NOT NULL
DEFAULT '' for a new column, and deliberately. Every row that predates
it has no meaningful value, and under a unique index an empty string
collides with every other empty string while NULL collides with nothing.
The convention exists because deleted_at's nullability broke a unique
index; here nullability is what makes one possible.

The unique index on sys_api cannot simply be created: a live database is
known to hold historical duplicates - the demo site had eighteen. The
migration looks first and refuses while naming the offending rows,
rather than letting CREATE UNIQUE INDEX fail with a constraint error
that names none. Same shape as 1786700003000's refuseOnDuplicates.

Run against SQLite, MySQL 8.0 and PostgreSQL 15, including the CONCAT
duplicate check, which had only ever been executed by SQLite's driver.
2026-09-08 19:26:42 +08:00
zhangwenjian 691df82016 feat✨: add the sys_app registry and the casbin grant ledger
sys_app is one row per installed application. It is physically deleted
on uninstall rather than following the millisecond soft-delete marker
the other sys_ tables use: an installed-app registry has no "deleted by
accident, needs recovering" case, and a physical delete is what lets the
same code be installed again afterwards.

status is installing/installed/failed rather than a boolean, because an
install spanning several migration files is not atomic on MySQL - DDL
commits implicitly, so a run can stop in the middle. failed_version and
last_error are diagnostic snapshots for a person to read; nothing may
decide anything from them, and the field comments say so. Where to
resume is answered by sys_migration, which cannot drift from what was
actually applied.

sys_app_casbin_grant records which casbin_rule rows an install created,
keyed by casbin_rule's own natural key. That table is not extended
instead: gorm-adapter's SavePolicy truncates and reloads it from an
in-memory model, which would drop any column added here without a word.

Built against SQLite, MySQL 8.0 and PostgreSQL 15.
2026-09-08 19:26:40 +08:00
zhangwenjian ea348fa9d1 build📦: pin go-admin-core v2.8.0
v2.8.0 adds sdk/contract/app - an application's manifest, and the one
comparator for its version - which the installer in this batch is built
on. Nothing here uses it yet; this is the dependency arriving.

Checked that the release is consumable rather than only tagged: a
program built against the published module registers a manifest, reads
it back, and gets -1 from Compare("1.9.0", "1.10.0"), which is the
multi-digit case a string comparison would order backwards.

25 packages pass and checksilent is clean on the new version.
2026-09-08 18:35:13 +08:00
wenjianzhang 925c6772a6 Merge pull request #921 from go-admin-team/fix/920-schema-readiness
Fail readiness while the database is behind the migrations
2026-09-08 17:18:34 +08:00
wenjianzhang 0c60e44aee Merge pull request #922 from go-admin-team/fix/919-drop-index-postgres
Drop the index with SQL this dialect can parse
2026-09-08 17:18:08 +08:00
zhangwenjian a716086295 test✅: fail these tests when the database does not answer
Every Create and Raw dropped its error. Most of them would have failed
an assertion further down anyway, with a message describing the wrong
problem - but one of them would not.

The last count in the index test reads how many indexes survived and
expects zero. An unchecked query that fails leaves the variable at zero,
and zero is what success looks like: a test that cannot reach the
database reports that the indexes were dropped.

Demonstrated rather than assumed, by breaking that one query both ways:

    with the check:    FAIL  counting the indexes after: relation
                             "pg_indexes_nope" does not exist
    without it:        PASS

Raised by Copilot on #922, as a consistency point with the SQLite tests
in this package. It is that as well, but the reason it is worth doing is
the row above.
2026-09-08 17:04:31 +08:00
zhangwenjian f8a5066a40 test✅: run the migration against PostgreSQL in CI
The rest of this package's tests run on SQLite, where dropping an index
through the migrator works. That is why a migration which failed on
every PostgreSQL database it was pointed at had a green suite: the
defect cannot occur on the backend being tested.

A test alone would not have helped either - without a service it skips,
and a test that never runs is the same as no test. So the workflow gains
a postgres service and the DSN, and the helper refuses to skip when CI
is set: a workflow that drops the service or renames the variable fails
rather than going quietly green, which is the shape of the original
defect.

Counter-proved by putting Migrator().DropIndex back, which reproduces
the statement verbatim:

    DROP INDEX CURRENT_SCHEMA()."idx_sd_pg_user_deleted_at"
    ERROR: syntax error at or near "CURRENT_SCHEMA" (SQLSTATE 42601)

The conversion test also checks the timestamp survives as a marker,
since a conversion that dropped it would bring deleted rows back live
while still passing a column-type assertion.
2026-09-08 16:57:17 +08:00
zhangwenjian f978967ef1 fix🐛: drop the index with SQL this dialect can parse
The soft-delete conversion dropped the indexes on deleted_at through
gorm's Migrator().DropIndex. Its PostgreSQL driver resolves a schema for
the statement and falls back to an expression when it cannot:

    currentSchema, _ := m.CurrentSchema(stmt, stmt.Table)
    m.DB.Exec("DROP INDEX ?.?", currentSchema, clause.Column{Name: name})

DROP INDEX takes an identifier in that position, so what reached the
server was

    DROP INDEX CURRENT_SCHEMA()."idx_sys_api_deleted_at"
    ERROR: syntax error at or near "CURRENT_SCHEMA" (SQLSTATE 42601)

The schema is unresolvable for every call this migration makes, because
it passes a table name as a string rather than a model. So it failed on
every PostgreSQL database rather than intermittently, and stopped the
whole conversion at the first table.

What that looked like from outside is go-admin#919: an upgrade that
could not complete, and a login rejecting a correct password, because
deleted_at was still a timestamptz while the current query compares it
to 0. Neither symptom names a migration.

Written per dialect, for the same reason addBigIntColumn and
renameColumn already are. MySQL and SQL Server name the table and have
no IF EXISTS for it; PostgreSQL and SQLite name the index alone.

Verified by running the shipped migration against PostgreSQL 15 and
MySQL 8.0 in containers, and SQLite through this package's tests. The
SQL Server form is from its documentation and has not been run - there
is no SQL Server here to run it against, and saying so is better than
implying four dialects were checked.
2026-09-08 16:57:15 +08:00
zhangwenjian d6e2c02fda fix🐛: fail readiness while the database is behind the migrations
The process started, both probes passed, and the first sign that the
schema did not match was a login failing with a driver-level encoding
error - go-admin#919, where an operator upgraded the binary and
restarted the API without running migrate. Nothing between those two
events had an opinion about the schema.

Readiness is where this belongs. Liveness asks "restart me", and a
process whose database is on the wrong schema comes back to the same
schema. Readiness asks "send me requests", and the answer is no. A
rolling update then stalls at the deploy - new instances never become
ready, the old ones keep serving - rather than at somebody's login, and
running migrate clears it without a restart because the check is
evaluated per request.

Any tenant database being behind fails the check, not only the one being
served: migrations are applied to every database in one run, so one
behind means that run did not finish, and serving the rest would let a
half-applied deploy look like a partial success.

A missing sys_migration table is nothing applied rather than an error.
That is a first deploy, where every migration is pending and the
operator can act on being told so.

The one test that matters is the one that cannot be written normally.
The registry is filled by init() in packages cmd/api does not import, so
a test that imported them to look at it would pass whatever the real
binary links - and a binary that links none of them gives a check that
reports every database current, forever, with every other test here
still green. TestTheServingBinaryLinksTheMigrationRegistry asks the
build instead, with a negative control so that a query matching
everything fails rather than passes.

Closes #920.
2026-09-08 15:42:14 +08:00
zhangwenjian e98b65cf90 feat✨: let the host add a readiness check this package cannot make
Whether the schema matches what the binary expects is answered by the
migration registry, which lives under cmd/. common/ has never imported
cmd/, and starting with this would put the shared layer behind the
command layer for one check.

Register instead, from where both are already in scope. A duplicate name
panics rather than appending: two checks under one name make the failing
one impossible to identify from the response body, and registering the
same one twice is a wiring mistake better heard at start-up than never.

The registered check is run through the same guard as the built-in ones,
so one that panics fails its check rather than taking down the probe
that asked.
2026-09-08 15:42:14 +08:00
zhangwenjian bc5411c30c feat✨: report what the migration registry holds without a database
Status answers what is registered, what is applied, and what is applied
while nothing registers it - and needs a database to do it. A readiness
check needs only the first half, and it already holds the databases it
is asking about.

Without this it would have to call SetDb to reuse Status, writing this
package's shared state from a request path, for a question that does not
depend on any database at all.
2026-09-08 15:42:11 +08:00
wenjianzhang 27f23121f0 Merge pull request #918 from go-admin-team/fix/911-queue-shutdown
Drain the queue on the way out instead of abandoning it
2026-09-07 17:39:15 +08:00
zhangwenjian 047b23872c fix🐛: let a drain that finishes on the deadline count as finished
The wait was one select over done and ctx.Done(). Both can be ready when
it runs, select picks at random among ready cases, and so a queue that
drained in the same instant the budget expired was reported as an
overrun about half the times it landed there - often enough to be read
as noise, and pointing at the wrong thing when it was not. core's own
RunShutdown re-checks for this reason; this did not.

The tie-break is now a function taking channels rather than a queue,
which is what lets a test hand it a closed done and an expired ctx
together. That state is the whole of the bug and cannot be arrived at
reliably from the outside; over 1000 iterations the single-select
version fails, and the second look does not.

The test for giving up on the deadline read the call counter straight
after shutdownQueue returned, while Shutdown runs on a goroutine nobody
joins. It passed because the goroutine is scheduled promptly, not
because anything ordered the two. The fake now signals that Shutdown has
been entered and the test waits for it.

Both raised by Copilot on #918.

The first attempt at the tie-break test was wrong and is not what
landed: it asserted that an immediately-returning Shutdown always counts
as drained under an already-expired context, which is not true and
should not be - if the goroutine has not run, nothing has drained. That
test failed, correctly. What is being claimed is narrower: when both are
ready, done wins.
2026-09-07 17:21:03 +08:00
zhangwenjian 84bd87dcc9 fix🐛: drain the queue on the way out instead of abandoning it
Nothing stopped the queue when the process exited. core v2.7.0 made the
drain work - Memory.Shutdown closes the queue and waits for every
consumer to finish what it holds, and the legacy adapter cancels its
context and closes the underlying queue - but no call site ever reached
it. The only Shutdown() in this repository applies to the previous
adapter during a reload, so the installed one was simply left. The login
log, the operation log and the API sync all publish through it, so a
rolling restart dropped whatever had not been consumed, on the path
where the process exits 0 and reports "Server exiting".

Setup now registers a BeforeExit callback that shuts down the adapter
this package installed.

Three things it has to get right, each with a test.

It reads `installed` when it runs, not when it registers. A reload
replaces the adapter, and the one from start-up is a queue nobody has
published to since.

It never goes through sdk.Runtime.GetQueueAdapter. That accessor never
returns nil - with no queue section configured it wraps the runtime's
own fallback - so it would look like it worked while closing a queue
this package neither built nor started. That is the same trap setupQueue
already had to drop an `if q != nil` for.

It registers once. Setup is re-run on every configuration change, and a
callback per reload would leave the shutdown phase holding a row of
identical entries, each eligible to be named as the one that overran the
budget.

That last one needed a seam. shutdownQueue takes the adapter on its
first run, so the second and third callbacks find nothing and return -
three registrations produce exactly the same observable result as one,
and a test going through the effect passes either way. It did: the
counter-proof for "register on every reload" came back green until the
registration was counted at the seam instead.

The wait is bounded here rather than left to the phase. Shutdown takes
no context, so a consumer that never finishes would hold the process
until SIGKILL; the callback gives up and says what is being lost, which
the phase's generic overrun message cannot.

Ordering falls out of the phase rather than being arranged: callbacks
run in reverse registration order, this one registers during setup and
the job scheduler's registers on AfterListen, so the schedulers stop
before the queue drains. Verified against core v2.7.0 rather than read
off the source.

Closes #911.
2026-09-07 17:10:13 +08:00
wenjianzhang 0e7a13aeba Merge pull request #917 from go-admin-team/fix/914-gen-write-guard
Register the generator's writing endpoints only in a development mode
2026-09-07 15:32:13 +08:00
zhangwenjian 85d50da494 fix🐛: tell the start-up warning's reader to restart, not only to reconfigure
The warning said to set application.mode and stopped there. Following
that on a running process does not close anything: buildRouter has one
call site, in run(), and route registration is on no phase and no reload
callback, so a configuration reload moves the mode and leaves the routes
exactly where they were.

The reader is then worse off than before they acted. The mode now says
prod, GenWriteRoutesEnabled agrees, and the endpoints are still served -
so the one thing they could check to confirm the fix reports success
while the exposure is untouched, until something restarts the process.

A test pins the gap rather than the prose: build under dev, move the
mode to prod, and the routes are still in the engine. It fails if
registration ever becomes dynamic, which is the change that would make
the new sentence wrong.

That test degrades differently from the others - making it fail means
rewriting registration, not weakening it - so what was checked instead
is that it cannot go vacuous. Both of its premises are guarded: with the
gate always refusing it reports building under dev without the writing
routes, and with the gate always allowing it reports the predicate still
allowing prod. Neither failure can be mistaken for the assertion passing.

Raised by Copilot on #917.
2026-09-07 15:20:36 +08:00
zhangwenjian 1d9def4314 test✅: restore the mode before the route helper returns
registeredRoutes set config.ApplicationConfig.Mode and gave it back with
t.Cleanup, which runs at the end of the test rather than at the end of
the helper. Everything the caller did after the call therefore ran under
the mode the helper had been asked about, not one the caller chose.

Nothing was wrong yet: the one caller that reads the mode afterwards
sets it itself, and the two cleanups happen to unwind in an order that
leaves the right value. Both of those are accidents, and neither is
visible at the call site.

A defer inside the helper makes the borrowing end where it starts. The
doc comment said the mode was put back before returning while the code
did not, so that is now true rather than aspirational.

The counter-proof is the reason this has a test of its own: with
t.Cleanup back in place TestRegisteredRoutesRestoresTheModeBeforeReturning
fails and nothing else does, which is what a leak this quiet looks like
when something is actually watching for it.

Raised by Copilot on #917.
2026-09-07 15:14:39 +08:00
zhangwenjian ed9bbd01e2 feat✨: warn at start-up when the generator can write to this host
The gate in the previous commit is decided by application.mode, and the
shipped configuration says dev. So the deployment most likely to be
serving the writing endpoints is the one that changed nothing, and that
is also the one least likely to go looking for them. A gate whose
default is open needs to say so.

Nothing is said in demo mode. The routes are registered there, but
DemoEvn refuses all three by name, so a warning would describe an
exposure that is not present.

The decision is split from the logging so it can be tested. Three
counter-proofs: warning in demo as well fails mode=demo; a warning that
never fires fails mode=dev, which is what shows the line can be reached
at all; and one that always fires fails every mode but dev.
2026-09-07 15:07:07 +08:00
zhangwenjian 523d6a3649 fix🐛: register the generator's writing endpoints only in a development mode
Three of the code generator's endpoints do not read. /gen/toproject
writes seven Go and Vue source files onto the host, one of them under
the path gen.frontpath names; /gen/apitofile writes a migration;
/gen/todb inserts menus and APIs. All three are GET, and all three are
listed in CasbinExclude - which AuthCheckRole skips - so Enforce never
runs for them. Any account that can log in could call them, on every
deployment.

They are now registered only where application.mode is dev or demo. dev
is the shipped default and is where the generator is meant to be used.
demo keeps them because demo mode already has a better answer than a
404: DemoEvn refuses these three by name and explains itself, which is
what the demo is for. test and prod get nothing, and so does a process
whose mode was never set.

This does not make the endpoints safe where they exist; it stops them
existing where nobody should be calling them. A host left on the shipped
dev is still open, which is why the next commit says so at start-up.

CasbinExclude is left alone on purpose. Taking the three off that list
would make them require a permission no existing deployment has granted,
so every non-admin user would start getting 403 from a tool that worked
yesterday. That is a migration, not a guard, and it belongs with a
release that can carry one.

Four counter-proofs, each red on the test that names the behaviour and
green everywhere else: a gate that always allows fails test/prod/unset
only; a gate that always refuses fails dev/demo and takes
TestEveryRouteDemoModeRefusesStillExists with it; moving a read-only
route inside the gate fails the reading test; and spelling the condition
at the registration site instead of calling the predicate fails the
agreement test, which is what keeps that test from being a tautology.
2026-09-07 15:05:24 +08:00
zhangwenjian 6326962862 style🎨: gofmt gen_router.go
Two pre-existing deviations: a space before the comma in
sysNoCheckRoleRouter's parameter list, and no newline at end of file.
Separated from the change that follows so its diff is only the change.
2026-09-07 15:02:33 +08:00
wenjianzhang cd7c8375c0 Merge pull request #916 from go-admin-team/docs/replicas-constraint
Name the job scheduler as the other reason for one replica, and stop the manifests redeploying the demo
2026-09-07 14:52:53 +08:00
zhangwenjian 63bcc912ef ci👷: stop the k8s manifests from redeploying the demo
The comment at the top of this workflow says documentation-only changes
skip it, because a push to master pushes an image, runs the migrations
and restarts the demo container. The ignore list did not cover
scripts/k8s, so editing a manifest that the deploy never reads bought
the site an outage.

The pattern is scripts/k8s/** rather than scripts/** because
scripts/Dockerfile is a build input - go.yml builds the release image
from it on a tag.

This workflow file stays outside the list on purpose. paths-ignore skips
only when every changed path matches, so a change that edits the deploy
still runs it, which is the point.
2026-09-07 14:14:39 +08:00
zhangwenjian adcdd2edcd docs📝: name the job scheduler as the other reason for one replica
The comment on replicas gave one obstacle to raising it, the shared log
volume, which reads as the only one. Someone who moves the log path off
that volume would conclude the way is clear.

The scheduler in app/jobs is the second, and it is the one that does not
announce itself. Its handle on a job lives in sys_job.entry_id, one
column shared by every process, and startup zeroes the whole column
before writing its own ids. A second pod therefore erases the first
pod's, and both pods run the full enabled list. Stopping a job from the
UI then removes an entry from whichever process is asked, by an id that
belongs to another one, and answers 200.

See #915.
2026-09-07 13:32:43 +08:00
wenjianzhang 29406f839e Merge pull request #913 from go-admin-team/fix/demo-mode-guard
fix🐛: demo 模式放行了注册成 GET 的写接口
2026-09-07 08:11:14 +08:00
zhangwenjian a43133ab7b fix🐛: stop demo mode serving the writes that are registered as GET
DemoEvn decided by HTTP method: GET and OPTIONS through, everything else
refused. Three of the code generator's routes are registered as GET and write
anyway - two emit Go source files onto the server's filesystem, and the third
inserts menus, APIs and casbin rules into the database. They sit in a group
whose own name says it does no role check, and a demo deployment lets anybody
log in. So on the demo host any visitor could write to the machine and to the
database, and the one that writes menus had in fact been used: three generated
SysCasbinRule entries is how this was noticed.

The guard now also looks at the matched route. The method cannot answer the
question - whether a request changes anything is not something the verb reports
truthfully here - so the three are named, as gin route patterns, which is what
Context.FullPath returns and how CasbinExclude already spells them.

Changing them to POST would be the better shape and is not this change. sys_api
records an endpoint by method and path and the casbin policy follows it, so
flipping the verb needs a migration and a policy resync; until both land, every
existing deployment would start answering 403 to a role that could use the
generator the day before.

The read-only half stays reachable: preview, the table tree, and the two
database listings. A demo host that cannot demonstrate the generator is as
broken as one that lets visitors write to it - refusing too much is the same
defect facing the other way, and there is a test for that direction too.

Half of the general hole is closed and the other half is written down. The
closed half is a test beside the route registrations: it builds the generator's
routes, enumerates them, and fails if any entry in the guard has stopped being
a real route, so renaming one turns the list red instead of quietly making it
match nothing. It lives there because common/ may not import app/ - which is
also why the guard cannot check its own list from where it is. The open half is
that no static check can tell a handler that writes from one that reads, so the
next GET that writes has to be added by hand. The comment says that rather than
leaving the impression the class is covered.

application.demomsg was configuration nothing read. The message was hard-coded
in the middleware, and the demo host's configured string happened to be
identical, so the setting looked like it worked and never had. It is read now,
with the old string kept verbatim as the fallback, so a deployment that never
set it is answered exactly as before.

This covers demo mode only. On a deployment that is not a demo those three
routes remain in CasbinExclude and stay reachable by any authenticated user
whatever their role; that is a separate decision and is not touched here.
2026-09-07 08:03:40 +08:00
wenjianzhang 7002cd4065 Merge pull request #912 from go-admin-team/feat/007-drain-window
feat✨: 关闭时先排空再停止接收——让 /ready 的 503 真的能被采到
2026-09-07 08:01:59 +08:00
zhangwenjian 8faa8d2aed feat✨: check the shutdown budget against every stop deadline
The budget is one number in config/settings.yml. The deadlines that have to
cover it are in four other files, none of which anybody edits while thinking
about shutdown - so raising the budget passes every test, deploys, and has the
cleanup callbacks killed on the next release.

Two checks share one arithmetic and one five-second margin.

shutdown-budget-overruns-grace compares preStop + drain + server + cleanup
against terminationGracePeriodSeconds in the shipped manifest. Those two files
are not merely adjacent examples: scripts/k8s/prerun.sh builds the
settings-admin ConfigMap out of config/settings.yml and the Deployment mounts
it, so the manifest deploys that file.

docker-stop-cuts-shutdown-short covers the three ways this container is
stopped: `docker stop` in the release workflow, the same in the Makefile, and
stop_grace_period on a compose service that runs this repository's own image. A
service running a database is not this process and is left alone. The duration
is parsed rather than scanned for digits - compose accepts 1m30s, and reading
the first number out of it would call ninety seconds one.

All three spellings of the deadline are read: --timeout, the deprecated --time,
and the short -t. A deadline the check cannot read is reported as no deadline at
all, so recognising only one of them would call a correct command broken and
send whoever fixed it towards the spelling docker is retiring. The message
quotes the flag back in the spelling it was written in, for the same reason:
suggesting a flag the line does not use is how a tool teaches people to
disbelieve it.

What neither covers is `docker rm -f`, which has no deadline to compare
against: it is SIGKILL by definition. That gap is deliberate, and it is why the
previous commit changed the one place that used it on a container that might
still be running.

Both report at two levels. A budget that already overruns is an ERROR; one that
fits with nothing to spare is a WARN, because it works today and failing the
build on a working configuration is how a project teaches people to ignore its
warnings. The two are exclusive: an overrun satisfies the headroom condition as
well, and an ERROR that always drags a duplicate WARN behind it teaches the same
lesson.

preStop is in the sum although the shipped manifest has no hook. That is the
point - a hook added later is spent before the process is told anything, and a
self-check that could not see it would understate the real budget by however
long somebody set it to, which is worse than not checking. A hook whose duration
cannot be read is reported rather than counted as zero.

The fallbacks for fields the settings file leaves out are read from the
constants in the scanned tree, not copied here; if they are renamed the run
stops instead of going quiet with the wrong numbers.

The wording differs by audience on purpose. At run time this is somebody else's
deployment under constraints the process cannot see, so the log states a
minimum. These checks read files this repository owns, where there is standing
to ask for headroom, so they name a target.

The table in AGENTS.md is relisted while it is being touched: the two new
checks, plus datascope-route-unguarded, which has been missing since it was
added. The hard-coded count is gone - it said seven and there were ten, which is
what a written-down count does. AGENTS.md and docs/contract.md both sent readers
to `go run ./tools/checksilent -h` for the list of checks; that prints
command-line flags and has never printed a check, so both now point at
runChecks.

The yaml parser moves from an indirect requirement to a direct one - it was
already in the module graph - and tidy drops four go.sum lines left over from
two older releases of core.
2026-09-06 22:07:08 +08:00
zhangwenjian 705427178d fix🐛: give every stop path enough time for the shutdown budget
Stopping this process takes drain + server + cleanup seconds: eight out of the
box, and more for anyone who configures a drain window. Three places decide
whether it gets that long, and none of them was written with it in mind.

The release workflow stopped the previous container with the default deadline,
which docker sets at ten seconds. The compose file - which the Makefile calls
the first way to run this - set no stop_grace_period, so it took the same ten.
Under either, a drain window over two seconds would have been cut off by
SIGKILL part-way through the cleanup callbacks: this project's own deployments
could not have run the capability it ships.

The third was worse. `make run` removed the previous container with `docker rm
-f`, and the force flag kills a running container outright - "uses SIGKILL", in
docker's own words - with no grace at all. Restarting locally cut every
shutdown short, so the drain window would never once have been reached on a
developer's machine. It now stops with a deadline and then removes, which
leaves what gets removed unchanged: on a container that has already stopped,
stop is a no-op.

So: --timeout 30 in the workflow, stop_grace_period: 30s on the compose
service, and stop --timeout 30 before the removal in the Makefile. --timeout
rather than --time, which docker still honours but has deprecated - it prints a
warning on every use, and a deploy log that always carries a warning is one
nobody reads.

The three remaining `docker rm -f` calls in the workflow are left alone. Two
remove containers that have already been stopped and one is the rollback path,
and nothing static can tell those apart from a container that is still running -
which is also why the check added next does not look at `rm -f` at all: a forced
removal has no deadline to compare against. What keeps that path honest is the
line above it, not a check.

Thirty will drift the first time somebody raises a budget. The next commit is
what notices, which is also why these comments name a check that does not exist
yet.
2026-09-06 22:06:35 +08:00
zhangwenjian 8f10d202e6 feat✨: give the shipped manifest probes and a stop grace period
The manifest in this repository had no probes at all. A pod was sent traffic as
soon as its container was running, whether or not the database it needs was
reachable, and it was stopped with whatever grace period Kubernetes defaults to
rather than one chosen against what this process actually spends shutting down.

It now mounts both probes, at the endpoint that answers each question:
readiness at /ready, which fails while a dependency is unreachable, and
liveness at /health, which is a bare 200 because restarting a process whose
database is down turns one outage into a crash loop. Both skip the rate
limiter, which is why that had to land first.

timeoutSeconds is 3, not the default 1. The handler allows its checks two
seconds, so at the default a database answering in 1.2s would be recorded as a
failed check while the handler was returning 200 - the probe would be failing on
the orchestrator's stopwatch, not on its own. The comment beside that constant
said the constraint was the polling period; the constraint is the per-check
timeout, and it is now written down correctly.

terminationGracePeriodSeconds is 30, against a shipped budget of 0 + 5 + 3.
Raising drain means raising this too, in the same commit; the check that
notices when somebody does not arrives two commits from here.

replicas stays at 1, and the comment says why that makes the drain window worth
nothing: there is nowhere to send the traffic this pod stops taking. Raising it
needs one more change than the number - the volume is shared by every replica
and the log path lives on it, so a second pod would append to the same rotating
file. The reason not to raise it is not the one the review assumed: the claim
was that the PVC is ReadWriteOnce, and it is not, it is ReadWriteMany on nfs-csi.

There is no preStop hook. How long one should sleep depends on how fast the
thing in front removes this instance, which the repository cannot know, and a
manifest carrying both a preStop sleep and a drain window is the double-counting
trap - the budget would be spent twice and the start-up line would report half
of it.
2026-09-06 21:49:17 +08:00
zhangwenjian 5648bd1dcf feat✨: state the shutdown budget at start-up
The three budgets are spent one after the other, so what has to fit inside the
orchestrator's grace period is their sum - and nothing said what that sum was.
Working it out meant reading a configuration file, remembering which fields
were absent, and knowing what each one falls back to.

Start-up now prints it: the three values and the total, taken from the resolved
budget rather than from the file. A field left out still costs its default, so
adding up what was written down understates the total by exactly the fields
nobody wrote - which is the arithmetic somebody doing this by hand gets wrong.

Whether the total fits is a separate question, and the framework cannot answer
it alone: `docker stop` allows ten seconds and Kubernetes thirty, three times
apart. A fixed threshold would have warned about the manifest this repository
is about to ship. So extend.shutdown.grace is optional, nothing reads it during
a shutdown, and when it is absent the line says so and prints both reference
values instead of judging.

When it is set and the budget does not fit, the warning names the shortfall:
how many more seconds are needed. A minimum, not a target - this is somebody
else's deployment under constraints this process cannot see, and asking them to
leave headroom on top is not this line's business. Equal does not fit either;
the grace period is when SIGKILL is sent, so a budget that ends exactly then
leaves the last callback no time to return.
2026-09-06 21:49:17 +08:00
zhangwenjian a442eadb96 feat✨: keep serving for a configurable window before the listener closes
/ready has failed from the moment shutdown begins since the readiness probe was
added, and the order it does that in is right: reversed, the state would be
reported after the connections were already cut. But order alone does not
produce a window. Nothing waited between the flip and Shutdown, so the two were
microseconds apart, and a poller on a multi-second interval never saw the 503 -
it saw a refused connection, which is the thing the probe was supposed to
avoid. Polling a container through a SIGTERM on the demo host recorded exactly
that: 200, then connection refused, and no 503 in between.

extend.shutdown.drain is that wait. The process keeps serving normally for it -
answering requests, not refusing them, because refusing them would move the
outage earlier rather than avoid it - and only then closes the listener.

It is zero by default, so nothing changes for a deployment that does not ask
for it. That is not timidity: the budgets are spent one after another, and a
non-zero default would push every existing shutdown closer to the orchestrator's
grace period, where being cut off part-way through the cleanup callbacks is
worse than never draining at all.

Keep-alive is switched off with the flip. The server keeps connections alive
until Shutdown sets shuttingDown() itself, so without this the pooled
connections a balancer holds would sit untouched for the whole window and be
cut at the end of it anyway - the cost of the window without its benefit. This
is the switch Shutdown flips, moved earlier by the window's length.

The signal disposition is restored after the window rather than on the first
signal. Before there was a window, the interval where a second signal killed
the process outright was only reachable while a cleanup callback hung; putting
a multi-second wait inside it would have made every ordinary shutdown
interruptible for the length of the drain. A second signal during the window is
taken by the channel and ends the window early instead - somebody sending
another kill wants this over with sooner - and the escape hatch comes back the
moment the window does.

What the window is worth depends on who removes this instance. A balancer that
polls /ready acts on the 503 and needs the window to cover its check interval
times its failure threshold; a Kubernetes Service withdraws the endpoint when
the Pod is deleted, concurrently with SIGTERM and regardless of what the probe
returns, and there the window covers the delay in that removal reaching every
node. The three comments that used to say a balancer "has a chance to" take the
instance out said it without either qualification, which is how a claim comes to
be repeated after a live test has refuted it.

The subprocess test polls the real probes on a connection it opens after the
signal - a reused one can be served after the listener is closed, which would
let this pass against a shutdown that had already broken it - and asserts on the
draining answer in the body, not on the status code. With no database the status
is 503 from start-up, so a status-code assertion would hold even with
BeginDraining deleted. Two window lengths, because one proves only that
something takes that long.
2026-09-06 21:49:17 +08:00
zhangwenjian f3b67e9abc fix🐛: keep the rate limiter away from the health probes
The limiter is installed on the engine and the probes are routes like any
other, so above the threshold they are answered with 429 too. Point a liveness
probe at one and the failure mode writes itself: traffic crosses the threshold,
the probe collects three 429s, the kubelet restarts the container, the capacity
that was already short gets shorter, and the instances that are left are pushed
further past the threshold. The limiter working exactly as designed is what
kills the pod.

It is the argument common/health already makes about restarting a process whose
database is unreachable, applied to load: turning one outage into a crash loop
is not an improvement on the outage.

Nothing points a liveness probe at these routes yet. The manifest that will is
two commits away, and this has to land first, because that manifest without
this change would be actively harmful.

The exemption wraps the middleware rather than teaching the limiter about these
paths. common/ may not import app/ - the contract check enforces it - so the
limiter cannot name routes that are registered over there. Wrapping it in the
command package, which imports both, is what keeps the boundary.

Naming those routes needs them exported, so the group prefix and the two paths
become constants and the router function becomes RegisterMonitorRouter. That
also gives a test something real to mount: a probe asserted against a
re-implementation of itself is a test of the copy.

The check that the middleware never runs is separate from the check that the
answer is not 429, because a probe can produce a 429 on its own. What has to be
true is that the request never reached the limiter.
2026-09-06 21:49:16 +08:00
zhangwenjian 4e51f56623 feat✨: make the shutdown budgets configurable
How long a shutdown may spend waiting for in-flight requests, and how long the
cleanup callbacks get after that, were compile-time constants. The two together
have to fit inside whatever grace period the orchestrator allows before it
sends SIGKILL, and that number is not the same everywhere - `docker stop`
allows ten seconds, Kubernetes thirty by default - so the one deployment shape
these constants suited was the one they were written for.

They now come from extend.shutdown, beside rateLimit. Not from application:
that section is a fixed struct in core, and the decoder discards keys it has no
field for without an error, so a budget written there would be accepted and
never applied. That is the failure this whole change is about, and putting the
configuration where it cannot be read would have reproduced it.

Both fields are pointers, following RateLimit.InboundQPS: nil means "not
configured" and takes the default, and a number that was written down is spent
literally, zero included. Without that separation `server: 0` - do not wait for
in-flight requests at all, which is a reasonable thing to ask when the grace
period is very short - could not be expressed, and the section would need a
paragraph explaining which zeros mean what.

A negative is refused rather than clamped. Correcting a value quietly is the
same failure in a different costume, and Budget returns the error instead of
ending the process so that the rule can be tested without a subprocess.

The defaults live in config as seconds and in cmd/api as durations, both from
the same constants, and a test asserts the two agree - a deployment that
configures nothing is entitled to one answer about what it spends, not two.

The last test loads the two settings files this repository ships through the
real loader and asserts the section arrives with the documented values. Nothing
weaker can tell "the key is read" from "the key is discarded": the struct
compiles either way.
2026-09-06 21:49:16 +08:00
zhangwenjian 7e4e17bbcf test✅: run the real shutdown sequence in the signal tests
The child process built a server, restored its own signal disposition and
called shutdownServer and runShutdownHooks itself, in an order it chose. It
never called anything run() calls. So the assertions were about a copy of the
sequence: move a step in the real one, or drop it, and every test here stays
green. The acceptance criteria these back are worth exactly as much as that.

The child now calls gracefulShutdown and asserts on what comes out of it. The
budget it spends is defaultBudget with one field shortened where a test needs a
deterministic timeout, which is also how the two waits stop being wired by
hand.

The stuck-shutdown case changes shape as a result. It used to sleep inside the
child, between the steps it had copied; there is no "between" to sleep in any
more, so it registers a BeforeExit callback that never returns and gives it a
budget long enough to hang on. That is where a shutdown actually hangs, and it
now runs through the same function - which means this test also pins where the
signal disposition is restored, rather than just asserting that the child dies.

It signals repeatedly rather than once. The marker is printed immediately
before gracefulShutdown is entered, so a single signal sent on seeing it can
still arrive before the disposition is restored, land in the buffered channel
and be dropped. Which signal does the killing is not the assertion; that one of
them can is.
2026-09-06 21:49:16 +08:00
zhangwenjian 799e892a68 refactor♻️: run the shutdown sequence from one function
The steps between the stop signal and the last log line were written inline in
run(), which left nothing for a test to call. The signal tests reproduce that
sequence instead: they build their own server, restore their own disposition,
and call shutdownServer and runShutdownHooks in an order of their own. So they
assert against a copy - reorder the real sequence, or drop a step from it, and
they stay green.

The sequence now lives in gracefulShutdown, and the waits it spends are a
budget rather than two constants read at the point of use. Nothing changes
about what happens or in what order: the same disposition is restored first,
the same two announcements are made, the same waits are spent, and run() logs
the same two errors with the same messages.

Returning those errors instead of logging them inside is what lets a caller
other than run() react to them. That matters for the next commit, where the
tests stop reproducing this sequence and start running it.
2026-09-06 21:49:16 +08:00
wenjianzhang 0e2adb3165 Merge pull request #909 from go-admin-team/fix/queue-swap-order
fix🐛: 热重载期间生产者被指向已关闭的队列
2026-09-06 21:23:01 +08:00
wenjianzhang 249e044ded Merge pull request #910 from go-admin-team/docs/readiness-claim
docs📝: 摘除实例的不是 readiness——订正 #908 的三处注释
2026-09-06 20:32:51 +08:00
zhangwenjian d6309c75be docs📝: state what the draining answer is worth
Three comments said readiness failing before the server stops accepting gives
a load balancer the chance to withdraw the instance before its connections are
cut. Nothing between the two lines makes that possible: BeginDraining is
immediately followed by the shutdown, and a poller on a multi-second interval
never observes the flip.

On Kubernetes the endpoint is withdrawn when the Pod receives a
deletionTimestamp, concurrently with SIGTERM and independent of what the probe
returns, so the probe result is not the mechanism there either.

The order itself stands - reporting the state after the connections are cut is
worse - so the comments now say the order is necessary and not sufficient, and
that a window needs a configured delay that does not exist yet.
2026-09-06 19:04:42 +08:00
zhangwenjian 211ae85a4e test✅: fail on an Append error this test does not model
Only ErrQueueClosed was examined and every other error was discarded, so a
run where nothing reached a queue at all would leave the refusal count at
zero and the test green. The first unexpected error is now kept and fails
the test.

The pool is sized so a full queue cannot be one of them: it returns an error
of its own and is expected while the consumer is held, which would otherwise
make the new check fire on the normal path.
2026-09-06 19:00:30 +08:00
zhangwenjian 3c3d94ca76 test✅: wait for the sample rather than for the clock
Two fixed sleeps decided when this test looked: one to let the consumer pick
a message up, one to let publishes accumulate inside the reload. Both are
guesses about how fast the runner is. The consumer now signals on its first
delivery, and the measurement waits until enough publishes have landed.

The "nothing was published" guard goes with them. It was the weaker form of
the same check, and it ran after the fact instead of holding the window open
until there was something to measure.
2026-09-06 19:00:25 +08:00
zhangwenjian 6138d2d74c test✅: assert the bound installing first actually gives
The assertion demanded zero refusals during a reload, and this ordering
cannot deliver that. GetQueuePrefix returns a wrapper that captured the
adapter, so a producer that fetched before the swap and appends after
Shutdown has begun is still holding the old queue. That window is one call
wide; closing it means resolving the adapter inside Append, which is core's
to change.

What the ordering removes is the sustained window - every producer that
fetches during the wait. Measured on a race-enabled run: 174 of 174 publishes
refused with the old order, 1 of 174 with the new one. The old assertion
therefore failed about half the time on a change that works.
2026-09-06 19:00:20 +08:00
zhangwenjian d5de79f75b test✅: join the producer before the test returns
The publishing goroutine was told to stop and never waited for. t.Cleanup
restores sdk.Runtime while a producer that has not yet noticed the stop is
still reading it, which -race reports as a write and a read on the same
package variable. Signalling is not joining.
2026-09-06 19:00:10 +08:00
zhangwenjian 46e793972c fix🐛: install the new queue before shutting the old one down
Shutdown now waits for its consumers to deliver what the queue still holds, and
setupQueue called it first. For that whole wait sdk.Runtime still pointed at the
adapter that had stopped accepting, so every Append landing in the window came
back ErrQueueClosed - and both call sites in common/middleware log that at error
level while the row never reaches the database.

Measured with a held consumer: 177 of 178 publishes during one reload.

Installing first leaves no window. A producer fetches the adapter per call and
gets either the new queue or the old one, and both accept; the old one still
drains, because Shutdown is what waits for that.

The difference only exists during the wait - after Setup returns the two orders
look identical, which is why the test holds a consumer and publishes throughout
the reload rather than checking the state afterwards.

The counter-proof compiles and reports the 177.
2026-09-06 17:37:01 +08:00
zhangwenjian 46f4092b43 build🔧: require go-admin-core v2.7.0
It carries the queue shutdown fixes: a closed queue now delivers what it already
accepted and stops the goroutines consuming it. Both matter here, because this
host rebuilds its queue adapter on every configuration reload and shuts one down
on the way out.

Nothing in this commit uses the new behaviour. The one place that has to change
because of it follows.
2026-09-06 17:37:01 +08:00
wenjianzhang 196195357b Merge pull request #908 from go-admin-team/feat/readiness-probe
feat✨: 新增 /ready,并让它在关闭一开始就失败
2026-09-06 10:07:45 +08:00
zhangwenjian c579c5f84c fix🐛: give every cache probe its own key
One fixed key meant two probes overlapping - two /ready requests, or two
instances against the one redis, which is the normal deployment - overwrote each
other's value between the write and the read, and each concluded the cache was
broken. A readiness probe that reports false negatives under load pulls healthy
instances out of the pool, which is worse than not probing.

The key now carries eight random bytes. The written value is still read back,
because that is what tells a healthy cache apart from one that answers "miss"
for everything, and the key is deleted afterwards on a best-effort basis - its
error is dropped deliberately, since the verdict is already decided and a cache
that cannot delete what it just wrote is not a reason to refuse traffic.

The first version of the concurrency test had no teeth: the probes are short
enough that the scheduler ran them one after another, so the fixed-key
counter-proof passed three times out of three. The fake cache now holds every
writer until all of them have written, which makes the interleaving the test is
about actually happen. With one key that is a deterministic 15 failures out of
16 - only the last writer's value survives - and with a key per probe, none.
2026-09-06 09:55:12 +08:00
zhangwenjian 241c27358b feat✨: answer readiness separately from liveness, and fail it while draining
/health returned 200 without asking anything. Whatever it was meant to say, an
orchestrator reading it learned only that a process was accepting connections.

The two questions are not the same one, and the answers differ:

  - /health stays a bare 200. It answers "should I restart you", and a process
    whose database is unreachable does not want restarting - that turns one
    outage into a crash loop and discards the connection pool, the cache and
    every request in flight on the way.
  - /ready is new. It answers "should I send you requests", fails while a
    dependency is unreachable, and fails from the moment shutdown begins.

That last part is what the life-cycle phases bought. BeginDraining sits next to
BeginShutdown, before the server stops accepting, so a load balancer is told to
stop sending while this instance can still finish what it holds. Reversed - and
that is where it was - the connections are cut first and the probe reports it
afterwards.

The queue is deliberately not checked. Nothing on AdapterQueue answers "are you
reachable" without publishing something, the memory backend cannot fail, and a
queue that is down degrades logging rather than stopping requests: a reason to
alert, not a reason to leave the pool.

The cache probe writes and reads back rather than only reading. A cache that
answers "miss" for every key - a client pointed at the wrong server - is
indistinguishable from a healthy one on a read alone.

Every check runs behind a recover, and that is not defensive habit. The test for
"nothing configured" found the reason: GetCacheAdapter builds a wrapper around
whatever is configured and returns it even when nothing is, so the value is not
nil, the cache inside it is, and Set dereferences it. A nil check cannot see
that, and GetQueueAdapter behaves the same way. Whatever the cause, a probe is
the last thing that should be able to take the process down - the caller is
asking whether this instance is well, and killing it is the wrong reply.

The counter-proof compiles and fails: without the recover, the unconfigured case
panics rather than reporting two failed checks.
2026-09-06 09:47:55 +08:00
wenjianzhang aa3c9866cb Merge pull request #907 from go-admin-team/test/892-reload-generation
test✅: 补上 #892 修复中没有断言的那一半
2026-09-06 09:47:07 +08:00
zhangwenjian 2ac01ea584 test✅: restore what Setup writes outside this test
Setup installs the cache and queue adapters on sdk.Runtime, which is a
package-level singleton, and records what it installed in this package's own
variables. The test left all of it behind, so what a later test in this binary
saw depended on whether this one had run - the leak cmd/api's freshRuntime and
common/middleware's copy of it exist to prevent.

sdk.Runtime is swapped for a fresh one and everything is put back in Cleanup.
2026-09-05 23:18:01 +08:00
zhangwenjian 7f9cc1e435 test✅: a configuration reload installs a new queue for the consumers to notice
Issue #892 is that a reload replaces the queue adapter and the consumers
registered against the previous one are left attached to a queue nobody
publishes to any more.

The fix has two halves and only one of them was covered. attachQueueConsumers
gives a new queue its own consumers and the same queue none, which cmd/api
tests against a queue it controls by passing generation numbers in by hand. What
nothing asserted is that a reload actually produces a new generation for it to
notice - the half that lives in this package.

Setup is what config re-runs on every change, so calling it twice is what a
reload does here. The generation goes 0, 1, 2.

The counter-proof compiles and fails: dropping the increment in setupQueue
reports that the first Setup installed no queue at all, because a generation
that never moves is indistinguishable from never having been set.

This closes the loop rather than adding coverage for its own sake: with both
halves asserted, the claim that #892 is fixed rests on tests instead of on
reading the two functions and believing they meet.
2026-09-05 23:01:27 +08:00
wenjianzhang 4fb0529d2d Merge pull request #906 from go-admin-team/feat/005-host-wiring
feat✨: 生命周期阶段接线,并修掉队列注册顺序与从未停止的 cron
2026-09-05 22:49:50 +08:00
zhangwenjian 8ee4141af6 fix🐛: check the certificate before announcing that the port is reachable
AfterListen promises a hook that the port answers. The bind was moved onto the
caller's goroutine to keep that promise, but with ssl enabled there was a second
way to fail after the announcement: ServeTLS reads the certificate files itself,
on the serving goroutine, so a bad path or an unreadable key surfaced once the
hooks had already run.

tls.LoadX509KeyPair now runs before anything is announced, and its error is
returned from startServing the way a failed bind is. ServeTLS still does the
real work - handing it a tls.Listener built here instead would take over the
HTTP/2 negotiation it sets up, and quietly drop h2 for every TLS deployment. The
cost is one extra read of the certificate at startup.

The fatal in the serving goroutine said "listen:". Neither the bind nor the
certificate reaches it any more, so it says "serve:".

The test covers the certificate path alongside the bind: neither may announce
the phase, and neither may seal it.

The counter-proof is not clean, and saying so is the point. Removing the check
does turn the run red, but through log.Fatal killing the process from the
serving goroutine - "fatal serve: open no-such.pem: no such file or directory" -
rather than through the assertion. That still demonstrates the defect, because
the process could only get there after startServing had returned successfully
and the phase had been announced; it cannot be observed from inside the test,
because the fatal races the assertion that would report it.

Also: the redis-backed queue tests now fail instead of skipping when CI is set
and GO_ADMIN_TEST_REDIS_ADDR is not. A workflow that renamed the variable or
dropped the service would otherwise stay green while those two tests quietly did
nothing - the same shape as the defect they exist to cover. Locally, with no CI
in the environment, they still skip.
2026-09-05 22:41:53 +08:00
zhangwenjian 36f2549172 test✅: pin BeforeRouter to the moment before the engine exists
buildRouter is split out of run() so the order it establishes can be asserted:
the phase is announced, then initRouter builds the engine, then runStartupHooks
drains the registries.

The test covers both halves of the distinction the contract draws. A
BeforeRouter callback sees no engine - that is what the phase means, the last
point at which a module can still affect how routes are built. A callback in the
before registry, two lines later, sees one. The names invite treating them as
the same moment and they are not.

No database is involved. AuthInit reads ApplicationConfig.Mode and JwtConfig and
nothing else, and building a router registers handlers rather than calling them,
so the whole sequence runs in a package test with two package-level values set.

freshRuntime swaps the global runtime for the duration: runStartupHooks seals
the registries it drains, and a sealed registry silently drops everything
registered afterwards, which would leave every later test in this binary passing
while proving nothing.

The counter-proof compiles and fails - announcing the phase after initRouter
reports "BeforeRouter saw engine &{...}, want nil".
2026-09-05 22:31:12 +08:00
zhangwenjian dff0e64f51 test✅: run the queue's ordering rule against a real redis in CI
The rule that consumers are registered before the queue is started had no test
that could fail on the backend it exists for. Everything so far ran on the
memory queue, which is the default: queue.Memory's Register starts another
consumer goroutine whatever the state, so the wrong order passes there. A suite
that only exercises the default reports success for a queue that accepts no
consumers at all.

CI gets a redis service, and two tests build the queue the way setupQueue does -
through config.QueueConfig.Setup, so what is under test is the adapter this
repository actually gets, LegacyQueueAdapter included. They skip without
GO_ADMIN_TEST_REDIS_ADDR, so a developer with no server still gets a green run.

Registering first and starting second delivers the message. Starting first and
registering second is refused: no consumer group was created, so Append comes
back with storage.ErrNoHandler. Pinning that particular error rather than "some
error" is deliberate - the test is about the missing consumer, and a connection
failure that happened to error too would otherwise pass for it.

Running it corrected something written two commits ago. The claim there was
that a late registration loses consumers "with nothing said". Only half of that
holds: the registration is silent, because Register returns nothing, but every
publish afterwards fails loudly - ErrNoHandler, logged at error level by both
call sites in common/middleware - while the log rows are never written. The
symptom is missing rows plus a lot of noise, not a quiet nothing. That commit's
message and the contract doc both say so now.
2026-09-05 22:31:12 +08:00
zhangwenjian 0b78bc1e2e docs📝: write down where the life-cycle phases land in this host
core's contract says what the four phases promise. This says which line of
cmd/api/server.go each of them is, which is the part an application author
cannot read out of core.

Three things are recorded because getting them wrong is silent:

  - BeforeRouter is not the before registry. Those callbacks run from
    runStartupHooks, which is after initRouter has built the engine; the phase
    is before it. The two are one line apart in the same function and describing
    them as equivalent is the mistake this paragraph exists to prevent.
  - AfterResource runs again after every configuration reload, so a callback
    there is idempotent with respect to a resource rather than doing nothing
    the second time. The queue consumers are the worked example, in both
    directions: a new adapter must get consumers, the same adapter must not get
    them twice. Identity has to come from where the resource is created -
    GetQueueAdapter and GetQueuePrefix build a fresh wrapper per call, so two
    of them never compare equal however often the adapter underneath changed.
  - Consumers must be registered before the queue is started, and which
    implementation is behind the interface decides whether that matters:
    QueueConfig.Setup hands back queue.NewMemory when there is no redis
    section, and that one does not care; a redis section reaches
    LegacyQueueAdapter, whose Register cannot report a refusal. The table says
    so rather than leaving "only on redis" as a claim. What is silent is the
    registration alone - every publish afterwards is refused with ErrNoHandler
    and logged at error level, so the symptom is missing log rows plus a lot of
    noise, not a quiet nothing.

The third-layer table loses its "queue consumers are lost after a hot reload"
row, which is what AfterResource is for, and gains the honest replacement: the
four phases are the only mount points there are. There is no "after the routes
are installed, before the socket is listening".

AfterListen is described as the port being bound rather than Serve being in its
accept loop. Serve runs on another goroutine and may not have reached the first
Accept; what is true is that the bind returned, so the kernel is queueing
connections. A bind that fails produces no phase at all - the error returns
from run() and the banner never prints.
2026-09-05 22:31:12 +08:00
zhangwenjian 94163f9afb fix🐛: stop the job scheduler on the way out, and start one per tenant
The per-tenant setup ended with `defer crontab.Stop()` on the line above
`select {}`. The select never returned, so the deferred call was unreachable
for the life of the process: the scheduler had never once been stopped. And
because setup never returned, the `for k, db := range dbs` loop in Setup never
reached its second iteration - with several tenant databases configured, only
whichever one came first out of the map ever got a scheduler at all.

Both fall out of deleting the select, which was blocking for nothing: cron.Start
is `go c.run()` and has never needed anything to hold the caller.

The stop becomes a BeforeExit callback. cron.Stop returns a context that closes
once the jobs already running have finished, so the shutdown budget has
something real to bound - and giving up on that wait leaves those jobs running
until the process exits, which is better than holding the whole shutdown open
for one job that will not end.

Startup moves from a bare goroutine in run() onto AfterListen. Two reasons: the
phase runs behind core's panic guard, which does not reach across a goroutine
boundary, so a panic while loading jobs used to take the process down; and the
jobs it starts can call the API, which is only true once the socket is
accepting. It can be synchronous now precisely because setup returns.

Tested where it can be: startCrontab is split out so a scheduler can be started
with no database in sight. The job runs every second; after RunShutdown, two
and a half seconds of silence is the assertion. The counter-proof - registering
no callback, which is what this commit replaces - compiles and reports "the job
fired 2 more times after shutdown".

There is one test, not several, because BeforeExit closes to further
registration once it has run; a second RunShutdown in the same binary would
find an empty registry and pass while proving nothing.

**The multi-tenant half has no test.** setup needs a *gorm.DB per tenant before
it reaches the line that was blocking, and this repository's CI has no database
- `make build` is CGO_ENABLED=0 with no sqlite tag. It is the same defect
though: the loop could not advance past a call that never returned.
2026-09-05 22:31:12 +08:00
zhangwenjian 4510b06959 fix🐛: register the queue consumers before the queue is started
setupQueue ended with `go queueAdapter.Run()`, and the three log consumers were
registered afterwards, from setup(). The contract implementations refuse a
registration once the queue is running - memqueue and the redis queue both
answer storage.ErrQueueAlreadyStarted - and Register cannot report it: it
returns nothing, which its own comment in core records as the reason the
interface is deprecated. Start first and register second, across two
goroutines, and the registration is dropped without the caller being able to
tell.

What follows is not quiet. No consumer group was created, so redis refuses
every later publish with storage.ErrNoHandler, and go-admin logs that at error
level from both call sites while the login and operation log rows are simply
never written. The silence is in the registration; the cost shows up on every
request after it.

Which implementation is behind the interface depends on the configuration.
config.QueueConfig.Setup returns queue.NewMemory directly when there is no
redis section - and that one does not care about the order, because its
Register just starts another consumer goroutine. Only a redis section reaches
storage.LegacyQueueAdapter, which wraps the contract implementation and
therefore refuses. So the defect is invisible in the default deployment and
shows up only where redis is configured, dropping the login log, the operation
log and the api check - the three things #892 was about.

The start therefore moves to the code that registers, and nothing starts the
queue but that.

The registration also moves onto AfterResource. It has to: a reload rebuilds
the adapter, and consumers attached to the one that existed at start-up are
attached to a queue nobody publishes to any more. Being on that phase means
running again on every reload, so the callback is idempotent with respect to a
given queue rather than "does nothing the second time" - registering twice on
the same queue would give every message two consumers and write every log row
twice.

Identity for that comes from common/storage, where the adapter is built, as a
generation counter. It cannot come from the accessors: GetQueueAdapter and
GetQueuePrefix build a fresh runtime.Queue wrapper on every call, so comparing
two of them compares two wrappers and never matches however many times the
adapter underneath has been replaced. A counter also keeps the comparison on a
uint64 rather than an `==` between two interface values, which would panic on
an adapter type that is not comparable.

Generation 0 means the configuration has no queue section, so nothing was
installed and the runtime hands back its own memory queue. That case still gets
consumers, because the registration this replaces was unconditional and
dropping it would stop the logs for anyone who commented the section out.

Two things fixed on the way past:

  - `if q := sdk.Runtime.GetQueueAdapter(); q != nil { q.Shutdown() }` was
    always true. GetQueueAdapter never returns nil - with nothing configured
    the runtime falls back to its own memory queue and wraps that - so the
    first start shut down the fallback queue before anything had used it. Only
    an adapter this package installed is shut down now.
  - config.Setup becomes bootstrap.SetupConfig, which is what announces
    AfterResource, and announces it after the callbacks that build the
    resources rather than before.

attachConsumersOnce is split out so the order and the once-ness can be checked
against a queue the test controls; neither can be read back out of a real
adapter. Four tests cover the ordering, both directions of the idempotency
rule, and the unconfigured case. Both counter-proofs compile and fail: calling
Run before the registrations reports each of the three as "came after Run", and
dropping the generation guard reports eight calls where four are wanted.

One honest limit: the counter-proof for the ordering makes Run synchronous.
The original arrangement started the queue on another goroutine, and a race
cannot be made to fail every time - which is the reason the order is enforced
by structure here instead of being left to be noticed in use.
2026-09-05 22:31:12 +08:00
zhangwenjian 750c7c744e feat✨: run the BeforeExit callbacks on the way out
A module can now register cleanup and have it happen. Until this commit the
process stopped serving and returned; anything a module had set up went down
with the process rather than being taken down.

BeginShutdown is said first, before anything is dismantled. Without it a
configuration reload arriving in this window re-runs AfterResource - rebuilding
the pool and the queue adapter and re-registering consumers - on top of cleanup
that has already run.

The cleanup runs whether or not Shutdown reported an error, which is the whole
reason that error stopped being fatal in the first place: Shutdown fails exactly
when connections were still in flight, and that is when there is most left to
take down.

The two budgets are spent one after the other, so what has to fit inside the
orchestrator's grace period is their sum. `docker stop` allows 10s by default
before SIGKILL; 5+3 leaves room to finish returning. Raising one without
lowering the other buys nothing.

Both halves are tested through the existing subprocess child, which now
registers a BeforeExit callback of its own:

  - after a Shutdown that timed out, the callback still runs. Moving the call
    into the success branch reports "the BeforeExit callback did not run after
    a failed Shutdown".
  - a callback that outlasts its budget is abandoned, not awaited. It sleeps
    two seconds against a 300ms budget; RunShutdown reports the deadline, the
    process exits cleanly inside one second, and the callback's own marker
    never appears. Widening the budget to five seconds makes the test time out
    waiting for the exit, which is what "awaited" looks like.

Both counter-proofs compile and fail.
2026-09-05 22:31:12 +08:00
zhangwenjian d52dca1cb6 feat✨: announce BeforeRouter and AfterListen, and bind before either
Two phases are now announced from the command that serves traffic, so a module
can attach to them instead of being called by name from here.

The listener is opened by this goroutine rather than left to ListenAndServe,
which binds on the goroutine that serves. That mattered for the phase: a hook
on AfterListen is promised a reachable port, and with the bind happening out of
sight there was no way to keep that promise - "address already in use" surfaced
on a goroutine nobody read, after the banner had already announced the server
was up. It is now returned from run() and the process exits non-zero without
claiming anything.

AfterListen is announced synchronously. Moving it to a goroutine to save the
few milliseconds would let it overlap the shutdown, and on a fast SIGTERM the
cleanup callbacks could finish before the startup ones did.

What is left in the serving goroutine is still log.Fatal, deliberately: the
bind is no longer among the errors that reach it, so what remains is a serve
that failed after the port was taken, and carrying on would park the process on
<-quit with nothing serving. ServeTLS is the one case that can still fail
immediately, since it reads the certificate files - with ssl enabled a hook can
still run against a server on its way down. That is not a regression (the old
code printed the banner in the same situation) and it is not fixed here.

BeforeRouter is placed before initRouter, which is a different moment from the
before registry runStartupHooks drains: those callbacks run after the engine
has been built, not before it.

AfterListen is tested here, in one test rather than two because the phase seals
itself once it has run: a second test would find a closed registry and pass
while proving nothing. Both counter-proofs compile and fail - announcing on a
failed bind reports "AfterListen ran 1 times after a failed bind", and
`go RunPhase(...)` reports "ran 0 times, want 1" against the hook's own pause.

BeforeRouter's placement is not asserted in this commit. The test for it comes
with the buildRouter extraction later in this branch.
2026-09-05 22:31:11 +08:00
zhangwenjian 71413a4248 build🔧: require go-admin-core v2.6.0
It carries the life-cycle phases and the shutdown registry. Nothing here uses
them yet - the wiring is the commits that follow, and keeping the bump on its
own means a bisect can tell "the dependency moved" apart from "the host
started calling into it".
2026-09-05 22:31:11 +08:00
wenjianzhang 4fede43254 Merge pull request #905 from go-admin-team/fix/checksilent-datascope-route
feat✨(checksilent): 抓「handler 读数据权限、路由却没挂中间件」
2026-09-05 21:56:08 +08:00
zhangwenjian 0a629e2f3f docs📝(checksilent): describe the two passes the code actually makes
The comment claimed bindings were collected as the body was walked so that a
registration only saw definitions above it. That is the single-pass design this
started as. The code does two passes - one to collect, one to report - and a
registration therefore sees every binding in the function.

That is the point rather than an accident: a `.Use` written below a route is
still part of the chain, because the chain is assembled before anything is
served. The price is that a name reused for two different things in one
function resolves to the last assignment, which the comment now says instead of
promising an ordering the code does not keep.
2026-09-05 21:51:51 +08:00
wenjianzhang 37065fb089 Merge pull request #904 from go-admin-team/fix/getinfo-data-permission
fix🐛: 开了数据权限就登不进去——/getinfo 拿不到它要的 DataPermission
2026-09-05 21:34:13 +08:00
zhangwenjian 6966f14dd4 feat✨(checksilent): report a route whose handler reads a data permission nobody supplies
GetPermissionFromContext cannot fail. When no middleware put a *DataPermission
in the context it returns the zero value, and the zero value's DataScope is the
empty string - which is not one of the five scopes Permission recognises, so it
takes the default branch and fails closed. The query is handed `1 = 0` and
matches nothing.

The endpoint then reports "not found" or "no permission" for rows that plainly
exist, and only where enabledp is true. With data permissions off - the
repository default - Permission returns the query untouched and the missing
middleware costs nothing at all. A test suite and a CI that run on the default
cannot see it.

That is what happened to /api/v1/getinfo: it read the permission on a group
carrying only the JWT middleware, so every login on a deployment with data
permissions enabled ended in a 401 from the endpoint the browser calls
immediately after signing in, and went back to the login page. Three /sys-api
routes had the same shape.

The check matches a handler to the group it is registered on, through the AST
rather than through the text - a scratch grep for the same thing reported four
false positives from a comment that happened to contain the function's name,
and before that, a dozen from matching handler names across packages. Handlers
are keyed by package, type and method, so two SysUser types are two handlers.
Subgroups inherit their parent's chain, as gin does, and a `.Use` written below
a registration still counts, because the chain is assembled before anything is
served.

Either half is a fix and the message says both, because which one is right
depends on the route. A handler reading other people's rows wants the
middleware. A handler reading the caller's own row - id from the token - wants
no scope at all: DataScopeSelf matches on create_by, so scoping a self-read
rejects every user who did not create their own account. Reporting only "add
the middleware" would have turned /getinfo from broken into worse.

Five tests: the mistake, both fixes, subgroup inheritance, and a same-named
handler in another package. TestThisRepositoryIsClean covers the real tree, and
it is what fails on the commit before this one - four findings, all real.
2026-09-05 21:31:45 +08:00
90 changed files with 13229 additions and 278 deletions
+62 -2
View File
@@ -1,12 +1,25 @@
name: Build
# Documentation-only changes skip this workflow entirely.
# Documentation-only changes, and changes confined to the Kubernetes
# manifests, skip this workflow entirely.
#
# A push to master here does not just build - it pushes an image, runs the
# migrations and restarts the demo container, so the site takes a short outage.
# Paying that for a README edit is waste at best; at worst a deploy fails for a
# reason unrelated to anything in the change. Code coverage is unaffected,
# because go.yml still builds every push and pull request.
#
# scripts/k8s holds deploy.yml, storage.yml and prerun.sh, and the deploy below
# reads none of them - it is an ssh into one host that runs docker, building the
# Dockerfile at the repository root. Those manifests are for people deploying to
# a cluster of their own. The pattern is scripts/k8s/** rather than scripts/**
# because scripts/Dockerfile is a build input: go.yml builds the release image
# from it on a tag.
#
# A file outside these patterns still runs the workflow even when the rest of
# the change is ignorable: paths-ignore skips only when every changed path
# matches. Editing this file is one such case, on purpose - a deploy script
# that is never exercised by the change that broke it is worse than an outage.
on:
push:
branches: [ master ]
@@ -15,6 +28,7 @@ on:
- 'docs/**'
- 'LICENSE*'
- '.github/ISSUE_TEMPLATE/**'
- 'scripts/k8s/**'
pull_request:
branches: [ master ]
paths-ignore:
@@ -22,6 +36,7 @@ on:
- 'docs/**'
- 'LICENSE*'
- '.github/ISSUE_TEMPLATE/**'
- 'scripts/k8s/**'
# One deploy at a time. Two merges seconds apart raced here: both runs did
# docker rm -f then docker run, the second removed the container the first had
@@ -102,7 +117,41 @@ jobs:
test -f "$CFG" || { echo "宿主机配置缺失,中止部署"; exit 1; }
# Old images of this repository are removed here and nowhere else.
# Every deployment pulls one tagged with its commit and nothing ever
# removed the previous one, so they only accumulated: 68 of them
# filled the disk and the next deployment could not pull.
#
# Three are kept so a release can be re-run by tag by hand.
#
# Only this repository's images are listed, because the host runs
# other services whose images are not this script's business. The
# image the live container is on is excluded by id rather than by
# position, so it survives even if the listing order is not what
# it looks like. With no container to ask, the function returns
# rather than running the pipeline on an empty id - which would
# also delete nothing, but by way of grep -v matching every line,
# which reads like the opposite of what it does. No -f, so an image
# any container still holds - including the one kept for rollback -
# is refused rather than taken away from it.
prune_old_images() {
REPO="${IMG%:*}"
LIVE=$(sudo docker inspect -f '{{.Image}}' "$NAME" 2>/dev/null | sed 's/^sha256://' | cut -c1-12)
[ -n "$LIVE" ] || return 0
sudo docker images "$REPO" --format '{{.ID}} {{.Repository}}:{{.Tag}}' \
| grep -v "^$LIVE" \
| tail -n +3 \
| awk '{print $2}' \
| xargs -r -n1 sudo docker rmi >/dev/null 2>&1 || true
}
sudo docker login --username=${{ secrets.DOCKER_USERNAME }} registry.ap-northeast-1.aliyuncs.com --password=${{ secrets.DOCKER_PASSWORD }}
# Before the pull, not only after a successful deploy. The pull
# is the first thing here that needs space and it is where a full
# disk stops this script, so a cleanup that only runs afterwards
# never runs on the host that needs it: rerunning the workflow
# fails at the same pull, and the disk has to be cleared by hand.
prune_old_images
sudo docker pull "$IMG" || { echo "拉取镜像失败,中止部署"; exit 1; }
# 迁移用新镜像跑。失败时线上仍是旧版本配旧 schema,是自洽的;
@@ -115,7 +164,14 @@ jobs:
if sudo docker ps -a --format '{{.Names}}' | grep -qx "$NAME"; then
sudo docker rm -f "$PREV" >/dev/null 2>&1 || true
sudo docker rename "$NAME" "$PREV"
sudo docker stop "$PREV" >/dev/null
# --timeout, because the default is 10 seconds and the process
# spends drain + server + cleanup from extend.shutdown before it
# exits - 8 seconds out of the box, and more for anyone who
# configures a drain window. Past the deadline docker sends
# SIGKILL and the cleanup callbacks are cut off part-way through.
# checksilent's docker-stop-cuts-shutdown-short check compares
# this number against config/settings.yml.
sudo docker stop --timeout 30 "$PREV" >/dev/null
fi
sudo docker run -d -p 8000:8000 \
@@ -133,6 +189,10 @@ jobs:
if [ "$ok" = "1" ]; then
sudo docker rm -f "$PREV" >/dev/null 2>&1 || true
# Again, so the image this deployment replaced falls out of the
# window rather than waiting for the next deployment to notice.
prune_old_images
else
echo "健康检查失败,回滚到上一版本"
sudo docker logs --tail 40 "$NAME" 2>&1 || true
+80
View File
@@ -15,6 +15,69 @@ jobs:
build:
name: Build
runs-on: ubuntu-latest
# The queue's ordering rule - consumers registered before the queue is
# started - is invisible on the memory backend, which is the default and
# therefore what every other test runs on: queue.Memory's Register starts a
# consumer goroutine whatever the state. Only redis refuses a late
# registration, so without a server here the tests that cover it would skip
# and the suite would report success for a queue that accepts no consumers.
services:
redis:
image: redis:7-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 3s
--health-retries 10
postgres:
image: postgres:15-alpine
env:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: goadmin_test
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 3s
--health-retries 10
# The fourth registered driver, and the one that disagrees with the
# other three about NULL: its unique index treats two NULLs as equal and
# permits one. A migration that builds a unique index over a nullable
# column therefore fails here and nowhere else, which is how one shipped
# that no SQL Server database could apply at all - not even an empty
# one. The password is this container's only credential and the
# container lives for the length of one job.
sqlserver:
image: mcr.microsoft.com/mssql/server:2022-latest
env:
ACCEPT_EULA: "Y"
MSSQL_SA_PASSWORD: GoAdmin_Test1
MSSQL_PID: Developer
ports:
- 1433:1433
options: >-
--health-cmd "/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P GoAdmin_Test1 -C -Q 'SELECT 1'"
--health-interval 10s
--health-timeout 5s
--health-retries 20
--health-start-period 20s
env:
GO_ADMIN_TEST_REDIS_ADDR: 127.0.0.1:6379
# The soft-delete conversion drops an index, and gorm's PostgreSQL driver
# produced unparseable SQL for that - on SQLite, where the rest of these
# tests run, the same code works. The suite reported success for a
# migration that failed on every PostgreSQL database it was pointed at.
# See go-admin#919.
GO_ADMIN_TEST_POSTGRES_DSN: "host=127.0.0.1 port=5432 user=postgres password=postgres dbname=goadmin_test sslmode=disable"
GO_ADMIN_TEST_SQLSERVER_DSN: "sqlserver://sa:GoAdmin_Test1@127.0.0.1:1433?database=goadmin_test"
steps:
- name: Set up Go 1.26
@@ -26,6 +89,14 @@ jobs:
- name: Check out code into the Go module directory
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
# SQL Server has no equivalent of POSTGRES_DB, so the database the DSN
# names has to be created before the tests run.
- name: Create the SQL Server test database
run: |
docker exec ${{ job.services.sqlserver.id }} /opt/mssql-tools18/bin/sqlcmd \
-S localhost -U sa -P GoAdmin_Test1 -C \
-Q "IF DB_ID('goadmin_test') IS NULL CREATE DATABASE goadmin_test"
- name: Get dependencies
run: go mod tidy
@@ -38,6 +109,15 @@ jobs:
- name: Build
run: make build
# A separate module, so none of the steps above see it: the main module's
# go.mod, its build and its tests are all unaware of the example
# application. This is the only thing that exercises an application being
# installed at all - everything below it runs against an injected engine
# and a hand-built schema, and none of that can catch an application's
# init() reaching one registry and not the other.
- name: End-to-end install and uninstall
run: make test-e2e
# Fails the build on the silent-failure classes listed in
# tools/checksilent, one of which is the contract boundary: nothing under
# common/ may import app/. A boundary that is only written down erodes; this
+10 -2
View File
@@ -227,8 +227,9 @@ go run -tags sqlite3 . server -c config/settings.sqlite.yml
## 静默失败校验
`make checksilent` 检查七类**不报错、不记日志、行为悄悄变得不对**的问题,
CI 会跑,命中 ERROR 即失败:
`make checksilent` 逐条检查那些**不报错、不记日志、行为悄悄变得不对**的问题,
CI 会跑,命中 ERROR 即失败。这里不写条数——写死的数字会悄悄过时,
真正的清单是 `tools/checksilent/checks.go` 里 `runChecks` 跑的那几个:
| 检查 | 级别 | 静默后果 |
|---|---|---|
@@ -238,8 +239,15 @@ CI 会跑,命中 ERROR 即失败:
| `menu-id-collision` | ERROR | 两个模块硬编码同一菜单 ID,互相覆盖 |
| `contract-import-boundary` | ERROR | 契约包 import `app/`,应用无法独立编译 |
| `contract-shim-alias` | ERROR | 契约薄壳写成 defined type 而非别名,方法集丢失,本仓可能照常编译、第三方应用编译不过 |
| `datascope-route-unguarded` | ERROR | handler 读调用方的数据权限,而注册它的路由组没装提供权限的中间件。取不到时拿到零值、走 fail-closed 分支,查询被塞进 `1 = 0`:接口对确实存在的行返回「查不到」,且只在 `enabledp: true` 的部署上出现 |
| `shutdown-budget-overruns-grace` | ERROR / WARN | `settings.yml` 的 `extend.shutdown` 预算(含清单里的 `preStop`)放不进自带 k8s 清单的 `terminationGracePeriodSeconds`,SIGKILL 在清理回调跑到一半时到达 |
| `docker-stop-cuts-shutdown-short` | ERROR / WARN | 停止容器的两条路径——脚本/工作流里的 `docker stop`,和 `docker-compose.yml` 的 `stop_grace_period`——没写或写得不够关闭预算用。两边默认都是 10 秒,而这个数字离命令很远,调大预算的人不会想起它 |
| `menu-name-mismatch` | WARN | 菜单名与前端组件 `name` 不一致,keep-alive 缓存静默失效 |
两条关闭预算检查分两级,用的是同一条算术和同一个 5 秒边际:真的超限报 ERROR,
放得进但余量不足 5 秒报 WARN。余量不足做 WARN 不做 ERROR,是因为那是个技术上
跑得通的配置——**一条在正确配置下也会响的 ERROR,训练的是忽略它**。
最后一条要跨仓库比对,只能做正则启发式,因此是 WARN,**不影响退出码**,
且默认跳过;要跑它得指定前端目录:
+21 -5
View File
@@ -4,10 +4,26 @@ FROM alpine
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
RUN apk update --no-cache
RUN apk add --update gcc g++ libc6-compat
RUN apk add --no-cache ca-certificates
RUN apk add --no-cache tzdata
# Runtime packages only.
#
# gcc and g++ used to be installed here and were 273MB of a 381MB image. The
# binary this image runs is compiled and linked before the image is built and
# arrives as a COPY, so nothing in the container ever invokes a compiler -
# there is no toolchain to drive it with either, since Go itself is not
# installed.
#
# That layer was also why a host could not share storage between images. apk
# resolves against an index that moves, so the layer digest differed on every
# build and no two images shared it: a host keeping one image per deployed
# commit stored a private 273MB copy each time. 68 of them filled the disk
# and the next deployment could not pull.
#
# libc6-compat stays. Nothing measured needs it - the binary CI produces is
# statically linked, and a container built without libc6-compat resolves a
# hostname and opens a database connection exactly as one built with it - but
# it is half a megabyte and it covers a ./main that was linked dynamically,
# which this Dockerfile has no way to check.
RUN apk add --no-cache ca-certificates tzdata libc6-compat
ENV TZ Asia/Shanghai
COPY ./main /main
@@ -15,4 +31,4 @@ COPY ./config/settings.demo.yml /config/settings.yml
COPY ./go-admin-db.db /go-admin-db.db
EXPOSE 8000
RUN chmod +x /main
CMD ["/main","server","-c", "/config/settings.yml"]
CMD ["/main","server","-c", "/config/settings.yml"]
+22 -1
View File
@@ -15,7 +15,16 @@ build-sqlite:
# make run
run:
# delete go-admin-api container
@if [ $(shell docker ps -aq --filter name=go-admin --filter publish=8000) ]; then docker rm -f go-admin; fi
#
# stop then rm, rather than `rm -f`. The force flag kills a running
# container with SIGKILL and no grace at all, so restarting locally cut
# short every shutdown this application does - the drain window was never
# once reached on a developer's machine. --timeout has to cover
# extend.shutdown's drain + server + cleanup; checksilent's
# docker-stop-cuts-shutdown-short check compares it against
# config/settings.yml. On a container that has already stopped, stop is a
# no-op and the removal is unchanged.
@if [ $(shell docker ps -aq --filter name=go-admin --filter publish=8000) ]; then docker stop --timeout 30 go-admin && docker rm go-admin; fi
# 启动方法一 run go-admin-api container docker-compose 启动方式
# 进入到项目根目录 执行 make run 命令
@@ -45,6 +54,18 @@ stop:
test:
go test -race -cover ./...
# The end-to-end install, which `test` above cannot reach: test/e2e-apporder
# is its own module, so `./...` in this one does not include it. It builds a
# go-admin binary with the example application linked in and drives
# `migrate install` / `migrate uninstall` against a real database.
#
# Its own target rather than a line in the CI workflow, so the one thing in
# the build that exercises installing an application is also the one thing
# somebody can run before pushing.
.PHONY: test-e2e
test-e2e:
cd test/e2e-apporder && go test ./... -count=1
# Reports the failures that do not announce themselves - see
# tools/checksilent. Exits non-zero on an ERROR; the one WARN-level check
# prints and does not fail the build.
-6
View File
@@ -277,15 +277,11 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/ninstein" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/580303?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/kikiyou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/17959053?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/horizonzy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/22524871?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Cassuis" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/48005724?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/hqcchina" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/5179057?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/nodece" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16235121?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stephenzhang0713" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/18169290?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/zhouxixi-dev" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/100399679?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Jalins" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31172582?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wkf928592" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6063351?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxxiong6" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6983441?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Silicon-He" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/52478309?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/GizmoOAO" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20385106?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/bestgopher" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/36840497?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxb1207" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20775558?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
@@ -299,8 +295,6 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/infnan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/38274826?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/d1y" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45585937?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/qlijin" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/515900?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/logtous
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/88697234?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stepway
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/9927079?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/NaturalGao
-6
View File
@@ -277,15 +277,11 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/ninstein" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/580303?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/kikiyou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/17959053?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/horizonzy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/22524871?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Cassuis" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/48005724?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/hqcchina" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/5179057?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/nodece" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16235121?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stephenzhang0713" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/18169290?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/zhouxixi-dev" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/100399679?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Jalins" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31172582?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wkf928592" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6063351?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxxiong6" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6983441?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Silicon-He" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/52478309?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/GizmoOAO" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20385106?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/bestgopher" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/36840497?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxb1207" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20775558?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
@@ -299,8 +295,6 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/infnan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/38274826?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/d1y" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45585937?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/qlijin" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/515900?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/logtous
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/88697234?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stepway
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/9927079?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/NaturalGao
-6
View File
@@ -263,15 +263,11 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/ninstein" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/580303?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/kikiyou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/17959053?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/horizonzy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/22524871?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Cassuis" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/48005724?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/hqcchina" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/5179057?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/nodece" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16235121?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stephenzhang0713" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/18169290?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/zhouxixi-dev" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/100399679?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Jalins" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31172582?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wkf928592" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6063351?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxxiong6" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6983441?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Silicon-He" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/52478309?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/GizmoOAO" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20385106?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/bestgopher" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/36840497?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxb1207" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20775558?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
@@ -285,8 +281,6 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/infnan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/38274826?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/d1y" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45585937?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/qlijin" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/515900?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/logtous
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/88697234?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stepway
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/9927079?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/NaturalGao
-6
View File
@@ -277,15 +277,11 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/ninstein" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/580303?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/kikiyou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/17959053?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/horizonzy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/22524871?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Cassuis" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/48005724?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/hqcchina" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/5179057?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/nodece" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16235121?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stephenzhang0713" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/18169290?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/zhouxixi-dev" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/100399679?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Jalins" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31172582?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wkf928592" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6063351?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxxiong6" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6983441?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/Silicon-He" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/52478309?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/GizmoOAO" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20385106?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/bestgopher" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/36840497?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/wxb1207" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20775558?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
@@ -299,8 +295,6 @@ pnpm dev
<span style="margin: 0 5px;" ><a href="https://github.com/infnan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/38274826?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/d1y" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45585937?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/qlijin" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/515900?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/logtous
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/88697234?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/stepway
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/9927079?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
<span style="margin: 0 5px;" ><a href="https://github.com/NaturalGao
+106
View File
@@ -0,0 +1,106 @@
package models
import (
"time"
"go-admin/common/models"
)
// The values sys_app.status takes.
//
// Three states rather than a single "installed", because an install that
// stopped partway has to be an observable row rather than the absence of one:
// the versions an app installs are separate migration files, and on MySQL a
// DDL statement commits the transaction around it - taking an outer
// transaction and every savepoint under it with it - so they cannot be
// wrapped in one.
//
// AppInstalling is also what a row reads as after the process was killed
// mid-install, which is why it is not treated as "installed" by anything.
const (
AppInstalling = 1
AppInstalled = 2
AppFailed = 3
)
// SysApp is the sys_app row model: one row per installed application (PRD
// 008 F2). It deliberately does not embed models.ModelTime - see the design
// doc (docs-prd/008-应用清单与安装器/数据库变更.md) §1.1 for why an
// installed-app registry does not need the millisecond soft-delete marker
// every other sys_* table follows. Uninstalling an app deletes its row
// outright; a later reinstall creates a fresh one.
type SysApp struct {
models.Model // Id int, primary key, autoincrement
// AppCode is the app.Manifest.Code / migration.ForApp / seed.SeedMenus
// identity, already lower-cased by migration.NormalizeAppCode before
// anything reaches this table. Unique: row existence alone answers G2
// ("is app X installed").
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;uniqueIndex:uk_sys_app_app_code;comment:app code"`
Name string `json:"name" gorm:"size:128;not null;comment:display name, from Manifest.Name"`
// Version is the version this row currently reflects - attempted or
// confirmed, disambiguated by Status. It does not drive which
// migrations run next; sys_migration's per-version rows do that (see
// design doc §1.5's resume flow). This field is descriptive, refreshed
// from the manifest on every install/upgrade/resume attempt.
Version string `json:"version" gorm:"size:32;not null;comment:version this row currently reflects, see Status"`
Description string `json:"description" gorm:"size:255;not null;default:'';comment:from Manifest.Description"`
Author string `json:"author" gorm:"size:128;not null;default:'';comment:from Manifest.Author"`
// Requires is a comma-separated list of app codes this app declared as
// dependencies (Manifest.Requires). Stored as plain VARCHAR CSV, not
// JSON - see design doc §1.3 for why. F8 (P1) is what validates and
// orders on this; this batch only stores what the manifest declared.
Requires string `json:"requires" gorm:"size:255;not null;default:'';comment:declared dependency app codes, comma separated"`
// Pricing/License are reserved passthrough fields (PRD 003; PRD 008
// open question 1). This batch stores whatever the manifest carries and
// does not interpret either one.
Pricing string `json:"pricing" gorm:"size:64;not null;default:'';comment:reserved, not interpreted by this batch"`
License string `json:"license" gorm:"size:64;not null;default:'';comment:reserved, not interpreted by this batch"`
// Status: 1=installing 2=installed 3=failed. Three states, not a
// single "1=installed", because a partial, stuck install has to be an
// observable row rather than "the row doesn't exist yet" - see design
// doc §1.5 for why cross-migration-file atomicity is not available on
// MySQL (implicit commit on DDL).
Status int `json:"status" gorm:"size:4;not null;default:1;comment:1=installing 2=installed 3=failed"`
// FailedVersion and LastError are DIAGNOSTIC TEXT ONLY - what a human
// looking at this row is told about the last failure, nothing more. No
// code anywhere may read either one to decide what to do next.
//
// The question "where should a resume pick up" has exactly one
// authoritative answer, and it is not these two columns: subtract
// sys_migration's applied rows for this app_code from what the app's
// own compiled-in code has registered (migration.Snapshot()/ForApp -
// the same set F7's `migrate status` already walks). That answer can
// never go stale, because it is not stored anywhere to go stale - it is
// recomputed from sys_migration every time it is asked. FailedVersion
// is a snapshot of what that computation returned at the moment of
// failure, kept only so an operator does not have to go find the
// process's logs; if it and a fresh recomputation from sys_migration
// ever disagree, sys_migration is right and this column is stale, by
// definition, and nothing should ever notice or care except a human
// reading the row.
FailedVersion string `json:"failedVersion" gorm:"size:64;not null;default:'';comment:diagnostic snapshot only, not a judgment basis; meaningful only when status=3"`
LastError string `json:"lastError" gorm:"size:255;not null;default:'';comment:diagnostic text only, not a judgment basis; meaningful only when status=3"`
// InstalledAt is when this app first reached status=installed - set
// once, never moved by a later upgrade (see design doc §1.4). Nullable,
// unlike every other column here: a row can exist before it has a
// value (a fresh install starts at status=installing). This is not the
// deleted_at problem 1786700003000_soft_delete_marker.go fixed - that
// column sat inside a unique index, where NULL <> NULL let two live
// rows coexist under the same key. InstalledAt is in no index at all,
// so nullability here opens no such hole.
InstalledAt *time.Time `json:"installedAt" gorm:"comment:first successful install time; null until status first reaches installed"`
UpdatedAt time.Time `json:"updatedAt" gorm:"comment:last updated time"`
models.ControlBy // CreateBy/UpdateBy: which operator triggered the attempt
}
func (*SysApp) TableName() string {
return "sys_app"
}
+43
View File
@@ -0,0 +1,43 @@
package models
import "time"
// SysAppCasbinGrant is a ledger of casbin_rule rows an app install created,
// keyed by the exact natural key casbin_rule itself is unique on. It exists
// because casbin_rule is not a table this project owns (see design doc
// docs-prd/008-应用清单与安装器/数据库变更.md §2.2): we cannot add an
// app_code column to it without that column being silently zeroed the first
// time anything calls the gorm-adapter's SavePolicy/SavePolicyCtx. Recording
// the natural key here, instead of a foreign key into casbin_rule, is also
// what survives SysRole.Update's RemoveFilteredPolicy+re-add cycle for a
// role's policies (app/admin/service/sys_role.go): that cycle replaces the
// underlying row (a new auto-increment ID) but reproduces the same
// (ptype,v0,v1,v2) tuple from the same sys_menu/sys_api data, so a
// natural-key match here still finds it. What it does not survive is the
// role being renamed, or the tuple being rebuilt from a completely different
// source (a future SavePolicy call from outside this seeder) - in both cases
// the match legitimately fails, and business rule 3 says the uninstaller
// should report and skip, not delete something else that happens to look
// the same.
type SysAppCasbinGrant struct {
Id int `json:"id" gorm:"primaryKey;autoIncrement"`
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;index:idx_sys_app_casbin_grant_app_code;comment:app code that created this grant"`
// Column widths mirror gorm-adapter's own CasbinRule struct exactly, so
// a value that fits into casbin_rule always fits here, and the unique
// index below matches the one createTable() puts on casbin_rule itself.
Ptype string `json:"ptype" gorm:"size:100;not null;uniqueIndex:uk_sys_app_casbin_grant_rule;comment:casbin ptype, 'p' today"`
V0 string `json:"v0" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:role_key at grant time"`
V1 string `json:"v1" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:api path"`
V2 string `json:"v2" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:http method"`
V3 string `json:"v3" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
V4 string `json:"v4" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
V5 string `json:"v5" gorm:"size:100;not null;default:'';uniqueIndex:uk_sys_app_casbin_grant_rule;comment:unused today"`
CreatedAt time.Time `json:"createdAt" gorm:"comment:when this grant was recorded"`
}
func (*SysAppCasbinGrant) TableName() string {
return "sys_app_casbin_grant"
}
+24
View File
@@ -32,6 +32,30 @@ type SysMenu struct {
// AutoMigrate adding this column to an existing table leaves every
// pre-existing row reading back as "" rather than NULL.
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;default:'';index:idx_sys_menu_app_code;comment:AppCode"`
// SeedCode is the raw seed.MenuSpec.Code this row was created from, kept
// so seedMenuTree can ask "did I already write this node" without
// relying on MenuName's PascalCase concatenation, which is not
// injective (see design doc §1.6). Nullable, unlike AppCode: every row
// seed.SeedMenus writes sets a real value, but every pre-existing row -
// the host's own hand-placed menus, and every app-seeded row written
// before this column existed - has none, and there is no way to
// backfill one that means anything. NULL is what lets an unbounded
// number of those coexist under the same app_code without tripping the
// unique index below: the database never treats two NULLs as equal, so
// only rows that do carry a real code participate in the uniqueness
// check at all.
// uk_sys_menu_app_seed_code_del is created by the migration, not from
// this tag, and deliberately: it covers (app_code, seed_code,
// deleted_at), and this struct cannot say so. A named uniqueIndex tag
// puts every field carrying that name into one index, and deleted_at
// comes from the shared ModelTime embed, which no single model can add a
// tag to. Naming it here anyway declared a unique index on seed_code
// alone under the same name - stricter than the real one, forbidding two
// applications from both having a "dir" node - and AutoMigrate on this
// model would have created that one first, after which the migration's
// HasIndex guard finds the name taken and leaves the wrong index in
// place.
SeedCode *string `json:"seedCode" gorm:"size:64;comment:raw MenuSpec.Code, null for rows not written through SeedMenus"`
models.ControlBy
models.ModelTime
}
@@ -0,0 +1,49 @@
package models
import (
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
)
func ptr(s string) *string { return &s }
// uk_sys_menu_app_seed_code_del covers (app_code, seed_code, deleted_at) and
// is created by 1786700008000, not from a struct tag. It cannot come from a
// tag: a named uniqueIndex collects every field carrying that name, and
// deleted_at lives in the shared ModelTime embed that no single model can tag.
//
// Naming it on SeedCode alone anyway produced a unique index on seed_code by
// itself under the same name - stricter than the real one - and AutoMigrate
// here would create that one, after which the migration's HasIndex guard
// finds the name taken and leaves the wrong index in place. Nothing in
// production AutoMigrates this model (the initial table migration uses a
// frozen snapshot that has neither column), which is why this never showed up
// as a broken database; it showed up the first time a test built the schema
// from the live model and seeded two applications.
func TestSysMenuDeclaresNoSeedCodeIndexOfItsOwn(t *testing.T) {
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&SysMenu{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
if db.Migrator().HasIndex(&SysMenu{}, "uk_sys_menu_app_seed_code_del") {
t.Error("AutoMigrate created uk_sys_menu_app_seed_code_del from a tag; " +
"the migration's HasIndex guard will now skip the composite index it should create")
}
// Two applications, the same seed code. The real index allows it because
// app_code is part of the key; an index on seed_code alone does not.
for _, app := range []string{"order", "crm"} {
row := SysMenu{MenuName: app + "Dir", AppCode: app, SeedCode: ptr("dir")}
if err := db.Create(&row).Error; err != nil {
t.Fatalf("%s could not use the seed code \"dir\": %v", app, err)
}
}
}
+339 -26
View File
@@ -5,6 +5,7 @@ import (
"fmt"
"strconv"
"strings"
"time"
"gorm.io/gorm"
@@ -76,7 +77,7 @@ func (adminSeeder) SeedMenus(tx *gorm.DB, appCode string, menus []seed.MenuSpec,
if len(menuIDs) == 0 && len(apiRows) == 0 {
return nil
}
return grantToAdminRole(tx, menuIDs, apiRows)
return grantToAdminRole(tx, appCode, menuIDs, apiRows)
}
// seedApis writes one sys_api row per ApiSpec and returns them keyed by
@@ -90,6 +91,23 @@ func (adminSeeder) SeedMenus(tx *gorm.DB, appCode string, menus []seed.MenuSpec,
// application's ids in the module cache. Never accepting a caller-chosen id
// here removes the collision this Seeder has no way to detect instead of
// trying to detect it after the fact.
//
// The natural key is (app_code, path, action) - the same three columns
// 1786700002000_remove_refresh_token_api.go already used to identify a
// single API by hand, and the ones 1786700008000_seed_natural_keys.go put a
// unique index on. Before inserting, this looks for a live row (deleted_at
// = 0, applied automatically by the soft-delete plugin on every query
// against models.SysApi) already holding that key and reuses it instead of
// inserting a second one - see the design doc §1.6: a migration retried
// after a partial failure previously re-ran this as a bare tx.Create and
// produced duplicate rows on the demo site.
//
// Unlike seedMenuTree's reuse branch, this one has nothing left to repair
// after finding an existing row: models.SysApi carries no association
// (nothing like SysMenu's many2many SysApi field) and this function writes
// nothing beyond the row itself - no second statement comparable to
// seedMenuTree's paths UPDATE follows tx.Create below. An interrupted retry
// can therefore only ever find this row complete or not find it at all.
func seedApis(tx *gorm.DB, appCode string, apis []seed.ApiSpec) (map[string]models.SysApi, error) {
seen := make(map[string]bool, len(apis))
rows := make(map[string]models.SysApi, len(apis))
@@ -102,6 +120,19 @@ func seedApis(tx *gorm.DB, appCode string, apis []seed.ApiSpec) (map[string]mode
}
seen[a.Code] = true
var existing models.SysApi
err := tx.Where("app_code = ? AND path = ? AND action = ?", appCode, a.Path, a.Method).
First(&existing).Error
switch {
case err == nil:
rows[a.Code] = existing
continue
case errors.Is(err, gorm.ErrRecordNotFound):
// Not seen yet; fall through to insert it.
default:
return nil, fmt.Errorf("api %q: checking for an existing row: %w", a.Code, err)
}
row := models.SysApi{
Handle: a.Handle,
Title: a.Title,
@@ -153,6 +184,12 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
continue
}
// Resolved before the idempotency check below, whether or not
// this spec's own row turns out to already exist: repairing an
// existing-but-incomplete row's paths needs the parent's
// already-resolved Paths exactly as much as creating a fresh
// row does (see repairExistingMenu), so both have to wait for
// it the same way.
var parentRow models.SysMenu
if s.Parent != "" {
parent, ok := created[s.Parent]
@@ -165,24 +202,42 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
parentRow = parent
}
row := models.SysMenu{
MenuName: menuName(appCode, s.Code),
Title: s.Title,
Icon: s.Icon,
Path: s.Path,
MenuType: s.Kind,
Permission: s.Permission,
ParentId: parentRow.MenuId,
Component: s.Component,
Sort: s.Sort,
// Visible "0" is shown, not hidden - the same defaults
// 1786700001000_demo_menu.go seeds its own menu with. A
// freshly installed application's menu should not need an
// administrator to first find and unhide it.
Visible: "0",
IsFrame: "1",
AppCode: appCode,
// Idempotency check: does this node already have a row, from
// an earlier, possibly-interrupted attempt? The natural key is
// (app_code, seed_code) - menu_name's PascalCase concatenation
// is not injective and cannot be used for this (see menuName's
// doc comment and the design doc §1.6). Only a live row counts;
// the soft-delete plugin scopes deleted_at = 0 automatically on
// every query against models.SysMenu.
var existing models.SysMenu
found := false
err := tx.Where("app_code = ? AND seed_code = ?", appCode, s.Code).First(&existing).Error
switch {
case err == nil:
found = true
case errors.Is(err, gorm.ErrRecordNotFound):
// Nothing under the natural key. It may still be here from
// before seed_code existed, under the name that identified
// it then.
existing, found, err = adoptLegacyMenu(tx, appCode, s)
if err != nil {
return nil, fmt.Errorf("%q: %w", s.Code, err)
}
default:
return nil, fmt.Errorf("%q: checking for an existing row: %w", s.Code, err)
}
if found {
row, err := repairExistingMenu(tx, existing, appCode, s, parentRow, apiRows)
if err != nil {
return nil, fmt.Errorf("%q: repairing an existing row: %w", s.Code, err)
}
created[s.Code] = row
ids = append(ids, row.MenuId)
progressed = true
continue
}
row := menuRowFor(appCode, s, parentRow)
for _, code := range s.ApiCodes {
api, ok := apiRows[code]
if !ok {
@@ -205,11 +260,7 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
// two-step create-then-update 1786700001000_demo_menu.go's
// hand-assigned ids let it do in one literal, sequenced here
// instead.
if s.Parent == "" {
row.Paths = "/0/" + strconv.Itoa(row.MenuId)
} else {
row.Paths = parentRow.Paths + "/" + strconv.Itoa(row.MenuId)
}
row.Paths = expectedPaths(row.MenuId, s.Parent, parentRow)
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", row.MenuId).
Update("paths", row.Paths).Error; err != nil {
return nil, fmt.Errorf("%q: writing paths: %w", s.Code, err)
@@ -226,6 +277,103 @@ func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows ma
return ids, nil
}
// expectedPaths is the materialized path a fresh insert of menuID under
// parent (or at the root, if parent is "") computes - factored out so
// repairExistingMenu can ask the same question about a row it did not just
// create.
func expectedPaths(menuID int, parent string, parentRow models.SysMenu) string {
if parent == "" {
return "/0/" + strconv.Itoa(menuID)
}
return parentRow.Paths + "/" + strconv.Itoa(menuID)
}
// repairExistingMenu brings a row seedMenuTree's idempotency check found up
// to what a fresh insert of the same spec would have produced.
//
// A row can be found and still be incomplete: tx.Create's own association
// write (the sys_menu_api_rule bindings from row.SysApi) and the paths
// UPDATE that follows it are each their own statement, and design doc §1.5
// establishes that nothing after the first DDL in a migration function can
// be rolled back together - a process interrupted between the row insert
// and either of those two steps leaves exactly this row: present, findable
// by its natural key, but missing what makes it a working menu entry. A
// retry that only checked "does the row exist" and stopped there would
// report success while the sys_menu_api_rule binding stays missing (the
// api is granted to no one) or paths stays empty (a materialized-path
// break that orphans the rest of the subtree from the root) - as silent as
// the duplicate-row defect the idempotency check itself was written to
// close.
//
// Both checks are read-before-write, so a row that is already complete -
// the ordinary case on every retry after the first successful one - causes
// no writes at all: existing.Paths already equals what expectedPaths
// computes, and the sys_menu_api_rule INSERT is itself guarded by
// WHERE NOT EXISTS, the same idempotent-insert shape grantToAdminRole
// already uses for sys_role_menu/casbin_rule. Never DELETEs an existing
// binding to rebuild it - that is the FullSaveAssociations mistake
// sys_role.go's SysRole.Update makes for sys_role_menu/casbin_rule
// (app/admin/service/sys_role.go:148-153), the exact pattern this design
// went out of its way to avoid for the tables that do use it.
//
// Insert-only cuts both ways, deliberately. A binding an administrator
// added by hand through the menu management UI, for an api never in
// s.ApiCodes at all, is never touched by this loop and survives every
// later retry (TestSeedMenusPreservesAHandAddedBinding is the reproduction
// case for the opposite mistake: delete-then-reinsert wipes it silently,
// the same shape as sys_role_menu/casbin_rule getting zeroed by a role
// edit, just with this code as the actor instead of the victim). The
// converse case - a MenuSpec that used to list an ApiCode and no longer
// does - is not handled here either, and that half is intentional rather
// than an oversight: this loop only ever adds rows for codes the *current*
// call's ApiCodes names, so a binding for a code an earlier version
// granted and the current one dropped is left in place, stale. Reconciling
// that is deleting something, which needs the same certainty about
// ownership uninstall's design (see design doc §5) already requires -
// this function has no way to tell "stale, from an older version of this
// same app" apart from "hand-added, for a reason", and business rule 3
// ("uninstall deletes only what it can attribute with certainty") applies
// here just as much as it does there. Reconciling stale seed-driven
// bindings, if it is ever wanted, belongs in the upgrade path with that
// same ownership check - not silently inside every retry of every install.
func repairExistingMenu(tx *gorm.DB, existing models.SysMenu, appCode string, s seed.MenuSpec, parentRow models.SysMenu, apiRows map[string]models.SysApi) (models.SysMenu, error) {
// Every column the spec decides, not just the two this used to touch. A
// menu whose parent was removed and reseeded kept parent_id pointing at
// the dead row while its paths named the new one, and the tree is built
// from parent_id - so the menu vanished from the sidebar with the
// migration reporting success. An application that renamed a menu or
// moved its component between versions had its change silently ignored
// for the same reason: nothing here wrote those columns.
want := menuRowFor(appCode, s, parentRow)
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", existing.MenuId).
Select(specMenuFields).Updates(want).Error; err != nil {
return models.SysMenu{}, fmt.Errorf("bringing the row up to the spec: %w", err)
}
want.MenuId = existing.MenuId
want.Paths = existing.Paths
want.Visible, want.IsFrame = existing.Visible, existing.IsFrame
if err := repairPaths(tx, &want, s, parentRow); err != nil {
return models.SysMenu{}, err
}
existing = want
for _, code := range s.ApiCodes {
api, ok := apiRows[code]
if !ok {
return models.SysMenu{}, fmt.Errorf("ApiCodes references %q, which is not an ApiSpec.Code in this call", code)
}
if err := tx.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) SELECT ?, ? WHERE NOT EXISTS (SELECT 1 FROM sys_menu_api_rule WHERE sys_menu_menu_id = ? AND sys_api_id = ?)",
existing.MenuId, api.Id, existing.MenuId, api.Id,
).Error; err != nil {
return models.SysMenu{}, fmt.Errorf("binding %q: %w", code, err)
}
}
return existing, nil
}
// validateMenuSpec rejects the malformed input tools/checksilent's
// menu-sort-overflow and Kind-adjacent checks would catch for an in-tree
// seed but cannot for a third-party application's - see menuSortRange's doc
@@ -278,7 +426,7 @@ func pascalCase(s string) string {
// framework migration sorts before every app-prefixed one - means that
// should not happen in practice, but failing this call over it would be
// worse than a menu with no grant yet.
func grantToAdminRole(tx *gorm.DB, menuIDs []int, apiRows map[string]models.SysApi) error {
func grantToAdminRole(tx *gorm.DB, appCode string, menuIDs []int, apiRows map[string]models.SysApi) error {
var role models.SysRole
if err := tx.Where("role_key = ?", adminRoleKey).First(&role).Error; err != nil {
if errors.Is(err, gorm.ErrRecordNotFound) {
@@ -297,12 +445,177 @@ func grantToAdminRole(tx *gorm.DB, menuIDs []int, apiRows map[string]models.SysA
}
for _, a := range apiRows {
if err := tx.Exec(
res := tx.Exec(
"INSERT INTO casbin_rule (ptype, v0, v1, v2, v3, v4, v5) SELECT 'p', ?, ?, ?, '', '', '' WHERE NOT EXISTS (SELECT 1 FROM casbin_rule WHERE ptype='p' AND v0=? AND v1=? AND v2=?)",
role.RoleKey, a.Path, a.Action, role.RoleKey, a.Path, a.Action,
).Error; err != nil {
)
if res.Error != nil {
return res.Error
}
if res.RowsAffected == 0 {
// The policy was already there, so this install did not create
// it and it is not this app's to take away. Leaving it out of
// the ledger is what makes an uninstall report it instead of
// deleting it.
//
// The two ways this can be wrong are not equally bad, which is
// what settles it. Under-recording leaves a policy behind and
// the uninstall says so, because a policy naming an app's own
// path with no ledger entry is exactly what it lists as an
// orphan. Over-recording deletes a grant somebody else made,
// silently. Between a visible leftover and an invisible
// deletion of somebody's authorization, take the leftover.
continue
}
if err := recordGrant(tx, appCode, role.RoleKey, a.Path, a.Action); err != nil {
return err
}
}
return nil
}
// recordGrant writes down that this application's install created one casbin
// policy, keyed by the same tuple casbin_rule is unique on.
//
// A ledger rather than a column on casbin_rule, because casbin_rule is not
// this project's table: the gorm adapter's SavePolicy truncates it and writes
// it back from memory, which would drop any column added here, and
// SysRole.Update replaces a role's policy rows wholesale. The tuple survives
// both, because both rebuild it from the same sys_menu/sys_api data.
//
// Written with the same INSERT ... WHERE NOT EXISTS shape as the policy above
// rather than a plain insert: the ledger's unique index covers the tuple
// alone, so a duplicate would abort the whole seed instead of being the
// no-op it should be.
func recordGrant(tx *gorm.DB, appCode, roleKey, path, action string) error {
return tx.Exec(
"INSERT INTO sys_app_casbin_grant (app_code, ptype, v0, v1, v2, v3, v4, v5, created_at) "+
"SELECT ?, 'p', ?, ?, ?, '', '', '', ? WHERE NOT EXISTS "+
"(SELECT 1 FROM sys_app_casbin_grant WHERE ptype='p' AND v0=? AND v1=? AND v2=? AND v3='' AND v4='' AND v5='')",
appCode, roleKey, path, action, time.Now(), roleKey, path, action,
).Error
}
// specMenuFields are the sys_menu columns a MenuSpec decides, and the only
// ones a reseed rewrites on a row that is already there.
//
// Visible and IsFrame are not in the list. They are seeding defaults the
// application never expressed, so an administrator who hid a seeded menu
// keeps it hidden. app_code and seed_code are not either: they are the
// natural key the row was found by, and writing them back would be writing
// what was just matched.
var specMenuFields = []string{
"MenuName", "Title", "Icon", "Path", "MenuType",
"Permission", "ParentId", "Component", "Sort",
}
// menuRowFor is the row a MenuSpec describes. One definition, so the insert
// path and the repair path cannot drift into disagreeing about what a spec
// decides.
func menuRowFor(appCode string, s seed.MenuSpec, parentRow models.SysMenu) models.SysMenu {
seedCode := s.Code
return models.SysMenu{
MenuName: menuName(appCode, s.Code),
Title: s.Title,
Icon: s.Icon,
Path: s.Path,
MenuType: s.Kind,
Permission: s.Permission,
ParentId: parentRow.MenuId,
Component: s.Component,
Sort: s.Sort,
// Visible "0" is shown, not hidden - the same defaults
// 1786700001000_demo_menu.go seeds its own menu with. A freshly
// installed application's menu should not need an administrator to
// first find and unhide it. Only written when the row is created;
// see specMenuFields.
Visible: "0",
IsFrame: "1",
AppCode: appCode,
SeedCode: &seedCode,
}
}
// repairPaths writes row.Paths, and moves whatever is underneath it.
//
// The subtree matters because it is not all in this call's specs: a menu an
// administrator added under a seeded one keeps the old prefix, and nothing
// else in the codebase would ever rewrite it. SysMenu.Update does the same
// cascade for the same column when somebody moves a menu by hand.
//
// The predicate is the row itself or a row strictly under it, rather than
// `paths LIKE old || '%'`, which also matches /0/10 when old is /0/1.
func repairPaths(tx *gorm.DB, row *models.SysMenu, s seed.MenuSpec, parentRow models.SysMenu) error {
want := expectedPaths(row.MenuId, s.Parent, parentRow)
old := row.Paths
if old == want {
return nil
}
if old == "" {
// A row whose paths was never written - an interrupted create. It
// has no subtree to speak of, and `LIKE '/%'` would match the whole
// table.
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", row.MenuId).
Update("paths", want).Error; err != nil {
return fmt.Errorf("writing paths: %w", err)
}
row.Paths = want
return nil
}
var subtree []models.SysMenu
if err := tx.Where("paths = ? OR paths LIKE ?", old, old+"/%").Find(&subtree).Error; err != nil {
return fmt.Errorf("reading the subtree under %s: %w", old, err)
}
for _, d := range subtree {
moved := want + strings.TrimPrefix(d.Paths, old)
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", d.MenuId).
Update("paths", moved).Error; err != nil {
return fmt.Errorf("moving %d from %s to %s: %w", d.MenuId, d.Paths, moved, err)
}
}
row.Paths = want
return nil
}
// adoptLegacyMenu claims a row this application wrote before sys_menu had a
// seed_code column, so a reseed repairs it instead of inserting a second copy
// beside it.
//
// 1786700008000 added the column and left it NULL on every row already there,
// which is right for the host's own hand-placed menus - there is nothing to
// derive one from. An application's rows are in that population too, and for
// those the value is derivable, because menu_name is what identified them
// before the column existed. Without this the natural-key lookup misses them,
// the seed inserts a duplicate, and the unique index cannot object: NULL
// never collides.
//
// Ambiguity is refused rather than guessed. menuName concatenates two
// pascalCase strings and pascalCase is not injective, so two specs can land
// on one name; picking one of several rows would attach an application's
// menu to whichever the database returned first.
func adoptLegacyMenu(tx *gorm.DB, appCode string, s seed.MenuSpec) (models.SysMenu, bool, error) {
name := menuName(appCode, s.Code)
var rows []models.SysMenu
if err := tx.Where("app_code = ? AND menu_name = ? AND seed_code IS NULL", appCode, name).
Find(&rows).Error; err != nil {
return models.SysMenu{}, false, fmt.Errorf("looking for a row written before seed_code existed: %w", err)
}
switch len(rows) {
case 0:
return models.SysMenu{}, false, nil
case 1:
default:
return models.SysMenu{}, false, fmt.Errorf(
"%d rows carry menu_name %q with no seed_code; which of them belongs to %q cannot be decided here, because menuName is not reversible - reconcile them by hand",
len(rows), name, s.Code)
}
seedCode := s.Code
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", rows[0].MenuId).
Update("seed_code", seedCode).Error; err != nil {
return models.SysMenu{}, false, fmt.Errorf("claiming the row written before seed_code existed: %w", err)
}
rows[0].SeedCode = &seedCode
return rows[0], true, nil
}
+963 -1
View File
@@ -1,13 +1,18 @@
package service
import (
"context"
"errors"
"fmt"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
gormlogger "gorm.io/gorm/logger"
contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/seed"
@@ -28,7 +33,14 @@ func newSeedTestDB(t *testing.T) *gorm.DB {
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&models.SysMenu{}, &models.SysApi{}, &models.SysRole{}); err != nil {
// sys_app_casbin_grant is where grantToAdminRole records which policies
// this install created, so an uninstall can tell them from the ones
// somebody granted by hand. In a real database it is created by
// 1786700007000, which is a framework migration and therefore runs ahead
// of every application's - version strings sort bare digits before any
// app-prefixed one.
if err := db.AutoMigrate(&models.SysMenu{}, &models.SysApi{}, &models.SysRole{},
&models.SysAppCasbinGrant{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
if err := db.Exec(`CREATE TABLE casbin_rule (
@@ -289,3 +301,953 @@ func TestSeedMenusWithNothingRegisteredWritesNothing(t *testing.T) {
}
}
}
// newSeedTestDB's AutoMigrate builds a unique index on seed_code alone,
// because SysMenu.SeedCode is the only field in the struct carrying the
// uk_sys_menu_app_seed_code_del tag - app_code already carries a different,
// non-unique index name of its own, and the embedded ModelTime's
// DeletedAt (aliased from go-admin-core) cannot be given a third one. The
// real migration (cmd/migrate/migration/version/1786700008000_seed_natural_keys.go)
// never lets AutoMigrate touch this table for exactly that reason: it
// builds the composite (app_code, seed_code, deleted_at) index by hand
// instead. Reproduce that by hand here too, so a test that seeds two rows
// sharing a seed_code under different deleted_at values sees what a real
// install would, not gorm's narrower default.
func useCompositeSeedCodeIndex(t *testing.T, db *gorm.DB) {
t.Helper()
if db.Migrator().HasIndex(&models.SysMenu{}, "uk_sys_menu_app_seed_code_del") {
if err := db.Migrator().DropIndex(&models.SysMenu{}, "uk_sys_menu_app_seed_code_del"); err != nil {
t.Fatalf("drop the single-column seed_code index: %v", err)
}
}
if err := db.Exec(
"CREATE UNIQUE INDEX uk_sys_menu_app_seed_code_del ON sys_menu (app_code, seed_code, deleted_at)",
).Error; err != nil {
t.Fatalf("create the composite seed_code index: %v", err)
}
}
// A retried migration - one that failed partway through and is run again,
// or simply run twice by mistake - must not create a second sys_api or
// sys_menu row for the same (appCode, natural key). This is the defect the
// demo site hit in production: duplicate sys_menu/casbin_rule rows from a
// bare tx.Create on a natural key nothing was checking.
func TestSeedMenusIsIdempotentAcrossARetry(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{
{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10},
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: "Orders", Sort: 1, ApiCodes: []string{"list"}},
}
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"},
}
run := func() {
t.Helper()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
}
run()
firstMenuIDs := allMenuIDs(t, db, "order")
firstApiIDs := allApiIDs(t, db, "order")
run() // the retry
if got := allMenuIDs(t, db, "order"); !sameIDs(got, firstMenuIDs) {
t.Errorf("sys_menu ids after retry = %v, want unchanged %v (a second call inserted new rows)", got, firstMenuIDs)
}
if got := allApiIDs(t, db, "order"); !sameIDs(got, firstApiIDs) {
t.Errorf("sys_api ids after retry = %v, want unchanged %v (a second call inserted new rows)", got, firstApiIDs)
}
assertRowCount(t, db, "sys_api", 1)
assertRowCount(t, db, "sys_menu", 2)
assertRowCount(t, db, "sys_menu_api_rule", 1)
assertRowCount(t, db, "sys_role_menu", 2)
assertRowCount(t, db, "casbin_rule", 1)
}
// Only a live row counts as "already written". A row a prior, unrelated
// soft-delete already retired must not be reused - seedApis/seedMenuTree
// have to insert a fresh one under the same natural key, the same way the
// unique indexes 1786700008000_seed_natural_keys.go builds only bind live
// rows.
func TestSeedMenusOnlyReusesLiveRows(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10}}
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
// Soft-delete both rows this first call wrote, as if an operator (or an
// earlier uninstall) had retired them, independently of this migration
// ever running again.
if err := db.Exec("UPDATE sys_menu SET deleted_at = 1").Error; err != nil {
t.Fatalf("soft-delete sys_menu: %v", err)
}
if err := db.Exec("UPDATE sys_api SET deleted_at = 1").Error; err != nil {
t.Fatalf("soft-delete sys_api: %v", err)
}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus after soft-delete: %v", err)
}
// Two rows total: the soft-deleted original, plus a fresh one - not the
// dead row resurrected in place, and not left with zero live rows.
assertRowCount(t, db, "sys_menu", 2)
assertRowCount(t, db, "sys_api", 2)
var liveMenus, liveApis int64
db.Model(&models.SysMenu{}).Where("app_code = ?", "order").Count(&liveMenus)
db.Model(&models.SysApi{}).Where("app_code = ?", "order").Count(&liveApis)
if liveMenus != 1 {
t.Errorf("live sys_menu rows = %d, want 1", liveMenus)
}
if liveApis != 1 {
t.Errorf("live sys_api rows = %d, want 1", liveApis)
}
}
// app_code is part of the natural key, not a descriptive column alongside
// it. Two applications that happen to register an identical (path, action)
// or seed_code must each get their own row - reusing one app's row for
// another's install would make an uninstall of the first delete a row the
// second considers its own.
func TestSeedMenusScopesTheNaturalKeyByAppCode(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus := []seed.MenuSpec{{Code: "dir", Kind: contractmodels.Directory, Title: "Dir", Sort: 10}}
apis := []seed.ApiSpec{{Code: "list", Title: "Shared endpoint", Path: "/api/v1/shared", Method: "GET"}}
for _, appCode := range []string{"order", "billing"} {
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, appCode, menus, apis)
}); err != nil {
t.Fatalf("SeedMenus(%q): %v", appCode, err)
}
}
var apiRows []models.SysApi
if err := db.Where("path = ? AND action = ?", "/api/v1/shared", "GET").
Order("app_code").Find(&apiRows).Error; err != nil {
t.Fatalf("read sys_api: %v", err)
}
if len(apiRows) != 2 {
t.Fatalf("sys_api has %d row(s) for the shared (path, action), want 2 - one per app", len(apiRows))
}
if apiRows[0].AppCode != "billing" || apiRows[1].AppCode != "order" {
t.Errorf("sys_api app_codes = [%s %s], want [billing order]", apiRows[0].AppCode, apiRows[1].AppCode)
}
var menuRows []models.SysMenu
if err := db.Where("seed_code = ?", "dir").Order("app_code").Find(&menuRows).Error; err != nil {
t.Fatalf("read sys_menu: %v", err)
}
if len(menuRows) != 2 {
t.Fatalf("sys_menu has %d row(s) for the shared seed_code, want 2 - one per app", len(menuRows))
}
if menuRows[0].AppCode != "billing" || menuRows[1].AppCode != "order" {
t.Errorf("sys_menu app_codes = [%s %s], want [billing order]", menuRows[0].AppCode, menuRows[1].AppCode)
}
}
func allMenuIDs(t *testing.T, db *gorm.DB, appCode string) []int {
t.Helper()
var rows []models.SysMenu
if err := db.Where("app_code = ?", appCode).Order("menu_id").Find(&rows).Error; err != nil {
t.Fatalf("read sys_menu: %v", err)
}
ids := make([]int, len(rows))
for i, r := range rows {
ids[i] = r.MenuId
}
return ids
}
func allApiIDs(t *testing.T, db *gorm.DB, appCode string) []int {
t.Helper()
var rows []models.SysApi
if err := db.Where("app_code = ?", appCode).Order("id").Find(&rows).Error; err != nil {
t.Fatalf("read sys_api: %v", err)
}
ids := make([]int, len(rows))
for i, r := range rows {
ids[i] = r.Id
}
return ids
}
func sameIDs(a, b []int) bool {
if len(a) != len(b) {
return false
}
for i := range a {
if a[i] != b[i] {
return false
}
}
return true
}
func assertRowCount(t *testing.T, db *gorm.DB, table string, want int64) {
t.Helper()
var n int64
if err := db.Table(table).Count(&n).Error; err != nil {
t.Fatalf("count %s: %v", table, err)
}
if n != want {
t.Errorf("%s has %d row(s), want %d", table, n, want)
}
}
// A retried migration does not just risk inserting a second copy of a row
// it already wrote (that gap is closed above) - the reuse path itself has
// to leave the row in the same state a fresh insert would have. Before
// this defect was fixed, the reuse branch (seed.go's "case err == nil")
// stopped at reusing the row's id and skipped everything a fresh insert
// does afterwards: the sys_menu_api_rule binding gorm's association save
// writes as part of Create, and the paths UPDATE that follows Create as a
// separate statement. A row a prior attempt inserted but did not finish -
// exactly the shape design doc §1.5 says a non-transactional retry can
// leave behind - would be "found" and then left broken forever, with the
// migration reporting success.
//
// existingHalfWrittenMenu inserts a sys_menu row the way seedMenuTree's own
// tx.Create leaves one when interrupted immediately afterwards: the row
// exists with its natural key, but paths was never computed and no
// sys_menu_api_rule binding was ever written for it - Create's association
// save and the paths UPDATE are each a separate statement from the row
// insert itself.
func existingHalfWrittenMenu(t *testing.T, db *gorm.DB, appCode, seedCode string, parentID int) models.SysMenu {
t.Helper()
code := seedCode
row := models.SysMenu{
MenuName: menuName(appCode, seedCode),
AppCode: appCode,
SeedCode: &code,
ParentId: parentID,
Visible: "0",
IsFrame: "1",
// Paths deliberately left "" - never computed, the same as a row
// whose Create succeeded but whose follow-up paths UPDATE never ran.
}
if err := db.Create(&row).Error; err != nil {
t.Fatalf("seed half-written menu %q: %v", seedCode, err)
}
return row
}
func bindingCount(t *testing.T, db *gorm.DB, menuID, apiID int) int64 {
t.Helper()
var n int64
if err := db.Table("sys_menu_api_rule").
Where("sys_menu_menu_id = ? AND sys_api_id = ?", menuID, apiID).Count(&n).Error; err != nil {
t.Fatalf("count sys_menu_api_rule: %v", err)
}
return n
}
// TestSeedMenusRepairsAnIncompleteExistingRow is the reproduction case:
// both paths and the api binding are missing on the row seedMenuTree finds
// through its idempotency check, the shape a real interrupted retry leaves
// behind. Run against the unfixed reuse branch, this must fail - that is
// what proves the defect is real rather than a three-way guess.
func TestSeedMenusRepairsAnIncompleteExistingRow(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
dir := existingHalfWrittenMenu(t, db, "order", "dir", 0)
list := existingHalfWrittenMenu(t, db, "order", "list", dir.MenuId)
menus := []seed.MenuSpec{
{Code: "dir", Kind: contractmodels.Directory, Title: "Order", Sort: 10},
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: "Orders", Sort: 1, ApiCodes: []string{"list"}},
}
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
wantDirPaths := "/0/" + strconv.Itoa(dir.MenuId)
wantListPaths := wantDirPaths + "/" + strconv.Itoa(list.MenuId)
var gotDir, gotList models.SysMenu
if err := db.First(&gotDir, dir.MenuId).Error; err != nil {
t.Fatalf("read dir: %v", err)
}
if err := db.First(&gotList, list.MenuId).Error; err != nil {
t.Fatalf("read list: %v", err)
}
if gotDir.Paths != wantDirPaths {
t.Errorf("dir.Paths = %q, want %q - a retried install left a root menu with no materialized path", gotDir.Paths, wantDirPaths)
}
if gotList.Paths != wantListPaths {
t.Errorf("list.Paths = %q, want %q - a retried install left the seeded subtree with a broken materialized path", gotList.Paths, wantListPaths)
}
if n := bindingCount(t, db, list.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("sys_menu_api_rule binding count for list = %d, want 1 - a retried install left the menu with its api granted to no one", n)
}
}
// soloMenuSpec is a single, parent-less menu with one api binding - the
// smallest shape that can exhibit "paths wrong" and "binding missing"
// independently of each other, used by the three tests below to isolate
// one repair path at a time from TestSeedMenusRepairsAnIncompleteExistingRow's
// combined (both broken) case.
func soloMenuSpec() []seed.MenuSpec {
return []seed.MenuSpec{{Code: "solo", Kind: contractmodels.Menu, Title: "Solo", Sort: 1, ApiCodes: []string{"list"}}}
}
// Only the binding is missing; paths is already correct. The repair must
// add the binding and must not touch the already-correct paths value.
func TestSeedMenusRepairsOnlyAMissingBinding(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
solo := existingHalfWrittenMenu(t, db, "order", "solo", 0)
wantPaths := "/0/" + strconv.Itoa(solo.MenuId)
if err := db.Model(&models.SysMenu{}).Where("menu_id = ?", solo.MenuId).
Update("paths", wantPaths).Error; err != nil {
t.Fatalf("set paths: %v", err)
}
// The binding is deliberately left unwritten.
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", soloMenuSpec(), apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
var got models.SysMenu
if err := db.First(&got, solo.MenuId).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
if got.Paths != wantPaths {
t.Errorf("paths changed from %q to %q; repairing a missing binding must not touch an already-correct path", wantPaths, got.Paths)
}
if n := bindingCount(t, db, solo.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("binding count = %d, want 1", n)
}
}
// Only paths is missing; the binding already exists (as if Create's own
// association write had succeeded but the paths UPDATE that follows it
// never ran). The repair must fix paths and must not duplicate the
// already-correct binding.
func TestSeedMenusRepairsOnlyMissingPaths(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
apiRows, err := seedApis(db, "order", apis)
if err != nil {
t.Fatalf("seedApis: %v", err)
}
solo := existingHalfWrittenMenu(t, db, "order", "solo", 0)
if err := db.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) VALUES (?, ?)",
solo.MenuId, apiRows["list"].Id,
).Error; err != nil {
t.Fatalf("seed binding: %v", err)
}
// solo.Paths is deliberately left "" by existingHalfWrittenMenu.
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", soloMenuSpec(), apis)
}); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
wantPaths := "/0/" + strconv.Itoa(solo.MenuId)
var got models.SysMenu
if err := db.First(&got, solo.MenuId).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
if got.Paths != wantPaths {
t.Errorf("paths = %q, want %q", got.Paths, wantPaths)
}
if n := bindingCount(t, db, solo.MenuId, apiRows["list"].Id); n != 1 {
t.Errorf("binding count = %d, want 1 - repairing paths must not duplicate an already-correct binding", n)
}
}
// capturingLogger records every SQL statement gorm actually executes, so a
// test can assert that a fully-consistent retry performs no write at all -
// not just that its net effect happens to be zero rows changed. Mirrors
// common/actions/crud_shim_test.go's logger of the same name and shape;
// duplicated locally rather than exported and shared, matching how small
// gorm-facing test doubles are kept next to the test that needs them
// elsewhere in this repository.
type capturingLogger struct {
gormlogger.Interface
mu sync.Mutex
stmts []string
}
func (l *capturingLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
sql, _ := fc()
l.mu.Lock()
l.stmts = append(l.stmts, sql)
l.mu.Unlock()
}
func (l *capturingLogger) all() string {
l.mu.Lock()
defer l.mu.Unlock()
return strings.Join(l.stmts, "\n")
}
// Both paths and the binding are already correct - the ordinary shape of
// every retry after the first one succeeds in full. Repairing an
// already-consistent row must not touch it: paths is read-before-write and
// so must not be UPDATEd at all (asserted directly, by statement, since the
// code gates that call behind a value comparison); the binding's own
// insert is guarded by WHERE NOT EXISTS the same way grantToAdminRole's
// already are, so its row count staying put is the meaningful claim - the
// guarded statement itself may still be sent, the same way it already is
// for sys_role_menu/casbin_rule.
func TestSeedMenusFullyConsistentRowCausesNoPathsUpdate(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
menus := soloMenuSpec()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (first): %v", err)
}
var apiRows []models.SysApi
db.Where("app_code = ?", "order").Find(&apiRows)
var soloRow models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "solo").First(&soloRow).Error; err != nil {
t.Fatalf("read solo after first call: %v", err)
}
if soloRow.Paths == "" {
t.Fatalf("solo.Paths is empty after the first call; the fixture itself is broken, not what this test means to check")
}
wantBindings := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id)
if wantBindings != 1 {
t.Fatalf("binding count after the first call = %d, want 1; the fixture itself is broken", wantBindings)
}
capturing := &capturingLogger{Interface: gormlogger.Default.LogMode(gormlogger.Info)}
captured := db.Session(&gorm.Session{Logger: capturing})
if err := captured.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (retry): %v", err)
}
all := strings.ToUpper(capturing.all())
if strings.Contains(all, "UPDATE") && strings.Contains(all, "SYS_MENU") && strings.Contains(all, "PATHS") {
t.Errorf("a fully consistent retry executed a paths UPDATE against sys_menu:\n%s", capturing.all())
}
if got := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id); got != 1 {
t.Errorf("binding count after the retry = %d, want 1 (unchanged)", got)
}
}
// An administrator can bind a menu to an additional api by hand through
// the menu management UI - a sys_menu_api_rule row for an api never in
// s.ApiCodes at all. A retried SeedMenus call must not touch it: deleting
// every binding for the menu and reinserting only what s.ApiCodes lists
// would wipe it out silently, the same shape as sys_role_menu/casbin_rule
// getting zeroed by SysRole.Update's FullSaveAssociations save - just with
// this code as the actor instead of the victim this time.
func TestSeedMenusPreservesAHandAddedBinding(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
apis := []seed.ApiSpec{{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET"}}
menus := soloMenuSpec()
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (first): %v", err)
}
var soloRow models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "solo").First(&soloRow).Error; err != nil {
t.Fatalf("read solo: %v", err)
}
// An api this call's ApiSpec list never mentions - standing in for one
// belonging to some other feature entirely, bound to this menu by an
// administrator, not by any SeedMenus call.
handAdded := models.SysApi{Path: "/api/v1/order/export", Action: "GET", Type: "SYS", AppCode: "order"}
if err := db.Create(&handAdded).Error; err != nil {
t.Fatalf("seed the hand-added api: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu_api_rule (sys_menu_menu_id, sys_api_id) VALUES (?, ?)",
soloRow.MenuId, handAdded.Id,
).Error; err != nil {
t.Fatalf("seed the hand-added binding: %v", err)
}
// A retry with the exact same specs - solo's ApiCodes still names only
// "list".
if err := db.Transaction(func(tx *gorm.DB) error {
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
}); err != nil {
t.Fatalf("SeedMenus (retry): %v", err)
}
if n := bindingCount(t, db, soloRow.MenuId, handAdded.Id); n != 1 {
t.Errorf("hand-added binding count = %d, want 1 - a retry silently deleted a binding it does not own", n)
}
var apiRows []models.SysApi
db.Where("app_code = ? AND path = ?", "order", "/api/v1/order").Find(&apiRows)
if len(apiRows) != 1 {
t.Fatalf("seeded api not found as expected: %+v", apiRows)
}
if n := bindingCount(t, db, soloRow.MenuId, apiRows[0].Id); n != 1 {
t.Errorf("the seed's own binding count = %d, want 1 - it must survive the retry too", n)
}
}
// Every policy grantToAdminRole creates has to be written down, or an
// uninstall has no way to tell this app's grants from a hand-made one and
// leaves all of them behind.
func TestSeedMenusRecordsTheGrantsItCreated(t *testing.T) {
db := newSeedTestDB(t)
role := seedAdminRole(t, db)
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET", Handle: "apis.Order.GetPage-fm"},
{Code: "create", Title: "Create order", Path: "/api/v1/order", Method: "POST", Handle: "apis.Order.Insert-fm"},
}
if err := (adminSeeder{}).SeedMenus(db, "order", nil, apis); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
for _, a := range apis {
var n int64
db.Model(&models.SysAppCasbinGrant{}).
Where("app_code = ? AND ptype = 'p' AND v0 = ? AND v1 = ? AND v2 = ?",
"order", role.RoleKey, a.Path, a.Method).
Count(&n)
if n != 1 {
t.Errorf("ledger rows for %s %s = %d, want 1", a.Method, a.Path, n)
}
}
}
// A policy that was already there was not created by this install, so it is
// not this app's to take away later. Recording it would mean an uninstall
// deletes a grant somebody else made, and deletes it silently - the opposite
// mistake leaves a policy behind, which the uninstall reports.
func TestSeedMenusDoesNotClaimAPolicyItDidNotCreate(t *testing.T) {
db := newSeedTestDB(t)
role := seedAdminRole(t, db)
if err := db.Exec(
"INSERT INTO casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', ?, '/api/v1/order', 'GET', '', '', '')",
role.RoleKey,
).Error; err != nil {
t.Fatalf("pre-existing policy: %v", err)
}
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET", Handle: "apis.Order.GetPage-fm"},
{Code: "create", Title: "Create order", Path: "/api/v1/order", Method: "POST", Handle: "apis.Order.Insert-fm"},
}
if err := (adminSeeder{}).SeedMenus(db, "order", nil, apis); err != nil {
t.Fatalf("SeedMenus: %v", err)
}
var claimed int64
db.Model(&models.SysAppCasbinGrant{}).
Where("v1 = ? AND v2 = ?", "/api/v1/order", "GET").Count(&claimed)
if claimed != 0 {
t.Errorf("the ledger claimed a policy that was already there (%d rows)", claimed)
}
// The one it did create is still recorded: the skip is per policy, not
// for the whole call.
var created int64
db.Model(&models.SysAppCasbinGrant{}).
Where("v1 = ? AND v2 = ?", "/api/v1/order", "POST").Count(&created)
if created != 1 {
t.Errorf("ledger rows for the policy it did create = %d, want 1", created)
}
// And the pre-existing policy itself is untouched.
var policies int64
db.Table("casbin_rule").Where("v1 = ? AND v2 = ?", "/api/v1/order", "GET").Count(&policies)
if policies != 1 {
t.Errorf("casbin_rule rows = %d, want the one that was already there", policies)
}
}
// A migration that failed partway is re-run whole. The ledger must come out
// of a second run the same as the first, not with a duplicate or an error
// from its own unique index.
func TestSeedMenusLedgerSurvivesARetry(t *testing.T) {
db := newSeedTestDB(t)
seedAdminRole(t, db)
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET", Handle: "apis.Order.GetPage-fm"},
}
for i := 0; i < 2; i++ {
if err := (adminSeeder{}).SeedMenus(db, "order", nil, apis); err != nil {
t.Fatalf("SeedMenus run %d: %v", i+1, err)
}
}
var n int64
db.Model(&models.SysAppCasbinGrant{}).Count(&n)
if n != 1 {
t.Errorf("ledger has %d rows after two runs, want 1", n)
}
}
// The ledger's own guard against a duplicate, which the plain retry above
// never reaches: there the policy still exists, so the insert is skipped
// before the ledger is touched at all. This is the case that does reach it -
// the policy row was removed while its ledger entry stayed, so the seed
// creates the policy again and writes a ledger entry that is already there.
// A plain insert would abort the whole seed on the ledger's unique index.
func TestSeedMenusLedgerToleratesAnEntryWhosePolicyWasRemoved(t *testing.T) {
db := newSeedTestDB(t)
seedAdminRole(t, db)
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET", Handle: "apis.Order.GetPage-fm"},
}
if err := (adminSeeder{}).SeedMenus(db, "order", nil, apis); err != nil {
t.Fatalf("first run: %v", err)
}
if err := db.Exec("DELETE FROM casbin_rule WHERE v1 = ? AND v2 = ?", "/api/v1/order", "GET").Error; err != nil {
t.Fatalf("removing the policy: %v", err)
}
var ledger int64
db.Model(&models.SysAppCasbinGrant{}).Count(&ledger)
if ledger != 1 {
t.Fatalf("the ledger entry is gone, so this test is not set up: %d rows", ledger)
}
if err := (adminSeeder{}).SeedMenus(db, "order", nil, apis); err != nil {
t.Fatalf("second run: %v", err)
}
db.Model(&models.SysAppCasbinGrant{}).Count(&ledger)
if ledger != 1 {
t.Errorf("ledger has %d rows, want 1", ledger)
}
var policies int64
db.Table("casbin_rule").Where("v1 = ? AND v2 = ?", "/api/v1/order", "GET").Count(&policies)
if policies != 1 {
t.Errorf("the policy was not put back: %d rows", policies)
}
}
// The tree is built from parent_id, not from paths. A menu whose parent was
// removed and written again kept parent_id on the dead row while its paths
// named the new one, so the menu was gone from the sidebar and the migration
// said it had succeeded.
func TestSeedMenusRepairsParentIdAfterTheParentWasRemoved(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus, apis := orderMenuSpecs("Orders", "apps/order/index")
if err := (adminSeeder{}).SeedMenus(db, "order", menus, apis); err != nil {
t.Fatalf("first seed: %v", err)
}
var dir models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "dir").First(&dir).Error; err != nil {
t.Fatal(err)
}
if err := db.Delete(&models.SysMenu{}, "menu_id = ?", dir.MenuId).Error; err != nil {
t.Fatalf("removing the parent: %v", err)
}
if err := (adminSeeder{}).SeedMenus(db, "order", menus, apis); err != nil {
t.Fatalf("second seed: %v", err)
}
var newDir, list models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "dir").First(&newDir).Error; err != nil {
t.Fatal(err)
}
if err := db.Where("app_code = ? AND seed_code = ?", "order", "list").First(&list).Error; err != nil {
t.Fatal(err)
}
if newDir.MenuId == dir.MenuId {
t.Fatal("the removed parent was reused, so this test proves nothing")
}
if list.ParentId != newDir.MenuId {
t.Errorf("parent_id = %d, want the new parent %d; the menu hangs off a row that is gone",
list.ParentId, newDir.MenuId)
}
if want := newDir.Paths + "/" + strconv.Itoa(list.MenuId); list.Paths != want {
t.Errorf("paths = %q, want %q", list.Paths, want)
}
}
// A menu somebody added under a seeded one is not in any spec, so nothing but
// this would ever rewrite its path when its ancestor moves.
func TestSeedMenusMovesTheSubtreeUnderARepairedMenu(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus, apis := orderMenuSpecs("Orders", "apps/order/index")
if err := (adminSeeder{}).SeedMenus(db, "order", menus, apis); err != nil {
t.Fatalf("first seed: %v", err)
}
var dir, list models.SysMenu
db.Where("app_code = ? AND seed_code = ?", "order", "dir").First(&dir)
db.Where("app_code = ? AND seed_code = ?", "order", "list").First(&list)
// By hand, under the seeded menu, the way an administrator would.
hand := models.SysMenu{MenuName: "HandMade", Title: "By hand", MenuType: contractmodels.Menu,
ParentId: list.MenuId}
if err := db.Create(&hand).Error; err != nil {
t.Fatal(err)
}
hand.Paths = list.Paths + "/" + strconv.Itoa(hand.MenuId)
db.Model(&models.SysMenu{}).Where("menu_id = ?", hand.MenuId).Update("paths", hand.Paths)
// Rows whose paths start with the moving one's as a string and are not
// underneath it as a path. /0/1/2 is a string prefix of /0/1/20, and a
// LIKE on the bare prefix cannot tell the two apart - so these have to
// be built against the path that actually moves, which is the one this
// repair rewrites.
var decoys []models.SysMenu
for _, suffix := range []string{"0", "1", "9"} {
d := models.SysMenu{MenuName: "Decoy" + suffix, Title: "decoy", MenuType: contractmodels.Menu}
if err := db.Create(&d).Error; err != nil {
t.Fatal(err)
}
d.Paths = list.Paths + suffix
db.Model(&models.SysMenu{}).Where("menu_id = ?", d.MenuId).Update("paths", d.Paths)
decoys = append(decoys, d)
}
if err := db.Delete(&models.SysMenu{}, "menu_id = ?", dir.MenuId).Error; err != nil {
t.Fatal(err)
}
if err := (adminSeeder{}).SeedMenus(db, "order", menus, apis); err != nil {
t.Fatalf("second seed: %v", err)
}
var newList, movedHand models.SysMenu
db.Where("app_code = ? AND seed_code = ?", "order", "list").First(&newList)
db.Where("menu_id = ?", hand.MenuId).First(&movedHand)
if want := newList.Paths + "/" + strconv.Itoa(hand.MenuId); movedHand.Paths != want {
t.Errorf("the hand-made menu's paths = %q, want %q; it no longer names its ancestors",
movedHand.Paths, want)
}
for _, d := range decoys {
var after models.SysMenu
db.Where("menu_id = ?", d.MenuId).First(&after)
if after.Paths != d.Paths {
t.Errorf("decoy %d moved from %q to %q; a prefix match caught a row that is not underneath",
d.MenuId, d.Paths, after.Paths)
}
}
}
// An application that renames a menu or moves its component in a new version
// had the change ignored: the row was found and returned untouched.
func TestSeedMenusRefreshesWhatTheSpecDecides(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
menus, apis := orderMenuSpecs("Orders", "apps/order/index")
if err := (adminSeeder{}).SeedMenus(db, "order", menus, apis); err != nil {
t.Fatalf("first seed: %v", err)
}
// An administrator hides it. That is not something the spec expresses,
// so a reseed has no business turning it back on.
if err := db.Model(&models.SysMenu{}).Where("app_code = ? AND seed_code = ?", "order", "list").
Update("visible", "1").Error; err != nil {
t.Fatal(err)
}
menus2, apis2 := orderMenuSpecs("Sales orders", "apps/order/list/index")
if err := (adminSeeder{}).SeedMenus(db, "order", menus2, apis2); err != nil {
t.Fatalf("second seed: %v", err)
}
var list models.SysMenu
db.Where("app_code = ? AND seed_code = ?", "order", "list").First(&list)
if list.Title != "Sales orders" {
t.Errorf("title = %q, want the new one", list.Title)
}
if list.Component != "apps/order/list/index" {
t.Errorf("component = %q, want the new one", list.Component)
}
if list.Visible != "1" {
t.Errorf("visible = %q; a reseed unhid a menu an administrator had hidden", list.Visible)
}
}
// orderMenuSpecs is a two-level tree plus one api, parameterised on the two
// columns the upgrade test changes.
func orderMenuSpecs(title, component string) ([]seed.MenuSpec, []seed.ApiSpec) {
menus := []seed.MenuSpec{
{Code: "dir", Kind: contractmodels.Directory, Title: "Order Example", Path: "/apps/order", Component: "Layout", Sort: 10},
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: title, Path: "list", Component: component, Sort: 1, ApiCodes: []string{"list"}},
}
apis := []seed.ApiSpec{
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET", Handle: "apis.Order.GetPage-fm"},
}
return menus, apis
}
// 1786700008000 added seed_code and left it NULL on every row already there.
// An application's rows are in that population, and the natural-key lookup
// misses them, so the seed used to insert a second copy beside each one -
// which the unique index cannot object to, because NULL never collides.
func TestSeedMenusAdoptsARowWrittenBeforeSeedCodeExisted(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
// What an older SeedMenus left: app_code set, seed_code absent, and the
// name that identified it then.
legacy := models.SysMenu{
MenuName: menuName("order", "dir"), AppCode: "order", Title: "the old title",
MenuType: contractmodels.Directory, Path: "/apps/order", Component: "Layout", Sort: 10,
}
if err := db.Create(&legacy).Error; err != nil {
t.Fatal(err)
}
menus, apis := orderMenuSpecs("Orders", "apps/order/index")
if err := (adminSeeder{}).SeedMenus(db, "order", menus, apis); err != nil {
t.Fatalf("seed: %v", err)
}
var rows []models.SysMenu
if err := db.Where("app_code = ? AND menu_name = ?", "order", menuName("order", "dir")).
Find(&rows).Error; err != nil {
t.Fatal(err)
}
if len(rows) != 1 {
t.Fatalf("%d rows carry that name; the row from before the column existed was not found", len(rows))
}
if rows[0].MenuId != legacy.MenuId {
t.Errorf("menu_id = %d, want the row that was already there (%d)", rows[0].MenuId, legacy.MenuId)
}
if rows[0].SeedCode == nil || *rows[0].SeedCode != "dir" {
t.Errorf("seed_code = %v, want it claimed", rows[0].SeedCode)
}
// Adopted and then repaired, like any other existing row.
if rows[0].Title != "Order Example" {
t.Errorf("title = %q; the adopted row was not brought up to the spec", rows[0].Title)
}
}
// menuName concatenates two pascalCase strings and pascalCase is not
// injective, so two specs can land on one name. Picking one of several rows
// would attach an application's menu to whichever the database returned
// first.
func TestSeedMenusRefusesAnAmbiguousAdoption(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
for i := 0; i < 2; i++ {
row := models.SysMenu{
MenuName: menuName("order", "dir"), AppCode: "order", Title: fmt.Sprintf("copy %d", i),
MenuType: contractmodels.Directory,
}
if err := db.Create(&row).Error; err != nil {
t.Fatal(err)
}
}
menus, apis := orderMenuSpecs("Orders", "apps/order/index")
err := (adminSeeder{}).SeedMenus(db, "order", menus, apis)
if err == nil {
t.Fatal("an ambiguous adoption was accepted")
}
if !strings.Contains(err.Error(), "2 rows") || !strings.Contains(err.Error(), "by hand") {
t.Errorf("error = %q, it has to say how many and that it is not deciding", err)
}
// And it did not write a third.
var n int64
db.Model(&models.SysMenu{}).Where("app_code = ? AND menu_name = ?", "order", menuName("order", "dir")).Count(&n)
if n != 2 {
t.Errorf("%d rows carry that name; the refusal still inserted", n)
}
}
// A row belonging to another application, or to the host, carries a different
// app_code and is not this application's to claim.
func TestSeedMenusDoesNotAdoptAnotherApplicationsRow(t *testing.T) {
db := newSeedTestDB(t)
useCompositeSeedCodeIndex(t, db)
seedAdminRole(t, db)
other := models.SysMenu{
MenuName: menuName("order", "dir"), AppCode: "crm", Title: "crm's own",
MenuType: contractmodels.Directory,
}
if err := db.Create(&other).Error; err != nil {
t.Fatal(err)
}
menus, apis := orderMenuSpecs("Orders", "apps/order/index")
if err := (adminSeeder{}).SeedMenus(db, "order", menus, apis); err != nil {
t.Fatalf("seed: %v", err)
}
var after models.SysMenu
db.Where("menu_id = ?", other.MenuId).First(&after)
if after.SeedCode != nil || after.Title != "crm's own" {
t.Errorf("another application's row was claimed: %+v", after)
}
var mine models.SysMenu
if err := db.Where("app_code = ? AND seed_code = ?", "order", "dir").First(&mine).Error; err != nil {
t.Fatalf("this application's own row was not created: %v", err)
}
}
+30 -4
View File
@@ -1,6 +1,7 @@
package jobs
import (
"context"
"fmt"
log "github.com/go-admin-team/go-admin-core/v2/logger"
"github.com/go-admin-team/go-admin-core/v2/sdk"
@@ -145,11 +146,36 @@ func setup(key string, db *gorm.DB) {
}
// 其中任务
crontab.Start()
startCrontab(crontab)
}
// startCrontab starts c and arranges for it to be stopped on the way out.
//
// The stop used to be `defer crontab.Stop()` followed by `select {}`. The
// select never returned, so the defer never ran and the scheduler was never
// stopped; and because setup never returned, the loop in Setup never reached
// the second tenant - only whichever database came first out of the map ever
// got a scheduler at all. cron.Start is itself `go c.run()`, so the select was
// blocking for nothing.
//
// cron.Stop returns a context that closes once the jobs already running have
// finished. That is the wait the shutdown budget exists to bound: giving up on
// it leaves those jobs running until the process exits, which is better than
// holding the whole shutdown open for one job that will not end.
func startCrontab(c *cron.Cron) {
c.Start()
fmt.Println(time.Now().Format(timeFormat), " [INFO] JobCore start success.")
// 关闭任务
defer crontab.Stop()
select {}
sdk.Runtime.SetShutdown(func(ctx context.Context) {
stopped := c.Stop()
select {
case <-stopped.Done():
fmt.Println(time.Now().Format(timeFormat), " [INFO] JobCore stopped.")
case <-ctx.Done():
fmt.Println(time.Now().Format(timeFormat), " [WARN] JobCore stop gave up waiting for running jobs")
}
})
}
// AddJob 添加任务 AddJob(invokeTarget string, jobId int, jobName string, cronExpression string)
@@ -183,7 +209,7 @@ func (e *ExecJob) addJob(c *cron.Cron) (int, error) {
// Remove 移除任务
func Remove(c *cron.Cron, entryID int) chan bool {
ch := make(chan bool)
ch := make(chan bool, 1)
go func() {
c.Remove(cron.EntryID(entryID))
fmt.Println(time.Now().Format(timeFormat), " [INFO] JobCore Remove success ,info entryID :", entryID)
+52
View File
@@ -0,0 +1,52 @@
package jobs
import (
"context"
"sync/atomic"
"testing"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg/cronjob"
)
// The scheduler had never been stopped. `defer crontab.Stop()` sat directly
// above a `select {}` that never returned, so the deferred call was
// unreachable for the life of the process.
//
// There is one test rather than several because BeforeExit closes to further
// registration once it has run: a second RunShutdown in this binary would find
// an empty registry and pass while proving nothing.
func TestTheSchedulerIsStoppedOnTheWayOut(t *testing.T) {
var ticks atomic.Int64
c := cronjob.NewWithSeconds()
if _, err := c.AddFunc("* * * * * *", func() { ticks.Add(1) }); err != nil {
t.Fatalf("AddFunc: %v", err)
}
startCrontab(c)
// It has to be running before stopping it can mean anything.
deadline := time.Now().Add(5 * time.Second)
for ticks.Load() == 0 && time.Now().Before(deadline) {
time.Sleep(20 * time.Millisecond)
}
if ticks.Load() == 0 {
t.Fatal("the scheduler never ran the job, so this test cannot show it was stopped")
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
if err := sdk.Runtime.RunShutdown(ctx); err != nil {
t.Fatalf("RunShutdown: %v", err)
}
// Two and a half seconds is two more firings of a job that runs every
// second, so silence here is the assertion.
at := ticks.Load()
time.Sleep(2500 * time.Millisecond)
if n := ticks.Load() - at; n > 0 {
t.Errorf("the job fired %d more times after shutdown: the scheduler is still running", n)
}
}
+1 -1
View File
@@ -39,7 +39,7 @@ func (e *SysJob) RemoveJob(c *dto.GeneralDelDto) error {
}
case <-time.After(time.Second * 1):
e.Msg = "操作超时!"
return nil
return errors.New(e.Msg)
}
return nil
}
+88
View File
@@ -0,0 +1,88 @@
package service
import (
"testing"
"time"
"github.com/glebarez/sqlite"
coreservice "github.com/go-admin-team/go-admin-core/v2/sdk/service"
"github.com/robfig/cron/v3"
"gorm.io/gorm"
"go-admin/app/jobs/models"
"go-admin/common/dto"
)
type blockedSchedule struct {
started chan struct{}
release chan struct{}
}
func (s blockedSchedule) Next(now time.Time) time.Time {
close(s.started)
<-s.release
return now.Add(time.Hour)
}
func TestRemoveJob(t *testing.T) {
for _, blocked := range []bool{false, true} {
name := "success"
if blocked {
name = "timeout"
}
t.Run(name, func(t *testing.T) {
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatal(err)
}
sqlDB, err := db.DB()
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = sqlDB.Close() })
if err := db.AutoMigrate(&models.SysJob{}); err != nil {
t.Fatal(err)
}
c := cron.New()
schedule := blockedSchedule{make(chan struct{}), make(chan struct{})}
entryID := c.Schedule(schedule, cron.FuncJob(func() {}))
if blocked {
c.Start()
t.Cleanup(func() {
close(schedule.release)
<-c.Stop().Done()
})
select {
case <-schedule.started:
case <-time.After(5 * time.Second):
t.Fatal("scheduler did not start")
}
}
job := models.SysJob{EntryId: int(entryID)}
if err := db.Create(&job).Error; err != nil {
t.Fatal(err)
}
s := SysJob{Service: coreservice.Service{Orm: db}, Cron: c}
err = s.RemoveJob(&dto.GeneralDelDto{Id: job.JobId})
if blocked {
if err == nil || err.Error() != "操作超时!" {
t.Errorf("RemoveJob error = %v, want timeout error", err)
}
} else if err != nil {
t.Fatal(err)
}
var saved models.SysJob
if err := db.First(&saved, job.JobId).Error; err != nil {
t.Fatal(err)
}
wantEntryID := 0
if blocked {
wantEntryID = int(entryID)
}
if saved.EntryId != wantEntryID {
t.Errorf("entry_id = %d, want %d", saved.EntryId, wantEntryID)
}
})
}
}
+7 -1
View File
@@ -8,6 +8,12 @@ import (
"go-admin/app/other/models/tools"
)
// emptyTableNameMsg is what the generator's endpoints answer with when the
// request named no table. Declared once because the tests assert on it: spelled
// out again at each site, a reworded message would leave them asserting on a
// string the server no longer sends, and still passing.
const emptyTableNameMsg = "table name cannot be empty!"
// GetDBColumnList 分页列表数据
// @Summary 分页列表数据 / page list data
// @Description 数据库表列分页列表 / database table column page list
@@ -41,7 +47,7 @@ func (e Gen) GetDBColumnList(c *gin.Context) {
}
data.TableName = c.Request.FormValue("tableName")
pkg.Assert(data.TableName != "", "table name cannot be empty!", 500)
pkg.Assert(data.TableName != "", emptyTableNameMsg, 500)
result, count, err := data.GetPage(db, pageSize, pageIndex)
if err != nil {
log.Errorf("GetPage error, %s", err.Error())
+26 -10
View File
@@ -16,17 +16,21 @@ import (
"go-admin/common/middleware"
)
const emptyTableNameMsg = "table name cannot be empty!"
// bodyOf covers both the success and the CustomError shape: both carry msg.
type bodyOf struct {
Code int `json:"code"`
Msg string `json:"msg"`
}
// newColumnListEngine wires the handler the way the router does, including the
// newEngine wires one generator handler the way the router does, including the
// middleware that turns pkg.Assert's panic into a response.
func newColumnListEngine(t *testing.T) *gin.Engine {
//
// The generator's queries target MySQL's information_schema and cannot run on
// the sqlite connection behind them; the driver setting only has to select that
// branch, since no statement here is expected to succeed. That makes this
// serviceable for any handler in this package whose behaviour is decided before
// the query goes out -- which is what these tests are about.
func newEngine(t *testing.T, method, path string, h gin.HandlerFunc) *gin.Engine {
t.Helper()
gin.SetMode(gin.TestMode)
@@ -35,26 +39,33 @@ func newColumnListEngine(t *testing.T) *gin.Engine {
t.Fatalf("open sqlite: %v", err)
}
// The query targets MySQL's information_schema; the driver setting only has
// to select that branch, the statement itself is never expected to succeed.
previous := config.DatabaseConfig.Driver
config.DatabaseConfig.Driver = "mysql"
t.Cleanup(func() { config.DatabaseConfig.Driver = previous })
r := gin.New()
r.Use(middleware.CustomError)
r.GET("/db/columns/page", func(c *gin.Context) {
r.Handle(method, path, func(c *gin.Context) {
c.Set("db", db)
c.Set(pkg.LoggerKey, logger.NewHelper(logger.DefaultLogger))
Gen{}.GetDBColumnList(c)
h(c)
})
return r
}
func columnListMsg(t *testing.T, r *gin.Engine, query string) bodyOf {
func newColumnListEngine(t *testing.T) *gin.Engine {
t.Helper()
return newEngine(t, http.MethodGet, "/db/columns/page", Gen{}.GetDBColumnList)
}
// serveJSON runs one request through the engine and decodes the envelope every
// handler here answers with. A body that will not decode fails the test rather
// than being reported as a mismatched message, which reads as the handler
// having answered something unexpected instead of not having answered at all.
func serveJSON(t *testing.T, r *gin.Engine, req *http.Request) bodyOf {
t.Helper()
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/db/columns/page"+query, nil))
r.ServeHTTP(w, req)
var body bodyOf
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
@@ -63,6 +74,11 @@ func columnListMsg(t *testing.T, r *gin.Engine, query string) bodyOf {
return body
}
func columnListMsg(t *testing.T, r *gin.Engine, query string) bodyOf {
t.Helper()
return serveJSON(t, r, httptest.NewRequest(http.MethodGet, "/db/columns/page"+query, nil))
}
func TestGetDBColumnList_AcceptsATableName(t *testing.T) {
body := columnListMsg(t, newColumnListEngine(t), "?tableName=sys_user")
if body.Msg == emptyTableNameMsg {
+49 -2
View File
@@ -1,6 +1,7 @@
package tools
import (
"errors"
"strings"
"github.com/gin-gonic/gin"
@@ -148,7 +149,8 @@ func (e SysTable) GetSysTablesTree(c *gin.Context) {
// @Tags 工具 / 生成工具
// @Accept application/json
// @Product application/json
// @Param tables query string false "tableName / 数据表名称"
// @Param tables query string false "tableName / 数据表名称,逗号分隔"
// @Param data body object false "tables / 同上,query 未带时从 JSON body 读"
// @Success 200 {string} string "{"code": 200, "message": "添加成功"}"
// @Success 200 {string} string "{"code": -1, "message": "添加失败"}"
// @Router /api/v1/sys/tables/info [post]
@@ -163,7 +165,13 @@ func (e SysTable) Insert(c *gin.Context) {
return
}
tablesList := strings.Split(c.Request.FormValue("tables"), ",")
tablesList, err := tablesToImport(c)
if err != nil {
log.Errorf("read the table list, %s", err.Error())
e.Error(500, err, "")
return
}
for i := 0; i < len(tablesList); i++ {
data, err := genTableInit(db, tablesList, i, c)
@@ -184,6 +192,45 @@ func (e SysTable) Insert(c *gin.Context) {
}
// tablesToImport reads the comma-separated table list carried by an import
// request, from the query string or from a JSON body.
//
// The list has only ever travelled in the query string, which is the single
// place FormValue looks once the request declares itself as JSON. A front end
// that puts it in the body instead therefore left this empty, and the import
// went on to ask information_schema for a table named "" -- go-admin-ui v3.2.0
// shipped exactly that, and every import failed with the message below.
// Reading the body when the query has nothing keeps either front end working.
func tablesToImport(c *gin.Context) ([]string, error) {
raw := c.Request.FormValue("tables")
if raw == "" {
var body struct {
Tables string `json:"tables"`
}
// A body that is absent, or shaped some other way, is not itself worth
// reporting: the list is missing either way, and the message below says
// so in the terms the caller asked in.
if err := c.ShouldBindJSON(&body); err == nil {
raw = body.Tables
}
}
parts := strings.Split(raw, ",")
names := make([]string, 0, len(parts))
for _, name := range parts {
// Splitting "" yields one empty name rather than nothing at all, so
// without this an empty list reads as a request to import one table
// whose name happens to be blank.
if name = strings.TrimSpace(name); name != "" {
names = append(names, name)
}
}
if len(names) == 0 {
return nil, errors.New(emptyTableNameMsg)
}
return names, nil
}
func genTableInit(tx *gorm.DB, tablesList []string, i int, c *gin.Context) (tools.SysTables, error) {
var data tools.SysTables
var dbTable tools.DBTables
+124
View File
@@ -0,0 +1,124 @@
package tools
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gin-gonic/gin"
)
// newImportRequest builds the request an import arrives in. Where the table
// list sits -- query or body -- is exactly what these tests are about, and it
// is net/http's form parsing that decides what a handler can reach, so these go
// through a real *http.Request rather than a hand-built one.
func newImportRequest(target, contentType, body string) *http.Request {
req := httptest.NewRequest(http.MethodPost, target, strings.NewReader(body))
if contentType != "" {
req.Header.Set("Content-Type", contentType)
}
return req
}
func TestTablesToImport(t *testing.T) {
cases := []struct {
name string
target string
contentType string
body string
want []string
wantErr bool
}{{
name: "from the query, as every front end before v3.2.0 sent it",
target: "/sys/tables/info?tables=sys_user,sys_post",
want: []string{"sys_user", "sys_post"},
}, {
name: "from a JSON body, as go-admin-ui v3.2.0 sends it",
target: "/sys/tables/info",
contentType: "application/json",
body: `{"tables":"sys_user,sys_post"}`,
want: []string{"sys_user", "sys_post"},
}, {
name: "the query wins when a request carries both",
target: "/sys/tables/info?tables=sys_user",
contentType: "application/json",
body: `{"tables":"sys_post"}`,
want: []string{"sys_user"},
}, {
name: "blank entries are dropped rather than imported as a nameless table",
target: "/sys/tables/info?tables=sys_user,,%20,sys_post",
want: []string{"sys_user", "sys_post"},
}, {
name: "a body carrying an empty list is an error",
target: "/sys/tables/info",
contentType: "application/json",
body: `{"tables":""}`,
wantErr: true,
}, {
name: "a body that is not JSON at all is an error, not a panic",
target: "/sys/tables/info",
contentType: "application/json",
body: "sys_user",
wantErr: true,
}}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
c, _ := gin.CreateTestContext(httptest.NewRecorder())
c.Request = newImportRequest(tc.target, tc.contentType, tc.body)
got, err := tablesToImport(c)
if tc.wantErr {
if err == nil {
t.Fatalf("expected an error, got %q", got)
}
if err.Error() != emptyTableNameMsg {
t.Fatalf("message should be the one the front end shows, got %q", err.Error())
}
return
}
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if strings.Join(got, ",") != strings.Join(tc.want, ",") {
t.Fatalf("got %q, want %q", got, tc.want)
}
})
}
}
// insertMsg runs one import through the wired handler. It asserts nothing about
// the import succeeding -- it cannot, over sqlite -- only about how far the
// request got, which the empty-list message is what distinguishes.
func insertMsg(t *testing.T, target, contentType, body string) bodyOf {
t.Helper()
return serveJSON(t,
newEngine(t, http.MethodPost, "/sys/tables/info", SysTable{}.Insert),
newImportRequest(target, contentType, body))
}
func TestInsert_ReadsTheTableListFromEitherPlace(t *testing.T) {
for _, tc := range []struct {
name string
target string
contentType string
body string
}{
{"query", "/sys/tables/info?tables=sys_user", "", ""},
{"JSON body", "/sys/tables/info", "application/json", `{"tables":"sys_user"}`},
} {
t.Run(tc.name, func(t *testing.T) {
if got := insertMsg(t, tc.target, tc.contentType, tc.body); got.Msg == emptyTableNameMsg {
t.Fatalf("request carried a table name and was still rejected as empty: %+v", got)
}
})
}
}
func TestInsert_RejectsAMissingTableList(t *testing.T) {
if got := insertMsg(t, "/sys/tables/info", "", ""); got.Msg != emptyTableNameMsg {
t.Fatalf("missing table list should be rejected, got %+v", got)
}
}
+91
View File
@@ -0,0 +1,91 @@
package router
import (
"testing"
"github.com/gin-gonic/gin"
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"go-admin/common/middleware"
)
// registeredRoutes builds the generator's routes on an engine of its own and
// reports the patterns they were registered under.
//
// The mode has to be given rather than inherited, because it now decides what
// gets registered: a test that leaves it at the zero value would be asking
// about a mode no deployment runs in, and would pass whether or not the gate
// works.
//
// It is put back before this returns, not at the end of the test. t.Cleanup
// would leave the mode set for everything the caller does afterwards, so a
// caller that went on to assert something mode-dependent would be reading a
// value this helper left behind rather than one it chose. A caller that does
// want the mode set has to set it, which is visible where it happens.
//
// The JWT middleware is a zero value. MiddlewareFunc only closes over the
// receiver and is never called here - no request is served, the engine is
// asked what it has - so nothing dereferences it.
func registeredRoutes(t *testing.T, mode string) map[string]bool {
t.Helper()
gin.SetMode(gin.TestMode)
previous := config.ApplicationConfig.Mode
defer func() { config.ApplicationConfig.Mode = previous }()
config.ApplicationConfig.Mode = mode
r := gin.New()
v1 := r.Group("/api/v1")
sysNoCheckRoleRouter(v1, &jwt.GinJWTMiddleware{})
registerDBRouter(v1, &jwt.GinJWTMiddleware{})
out := map[string]bool{}
for _, route := range r.Routes() {
out[route.Path] = true
}
return out
}
// The list of routes demo mode refuses lives in common/middleware, which may
// not import app/ and therefore cannot see whether any of them is still a
// route. This is the half that can be checked, and it is checked here because
// this is where the routes are declared: rename one, and the entry over there
// stops matching anything, demo mode silently starts serving it again, and
// nothing else would say so.
func TestEveryRouteDemoModeRefusesStillExists(t *testing.T) {
routes := registeredRoutes(t, "demo")
for _, guarded := range middleware.DemoWriteRoutes() {
if !routes[guarded] {
t.Errorf("demo mode refuses %q, but no route is registered under that pattern - "+
"either it was renamed, or it moved to another file; the guard now matches nothing",
guarded)
}
}
}
// The other direction, and the one the demo host cares about: the generator's
// read-only routes have to stay reachable, or a demo deployment cannot show
// the feature at all. Refusing too much is as much of a defect as refusing too
// little.
func TestTheGeneratorsReadOnlyRoutesAreNotRefused(t *testing.T) {
refused := map[string]bool{}
for _, guarded := range middleware.DemoWriteRoutes() {
refused[guarded] = true
}
for _, readOnly := range []string{
"/api/v1/gen/preview/:tableId",
"/api/v1/gen/tabletree",
"/api/v1/db/tables/page",
"/api/v1/db/columns/page",
} {
if !registeredRoutes(t, "demo")[readOnly] {
t.Fatalf("%s is not registered, so this test is asserting against nothing", readOnly)
}
if refused[readOnly] {
t.Errorf("demo mode refuses %s, which only reads - the demo host needs it to "+
"demonstrate the generator", readOnly)
}
}
}
+41 -5
View File
@@ -3,15 +3,49 @@ package router
import (
"github.com/gin-gonic/gin"
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"go-admin/app/admin/apis"
"go-admin/app/other/apis/tools"
)
// GenWriteRoutesEnabled reports whether the code generator's writing endpoints
// are registered in this process.
//
// Three of the generator's endpoints do not read. /gen/toproject writes seven
// Go and Vue source files onto this host, one of them under the path
// gen.frontpath names; /gen/apitofile writes a migration; /gen/todb inserts
// menus and APIs. All three are GET, all three are listed in CasbinExclude,
// and AuthCheckRole skips what is on that list - so Enforce never runs for
// them and any account that can log in may call them. That is a bargain a
// workstation can make and a deployment cannot.
//
// dev is the shipped default and is where the generator is meant to be used.
// demo keeps them because demo mode already has a better answer than a 404:
// DemoEvn refuses these three by name and explains itself, which is the thing
// the demo exists to show. test and prod get nothing.
//
// The mode is read once, while the routes are being built. Changing
// application.mode in a running process adds and removes nothing - a
// configuration reload rebuilds neither the engine nor its routes.
//
// core has constants for dev, test and prod but none for demo, which this
// repository spells as a literal in common/middleware/demo.go. Both are
// literals here so that the two read as one set rather than two conventions.
func GenWriteRoutesEnabled() bool {
switch config.ApplicationConfig.Mode {
case "dev", "demo":
return true
default:
return false
}
}
func init() {
routerCheckRole = append(routerCheckRole, sysNoCheckRoleRouter, registerDBRouter, registerSysTableRouter)
}
func sysNoCheckRoleRouter(v1 *gin.RouterGroup ,authMiddleware *jwt.GinJWTMiddleware) {
func sysNoCheckRoleRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddleware) {
r1 := v1.Group("")
{
sys := apis.System{}
@@ -22,9 +56,11 @@ func sysNoCheckRoleRouter(v1 *gin.RouterGroup ,authMiddleware *jwt.GinJWTMiddlew
{
gen := tools.Gen{}
r.GET("/gen/preview/:tableId", gen.Preview)
r.GET("/gen/toproject/:tableId", gen.GenCode)
r.GET("/gen/apitofile/:tableId", gen.GenApiToFile)
r.GET("/gen/todb/:tableId", gen.GenMenuAndApi)
if GenWriteRoutesEnabled() {
r.GET("/gen/toproject/:tableId", gen.GenCode)
r.GET("/gen/apitofile/:tableId", gen.GenApiToFile)
r.GET("/gen/todb/:tableId", gen.GenMenuAndApi)
}
sysTable := tools.SysTable{}
r.GET("/gen/tabletree", sysTable.GetSysTablesTree)
}
@@ -53,4 +89,4 @@ func registerSysTableRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddl
tablesInfo.GET("", sysTable.GetSysTablesInfo)
}
}
}
}
+151
View File
@@ -0,0 +1,151 @@
package router
import (
"testing"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
// genWritingRoutes are the three that do not read. They write Go and Vue
// source onto the host, a migration, and rows in sys_menu.
var genWritingRoutes = []string{
"/api/v1/gen/toproject/:tableId",
"/api/v1/gen/apitofile/:tableId",
"/api/v1/gen/todb/:tableId",
}
// genReadingRoutes are the rest of the generator's surface. Gating the three
// above must not cost any of these: a deployment that cannot list its tables
// or preview a template has lost the feature, not secured it.
var genReadingRoutes = []string{
"/api/v1/gen/preview/:tableId",
"/api/v1/gen/tabletree",
"/api/v1/db/tables/page",
"/api/v1/db/columns/page",
}
// The endpoints that write are registered where the mode says development and
// nowhere else.
//
// They are in CasbinExclude, so Enforce never runs for them and any account
// that can log in may call them. dev is the shipped default and is where the
// generator is meant to be used; demo keeps them because DemoEvn refuses these
// three by name and saying so is the thing the demo is for. Everything else,
// including the empty mode a process gets when nothing set one, is refused by
// not existing.
func TestGeneratorWritingRoutesExistOnlyWhereTheModeAllowsIt(t *testing.T) {
for _, tc := range []struct {
mode string
expected bool
why string
}{
{"dev", true, "the shipped default, and where the generator is used"},
{"demo", true, "registered so demo mode can refuse them by name"},
{"test", false, "a deployment, however much it is called a test"},
{"prod", false, "a deployment"},
{"", false, "no mode configured is not a reason to trust the caller"},
} {
t.Run("mode="+tc.mode, func(t *testing.T) {
routes := registeredRoutes(t, tc.mode)
for _, writing := range genWritingRoutes {
if got := routes[writing]; got != tc.expected {
t.Errorf("mode %q: %s registered = %v, want %v (%s)",
tc.mode, writing, got, tc.expected, tc.why)
}
}
})
}
}
// The other direction. Refusing too much is as much of a defect as refusing
// too little, and the read-only half of the generator is what a demo shows.
func TestGeneratorReadingRoutesExistInEveryMode(t *testing.T) {
for _, mode := range []string{"dev", "demo", "test", "prod", ""} {
t.Run("mode="+mode, func(t *testing.T) {
routes := registeredRoutes(t, mode)
for _, reading := range genReadingRoutes {
if !routes[reading] {
t.Errorf("mode %q: %s is not registered - the gate took a route that only reads",
mode, reading)
}
}
})
}
}
// GenWriteRoutesEnabled is what cmd/api reads to decide whether to warn at
// start-up. If it and the registration ever disagree, the log says one thing
// and the engine does another, so they are checked against each other rather
// than each against a list.
func TestGenWriteRoutesEnabledAgreesWithWhatWasRegistered(t *testing.T) {
for _, mode := range []string{"dev", "demo", "test", "prod", ""} {
t.Run("mode="+mode, func(t *testing.T) {
routes := registeredRoutes(t, mode)
// registeredRoutes puts the mode back before it returns, so ask
// the predicate under a mode set here - about the same value the
// engine was just built under.
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = mode
claimed := GenWriteRoutesEnabled()
actual := routes["/api/v1/gen/todb/:tableId"]
if claimed != actual {
t.Errorf("mode %q: GenWriteRoutesEnabled() = %v but the route was registered = %v",
mode, claimed, actual)
}
})
}
}
// The helper restores the mode before it returns, so nothing it was asked
// about leaks into what the caller does next.
//
// Worth a test of its own because the failure is silent: a helper that left
// the mode set would make every assertion after the call read a value the
// caller did not choose, and each of those assertions would still pass for as
// long as the leaked value happened to be the right one.
func TestRegisteredRoutesRestoresTheModeBeforeReturning(t *testing.T) {
const sentinel = "not-a-mode"
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = sentinel
registeredRoutes(t, "prod")
if got := config.ApplicationConfig.Mode; got != sentinel {
t.Errorf("mode after the helper returned = %q, want %q - it was left set to what "+
"the helper was asked about", got, sentinel)
}
}
// Changing the mode after the routes were built unregisters nothing.
//
// buildRouter has one call site, in run(), and route registration is not on
// any phase or reload callback - so a configuration reload moves
// config.ApplicationConfig.Mode without moving the routes. From that moment
// GenWriteRoutesEnabled answers about a mode the engine was not built under.
//
// That gap is why the start-up warning tells the reader to restart rather than
// only to change the mode. This pins it: if registration ever becomes dynamic,
// this test fails and the message it justifies has to be revisited.
func TestChangingTheModeDoesNotUnregisterWhatWasAlreadyBuilt(t *testing.T) {
built := registeredRoutes(t, "dev")
if !built["/api/v1/gen/todb/:tableId"] {
t.Fatal("built under dev without the writing routes, so this test asserts nothing")
}
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = "prod"
if GenWriteRoutesEnabled() {
t.Fatal("the predicate still allows prod, so the disagreement below is not the one meant")
}
if !built["/api/v1/gen/todb/:tableId"] {
t.Error("the route left the engine when the mode changed - registration has become " +
"dynamic, and the start-up warning's advice to restart is now wrong")
}
}
+65 -6
View File
@@ -1,23 +1,82 @@
package router
import (
"context"
"net/http"
"time"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/tools/transfer"
"github.com/prometheus/client_golang/prometheus/promhttp"
"go-admin/common/health"
)
func init() {
routerNoCheckRole = append(routerNoCheckRole, registerMonitorRouter)
routerNoCheckRole = append(routerNoCheckRole, RegisterMonitorRouter)
}
// 需认证的路由代码
func registerMonitorRouter(v1 *gin.RouterGroup) {
// readyTimeout bounds the whole probe. What constrains it is the orchestrator's
// per-check timeout rather than its polling period: Kubernetes allows a probe
// one second by default, so a dependency that answers in 1.2s is recorded as a
// failed check however promptly this handler returns. A manifest that mounts
// this probe has to raise timeoutSeconds above this value, and
// scripts/k8s/deploy.yml does.
const readyTimeout = 2 * time.Second
// HealthPath and ReadyPath are the two probe routes, relative to APIPrefix.
//
// Exported for the same reason as the prefix: the rate limiter has to be told
// to skip them, and it is installed in a package that cannot import this one.
const (
HealthPath = "/health"
ReadyPath = "/ready"
)
// RegisterMonitorRouter mounts the metrics endpoint and the two probes on v1.
//
// Exported so that a test can put the real probes on a server of its own. The
// alternative - a test that re-implements the handler it means to check - is
// how a probe comes to be asserted against a copy of itself.
//
// 无需认证的路由代码
func RegisterMonitorRouter(v1 *gin.RouterGroup) {
v1.GET("/metrics", transfer.Handler(promhttp.Handler()))
//健康检查
v1.GET("/health", func(c *gin.Context) {
// 健康检查(存活)
//
// Stays a bare 200 on purpose. This is the answer to "should I restart
// you", and a process whose database is unreachable does not want
// restarting - that turns one outage into a crash loop and throws away the
// connection pool, the cache and every in-flight request along the way.
v1.GET(HealthPath, func(c *gin.Context) {
c.Status(http.StatusOK)
})
}
// 就绪检查
//
// The answer to "should I send you requests". It fails while a dependency
// is unreachable, and from the moment shutdown begins - for as long as
// extend.shutdown.drain says, which is zero unless it is configured. The
// package comment in common/health says what that window is worth, and to
// whom.
v1.GET(ReadyPath, func(c *gin.Context) {
if health.Draining() {
c.JSON(http.StatusServiceUnavailable, gin.H{
"status": "draining",
"checks": []health.Check{},
})
return
}
ctx, cancel := context.WithTimeout(c.Request.Context(), readyTimeout)
defer cancel()
checks := health.Ready(ctx)
status := http.StatusOK
if !health.Healthy(checks) {
status = http.StatusServiceUnavailable
}
c.JSON(status, gin.H{"status": http.StatusText(status), "checks": checks})
})
}
+9 -2
View File
@@ -10,6 +10,13 @@ var (
routerCheckRole = make([]func(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddleware), 0)
)
// APIPrefix is the group every route below is registered under.
//
// Exported because the middleware chain in cmd/api has to name two of those
// routes in full - the rate limiter is installed on the engine and must skip
// the probes - and a prefix spelled in two places is a prefix that drifts.
const APIPrefix = "/api/v1"
// initRouter 路由示例
func initRouter(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware) *gin.Engine {
@@ -24,7 +31,7 @@ func initRouter(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware) *gin.Engine
// noCheckRoleRouter 无需认证的路由示例
func noCheckRoleRouter(r *gin.Engine) {
// 可根据业务需求来设置接口版本
v1 := r.Group("/api/v1")
v1 := r.Group(APIPrefix)
for _, f := range routerNoCheckRole {
f(v1)
@@ -34,7 +41,7 @@ func noCheckRoleRouter(r *gin.Engine) {
// checkRoleRouter 需要认证的路由示例
func checkRoleRouter(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware) {
// 可根据业务需求来设置接口版本
v1 := r.Group("/api/v1")
v1 := r.Group(APIPrefix)
for _, f := range routerCheckRole {
f(v1, authMiddleware)
+38
View File
@@ -0,0 +1,38 @@
package api
import (
"testing"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
// The warning fires exactly where the generator's writing endpoints are served
// and nothing else refuses them.
//
// dev is the case the warning exists for: it is the shipped default, so it is
// the mode a deployment that changed nothing is running in. demo serves the
// routes too, but DemoEvn refuses all three by name, so warning there would
// describe an exposure that is not there.
func TestGeneratorWriteRoutesWarningFiresWhereTheExposureIs(t *testing.T) {
for _, tc := range []struct {
mode string
want bool
why string
}{
{"dev", true, "shipped default, endpoints served and not refused"},
{"demo", false, "served, but DemoEvn refuses all three"},
{"test", false, "not served"},
{"prod", false, "not served"},
{"", false, "not served"},
} {
t.Run("mode="+tc.mode, func(t *testing.T) {
previous := config.ApplicationConfig.Mode
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
config.ApplicationConfig.Mode = tc.mode
if got := generatorWriteRoutesNeedWarning(); got != tc.want {
t.Errorf("mode %q: warning = %v, want %v (%s)", tc.mode, got, tc.want, tc.why)
}
})
}
}
+160
View File
@@ -0,0 +1,160 @@
package api
import (
"fmt"
"net"
"net/http"
"testing"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
)
// freePort returns a port nothing is listening on. It is inherently a guess -
// the port is free when it is handed back and could be taken a moment later -
// but every alternative needs the caller to hold the listener, which is the one
// thing these tests cannot do.
func freePort(t *testing.T) int {
t.Helper()
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("probe listen: %v", err)
}
port := ln.Addr().(*net.TCPAddr).Port
_ = ln.Close()
return port
}
// AfterListen promises a hook that the port is reachable. Both halves of that
// are asserted here, and in one test rather than two, because the phase seals
// itself once it has run: a second test calling RunPhase again would find a
// closed registry and pass while proving nothing.
//
// The failing bind comes first for the same reason. It must leave the phase
// unsealed, which is only visible if nothing has sealed it yet.
func TestAfterListenIsAnnouncedOnlyOnceThePortIsBound(t *testing.T) {
// The pause makes the "announced synchronously" claim testable: if the
// announcement were moved onto a goroutine, startServing would return
// while the hook was still sleeping and the count below would be zero.
var ran int
sdk.Runtime.SetPhase(runtime.AfterListen, func() {
time.Sleep(50 * time.Millisecond)
ran++
})
// Somebody else already has the port. Under ListenAndServe this surfaced
// on the serving goroutine, far too late to stop the announcement.
taken, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("occupy: %v", err)
}
defer func() { _ = taken.Close() }()
blocked := &http.Server{Addr: taken.Addr().String(), Handler: http.NewServeMux()}
if err := startServing(blocked, false, "", ""); err == nil {
t.Fatal("startServing returned no error for a port that was already taken")
}
if ran != 0 {
t.Errorf("AfterListen ran %d times after a failed bind; a hook there is told the port is reachable", ran)
}
if sdk.Runtime.PhaseSealed(runtime.AfterListen) {
t.Error("a failed bind sealed AfterListen, so the phase could never run for a server that did start")
}
// A certificate that cannot be read is the other way to fail before there
// is anything to announce. ServeTLS reads it on the serving goroutine, so
// without the check in startServing this would be a hook told the port was
// reachable while the server was already on its way down.
if err := startServing(&http.Server{Addr: "127.0.0.1:0"}, true, "no-such.pem", "no-such.key"); err == nil {
t.Fatal("startServing returned no error for a certificate that does not exist")
}
if ran != 0 {
t.Errorf("AfterListen ran %d times after a certificate failure", ran)
}
if sdk.Runtime.PhaseSealed(runtime.AfterListen) {
t.Error("a certificate failure sealed AfterListen")
}
// And now a bind that works.
port := freePort(t)
srv := &http.Server{Addr: fmt.Sprintf("127.0.0.1:%d", port), Handler: http.NewServeMux()}
if err := startServing(srv, false, "", ""); err != nil {
t.Fatalf("startServing on a free port: %v", err)
}
defer func() { _ = srv.Close() }()
// Checked the instant startServing returns, so this is also the assertion
// that it did not return early: an asynchronous announcement would still
// be inside the sleep. Synchrony matters because an announcement that
// overlaps the wait below could, on a fast SIGTERM, have the shutdown
// callbacks finish before the startup ones.
if ran != 1 {
t.Fatalf("AfterListen ran %d times, want 1", ran)
}
// The claim is not "Serve was called" but "the port answers". Dial it.
c, err := net.DialTimeout("tcp", srv.Addr, 5*time.Second)
if err != nil {
t.Fatalf("AfterListen ran but the port does not answer: %v", err)
}
_ = c.Close()
}
// BeforeRouter is the last point at which a module can still affect how routes
// are built, so it has to run while there is no engine yet. The before registry
// is a different moment despite the name: those callbacks run after initRouter
// has built the engine.
//
// The two are two lines apart in buildRouter, and calling them equivalent is a
// mistake this repository has already made in writing. Until this test the
// ordering was checked by reading - which is how the stop signals came to be
// armed after the readiness banner in the same file.
func TestBeforeRouterRunsWhileThereIsNoEngine(t *testing.T) {
freshRuntime(t)
// AuthInit reads these two package-level values and nothing else. No
// database is involved in building a router: the handlers are registered,
// not called.
config.ApplicationConfig.Mode = "dev"
config.JwtConfig.Secret = "test-secret-for-the-router-build"
type observation struct {
ran int
engineWas interface{}
engineSeen bool
}
var phase, before observation
sdk.Runtime.SetPhase(runtime.BeforeRouter, func() {
phase.ran++
phase.engineWas = sdk.Runtime.GetEngine()
phase.engineSeen = true
})
sdk.Runtime.SetBefore(func() {
before.ran++
before.engineWas = sdk.Runtime.GetEngine()
before.engineSeen = true
})
buildRouter()
if phase.ran != 1 {
t.Fatalf("BeforeRouter ran %d times, want 1", phase.ran)
}
if !phase.engineSeen || phase.engineWas != nil {
t.Errorf("BeforeRouter saw engine %v, want nil: it is meant to run before initRouter builds one", phase.engineWas)
}
if before.ran != 1 {
t.Fatalf("the before registry ran %d times, want 1", before.ran)
}
if before.engineWas == nil {
t.Error("a before callback saw no engine; that registry is meant to run after initRouter, and describing it as equivalent to BeforeRouter is the error this asserts against")
}
if sdk.Runtime.GetEngine() == nil {
t.Error("buildRouter returned with no engine built")
}
}
+167
View File
@@ -0,0 +1,167 @@
package api
import (
"strings"
"sync"
"testing"
"time"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
)
// recordingQueue records what was done to it, in order. Register and Run are
// the two calls whose order is the point of this file; Append and Shutdown are
// here to satisfy the interface.
type recordingQueue struct {
mu sync.Mutex
events []string
ran chan struct{}
}
func newRecordingQueue() *recordingQueue {
return &recordingQueue{ran: make(chan struct{}, 4)}
}
func (q *recordingQueue) record(e string) {
q.mu.Lock()
q.events = append(q.events, e)
q.mu.Unlock()
}
func (q *recordingQueue) seen() []string {
q.mu.Lock()
defer q.mu.Unlock()
return append([]string(nil), q.events...)
}
func (q *recordingQueue) String() string { return "recording" }
func (q *recordingQueue) Append(corestorage.Messager) error { return nil }
func (q *recordingQueue) Register(name string, _ corestorage.ConsumerFunc) {
q.record("register:" + name)
}
func (q *recordingQueue) Shutdown() {}
func (q *recordingQueue) Run() {
q.record("run")
select {
case q.ran <- struct{}{}:
default:
}
}
// waitForRun waits for Run, which is started on a goroutine.
func (q *recordingQueue) waitForRun(t *testing.T) {
t.Helper()
select {
case <-q.ran:
case <-time.After(5 * time.Second):
t.Fatalf("Run was never called; saw: %v", q.seen())
}
}
// The consumers must be registered before the queue is started. A queue that
// is already running refuses further registration - the contract
// implementations answer storage.ErrQueueAlreadyStarted - and the legacy
// adapter this path goes through drops that error, so the wrong order loses
// consumers with nothing said about it. The memory backend does not care,
// which is exactly why this cannot be left to be noticed in use.
func TestConsumersAreRegisteredBeforeTheQueueIsStarted(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
q := newRecordingQueue()
attachConsumersOnce(1, q)
q.waitForRun(t)
seen := q.seen()
runAt := -1
registers := 0
for i, e := range seen {
switch {
case e == "run":
if runAt < 0 {
runAt = i
}
case strings.HasPrefix(e, "register:"):
registers++
if runAt >= 0 {
t.Errorf("%q came after Run; a running queue refuses registration", e)
}
}
}
if registers != 3 {
t.Errorf("registered %d consumers, want 3; saw %v", registers, seen)
}
if runAt < 0 {
t.Errorf("the queue was never started; saw %v", seen)
}
}
// AfterResource runs again on every configuration reload, so the hook has to
// be idempotent with respect to a given queue - not "does nothing the second
// time". Registering twice on the same queue would give every message two
// consumers and write every log row twice.
func TestTheSameQueueIsNotGivenConsumersTwice(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
q := newRecordingQueue()
attachConsumersOnce(1, q)
q.waitForRun(t)
attachConsumersOnce(1, q)
// Nothing to wait for on the second call, so give a wrong implementation
// the time it would need to show up.
time.Sleep(200 * time.Millisecond)
if n := len(q.seen()); n != 4 {
t.Errorf("%d calls after attaching twice to the same queue, want 4 (3 registers + 1 run); saw %v", n, q.seen())
}
}
// The other half of the same rule: a reload builds a new adapter, and the
// consumers on the old one are attached to a queue nobody publishes to any
// more. A new generation must get its own set.
func TestANewQueueGetsItsOwnConsumers(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
first := newRecordingQueue()
attachConsumersOnce(1, first)
first.waitForRun(t)
second := newRecordingQueue()
attachConsumersOnce(2, second)
second.waitForRun(t)
if n := len(second.seen()); n != 4 {
t.Errorf("the queue from the second generation saw %d calls, want 4; saw %v", n, second.seen())
}
if n := len(first.seen()); n != 4 {
t.Errorf("the queue from the first generation saw %d calls, want 4 - it should not have been touched again; saw %v", n, first.seen())
}
}
// Generation 0 means the configuration has no queue section at all, so nothing
// was installed and the runtime hands back its own memory queue. That case
// still has to get consumers - the registration it replaces was unconditional,
// and dropping it would stop the login and operation logs for anyone who
// commented the section out.
func TestAnUnconfiguredQueueStillGetsConsumers(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
q := newRecordingQueue()
attachConsumersOnce(0, q)
q.waitForRun(t)
if n := len(q.seen()); n != 4 {
t.Errorf("an unconfigured queue saw %d calls, want 4; saw %v", n, q.seen())
}
// And still only once.
attachConsumersOnce(0, q)
time.Sleep(200 * time.Millisecond)
if n := len(q.seen()); n != 4 {
t.Errorf("generation 0 was attached to twice: %d calls, want 4; saw %v", n, q.seen())
}
}
+117
View File
@@ -0,0 +1,117 @@
package api
import (
"context"
"fmt"
"sort"
"strings"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"gorm.io/gorm"
"go-admin/cmd/migrate/migration"
"go-admin/common/health"
commonmodels "go-admin/common/models"
)
// schemaCheckName is what a failing schema reports itself as in /ready's body.
const schemaCheckName = "schema"
// registerSchemaCheck adds the pending-migration check to readiness.
//
// Readiness rather than a refusal to start, and rather than a log line alone.
// The two probes answer different questions: liveness is "restart me", and a
// process whose database is on the wrong schema comes back to the same schema,
// so restarting is not the answer. Readiness is "send me requests", and with a
// schema the binary does not match the answer is no.
//
// Issue #919 is what the absence of this looked like: the process started,
// both probes passed, and the first sign of trouble was a login failing with a
// driver-level encoding error. Refusing to start would have been the wrong fix
// - a process that exits tells an operator less than one that runs and says
// why, and under an orchestrator it crash-loops - while a log line alone is
// not something an orchestrator can act on.
func registerSchemaCheck() {
health.Register(schemaCheckName, schemaCheck)
}
// schemaCheck fails while any tenant database is behind the migrations this
// binary registers.
//
// Any one of them, rather than only the tenant being served: migrations are
// applied to every database in one run, so one database behind means that run
// did not finish. Serving the rest would let a half-applied deploy look like a
// partial success.
//
// Evaluated per request rather than decided at start-up, so that running
// migrate clears it without a restart.
func schemaCheck(ctx context.Context) error {
registered := migration.RegisteredVersions()
if len(registered) == 0 {
// Nothing registered means nothing can be pending, which is the honest
// answer for a tree with no migrations. It is also what a broken build
// would produce - the registry is filled by init() in packages the
// binary has to link - so cmd/api's dependency test asserts the real
// binary links them.
return nil
}
behind := make([]string, 0, 2)
for name, db := range sdk.Runtime.GetAllDb() {
applied, err := appliedVersions(ctx, db)
if err != nil {
return fmt.Errorf("reading applied migrations for %q: %w", name, err)
}
if pending := pendingVersions(registered, applied); len(pending) > 0 {
behind = append(behind, fmt.Sprintf("%s is %d behind, first pending %s",
name, len(pending), pending[0]))
}
}
if len(behind) == 0 {
return nil
}
sort.Strings(behind)
return fmt.Errorf("%s; run `go-admin migrate -c <config>` and see `go-admin migrate status`",
strings.Join(behind, "; "))
}
// appliedVersions reads what sys_migration records for one database.
//
// A missing table is not an error: a database that has never been migrated has
// applied nothing, which is exactly what the caller needs to hear, and is the
// state a first deploy is in.
func appliedVersions(ctx context.Context, db *gorm.DB) (map[string]bool, error) {
if db == nil {
return nil, fmt.Errorf("no database")
}
db = db.WithContext(ctx)
if !db.Migrator().HasTable(&commonmodels.Migration{}) {
return map[string]bool{}, nil
}
var rows []commonmodels.Migration
if err := db.Select("version").Find(&rows).Error; err != nil {
return nil, err
}
out := make(map[string]bool, len(rows))
for _, r := range rows {
out[r.Version] = true
}
return out, nil
}
// pendingVersions returns the registered versions applied does not contain.
//
// Split out and taking both sides as arguments because the registry is
// process-wide and filled by init() in packages cmd/api does not import: a
// test in this package cannot arrange it, so the arranging part is the part
// that is not tested here.
func pendingVersions(registered []string, applied map[string]bool) []string {
out := make([]string, 0)
for _, v := range registered {
if !applied[v] {
out = append(out, v)
}
}
sort.Strings(out)
return out
}
+124
View File
@@ -0,0 +1,124 @@
package api
import (
"context"
"os/exec"
"strings"
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
commonmodels "go-admin/common/models"
)
func TestPendingVersionsReportsOnlyWhatIsNotApplied(t *testing.T) {
registered := []string{"1000_a", "2000_b", "3000_c"}
applied := map[string]bool{"1000_a": true, "3000_c": true}
got := pendingVersions(registered, applied)
if len(got) != 1 || got[0] != "2000_b" {
t.Errorf("pending = %v, want [2000_b]", got)
}
}
func TestPendingVersionsIsEmptyWhenTheDatabaseIsCurrent(t *testing.T) {
registered := []string{"1000_a", "2000_b"}
applied := map[string]bool{"1000_a": true, "2000_b": true}
if got := pendingVersions(registered, applied); len(got) != 0 {
t.Errorf("pending = %v, want none", got)
}
}
// A row recorded that this binary no longer registers is not pending. It is
// the orphan `migrate status` already reports, and readiness has nothing to
// say about it: the schema is ahead, not behind, and requests will be served
// correctly.
func TestPendingVersionsIgnoresAppliedRowsNothingRegisters(t *testing.T) {
registered := []string{"1000_a"}
applied := map[string]bool{"1000_a": true, "9999_gone": true}
if got := pendingVersions(registered, applied); len(got) != 0 {
t.Errorf("pending = %v, want none - an orphaned row is not a pending migration", got)
}
}
func memoryDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file::memory:?cache=shared"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { db.Migrator().DropTable(&commonmodels.Migration{}) })
return db
}
// A first deploy has no sys_migration table. That is "nothing applied", not an
// error: reporting it as one would make the check fail for a reason the
// operator cannot act on, on the one deployment where every migration really
// is pending.
func TestAppliedVersionsTreatsAMissingTableAsNothingApplied(t *testing.T) {
db := memoryDB(t)
db.Migrator().DropTable(&commonmodels.Migration{})
got, err := appliedVersions(context.Background(), db)
if err != nil {
t.Fatalf("appliedVersions: %v", err)
}
if len(got) != 0 {
t.Errorf("applied = %v, want empty", got)
}
}
func TestAppliedVersionsReadsWhatTheTableHolds(t *testing.T) {
db := memoryDB(t)
if err := db.AutoMigrate(&commonmodels.Migration{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
db.Create(&commonmodels.Migration{Version: "1000_a"})
db.Create(&commonmodels.Migration{Version: "2000_b"})
got, err := appliedVersions(context.Background(), db)
if err != nil {
t.Fatalf("appliedVersions: %v", err)
}
if !got["1000_a"] || !got["2000_b"] || len(got) != 2 {
t.Errorf("applied = %v, want the two rows written", got)
}
}
// The check is only worth anything if the registry it reads is populated in
// the binary that serves requests, and it is filled by init() in packages
// cmd/api does not import - cmd/migrate blank-imports them, and cmd wires both
// subcommands into one binary.
//
// This cannot be asserted from an ordinary test: importing the version package
// to look at the registry would put it in the test binary's dependency graph
// and pass whatever the real binary links. So ask the build instead.
//
// Without this, dropping those blank imports leaves a check that reports
// "nothing pending" for every database forever, and every test above still
// passes.
func TestTheServingBinaryLinksTheMigrationRegistry(t *testing.T) {
out, err := exec.Command("go", "list", "-deps", "go-admin").Output()
if err != nil {
t.Skipf("go list unavailable: %v", err)
}
deps := string(out)
const versions = "go-admin/cmd/migrate/migration/version"
if !strings.Contains(deps, versions+"\n") {
t.Errorf("the main package does not link %s, so the schema check would "+
"read an empty registry and report every database as current", versions)
}
// Negative control: a package the binary genuinely must not link, so that a
// `deps` that somehow contained everything would fail here rather than pass
// the assertion above for the wrong reason.
const notLinked = "go-admin/tools/checksilent"
if strings.Contains(deps, notLinked+"\n") {
t.Errorf("%s is in the binary's dependency closure, so this test cannot "+
"tell a real link from a query that matches anything", notLinked)
}
}
+437 -42
View File
@@ -2,10 +2,13 @@ package api
import (
"context"
"crypto/tls"
"fmt"
"net"
"net/http"
"os"
"os/signal"
"sync/atomic"
"syscall"
"time"
@@ -14,16 +17,21 @@ import (
log "github.com/go-admin-team/go-admin-core/v2/logger"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/api"
"github.com/go-admin-team/go-admin-core/v2/sdk/bootstrap"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"go-admin/app/admin/models"
"go-admin/app/admin/router"
"go-admin/app/jobs"
otherrouter "go-admin/app/other/router"
"go-admin/common/database"
"go-admin/common/global"
"go-admin/common/health"
common "go-admin/common/middleware"
"go-admin/common/middleware/handler"
"go-admin/common/storage"
@@ -60,30 +68,121 @@ func init() {
func setup() {
// 注入配置扩展项
config.ExtendConfig = &ext.ExtConfig
// Registered before the configuration is read. SetupConfig announces
// AfterResource as soon as the callbacks that build the resources have
// run, so a hook added after that call would miss the first round and the
// queue would have no consumers until somebody edited the config file.
sdk.Runtime.SetPhase(runtime.AfterResource, attachQueueConsumers)
// On AfterListen rather than on a bare goroutine from run(). Two reasons:
// the phase runs behind core's panic guard, which does not reach across a
// goroutine boundary - a panic while loading jobs used to take the whole
// process down with a stack that named this file - and the jobs it starts
// can call the API, which is only true once the socket is accepting.
sdk.Runtime.SetPhase(runtime.AfterListen, startCronJobs)
// Registered before the configuration is read, because it registers a
// callback rather than reading anything: the check runs per request and
// asks the databases that exist then.
registerSchemaCheck()
//1. 读取配置
config.Setup(
bootstrap.SetupConfig(
file.NewSource(file.WithPath(configYml)),
database.Setup,
storage.Setup,
)
//注册监听函数
queue := sdk.Runtime.GetQueuePrefix("")
queue.Register(global.LoginLog, models.SaveLoginLog)
queue.Register(global.OperateLog, models.SaveOperaLog)
queue.Register(global.ApiCheck, models.SaveSysApi)
go queue.Run()
usageStr := `starting api server...`
log.Info(usageStr)
}
// startCronJobs registers the job implementations and starts a scheduler for
// every tenant database.
//
// It is synchronous, like the phase that runs it. jobs.Setup returns now that
// the `select {}` at the end of its per-tenant setup is gone, which is what
// makes that possible; while it was there this could only be a goroutine, and
// a goroutine is outside the panic guard.
func startCronJobs() {
jobs.InitJob()
jobs.Setup(sdk.Runtime.GetAllDb())
}
// attachedQueue is the queue generation the consumers are attached to, plus
// one, so that the zero value means "attached to nothing yet". Written from
// the goroutine running the phase, read from the next one - rounds never
// overlap, but they are not the same goroutine.
var attachedQueue atomic.Uint64
// attachQueueConsumers registers the log consumers against the queue that is
// current, and starts it.
//
// It runs on AfterResource, so it runs again after every configuration reload
// - and it has to. A reload rebuilds the queue adapter, and consumers
// registered against the one that existed at start-up are attached to an
// adapter nobody publishes to any more, so the login and operation logs stop
// being written with nothing said about it.
//
// It is therefore idempotent with respect to a given queue rather than "does
// nothing the second time": a new adapter gets a fresh set of consumers, the
// same one gets none. Registering twice on the same queue would give every
// message two consumers and write every log row twice.
//
// Generation 0 means the configuration has no queue section, so nothing was
// installed and GetQueuePrefix hands back the runtime's own memory queue.
// That case still gets consumers - it is what the previous unconditional
// registration did, and dropping it would silently stop logging for anyone who
// commented the section out - it just never gets them twice.
func attachQueueConsumers() {
attachConsumersOnce(storage.QueueGeneration(), sdk.Runtime.GetQueuePrefix(""))
}
// attachConsumersOnce puts the log consumers on q and starts it, unless gen
// says this queue already has them.
//
// Split out from attachQueueConsumers so that the order and the once-ness can
// be checked against a queue the test controls: the sequence that matters here
// cannot be read back out of a real adapter.
func attachConsumersOnce(gen uint64, q corestorage.AdapterQueue) {
if attachedQueue.Load() == gen+1 {
return
}
attachedQueue.Store(gen + 1)
//注册监听函数
q.Register(global.LoginLog, models.SaveLoginLog)
q.Register(global.OperateLog, models.SaveOperaLog)
q.Register(global.ApiCheck, models.SaveSysApi)
// Started only now, and by whoever registered. setupQueue deliberately
// leaves it stopped: a queue that is already running refuses further
// registration, and the adapter in this path drops that error on the
// floor, so starting first loses consumers without a word.
go q.Run()
}
func run() error {
// Resolved first, and used both for the line it prints and for the
// shutdown that spends it. Reading the configuration again at signal time
// would let the two disagree, and the sum that gets printed is the whole
// point of printing it.
//
// Refused rather than corrected, and refused before anything is built: a
// budget that cannot be spent as written is a configuration error, and the
// moment to say so is while nothing depends on this process yet.
seconds, err := ext.ExtConfig.Shutdown.Budget()
if err != nil {
return err
}
reportShutdownBudget(seconds)
if config.ApplicationConfig.Mode == pkg.ModeProd.String() {
gin.SetMode(gin.ReleaseMode)
}
initRouter()
runStartupHooks()
buildRouter()
reportGeneratorWriteRoutes()
srv := &http.Server{
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
@@ -92,12 +191,6 @@ func run() error {
WriteTimeout: time.Duration(config.ApplicationConfig.WriterTimeout) * time.Second,
}
go func() {
jobs.InitJob()
jobs.Setup(sdk.Runtime.GetAllDb())
}()
if apiCheck {
var routers = sdk.Runtime.GetRouter()
q := sdk.Runtime.GetQueuePrefix("")
@@ -122,18 +215,10 @@ func run() error {
// arming is separate from waiting.
quit, disarmStopSignals := armStopSignals()
go func() {
// 服务连接
if config.SslConfig.Enable {
if err := srv.ListenAndServeTLS(config.SslConfig.Pem, config.SslConfig.KeyStr); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Fatal("listen: ", err)
}
} else {
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
log.Fatal("listen: ", err)
}
}
}()
if err := startServing(srv, config.SslConfig.Enable, config.SslConfig.Pem, config.SslConfig.KeyStr); err != nil {
return err
}
fmt.Println(pkg.Red(string(global.LogoContent)))
tip()
fmt.Println(pkg.Green("Server run at:"))
@@ -145,26 +230,220 @@ func run() error {
fmt.Printf("%s Enter Control + C Shutdown Server \r\n", pkg.GetCurrentTimeStr())
<-quit
// Restored here, not deferred: from this point a second signal must reach
// the default handler, so a shutdown that hangs can still be interrupted.
disarmStopSignals()
log.Info("Shutdown Server ... ")
if err := shutdownServer(srv, shutdownTimeout); err != nil {
serverErr, cleanupErr := gracefulShutdown(srv, quit, disarmStopSignals, budgetFrom(seconds))
if serverErr != nil {
// Not log.Fatal: that is an unconditional os.Exit(1), and Shutdown
// reports an error exactly when connections were still in flight -
// which is when the cleanup that follows matters most.
log.Error("Server Shutdown: ", err)
// which is when the cleanup that ran after it mattered most.
log.Error("Server Shutdown: ", serverErr)
}
if cleanupErr != nil {
log.Error("Cleanup: ", cleanupErr)
}
log.Info("Server exiting")
return nil
}
// shutdownTimeout is how long Shutdown waits for in-flight requests. It plus
// whatever cleanup follows has to stay inside the orchestrator's grace period
// - `docker stop` allows 10s by default before it sends SIGKILL.
const shutdownTimeout = 5 * time.Second
// budget is the three waits a shutdown spends, in the order it spends them.
type budget struct {
drain time.Duration
server time.Duration
cleanup time.Duration
}
// budgetFrom turns the resolved seconds into the durations the sequence waits
// on.
func budgetFrom(s ext.ShutdownBudget) budget {
return budget{
drain: time.Duration(s.Drain) * time.Second,
server: time.Duration(s.Server) * time.Second,
cleanup: time.Duration(s.Cleanup) * time.Second,
}
}
// defaultBudget is what a process with no extend.shutdown section spends.
func defaultBudget() budget {
return budget{drain: drainTimeout, server: shutdownTimeout, cleanup: cleanupTimeout}
}
// gracefulShutdown takes the process down in the order that gives something
// else a chance to notice first.
//
// The whole order lives here, and run() is not the only caller: the signal
// tests run this function rather than reproducing it. A test that reproduces a
// sequence asserts against its own copy and stays green while the sequence it
// was written for regresses.
//
// The caller has already taken the first signal off quit. quit is handed on
// because a second signal during the drain window ends the window early -
// somebody sending another kill wants this over with sooner - and because
// until the window is over that signal must not reach the default handler and
// kill the process outright.
//
// disarm is therefore called at the end of the window rather than on the first
// signal. After it, a second signal is handled by the default disposition
// again, which is the only way out of a Shutdown or a cleanup callback that
// never returns. Restoring it any earlier would put every ordinary shutdown
// inside that escape hatch for the whole length of the drain, where before
// this window existed only a hung callback could reach it.
//
// The two waits' errors are returned separately rather than logged: they fail
// for different reasons, and the caller decides what each is worth.
func gracefulShutdown(srv *http.Server, quit <-chan os.Signal, disarm func(), b budget) (serverErr, cleanupErr error) {
// Said before anything is taken apart. A configuration reload arriving in
// this window would otherwise re-run AfterResource - rebuilding the pool
// and the queue adapter, and re-registering consumers - on top of cleanup
// that has already run.
sdk.Runtime.BeginShutdown()
// Readiness fails from here, which is before the server stops accepting.
// That order is necessary and not sufficient: with nothing between this
// line and the listener closing, the two are microseconds apart and a
// poller on a multi-second interval sees the refused connection instead of
// the 503. The window below is what turns the order into something
// observable - extend.shutdown.drain, which is zero unless it is
// configured.
health.BeginDraining()
// Keep-alive off for the same window, and for the same reason. The server
// keeps connections alive while !disableKeepAlives && !shuttingDown(), and
// shuttingDown() is only set by Shutdown itself - so without this line
// every pooled connection stays open for the whole drain and is cut at the
// end of it anyway, which is the cost of the window without its benefit.
// This is the switch Shutdown flips, moved earlier by the window's length:
// answers now carry Connection: close, and the idle connections a balancer
// is holding are closed at once rather than when it next tries to use one.
srv.SetKeepAlivesEnabled(false)
drain(quit, b.drain)
// Restored here, not on the first signal: from this point a second signal
// must reach the default handler, so a shutdown that hangs can still be
// interrupted.
disarm()
log.Info("Shutdown Server ... ")
serverErr = shutdownServer(srv, b.server)
// Runs whether or not the wait above failed, and deliberately so: Shutdown
// reports an error exactly when connections were still in flight, which is
// when there is most left to clean up after.
cleanupErr = runShutdownHooks(b.cleanup)
log.Info("Server exiting")
return serverErr, cleanupErr
}
// drain keeps serving for d, or until another stop signal arrives.
//
// Requests are answered normally throughout. Refusing them would move the
// outage earlier rather than avoid it - the point of the window is that this
// instance is still able to work while whoever routes to it stops routing.
func drain(quit <-chan os.Signal, d time.Duration) {
if d <= 0 {
return
}
log.Infof("Draining for %s: still serving, /ready answers 503 from here", d)
timer := time.NewTimer(d)
defer timer.Stop()
select {
case <-quit:
log.Info("Second signal during the drain window, closing the listener now")
case <-timer.C:
}
}
// Reference stop grace periods, printed when nothing was configured to compare
// against. They are three times apart, which is why the check below needs a
// configured value rather than a constant of its own: a budget that overruns
// under one of them fits comfortably under the other.
const (
dockerStopGraceSeconds = 10
kubernetesGraceSeconds = 30
)
// reportGeneratorWriteRoutes says whether this process serves the code
// generator's writing endpoints, and to whom.
//
// The endpoints are gated on the mode, and the shipped configuration says dev -
// so the deployment most likely to be exposed is the one that changed nothing,
// and the one least likely to go looking. Silence there would leave the gate
// technically correct and practically useless.
//
// Nothing is said in demo mode. The routes are registered, but DemoEvn refuses
// all three by name, so a warning would describe an exposure that is not there.
func reportGeneratorWriteRoutes() {
if !generatorWriteRoutesNeedWarning() {
return
}
log.Warnf("the code generator's writing endpoints are served in mode %q: "+
"/api/v1/gen/{toproject,apitofile,todb} write Go and Vue source onto this host and rows "+
"into this database, and they are in CasbinExclude, so any account that can log in may "+
"call them. Set application.mode to prod or test on anything that is not a workstation, "+
"then restart: these routes were registered at start-up and a configuration reload does "+
"not rebuild them.",
config.ApplicationConfig.Mode)
}
// generatorWriteRoutesNeedWarning reports whether there is an exposure to warn
// about: the endpoints are served, and nothing else is refusing them.
//
// Split from the logging so the decision can be tested. A warning nobody can
// make fire is indistinguishable from no warning at all, and this one exists
// precisely for the case nobody is looking at.
func generatorWriteRoutesNeedWarning() bool {
return otherrouter.GenWriteRoutesEnabled() && config.ApplicationConfig.Mode != "demo"
}
// reportShutdownBudget states what a shutdown will spend and whether it fits.
//
// The sum is taken from the resolved values, not from the configuration file:
// a field left out of extend.shutdown still costs its default, so adding up
// what was written down understates the budget by exactly the fields nobody
// wrote.
func reportShutdownBudget(s ext.ShutdownBudget) {
log.Infof("shutdown budget: drain %ds + server %ds + cleanup %ds = %ds",
s.Drain, s.Server, s.Cleanup, s.Total())
if s.Grace <= 0 {
log.Infof("shutdown budget: extend.shutdown.grace is not set, so nothing is compared against it - "+
"for reference `docker stop` allows %ds and Kubernetes terminationGracePeriodSeconds defaults to %ds",
dockerStopGraceSeconds, kubernetesGraceSeconds)
return
}
if over := s.Overrun(); over > 0 {
// A minimum, not a target. This is somebody else's deployment under
// constraints this process cannot see, so the honest thing to state is
// how much is missing - the repository's own files are where there is
// standing to ask for headroom on top, and checksilent does that.
log.Warnf("shutdown budget of %ds does not fit inside the %ds of extend.shutdown.grace: "+
"SIGKILL arrives while the cleanup callbacks are still running, and the work they "+
"were about to finish is lost. It needs at least %ds more, or %ds less budget.",
s.Total(), s.Grace, over, over)
return
}
log.Infof("shutdown budget of %ds fits inside the %ds of extend.shutdown.grace", s.Total(), s.Grace)
}
// The budgets a shutdown spends when extend.shutdown configures nothing:
// drainTimeout keeps the process serving after the stop signal, then
// shutdownTimeout waits for in-flight requests, then cleanupTimeout is what
// the BeforeExit callbacks get.
//
// The seconds come from config, which is where an absent field falls back, so
// the default is one number rather than two that can drift apart.
//
// They are consumed one after the other, so their sum is what has to stay
// inside the orchestrator's grace period: `docker stop` allows 10s by default
// before it sends SIGKILL, and 0+5+3 leaves room for the process to finish
// returning. Raising one without lowering another buys nothing - the budget
// that runs out is the orchestrator's, and reportShutdownBudget is what says
// so at start-up.
var (
drainTimeout = time.Duration(ext.DefaultDrainSeconds) * time.Second
shutdownTimeout = time.Duration(ext.DefaultServerSeconds) * time.Second
cleanupTimeout = time.Duration(ext.DefaultCleanupSeconds) * time.Second
)
// armStopSignals registers for the stop signals and returns the channel they
// arrive on together with the function that restores the default disposition.
@@ -185,6 +464,66 @@ func armStopSignals() (<-chan os.Signal, func()) {
return quit, func() { signal.Stop(quit) }
}
// startServing binds srv.Addr, hands the listener to srv on its own goroutine,
// and announces AfterListen.
//
// The bind is done here rather than left to ListenAndServe, which binds on the
// goroutine that serves. That put the failure every deployment actually hits -
// "address already in use" - on a goroutine nobody was reading, so the banner
// went on to claim the server was up, and there would be no way to keep
// AfterListen from announcing a socket that does not exist. A hook there is
// promised a reachable port; the only way to keep that promise is for the bind
// to have already happened on this goroutine.
//
// AfterListen is announced synchronously. Running it in a goroutine to save the
// few milliseconds would let it overlap the shutdown: on a fast SIGTERM the
// cleanup callbacks could finish before the startup ones had.
//
// Both ways of failing to start are therefore checked before the announcement:
// the bind, and - with ssl enabled - the certificate.
func startServing(srv *http.Server, useTLS bool, pem, key string) error {
if useTLS {
// Read the certificate before anything is announced. ServeTLS reads
// these files itself, but on the serving goroutine - so a bad
// certificate used to surface after AfterListen had already promised a
// reachable port. Loading it here costs one extra read and moves the
// failure onto this goroutine, where run() can return it.
//
// ServeTLS still does the real work below rather than this handing it a
// tls.Listener: that is what sets up HTTP/2 negotiation, and taking it
// over here would quietly drop h2 for every TLS deployment.
if _, err := tls.LoadX509KeyPair(pem, key); err != nil {
return errors.Wrap(err, "tls certificate")
}
}
ln, err := net.Listen("tcp", srv.Addr)
if err != nil {
return errors.Wrap(err, "listen")
}
go func() {
// 服务连接
var err error
if useTLS {
err = srv.ServeTLS(ln, pem, key)
} else {
err = srv.Serve(ln)
}
if err != nil && !errors.Is(err, http.ErrServerClosed) {
// Still fatal, as it was. Neither the bind nor the certificate is
// among the errors that reach here any more - both are checked
// above, on the caller's goroutine. What is left is a serve that
// failed after the port was taken, and carrying on would park the
// process on <-quit with nothing serving.
log.Fatal("serve: ", err)
}
}()
sdk.Runtime.RunPhase(runtime.AfterListen)
return nil
}
// shutdownServer stops srv, giving in-flight requests up to timeout to finish.
//
// It returns the error instead of exiting on it. A caller that exits here skips
@@ -196,6 +535,34 @@ func shutdownServer(srv *http.Server, timeout time.Duration) error {
return srv.Shutdown(ctx)
}
// runShutdownHooks runs the BeforeExit callbacks with timeout to share.
//
// What the budget bounds is the wait, not the work. When it is gone RunShutdown
// stops waiting and returns; a callback that never looks at its context carries
// on until the process exits, and may leave a partial write behind. Go cannot
// cancel a function that does not check for cancellation, which is why the
// callbacks are handed a context at all.
func runShutdownHooks(timeout time.Duration) error {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
return sdk.Runtime.RunShutdown(ctx)
}
// buildRouter announces BeforeRouter, builds the engine, and then drains the
// startup registries.
//
// The order is the contract. BeforeRouter is the last point at which a module
// can still affect how routes are built, so it has to run while there is no
// engine yet. The before registry runStartupHooks drains is a different moment
// despite the name: those callbacks run after initRouter has built the engine.
// Two lines apart, and describing them as equivalent is a mistake this
// repository has already made once in writing.
func buildRouter() {
sdk.Runtime.RunPhase(runtime.BeforeRouter)
initRouter()
runStartupHooks()
}
// runStartupHooks runs the router registries and then the before callbacks.
//
// The package-level slice runs first and in its existing order, so a fork that
@@ -243,10 +610,38 @@ func initRouter() {
r.Use(handler.TlsHandler())
}
//r.Use(middleware.Metrics())
r.Use(common.Sentinel()).
r.Use(exemptProbes(common.Sentinel())).
Use(common.RequestId(pkg.TrafficKey)).
Use(api.SetRequestLogger)
common.InitMiddleware(r)
}
// probePaths are the two routes the rate limiter must not answer for.
var probePaths = map[string]bool{
otherrouter.APIPrefix + otherrouter.HealthPath: true,
otherrouter.APIPrefix + otherrouter.ReadyPath: true,
}
// exemptProbes wraps a middleware so the health and readiness routes skip it.
//
// The limiter is installed on the engine and the probes are routes like any
// other, so above the threshold they are answered with 429 as well. A liveness
// probe that collects 429s fails its threshold and the container is restarted,
// which takes capacity out of a deployment that is already short of it and
// pushes the rest closer to the threshold - the limiter working exactly as
// intended is what causes it. It is the argument common/health makes about
// restarting a process whose database is unreachable, applied to load.
//
// Wrapping rather than teaching the limiter about these paths: the limiter
// lives under common/, which may not import the package that registers them.
func exemptProbes(h gin.HandlerFunc) gin.HandlerFunc {
return func(c *gin.Context) {
if probePaths[c.FullPath()] {
c.Next()
return
}
h(c)
}
}
+139
View File
@@ -0,0 +1,139 @@
package api
import (
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/gin-gonic/gin"
otherrouter "go-admin/app/other/router"
"go-admin/common/health"
ext "go-admin/config"
)
// The seconds in the configuration and the durations the sequence waits on are
// two spellings of one budget, and only one of them is printed at start-up.
func TestBudgetFromSeconds(t *testing.T) {
got := budgetFrom(ext.ShutdownBudget{Drain: 10, Server: 5, Cleanup: 3})
want := budget{
drain: 10 * time.Second,
server: 5 * time.Second,
cleanup: 3 * time.Second,
}
if got != want {
t.Errorf("budgetFrom = %+v, want %+v", got, want)
}
}
// The package variables and config.Default*Seconds have to say the same thing.
// They are the same default written twice - once as durations for the shutdown
// and once as seconds for the fallback - and a deployment that configures
// nothing is entitled to one answer, not two.
func TestDefaultBudgetIsTheConfiguredFallback(t *testing.T) {
unconfigured, err := ext.Shutdown{}.Budget()
if err != nil {
t.Fatalf("the empty section did not resolve: %v", err)
}
if got, want := defaultBudget(), budgetFrom(unconfigured); got != want {
t.Errorf("defaultBudget = %+v, want the unconfigured budget %+v", got, want)
}
}
// The rate limiter must not answer for the probes.
//
// It is installed on the engine, so without this the probes are limited like
// any other route and answer 429 above the threshold. A liveness probe that
// collects 429s fails its threshold and the container is restarted - taking
// capacity out of a deployment that is already short of it and pushing the
// rest closer to the threshold. The limiter working exactly as designed is
// what would cause it.
//
// The stand-in rejects everything rather than being a real limiter: what is
// under test is which requests reach it, and a real one would need the traffic
// to cross a threshold before it said anything.
func TestTheProbesSkipTheRateLimiter(t *testing.T) {
gin.SetMode(gin.TestMode)
var reached []string
r := gin.New()
r.Use(exemptProbes(func(c *gin.Context) {
reached = append(reached, c.FullPath())
c.AbortWithStatus(http.StatusTooManyRequests)
}))
v1 := r.Group(otherrouter.APIPrefix)
otherrouter.RegisterMonitorRouter(v1)
v1.GET("/business", func(c *gin.Context) { c.Status(http.StatusOK) })
for _, tc := range []struct {
path string
limited bool
}{
{otherrouter.APIPrefix + otherrouter.HealthPath, false},
{otherrouter.APIPrefix + otherrouter.ReadyPath, false},
// Not a probe, and deliberately not exempt: the exemption is for the
// two routes an orchestrator acts on, not for everything under
// /api/v1 that happens to be unauthenticated.
{otherrouter.APIPrefix + "/metrics", true},
{otherrouter.APIPrefix + "/business", true},
} {
t.Run(tc.path, func(t *testing.T) {
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, tc.path, nil))
if tc.limited {
if w.Code != http.StatusTooManyRequests {
t.Errorf("answered %d, want the middleware's 429 - it was skipped for a route that is not a probe", w.Code)
}
return
}
if w.Code == http.StatusTooManyRequests {
t.Errorf("answered 429; a probe that can be rate-limited gets the container restarted under load")
}
})
}
// Said separately, because a probe could also answer 429 by itself: what
// has to be true is that the middleware never saw the request.
for _, p := range reached {
if probePaths[p] {
t.Errorf("the middleware ran for %s", p)
}
}
}
// /health has to stay 200 while draining, and it is the assertion most easily
// lost by accident: making the liveness probe follow the readiness flag reads
// like tidying up, and it turns every rolling restart into a kubelet-issued
// kill part-way through the drain.
func TestHealthStaysUpWhileDraining(t *testing.T) {
gin.SetMode(gin.TestMode)
r := gin.New()
v1 := r.Group(otherrouter.APIPrefix)
otherrouter.RegisterMonitorRouter(v1)
ask := func(path string) int {
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, path, nil))
return w.Code
}
if got := ask(otherrouter.APIPrefix + otherrouter.HealthPath); got != http.StatusOK {
t.Fatalf("/health answered %d before draining, want 200", got)
}
// Process-wide and one-way - nothing clears it - so this is the last thing
// in this package that may run in-process and care. Everything else that
// exercises draining does so in a child process of its own.
health.BeginDraining()
if got := ask(otherrouter.APIPrefix + otherrouter.HealthPath); got != http.StatusOK {
t.Errorf("/health answered %d while draining, want 200 - liveness is "+
"\"should I restart you\", and the answer during a drain is no", got)
}
if got := ask(otherrouter.APIPrefix + otherrouter.ReadyPath); got != http.StatusServiceUnavailable {
t.Errorf("/ready answered %d while draining, want 503", got)
}
}
+496 -49
View File
@@ -1,33 +1,65 @@
package api
import (
"context"
"fmt"
"io"
"net"
"net/http"
"os"
"os/exec"
"strconv"
"strings"
"sync"
"syscall"
"testing"
"time"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/sdk"
otherrouter "go-admin/app/other/router"
)
// The signal path cannot be exercised in-process: delivering a signal to the
// test binary would race with the test framework, and the disposition changes
// are global. So the test re-executes itself as a child, and the child runs the
// same armStopSignals / shutdownServer the server does.
// are global. So the test re-executes itself as a child, and the child runs
// gracefulShutdown - the same function run() runs, not a second copy of the
// sequence. A test that reproduces the sequence asserts against its own copy:
// move BeginDraining after the drain window and the process regresses while
// the test stays green, which is the failure mode this file exists to avoid.
//
// The child deliberately serves an empty http.Server rather than the real one:
// this repository's CI has no database (.github/workflows/go.yml runs neither
// MySQL nor a sqlite-tagged build), and none of what is under test needs one.
// The child serves the real probe routes on an http.Server of its own rather
// than the configured one: this repository's CI has no database
// (.github/workflows/go.yml runs neither MySQL nor a sqlite-tagged build), and
// none of what is under test needs one. /ready answers 503 either way - with
// no database its checks fail - so the assertions below are on the draining
// answer specifically, not on the status code alone.
const (
childEnv = "GO_ADMIN_SIGNAL_CHILD"
childStuckEnv = "GO_ADMIN_SIGNAL_CHILD_STUCK"
childHangConn = "GO_ADMIN_SIGNAL_CHILD_HANGCONN"
markerReady = "CHILD-READY"
markerSignal = "CHILD-SIGNAL"
markerShutdown = "CHILD-SHUTDOWN-OK"
markerExiting = "CHILD-EXITING"
childEnv = "GO_ADMIN_SIGNAL_CHILD"
childStuckEnv = "GO_ADMIN_SIGNAL_CHILD_STUCK"
childHangConn = "GO_ADMIN_SIGNAL_CHILD_HANGCONN"
childSlowCleanup = "GO_ADMIN_SIGNAL_CHILD_SLOWCLEANUP"
childDrainMS = "GO_ADMIN_SIGNAL_CHILD_DRAIN_MS"
markerAddr = "CHILD-ADDR"
markerReady = "CHILD-READY"
markerSignal = "CHILD-SIGNAL"
markerShutdown = "CHILD-SHUTDOWN-OK"
markerCleanup = "CHILD-CLEANUP-RAN"
markerTook = "CHILD-TOOK-NS"
markerExiting = "CHILD-EXITING"
)
// childPingRoute is an ordinary route, registered beside the probes so the
// window can be checked for what it promises: requests arriving inside it are
// served, not refused. Refusing them would move the outage earlier instead of
// avoiding it.
const childPingRoute = "/signal-test-ping"
var (
readyPath = otherrouter.APIPrefix + otherrouter.ReadyPath
healthPath = otherrouter.APIPrefix + otherrouter.HealthPath
pingPath = otherrouter.APIPrefix + childPingRoute
)
// TestSignalChild is the child process. It is skipped in a normal run.
@@ -36,6 +68,12 @@ func TestSignalChild(t *testing.T) {
t.Skip("child process entry point")
}
gin.SetMode(gin.TestMode)
engine := gin.New()
v1 := engine.Group(otherrouter.APIPrefix)
otherrouter.RegisterMonitorRouter(v1)
v1.GET(childPingRoute, func(c *gin.Context) { c.String(http.StatusOK, "pong") })
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
fmt.Println("listen:", err)
@@ -47,7 +85,7 @@ func TestSignalChild(t *testing.T) {
// finds nothing to wait for and returns immediately.
accepted := make(chan struct{}, 1)
srv := &http.Server{
Handler: http.NewServeMux(),
Handler: engine,
ConnState: func(_ net.Conn, state http.ConnState) {
if state == http.StateNew {
select {
@@ -59,6 +97,45 @@ func TestSignalChild(t *testing.T) {
}
go func() { _ = srv.Serve(ln) }()
// The budget the child spends. Nothing here calls bootstrap.SetupConfig, so
// with no environment set this is the budget of a deployment that
// configures no extend.shutdown section at all.
b := defaultBudget()
if ms := os.Getenv(childDrainMS); ms != "" {
n, err := strconv.Atoi(ms)
if err != nil {
fmt.Println("drain:", err)
os.Exit(4)
}
b.drain = time.Duration(n) * time.Millisecond
}
// A BeforeExit callback, registered the way a module would. What the tests
// below care about is whether it runs at all - after a Shutdown that
// failed, and after its own budget has been spent.
sdk.Runtime.SetShutdown(func(ctx context.Context) {
switch {
case os.Getenv(childStuckEnv) == "1":
// Stands in for a cleanup hook that never finishes. The point of
// restoring the signal disposition after the drain window is that
// a second signal still reaches the default handler and kills this.
time.Sleep(2 * time.Minute)
case os.Getenv(childSlowCleanup) == "1":
// Outlasts the budget on purpose, and does not consult ctx - which
// is the case the contract is explicit about: what the context
// bounds is the wait, not the work.
time.Sleep(2 * time.Second)
}
fmt.Println(markerCleanup)
_ = os.Stdout.Sync()
})
switch {
case os.Getenv(childStuckEnv) == "1":
b.cleanup = 2 * time.Minute
case os.Getenv(childSlowCleanup) == "1":
b.cleanup = 300 * time.Millisecond
}
// Arm before announcing readiness. Doing it the other way round leaves a
// window in which the parent's signal reaches the default handler and
// kills the child before any of this runs - which is exactly the failure
@@ -66,22 +143,14 @@ func TestSignalChild(t *testing.T) {
// accident.
quit, disarm := armStopSignals()
fmt.Println(markerAddr, ln.Addr().String())
fmt.Println(markerReady)
os.Stdout.Sync()
_ = os.Stdout.Sync()
sig := <-quit
disarm()
fmt.Println(markerSignal, sig)
os.Stdout.Sync()
_ = os.Stdout.Sync()
if os.Getenv(childStuckEnv) == "1" {
// Stand in for a cleanup hook that never finishes. The point of
// restoring the signal disposition is that a second signal still
// reaches the default handler and kills this.
time.Sleep(2 * time.Minute)
}
timeout := shutdownTimeout
if os.Getenv(childHangConn) == "1" {
// Dialled here, not at start-up. net/http stops counting a StateNew
// connection against Shutdown once it is more than five seconds old,
@@ -107,21 +176,29 @@ func TestSignalChild(t *testing.T) {
// only treats a StateNew connection as idle once it is more than five
// seconds old. A short budget makes the timeout deterministic without
// waiting out the real one.
timeout = 300 * time.Millisecond
b.server = 300 * time.Millisecond
}
if err := shutdownServer(srv, timeout); err != nil {
started := time.Now()
serverErr, cleanupErr := gracefulShutdown(srv, quit, disarm, b)
spent := time.Since(started)
if serverErr != nil {
// Deliberately not fatal, and deliberately not a bare return: the
// point is that whatever follows still runs.
fmt.Println("shutdown error:", err)
fmt.Println("shutdown error:", serverErr)
} else {
fmt.Println(markerShutdown)
}
if cleanupErr != nil {
fmt.Println("cleanup error:", cleanupErr)
}
fmt.Println(markerTook, spent.Nanoseconds())
fmt.Println(markerExiting)
os.Stdout.Sync()
_ = os.Stdout.Sync()
}
func startChild(t *testing.T, stuck bool, extraEnv ...string) (*exec.Cmd, *os.File, chan string) {
func startChild(t *testing.T, stuck bool, extraEnv ...string) (*exec.Cmd, chan string) {
t.Helper()
r, w, err := os.Pipe()
@@ -141,7 +218,7 @@ func startChild(t *testing.T, stuck bool, extraEnv ...string) (*exec.Cmd, *os.Fi
}
_ = w.Close()
lines := make(chan string, 64)
lines := make(chan string, 256)
go func() {
defer close(lines)
buf := make([]byte, 4096)
@@ -175,12 +252,13 @@ func startChild(t *testing.T, stuck bool, extraEnv ...string) (*exec.Cmd, *os.Fi
_, _ = cmd.Process.Wait()
_ = r.Close()
})
return cmd, r, lines
return cmd, lines
}
// await drains lines until one contains want, or the deadline passes. It
// returns everything it saw, so a failure says what the child actually did.
func await(t *testing.T, lines chan string, want string, d time.Duration) []string {
// returns everything it saw, so a failure says what the child actually did,
// and the matching line, so a marker can carry a value.
func await(t *testing.T, lines chan string, want string, d time.Duration) ([]string, string) {
t.Helper()
var seen []string
deadline := time.After(d)
@@ -192,7 +270,7 @@ func await(t *testing.T, lines chan string, want string, d time.Duration) []stri
}
seen = append(seen, l)
if strings.Contains(l, want) {
return seen
return seen, l
}
case <-deadline:
t.Fatalf("timed out waiting for %q; saw:\n%s", want, strings.Join(seen, "\n"))
@@ -200,6 +278,134 @@ func await(t *testing.T, lines chan string, want string, d time.Duration) []stri
}
}
// childAddr waits for the address the child is listening on.
func childAddr(t *testing.T, lines chan string) string {
t.Helper()
_, line := await(t, lines, markerAddr, 30*time.Second)
fields := strings.Fields(line)
return fields[len(fields)-1]
}
// took reads the nanoseconds gracefulShutdown spent, as the child measured
// them. Measured inside the child on purpose: the parent's own clock includes
// process scheduling, which is the noise the tightest assertion here cannot
// afford.
func took(t *testing.T, lines chan string, d time.Duration) time.Duration {
t.Helper()
_, line := await(t, lines, markerTook, d)
fields := strings.Fields(line)
ns, err := strconv.ParseInt(fields[len(fields)-1], 10, 64)
if err != nil {
t.Fatalf("unreadable %s line %q: %v", markerTook, line, err)
}
return time.Duration(ns)
}
// sample is one answer, or the refusal that replaced it.
type sample struct {
at time.Time
path string
// status is zero when the connection could not be made at all, which is
// what a closed listener looks like from outside.
status int
draining bool
// willClose is what the server answered about the connection: the header
// it sends is Connection: close, which the transport consumes and reports
// here rather than leaving in Response.Header.
willClose bool
}
// probe asks once, on a connection of its own.
//
// A new transport per request, because a connection opened before the signal
// can still be served after the listener is closed: reusing one would let this
// test pass against a shutdown that had already broken the listener. Keep-alive
// is left enabled so the server's own Connection: close is observable - a
// client that asked for close would get that header back either way, and the
// assertion would prove nothing.
func probe(addr, path string) sample {
tr := &http.Transport{}
defer tr.CloseIdleConnections()
c := &http.Client{Transport: tr, Timeout: 3 * time.Second}
s := sample{at: time.Now(), path: path}
resp, err := c.Get("http://" + addr + path)
if err != nil {
return s
}
defer func() { _ = resp.Body.Close() }()
body, _ := io.ReadAll(resp.Body)
s.status = resp.StatusCode
s.willClose = resp.Close
s.draining = strings.Contains(string(body), `"status":"draining"`)
return s
}
// watcher polls the child until it stops accepting connections, keeping every
// answer.
type watcher struct {
mu sync.Mutex
samples []sample
done chan struct{}
}
func watch(addr string, paths ...string) *watcher {
w := &watcher{done: make(chan struct{})}
go func() {
defer close(w.done)
for {
refused := false
for _, p := range paths {
s := probe(addr, p)
w.mu.Lock()
w.samples = append(w.samples, s)
w.mu.Unlock()
if s.status == 0 {
refused = true
}
}
if refused {
return
}
time.Sleep(20 * time.Millisecond)
}
}()
return w
}
// sawDraining reports whether /ready has answered "draining" yet.
func (w *watcher) sawDraining() bool {
w.mu.Lock()
defer w.mu.Unlock()
for _, s := range w.samples {
if s.path == readyPath && s.draining {
return true
}
}
return false
}
func (w *watcher) wait(t *testing.T, d time.Duration) []sample {
t.Helper()
select {
case <-w.done:
case <-time.After(d):
t.Fatal("the child never stopped accepting connections")
}
w.mu.Lock()
defer w.mu.Unlock()
return w.samples
}
func describe(samples []sample) string {
var b strings.Builder
for _, s := range samples {
fmt.Fprintf(&b, " %s %s -> %d draining=%v willClose=%v\n",
s.at.Format("15:04:05.000"), s.path, s.status, s.draining, s.willClose)
}
return b.String()
}
// Acceptance 19. Registering only os.Interrupt meant SIGTERM - the signal
// `docker stop`, Kubernetes and systemd all send - terminated the process
// before any of the shutdown path ran. Both must now reach it.
@@ -212,7 +418,7 @@ func TestBothSignalsRunTheShutdownPath(t *testing.T) {
{"SIGTERM", syscall.SIGTERM},
} {
t.Run(tc.name, func(t *testing.T) {
cmd, _, lines := startChild(t, false)
cmd, lines := startChild(t, false)
await(t, lines, markerReady, 30*time.Second)
if err := cmd.Process.Signal(tc.sig); err != nil {
@@ -234,8 +440,14 @@ func TestBothSignalsRunTheShutdownPath(t *testing.T) {
// without restoring the disposition a second signal only refills the buffer:
// once SIGTERM is registered, a shutdown that hangs could not be interrupted by
// anything short of SIGKILL.
//
// The hang is now a cleanup callback that never returns, which is where a
// shutdown actually hangs, and it is reached through gracefulShutdown - so this
// also pins where the disposition is restored. Restore it before the drain
// window and the window itself becomes the interruptible part; restore it never
// and this test hangs.
func TestASecondSignalStillKillsAStuckShutdown(t *testing.T) {
cmd, _, lines := startChild(t, true)
cmd, lines := startChild(t, true)
await(t, lines, markerReady, 30*time.Second)
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
@@ -243,21 +455,33 @@ func TestASecondSignalStillKillsAStuckShutdown(t *testing.T) {
}
await(t, lines, markerSignal, 10*time.Second)
// The child is now inside a cleanup that will not finish on its own.
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("second signal: %v", err)
}
// The child is now on its way into a cleanup that will not finish on its
// own. Signalled repeatedly rather than once: the marker is printed just
// before gracefulShutdown is entered, and the disposition is not restored
// until the drain window is over - zero seconds here, but not zero
// instructions - so a single signal sent immediately after the marker can
// still land in the buffered channel and be dropped. Which of them does
// the killing is not the assertion; that one of them can is.
done := make(chan error, 1)
go func() { done <- cmd.Wait() }()
select {
case err := <-done:
if err == nil {
t.Fatal("child exited cleanly; it was supposed to be killed by the second signal")
retry := time.NewTicker(200 * time.Millisecond)
defer retry.Stop()
deadline := time.After(15 * time.Second)
for {
select {
case err := <-done:
if err == nil {
t.Fatal("child exited cleanly; it was supposed to be killed by the second signal")
}
return
case <-retry.C:
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("second signal: %v", err)
}
case <-deadline:
t.Fatal("the second signal did not kill a stuck shutdown - the escape hatch is gone")
}
case <-time.After(15 * time.Second):
t.Fatal("the second signal did not kill a stuck shutdown - the escape hatch is gone")
}
}
@@ -266,7 +490,7 @@ func TestASecondSignalStillKillsAStuckShutdown(t *testing.T) {
// unconditional os.Exit(1). Everything after it, which is where the cleanup
// hooks will hang, never ran. A failed Shutdown must not end the process.
func TestShutdownTimeoutDoesNotStopWhatFollows(t *testing.T) {
cmd, _, lines := startChild(t, false, childHangConn+"=1")
cmd, lines := startChild(t, false, childHangConn+"=1")
await(t, lines, markerReady, 30*time.Second)
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
@@ -274,7 +498,7 @@ func TestShutdownTimeoutDoesNotStopWhatFollows(t *testing.T) {
}
await(t, lines, markerSignal, 10*time.Second)
seen := await(t, lines, markerExiting, 20*time.Second)
seen, _ := await(t, lines, markerExiting, 20*time.Second)
var timedOut bool
for _, l := range seen {
@@ -286,7 +510,230 @@ func TestShutdownTimeoutDoesNotStopWhatFollows(t *testing.T) {
t.Fatalf("Shutdown did not time out, so this test proves nothing; saw:\n%s",
strings.Join(seen, "\n"))
}
var cleaned bool
for _, l := range seen {
if strings.Contains(l, markerCleanup) {
cleaned = true
}
}
if !cleaned {
t.Fatalf("the BeforeExit callback did not run after a failed Shutdown; saw:\n%s",
strings.Join(seen, "\n"))
}
if err := cmd.Wait(); err != nil {
t.Fatalf("child exited with %v after a failed Shutdown, want a clean exit", err)
}
}
// A callback that outlasts its budget must not take the process with it, and
// must not be waited for: RunShutdown reports the deadline and returns, the
// callback carries on, and the process still exits cleanly. This is the half of
// the contract that is easy to get backwards - the context bounds the wait, not
// the work, because Go cannot cancel a function that does not check for it.
func TestACleanupThatOutlastsItsBudgetIsAbandonedNotAwaited(t *testing.T) {
cmd, lines := startChild(t, false, childSlowCleanup+"=1")
await(t, lines, markerReady, 30*time.Second)
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("signal: %v", err)
}
await(t, lines, markerSignal, 10*time.Second)
// The budget is 300ms and the callback sleeps two seconds. If RunShutdown
// waited for it, this marker would not arrive for two seconds; the one
// second here is what makes "abandoned, not awaited" the thing asserted.
seen, _ := await(t, lines, markerExiting, 1*time.Second)
var reported bool
for _, l := range seen {
if strings.Contains(l, "cleanup error:") {
reported = true
}
if strings.Contains(l, markerCleanup) {
t.Fatalf("the slow callback finished before the process moved on, so nothing was abandoned; saw:\n%s",
strings.Join(seen, "\n"))
}
}
if !reported {
t.Fatalf("RunShutdown returned no error for a callback that outlasted the budget; saw:\n%s",
strings.Join(seen, "\n"))
}
if err := cmd.Wait(); err != nil {
t.Fatalf("child exited with %v, want a clean exit despite the abandoned callback", err)
}
}
// The core acceptance: with a drain window configured, something outside the
// process can observe that this instance is draining, on a connection it opens
// after the signal, and can still be served while it does.
//
// Two windows rather than one. A single value proves only that something takes
// that long, which a hard-coded sleep anywhere in the sequence would satisfy;
// two say the wait is the configured one.
//
// What each answer is for:
//
// - /ready reporting "draining" is the window being observable at all. The
// status code alone would not say it: with no database configured the
// probe's own checks fail and 503 is also the answer before the signal.
// - The server refusing to keep those connections alive is the window being
// useful. It keeps them alive until Shutdown sets shuttingDown(), so
// without switching keep-alive off here a balancer's pool would sit
// untouched for the whole window and be cut at the end of it anyway. The
// header saying so is Connection: close; the transport consumes it and
// reports it as Response.Close, which is what a sample records.
// - /health staying 200 is the window not asking to be restarted, and the
// ordinary route staying 200 is the window not refusing work. Draining is
// "stop sending me new work", not "reject what arrives".
func TestTheDrainWindowIsObservableWhileStillServing(t *testing.T) {
for _, drain := range []time.Duration{300 * time.Millisecond, 1200 * time.Millisecond} {
t.Run(drain.String(), func(t *testing.T) {
cmd, lines := startChild(t, false,
fmt.Sprintf("%s=%d", childDrainMS, drain.Milliseconds()))
addr := childAddr(t, lines)
await(t, lines, markerReady, 30*time.Second)
w := watch(addr, readyPath, healthPath, pingPath)
// Long enough for a round of answers from a server that is not yet
// draining, which is what the keep-alive assertion below compares
// against.
time.Sleep(150 * time.Millisecond)
signalAt := time.Now()
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("signal: %v", err)
}
samples := w.wait(t, drain+30*time.Second)
spent := took(t, lines, 10*time.Second)
await(t, lines, markerExiting, 10*time.Second)
if spent < drain {
t.Errorf("the shutdown took %s, want at least the %s window", spent, drain)
}
var refusedAt = -1
for i, s := range samples {
if s.status == 0 {
refusedAt = i
break
}
}
if refusedAt < 0 {
t.Fatalf("the child never stopped accepting; saw:\n%s", describe(samples))
}
var keptAliveBefore, drainingInside, closedInside bool
for _, s := range samples[:refusedAt] {
switch s.path {
case readyPath:
if s.at.Before(signalAt) && !s.draining && !s.willClose {
keptAliveBefore = true
}
if s.at.After(signalAt) && s.draining {
drainingInside = true
if s.willClose {
closedInside = true
}
}
case healthPath, pingPath:
if s.status != http.StatusOK {
t.Errorf("%s answered %d before the listener closed, want 200;\n%s",
s.path, s.status, describe(samples))
}
}
}
if !keptAliveBefore {
t.Fatalf("no answer before the signal kept the connection alive, so the header assertion below proves nothing;\n%s",
describe(samples))
}
if !drainingInside {
t.Errorf("no answer inside the window reported draining; the flip and the closed listener were not far enough apart to observe;\n%s",
describe(samples))
}
if !closedInside {
t.Errorf("answers inside the window still kept the connection alive, so a pooled connection survives the whole window and is cut at the end of it anyway;\n%s",
describe(samples))
}
if err := cmd.Wait(); err != nil {
t.Fatalf("child exited with %v, want a clean exit", err)
}
})
}
}
// The default has to be no window at all: a process that configures no
// extend.shutdown section must shut down the way it did before the section
// existed.
//
// Asserted as a sequence rather than as a duration. How long a shutdown takes
// is decided by how much the cleanup callbacks have to do, so "as fast as
// before" is not falsifiable; "nothing was inserted between the signal and the
// listener closing" is.
func TestAnUnconfiguredShutdownAddsNoWindow(t *testing.T) {
cmd, lines := startChild(t, false)
await(t, lines, markerReady, 30*time.Second)
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("signal: %v", err)
}
await(t, lines, markerSignal, 10*time.Second)
spent := took(t, lines, 10*time.Second)
if spent > 100*time.Millisecond {
t.Errorf("an unconfigured shutdown spent %s between the signal and exiting; "+
"with no drain window and no cleanup callbacks it must be immediate", spent)
}
if err := cmd.Wait(); err != nil {
t.Fatalf("child exited with %v, want a clean exit", err)
}
}
// A second signal during the window ends it early rather than killing the
// process. Somebody sending another kill wants this over with sooner, and the
// answer to that is to stop draining - not to skip the cleanup, which is what
// the default disposition would do.
//
// This is the pair to TestASecondSignalStillKillsAStuckShutdown: the escape
// hatch has to be closed for the length of the window and open after it.
func TestASecondSignalEndsTheDrainWindowEarly(t *testing.T) {
// Long enough that the shutdown cannot plausibly have taken this long on
// its own, short enough that the test does not sit out the whole window
// when the early exit is missing - it fails on the reported duration
// instead of on a timeout, which says which of the two broke.
const window = 10 * time.Second
cmd, lines := startChild(t, false,
fmt.Sprintf("%s=%d", childDrainMS, window.Milliseconds()))
addr := childAddr(t, lines)
await(t, lines, markerReady, 30*time.Second)
w := watch(addr, readyPath)
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("first signal: %v", err)
}
deadline := time.Now().Add(15 * time.Second)
for !w.sawDraining() {
if time.Now().After(deadline) {
t.Fatal("the child never reported draining, so the second signal below would not land inside the window")
}
time.Sleep(20 * time.Millisecond)
}
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
t.Fatalf("second signal: %v", err)
}
spent := took(t, lines, window+20*time.Second)
if spent >= window {
t.Errorf("the window ran its full %s despite a second signal (%s); the signal was ignored", window, spent)
}
await(t, lines, markerExiting, 10*time.Second)
if err := cmd.Wait(); err != nil {
t.Fatalf("child exited with %v; a second signal inside the window must end the window, not the process", err)
}
}
+45
View File
@@ -0,0 +1,45 @@
package migrate
import (
"bytes"
"errors"
"strings"
"testing"
)
// A deployment decides whether to start the new version on this command's
// exit code. Before this batch the only failure that produced one was a
// failing migration function, and it produced it by ending the process from
// inside the migration engine; moving that out would have taken the last
// reported failure with it.
func TestExitOnErrorEndsTheCommandNonZero(t *testing.T) {
var codes []int
osExit = func(c int) { codes = append(codes, c) }
t.Cleanup(func() { osExit = origExit })
var out bytes.Buffer
exitOnError(&out, errors.New("the tenant database is unreachable"))
if len(codes) != 1 || codes[0] != 1 {
t.Errorf("exit codes = %v, want [1]", codes)
}
if !strings.Contains(out.String(), "the tenant database is unreachable") {
t.Errorf("the reason was not reported: %q", out.String())
}
}
func TestExitOnErrorLetsSuccessThrough(t *testing.T) {
var codes []int
osExit = func(c int) { codes = append(codes, c) }
t.Cleanup(func() { osExit = origExit })
var out bytes.Buffer
exitOnError(&out, nil)
if len(codes) != 0 {
t.Errorf("a successful migration exited with %v", codes)
}
if out.Len() != 0 {
t.Errorf("a successful migration wrote %q", out.String())
}
}
+472
View File
@@ -0,0 +1,472 @@
package migrate
import (
"errors"
"fmt"
"io"
"slices"
"sort"
"strings"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/app"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
)
// engine is the part of the migration engine the installer drives.
//
// An interface rather than *migration.Migration because the concrete type is
// a package-level singleton with no exported constructor, so a test that took
// it would be sharing one registry with every other test in the process.
type engine interface {
SetDb(*gorm.DB)
Status() ([]migration.StatusEntry, error)
MigrateApp(string) error
}
// installReport is what an install did, for the command to print.
type installReport struct {
Code string
// Version is the manifest version this run recorded.
Version string
// Previous is the version sys_app held before this run, empty when this
// is the first install.
Previous string
// Applied lists the versions this run brought in, in the order they were
// applied. Empty on a no-op, and also empty on a run that only corrected
// sys_app - the difference is NoOp.
Applied []string
// NoOp says nothing was left to do: the app is recorded as installed, at
// this same version, with no migration outstanding.
NoOp bool
}
// install brings one application up to the version its manifest declares.
//
// Three phases, each committing on its own. They are not one transaction and
// cannot be: an application's versions are separate migration files, and a
// DDL statement inside any of them commits the transaction around it on
// MySQL, which destroys an outer transaction and every savepoint taken from
// it. So this does not
// promise that a half-installed application cannot happen. It promises that
// one is visible when it does: phase A writes "installing" before anything
// that can fail, and phase C turns that into "installed" or "failed".
//
// What is left to apply comes from sys_migration, never from sys_app.
// sys_app is a derived view - a summary for a human, and the answer to "which
// version does this app think it is at". If it were the authority, then an
// operator who deleted sys_migration rows by hand would be told an app is
// installed while its schema is not, which is worse than not knowing.
func install(db *gorm.DB, eng engine, m app.Manifest) (installReport, error) {
code := migration.NormalizeAppCode(m.Code)
rep := installReport{Code: code, Version: m.Version}
if code == "" {
return rep, errors.New("the manifest declares no app code")
}
if code == migration.FrameworkAppCode {
// Installing the framework is what `migrate` is, and the framework
// has no manifest and no sys_app row. Saying so beats writing a row
// that nothing else in this batch expects to exist.
return rep, fmt.Errorf("%q is the framework's own migrations, not an application; run `migrate` for those", code)
}
if !db.Migrator().HasTable(&adminmodels.SysApp{}) {
return rep, errors.New("sys_app does not exist; run `migrate` first to bring the framework's own tables up to date")
}
eng.SetDb(db)
row, found, err := loadApp(db, code)
if err != nil {
return rep, err
}
// sameVersion is only meaningful when found; it stays false otherwise.
// The comparison happens here, before phase A, so an unparseable
// recorded version is refused while it is still readable rather than
// after being overwritten.
sameVersion := false
if found {
rep.Previous = row.Version
cmp, err := app.Compare(m.Version, row.Version)
if err != nil {
return rep, fmt.Errorf("comparing %s against the recorded %s: %w", m.Version, row.Version, err)
}
if cmp < 0 {
return rep, fmt.Errorf("%s is recorded at %s; installing %s would be a downgrade, which is not supported",
code, row.Version, m.Version)
}
sameVersion = cmp == 0
}
if err := requiresInstalled(db, code, m); err != nil {
return rep, err
}
pending, err := pendingFor(eng, code)
if err != nil {
return rep, err
}
// Nothing outstanding, recorded as installed, at this same version. All
// three, and the first one comes from sys_migration: a row that says
// installed while a migration of its has never run is exactly the case
// sys_app must not be believed about. AppInstalling is not installed -
// it is what a row reads as after the process was killed partway.
if found && sameVersion && row.Status == adminmodels.AppInstalled && len(pending) == 0 {
rep.NoOp = true
return rep, nil
}
// Phase A: the attempt is on disk before anything that can fail.
now := time.Now()
if err := beginInstall(db, &row, m, code, found, now); err != nil {
return rep, err
}
// Phase B: no atomicity across these, by the nature of the thing.
runErr := eng.MigrateApp(code)
// Phase C.
if runErr != nil {
failed := ""
var vf *migration.VersionFailure
if errors.As(runErr, &vf) {
failed = vf.Version
}
if err := markFailed(db, code, failed, runErr, time.Now()); err != nil {
return rep, errors.Join(runErr, fmt.Errorf("recording the failure on sys_app: %w", err))
}
return rep, runErr
}
if err := markInstalled(db, code, row.InstalledAt, time.Now()); err != nil {
return rep, err
}
rep.Applied = pending
return rep, nil
}
// loadApp reads the sys_app row for code. A missing row is not an error: it
// is what a first install looks like.
func loadApp(db *gorm.DB, code string) (adminmodels.SysApp, bool, error) {
var row adminmodels.SysApp
err := db.Where("app_code = ?", code).First(&row).Error
if err == nil {
return row, true, nil
}
if errors.Is(err, gorm.ErrRecordNotFound) {
return adminmodels.SysApp{}, false, nil
}
return adminmodels.SysApp{}, false, fmt.Errorf("reading sys_app for %q: %w", code, err)
}
// requiresInstalled refuses an install whose declared dependencies are not
// installed, and names the ones that are not.
//
// It does not install them. "Install this application" would otherwise mean
// "and everything it happens to name, and everything those name" - a blast
// radius the operator did not ask for and cannot see before it happens. What
// they get instead is a list and the order to do it in.
//
// An unfinished or failed dependency counts as missing, and says which it is:
// "not installed" sends someone to install it, "did not finish" sends them to
// look at why.
func requiresInstalled(db *gorm.DB, code string, m app.Manifest) error {
if len(m.Requires) == 0 {
return nil
}
apps, err := loadApps(db)
if err != nil {
return err
}
var why, what []string
for _, req := range m.Requires {
want := migration.NormalizeAppCode(req)
if want == "" {
continue
}
var reason string
switch row, ok := apps[want]; {
case !ok:
reason = "not installed"
case row.Status == adminmodels.AppFailed:
reason = "its install failed"
case row.Status == adminmodels.AppInstalling:
reason = "its install did not finish"
case row.Status != adminmodels.AppInstalled:
// A status this binary has no name for. Saying so beats the
// catch-all this used to be, which read any future value as
// "did not finish" - a sentence that would be wrong for
// whatever reason the value was added.
reason = fmt.Sprintf("its status is %d, which this binary does not recognise", row.Status)
default:
continue
}
why = append(why, want+" ("+reason+")")
what = append(what, want)
}
if len(why) > 0 {
return fmt.Errorf("%s requires %s; install %s first",
code, strings.Join(why, ", "), strings.Join(what, " and "))
}
return nil
}
// refuseOnDependencyCycle reports a cycle anywhere in the registered
// manifests, whether or not the application being installed is part of it.
//
// Over the whole set rather than one application's closure, because a cycle
// between two applications neither of which is the one being installed is
// still an authoring mistake, and finding it the day somebody happens to
// install into it - with an error naming two applications they did not ask
// for - is the worse time to find it.
//
// Requires naming an application that is not registered is not a cycle and
// not reported here; that is requiresInstalled's answer to give, against the
// database, at the time it matters.
func refuseOnDependencyCycle(manifests map[string]app.Manifest) error {
const (
white = 0 // not visited
grey = 1 // on the current path
black = 2 // finished
)
colour := make(map[string]int, len(manifests))
codes := make([]string, 0, len(manifests))
for code := range manifests {
codes = append(codes, code)
}
// Sorted, so the same set of manifests always reports the same cycle
// rather than whichever one the map happened to hand over first.
sort.Strings(codes)
var path []string
var walk func(code string) error
walk = func(code string) error {
switch colour[code] {
case grey:
// Trim the path to where this code first appears, so the error
// is the cycle and not the walk that reached it. grey is only
// ever set together with the append below, and cleared together
// with the matching trim, so the code is always on the path.
cycle := append(slices.Clone(path[slices.Index(path, code):]), code)
return fmt.Errorf("the declared dependencies form a cycle: %s",
strings.Join(cycle, " -> "))
case black:
return nil
}
colour[code] = grey
path = append(path, code)
// In the order the manifest declared them, which is a fixed order
// already - sorting here would only make the reported cycle harder
// to line up against the manifest that caused it. The determinism
// that matters comes from the sorted outer loop, because that one
// walks a map.
for _, r := range manifests[code].Requires {
n := migration.NormalizeAppCode(r)
if _, registered := manifests[n]; !registered {
// Including the empty string, which Register rejects, so
// no manifest is filed under it.
continue
}
if err := walk(n); err != nil {
return err
}
}
path = path[:len(path)-1]
colour[code] = black
return nil
}
for _, code := range codes {
if err := walk(code); err != nil {
return err
}
}
return nil
}
// loadApps reads every sys_app row, keyed by app code.
//
// A database that has never had 1786700007000 applied has no such table, and
// that is not an error here: `migrate status` has to keep working on a
// database that has not been migrated at all, which is when it is most wanted.
// A nil map is the honest answer there, and the caller prints what it always
// printed.
func loadApps(db *gorm.DB) (map[string]adminmodels.SysApp, error) {
if !db.Migrator().HasTable(&adminmodels.SysApp{}) {
return nil, nil
}
var rows []adminmodels.SysApp
if err := db.Find(&rows).Error; err != nil {
return nil, fmt.Errorf("reading sys_app: %w", err)
}
out := make(map[string]adminmodels.SysApp, len(rows))
for _, r := range rows {
// An application cannot be filed under the empty code or the one
// reserved for the framework - Register rejects both - so a row
// carrying either was not written by an install. Dropping it here
// is the one place that settles it: every reader of this map would
// otherwise have to decide separately, and `migrate status` would
// merge such a row into the framework's own group.
if r.AppCode == "" || r.AppCode == migration.FrameworkAppCode {
continue
}
out[r.AppCode] = r
}
return out, nil
}
// pendingFor is the authoritative answer to "what is left to apply", and it
// is recomputed every time rather than stored: what is registered in this
// process, minus what sys_migration says has run. sys_app.failed_version is a
// snapshot of what this returned once and may be stale by now; nothing may
// read it to decide this.
func pendingFor(eng engine, code string) ([]string, error) {
entries, err := eng.Status()
if err != nil {
return nil, err
}
var out []string
for _, e := range entries {
if e.AppCode == code && e.Registered && !e.Applied {
out = append(out, e.Version)
}
}
sort.Strings(out)
return out, nil
}
// beginInstall is phase A. It refreshes every descriptive column from the
// manifest, because those are the manifest's to say and the row is only a
// copy, and it clears the two diagnostic columns so a stale failure from a
// previous attempt cannot be read as this one's.
func beginInstall(db *gorm.DB, row *adminmodels.SysApp, m app.Manifest, code string, found bool, now time.Time) error {
row.AppCode = code
row.Name = m.Name
row.Version = m.Version
row.Description = m.Description
row.Author = m.Author
row.Requires = strings.Join(m.Requires, ",")
row.Pricing = m.Pricing
row.License = m.License
row.Status = adminmodels.AppInstalling
row.FailedVersion = ""
row.LastError = ""
row.UpdatedAt = now
if !found {
if err := db.Create(row).Error; err != nil {
return fmt.Errorf("recording the install attempt for %q: %w", code, err)
}
return nil
}
if err := db.Save(row).Error; err != nil {
return fmt.Errorf("recording the install attempt for %q: %w", code, err)
}
return nil
}
// markInstalled is the success half of phase C. installed_at is set once and
// never moved: an upgrade keeps the time of the first install, which is what
// the column is for.
//
// Computed here rather than with COALESCE so the statement is the same on all
// four drivers this repository supports.
func markInstalled(db *gorm.DB, code string, installedAt *time.Time, now time.Time) error {
updates := map[string]any{
"status": adminmodels.AppInstalled,
"updated_at": now,
}
if installedAt == nil {
updates["installed_at"] = now
}
err := db.Model(&adminmodels.SysApp{}).Where("app_code = ?", code).Updates(updates).Error
if err != nil {
return fmt.Errorf("recording %q as installed: %w", code, err)
}
return nil
}
// markFailed is the other half. Both columns it writes are diagnostic text
// for whoever reads the row; no code may branch on either one.
func markFailed(db *gorm.DB, code, failedVersion string, cause error, now time.Time) error {
updates := map[string]any{
"status": adminmodels.AppFailed,
"failed_version": truncate(failedVersion, 64),
"last_error": truncate(cause.Error(), 255),
"updated_at": now,
}
return db.Model(&adminmodels.SysApp{}).Where("app_code = ?", code).Updates(updates).Error
}
// truncate cuts s to at most n runes, not bytes: these columns are declared in
// characters, and a message that is partly Chinese would otherwise be cut in
// the middle of one and stored as an invalid sequence.
func truncate(s string, n int) string {
r := []rune(s)
if len(r) <= n {
return s
}
return string(r[:n])
}
// reportInstall prints what happened, and says that the data is in place but
// the code is not.
//
// That last sentence is not a pleasantry. Go links its applications at build
// time and Vite resolves its import globs at build time, so installing an
// application writes its menus, its APIs and its permissions and cannot make
// one line of its code run. An operator who is not told that sees the menus
// appear and reasonably concludes the thing is live.
func reportInstall(w io.Writer, rep installReport) {
if rep.NoOp {
fmt.Fprintf(w, "%s %s is already installed; nothing to do\n", rep.Code, rep.Version)
return
}
switch {
case rep.Previous == "":
fmt.Fprintf(w, "installed %s %s\n", rep.Code, rep.Version)
case rep.Previous == rep.Version:
fmt.Fprintf(w, "brought %s %s the rest of the way\n", rep.Code, rep.Version)
default:
fmt.Fprintf(w, "upgraded %s from %s to %s\n", rep.Code, rep.Previous, rep.Version)
}
if len(rep.Applied) > 0 {
fmt.Fprintf(w, "applied %d migration(s): %s\n", len(rep.Applied), strings.Join(rep.Applied, ", "))
} else {
fmt.Fprintln(w, "no migration was outstanding; only sys_app was brought up to date")
}
fmt.Fprintln(w, "the database is up to date, but the application's code is not running yet:")
fmt.Fprintln(w, "rebuild and restart the server before expecting its routes to answer.")
}
// manifestFor finds the manifest an application registered for code.
//
// A code nothing registered is an error naming what is registered, for the
// same reason exitUnlessAppRegistered exists: the alternative is telling an
// operator who typed `install ordr` that there was nothing to do.
// Takes the snapshot rather than reading it, so this lookup and the caller's
// cycle check see the same set. Two calls to app.Snapshot() would also be two
// deep copies of the registry for one install.
func manifestFor(all map[string]app.Manifest, code string) (app.Manifest, error) {
want := migration.NormalizeAppCode(code)
if m, ok := all[want]; ok {
return m, nil
}
codes := make([]string, 0, len(all))
for c := range all {
codes = append(codes, c)
}
sort.Strings(codes)
if len(codes) == 0 {
// Worth its own sentence: no application is compiled into this
// binary at all, which is a different thing from having typed the
// wrong one of several.
return app.Manifest{}, fmt.Errorf(
"no application registers a manifest in this binary, so %q cannot be installed; "+
"an application has to be compiled in before it can be installed", want)
}
return app.Manifest{}, fmt.Errorf("no application registers the code %q; registered: %s",
want, strings.Join(codes, ", "))
}
+649
View File
@@ -0,0 +1,649 @@
package migrate
import (
"errors"
"strings"
"testing"
"github.com/glebarez/sqlite"
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/app"
"gorm.io/gorm"
"gorm.io/gorm/logger"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
)
func newInstallDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
if err := db.AutoMigrate(&adminmodels.SysApp{}); err != nil {
t.Fatalf("automigrate sys_app: %v", err)
}
return db
}
// fakeEngine stands in for the migration engine. The real one is a
// package-level singleton with no exported constructor, so a test taking it
// would share one registry with every other test in this process.
type fakeEngine struct {
entries []migration.StatusEntry
// failWith, when set, is what MigrateApp returns instead of applying.
failWith error
calls []string
}
func (f *fakeEngine) SetDb(*gorm.DB) {}
func (f *fakeEngine) Status() ([]migration.StatusEntry, error) {
out := make([]migration.StatusEntry, len(f.entries))
copy(out, f.entries)
return out, nil
}
func (f *fakeEngine) MigrateApp(code string) error {
f.calls = append(f.calls, code)
if f.failWith != nil {
return f.failWith
}
for i := range f.entries {
if f.entries[i].AppCode == code && f.entries[i].Registered {
f.entries[i].Applied = true
}
}
return nil
}
func orderManifest(version string) app.Manifest {
return app.Manifest{
Code: "order",
Name: "Orders",
Version: version,
Description: "order management",
Author: "go-admin",
// No dependency by default: these tests are about installing, and a
// declared requirement would make every one of them set up a second
// application first. requiresInstalled has its own tests below.
Requires: nil,
Pricing: "free",
License: "MIT",
}
}
// appRow writes one sys_app row: what another application looks like to the
// installer, in whichever state the caller is testing against.
func appRow(t *testing.T, db *gorm.DB, code string, status int) {
t.Helper()
if err := db.Create(&adminmodels.SysApp{
AppCode: code, Name: code, Version: "1.0.0", Status: status,
}).Error; err != nil {
t.Fatalf("seeding %q with status %d: %v", code, status, err)
}
}
func loadRow(t *testing.T, db *gorm.DB, code string) adminmodels.SysApp {
t.Helper()
var row adminmodels.SysApp
if err := db.Where("app_code = ?", code).First(&row).Error; err != nil {
t.Fatalf("sys_app has no row for %q: %v", code, err)
}
return row
}
// A1: a first install records the app, at the version the manifest declares,
// with every descriptive column copied from it.
func TestInstallRecordsAFirstInstall(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
{Version: "order-1786800002000", AppCode: "order", Registered: true},
{Version: "crm-1786800001000", AppCode: "crm", Registered: true},
}}
rep, err := install(db, eng, orderManifest("1.0.0"))
if err != nil {
t.Fatalf("install: %v", err)
}
if rep.NoOp {
t.Error("a first install reported nothing to do")
}
if got, want := len(rep.Applied), 2; got != want {
t.Errorf("applied %v, want %d versions", rep.Applied, want)
}
// Only this app's migrations, not every pending one in the process.
if len(eng.calls) != 1 || eng.calls[0] != "order" {
t.Errorf("MigrateApp calls = %v", eng.calls)
}
row := loadRow(t, db, "order")
if row.Status != adminmodels.AppInstalled {
t.Errorf("status = %d, want installed", row.Status)
}
if row.Version != "1.0.0" {
t.Errorf("version = %q", row.Version)
}
if row.InstalledAt == nil {
t.Error("installed_at was not set")
}
if row.Name != "Orders" || row.Author != "go-admin" || row.Description != "order management" {
t.Errorf("descriptive columns not copied from the manifest: %+v", row)
}
if row.Pricing != "free" || row.License != "MIT" {
t.Errorf("the reserved fields were not carried through: %+v", row)
}
}
// A2: installing the same version again is a no-op, and says so.
func TestInstallIsANoOpAtTheSameVersion(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
}}
if _, err := install(db, eng, orderManifest("1.0.0")); err != nil {
t.Fatalf("first install: %v", err)
}
before := loadRow(t, db, "order")
rep, err := install(db, eng, orderManifest("1.0.0"))
if err != nil {
t.Fatalf("second install: %v", err)
}
if !rep.NoOp {
t.Error("installing the same version again was not reported as a no-op")
}
if len(eng.calls) != 1 {
t.Errorf("the engine was driven again: %v", eng.calls)
}
after := loadRow(t, db, "order")
if !after.UpdatedAt.Equal(before.UpdatedAt) {
t.Error("a no-op rewrote the row")
}
var n int64
db.Model(&adminmodels.SysApp{}).Count(&n)
if n != 1 {
t.Errorf("sys_app has %d rows, want 1", n)
}
}
// A no-op is only a no-op when nothing is outstanding. A row that says
// installed while a migration of its has never run is the case sys_app must
// not be believed over sys_migration.
func TestInstallRunsWhenTheRowSaysInstalledButAMigrationIsPending(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true, Applied: true},
}}
if _, err := install(db, eng, orderManifest("1.0.0")); err != nil {
t.Fatalf("first install: %v", err)
}
// A second version of the same app appears - the app was rebuilt with
// one more migration file, without its version changing.
eng.entries = append(eng.entries, migration.StatusEntry{
Version: "order-1786800002000", AppCode: "order", Registered: true,
})
rep, err := install(db, eng, orderManifest("1.0.0"))
if err != nil {
t.Fatalf("install: %v", err)
}
if rep.NoOp {
t.Fatal("an outstanding migration was reported as nothing to do")
}
if len(rep.Applied) != 1 || rep.Applied[0] != "order-1786800002000" {
t.Errorf("applied = %v", rep.Applied)
}
}
// A9: an upgrade is in place. installed_at is the first install's, not this
// one's.
func TestInstallUpgradesInPlaceAndKeepsTheFirstInstallTime(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
}}
if _, err := install(db, eng, orderManifest("1.0.0")); err != nil {
t.Fatalf("first install: %v", err)
}
first := loadRow(t, db, "order")
if first.InstalledAt == nil {
t.Fatal("installed_at was not set by the first install")
}
eng.entries = append(eng.entries, migration.StatusEntry{
Version: "order-1786800002000", AppCode: "order", Registered: true,
})
rep, err := install(db, eng, orderManifest("2.0.0"))
if err != nil {
t.Fatalf("upgrade: %v", err)
}
if rep.Previous != "1.0.0" {
t.Errorf("previous = %q, want 1.0.0", rep.Previous)
}
row := loadRow(t, db, "order")
if row.Version != "2.0.0" {
t.Errorf("version = %q, want 2.0.0", row.Version)
}
if row.Status != adminmodels.AppInstalled {
t.Errorf("status = %d, want installed", row.Status)
}
if !row.InstalledAt.Equal(*first.InstalledAt) {
t.Errorf("installed_at moved from %v to %v; an upgrade keeps the first install's time",
first.InstalledAt, row.InstalledAt)
}
}
// A10: a downgrade is refused, and refused before anything is written.
func TestInstallRefusesADowngrade(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
}}
if _, err := install(db, eng, orderManifest("2.0.0")); err != nil {
t.Fatalf("first install: %v", err)
}
before := loadRow(t, db, "order")
_, err := install(db, eng, orderManifest("1.0.0"))
if err == nil {
t.Fatal("a downgrade was accepted")
}
if !strings.Contains(err.Error(), "downgrade") {
t.Errorf("error = %q, it has to say what it refused", err)
}
after := loadRow(t, db, "order")
if after.Version != before.Version || after.Status != before.Status {
t.Errorf("the refused downgrade still wrote to the row: %+v -> %+v", before, after)
}
}
// A5: a failing migration leaves a row that says so, and says where.
func TestInstallRecordsAFailure(t *testing.T) {
db := newInstallDB(t)
boom := errors.New("the seed hit a duplicate")
eng := &fakeEngine{
entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
},
failWith: &migration.VersionFailure{Version: "order-1786800001000", Err: boom},
}
_, err := install(db, eng, orderManifest("1.0.0"))
if err == nil {
t.Fatal("a failed install reported success")
}
if !errors.Is(err, boom) {
t.Errorf("the cause is not reachable: %v", err)
}
row := loadRow(t, db, "order")
if row.Status != adminmodels.AppFailed {
t.Errorf("status = %d, want failed", row.Status)
}
if row.FailedVersion != "order-1786800001000" {
t.Errorf("failed_version = %q", row.FailedVersion)
}
if !strings.Contains(row.LastError, "duplicate") {
t.Errorf("last_error = %q", row.LastError)
}
if row.InstalledAt != nil {
t.Error("installed_at was set by an install that failed")
}
}
// A failed install is retried by running it again - not by any special
// command, and without the previous attempt's diagnostics surviving into a
// row that now says installed.
func TestInstallResumesAfterAFailure(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{
entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
},
failWith: &migration.VersionFailure{Version: "order-1786800001000", Err: errors.New("boom")},
}
if _, err := install(db, eng, orderManifest("1.0.0")); err == nil {
t.Fatal("the first attempt did not fail")
}
eng.failWith = nil
rep, err := install(db, eng, orderManifest("1.0.0"))
if err != nil {
t.Fatalf("retry: %v", err)
}
if rep.NoOp {
t.Error("a failed row was treated as installed")
}
row := loadRow(t, db, "order")
if row.Status != adminmodels.AppInstalled {
t.Errorf("status = %d, want installed", row.Status)
}
if row.FailedVersion != "" || row.LastError != "" {
t.Errorf("the previous failure survived onto a row that now says installed: %q / %q",
row.FailedVersion, row.LastError)
}
if row.InstalledAt == nil {
t.Error("installed_at was not set by the attempt that succeeded")
}
}
// A row stuck at installing - the process was killed partway - is not
// installed, and must not be mistaken for it.
func TestInstallRetriesARowStuckAtInstalling(t *testing.T) {
db := newInstallDB(t)
if err := db.Create(&adminmodels.SysApp{
AppCode: "order", Name: "Orders", Version: "1.0.0",
Status: adminmodels.AppInstalling,
}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true, Applied: true},
}}
rep, err := install(db, eng, orderManifest("1.0.0"))
if err != nil {
t.Fatalf("install: %v", err)
}
if rep.NoOp {
t.Fatal("a row stuck at installing was reported as already installed")
}
if row := loadRow(t, db, "order"); row.Status != adminmodels.AppInstalled {
t.Errorf("status = %d, want installed", row.Status)
}
}
func TestInstallRejectsTheFrameworkCode(t *testing.T) {
db := newInstallDB(t)
m := orderManifest("1.0.0")
m.Code = migration.FrameworkAppCode
_, err := install(db, &fakeEngine{}, m)
if err == nil {
t.Fatal("the framework was installed as an application")
}
if !strings.Contains(err.Error(), "migrate") {
t.Errorf("error = %q, it should point at the command that does this", err)
}
}
func TestInstallRefusesAnUnparseableRecordedVersion(t *testing.T) {
db := newInstallDB(t)
if err := db.Create(&adminmodels.SysApp{
AppCode: "order", Name: "Orders", Version: "v1.0", Status: adminmodels.AppInstalled,
}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
_, err := install(db, &fakeEngine{}, orderManifest("1.0.0"))
if err == nil {
t.Fatal("an unparseable recorded version was compared anyway")
}
row := loadRow(t, db, "order")
if row.Status != adminmodels.AppInstalled || row.Version != "v1.0" {
t.Errorf("the row was overwritten before the comparison failed: %+v", row)
}
}
// A7: the report has to say the code is not running yet. Menus appearing is
// exactly what makes an operator think it is.
func TestReportInstallSaysTheCodeIsNotRunningYet(t *testing.T) {
var out strings.Builder
reportInstall(&out, installReport{Code: "order", Version: "1.0.0", Applied: []string{"order-1786800001000"}})
got := out.String()
if !strings.Contains(got, "rebuild") || !strings.Contains(got, "restart") {
t.Errorf("the report does not say the binary has to be rebuilt: %q", got)
}
if !strings.Contains(got, "order-1786800001000") {
t.Errorf("the report does not name what it applied: %q", got)
}
}
func TestReportInstallOnANoOp(t *testing.T) {
var out strings.Builder
reportInstall(&out, installReport{Code: "order", Version: "1.0.0", NoOp: true})
if !strings.Contains(out.String(), "already installed") {
t.Errorf("output = %q", out.String())
}
}
// last_error is a varchar(255) declared in characters. A message that is
// partly Chinese would be cut mid-rune by a byte-wise truncation and stored
// as an invalid sequence.
func TestTruncateCutsRunesNotBytes(t *testing.T) {
s := strings.Repeat("迁", 300)
got := truncate(s, 255)
if n := len([]rune(got)); n != 255 {
t.Errorf("kept %d runes, want 255", n)
}
if !strings.HasPrefix(s, got) {
t.Error("truncation did not cut at a rune boundary")
}
if short := truncate("ok", 255); short != "ok" {
t.Errorf("a short message was altered: %q", short)
}
}
func TestInstallNeedsSysApp(t *testing.T) {
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
_, err = install(db, &fakeEngine{}, orderManifest("1.0.0"))
if err == nil {
t.Fatal("install ran against a database with no sys_app")
}
if !strings.Contains(err.Error(), "migrate") {
t.Errorf("error = %q, it should say what to run first", err)
}
}
// The code written to sys_app and handed to the engine is the normalized one.
// A manifest whose Code was typed with different case or stray spaces has to
// land on the same identity migration.ForApp and seed.SeedMenus already use,
// or the row and the migrations it stands for are filed under two names.
func TestInstallNormalizesTheAppCode(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
}}
m := orderManifest("1.0.0")
m.Code = " Order "
rep, err := install(db, eng, m)
if err != nil {
t.Fatalf("install: %v", err)
}
if rep.Code != "order" {
t.Errorf("reported code = %q, want order", rep.Code)
}
if len(eng.calls) != 1 || eng.calls[0] != "order" {
t.Errorf("the engine was asked for %v, want [order]", eng.calls)
}
// The row has to be findable by the normalized code, which is what every
// other table in this batch is keyed by.
row := loadRow(t, db, "order")
if row.AppCode != "order" {
t.Errorf("app_code = %q", row.AppCode)
}
if len(rep.Applied) != 1 {
t.Errorf("applied = %v; the normalized code has to match what Status reports", rep.Applied)
}
}
// The manifest's dependency list is stored as it was declared, in the CSV
// shape sys_app.requires carries.
func TestInstallStoresTheDeclaredRequires(t *testing.T) {
db := newInstallDB(t)
appRow(t, db, "crm", adminmodels.AppInstalled)
appRow(t, db, "billing", adminmodels.AppInstalled)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
}}
m := orderManifest("1.0.0")
m.Requires = []string{"crm", "billing"}
if _, err := install(db, eng, m); err != nil {
t.Fatalf("install: %v", err)
}
if row := loadRow(t, db, "order"); row.Requires != "crm,billing" {
t.Errorf("requires = %q, want the manifest's list as CSV", row.Requires)
}
}
// An application is not installed for you because something else names it.
// "Install this" would otherwise mean "and everything it happens to name, and
// everything those name".
func TestInstallRefusesWhenADependencyIsNotInstalled(t *testing.T) {
db := newInstallDB(t)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
}}
m := orderManifest("1.0.0")
m.Requires = []string{"crm"}
_, err := install(db, eng, m)
if err == nil {
t.Fatal("an application with an uninstalled dependency was installed")
}
if !strings.Contains(err.Error(), "crm") || !strings.Contains(err.Error(), "not installed") {
t.Errorf("error = %q, it has to name what is missing and why", err)
}
if len(eng.calls) != 0 {
t.Errorf("the engine ran anyway: %v", eng.calls)
}
// Refused before phase A, so a refusal leaves nothing behind.
if n := count(t, db, "sys_app", "app_code = ?", "order"); n != 0 {
t.Errorf("a refused install wrote %d sys_app row(s)", n)
}
}
// A dependency whose own install failed or never finished is not a dependency
// that is there, and the two say which they are - one sends you to install it,
// the other to look at why.
func TestInstallRefusesWhenADependencyIsNotFinished(t *testing.T) {
for _, tc := range []struct {
name string
status int
want string
}{
{"failed", adminmodels.AppFailed, "its install failed"},
{"installing", adminmodels.AppInstalling, "did not finish"},
} {
t.Run(tc.name, func(t *testing.T) {
db := newInstallDB(t)
appRow(t, db, "crm", tc.status)
m := orderManifest("1.0.0")
m.Requires = []string{"crm"}
_, err := install(db, &fakeEngine{}, m)
if err == nil {
t.Fatal("the dependency was accepted")
}
if !strings.Contains(err.Error(), tc.want) {
t.Errorf("error = %q, want it to say %q", err, tc.want)
}
})
}
}
func TestInstallAcceptsASatisfiedDependency(t *testing.T) {
db := newInstallDB(t)
appRow(t, db, "crm", adminmodels.AppInstalled)
eng := &fakeEngine{entries: []migration.StatusEntry{
{Version: "order-1786800001000", AppCode: "order", Registered: true},
}}
m := orderManifest("1.0.0")
m.Requires = []string{"crm"}
if _, err := install(db, eng, m); err != nil {
t.Fatalf("install: %v", err)
}
if row := loadRow(t, db, "order"); row.Status != adminmodels.AppInstalled {
t.Errorf("status = %d, want installed", row.Status)
}
}
func TestDependencyCycleIsRefused(t *testing.T) {
manifests := map[string]app.Manifest{
"a": {Code: "a", Requires: []string{"b"}},
"b": {Code: "b", Requires: []string{"c"}},
"c": {Code: "c", Requires: []string{"a"}},
}
err := refuseOnDependencyCycle(manifests)
if err == nil {
t.Fatal("a cycle was accepted")
}
// The error is the cycle, not the walk that reached it.
if !strings.Contains(err.Error(), "a -> b -> c -> a") {
t.Errorf("error = %q", err)
}
}
// A cycle between two applications neither of which is being installed is
// still an authoring mistake, and the day somebody installs into it is the
// worse time to find out.
func TestDependencyCycleIsRefusedEvenAwayFromTheTarget(t *testing.T) {
manifests := map[string]app.Manifest{
"order": {Code: "order"},
"x": {Code: "x", Requires: []string{"y"}},
"y": {Code: "y", Requires: []string{"x"}},
}
if err := refuseOnDependencyCycle(manifests); err == nil {
t.Fatal("a cycle away from the target was accepted")
}
}
func TestDependencyGraphWithoutACycle(t *testing.T) {
manifests := map[string]app.Manifest{
"a": {Code: "a", Requires: []string{"b", "c"}},
"b": {Code: "b", Requires: []string{"c"}},
"c": {Code: "c"},
// Naming something that is not registered is not a cycle. Whether it
// is installed is a question for the database, at install time.
"d": {Code: "d", Requires: []string{"nowhere"}},
}
if err := refuseOnDependencyCycle(manifests); err != nil {
t.Errorf("a graph with no cycle was refused: %v", err)
}
}
// An application that names itself.
func TestDependencyCycleOfOne(t *testing.T) {
manifests := map[string]app.Manifest{"a": {Code: "a", Requires: []string{"a"}}}
err := refuseOnDependencyCycle(manifests)
if err == nil {
t.Fatal("an application requiring itself was accepted")
}
if !strings.Contains(err.Error(), "a -> a") {
t.Errorf("error = %q", err)
}
}
// The cycle reached from outside it. a is not part of anything circular; b
// and c are. Reporting the walk instead of the cycle would name a as well,
// and sending somebody to look at an application that is not involved is
// the whole reason the path is trimmed.
func TestDependencyCycleReportsOnlyTheCycleItReached(t *testing.T) {
manifests := map[string]app.Manifest{
"a": {Code: "a", Requires: []string{"b"}},
"b": {Code: "b", Requires: []string{"c"}},
"c": {Code: "c", Requires: []string{"b"}},
}
err := refuseOnDependencyCycle(manifests)
if err == nil {
t.Fatal("a cycle was accepted")
}
if !strings.Contains(err.Error(), "b -> c -> b") {
t.Errorf("error = %q, want just the cycle", err)
}
if strings.Contains(err.Error(), "a ->") {
t.Errorf("the walk that reached the cycle was reported as part of it: %q", err)
}
}
+64 -8
View File
@@ -185,6 +185,28 @@ func (e *Migration) mergedEntries() map[string]versionEntry {
return out
}
// RegisteredVersions returns every migration version this binary registers,
// sorted, without touching a database.
//
// Status answers a richer question - what is registered, what is applied, and
// what is applied while nothing registers it - and needs a database to do it.
// This is the half that can be asked of the process alone, which is what a
// readiness check needs: the check holds the databases it is asking about, and
// reusing Status would mean calling SetDb from a request handler, writing this
// package's shared state from a request path.
func (e *Migration) RegisteredVersions() []string {
all := e.mergedEntries()
out := make([]string, 0, len(all))
for k := range all {
out = append(out, k)
}
sort.Strings(out)
return out
}
// RegisteredVersions reports what the process-wide registry holds.
func RegisteredVersions() []string { return Migrate.RegisteredVersions() }
// StatusEntry is one row of migrate status.
type StatusEntry struct {
AppCode string
@@ -262,12 +284,33 @@ func (e *Migration) Status() ([]StatusEntry, error) {
}
// Migrate applies every registered migration that has not been applied yet,
// across all apps. Existing callers are unaffected.
func (e *Migration) Migrate() { e.run(allApps) }
// across all apps.
func (e *Migration) Migrate() error { return e.run(allApps) }
// MigrateApp applies only the migrations registered under appCode. Pass
// FrameworkAppCode for the framework's own migrations.
func (e *Migration) MigrateApp(appCode string) { e.run(AppFilter(appCode)) }
func (e *Migration) MigrateApp(appCode string) error { return e.run(AppFilter(appCode)) }
// VersionFailure names the migration that failed.
//
// The caller that needs this is an installer recording which version an
// install got stuck on. That is a diagnostic snapshot and nothing more: the
// authoritative answer to "where does a retry resume" is always recomputed
// by subtracting sys_migration's applied rows from what is registered, never
// read back from anywhere it was stored. Which is exactly why this carries
// the version rather than leaving the caller to infer it - inferring it
// would produce "what is pending now", a different question that happens to
// have the same answer most of the time.
type VersionFailure struct {
Version string
Err error
}
func (e *VersionFailure) Error() string {
return fmt.Sprintf("migration %s failed: %v", e.Version, e.Err)
}
func (e *VersionFailure) Unwrap() error { return e.Err }
// NormalizeAppCode applies the same rule ForApp does, so a code typed on the
// command line matches one written in an init().
@@ -310,7 +353,15 @@ func (e *Migration) AppCodes() []string {
return out
}
func (e *Migration) run(appCode string) {
// run applies the pending migrations selected by appCode.
//
// It reports failure instead of ending the process. It used to call
// log.Fatalf, which took the whole process down at the first failing
// migration - so a caller had nowhere to record what happened, and a test
// could not exercise a failing migration at all without killing the test
// binary. The exit now lives at the command layer, where the exit code is
// the command's business (see initDB in cmd/migrate/server.go).
func (e *Migration) run(appCode string) error {
all := e.mergedEntries()
versions := make([]string, 0, len(all))
entries := make(map[string]versionEntry, len(all))
@@ -325,10 +376,14 @@ func (e *Migration) run(appCode string) {
// A mistyped --app would otherwise select nothing and report "no
// migrations to apply", which reads exactly like "already up to date".
//
// The command layer rejects an unregistered code before any database
// work (exitUnlessAppRegistered), so on that path this is unreachable.
// It is reachable from an installer, which asks for one app by name and
// must not be told that installing an app nothing registered succeeded.
if appCode != allApps && len(versions) == 0 {
log.Printf("no migrations are registered for app %q; registered: %s",
return fmt.Errorf("no migrations are registered for app %q; registered: %s",
DisplayAppCode(appCode), strings.Join(e.AppCodes(), ", "))
return
}
var err error
@@ -337,7 +392,7 @@ func (e *Migration) run(appCode string) {
for _, v := range versions {
err = e.db.Table("sys_migration").Where("version = ?", v).Count(&count).Error
if err != nil {
log.Fatalln(err)
return fmt.Errorf("checking whether migration %s was applied: %w", v, err)
}
if count > 0 {
// Already applied. This used to print the bare count, so a mature
@@ -347,7 +402,7 @@ func (e *Migration) run(appCode string) {
}
log.Printf("applying migration %s", v)
if err = entries[v].fn(e.db.Debug(), v); err != nil {
log.Fatalf("migration %s failed: %v", v, err)
return &VersionFailure{Version: v, Err: err}
}
applied++
}
@@ -356,6 +411,7 @@ func (e *Migration) run(appCode string) {
} else {
log.Printf("applied %d migration(s)", applied)
}
return nil
}
// allApps is the sentinel run() takes to mean "do not filter". It is distinct
+90 -28
View File
@@ -1,9 +1,7 @@
package migration
import (
"bytes"
"log"
"os"
"errors"
"strings"
"testing"
"time"
@@ -81,7 +79,9 @@ func TestForAppRecordsItsAppCode(t *testing.T) {
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
return recordFor(db, version, appCode)
})
m.Migrate()
if err := m.Migrate(); err != nil {
t.Fatalf("m.Migrate(): %v", err)
}
rows := rowsByVersion(t, db)
row, ok := rows["x-1786800001000"]
@@ -104,7 +104,9 @@ func TestSetVersionStillRecordsTheFrameworkAsEmpty(t *testing.T) {
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
return db.Create(&common.Migration{Version: version}).Error
})
m.Migrate()
if err := m.Migrate(); err != nil {
t.Fatalf("m.Migrate(): %v", err)
}
rows := rowsByVersion(t, db)
row, ok := rows["1786700009000"]
@@ -136,7 +138,9 @@ func TestMigrateAppRunsOnlyThatApp(t *testing.T) {
return recordFor(db, version, appCode)
})
m.MigrateApp("x")
if err := m.MigrateApp("x"); err != nil {
t.Fatalf("m.MigrateApp(\"x\"): %v", err)
}
if !ran["x"] {
t.Error("x did not run")
@@ -167,7 +171,9 @@ func TestMigrateAppCoreSelectsTheFramework(t *testing.T) {
return recordFor(db, version, appCode)
})
m.MigrateApp(FrameworkAppCode)
if err := m.MigrateApp(FrameworkAppCode); err != nil {
t.Fatalf("m.MigrateApp(FrameworkAppCode): %v", err)
}
if !ran["core"] {
t.Error("framework migration did not run")
@@ -198,7 +204,9 @@ func TestMigrateRunsEveryApp(t *testing.T) {
return recordFor(db, version, appCode)
})
m.Migrate()
if err := m.Migrate(); err != nil {
t.Fatalf("m.Migrate(): %v", err)
}
// Namespacing puts every framework migration - bare digits - ahead of every
// app migration, and orders apps by code rather than by whose timestamp
@@ -232,7 +240,9 @@ func TestNamespacingKeepsTwoAppsWithTheSameTimestampApart(t *testing.T) {
return recordFor(db, version, appCode)
})
}
m.Migrate()
if err := m.Migrate(); err != nil {
t.Fatalf("m.Migrate(): %v", err)
}
if ran != 2 {
t.Errorf("ran %d migrations, want 2", ran)
@@ -357,11 +367,11 @@ func TestFailedMigrationLeavesNoRecord(t *testing.T) {
})
})
// run() calls log.Fatal on failure, which would take the test binary with
// it, so drive the registered function directly - the point here is the
// transaction boundary, not the scheduler.
entry := m.version["crm-1786800001000"]
if err := entry.fn(db, "crm-1786800001000"); err == nil {
// Driven through the scheduler, not by calling the registered function
// directly. That workaround was here because run() called log.Fatal and
// would have taken the test binary with it, which also meant nothing
// covered what the scheduler does with a failure.
if err := m.MigrateApp("crm"); err == nil {
t.Fatal("migration reported success")
}
if rows := rowsByVersion(t, db); len(rows) != 0 {
@@ -369,6 +379,49 @@ func TestFailedMigrationLeavesNoRecord(t *testing.T) {
}
}
// An installer records which version an attempt got stuck on. It gets that
// from the error rather than by asking the database what is still pending,
// which is a different question - see VersionFailure.
func TestRunReportsWhichVersionFailed(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
// Two versions, and the first one succeeds: the failure has to name the
// one that actually failed, which a report that just names the app, or
// the first version it looked at, would get wrong.
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
return recordFor(db, version, appCode)
})
m.ForApp("crm").SetVersion("1786800002000", func(db *gorm.DB, version, appCode string) error {
return errTestMigrationFailed
})
err := m.MigrateApp("crm")
if err == nil {
t.Fatal("MigrateApp reported success")
}
var vf *VersionFailure
if !errors.As(err, &vf) {
t.Fatalf("error is %T, want *VersionFailure: %v", err, err)
}
if vf.Version != "crm-1786800002000" {
t.Errorf("failed version = %q, want crm-1786800002000", vf.Version)
}
if !errors.Is(err, errTestMigrationFailed) {
t.Errorf("the cause is not reachable through the wrapper: %v", err)
}
// The one that succeeded before it stays recorded: a retry must not run
// it again.
rows := rowsByVersion(t, db)
if _, ok := rows["crm-1786800001000"]; !ok {
t.Errorf("the migration that succeeded was not recorded: %v", rows)
}
if _, ok := rows["crm-1786800002000"]; ok {
t.Errorf("the migration that failed was recorded: %v", rows)
}
}
var errTestMigrationFailed = &testError{"boom"}
type testError struct{ s string }
@@ -389,17 +442,18 @@ func TestMigrateAppOnAnUnknownCodeSaysSo(t *testing.T) {
return recordFor(db, version, appCode)
})
var buf bytes.Buffer
log.SetOutput(&buf)
t.Cleanup(func() { log.SetOutput(os.Stderr) })
m.MigrateApp("crmm")
if !strings.Contains(buf.String(), `no migrations are registered for app "crmm"`) {
t.Errorf("output = %q", buf.String())
// Reported as an error rather than a log line, so an installer asking
// for one app by name cannot be told that installing an app nothing
// registered succeeded.
err := m.MigrateApp("crmm")
if err == nil {
t.Fatal("a typo reported success")
}
if !strings.Contains(buf.String(), "registered: core, crm") {
t.Errorf("the message must list what is registered; got %q", buf.String())
if !strings.Contains(err.Error(), `no migrations are registered for app "crmm"`) {
t.Errorf("error = %q", err)
}
if !strings.Contains(err.Error(), "registered: core, crm") {
t.Errorf("the message must list what is registered; got %q", err)
}
if rows := rowsByVersion(t, db); len(rows) != 0 {
t.Errorf("a typo ran %v", rows)
@@ -425,7 +479,9 @@ func TestMergedEntriesRunsAContractRegisteredAppMigration(t *testing.T) {
return recordFor(db, version, appCode)
})
m.Migrate()
if err := m.Migrate(); err != nil {
t.Fatalf("m.Migrate(): %v", err)
}
if !ran {
t.Fatal("contract-registered migration did not run")
@@ -466,7 +522,9 @@ func TestMergedEntriesStatusIncludesContractRegisteredMigrations(t *testing.T) {
t.Fatalf("pending contract entry = %+v (ok=%v)", e, ok)
}
m.Migrate()
if err := m.Migrate(); err != nil {
t.Fatalf("m.Migrate(): %v", err)
}
entries, err = m.Status()
if err != nil {
@@ -522,7 +580,9 @@ func TestMergedEntriesMigrateAppRunsOnlyThatContractApp(t *testing.T) {
return recordFor(db, version, appCode)
})
m.MigrateApp("order")
if err := m.MigrateApp("order"); err != nil {
t.Fatalf("m.MigrateApp(\"order\"): %v", err)
}
if !ran["order"] {
t.Error("order did not run")
@@ -552,7 +612,9 @@ func TestMergedEntriesHostRegistrationWinsOnKeyCollision(t *testing.T) {
return recordFor(db, version, appCode)
})
m.Migrate()
if err := m.Migrate(); err != nil {
t.Fatalf("m.Migrate(): %v", err)
}
if !hostRan {
t.Error("host registration did not run")
@@ -237,13 +237,53 @@ func dropIndexesOn(db *gorm.DB, table, column string) error {
if !m.HasIndex(table, name) {
continue
}
if err := m.DropIndex(table, name); err != nil {
if err := db.Exec(dropIndex(db, table, name)).Error; err != nil {
return fmt.Errorf("dropping index %s: %w", name, err)
}
}
return nil
}
// dropIndex spells DROP INDEX for one dialect, rather than going through
// Migrator().DropIndex.
//
// The migrator cannot be used here on PostgreSQL. Its driver resolves a schema
// for the statement and falls back to an expression when it cannot:
//
// currentSchema, _ := m.CurrentSchema(stmt, stmt.Table) // CURRENT_SCHEMA()
// m.DB.Exec("DROP INDEX ?.?", currentSchema, clause.Column{Name: name})
//
// DROP INDEX takes an identifier in that position, not an expression, so the
// statement does not parse. The schema is unresolvable for every call made
// here, because this passes a table name as a string rather than a model - so
// it failed on every PostgreSQL database rather than intermittently, and took
// the whole conversion with it. Reported as go-admin#919, where the visible
// symptom was a login rejecting a correct password: the migration had stopped
// here, leaving deleted_at a timestamptz that the current query compares to 0.
//
// Written per dialect for the same reason addBigIntColumn and renameColumn
// already are.
//
// MySQL and SQL Server name the table in the statement and have no IF EXISTS
// for it; PostgreSQL and SQLite name the index alone, in its own namespace.
// The caller has already checked HasIndex, so IF EXISTS is only there to make
// the two that support it say nothing rather than fail on a race with another
// migrator.
//
// Verified against PostgreSQL 15, MySQL 8.0 and SQLite. The SQL Server form is
// from its documentation and has not been run - this repository has no SQL
// Server to run it against.
func dropIndex(db *gorm.DB, table, index string) string {
switch db.Dialector.Name() {
case "mysql":
return fmt.Sprintf("DROP INDEX `%s` ON `%s`", index, table)
case "sqlserver":
return fmt.Sprintf("DROP INDEX [%s] ON [%s]", index, table)
default:
return fmt.Sprintf(`DROP INDEX IF EXISTS "%s"`, index)
}
}
// indexNamesFor asks the database which indexes cover column.
func indexNamesFor(db *gorm.DB, table, column string) ([]string, error) {
indexes, err := db.Migrator().GetIndexes(table)
@@ -0,0 +1,156 @@
package version
import (
"os"
"testing"
"time"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
// postgresDSNEnv points these tests at a database. They are skipped without
// it, so a developer with no PostgreSQL running still gets a green run.
//
// The whole file exists because the rest of this package's tests run on
// SQLite, where the defect they cover cannot happen: dropping an index through
// gorm's migrator works there and produces unparseable SQL on PostgreSQL. A
// suite that only ever exercised SQLite reported success for a migration that
// failed on every PostgreSQL database it was pointed at - go-admin#919.
const postgresDSNEnv = "GO_ADMIN_TEST_POSTGRES_DSN"
func postgresDB(t *testing.T) *gorm.DB {
t.Helper()
dsn := os.Getenv(postgresDSNEnv)
if dsn == "" {
// Skipping locally is the point; skipping in CI is the failure this
// file exists to prevent. A workflow that renamed the variable or
// dropped the service would otherwise go green while these tests
// quietly did nothing - the same shape as the defect they cover.
if os.Getenv("CI") != "" {
t.Fatalf("%s is not set while CI is: the PostgreSQL migration tests must not skip here", postgresDSNEnv)
}
t.Skipf("%s is not set; skipping the PostgreSQL migration tests", postgresDSNEnv)
}
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatalf("connecting to %s: %v", postgresDSNEnv, err)
}
return db
}
// pgOldUser is the pre-migration shape: a nullable timestamp with an index on
// it, which is what makes dropping the column require dropping the index.
type pgOldUser struct {
UserId int64 `gorm:"column:user_id;primaryKey;autoIncrement"`
Username string
DeletedAt *time.Time `gorm:"index"`
}
func (pgOldUser) TableName() string { return "sd_pg_user" }
// The conversion completes on PostgreSQL.
//
// It did not. dropIndexesOn went through Migrator().DropIndex, whose
// PostgreSQL driver falls back to an expression when it cannot resolve a
// schema - which is every call made here, because the migration passes a table
// name as a string:
//
// DROP INDEX CURRENT_SCHEMA()."idx_sd_pg_user_deleted_at"
//
// DROP INDEX takes an identifier there, so it failed to parse and took the
// whole conversion with it. Every PostgreSQL deployment stopped at this
// migration, and the visible symptom was a login rejecting a correct password
// because deleted_at was still a timestamptz being compared to 0.
func TestConversionCompletesOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&pgOldUser{}) })
db.Migrator().DropTable(&pgOldUser{})
if err := db.AutoMigrate(&pgOldUser{}); err != nil {
t.Fatalf("building the old shape: %v", err)
}
deleted := time.Now().Add(-time.Hour)
// Checked rather than fired and forgotten: a failed insert leaves the
// assertions below reading an empty table, and "no rows" is a shape some
// of them cannot tell from success.
if err := db.Create(&pgOldUser{Username: "gone", DeletedAt: &deleted}).Error; err != nil {
t.Fatalf("seeding the deleted row: %v", err)
}
if err := db.Create(&pgOldUser{Username: "live"}).Error; err != nil {
t.Fatalf("seeding the live row: %v", err)
}
if err := convertDeletedAt(db, "sd_pg_user"); err != nil {
t.Fatalf("convertDeletedAt: %v", err)
}
var dataType string
if err := db.Raw(`SELECT data_type FROM information_schema.columns
WHERE table_name = 'sd_pg_user' AND column_name = 'deleted_at'`).Scan(&dataType).Error; err != nil {
t.Fatalf("reading the column type: %v", err)
}
if dataType != "bigint" {
t.Errorf("deleted_at is %q after the conversion, want bigint", dataType)
}
// The marker has to carry the timestamp across, or a row that was deleted
// comes back live.
var markers []int64
if err := db.Raw(`SELECT deleted_at FROM sd_pg_user ORDER BY user_id`).Scan(&markers).Error; err != nil {
t.Fatalf("reading the markers: %v", err)
}
if len(markers) != 2 {
t.Fatalf("read %d rows, want 2", len(markers))
}
if markers[0] == 0 {
t.Error("the deleted row came back live")
}
if markers[1] != 0 {
t.Errorf("the live row is marked deleted at %d", markers[1])
}
}
// The index on deleted_at is gone afterwards, which is the step that failed.
//
// Asserted separately from the conversion because the conversion can succeed
// on a table with no index at all, and this migration exists for tables that
// have one.
func TestTheIndexOnDeletedAtIsDroppedOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&pgOldUser{}) })
db.Migrator().DropTable(&pgOldUser{})
if err := db.AutoMigrate(&pgOldUser{}); err != nil {
t.Fatalf("building the old shape: %v", err)
}
var before int64
if err := db.Raw(`SELECT count(*) FROM pg_indexes
WHERE tablename = 'sd_pg_user' AND indexdef LIKE '%deleted_at%'`).Scan(&before).Error; err != nil {
t.Fatalf("counting the indexes before: %v", err)
}
if before == 0 {
t.Fatal("the old shape has no index on deleted_at, so this test asserts nothing")
}
if err := dropIndexesOn(db, "sd_pg_user", "deleted_at"); err != nil {
t.Fatalf("dropIndexesOn: %v", err)
}
// This one is why the errors are checked at all rather than as a matter of
// habit: a query that fails leaves after at zero, and zero is what success
// looks like. An unchecked error here is a test that passes when it cannot
// reach the database.
var after int64
if err := db.Raw(`SELECT count(*) FROM pg_indexes
WHERE tablename = 'sd_pg_user' AND indexdef LIKE '%deleted_at%'`).Scan(&after).Error; err != nil {
t.Fatalf("counting the indexes after: %v", err)
}
if after != 0 {
t.Errorf("%d index(es) on deleted_at survived", after)
}
}
@@ -0,0 +1,42 @@
package version
import (
"runtime"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
)
// Create sys_app (PRD 008 F2) and sys_app_casbin_grant (F4/F6's casbin
// attribution ledger - see the design doc's (docs-prd/008-应用清单与安装器/
// 数据库变更.md) §2.2/§3 for why casbin_rule itself is not touched:
// gorm-adapter's SavePolicyCtx truncates and reloads that table from its
// in-memory model, and any column this migration added to it would be
// silently zeroed the first time anything calls SavePolicy.
//
// Ordered after 1786700003000 (the soft-delete conversion), so importing
// cmd/migrate/migration/models is banned here - see
// schema_coverage_test.go's TestPostConversionMigrationsAvoidFrozenSeedModels.
// Both new tables are AutoMigrate'd from their runtime model shape under
// app/admin/models directly, which is also why neither one is added to
// 1786700003000's frozen softDeleteTables list: neither embeds
// common.ModelTime in the first place (see design doc §1.1).
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700007000AppRegistryTables)
}
func _1786700007000AppRegistryTables(db *gorm.DB, version string) error {
return db.Transaction(func(tx *gorm.DB) error {
if err := tx.Migrator().AutoMigrate(
new(adminmodels.SysApp),
new(adminmodels.SysAppCasbinGrant),
); err != nil {
return err
}
return tx.Create(&common.Migration{Version: version}).Error
})
}
@@ -0,0 +1,62 @@
package version
import (
"testing"
common "go-admin/common/models"
adminmodels "go-admin/app/admin/models"
)
// postgresDB is defined in 1786700003000_soft_delete_marker_postgres_test.go
// and shared across this package's PostgreSQL-only tests.
//
// This migration is plain AutoMigrate on two brand-new tables, unlike
// 1786700003000's DROP INDEX (go-admin#919's actual defect), so there is no
// dialect-specific SQL here for AutoMigrate itself to get wrong on
// PostgreSQL specifically. What is worth a real PostgreSQL run is
// 1786700008000's CONCAT()-based duplicate check next door - PostgreSQL has
// had CONCAT() since 9.1, but it was never verified against a real server
// until this file, only inferred from documentation - and the same
// AutoMigrate call this test makes, so a schema/character-set mistake
// AutoMigrate might make silently on a dialect nobody ran it against here
// has somewhere to surface.
func TestAppRegistryTablesAreCreatedOnPostgres(t *testing.T) {
db := postgresDB(t)
const version = "1786700007000-pg"
cleanup := func() {
db.Migrator().DropTable(&adminmodels.SysAppCasbinGrant{}, &adminmodels.SysApp{})
// Only this test's own row, not the whole shared sys_migration
// table: postgresDB points at a real, persistent database (unlike
// the SQLite tests' fresh in-memory one per run), so a version left
// behind by a previous run of this same binary collides with the
// wrapper's own INSERT the next time this test runs.
db.Exec("DELETE FROM sys_migration WHERE version = ?", version)
}
t.Cleanup(cleanup)
cleanup()
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("automigrate sys_migration: %v", err)
}
if err := _1786700007000AppRegistryTables(db, version); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order", Version: "v1"}).Error; err != nil {
t.Fatalf("insert sys_app: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "dup", Version: "v1"}).Error; err == nil {
t.Fatal("a second sys_app row with the same app_code was accepted on PostgreSQL")
}
grant := adminmodels.SysAppCasbinGrant{AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET"}
if err := db.Create(&grant).Error; err != nil {
t.Fatalf("insert sys_app_casbin_grant: %v", err)
}
dup := grant
dup.Id = 0
if err := db.Create(&dup).Error; err == nil {
t.Fatal("a second sys_app_casbin_grant row with the same natural key was accepted on PostgreSQL")
}
}
@@ -0,0 +1,130 @@
package version
import (
"testing"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
common "go-admin/common/models"
)
func openAppRegistryDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("automigrate sys_migration: %v", err)
}
return db
}
// The migration has to build both tables and record itself as applied -
// F2/F6's acceptance case is a row landing in either one, and neither is
// possible if the table it belongs to was never created.
func TestAppRegistryTablesAreCreated(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasTable(&adminmodels.SysApp{}) {
t.Fatal("sys_app was not created")
}
if !db.Migrator().HasTable(&adminmodels.SysAppCasbinGrant{}) {
t.Fatal("sys_app_casbin_grant was not created")
}
// A row that exercises every column, not just HasTable/HasColumn -
// AutoMigrate can build a column with the wrong type and still report
// that it exists.
if err := db.Create(&adminmodels.SysApp{
AppCode: "order", Name: "Order", Version: "v1", Description: "d", Author: "a",
Requires: "payment", Pricing: "free", License: "MIT", Status: 1,
}).Error; err != nil {
t.Fatalf("insert sys_app: %v", err)
}
if err := db.Create(&adminmodels.SysAppCasbinGrant{
AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET",
}).Error; err != nil {
t.Fatalf("insert sys_app_casbin_grant: %v", err)
}
var applied common.Migration
if err := db.Where("version = ?", "1786700007000").First(&applied).Error; err != nil {
t.Fatalf("sys_migration was not recorded: %v", err)
}
}
// Running it twice must be safe: DDL does not roll back on MySQL, so an
// operator whose first attempt failed partway through has nothing to do but
// run it again. This calls AutoMigrate directly rather than the wrapper,
// which also inserts a sys_migration row that a second call would collide
// on - a collision Migrate.run() itself prevents by never calling a
// function twice for the same recorded version, so it is not this
// migration's job to tolerate.
func TestAppRegistryTablesAutoMigrateIsRepeatable(t *testing.T) {
db := openAppRegistryDB(t)
for i := 0; i < 3; i++ {
if err := db.Migrator().AutoMigrate(
new(adminmodels.SysApp),
new(adminmodels.SysAppCasbinGrant),
); err != nil {
t.Fatalf("automigrate %d: %v", i, err)
}
}
}
// sys_app.app_code is the unique key G2 ("is app X installed") answers with
// - a second row for the same app code must be rejected, not tolerated.
func TestSysAppAppCodeIsUnique(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order", Version: "v1"}).Error; err != nil {
t.Fatalf("first insert: %v", err)
}
if err := db.Create(&adminmodels.SysApp{AppCode: "order", Name: "Order dup", Version: "v1"}).Error; err == nil {
t.Fatal("a second sys_app row with the same app_code was accepted")
}
}
// sys_app_casbin_grant's unique index mirrors casbin_rule's own natural key
// (ptype,v0..v5) exactly - see design doc §3. A duplicate grant for the
// same rule must be rejected the same way gorm-adapter's own unique index
// on casbin_rule would reject it.
func TestSysAppCasbinGrantNaturalKeyIsUnique(t *testing.T) {
db := openAppRegistryDB(t)
if err := _1786700007000AppRegistryTables(db, "1786700007000"); err != nil {
t.Fatalf("migrate: %v", err)
}
grant := adminmodels.SysAppCasbinGrant{AppCode: "order", Ptype: "p", V0: "admin", V1: "/api/v1/order", V2: "GET"}
if err := db.Create(&grant).Error; err != nil {
t.Fatalf("first insert: %v", err)
}
dup := grant
dup.Id = 0
if err := db.Create(&dup).Error; err == nil {
t.Fatal("a second sys_app_casbin_grant row with the same natural key was accepted")
}
// A grant for a different app, but the identical casbin natural key, is
// exactly the collision two applications granting the same api/role
// pair would produce - the natural key has to be the one thing that
// rejects it, app_code is descriptive only and not part of the index.
other := grant
other.Id = 0
other.AppCode = "another-app"
if err := db.Create(&other).Error; err == nil {
t.Fatal("a duplicate natural key under a different app_code was accepted")
}
}
@@ -0,0 +1,183 @@
package version
import (
"fmt"
"runtime"
"strings"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
common "go-admin/common/models"
)
// Give seed.SeedMenus's two write paths (seedApis, seedMenuTree in
// app/admin/service/seed.go) a real natural key to check before inserting,
// so a retried, partially-failed migration (see the design doc
// docs-prd/008-应用清单与安装器/数据库变更.md §1.5/§1.6) does not insert the
// same row twice. This has already happened in production once (duplicate
// sys_menu/casbin_rule rows on the demo site), not a theoretical risk.
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700008000SeedNaturalKeys)
}
func _1786700008000SeedNaturalKeys(db *gorm.DB, version string) error {
if err := seedNaturalKeys(db); err != nil {
return err
}
return db.Create(&common.Migration{Version: version}).Error
}
// seedNaturalKeys is split out from the wrapper above so tests can call it
// against a database that only has sys_menu/sys_api, without also standing
// up sys_migration - and so it can be called more than once in the same
// test to prove the re-run tolerance the doc comment above promises: DDL
// does not roll back on MySQL, so an operator whose first attempt failed
// partway through has nothing to do but run the whole migration again.
func seedNaturalKeys(db *gorm.DB) error {
m := db.Migrator()
// sys_menu.seed_code is a brand-new column: every existing row becomes
// NULL, and NULL never collides in the unique index built below, so
// this needs no pre-check.
if !m.HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
if err := m.AddColumn(&adminmodels.SysMenu{}, "SeedCode"); err != nil {
return err
}
}
if !m.HasIndex(&adminmodels.SysMenu{}, "uk_sys_menu_app_seed_code_del") {
if err := db.Exec(uniqueIndexOverNullable(db.Dialector.Name(),
"uk_sys_menu_app_seed_code_del", "sys_menu",
"app_code, seed_code, deleted_at", "seed_code"),
).Error; err != nil {
return err
}
}
// sys_api reuses existing, already-populated columns, which the demo
// site has already proven can hold duplicates. Refuse rather than let
// CREATE UNIQUE INDEX fail on an operator with no idea which rows to
// reconcile - same shape as 1786700003000_soft_delete_marker.go's
// refuseOnDuplicates.
if err := refuseOnDuplicateApis(db); err != nil {
return err
}
if !m.HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
if err := db.Exec(uniqueIndexOverNullable(db.Dialector.Name(),
"uk_sys_api_app_path_action_del", "sys_api",
"app_code, path, action, deleted_at", "path", "action"),
).Error; err != nil {
return err
}
}
return nil
}
// uniqueIndexOverNullable builds a CREATE UNIQUE INDEX whose key includes
// columns that can be NULL, and makes it mean the same thing on all four
// drivers this repository registers.
//
// Three of them treat two NULLs as different values, so any number of rows
// missing one of these columns coexist under the index. SQL Server does not:
// its unique index treats NULLs as equal and permits exactly one. The
// unfiltered statement therefore fails there on any database with two rows
// lacking a seed_code - which is every database, including a brand-new one,
// because 1786700001000 seeds five menus and none of them has one:
//
// Msg 1505 ... duplicate key ... The duplicate key value is (, <NULL>, 0).
//
// Adding the filter on SQL Server takes the rows that carry no value out of
// the index, which is what the other three do by not comparing their NULLs.
// It is not added elsewhere: MySQL has no filtered index at all, and on
// PostgreSQL and SQLite it would only restate what those engines already do.
//
// Only databases that have not applied this migration are affected, and no
// SQL Server database can have: it could not get past this statement.
//
// Takes the dialect by name rather than the connection, so the statement it
// builds for every driver can be checked without one of each running.
func uniqueIndexOverNullable(dialect, name, table, columns string, nullable ...string) string {
stmt := fmt.Sprintf("CREATE UNIQUE INDEX %s ON %s (%s)", name, table, columns)
if dialect != "sqlserver" || len(nullable) == 0 {
return stmt
}
preds := make([]string, 0, len(nullable))
for _, c := range nullable {
preds = append(preds, c+" IS NOT NULL")
}
return stmt + " WHERE " + strings.Join(preds, " AND ")
}
// refuseOnDuplicateApis reports the (app_code, path, action) values that
// would make the unique index impossible, rather than the index failing to
// build and saying only that it did. Only live rows count: a soft-deleted
// duplicate does not block the index it will never occupy a slot in.
//
// sys_api.path/action (app/admin/models/sys_api.go) carry no NOT NULL
// constraint, and that stays true here on purpose: tightening it is an
// independent, backward-incompatible change of its own - existing NULL
// rows in a real database would need reconciling or backfilling before
// ALTER TABLE ... NOT NULL could even run, which is a decision for
// whoever owns that data, not something this migration should force as a
// side effect of adding an unrelated index. So this function has to
// tolerate NULL path/action rather than assume they cannot occur - see the
// query below for how it does that without either crashing on them
// (MySQL's CONCAT) or wrongly flagging them (GROUP BY's NULL-equals-NULL).
//
// The two are independent bugs that happened to share one root cause, and
// SQLite's own test suite for this file would have caught neither on its
// own: MySQL's CONCAT() returns NULL if any argument is NULL, which turned
// a duplicate check against a NULL-holding library into "converting NULL
// to string is unsupported" instead of a report - but SQLite's (and
// PostgreSQL's) CONCAT() treats a NULL argument as an empty string
// instead, so the exact same query never errors there no matter how it is
// called. A suite that only ever ran on SQLite would report success for
// both defects; only a real MySQL server surfaces the first one at all -
// this migration's PostgreSQL-only sibling test file
// (1786700008000_seed_natural_keys_postgres_test.go) rules out one more
// dialect, but MySQL specifically has to be checked by hand, since this
// repository's test suite has no MySQL service to run against in CI.
func refuseOnDuplicateApis(db *gorm.DB) error {
var dupes []string
if err := db.Raw(
// This has to agree with what the unique index it guards actually
// enforces, not just with what looks like a duplicate at a glance.
// Two different SQL rules collide on a NULL: GROUP BY treats two
// NULLs as equal, so a naive query flags every pair of rows that
// share a NULL path or action - even a pair with only one of the
// two NULL, since GROUP BY's equality still holds on whichever
// column both rows leave NULL - but a UNIQUE INDEX treats every
// NULL as distinct from every other value, including another
// NULL, so the index itself accepts every one of those pairs
// without complaint. Excluding any row missing either column from
// consideration entirely is what makes the two agree: a row
// missing path, or missing action, or missing both, can never
// violate the index no matter how many other rows are also
// missing the same one, so none of them belong in this count.
//
// No COALESCE: with both columns excluded whenever either is
// NULL, CONCAT here never receives a NULL argument for path or
// action - app_code cannot be NULL at all (see its own NOT NULL
// tag) - so there is nothing left for COALESCE to guard against,
// and leaving it out is deliberate rather than an oversight. A
// future regression that removed the two IS NOT NULL conditions
// above would fail loudly on MySQL (the same Scan error this
// query used to produce) instead of quietly reporting a made-up
// "duplicate" whose path and action both print as empty - the
// failure this function exists to prevent in the first place.
`SELECT CONCAT(app_code, '|', path, '|', action) FROM sys_api
WHERE deleted_at = 0 AND path IS NOT NULL AND action IS NOT NULL
GROUP BY app_code, path, action HAVING COUNT(*) > 1`,
).Scan(&dupes).Error; err != nil {
return fmt.Errorf("checking sys_api for duplicates: %w", err)
}
if len(dupes) > 0 {
return fmt.Errorf(
"sys_api already holds duplicate (app_code,path,action) %v; reconcile them before this migration can add its unique index",
dupes)
}
return nil
}
@@ -0,0 +1,105 @@
package version
import (
"testing"
)
// postgresDB is defined in 1786700003000_soft_delete_marker_postgres_test.go.
//
// This file exists because refuseOnDuplicateApis's duplicate check is
// spelled with CONCAT(), a function this migration's design assumed
// PostgreSQL has carried since 9.1 but that nothing had run against a real
// PostgreSQL server before this test - only against the pure-Go SQLite
// driver, which happens to bundle a SQLite new enough to have grown its own
// CONCAT() only recently. A dialect where that assumption were wrong would
// otherwise only be discovered the first time an operator's install hit a
// genuine sys_api duplicate on PostgreSQL in production.
func TestSeedNaturalKeysRefusesDuplicateApisOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
for i := 0; i < 2; i++ {
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed duplicate %d: %v", i, err)
}
}
err := seedNaturalKeys(db)
if err == nil {
t.Fatal("PostgreSQL accepted sys_api rows that already hold a duplicate (app_code, path, action)")
}
if !contains(err.Error(), "order") || !contains(err.Error(), "/api/v1/order") {
t.Errorf("the error does not name the offending row: %v", err)
}
if db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was built despite the migration refusing")
}
}
// GROUP BY treats two NULLs as equal for grouping; a UNIQUE INDEX treats
// every NULL as distinct from every other value, including another NULL.
// Both are standard SQL, not a SQLite/PostgreSQL/MySQL difference - this
// file exists to confirm that on a real server rather than assume it, the
// same reason TestSeedNaturalKeysRefusesDuplicateApisOnPostgres above
// exists for CONCAT(). See TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAccept
// in the SQLite-backed test file for the full account of why this matters:
// a naive duplicate check that does not exclude NULL path/action refuses
// an install the unique index itself would accept without complaint.
func TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAcceptOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
for i := 0; i < 2; i++ {
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, NULL, 0)",
).Error; err != nil {
t.Fatalf("seed NULL row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts on PostgreSQL: %v", err)
}
if !db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built on PostgreSQL even though seedNaturalKeys reported success")
}
}
// The success path, on the same server: both columns and both unique
// indexes have to actually build on PostgreSQL, not merely fail to error
// out on SQLite. Mirrors TestSeedNaturalKeysIsRepeatable's SQLite coverage.
func TestSeedNaturalKeysBuildsOnPostgres(t *testing.T) {
db := postgresDB(t)
t.Cleanup(func() { db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{}) })
db.Migrator().DropTable(&oldSeedMenu{}, &oldSeedApi{})
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
for i := 0; i < 2; i++ {
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate %d: %v", i, err)
}
}
if !db.Migrator().HasColumn(&oldSeedMenu{}, "seed_code") {
t.Error("sys_menu.seed_code was not added on PostgreSQL")
}
if !db.Migrator().HasIndex(&oldSeedMenu{}, "uk_sys_menu_app_seed_code_del") {
t.Error("the sys_menu unique index was not built on PostgreSQL")
}
if !db.Migrator().HasIndex(&oldSeedApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the sys_api unique index was not built on PostgreSQL")
}
}
@@ -0,0 +1,148 @@
package version
import (
"os"
"testing"
"gorm.io/driver/sqlserver"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
)
// sqlserverDSNEnv points these tests at a database. They skip without it, so
// a developer with no SQL Server running still gets a green run.
//
// This file exists for the same reason the PostgreSQL one does, one driver
// further along. The rest of the package runs on SQLite, where the defect it
// covers cannot happen: SQLite, MySQL and PostgreSQL all treat two NULLs in a
// unique index as different values, and SQL Server treats them as equal and
// permits one. A suite that never pointed at SQL Server reported success for
// a migration that could not be applied to any SQL Server database at all,
// new or old.
const sqlserverDSNEnv = "GO_ADMIN_TEST_SQLSERVER_DSN"
func sqlserverDB(t *testing.T) *gorm.DB {
t.Helper()
dsn := os.Getenv(sqlserverDSNEnv)
if dsn == "" {
// Skipping locally is the point; skipping in CI is the failure this
// file exists to prevent.
if os.Getenv("CI") != "" {
t.Fatalf("%s is not set while CI is: the SQL Server migration tests must not skip here", sqlserverDSNEnv)
}
t.Skipf("%s is not set; skipping the SQL Server migration tests", sqlserverDSNEnv)
}
db, err := gorm.Open(sqlserver.Open(dsn), &gorm.Config{})
if err != nil {
t.Fatalf("connecting to %s: %v", sqlserverDSNEnv, err)
}
return db
}
// freshSQLServerTables drops and rebuilds the two tables this migration
// touches, so a rerun does not inherit the previous run's index.
func freshSQLServerTables(t *testing.T, db *gorm.DB) {
t.Helper()
for _, m := range []any{&adminmodels.SysMenu{}, &adminmodels.SysApi{}} {
if db.Migrator().HasTable(m) {
if err := db.Migrator().DropTable(m); err != nil {
t.Fatalf("dropping: %v", err)
}
}
}
if err := db.AutoMigrate(&adminmodels.SysMenu{}, &adminmodels.SysApi{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
}
// The migration completes on SQL Server.
//
// It did not. Five menus with no seed_code is what 1786700001000 leaves on
// every database, and the unfiltered index rejects the second of them:
//
// Msg 1505 ... duplicate key ... The duplicate key value is (, <NULL>, 0).
func TestSeedNaturalKeysOnSQLServer(t *testing.T) {
db := sqlserverDB(t)
freshSQLServerTables(t, db)
// Three rows in the state 1786700006000 leaves behind: an app_code that
// defaulted to empty, no seed_code, and live.
for _, name := range []string{"one", "two", "three"} {
if err := db.Exec(
"INSERT INTO sys_menu (menu_name, app_code, deleted_at) VALUES (?, '', 0)", name,
).Error; err != nil {
t.Fatalf("seeding %s: %v", name, err)
}
}
// sys_api's key has two nullable columns and either one is enough to
// collide, so both shapes are here. Two rows missing both, and two more
// that have a path and no action: a filter naming only path would let
// that second pair back into the index, where their equal NULLs collide.
for i := 0; i < 2; i++ {
if err := db.Exec("INSERT INTO sys_api (app_code, deleted_at) VALUES ('', 0)").Error; err != nil {
t.Fatalf("seeding sys_api: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, deleted_at) VALUES ('', '/api/v1/half', 0)",
).Error; err != nil {
t.Fatalf("seeding a sys_api row with no action: %v", err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys on SQL Server: %v", err)
}
for _, name := range []string{"uk_sys_menu_app_seed_code_del", "uk_sys_api_app_path_action_del"} {
var model any = &adminmodels.SysMenu{}
if name == "uk_sys_api_app_path_action_del" {
model = &adminmodels.SysApi{}
}
if !db.Migrator().HasIndex(model, name) {
t.Errorf("%s was not created", name)
}
}
// Rows that do carry a seed code still cannot collide - the filter takes
// the ones with no value out of the index, it does not turn the index off.
code := "dir"
first := adminmodels.SysMenu{MenuName: "d1", AppCode: "order", SeedCode: &code}
if err := db.Create(&first).Error; err != nil {
t.Fatalf("first seeded menu: %v", err)
}
second := adminmodels.SysMenu{MenuName: "d2", AppCode: "order", SeedCode: &code}
if err := db.Create(&second).Error; err == nil {
t.Error("a duplicate (app_code, seed_code) was accepted; the filtered index is not enforcing anything")
}
// A different app may reuse the same seed code, which is why the key is
// composite in the first place.
other := adminmodels.SysMenu{MenuName: "d3", AppCode: "crm", SeedCode: &code}
if err := db.Create(&other).Error; err != nil {
t.Errorf("another app could not reuse the seed code: %v", err)
}
}
// The control. Without the filter the statement fails on this engine, so the
// test above is passing because of the fix rather than because SQL Server
// turned out not to mind.
func TestSQLServerRejectsTheUnfilteredIndex(t *testing.T) {
db := sqlserverDB(t)
freshSQLServerTables(t, db)
for _, name := range []string{"one", "two"} {
if err := db.Exec(
"INSERT INTO sys_menu (menu_name, app_code, deleted_at) VALUES (?, '', 0)", name,
).Error; err != nil {
t.Fatalf("seeding %s: %v", name, err)
}
}
err := db.Exec(uniqueIndexOverNullable("postgres",
"uk_unfiltered_probe", "sys_menu", "app_code, seed_code, deleted_at", "seed_code")).Error
if err == nil {
t.Fatal("SQL Server accepted two NULLs in a unique index; the filter this migration adds is not needed")
}
t.Logf("as expected: %v", err)
}
@@ -0,0 +1,305 @@
package version
import (
"testing"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
common "go-admin/common/models"
)
// oldSeedMenu/oldSeedApi are the shape of sys_menu/sys_api immediately
// before this migration: post-1786700003000 (deleted_at is the NOT NULL
// millisecond marker) and post-1786700006000 (app_code exists), but before
// seed_code or either unique index. They stand in for the real runtime
// models, which by the time this file is read already carry the columns
// this migration adds - the same relationship oldUser bears to sys_user in
// 1786700003000_soft_delete_marker_test.go.
type oldSeedMenu struct {
MenuId int `gorm:"column:menu_id;primaryKey;autoIncrement"`
AppCode string `gorm:"column:app_code;type:varchar(64);not null;default:''"`
DeletedAt int64 `gorm:"column:deleted_at;not null;default:0"`
}
func (oldSeedMenu) TableName() string { return "sys_menu" }
type oldSeedApi struct {
Id int `gorm:"column:id;primaryKey;autoIncrement"`
AppCode string `gorm:"column:app_code;type:varchar(64);not null;default:''"`
Path string `gorm:"column:path;type:varchar(128)"`
Action string `gorm:"column:action;type:varchar(16)"`
DeletedAt int64 `gorm:"column:deleted_at;not null;default:0"`
}
func (oldSeedApi) TableName() string { return "sys_api" }
func openSeedNaturalKeysDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("open: %v", err)
}
if err := db.AutoMigrate(&oldSeedMenu{}, &oldSeedApi{}, &common.Migration{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
return db
}
// The host's own hand-placed menus, and every app-seeded row written
// before this column existed, have no seed_code at all - an unbounded
// number of those must coexist under the same app_code without tripping
// the new unique index (design doc §1.6: "NULL never treated as equal to
// NULL").
func TestSeedNaturalKeysToleratesManyPreExistingMenusWithNoSeedCode(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 3; i++ {
if err := db.Create(&oldSeedMenu{AppCode: ""}).Error; err != nil {
t.Fatalf("seed pre-existing menu %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
t.Fatal("sys_menu.seed_code was not added")
}
}
// The point of adding seed_code at all: a second row with the same
// (app_code, seed_code) while both are live is what seedMenuTree's
// idempotency check depends on the database to reject if the Go-level
// check above it is ever bypassed or raced.
func TestSeedNaturalKeysMenuUniqueIndexBindsLiveRowsOnly(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error; err != nil {
t.Fatalf("seed: %v", err)
}
t.Run("a second live row with the same natural key is rejected", func(t *testing.T) {
err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error
if err == nil {
t.Fatal("a duplicate (app_code, seed_code) was accepted while both rows were live")
}
})
t.Run("the key is free again once the row is soft-deleted", func(t *testing.T) {
if err := db.Exec("UPDATE sys_menu SET deleted_at = ? WHERE seed_code = 'dir'", time.Now().UnixMilli()).Error; err != nil {
t.Fatalf("soft-delete: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_menu (app_code, seed_code, deleted_at) VALUES ('order', 'dir', 0)",
).Error; err != nil {
t.Errorf("the key stayed taken after its row was soft-deleted: %v", err)
}
})
}
// The demo site has already proven sys_api can hold historical duplicates;
// the migration has to name them and refuse, not let CREATE UNIQUE INDEX
// fail on an operator with no idea which rows to reconcile.
func TestSeedNaturalKeysRefusesDuplicateApis(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed duplicate %d: %v", i, err)
}
}
err := seedNaturalKeys(db)
if err == nil {
t.Fatal("the migration accepted sys_api rows that already hold a duplicate (app_code, path, action)")
}
if !contains(err.Error(), "order") || !contains(err.Error(), "/api/v1/order") {
t.Errorf("the error does not name the offending row: %v", err)
}
if db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was built despite the migration refusing")
}
// sys_menu's column and index are independent of sys_api's outcome and
// should already be in place - a partial failure here still leaves a
// record of what succeeded, same as any other non-transactional DDL
// migration in this package.
if !db.Migrator().HasColumn(&adminmodels.SysMenu{}, "SeedCode") {
t.Error("sys_menu.seed_code was not added even though only the sys_api step failed")
}
}
// Only live rows count towards the duplicate check: a row a prior,
// unrelated soft-delete already retired does not block the index it will
// never occupy a slot in.
func TestSeedNaturalKeysIgnoresSoftDeletedApiDuplicates(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed live row: %v", err)
}
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET", DeletedAt: time.Now().UnixMilli()}).Error; err != nil {
t.Fatalf("seed soft-deleted row: %v", err)
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built")
}
}
// The point of the sys_api index, mirroring
// TestSeedNaturalKeysMenuUniqueIndexBindsLiveRowsOnly above: a second live
// row is rejected, and the key is free again once the row is
// soft-deleted.
func TestSeedNaturalKeysApiUniqueIndexBindsLiveRowsOnly(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate: %v", err)
}
t.Run("a second live row with the same natural key is rejected", func(t *testing.T) {
err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', 'GET', 0)",
).Error
if err == nil {
t.Fatal("a duplicate (app_code, path, action) was accepted while both rows were live")
}
})
t.Run("the key is free again once the row is soft-deleted", func(t *testing.T) {
if err := db.Exec(
"UPDATE sys_api SET deleted_at = ? WHERE path = '/api/v1/order'", time.Now().UnixMilli(),
).Error; err != nil {
t.Fatalf("soft-delete: %v", err)
}
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', 'GET', 0)",
).Error; err != nil {
t.Errorf("the key stayed taken after its row was soft-deleted: %v", err)
}
})
}
// Running it twice must be safe: DDL does not roll back on MySQL, so an
// operator whose first attempt failed partway through (say, sys_menu's step
// succeeded and sys_api's refused) has nothing to do but run the whole
// migration again once the duplicates are reconciled.
func TestSeedNaturalKeysIsRepeatable(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := db.Create(&oldSeedApi{AppCode: "order", Path: "/api/v1/order", Action: "GET"}).Error; err != nil {
t.Fatalf("seed: %v", err)
}
for i := 0; i < 3; i++ {
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("migrate %d: %v", i, err)
}
}
}
// The wrapper's contract with Migrate.run(): the version is only recorded
// once the whole thing - both columns, both indexes - succeeded.
func TestSeedNaturalKeysWrapperRecordsTheVersion(t *testing.T) {
db := openSeedNaturalKeysDB(t)
if err := _1786700008000SeedNaturalKeys(db, "1786700008000"); err != nil {
t.Fatalf("migrate: %v", err)
}
var applied common.Migration
if err := db.Where("version = ?", "1786700008000").First(&applied).Error; err != nil {
t.Fatalf("sys_migration was not recorded: %v", err)
}
}
// GROUP BY treats two NULLs as equal for grouping purposes; a UNIQUE INDEX
// treats every NULL as distinct from every other value, including another
// NULL - both are standard SQL semantics, not a quirk of one dialect (see
// the postgres-only test file next to this one for the same check against
// a real server). A duplicate check that groups on the raw columns without
// accounting for that difference refuses an install the index itself would
// accept without complaint, on data there is nothing to "reconcile" -
// worse than the index simply failing to build, because it stops a library
// that has nothing wrong with it.
//
// sys_api.path/action carry no NOT NULL constraint - see the design doc's
// note on this migration for why that stays true in this batch, changing
// it is an independent, backward-incompatible migration of its own - so
// this state is reachable in a real database even though seedApis's own
// Create call, which always writes the Go zero value "" rather than NULL,
// never produces it itself. Inserted via raw SQL for exactly that reason:
// models.SysApi's Path/Action are plain (non-pointer) Go strings, which
// cannot represent NULL through a normal Create call.
func TestSeedNaturalKeysDoesNotFlagWhatTheIndexWouldAccept(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Exec(
"INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, NULL, 0)",
).Error; err != nil {
t.Fatalf("seed NULL row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built even though seedNaturalKeys reported success")
}
}
// The case above has both path and action NULL on every row, which both
// of the query's two NULL-exclusion conditions independently catch - it
// cannot tell "only path IS NOT NULL is doing anything here" apart from
// "both conditions are doing something". A row missing only one of the
// two is exactly as real (an api registered with a path but no method,
// or vice versa) and exercises only one condition at a time: two rows
// sharing a real path but both NULL in action, or two rows sharing a real
// action but both NULL in path. GROUP BY treats each pair's shared NULL
// the same way it treats a shared (NULL, NULL) - as equal - and the
// unique index accepts both pairs for the same reason it accepts the
// (NULL, NULL) case, so neither belongs in the count either.
func TestSeedNaturalKeysDoesNotFlagPartiallyNullRows(t *testing.T) {
cases := []struct {
name string
insert string // two rows, sharing a value in exactly one of path/action
}{
{
name: "path is null, action repeats",
insert: "INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', NULL, 'GET', 0)",
},
{
name: "action is null, path repeats",
insert: "INSERT INTO sys_api (app_code, path, action, deleted_at) VALUES ('order', '/api/v1/order', NULL, 0)",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
db := openSeedNaturalKeysDB(t)
for i := 0; i < 2; i++ {
if err := db.Exec(tc.insert).Error; err != nil {
t.Fatalf("seed row %d: %v", i, err)
}
}
if err := seedNaturalKeys(db); err != nil {
t.Fatalf("seedNaturalKeys refused a library the unique index itself accepts: %v", err)
}
if !db.Migrator().HasIndex(&adminmodels.SysApi{}, "uk_sys_api_app_path_action_del") {
t.Error("the unique index was not built even though seedNaturalKeys reported success")
}
})
}
}
@@ -0,0 +1,51 @@
package version
import (
"strings"
"testing"
)
// The index has to mean the same thing on every driver this repository
// registers, and the drivers do not agree about NULL.
//
// MySQL, PostgreSQL and SQLite treat two NULLs as different values, so any
// number of rows missing one of these columns coexist under the index. SQL
// Server treats them as equal and permits exactly one, so the unfiltered
// statement fails there on any database with two rows lacking a seed_code -
// which is every database, a brand-new one included, because 1786700001000
// seeds five menus and none of them carries one.
func TestUniqueIndexOverNullableFiltersOnlyWhereItHasTo(t *testing.T) {
const plain = "CREATE UNIQUE INDEX uk ON sys_menu (app_code, seed_code, deleted_at)"
for _, dialect := range []string{"mysql", "postgres", "sqlite"} {
got := uniqueIndexOverNullable(dialect, "uk", "sys_menu", "app_code, seed_code, deleted_at", "seed_code")
if got != plain {
t.Errorf("%s: %q\n want %q", dialect, got, plain)
}
}
got := uniqueIndexOverNullable("sqlserver", "uk", "sys_menu", "app_code, seed_code, deleted_at", "seed_code")
want := plain + " WHERE seed_code IS NOT NULL"
if got != want {
t.Errorf("sqlserver: %q\n want %q", got, want)
}
}
// sys_api's key has two nullable columns, and either one being NULL is enough
// to collide on SQL Server.
func TestUniqueIndexOverNullableCoversEveryNullableColumn(t *testing.T) {
got := uniqueIndexOverNullable("sqlserver", "uk", "sys_api",
"app_code, path, action, deleted_at", "path", "action")
if !strings.HasSuffix(got, " WHERE path IS NOT NULL AND action IS NOT NULL") {
t.Errorf("got %q", got)
}
}
// A key with nothing nullable in it needs no filter anywhere, or SQL Server
// would get a WHERE clause naming no column.
func TestUniqueIndexOverNullableWithoutNullableColumns(t *testing.T) {
got := uniqueIndexOverNullable("sqlserver", "uk", "sys_menu", "app_code, deleted_at")
if strings.Contains(got, "WHERE") {
t.Errorf("got %q", got)
}
}
+128 -9
View File
@@ -3,6 +3,7 @@ package migrate
import (
"bytes"
"fmt"
"io"
"os"
"strconv"
"strings"
@@ -14,6 +15,7 @@ import (
"gorm.io/gorm"
"github.com/go-admin-team/go-admin-core/v2/config/source/file"
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/app"
"github.com/spf13/cobra"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
@@ -47,6 +49,31 @@ var (
runStatus()
},
}
// Under migrate rather than under the existing `app` command, which
// already means "generate the skeleton of a new app" - a directory that
// does not exist yet, not an application already compiled into this
// binary. Installing an application is running its migrations, which is
// what this command is; --app, --domain and resolveDB are all already
// here, including the guard that refuses a mistyped code instead of
// reporting a successful no-op.
installCmd = &cobra.Command{
Use: "install <code>",
Short: "Install one application: run its migrations and record it in sys_app",
Example: "go-admin migrate install order -c config/settings.yml",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
runInstall(args[0])
},
}
uninstallCmd = &cobra.Command{
Use: "uninstall <code>",
Short: "Remove one application's menus, apis and permission grants; its own tables are left alone",
Example: "go-admin migrate uninstall order -c config/settings.yml",
Args: cobra.ExactArgs(1),
Run: func(cmd *cobra.Command, args []string) {
runUninstall(args[0])
},
}
)
// fixme 在您看不见代码的时候运行迁移,我觉得是不安全的,所以编译后最好不要去执行迁移
@@ -64,6 +91,8 @@ func init() {
StartCmd.Flags().BoolVar(&dryRun, "dry-run", false, "list what would be applied, in order, and write nothing")
StartCmd.AddCommand(statusCmd)
StartCmd.AddCommand(installCmd)
StartCmd.AddCommand(uninstallCmd)
}
func run() {
@@ -162,11 +191,9 @@ func migrateModel() error {
}
migration.Migrate.SetDb(db.Debug())
if appCode != "" {
migration.Migrate.MigrateApp(appCode)
return nil
return migration.Migrate.MigrateApp(appCode)
}
migration.Migrate.Migrate()
return nil
return migration.Migrate.Migrate()
}
func initDB() {
@@ -197,13 +224,40 @@ func initDB() {
//4. 数据库迁移
fmt.Println("数据库迁移开始")
if err := migrateModel(); err != nil {
fmt.Println(err)
return
}
exitOnError(os.Stderr, migrateModel())
fmt.Println(`数据库基础数据初始化成功`)
}
// exitOnError ends the command non-zero when the migration did not go through.
//
// A caller that migrates before starting a server decides whether to go ahead
// on the exit code alone - the deploy workflow does exactly that. Every path
// out of migrateModel used to return without one: an unreachable tenant
// database or a failed AutoMigrate printed a line and exited 0, so a
// deployment carried on onto a schema that had not been brought forward. A
// failing migration function was the only one reported, and only because it
// ended the process from inside the migration engine - which is the call this
// batch moved out here, so without this the last reported failure would have
// stopped being reported too.
//
// Split from the exit itself, the way appRegistrationError is split from
// exitUnlessAppRegistered, so what it decides can be tested without a
// subprocess. osExit is a variable for the same reason.
func exitOnError(w io.Writer, err error) {
if err == nil {
return
}
fmt.Fprintln(w, err)
osExit(1)
}
// osExit is a variable so a test can watch the decision without ending the
// test binary; origExit is what it is put back to.
var (
osExit = os.Exit
origExit = os.Exit
)
func runStatus() {
config.Setup(
file.NewSource(file.WithPath(configYml)),
@@ -222,13 +276,78 @@ func runStatus() {
fmt.Println(err)
return
}
if err = printStatus(os.Stdout, entries, appCode); err != nil {
// Which applications exist is a different question from which
// migrations ran, and an install that stopped partway is only
// visible in the answer to the first.
apps, err := loadApps(db)
if err != nil {
fmt.Println(err)
return
}
if err = printStatus(os.Stdout, entries, apps, appCode); err != nil {
fmt.Println(err)
}
},
)
}
func runInstall(code string) {
config.Setup(
file.NewSource(file.WithPath(configYml)),
func() {
database.Setup()
db, err := resolveDB()
if err != nil {
exitOnError(os.Stderr, err)
return
}
registered := app.Snapshot()
m, err := manifestFor(registered, code)
if err != nil {
exitOnError(os.Stderr, err)
return
}
// Over every registered manifest, not just this one's closure: a
// cycle between two other applications is still an authoring
// mistake, and the day somebody installs into it is the worse
// time to find out.
if err := refuseOnDependencyCycle(registered); err != nil {
exitOnError(os.Stderr, err)
return
}
rep, err := install(db, migration.Migrate, m)
if err != nil {
exitOnError(os.Stderr, err)
return
}
reportInstall(os.Stdout, rep)
},
)
}
func runUninstall(code string) {
config.Setup(
file.NewSource(file.WithPath(configYml)),
func() {
database.Setup()
db, err := resolveDB()
if err != nil {
exitOnError(os.Stderr, err)
return
}
// No manifest lookup. An application whose code has already been
// taken out of the binary registers nothing, and that is exactly
// when somebody needs to clear its rows out of the database.
rep, err := uninstall(db, code)
if err != nil {
exitOnError(os.Stderr, err)
return
}
reportUninstall(os.Stdout, rep)
},
)
}
func genFile() error {
t1, err := template.ParseFiles("template/migrate.template")
if err != nil {
+79 -4
View File
@@ -7,6 +7,7 @@ import (
"strings"
"time"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
)
@@ -15,11 +16,24 @@ const applyTimeLayout = "2006-01-02 15:04:05"
// printStatus lists every migration this binary knows about together with every
// row already in sys_migration, grouped by app.
//
// apps is what sys_app says about each of them, keyed by app code, and it
// answers a different question from the migration rows: an install that
// stopped partway leaves migrations that all read "applied" and a row that
// says the install never finished. A nil map is a database from before
// sys_app existed, and the listing is then exactly what it was.
//
// The app list is the union of the two. Reading it from sys_app alone would
// drop an application whose migrations ran under plain `migrate` and which
// therefore has no row; reading it from the migration rows alone drops one
// whose code has been taken out of the binary, which is when somebody most
// wants to see it named.
//
// filter is an app code as typed on the command line; empty means every app.
func printStatus(w io.Writer, entries []migration.StatusEntry, filter string) error {
func printStatus(w io.Writer, entries []migration.StatusEntry, apps map[string]adminmodels.SysApp, filter string) error {
entries = filterByApp(entries, filter)
apps = filterAppsByApp(apps, filter)
groups, order := groupByApp(entries)
groups, order := groupByApp(entries, apps)
if len(order) == 0 {
_, err := fmt.Fprintln(w, "no migrations registered and none recorded")
return err
@@ -34,7 +48,14 @@ func printStatus(w io.Writer, entries []migration.StatusEntry, filter string) er
if i > 0 {
fmt.Fprintln(w)
}
fmt.Fprintf(w, "[%s]\n", app)
fmt.Fprintf(w, "[%s]%s\n", app, appSummary(apps, app))
if len(groups[app]) == 0 {
// A row in sys_app and not one migration, recorded or
// registered. Its code is out of this binary and its migration
// records have been removed, and the row is all that is left to
// say it was ever here.
fmt.Fprintln(w, " no migrations registered in this binary and none recorded")
}
for _, e := range groups[app] {
state := "pending"
switch {
@@ -133,12 +154,21 @@ func filterByApp(entries []migration.StatusEntry, filter string) []migration.Sta
// order to print them in: the framework first, then apps alphabetically. That
// is also the order a full run executes them in, because version strings sort
// as ASCII and the framework's are bare digits.
func groupByApp(entries []migration.StatusEntry) (map[string][]migration.StatusEntry, []string) {
func groupByApp(entries []migration.StatusEntry, apps map[string]adminmodels.SysApp) (map[string][]migration.StatusEntry, []string) {
groups := make(map[string][]migration.StatusEntry)
for _, e := range entries {
app := migration.DisplayAppCode(e.AppCode)
groups[app] = append(groups[app], e)
}
// An application sys_app knows about and no migration mentions still gets
// a group, empty. That is the one case the migration rows cannot report
// at all.
for code := range apps {
app := migration.DisplayAppCode(code)
if _, ok := groups[app]; !ok {
groups[app] = nil
}
}
order := make([]string, 0, len(groups))
for app := range groups {
order = append(order, app)
@@ -152,6 +182,51 @@ func groupByApp(entries []migration.StatusEntry) (map[string][]migration.StatusE
return groups, order
}
// appSummary is what sys_app says about one application, as a suffix for its
// group header. Empty when there is no row: an application whose migrations
// ran under plain `migrate` has none, and neither does any application on a
// database from before sys_app existed.
func appSummary(apps map[string]adminmodels.SysApp, display string) string {
// AppFilter, not NormalizeAppCode: this takes a display code back to the
// stored one, and only AppFilter is that inverse. It maps the framework
// to the empty string, which loadApps never files a row under, so the
// framework needs no branch of its own here.
row, ok := apps[migration.AppFilter(display)]
if !ok {
return ""
}
switch row.Status {
case adminmodels.AppInstalled:
return fmt.Sprintf(" %s installed", row.Version)
case adminmodels.AppFailed:
if row.FailedVersion != "" {
return fmt.Sprintf(" %s failed at %s", row.Version, row.FailedVersion)
}
return fmt.Sprintf(" %s failed", row.Version)
case adminmodels.AppInstalling:
// Not "installing" as in "right now": nothing holds this state while
// it works. It is what is left when an attempt did not reach either
// end, and running the install again is what clears it.
return fmt.Sprintf(" %s did not finish installing", row.Version)
default:
return fmt.Sprintf(" %s status %d", row.Version, row.Status)
}
}
// filterAppsByApp narrows the sys_app rows the same way filterByApp narrows
// the migrations, so --app names one application in both halves of the report.
func filterAppsByApp(apps map[string]adminmodels.SysApp, filter string) map[string]adminmodels.SysApp {
if filter == "" {
return apps
}
want := migration.AppFilter(filter)
out := make(map[string]adminmodels.SysApp, 1)
if row, ok := apps[want]; ok {
out[want] = row
}
return out
}
func formatApplyTime(t *time.Time) string {
if t == nil {
return ""
+119 -5
View File
@@ -6,6 +6,7 @@ import (
"testing"
"time"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
)
@@ -29,7 +30,7 @@ func sampleEntries() []migration.StatusEntry {
func TestPrintStatusGroupsByApp(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), ""); err != nil {
if err := printStatus(&buf, sampleEntries(), nil, ""); err != nil {
t.Fatal(err)
}
got := buf.String()
@@ -62,7 +63,7 @@ func TestPrintStatusMarksOrphanedRows(t *testing.T) {
Version: "gone-1786800000000", AppCode: "gone", Applied: true, ApplyTime: at("2026-08-01 09:00:00"),
})
var buf bytes.Buffer
if err := printStatus(&buf, entries, ""); err != nil {
if err := printStatus(&buf, entries, nil, ""); err != nil {
t.Fatal(err)
}
got := buf.String()
@@ -79,7 +80,7 @@ func TestPrintStatusMarksOrphanedRows(t *testing.T) {
func TestPrintStatusFiltersByApp(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), "crm"); err != nil {
if err := printStatus(&buf, sampleEntries(), nil, "crm"); err != nil {
t.Fatal(err)
}
got := buf.String()
@@ -94,7 +95,7 @@ func TestPrintStatusFiltersByApp(t *testing.T) {
// status prints [core]; --app core has to mean the same thing.
func TestPrintStatusAppCoreSelectsTheFramework(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), migration.FrameworkAppCode); err != nil {
if err := printStatus(&buf, sampleEntries(), nil, migration.FrameworkAppCode); err != nil {
t.Fatal(err)
}
got := buf.String()
@@ -108,7 +109,7 @@ func TestPrintStatusAppCoreSelectsTheFramework(t *testing.T) {
func TestPrintStatusOnAnEmptyRegistry(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, nil, ""); err != nil {
if err := printStatus(&buf, nil, nil, ""); err != nil {
t.Fatal(err)
}
if !strings.Contains(buf.String(), "no migrations registered and none recorded") {
@@ -168,3 +169,116 @@ func TestPrintPendingFiltersByApp(t *testing.T) {
t.Errorf("output = %s", got)
}
}
func appRows(rows ...adminmodels.SysApp) map[string]adminmodels.SysApp {
out := make(map[string]adminmodels.SysApp, len(rows))
for _, r := range rows {
out[r.AppCode] = r
}
return out
}
// The migration rows say every one of an application's migrations ran. Only
// sys_app can say the install that ran them never finished.
func TestPrintStatusShowsWhatSysAppSays(t *testing.T) {
apps := appRows(
adminmodels.SysApp{AppCode: "crm", Version: "1.2.0", Status: adminmodels.AppInstalled},
)
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), apps, ""); err != nil {
t.Fatal(err)
}
got := buf.String()
if !strings.Contains(got, "[crm] 1.2.0 installed") {
t.Errorf("the header does not carry what sys_app says:\n%s", got)
}
// The framework is not an application and has no row.
if strings.Contains(got, "[core] ") {
t.Errorf("the framework was given an application summary:\n%s", got)
}
}
func TestPrintStatusNamesAFailedInstallAndWhereItStopped(t *testing.T) {
apps := appRows(adminmodels.SysApp{
AppCode: "crm", Version: "1.2.0", Status: adminmodels.AppFailed,
FailedVersion: "crm-1786800002000",
})
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), apps, ""); err != nil {
t.Fatal(err)
}
if !strings.Contains(buf.String(), "[crm] 1.2.0 failed at crm-1786800002000") {
t.Errorf("output:\n%s", buf.String())
}
}
// A process killed partway leaves this, and nothing else records it.
func TestPrintStatusNamesAnInstallThatDidNotFinish(t *testing.T) {
apps := appRows(adminmodels.SysApp{AppCode: "crm", Version: "1.2.0", Status: adminmodels.AppInstalling})
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), apps, ""); err != nil {
t.Fatal(err)
}
if !strings.Contains(buf.String(), "did not finish installing") {
t.Errorf("output:\n%s", buf.String())
}
}
// An application whose code was taken out of the binary after its migration
// records were removed has nothing left but a sys_app row. The migration rows
// cannot report it at all.
func TestPrintStatusListsAnAppWithNoMigrationsAtAll(t *testing.T) {
apps := appRows(adminmodels.SysApp{AppCode: "billing", Version: "3.0.0", Status: adminmodels.AppInstalled})
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), apps, ""); err != nil {
t.Fatal(err)
}
got := buf.String()
if !strings.Contains(got, "[billing] 3.0.0 installed") {
t.Errorf("an application only sys_app knows about was not listed:\n%s", got)
}
if !strings.Contains(got, "no migrations registered in this binary and none recorded") {
t.Errorf("the empty group needs to say why it is empty:\n%s", got)
}
if !strings.Contains(got, "across 3 app(s)") {
t.Errorf("the count does not include it:\n%s", got)
}
}
// --app narrows both halves, or the report names one application and
// summarises another.
func TestPrintStatusFilterAppliesToSysAppToo(t *testing.T) {
apps := appRows(
adminmodels.SysApp{AppCode: "crm", Version: "1.2.0", Status: adminmodels.AppInstalled},
adminmodels.SysApp{AppCode: "billing", Version: "3.0.0", Status: adminmodels.AppInstalled},
)
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), apps, "crm"); err != nil {
t.Fatal(err)
}
got := buf.String()
if strings.Contains(got, "billing") {
t.Errorf("--app crm listed billing:\n%s", got)
}
if !strings.Contains(got, "across 1 app(s)") {
t.Errorf("output:\n%s", got)
}
}
// A database from before sys_app existed. The listing is what it always was,
// rather than an error or an empty report.
func TestPrintStatusWithoutSysApp(t *testing.T) {
var withRows, without bytes.Buffer
if err := printStatus(&without, sampleEntries(), nil, ""); err != nil {
t.Fatal(err)
}
if err := printStatus(&withRows, sampleEntries(), map[string]adminmodels.SysApp{}, ""); err != nil {
t.Fatal(err)
}
if without.String() != withRows.String() {
t.Errorf("an empty sys_app and no sys_app print differently:\n%s\n---\n%s", without.String(), withRows.String())
}
if !strings.Contains(without.String(), "[crm]\n") {
t.Errorf("the header carries a summary it has no row for:\n%s", without.String())
}
}
+296
View File
@@ -0,0 +1,296 @@
package migrate
import (
"errors"
"fmt"
"io"
"strings"
"gorm.io/gorm"
adminmodels "go-admin/app/admin/models"
"go-admin/cmd/migrate/migration"
commonmodels "go-admin/common/models"
)
// policyKey is one casbin_rule row identified the way casbin_rule is unique:
// by its tuple, not by its id. Ids do not survive SysRole.Update, which
// removes a role's policies and adds them back.
type policyKey struct {
Ptype string `gorm:"column:ptype"`
V0 string `gorm:"column:v0"`
V1 string `gorm:"column:v1"`
V2 string `gorm:"column:v2"`
V3 string `gorm:"column:v3"`
V4 string `gorm:"column:v4"`
V5 string `gorm:"column:v5"`
}
func (p policyKey) String() string {
return fmt.Sprintf("%s %s %s %s", p.Ptype, p.V0, p.V1, p.V2)
}
// uninstallReport is what an uninstall removed, and what it deliberately did
// not.
type uninstallReport struct {
Code string
// Found says whether sys_app had a row. An application whose migrations
// were applied by plain `migrate` rather than by `install` has its menus
// and its permissions without ever having had one.
Found bool
Version string
Menus int64
Apis int64
Bindings int64
RoleMenus int64
Policies int64
Migrations int64
// Skipped are ledger entries whose casbin_rule row was not there any
// more: something this install created and something else removed.
Skipped []policyKey
// Orphans are policies naming this application's paths that no ledger
// entry claims - somebody granted this app's API to another role by
// hand. Reported, never deleted.
Orphans []policyKey
}
// uninstall removes one application's menus, APIs and permission grants.
//
// It does not touch the application's own tables. Removing an order module
// is not the same decision as destroying the orders, and nothing here can
// tell the operator apart from someone who will reinstall tomorrow.
//
// One transaction, and this one really is one: every statement below is DML
// or a SELECT, so unlike an install there is no DDL to commit it out from
// under itself. Child rows go first, while the ids that identify them can
// still be read from the parents.
//
// A sys_app row is not required. `migrate` with no subcommand applies every
// registered migration, an application's included, so an application can
// have all of its data without ever having gone through the installer.
func uninstall(db *gorm.DB, code string) (uninstallReport, error) {
code = migration.NormalizeAppCode(code)
rep := uninstallReport{Code: code}
if code == "" {
return rep, errors.New("no app code given")
}
if code == migration.FrameworkAppCode {
return rep, fmt.Errorf("%q is the framework's own migrations; there is no uninstall for those", code)
}
err := db.Transaction(func(tx *gorm.DB) error {
row, found, err := loadApp(tx, code)
if err != nil {
return err
}
rep.Found = found
if found {
rep.Version = row.Version
}
// 1 and 2. Read before deleting: sys_api's rows are about to go, and
// step 5b needs their paths.
//
// Unscoped throughout. A row this application wrote that somebody
// soft-deleted from the UI is still this application's row, and
// leaving it behind would leave its join rows pointing at it.
var menuIDs []int
if err := tx.Unscoped().Model(&adminmodels.SysMenu{}).
Where("app_code = ?", code).Pluck("menu_id", &menuIDs).Error; err != nil {
return fmt.Errorf("reading this app's menus: %w", err)
}
var apiIDs []int
if err := tx.Unscoped().Model(&adminmodels.SysApi{}).
Where("app_code = ?", code).Pluck("id", &apiIDs).Error; err != nil {
return fmt.Errorf("reading this app's apis: %w", err)
}
var apiKeys []policyKey
if err := tx.Unscoped().Model(&adminmodels.SysApi{}).
Where("app_code = ?", code).
Select("path as v1, action as v2").Scan(&apiKeys).Error; err != nil {
return fmt.Errorf("reading this app's api paths: %w", err)
}
// 3. The many2many rows behind SysMenu.SysApi. Either side is enough
// to make a row this application's.
//
// The guard is intent, not necessity: GORM renders IN with an empty
// slice as a condition matching nothing rather than the empty IN
// list raw SQL would reject, so removing it changes no behaviour
// today. It says out loud that an application with no menus, or no
// apis, is a normal thing to uninstall.
if len(menuIDs) > 0 || len(apiIDs) > 0 {
q := tx.Table("sys_menu_api_rule")
switch {
case len(menuIDs) > 0 && len(apiIDs) > 0:
q = q.Where("sys_menu_menu_id IN ? OR sys_api_id IN ?", menuIDs, apiIDs)
case len(menuIDs) > 0:
q = q.Where("sys_menu_menu_id IN ?", menuIDs)
default:
q = q.Where("sys_api_id IN ?", apiIDs)
}
res := q.Delete(nil)
if res.Error != nil {
return fmt.Errorf("removing menu/api bindings: %w", res.Error)
}
rep.Bindings = res.RowsAffected
}
// 4. Role assignments. menu_id is a surrogate key, so a row here can
// only have come from a menu this application wrote - there is no
// "looks like it but is not". That is why this needs no ledger, and
// why a column on sys_role_menu would have been wrong: SysRole.Update
// deletes a role's rows and writes them back through GORM's
// many2many, which does not carry extra columns, so any such column
// would be silently blanked the first time somebody edits a role.
if len(menuIDs) > 0 {
res := tx.Table("sys_role_menu").Where("menu_id IN ?", menuIDs).Delete(nil)
if res.Error != nil {
return fmt.Errorf("removing role assignments: %w", res.Error)
}
rep.RoleMenus = res.RowsAffected
}
// 5. Policies, by ledger, one at a time and by exact tuple.
var grants []adminmodels.SysAppCasbinGrant
if err := tx.Where("app_code = ?", code).Find(&grants).Error; err != nil {
return fmt.Errorf("reading the grant ledger: %w", err)
}
for _, g := range grants {
k := policyKey{Ptype: g.Ptype, V0: g.V0, V1: g.V1, V2: g.V2, V3: g.V3, V4: g.V4, V5: g.V5}
res := tx.Table("casbin_rule").
Where("ptype = ? AND v0 = ? AND v1 = ? AND v2 = ? AND v3 = ? AND v4 = ? AND v5 = ?",
k.Ptype, k.V0, k.V1, k.V2, k.V3, k.V4, k.V5).
Delete(nil)
if res.Error != nil {
return fmt.Errorf("removing policy %s: %w", k, res.Error)
}
if res.RowsAffected == 0 {
// Something this install created is not there any more. Not
// an error: the uninstall's job was to remove it and it is
// gone. Reported because a policy this app created and did
// not remove means something else rewrote casbin_rule.
rep.Skipped = append(rep.Skipped, k)
continue
}
rep.Policies += res.RowsAffected
}
// The ledger's job ends here whether or not each row matched. Left
// behind it would only grow, and a reinstall writes its own entries.
if err := tx.Where("app_code = ?", code).
Delete(&adminmodels.SysAppCasbinGrant{}).Error; err != nil {
return fmt.Errorf("clearing the grant ledger: %w", err)
}
// 5b. Read-only. By now every policy the ledger could speak for has
// been dealt with, so a policy still matching one of this app's paths
// is one the ledger never claimed - somebody granted this app's API
// to another role by hand. Business rule 3 says do not delete what
// is not ours; without this step nobody would ever learn it is
// there, pointing at an API that is about to stop existing.
orphans, err := findOrphanPolicies(tx, apiKeys)
if err != nil {
return err
}
rep.Orphans = orphans
// 6 and 7.
res := tx.Unscoped().Where("app_code = ?", code).Delete(&adminmodels.SysApi{})
if res.Error != nil {
return fmt.Errorf("removing this app's apis: %w", res.Error)
}
rep.Apis = res.RowsAffected
res = tx.Unscoped().Where("app_code = ?", code).Delete(&adminmodels.SysMenu{})
if res.Error != nil {
return fmt.Errorf("removing this app's menus: %w", res.Error)
}
rep.Menus = res.RowsAffected
// 8. Without this a reinstall finds every version already applied,
// runs no migration, seeds nothing, and reports success. It is the
// easiest step to leave out, because a migration record does not
// look like the application's data.
res = tx.Where("app_code = ?", code).Delete(&commonmodels.Migration{})
if res.Error != nil {
return fmt.Errorf("removing this app's migration records: %w", res.Error)
}
rep.Migrations = res.RowsAffected
// 9.
if found {
if err := tx.Where("app_code = ?", code).
Delete(&adminmodels.SysApp{}).Error; err != nil {
return fmt.Errorf("removing the sys_app row: %w", err)
}
}
return nil
})
if err != nil {
return uninstallReport{Code: code}, err
}
return rep, nil
}
// findOrphanPolicies looks for policies naming any of this application's
// paths.
//
// Written as an OR chain rather than a row-value IN, which MySQL and modern
// SQLite accept and SQL Server does not; this repository supports all of
// them. Chunked because a driver's placeholder limit is reached long before
// an application runs out of endpoints.
func findOrphanPolicies(tx *gorm.DB, keys []policyKey) ([]policyKey, error) {
const chunk = 100
var out []policyKey
for start := 0; start < len(keys); start += chunk {
end := start + chunk
if end > len(keys) {
end = len(keys)
}
clauses := make([]string, 0, end-start)
args := make([]any, 0, (end-start)*2)
for _, k := range keys[start:end] {
clauses = append(clauses, "(v1 = ? AND v2 = ?)")
args = append(args, k.V1, k.V2)
}
var found []policyKey
if err := tx.Table("casbin_rule").
Where("ptype = ? AND ("+strings.Join(clauses, " OR ")+")", append([]any{"p"}, args...)...).
Scan(&found).Error; err != nil {
return nil, fmt.Errorf("looking for policies nothing claims: %w", err)
}
out = append(out, found...)
}
return out, nil
}
// reportUninstall prints what went and what stayed.
//
// The two lists are separate because they mean different things: one is
// something this application created that had already gone, the other is
// somebody else's grant that is now pointing at nothing. Merged into one
// "could not remove" list, neither would be actionable.
func reportUninstall(w io.Writer, rep uninstallReport) {
if !rep.Found {
fmt.Fprintf(w, "%s had no sys_app row; removed what its migrations had written\n", rep.Code)
} else {
fmt.Fprintf(w, "uninstalled %s %s\n", rep.Code, rep.Version)
}
fmt.Fprintf(w, "removed: %d menu(s), %d api(s), %d binding(s), %d role assignment(s), %d policy(ies), %d migration record(s)\n",
rep.Menus, rep.Apis, rep.Bindings, rep.RoleMenus, rep.Policies, rep.Migrations)
fmt.Fprintln(w, "the application's own tables were not touched.")
if len(rep.Skipped) > 0 {
fmt.Fprintf(w, "\n%d policy(ies) this install had created were already gone:\n", len(rep.Skipped))
for _, k := range rep.Skipped {
fmt.Fprintf(w, " %s\n", k)
}
}
if len(rep.Orphans) > 0 {
fmt.Fprintf(w, "\n%d policy(ies) name this application's paths and were granted by somebody else, so they were left alone:\n", len(rep.Orphans))
for _, k := range rep.Orphans {
fmt.Fprintf(w, " %s\n", k)
}
fmt.Fprintln(w, "they now point at APIs that no longer exist. Harmless to the running server, and yours to clear up.")
}
}
+500
View File
@@ -0,0 +1,500 @@
package migrate
import (
"strconv"
"strings"
"testing"
"time"
"github.com/glebarez/sqlite"
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/seed"
"gorm.io/gorm"
"gorm.io/gorm/logger"
adminmodels "go-admin/app/admin/models"
_ "go-admin/app/admin/service" // registers the seeder SeedMenus dispatches to
"go-admin/cmd/migrate/migration"
commonmodels "go-admin/common/models"
)
const adminRoleKey = "admin"
// newUninstallDB builds every table an install writes to, plus one table
// standing in for the application's own data, which an uninstall must not
// touch.
func newUninstallDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
if err := db.AutoMigrate(
&adminmodels.SysMenu{}, &adminmodels.SysApi{}, &adminmodels.SysRole{},
&adminmodels.SysApp{}, &adminmodels.SysAppCasbinGrant{}, &commonmodels.Migration{},
); err != nil {
t.Fatalf("automigrate: %v", err)
}
// casbin_rule has no GORM model in this repository; the columns are the
// ones grantToAdminRole's INSERT addresses.
if err := db.Exec(`CREATE TABLE casbin_rule (
id INTEGER PRIMARY KEY AUTOINCREMENT,
ptype TEXT, v0 TEXT, v1 TEXT, v2 TEXT, v3 TEXT, v4 TEXT, v5 TEXT
)`).Error; err != nil {
t.Fatalf("create casbin_rule: %v", err)
}
if err := db.Exec(`CREATE TABLE app_order (id INTEGER PRIMARY KEY, note TEXT)`).Error; err != nil {
t.Fatalf("create app_order: %v", err)
}
if err := db.Exec(`INSERT INTO app_order (id, note) VALUES (1, 'a real order')`).Error; err != nil {
t.Fatalf("seed app_order: %v", err)
}
if err := db.Create(&adminmodels.SysRole{RoleName: "Administrator", RoleKey: adminRoleKey}).Error; err != nil {
t.Fatalf("seed admin role: %v", err)
}
return db
}
// specsFor builds one application's menus and apis. The paths carry the app
// code, because two applications do not share an endpoint - and if a fixture
// let them, the second one's policies would already exist and its ledger
// would legitimately come out empty, which would make it a useless control.
func specsFor(code string) ([]seed.MenuSpec, []seed.ApiSpec) {
menus := []seed.MenuSpec{
{Code: "dir", Kind: "M", Title: code + " example", Path: "/apps/" + code, Component: "Layout", Sort: 10},
{Code: "list", Parent: "dir", Kind: "C", Title: code, Path: "list", Component: "apps/" + code + "/index", Sort: 1, ApiCodes: []string{"list"}},
}
apis := []seed.ApiSpec{
{Code: "list", Title: code + " list", Path: "/api/v1/" + code, Method: "GET", Handle: "apis." + code + ".GetPage-fm"},
{Code: "create", Title: "create " + code, Path: "/api/v1/" + code, Method: "POST", Handle: "apis." + code + ".Insert-fm"},
}
return menus, apis
}
// seedApp runs the real seeding path, so what the uninstaller has to undo is
// what an install actually writes rather than a hand-built approximation.
func seedApp(t *testing.T, db *gorm.DB, code string) {
t.Helper()
menus, apis := specsFor(code)
if err := db.Transaction(func(tx *gorm.DB) error {
return seed.SeedMenus(tx, code, menus, apis)
}); err != nil {
t.Fatalf("seeding %q: %v", code, err)
}
if err := db.Create(&commonmodels.Migration{
Version: code + "-1786800001000", AppCode: code, ApplyTime: time.Now(),
}).Error; err != nil {
t.Fatalf("recording the migration for %q: %v", code, err)
}
appRow(t, db, code, adminmodels.AppInstalled)
}
func count(t *testing.T, db *gorm.DB, table, where string, args ...any) int64 {
t.Helper()
var n int64
q := db.Table(table)
if where != "" {
q = q.Where(where, args...)
}
if err := q.Count(&n).Error; err != nil {
t.Fatalf("counting %s: %v", table, err)
}
return n
}
// A3: everything the install wrote goes, and the application's own table does
// not.
func TestUninstallRemovesWhatWasSeededAndNothingElse(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
if count(t, db, "sys_menu", "app_code = ?", "order") == 0 {
t.Fatal("nothing was seeded, so this test proves nothing")
}
rep, err := uninstall(db, "order")
if err != nil {
t.Fatalf("uninstall: %v", err)
}
if !rep.Found {
t.Error("the sys_app row was not found")
}
for _, c := range []struct {
table, where string
args []any
}{
{"sys_menu", "app_code = ?", []any{"order"}},
{"sys_api", "app_code = ?", []any{"order"}},
{"sys_menu_api_rule", "", nil},
{"sys_role_menu", "", nil},
{"casbin_rule", "v1 = ?", []any{"/api/v1/order"}},
{"sys_app_casbin_grant", "app_code = ?", []any{"order"}},
{"sys_migration", "app_code = ?", []any{"order"}},
{"sys_app", "app_code = ?", []any{"order"}},
} {
if n := count(t, db, c.table, c.where, c.args...); n != 0 {
t.Errorf("%s still has %d row(s)", c.table, n)
}
}
if n := count(t, db, "app_order", "", nil); n != 1 {
t.Errorf("app_order has %d row(s); the application's own data is not the uninstaller's to remove", n)
}
if len(rep.Skipped) != 0 || len(rep.Orphans) != 0 {
t.Errorf("a clean uninstall reported skipped=%v orphans=%v", rep.Skipped, rep.Orphans)
}
if rep.Menus == 0 || rep.Apis == 0 || rep.Policies == 0 || rep.Migrations == 0 {
t.Errorf("the report says nothing was removed: %+v", rep)
}
}
// Uninstalling one application must not reach into another's rows. Every
// delete here is filtered, and a missing filter is invisible on a database
// with only one application in it.
func TestUninstallLeavesAnotherApplicationAlone(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
seedApp(t, db, "crm")
before := map[string]int64{
"sys_menu": count(t, db, "sys_menu", "app_code = ?", "crm"),
"sys_api": count(t, db, "sys_api", "app_code = ?", "crm"),
"sys_app_casbin_grant": count(t, db, "sys_app_casbin_grant", "app_code = ?", "crm"),
"sys_migration": count(t, db, "sys_migration", "app_code = ?", "crm"),
"sys_app": count(t, db, "sys_app", "app_code = ?", "crm"),
}
for k, v := range before {
if v == 0 {
t.Fatalf("crm has no rows in %s, so this test proves nothing", k)
}
}
crmBindings := count(t, db, "sys_menu_api_rule", "", nil)
crmRoleMenus := count(t, db, "sys_role_menu", "", nil)
if _, err := uninstall(db, "order"); err != nil {
t.Fatalf("uninstall: %v", err)
}
for k, v := range before {
if n := count(t, db, k, "app_code = ?", "crm"); n != v {
t.Errorf("%s for crm went from %d to %d", k, v, n)
}
}
// crm's own bindings and role rows are half of each total, and must be
// exactly what is left.
if n := count(t, db, "sys_menu_api_rule", "", nil); n != crmBindings/2 {
t.Errorf("sys_menu_api_rule = %d, want %d (crm's half)", n, crmBindings/2)
}
if n := count(t, db, "sys_role_menu", "", nil); n != crmRoleMenus/2 {
t.Errorf("sys_role_menu = %d, want %d (crm's half)", n, crmRoleMenus/2)
}
// crm's policies name a different path, so they are untouched.
if n := count(t, db, "casbin_rule", "v1 = ?", "/api/v1/order"); n != 0 {
t.Errorf("order's policies survived: %d", n)
}
}
// A6b: somebody granted this application's API to another role by hand. That
// grant is not in the ledger, is not this uninstall's to remove, and would
// otherwise vanish from view entirely.
func TestUninstallReportsAGrantSomebodyElseMade(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
if err := db.Exec(
"INSERT INTO casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'ops', '/api/v1/order', 'GET', '', '', '')",
).Error; err != nil {
t.Fatalf("hand-made grant: %v", err)
}
rep, err := uninstall(db, "order")
if err != nil {
t.Fatalf("uninstall: %v", err)
}
if n := count(t, db, "casbin_rule", "v0 = ?", "ops"); n != 1 {
t.Errorf("somebody else's grant was deleted (%d rows left)", n)
}
if len(rep.Orphans) != 1 {
t.Fatalf("orphans = %v, want the one hand-made grant", rep.Orphans)
}
if rep.Orphans[0].V0 != "ops" {
t.Errorf("orphan = %+v", rep.Orphans[0])
}
// The admin grants it did own are gone.
if n := count(t, db, "casbin_rule", "v0 = ?", adminRoleKey); n != 0 {
t.Errorf("%d of this app's own policies survived", n)
}
var out strings.Builder
reportUninstall(&out, rep)
if !strings.Contains(out.String(), "ops") || !strings.Contains(out.String(), "left alone") {
t.Errorf("the report does not say what was left behind: %q", out.String())
}
}
// A6a: a policy this install created is not there any more. Not an error -
// the uninstall wanted it gone and it is - but reported, because something
// else rewrote casbin_rule.
func TestUninstallReportsALedgerEntryWhosePolicyIsGone(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
if err := db.Exec("DELETE FROM casbin_rule WHERE v2 = 'POST'").Error; err != nil {
t.Fatalf("removing a policy: %v", err)
}
rep, err := uninstall(db, "order")
if err != nil {
t.Fatalf("a missing policy made the uninstall fail: %v", err)
}
if len(rep.Skipped) != 1 {
t.Fatalf("skipped = %v, want the one that had gone", rep.Skipped)
}
if rep.Skipped[0].V2 != "POST" {
t.Errorf("skipped = %+v", rep.Skipped[0])
}
if n := count(t, db, "sys_app_casbin_grant", "", nil); n != 0 {
t.Errorf("the ledger kept %d row(s); its job ends with the uninstall", n)
}
// It still committed: a skip is a reported branch, not a failure.
if n := count(t, db, "sys_menu", "app_code = ?", "order"); n != 0 {
t.Errorf("the transaction rolled back over a skip: sys_menu has %d row(s)", n)
}
}
// G5/A4: without this the reinstall finds every version applied, runs no
// migration, seeds nothing, and reports success.
func TestUninstallClearsThisAppsMigrationRecordsOnly(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
if err := db.Create(&commonmodels.Migration{
Version: "1786700001000", AppCode: "", ApplyTime: time.Now(),
}).Error; err != nil {
t.Fatalf("framework migration row: %v", err)
}
if _, err := uninstall(db, "order"); err != nil {
t.Fatalf("uninstall: %v", err)
}
if n := count(t, db, "sys_migration", "app_code = ?", "order"); n != 0 {
t.Errorf("sys_migration still has %d row(s) for order; a reinstall would seed nothing", n)
}
if n := count(t, db, "sys_migration", "app_code = ?", ""); n != 1 {
t.Errorf("the framework's own migration record was removed (%d left)", n)
}
}
// A11: sys_role_menu is found by menu id, not by a column on it. A column
// would have been blanked the first time somebody edited a role, because
// SysRole.Update deletes the role's rows and writes them back through GORM's
// many2many, which does not carry extra columns. This reproduces that edit.
func TestUninstallSurvivesARoleMenuRewrite(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
var roleID int
if err := db.Model(&adminmodels.SysRole{}).Where("role_key = ?", adminRoleKey).
Pluck("role_id", &roleID).Error; err != nil {
t.Fatalf("reading the admin role: %v", err)
}
var menuIDs []int
if err := db.Model(&adminmodels.SysMenu{}).Where("app_code = ?", "order").
Pluck("menu_id", &menuIDs).Error; err != nil {
t.Fatalf("reading menus: %v", err)
}
if len(menuIDs) == 0 {
t.Fatal("no menus were seeded")
}
// What SysRole.Update does: drop every row for the role, then write them
// back with nothing but the two keys.
if err := db.Exec("DELETE FROM sys_role_menu WHERE role_id = ?", roleID).Error; err != nil {
t.Fatalf("clearing role menus: %v", err)
}
for _, id := range menuIDs {
if err := db.Exec("INSERT INTO sys_role_menu (role_id, menu_id) VALUES (?, ?)", roleID, id).Error; err != nil {
t.Fatalf("rewriting role menus: %v", err)
}
}
rep, err := uninstall(db, "order")
if err != nil {
t.Fatalf("uninstall: %v", err)
}
if rep.RoleMenus != int64(len(menuIDs)) {
t.Errorf("removed %d role assignment(s), want %d", rep.RoleMenus, len(menuIDs))
}
if n := count(t, db, "sys_role_menu", "", nil); n != 0 {
t.Errorf("sys_role_menu still has %d row(s) after a role edit", n)
}
}
// `migrate` with no subcommand applies every registered migration, an
// application's included, so an application can have all of its rows and
// never have had a sys_app row. Refusing to clean that up would leave the
// only case where nothing else can.
func TestUninstallWorksWithoutASysAppRow(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
if err := db.Where("app_code = ?", "order").Delete(&adminmodels.SysApp{}).Error; err != nil {
t.Fatalf("removing the sys_app row: %v", err)
}
rep, err := uninstall(db, "order")
if err != nil {
t.Fatalf("uninstall: %v", err)
}
if rep.Found {
t.Error("the report claims a sys_app row that was not there")
}
if n := count(t, db, "sys_menu", "app_code = ?", "order"); n != 0 {
t.Errorf("sys_menu still has %d row(s)", n)
}
var out strings.Builder
reportUninstall(&out, rep)
if !strings.Contains(out.String(), "no sys_app row") {
t.Errorf("the report does not say the row was missing: %q", out.String())
}
}
// One transaction, and it really is one: nothing here runs DDL, so unlike an
// install there is nothing to commit it out from under itself.
func TestUninstallRollsBackAsAWhole(t *testing.T) {
db := newUninstallDB(t)
seedApp(t, db, "order")
menusBefore := count(t, db, "sys_menu", "app_code = ?", "order")
policiesBefore := count(t, db, "casbin_rule", "", nil)
// Step 8's table is gone, so the uninstall fails after it has already
// deleted menus, apis, bindings and policies.
if err := db.Migrator().DropTable(&commonmodels.Migration{}); err != nil {
t.Fatalf("dropping sys_migration: %v", err)
}
if _, err := uninstall(db, "order"); err == nil {
t.Fatal("the uninstall reported success with sys_migration missing")
}
if n := count(t, db, "sys_menu", "app_code = ?", "order"); n != menusBefore {
t.Errorf("sys_menu = %d, want %d: the failed uninstall did not roll back", n, menusBefore)
}
if n := count(t, db, "casbin_rule", "", nil); n != policiesBefore {
t.Errorf("casbin_rule = %d, want %d: the failed uninstall did not roll back", n, policiesBefore)
}
}
func TestUninstallRefusesTheFrameworkCode(t *testing.T) {
db := newUninstallDB(t)
if _, err := uninstall(db, migration.FrameworkAppCode); err == nil {
t.Fatal("the framework was uninstalled")
}
}
// findOrphanPolicies chunks its OR chain because a driver runs out of
// placeholders long before an application runs out of endpoints. The
// boundary is where an off-by-one hides: a chunk size that drops the last
// element of each batch, or one that never advances, both leave policies
// unreported and nothing says so.
func TestFindOrphanPoliciesCoversEveryPathAcrossChunks(t *testing.T) {
db := newUninstallDB(t)
// Deliberately not a multiple of the chunk size, so the last batch is
// short, and large enough to need three of them.
const n = 205
keys := make([]policyKey, 0, n)
for i := 0; i < n; i++ {
path := "/api/v1/thing" + strconv.Itoa(i)
keys = append(keys, policyKey{V1: path, V2: "GET"})
if err := db.Exec(
"INSERT INTO casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'ops', ?, 'GET', '', '', '')",
path,
).Error; err != nil {
t.Fatalf("seeding policy %d: %v", i, err)
}
}
// One policy that must not match: a path no key names.
if err := db.Exec(
"INSERT INTO casbin_rule (ptype, v0, v1, v2, v3, v4, v5) VALUES ('p', 'ops', '/api/v1/elsewhere', 'GET', '', '', '')",
).Error; err != nil {
t.Fatalf("seeding the control policy: %v", err)
}
found, err := findOrphanPolicies(db, keys)
if err != nil {
t.Fatalf("findOrphanPolicies: %v", err)
}
if len(found) != n {
t.Fatalf("found %d policies, want %d", len(found), n)
}
seen := make(map[string]bool, len(found))
for _, f := range found {
seen[f.V1] = true
if f.V1 == "/api/v1/elsewhere" {
t.Error("a path no key names was reported")
}
}
for _, k := range keys {
if !seen[k.V1] {
t.Errorf("%s was not reported", k.V1)
}
}
}
// An application may register apis with no menus at all - endpoints another
// service calls - so either of the id lists an uninstall reads can be empty.
// The guard in front of the join-table delete turns out not to be what makes
// this work: GORM renders IN with an empty slice as a condition that matches
// nothing, rather than the empty IN list that would be a syntax error in raw
// SQL, and removing the guard leaves this test green. It stays as an explicit
// statement of intent rather than a reliance on that rendering.
func TestUninstallWithApisButNoMenus(t *testing.T) {
db := newUninstallDB(t)
apis := []seed.ApiSpec{
{Code: "hook", Title: "Inbound hook", Path: "/api/v1/hook", Method: "POST", Handle: "hook.Receive"},
}
if err := db.Transaction(func(tx *gorm.DB) error {
return seed.SeedMenus(tx, "hooks", nil, apis)
}); err != nil {
t.Fatalf("seeding: %v", err)
}
rep, err := uninstall(db, "hooks")
if err != nil {
t.Fatalf("uninstall: %v", err)
}
if rep.Apis != 1 {
t.Errorf("removed %d api(s), want 1", rep.Apis)
}
if rep.Policies != 1 {
t.Errorf("removed %d policy(ies), want 1", rep.Policies)
}
if n := count(t, db, "casbin_rule", "", nil); n != 0 {
t.Errorf("casbin_rule has %d row(s)", n)
}
}
// The mirror case: menus and no apis at all.
func TestUninstallWithMenusButNoApis(t *testing.T) {
db := newUninstallDB(t)
menus := []seed.MenuSpec{
{Code: "dir", Kind: "M", Title: "Reports", Path: "/apps/reports", Component: "Layout", Sort: 10},
}
if err := db.Transaction(func(tx *gorm.DB) error {
return seed.SeedMenus(tx, "reports", menus, nil)
}); err != nil {
t.Fatalf("seeding: %v", err)
}
rep, err := uninstall(db, "reports")
if err != nil {
t.Fatalf("uninstall: %v", err)
}
if rep.Menus != 1 {
t.Errorf("removed %d menu(s), want 1", rep.Menus)
}
if n := count(t, db, "sys_menu", "app_code = ?", "reports"); n != 0 {
t.Errorf("sys_menu has %d row(s)", n)
}
if len(rep.Orphans) != 0 {
t.Errorf("an application with no apis reported orphans: %v", rep.Orphans)
}
}
+231
View File
@@ -0,0 +1,231 @@
// Package health answers whether this process should be sent traffic.
//
// The two questions an orchestrator asks are not the same one, and go-admin
// answers them at two endpoints:
//
// - /health is liveness: is the process there at all. It stays a bare 200,
// because the honest answer to "should I restart you" is almost always no.
// Restarting a process because its database is unreachable turns one
// outage into a crash loop that also loses the connection pool, the cache
// and every in-flight request.
// - /ready is readiness: should this instance receive requests now. It fails
// while the dependencies are unreachable, and - the part that only exists
// because of the life-cycle phases - it fails as soon as shutdown begins,
// before the server stops accepting.
//
// # What the draining answer is worth
//
// Order alone does not produce a window. Answering before the server stops
// accepting is the right order - the reverse reports the state after the
// connections are already cut - but with nothing between the two they are
// microseconds apart, and a poller on a multi-second interval never sees the
// 503.
//
// The delay between them is extend.shutdown.drain, which is zero unless it is
// configured. On the shipped defaults this is therefore still an answer that
// can be read rather than one anything acts on; a deployment that sets a drain
// window is the one that gets a window to act in.
//
// What acts on it depends on who does the removing. A load balancer that polls
// /ready takes this instance out when it reads the 503, and the window has to
// cover its check interval times its failure threshold, plus however long the
// removal takes to apply. On Kubernetes the endpoint is withdrawn when the Pod
// receives a deletionTimestamp, concurrently with SIGTERM and regardless of
// what the probe returns - there the window covers the delay in that removal
// reaching every node, and the 503 is what makes the state observable.
package health
import (
"context"
"crypto/rand"
"encoding/hex"
"errors"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
)
// draining is set when the process starts shutting down.
//
// It is kept here rather than read back from core: BeginShutdown sets a flag on
// the Application, but nothing exports it, and one host wanting to know is not
// yet a reason to widen that interface.
var draining atomic.Bool
// BeginDraining records that shutdown has started, so readiness fails from now
// on. It is called with BeginShutdown, before anything is taken apart.
func BeginDraining() { draining.Store(true) }
// Draining reports whether shutdown has begun.
func Draining() bool { return draining.Load() }
// Check is one dependency and what asking it produced.
type Check struct {
Name string `json:"name"`
OK bool `json:"ok"`
Err string `json:"error,omitempty"`
}
// extra are checks a host registers that this package cannot make itself.
//
// The direction is why this exists. Whether the schema matches what the binary
// expects is answered by the migration registry, which lives under cmd/ - and
// common/ has never imported cmd/. Rather than start, the host registers the
// check from where both are already in scope.
var (
extraMu sync.RWMutex
extra []namedCheck
)
type namedCheck struct {
name string
fn func(context.Context) error
}
// Register adds a check to what Ready asks.
//
// It panics on a duplicate name rather than replacing or appending: two checks
// under one name make the failing one impossible to identify from the response,
// and registering the same one twice is a wiring mistake worth hearing about at
// start-up rather than never.
func Register(name string, fn func(context.Context) error) {
if name == "" {
panic("health: a registered check needs a name")
}
if fn == nil {
panic("health: check " + name + " is nil")
}
extraMu.Lock()
defer extraMu.Unlock()
for _, c := range extra {
if c.name == name {
panic("health: check " + name + " is already registered")
}
}
extra = append(extra, namedCheck{name: name, fn: fn})
}
// registered returns the checks a host has added, copied so that Ready is not
// iterating the slice while another goroutine appends to it.
func registered() []namedCheck {
extraMu.RLock()
defer extraMu.RUnlock()
out := make([]namedCheck, len(extra))
copy(out, extra)
return out
}
// Ready asks every dependency this process cannot serve a request without.
//
// The queue is deliberately absent. Nothing on AdapterQueue answers "are you
// reachable" without publishing something, the memory backend cannot fail, and
// a queue that is down degrades logging rather than stopping requests - which
// is a reason to alert, not a reason to leave the load balancer pool.
func Ready(ctx context.Context) []Check {
checks := []Check{
safely("database", func() error { return pingDB(ctx) }),
safely("cache", probeCache),
}
for _, c := range registered() {
checks = append(checks, safely(c.name, func() error { return c.fn(ctx) }))
}
return checks
}
// safely turns a panic into a failed check.
//
// Not defensive habit: the accessors hand back wrappers, not the resources.
// sdk.Runtime.GetCacheAdapter builds a runtime.Cache around whatever is
// configured and returns it even when nothing is - so the value is not nil, the
// cache inside it is, and the first call dereferences it. A nil check cannot
// see that, and the same is true of GetQueueAdapter.
//
// Whatever the reason, a probe is the last thing that should be able to take
// the process down: the caller is asking whether this instance is well, and
// killing it to answer is the wrong reply.
func safely(name string, fn func() error) (c Check) {
c = Check{Name: name}
defer func() {
if r := recover(); r != nil {
c.OK, c.Err = false, fmt.Sprintf("the check panicked: %v", r)
}
}()
if err := fn(); err != nil {
c.Err = err.Error()
return c
}
c.OK = true
return c
}
// Healthy reports whether every check passed.
func Healthy(checks []Check) bool {
for _, c := range checks {
if !c.OK {
return false
}
}
return true
}
func pingDB(ctx context.Context) error {
db := sdk.Runtime.GetDb()
if db == nil {
return errors.New("no database configured")
}
sqlDB, err := db.DB()
if err != nil {
return err
}
return sqlDB.PingContext(ctx)
}
// cacheProbePrefix names the probe's keys. The key itself is per probe, not
// fixed: two /ready requests arriving together - or two instances sharing one
// redis, which is the normal deployment - would otherwise overwrite each
// other's value between the write and the read and each conclude the cache was
// broken. A readiness probe that reports false negatives under load takes
// healthy instances out of the pool, which is worse than not probing.
const cacheProbePrefix = "go-admin:health:"
// cacheProbeTTL is short because these keys are write-once and never read
// again by anyone else; it only has to outlive the read that follows.
const cacheProbeTTL = 30
func probeCache() error {
adapter := sdk.Runtime.GetCacheAdapter()
if adapter == nil {
return errors.New("no cache configured")
}
suffix := make([]byte, 8)
if _, err := rand.Read(suffix); err != nil {
return fmt.Errorf("could not build a probe key: %w", err)
}
key := cacheProbePrefix + hex.EncodeToString(suffix)
// Written and read back rather than only read: a cache that answers "miss"
// for every key - a client pointed at the wrong server - is
// indistinguishable from a healthy one on a read alone.
want := time.Now().Format(time.RFC3339Nano)
if err := adapter.Set(key, want, cacheProbeTTL); err != nil {
return err
}
// Best effort, and its error is deliberately dropped: the verdict is
// already decided by the read below, and a cache that cannot delete a key
// it just wrote is not a reason to refuse traffic. The TTL is the real
// cleanup.
defer func() { _ = adapter.Del(key) }()
got, err := adapter.Get(key)
if err != nil {
return err
}
if got != want {
return errors.New("the cache returned a different value than was written")
}
return nil
}
+244
View File
@@ -0,0 +1,244 @@
package health
import (
"context"
"errors"
"sync"
"testing"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
)
func freshRuntime(t *testing.T) {
t.Helper()
previous := sdk.Runtime
t.Cleanup(func() { sdk.Runtime = previous })
sdk.Runtime = runtime.NewConfig()
}
// fakeCache answers whatever the test needs it to.
type fakeCache struct {
mu sync.Mutex
setErr error
getErr error
getBack string // returned instead of what was written, when non-empty
stored map[string]string
// oneSlot makes the cache keep a single value however many keys are
// written, which is what a shared probe key turns any cache into.
oneSlot bool
slot string
// setBarrier, when set, holds every writer until all of them have written.
// Without it the probes are short enough that the scheduler usually runs
// them one after another, and a shared key survives by luck rather than by
// design - which would leave the test below asserting nothing.
setBarrier *barrier
}
// barrier releases every waiter once n of them have arrived.
type barrier struct {
n int
mu sync.Mutex
got int
ch chan struct{}
}
func newBarrier(n int) *barrier { return &barrier{n: n, ch: make(chan struct{})} }
func (b *barrier) wait() {
b.mu.Lock()
b.got++
if b.got == b.n {
close(b.ch)
}
b.mu.Unlock()
<-b.ch
}
func (c *fakeCache) String() string { return "fake" }
func (c *fakeCache) Set(key string, val interface{}, _ int) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.setErr != nil {
return c.setErr
}
v, _ := val.(string)
if c.oneSlot {
c.slot = v
return nil
}
if c.stored == nil {
c.stored = map[string]string{}
}
c.stored[key] = v
c.mu.Unlock()
if c.setBarrier != nil {
// Outside the lock on purpose: waiting while holding it would deadlock
// every other writer before the barrier could fill.
c.setBarrier.wait()
}
c.mu.Lock()
return nil
}
func (c *fakeCache) Get(key string) (string, error) {
c.mu.Lock()
defer c.mu.Unlock()
if c.getErr != nil {
return "", c.getErr
}
if c.getBack != "" {
return c.getBack, nil
}
if c.oneSlot {
return c.slot, nil
}
return c.stored[key], nil
}
func (c *fakeCache) Del(key string) error {
c.mu.Lock()
defer c.mu.Unlock()
delete(c.stored, key)
return nil
}
func (c *fakeCache) HashGet(_, _ string) (string, error) { return "", nil }
func (c *fakeCache) HashDel(_, _ string) error { return nil }
func (c *fakeCache) Increase(string) error { return nil }
func (c *fakeCache) Decrease(string) error { return nil }
func (c *fakeCache) Expire(string, time.Duration) error { return nil }
var _ corestorage.AdapterCache = (*fakeCache)(nil)
func named(checks []Check, name string) Check {
for _, c := range checks {
if c.Name == name {
return c
}
}
return Check{Name: name, Err: "check not reported at all"}
}
// A cache that accepts writes and answers every read with a different value is
// the failure this probe exists for - a client pointed at the wrong server, or
// one that silently drops everything. A read alone cannot tell that apart from
// a healthy cache with a cold key, which is why the probe writes first.
func TestCacheProbeFailsWhenTheValueDoesNotComeBack(t *testing.T) {
freshRuntime(t)
sdk.Runtime.SetCacheAdapter(&fakeCache{getBack: "something else"})
got := named(Ready(context.Background()), "cache")
if got.OK {
t.Error("the cache check passed although the value written was not the value read back")
}
if got.Err == "" {
t.Error("the failing check reported no reason")
}
}
func TestCacheProbePassesWhenTheValueComesBack(t *testing.T) {
freshRuntime(t)
sdk.Runtime.SetCacheAdapter(&fakeCache{})
if got := named(Ready(context.Background()), "cache"); !got.OK {
t.Errorf("the cache check failed for a cache that works: %s", got.Err)
}
}
func TestCacheProbeReportsAWriteFailure(t *testing.T) {
freshRuntime(t)
sdk.Runtime.SetCacheAdapter(&fakeCache{setErr: errors.New("connection refused")})
got := named(Ready(context.Background()), "cache")
if got.OK {
t.Error("the cache check passed although the write failed")
}
}
// Nothing configured is the case that used to take the process down rather
// than answer. GetCacheAdapter builds a wrapper around whatever is configured
// and returns it even when nothing is, so the value is not nil, the cache
// inside it is, and Set dereferences it - a probe that panics is the worst
// possible answer to "are you well".
//
// Every check has to be reported, passing or not. A probe that omits what it
// could not reach reads as a shorter list of healthy things.
func TestEveryDependencyIsReportedEvenWithNothingConfigured(t *testing.T) {
freshRuntime(t)
checks := Ready(context.Background())
for _, name := range []string{"database", "cache"} {
c := named(checks, name)
if c.Err == "check not reported at all" {
t.Errorf("%s was not reported", name)
}
if c.OK {
t.Errorf("%s passed with nothing configured", name)
}
}
if Healthy(checks) {
t.Error("Healthy said yes for a process with no database and no cache")
}
}
// BeginDraining sets the flag and Draining reports it, before anything else is
// taken apart. That is the whole of what can be checked from inside the
// process: whether anyone outside gets to read it depends on
// extend.shutdown.drain, which is zero unless it is configured, and on who is
// routing traffic here - the package comment has both. The subprocess tests in
// cmd/api are where a reader on the other end of a socket sees the 503.
func TestDrainingIsObservableOnceItBegins(t *testing.T) {
previous := draining.Load()
t.Cleanup(func() { draining.Store(previous) })
draining.Store(false)
if Draining() {
t.Fatal("Draining reported true before shutdown began")
}
BeginDraining()
if !Draining() {
t.Error("Draining still reported false after BeginDraining")
}
}
// Two probes at once must both pass. With one fixed key they overwrite each
// other's value between the write and the read, and a readiness probe that
// reports false negatives under load takes healthy instances out of the pool -
// which is worse than not probing at all.
func TestConcurrentProbesDoNotOverwriteEachOther(t *testing.T) {
freshRuntime(t)
const probes = 16
sdk.Runtime.SetCacheAdapter(&fakeCache{setBarrier: newBarrier(probes)})
var wg sync.WaitGroup
failures := make(chan string, probes)
for i := 0; i < probes; i++ {
wg.Add(1)
go func() {
defer wg.Done()
if c := named(Ready(context.Background()), "cache"); !c.OK {
failures <- c.Err
}
}()
}
wg.Wait()
close(failures)
var n int
var first string
for err := range failures {
if n == 0 {
first = err
}
n++
}
if n > 0 {
t.Errorf("%d of %d concurrent probes called a healthy cache broken; first: %s", n, probes, first)
}
}
+125
View File
@@ -0,0 +1,125 @@
package health
import (
"context"
"errors"
"testing"
)
// isolate empties the registered checks and puts them back, so one test in
// this package cannot decide what the next one sees.
func isolate(t *testing.T) {
t.Helper()
extraMu.Lock()
previous := extra
extra = nil
extraMu.Unlock()
t.Cleanup(func() {
extraMu.Lock()
extra = previous
extraMu.Unlock()
})
}
func findCheck(checks []Check, name string) (Check, bool) {
for _, c := range checks {
if c.Name == name {
return c, true
}
}
return Check{}, false
}
// A registered check has to reach Ready's answer, or the host has wired
// something that never gets asked.
func TestARegisteredCheckIsAsked(t *testing.T) {
isolate(t)
Register("schema", func(context.Context) error { return errors.New("two behind") })
checks := Ready(context.Background())
c, ok := findCheck(checks, "schema")
if !ok {
t.Fatal("Ready did not ask the registered check")
}
if c.OK {
t.Error("the check returned an error and was still reported OK")
}
if c.Err != "two behind" {
t.Errorf("Err = %q, want the check's own message", c.Err)
}
if Healthy(checks) {
t.Error("Healthy said yes while a registered check was failing")
}
}
// The context Ready is given has to reach the check: it carries the probe's
// deadline, and a check that ignores it can hold the handler past it.
func TestTheRegisteredCheckIsGivenReadysContext(t *testing.T) {
isolate(t)
type key struct{}
Register("ctx", func(ctx context.Context) error {
if ctx.Value(key{}) != "carried" {
return errors.New("the check was handed a different context")
}
return nil
})
checks := Ready(context.WithValue(context.Background(), key{}, "carried"))
c, ok := findCheck(checks, "ctx")
if !ok {
t.Fatal("the registered check was not asked")
}
if !c.OK {
t.Errorf("check failed: %s", c.Err)
}
}
// A check that panics must not take the process down through the probe, the
// same guarantee the built-in checks have.
func TestARegisteredCheckThatPanicsFailsRatherThanCrashes(t *testing.T) {
isolate(t)
Register("boom", func(context.Context) error { panic("registry unreachable") })
checks := Ready(context.Background())
c, ok := findCheck(checks, "boom")
if !ok {
t.Fatal("the registered check was not asked")
}
if c.OK {
t.Error("a panicking check was reported OK")
}
}
func TestRegisteringTheSameNameTwicePanics(t *testing.T) {
isolate(t)
Register("dup", func(context.Context) error { return nil })
defer func() {
if recover() == nil {
t.Error("registering a duplicate name did not panic; two checks under " +
"one name make the failing one impossible to identify")
}
}()
Register("dup", func(context.Context) error { return nil })
}
func TestRegisterRefusesAnEmptyNameOrNilCheck(t *testing.T) {
isolate(t)
for _, tc := range []struct {
name string
fn func(context.Context) error
why string
}{
{"", func(context.Context) error { return nil }, "empty name"},
{"nilfn", nil, "nil function"},
} {
func() {
defer func() {
if recover() == nil {
t.Errorf("%s did not panic", tc.why)
}
}()
Register(tc.name, tc.fn)
}()
}
}
+91 -17
View File
@@ -1,29 +1,103 @@
package middleware
import (
"net/http"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"net/http"
)
// defaultDemoMsg is what a refused request is told when nothing is configured.
//
// It is the string this middleware used to carry hard-coded, kept verbatim so
// that a deployment which never set application.demomsg is answered exactly as
// it was before.
const defaultDemoMsg = "谢谢您的参与,但为了大家更好的体验,所以本次提交就算了吧!\U0001F600\U0001F600\U0001F600"
// demoWriteRoutes are routes that change something despite being registered as
// GET, so the method alone does not say whether they are safe to serve.
//
// All three belong to the code generator: two write Go source files onto the
// server's filesystem and the third inserts menus, APIs and casbin rules into
// the database. They are registered under a group whose own name says it does
// no role check, and a demo deployment lets anybody log in - so on a demo host
// they were reachable by any visitor, and the menus one had in fact been used.
//
// Spelled as gin route patterns, which is what Context.FullPath returns, so a
// path parameter matches whatever value it is given.
//
// This list cannot be checked from here: common/ may not import app/, so this
// package cannot see which routes exist. What keeps it honest is a test beside
// the routes themselves - see app/other/router - which registers them and
// fails if any entry here has stopped being a real route.
//
// It also does not close the general hole. Nothing stops the next GET handler
// that writes something from being added without an entry here, and no static
// check can tell a handler that writes from one that reads. Demo mode refuses
// the routes it has been told about; that is the whole of the guarantee.
var demoWriteRoutes = map[string]bool{
"/api/v1/gen/toproject/:tableId": true,
"/api/v1/gen/apitofile/:tableId": true,
"/api/v1/gen/todb/:tableId": true,
}
// DemoWriteRoutes returns the routes demo mode refuses despite their method.
//
// Exported only so the test that lives beside the route registrations can
// check every one of them still exists; nothing else should need it.
func DemoWriteRoutes() []string {
out := make([]string, 0, len(demoWriteRoutes))
for route := range demoWriteRoutes {
out = append(out, route)
}
return out
}
// demoAllows reports whether demo mode lets a request through.
//
// route is the matched gin route pattern and uri the raw request target; the
// two are different things and both are needed. The route is what identifies a
// handler regardless of the values in its path parameters, and it is empty for
// a request that matched nothing - which is why the login and logout checks
// still read the raw target, as they always did.
func demoAllows(method, route, uri string) bool {
if demoWriteRoutes[route] {
return false
}
return method == http.MethodGet ||
method == http.MethodOptions ||
uri == "/api/v1/login" ||
uri == "/api/v1/logout"
}
// demoMessage is the answer a refused request gets.
//
// application.demomsg has been in the configuration all along and nothing read
// it: the message was hard-coded here, and the demo host's configured string
// happened to be identical, so the setting looked like it worked. An empty
// value falls back rather than answering with nothing.
func demoMessage() string {
if msg := config.ApplicationConfig.DemoMsg; msg != "" {
return msg
}
return defaultDemoMsg
}
// DemoEvn refuses anything that would change state while mode is demo.
func DemoEvn() gin.HandlerFunc {
return func(c *gin.Context) {
method := c.Request.Method
if config.ApplicationConfig.Mode == "demo" {
if method == "GET" ||
method == "OPTIONS" ||
c.Request.RequestURI == "/api/v1/login" ||
c.Request.RequestURI == "/api/v1/logout" {
c.Next()
} else {
c.JSON(http.StatusOK, gin.H{
"code": 500,
"msg": "谢谢您的参与,但为了大家更好的体验,所以本次提交就算了吧!\U0001F600\U0001F600\U0001F600",
})
c.Abort()
return
}
if config.ApplicationConfig.Mode != "demo" {
c.Next()
return
}
c.Next()
if demoAllows(c.Request.Method, c.FullPath(), c.Request.RequestURI) {
c.Next()
return
}
c.JSON(http.StatusOK, gin.H{
"code": 500,
"msg": demoMessage(),
})
c.Abort()
}
}
+144
View File
@@ -0,0 +1,144 @@
package middleware
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
// demoMode puts the process in demo mode for one test and puts it back.
func demoMode(t *testing.T, mode, msg string) {
t.Helper()
previousMode, previousMsg := config.ApplicationConfig.Mode, config.ApplicationConfig.DemoMsg
t.Cleanup(func() {
config.ApplicationConfig.Mode = previousMode
config.ApplicationConfig.DemoMsg = previousMsg
})
config.ApplicationConfig.Mode, config.ApplicationConfig.DemoMsg = mode, msg
}
// The method is not enough on its own. Three of the generator's routes are
// registered as GET and write anyway - two of them onto the server's
// filesystem, one into the database - so a guard that reads only the method
// serves them to anybody who can log in, which on a demo host is everybody.
func TestDemoRefusesTheWritesThatAreServedOverGET(t *testing.T) {
const login = "/api/v1/login"
for _, tc := range []struct {
name string
method string
route, uri string
wantThrough bool
}{
{"a plain read", http.MethodGet, "/api/v1/dept", "/api/v1/dept", true},
{"a write, by method", http.MethodPost, "/api/v1/dept", "/api/v1/dept", false},
{"login is how a visitor gets in", http.MethodPost, login, login, true},
{"logout", http.MethodPost, "/api/v1/logout", "/api/v1/logout", true},
{"preflight", http.MethodOptions, "/api/v1/dept", "/api/v1/dept", true},
// A request that matched no route has an empty pattern, and the guard
// still has to refuse it by method - this is what a POST to a path
// that does not exist looks like from in here.
{"a write to nothing at all", http.MethodPost, "", "/api/v1/__probe__", false},
// The three this change is about.
{"generator writes the database", http.MethodGet,
"/api/v1/gen/todb/:tableId", "/api/v1/gen/todb/3", false},
{"generator writes source files", http.MethodGet,
"/api/v1/gen/toproject/:tableId", "/api/v1/gen/toproject/3", false},
{"generator writes an api file", http.MethodGet,
"/api/v1/gen/apitofile/:tableId", "/api/v1/gen/apitofile/3", false},
// The read-only half of the generator has to keep working, or the demo
// host cannot demonstrate the feature at all. Refusing too much is as
// much of a defect as refusing too little.
{"generator preview stays available", http.MethodGet,
"/api/v1/gen/preview/:tableId", "/api/v1/gen/preview/3", true},
{"generator table tree stays available", http.MethodGet,
"/api/v1/gen/tabletree", "/api/v1/gen/tabletree", true},
{"table list stays available", http.MethodGet,
"/api/v1/db/tables/page", "/api/v1/db/tables/page", true},
{"column list stays available", http.MethodGet,
"/api/v1/db/columns/page", "/api/v1/db/columns/page", true},
} {
t.Run(tc.name, func(t *testing.T) {
if got := demoAllows(tc.method, tc.route, tc.uri); got != tc.wantThrough {
t.Errorf("demoAllows(%s %s) = %v, want %v", tc.method, tc.route, got, tc.wantThrough)
}
})
}
}
// Everything above is about demo mode only. A deployment that is not a demo
// runs the generator for real, and a guard that reached it there would have
// taken the feature away from every production install.
func TestOutsideDemoModeNothingIsRefused(t *testing.T) {
demoMode(t, "prod", "")
gin.SetMode(gin.TestMode)
for _, route := range append(DemoWriteRoutes(), "/api/v1/dept") {
t.Run(route, func(t *testing.T) {
r := gin.New()
r.Use(DemoEvn())
r.GET(route, func(c *gin.Context) { c.String(http.StatusOK, "served") })
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, requestFor(route), nil))
if w.Body.String() != "served" {
t.Errorf("answered %q; outside demo mode the handler must run", w.Body.String())
}
})
}
}
// The refusal has to come back as the demo message rather than a 403 or a 404:
// the front end shows it to the visitor, and the point of a demo host is that
// being turned away is explained.
func TestDemoRefusalCarriesTheConfiguredMessage(t *testing.T) {
gin.SetMode(gin.TestMode)
const route = "/api/v1/gen/todb/:tableId"
for _, tc := range []struct {
name, configured, want string
}{
{"configured", "come back tomorrow", "come back tomorrow"},
// A deployment that never set application.demomsg keeps the answer it
// already had; an empty setting must not become an empty message.
{"not configured", "", defaultDemoMsg},
} {
t.Run(tc.name, func(t *testing.T) {
demoMode(t, "demo", tc.configured)
r := gin.New()
r.Use(DemoEvn())
r.GET(route, func(c *gin.Context) { c.String(http.StatusOK, "served") })
w := httptest.NewRecorder()
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, requestFor(route), nil))
if w.Code != http.StatusOK {
t.Errorf("answered %d, want 200 so the front end reads the body", w.Code)
}
if body := w.Body.String(); !strings.Contains(body, tc.want) {
t.Errorf("body %q does not carry %q", body, tc.want)
}
if strings.Contains(w.Body.String(), "served") {
t.Error("the handler ran; the request was supposed to be refused")
}
})
}
}
// requestFor turns a route pattern into a request target by giving every path
// parameter a value.
func requestFor(route string) string {
segments := strings.Split(route, "/")
for i, segment := range segments {
if strings.HasPrefix(segment, ":") {
segments[i] = "1"
}
}
return strings.Join(segments, "/")
}
+1
View File
@@ -34,6 +34,7 @@ var CasbinExclude = []UrlInfo{
{Url: "/api/v1/user/pwd", Method: "PUT"},
{Url: "/api/v1/metrics", Method: "GET"},
{Url: "/api/v1/health", Method: "GET"},
{Url: "/api/v1/ready", Method: "GET"},
{Url: "/", Method: "GET"},
{Url: "/api/v1/server-monitor", Method: "GET"},
{Url: "/api/v1/public/uploadFile", Method: "POST"},
+182 -5
View File
@@ -8,11 +8,14 @@
package storage
import (
"context"
"log"
"sync"
"github.com/go-admin-team/go-admin-core/v2/captcha"
corelog "github.com/go-admin-team/go-admin-core/v2/logger"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/captcha"
)
// Setup 配置storage组件
@@ -20,6 +23,7 @@ func Setup() {
setupCache()
setupCaptcha()
setupQueue()
registerQueueDrain()
}
func setupCache() {
@@ -34,17 +38,190 @@ func setupCaptcha() {
captcha.SetStore(captcha.NewCacheStore(sdk.Runtime.GetCacheAdapter(), 600))
}
var (
queueMu sync.Mutex
// installed is the adapter setupQueue built, kept so the next reload can
// shut it down, and counted so a consumer can tell one from the next.
installed interface{ Shutdown() }
installedGen uint64
// drainRegistered records that the BeforeExit callback is on the runtime,
// so that a reload does not add another one.
drainRegistered bool
)
// setShutdown is sdk.Runtime.SetShutdown, indirected so that registering can
// be observed.
//
// It has to be: the runtime does not report how many callbacks a phase holds,
// and shutdownQueue takes the adapter on its first run, so every registration
// after the first returns immediately and changes nothing anybody can see. A
// reload adding one callback per round would therefore be invisible from the
// outside - which is exactly how it would survive.
var setShutdown = func(f func(context.Context)) { sdk.Runtime.SetShutdown(f) }
// registerQueueDrain puts shutdownQueue on the BeforeExit phase, once.
//
// Setup is one of the callbacks bootstrap.SetupConfig re-runs on every
// configuration change, so registering from it without a guard would leave one
// callback per reload - each shutting down the same adapter, each reported
// separately when the budget runs out.
//
// A flag under the existing mutex rather than a sync.Once: the tests in this
// package already save and restore installed and installedGen to keep one test
// from deciding what the next one sees, and a sync.Once cannot be put back.
func registerQueueDrain() {
queueMu.Lock()
first := !drainRegistered
drainRegistered = true
queueMu.Unlock()
if first {
setShutdown(shutdownQueue)
}
}
// drainedInTime shuts q down and reports whether it finished before ctx expired.
func drainedInTime(ctx context.Context, q interface{ Shutdown() }) bool {
done := make(chan struct{})
go func() {
defer close(done)
q.Shutdown()
}()
return finishedBeforeDeadline(ctx, done)
}
// finishedBeforeDeadline waits for done or for ctx, and resolves a tie in
// favour of done.
//
// The tie is the reason this is a function of its own rather than one select
// inline. Both channels can be ready when the select runs, select picks at
// random among ready cases, and so a single look reports an overrun for a
// drain that completed - about half the times it lands there, which is exactly
// often enough to be dismissed as noise. core's own RunShutdown re-checks for
// this reason.
//
// Taking channels rather than a queue is what makes it testable: a closed done
// and an expired ctx can be handed in together, which is the state a race
// would otherwise have to be caught in.
func finishedBeforeDeadline(ctx context.Context, done <-chan struct{}) bool {
select {
case <-done:
return true
case <-ctx.Done():
}
select {
case <-done:
return true
default:
return false
}
}
// shutdownQueue drains the queue this package installed, on the way out.
//
// Nothing used to. core's Memory.Shutdown closes the queue and waits for every
// consumer to finish what it is holding, and the legacy adapter cancels its
// context and closes the underlying queue - but neither ran at exit, so the
// process left with the login log, the operation log and the API sync still
// buffered, and left reporting success.
//
// The adapter is read here rather than captured at registration because a
// reload replaces it. Registration happens once per process; this runs against
// whatever is current when the signal arrives.
//
// Only an adapter this package installed. sdk.Runtime.GetQueueAdapter never
// returns nil - with no queue section configured the runtime wraps its own
// fallback queue - so going through that accessor would shut down a queue this
// package neither built nor started.
//
// The adapter is taken, not read: after this the package owns nothing, so a
// reload arriving mid-shutdown builds a new one instead of being handed a
// closed one to shut down again. Both implementations tolerate a second
// Shutdown, so this is about who owns it rather than about a crash.
func shutdownQueue(ctx context.Context) {
queueMu.Lock()
q := installed
installed = nil
queueMu.Unlock()
if q == nil {
return
}
if drainedInTime(ctx, q) {
corelog.Info("queue: drained")
return
}
// The wait is what the budget bounds, not the work: Shutdown takes no
// context and is still running in that goroutine. Saying so here names what
// is being lost, which the generic overrun message cannot.
corelog.Warnf("queue: the shutdown budget ran out while the queue was still draining - " +
"whatever it had not delivered goes with the process. Raise extend.shutdown.cleanup " +
"if this recurs.")
}
// QueueGeneration reports how many times this package has installed a queue
// adapter. It changes every time setupQueue builds a new one, which is on
// every configuration reload, and stays 0 for as long as the configuration has
// no queue section at all - in which case nothing is installed and callers are
// working with the runtime's own fallback queue.
//
// It exists because there is no way to ask for the adapter's identity from the
// outside. sdk.Runtime.GetQueueAdapter and GetQueuePrefix build a fresh
// runtime.Queue wrapper on every call, so comparing what two calls return
// compares two wrappers and never matches, however many times the underlying
// adapter has been replaced. This package creates the adapter, so this is the
// only place that knows. A counter rather than the adapter itself keeps the
// comparison on a uint64: an adapter type that is not comparable would panic
// an `==` between two interface values.
func QueueGeneration() uint64 {
queueMu.Lock()
defer queueMu.Unlock()
return installedGen
}
func setupQueue() {
if config.QueueConfig.Empty() {
return
}
if q := sdk.Runtime.GetQueueAdapter(); q != nil {
q.Shutdown()
}
queueMu.Lock()
defer queueMu.Unlock()
queueAdapter, err := config.QueueConfig.Setup()
if err != nil {
log.Fatalf("queue setup error, %s\n", err.Error())
}
previous := installed
sdk.Runtime.SetQueueAdapter(queueAdapter)
go queueAdapter.Run()
installed = queueAdapter
installedGen++
// The previous adapter goes down after the new one is installed, not
// before. Shutdown waits for its consumers to deliver what it still holds,
// and for that whole wait the runtime would otherwise be handing producers
// a queue that has stopped accepting: every Append in the window comes back
// ErrQueueClosed, and both call sites in common/middleware log it. Swapping
// first leaves no such window - a producer gets the new queue or the old
// one, and both work.
//
// Only an adapter this package installed. GetQueueAdapter never returns
// nil - with nothing configured the runtime falls back to its own memory
// queue and wraps that - so the `if q := GetQueueAdapter(); q != nil` this
// replaces was always true, and shut down the fallback queue on the very
// first start, before anything had used it.
if previous != nil {
previous.Shutdown()
}
// Deliberately not started here. Run has to come after the consumers have
// registered: the contract implementations refuse a registration once the
// queue is running (storage.ErrQueueAlreadyStarted), and the legacy
// adapter this repository still goes through swallows that error rather
// than reporting it - its own comment says the interface gives it no way
// to tell the caller. Starting here and registering afterwards is
// therefore a race that loses consumers in silence. Whoever registers is
// the one that starts it.
}
+333
View File
@@ -0,0 +1,333 @@
package storage
import (
"context"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
)
// countingQueue stands in for an installed adapter. Only Shutdown is exercised
// - the drain callback never publishes or consumes - so the rest of
// AdapterQueue is deliberately absent: `installed` is typed on Shutdown alone,
// and widening the fake would only invite it to be used for something else.
type countingQueue struct {
calls atomic.Int32
block chan struct{}
// started is closed on the way into Shutdown, so a test can wait for the
// call rather than assume the goroutine that makes it was scheduled. The
// caller returns on its own deadline while Shutdown is still running, so
// reading calls straight after that return is a race with the increment.
startOnce sync.Once
started chan struct{}
}
func newCountingQueue() *countingQueue {
return &countingQueue{started: make(chan struct{})}
}
func (q *countingQueue) Shutdown() {
q.startOnce.Do(func() { close(q.started) })
q.calls.Add(1)
if q.block != nil {
<-q.block
}
}
// waitStarted blocks until Shutdown has been entered, or fails the test.
func (q *countingQueue) waitStarted(t *testing.T) {
t.Helper()
select {
case <-q.started:
case <-time.After(5 * time.Second):
t.Fatal("Shutdown was never called")
}
}
// isolate gives the test its own runtime and its own view of what this package
// has installed, and puts the process-wide state back afterwards.
//
// Same isolation as TestSetupBumpsTheQueueGenerationOnEveryReload, plus
// drainRegistered: it is what stops a reload registering a second callback, so
// leaving it set would make every later test in this binary see a package that
// has already registered.
func isolate(t *testing.T) {
t.Helper()
prevQ, prevC := config.QueueConfig, config.CacheConfig
prevRuntime := sdk.Runtime
queueMu.Lock()
prevInstalled, prevGen, prevRegistered := installed, installedGen, drainRegistered
queueMu.Unlock()
t.Cleanup(func() {
config.QueueConfig, config.CacheConfig = prevQ, prevC
sdk.Runtime = prevRuntime
queueMu.Lock()
installed, installedGen, drainRegistered = prevInstalled, prevGen, prevRegistered
queueMu.Unlock()
})
sdk.Runtime = runtime.NewConfig()
queueMu.Lock()
installed, drainRegistered = nil, false
queueMu.Unlock()
}
// setInstalled puts a fake where setupQueue would have left the real adapter.
//
// Legitimate because the callback reads `installed` when it runs rather than
// capturing it at registration - that is the property that lets a reload
// replace the adapter and still have the right one drained.
func setInstalled(q interface{ Shutdown() }) {
queueMu.Lock()
installed = q
queueMu.Unlock()
}
func currentInstalled() interface{ Shutdown() } {
queueMu.Lock()
defer queueMu.Unlock()
return installed
}
// Issue #911: nothing shut the queue down at exit, so whatever was buffered
// went with the process.
//
// This is the half that matters most - a callback is on BeforeExit and it
// reaches the adapter this package installed. It says nothing about how many
// times the callback was registered; see the test below for that.
func TestSetupPutsTheQueueDrainOnBeforeExit(t *testing.T) {
isolate(t)
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
Setup()
Setup()
Setup()
q := newCountingQueue()
setInstalled(q)
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
t.Fatalf("RunShutdown: %v", err)
}
if got := q.calls.Load(); got != 1 {
t.Errorf("Shutdown called %d times, want 1 - 0 means nothing registered the drain", got)
}
}
// Setup is re-run on every configuration change, so registering from it has to
// be guarded: a callback per reload would leave the shutdown phase holding a
// row of identical entries, each timed and each eligible to be named as the one
// that overran the budget.
//
// Counted at the seam rather than through the effect. The test above cannot see
// this - shutdownQueue takes the adapter on its first run, so the second and
// third callbacks find nothing and return, and three registrations produce
// exactly the same observable result as one. That is a good property of the
// callback and a blind spot for any test that goes through it.
func TestSetupRegistersTheDrainOncePerProcessHoweverManyReloads(t *testing.T) {
isolate(t)
previous := setShutdown
t.Cleanup(func() { setShutdown = previous })
registrations := 0
setShutdown = func(func(context.Context)) { registrations++ }
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
Setup()
Setup()
Setup()
if registrations != 1 {
t.Errorf("three reloads registered the drain %d times, want 1", registrations)
}
}
// The drain reaches the adapter that is current when the signal arrives, not
// one captured while wiring up. A reload replaces the adapter, and draining the
// one that was installed at start-up would drain something nobody has published
// to since.
func TestTheDrainRunsAgainstTheAdapterInstalledLast(t *testing.T) {
isolate(t)
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
Setup()
first, second := newCountingQueue(), newCountingQueue()
setInstalled(first)
setInstalled(second)
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
t.Fatalf("RunShutdown: %v", err)
}
if first.calls.Load() != 0 {
t.Error("the adapter that was replaced was shut down; the callback captured it instead of " +
"reading it when it ran")
}
if second.calls.Load() != 1 {
t.Errorf("the current adapter was shut down %d times, want 1", second.calls.Load())
}
}
// Nothing installed is the shipped default: settings.yml has no queue section,
// so setupQueue returns early and the runtime's own fallback queue is what
// callers get. Shutting that down would close a queue this package neither
// built nor started.
func TestTheDrainDoesNothingWhenThisPackageInstalledNothing(t *testing.T) {
isolate(t)
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{}
Setup()
if currentInstalled() != nil {
t.Fatal("an empty queue configuration installed an adapter, so this test asserts nothing")
}
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
t.Fatalf("RunShutdown: %v", err)
}
}
// The budget bounds the wait, not the work. A consumer that never finishes must
// not hold the process past its grace period - SIGKILL would arrive mid-write
// instead of at a point of the process's choosing.
func TestTheDrainStopsWaitingWhenTheBudgetIsGone(t *testing.T) {
isolate(t)
blocked := newCountingQueue()
blocked.block = make(chan struct{})
t.Cleanup(func() { close(blocked.block) })
setInstalled(blocked)
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
returned := make(chan struct{})
go func() {
defer close(returned)
shutdownQueue(ctx)
}()
select {
case <-returned:
case <-time.After(5 * time.Second):
t.Fatal("shutdownQueue did not return after its context expired - it waits on a Shutdown " +
"that takes no context, so the wait has to be bounded here")
}
// Waited for rather than read straight after the return: Shutdown runs on a
// goroutine that the caller does not join, so the increment is not ordered
// against the caller giving up on its deadline.
blocked.waitStarted(t)
if blocked.calls.Load() != 1 {
t.Errorf("Shutdown called %d times, want 1 - the drain has to be attempted even when it "+
"cannot be waited out", blocked.calls.Load())
}
}
// After the drain this package owns nothing. A reload arriving mid-shutdown
// then builds a new adapter rather than being handed a closed one as its
// `previous` to shut down again.
func TestTheDrainGivesUpOwnershipOfTheAdapter(t *testing.T) {
isolate(t)
setInstalled(newCountingQueue())
shutdownQueue(context.Background())
if got := currentInstalled(); got != nil {
t.Errorf("installed is %T after the drain, want nil", got)
}
}
// runtimeQueue is a full AdapterQueue, so it can be handed to the runtime
// rather than only to this package's own record of what it installed.
type runtimeQueue struct {
countingQueue
}
func (q *runtimeQueue) String() string { return "runtime-fake" }
func (q *runtimeQueue) Append(corestorage.Messager) error { return nil }
func (q *runtimeQueue) Register(string, corestorage.ConsumerFunc) {}
func (q *runtimeQueue) Run() {}
// The drain must not reach a queue this package did not install.
//
// sdk.Runtime.GetQueueAdapter never returns nil: with no queue section
// configured it wraps the runtime's own fallback, and the wrapper's Shutdown
// forwards. Reaching for the accessor would therefore look like it worked and
// would close a queue this package neither built nor started - the same shape
// as the `if q := GetQueueAdapter(); q != nil` that setupQueue already had to
// drop.
//
// The previous test's empty-configuration case cannot see this: it only checks
// that RunShutdown returns, which it would either way.
func TestTheDrainNeverReachesTheRuntimesOwnQueue(t *testing.T) {
isolate(t)
onTheRuntime := &runtimeQueue{countingQueue: *newCountingQueue()}
sdk.Runtime.SetQueueAdapter(onTheRuntime)
if currentInstalled() != nil {
t.Fatal("this package installed something, so the distinction under test is not set up")
}
if sdk.Runtime.GetQueueAdapter() == nil {
t.Fatal("the accessor returned nil, so it is no longer the trap this guards")
}
shutdownQueue(context.Background())
if got := onTheRuntime.calls.Load(); got != 0 {
t.Errorf("the runtime's queue was shut down %d times - the drain went through "+
"GetQueueAdapter instead of the adapter this package installed", got)
}
}
// A drain that finishes in the same instant the budget expires counts as
// finished.
//
// Both channels are ready when the select runs, and select picks at random
// among ready cases, so a single look reports an overrun for a drain that
// completed - roughly half the times it lands here. The repetition is what
// makes that visible: one iteration passes either way.
func TestATieBetweenTheDeadlineAndTheDrainGoesToTheDrain(t *testing.T) {
expired, cancel := context.WithCancel(context.Background())
cancel()
<-expired.Done()
done := make(chan struct{})
close(done)
for i := 0; i < 1000; i++ {
if !finishedBeforeDeadline(expired, done) {
t.Fatalf("iteration %d of 1000: both the deadline and the drain were ready and the "+
"deadline won - a drain that completed is being reported as an overrun", i)
}
}
}
// The other side of it. A drain that really has not finished has to be
// reported, or the warning never fires and the tie-break above has quietly
// turned into "always say it drained".
func TestADrainThatHasNotFinishedIsReportedAsAnOverrun(t *testing.T) {
expired, cancel := context.WithCancel(context.Background())
cancel()
<-expired.Done()
stillRunning := make(chan struct{}) // never closed
if finishedBeforeDeadline(expired, stillRunning) {
t.Error("an unfinished drain was reported as having finished in time")
}
}
+130
View File
@@ -0,0 +1,130 @@
package storage
import (
"errors"
"os"
"testing"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
"github.com/go-admin-team/go-admin-core/v2/storage/queue"
)
// redisAddrEnv points these tests at a server. They are skipped without it, so
// a developer with no redis running still gets a green run - and CI sets it,
// which is the point: the ordering rule they cover is invisible on the memory
// backend, and memory is the default. A suite that only ever exercised the
// default would report success for a queue that silently drops every consumer.
const redisAddrEnv = "GO_ADMIN_TEST_REDIS_ADDR"
func redisAddr(t *testing.T) string {
t.Helper()
addr := os.Getenv(redisAddrEnv)
if addr != "" {
return addr
}
// Skipping locally is the point; skipping in CI is the failure this whole
// file exists to prevent. A workflow that renamed the variable, or dropped
// the service, would otherwise go green while these two tests quietly did
// nothing - which is the same shape as the defect they cover.
if os.Getenv("CI") != "" {
t.Fatalf("%s is not set while CI is: the redis-backed queue tests must not skip here", redisAddrEnv)
}
t.Skipf("%s is not set; skipping the redis-backed queue tests", redisAddrEnv)
return ""
}
// newRedisQueue builds the queue the same way setupQueue does - through
// config.QueueConfig.Setup - so that what is under test is the adapter this
// repository actually gets, LegacyQueueAdapter and all, rather than a redis
// client wired up by the test.
func newRedisQueue(t *testing.T, prefix string) corestorage.AdapterQueue {
t.Helper()
previous := config.QueueConfig
t.Cleanup(func() { config.QueueConfig = previous })
config.QueueConfig = &config.Queue{
Redis: &config.RedisQueue{
RedisOptions: config.RedisOptions{Addr: redisAddr(t)},
Group: prefix,
KeyPrefix: prefix,
},
}
q, err := config.QueueConfig.Setup()
if err != nil {
t.Fatalf("queue setup: %v", err)
}
t.Cleanup(q.Shutdown)
return q
}
func message(t *testing.T, stream string) corestorage.Messager {
t.Helper()
m := &queue.Message{}
m.SetStream(stream)
m.SetValues(map[string]interface{}{"hello": "world"})
return m
}
// Registered first, then started: the consumer gets the message. This is the
// order setupQueue and attachQueueConsumers now produce between them.
func TestRedisQueueDeliversToAConsumerRegisteredBeforeTheStart(t *testing.T) {
stream := "t-ordered"
q := newRedisQueue(t, "gotest-ordered")
got := make(chan struct{}, 1)
q.Register(stream, func(corestorage.Messager) error {
select {
case got <- struct{}{}:
default:
}
return nil
})
go q.Run()
// Give Start a moment to reach its read loop before publishing.
time.Sleep(500 * time.Millisecond)
if err := q.Append(message(t, stream)); err != nil {
t.Fatalf("append: %v", err)
}
select {
case <-got:
case <-time.After(15 * time.Second):
t.Fatal("the consumer never received the message")
}
}
// Started first, then registered: the registration is refused and every
// publish afterwards fails.
//
// Subscribe answers ErrQueueAlreadyStarted, and LegacyQueueAdapter.Register
// returns nothing, so the caller cannot know - that part is silent. What is not
// silent is the consequence: no consumer group was created, so Publish refuses
// the topic with ErrNoHandler on every single request, and go-admin's call
// sites log that at error level while the login and operation log rows are
// never written.
//
// This is the test the memory backend cannot provide. queue.Memory's Register
// starts another consumer goroutine whatever the state, so the same code passes
// there - which is how the defect survived, memory being the default.
func TestRedisQueueRefusesAConsumerRegisteredAfterTheStart(t *testing.T) {
stream := "t-late"
q := newRedisQueue(t, "gotest-late")
go q.Run()
time.Sleep(500 * time.Millisecond)
q.Register(stream, func(corestorage.Messager) error { return nil })
err := q.Append(message(t, stream))
if err == nil {
t.Fatal("a message was accepted for a topic whose registration came after Start; " +
"if the backend now accepts late registration, the ordering rule in setupQueue can be revisited")
}
if !errors.Is(err, corestorage.ErrNoHandler) {
t.Fatalf("append failed with %v, want %v - the test is meant to pin the "+
"missing-consumer path, not any error at all", err, corestorage.ErrNoHandler)
}
}
+161
View File
@@ -0,0 +1,161 @@
package storage
import (
"errors"
"sync"
"sync/atomic"
"testing"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
"github.com/go-admin-team/go-admin-core/v2/storage/queue"
)
// sampleSize is how many publishes have to land inside the reload before the
// measurement is taken. Waiting on the count rather than on wall clock keeps
// the window the test covers the same on a loaded runner as on an idle one.
const sampleSize = 200
func swapMsg() corestorage.Messager {
m := new(queue.Message)
m.SetStream("t")
m.SetValues(map[string]interface{}{"a": "b"})
return m
}
// A reload must never leave producers holding a queue that has stopped
// accepting.
//
// Shutdown waits for its consumers to deliver what the queue still holds. Taking
// the old adapter down before installing the new one meant the runtime pointed
// at a closed queue for that entire wait: every Append in the window came back
// ErrQueueClosed, and both call sites in common/middleware log it at error
// level. Installing first leaves no window - a producer gets the new queue or
// the old one, and both accept.
//
// The difference is only visible during that wait, which is why the test holds
// a consumer rather than checking the state after Setup has returned: by then
// the two orders look identical.
//
// One refusal survives the fix and is not something this ordering can reach.
// GetQueuePrefix hands back a wrapper that captured the adapter, so a producer
// that fetched before the swap and appends after Shutdown has begun is still
// holding the old one. That window is one call wide and closing it means
// resolving the adapter inside Append, which is core's to change. What the
// ordering removes is the sustained window: every producer that fetches during
// the wait. The test publishes from a single goroutine, so at most one of its
// calls can straddle the swap - which is what makes "more than one" the line
// between the two orders rather than a tolerance.
func TestAReloadNeverPointsProducersAtAClosedQueue(t *testing.T) {
prevQ, prevC := config.QueueConfig, config.CacheConfig
prevRuntime := sdk.Runtime
prevInstalled, prevGen := installed, installedGen
t.Cleanup(func() {
config.QueueConfig, config.CacheConfig = prevQ, prevC
sdk.Runtime = prevRuntime
queueMu.Lock()
installed, installedGen = prevInstalled, prevGen
queueMu.Unlock()
})
sdk.Runtime = runtime.NewConfig()
config.CacheConfig = &config.Cache{Memory: struct{}{}}
// Sized so the buffer cannot fill while the consumer is held: a full queue
// returns an error of its own, and this test needs every error other than
// ErrQueueClosed to mean something it does not model has happened.
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 4096}}
Setup()
// A consumer that will not finish until this test lets it, so the reload's
// Shutdown has something to wait for.
release := make(chan struct{})
consuming := make(chan struct{})
var picked sync.Once
first := sdk.Runtime.GetQueuePrefix("")
first.Register("t", func(corestorage.Messager) error {
picked.Do(func() { close(consuming) })
<-release
return nil
})
go first.Run()
for i := 0; i < 4; i++ {
if err := first.Append(swapMsg()); err != nil {
t.Fatalf("seed append %d: %v", i, err)
}
}
select {
case <-consuming:
case <-time.After(10 * time.Second):
t.Fatal("the consumer never picked a message up, so the reload has nothing to wait for")
}
reloaded := make(chan struct{})
go func() { Setup(); close(reloaded) }()
// Publish continuously while the reload is in progress.
var refused atomic.Int64
var attempts atomic.Int64
unexpected := make(chan error, 1)
stop := make(chan struct{})
// publishing is closed by the producer on its way out. The test joins on it
// before returning: t.Cleanup restores sdk.Runtime, and a producer still in
// flight would be reading the variable that restore writes.
publishing := make(chan struct{})
go func() {
defer close(publishing)
for {
select {
case <-stop:
return
default:
}
attempts.Add(1)
err := sdk.Runtime.GetQueuePrefix("").Append(swapMsg())
switch {
case err == nil:
case errors.Is(err, corestorage.ErrQueueClosed):
refused.Add(1)
default:
// Kept rather than counted: an Append refused for some other
// reason would otherwise leave refused at zero and the test
// green while nothing was reaching a queue at all.
select {
case unexpected <- err:
default:
}
}
time.Sleep(time.Millisecond)
}
}()
deadline := time.After(30 * time.Second)
for attempts.Load() < sampleSize {
select {
case <-deadline:
t.Fatalf("only %d publishes landed inside the reload; the window was never sampled", attempts.Load())
case <-time.After(time.Millisecond):
}
}
close(release)
select {
case <-reloaded:
case <-time.After(30 * time.Second):
t.Fatal("the reload never finished")
}
close(stop)
<-publishing
select {
case err := <-unexpected:
t.Fatalf("a publish failed for a reason this test does not model: %v", err)
default:
}
if n := refused.Load(); n > 1 {
t.Errorf("%d of %d publishes during the reload were refused: producers were pointed at the closed queue",
n, attempts.Load())
}
}
+53
View File
@@ -0,0 +1,53 @@
package storage
import (
"testing"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
)
// Issue #892: a configuration reload replaces the queue adapter and the
// consumers registered against the previous one are attached to a queue nobody
// publishes to any more.
//
// The fix has two halves. attachQueueConsumers gives a new queue its own
// consumers and the same queue none, which cmd/api covers against a queue the
// test controls. This is the other half: that a reload actually produces a new
// queue for it to notice. Setup is what config re-runs on every change, so
// calling it twice is what a reload does to this package.
func TestSetupBumpsTheQueueGenerationOnEveryReload(t *testing.T) {
// Setup writes the process-wide sdk.Runtime - the cache and queue adapters -
// and this package's own record of what it installed. Restoring all of it
// keeps the test from deciding what a later test in this binary sees,
// which is the same isolation cmd/api's freshRuntime provides.
prevQ, prevC := config.QueueConfig, config.CacheConfig
prevRuntime := sdk.Runtime
prevInstalled, prevGen := installed, installedGen
t.Cleanup(func() {
config.QueueConfig, config.CacheConfig = prevQ, prevC
sdk.Runtime = prevRuntime
queueMu.Lock()
installed, installedGen = prevInstalled, prevGen
queueMu.Unlock()
})
sdk.Runtime = runtime.NewConfig()
config.CacheConfig = &config.Cache{Memory: struct{}{}}
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
before := QueueGeneration()
Setup()
first := QueueGeneration()
Setup()
second := QueueGeneration()
t.Logf("before=%d first=%d second=%d", before, first, second)
if first == before {
t.Fatal("the first Setup did not install a queue")
}
if second == first {
t.Fatal("a second Setup - which is what a configuration reload does - did not install a new one")
}
}
+136
View File
@@ -1,5 +1,10 @@
package config
import (
"fmt"
"strings"
)
var ExtConfig Extend
// Extend 扩展配置
@@ -13,6 +18,7 @@ type Extend struct {
AMap AMap // 这里配置对应配置文件的结构即可
FileStore FileStore
RateLimit RateLimit
Shutdown Shutdown
}
// DefaultInboundQPS is the limit applied when nothing is configured. It is the
@@ -72,3 +78,133 @@ type ObjectStore struct {
func (o ObjectStore) Configured() bool {
return o.Endpoint != "" && o.AccessKeyID != "" && o.AccessKeySecret != "" && o.BucketName != ""
}
// Default budgets for a graceful shutdown, in seconds. Each applies to the
// matching field of extend.shutdown when that field is absent, and together
// they are what the process spent before the section existed - so a deployment
// that configures nothing keeps the shutdown it already had.
//
// The drain default is zero deliberately. The three budgets are spent one
// after the other, and once their sum reaches the orchestrator's stop grace
// period the process is killed part-way through its cleanup callbacks, which
// is worse than not draining at all. `docker stop` allows ten seconds by
// default and 5+3 already leaves little room, so a non-zero default here would
// slow down every existing shutdown to buy something only a load balancer that
// polls /ready can collect.
const (
DefaultDrainSeconds = 0
DefaultServerSeconds = 5
DefaultCleanupSeconds = 3
)
// Shutdown is how long a graceful shutdown may spend, stage by stage.
//
// extend:
// shutdown:
// drain: 0
// server: 5
// cleanup: 3
// grace: 30
//
// Every field is a pointer for the reason RateLimit.InboundQPS is: nil means
// "not configured" and takes the default, while a value that was written down
// is taken literally, zero included. Without that separation `server: 0` - do
// not wait for in-flight requests, which is a reasonable thing to ask under a
// very short grace period - could not be said at all, and `drain: 0` would
// have to mean something different from `server: 0` in the same section.
type Shutdown struct {
// Drain is how long to keep serving normally after a stop signal arrives.
// Throughout it /ready answers 503 and keep-alive is switched off, which
// is what gives whatever routes traffic here time to stop routing it
// before the listener closes. Zero is no window: the readiness flip and
// the listener closing are then microseconds apart and nothing observes
// the first.
//
// What the window is worth depends on who does the removing and on what
// basis; the package comment in common/health has the two cases, and they
// do not want the same value.
Drain *int
// Server is how long the server waits for in-flight requests once the
// listener is closed.
Server *int
// Cleanup is how long the BeforeExit callbacks get after that.
Cleanup *int
// Grace is the stop grace period the orchestrator gives this process -
// `docker stop --timeout`, or terminationGracePeriodSeconds. Nothing reads
// it during a shutdown; it exists so start-up can say whether the budget
// fits inside it. Absent means no comparison is made, because the
// reference values differ threefold between runtimes and a fixed threshold
// would warn about configurations that are correct.
Grace *int
}
// ShutdownBudget is what a shutdown will actually spend, in seconds, after the
// fallbacks have been applied.
type ShutdownBudget struct {
Drain int
Server int
Cleanup int
// Grace is zero when extend.shutdown.grace was not configured.
Grace int
}
// Budget resolves the configured section into the values that will be spent.
//
// A negative is refused rather than corrected. A wait cannot be negative, so
// there is no reading of one to honour, and quietly turning it into zero would
// be the failure this whole section exists to remove: written down, accepted,
// and not what happens. It is returned as an error rather than reported here
// so that the rule can be checked without ending the process.
func (s Shutdown) Budget() (ShutdownBudget, error) {
var negative []string
for _, f := range []struct {
name string
value *int
}{
{"drain", s.Drain},
{"server", s.Server},
{"cleanup", s.Cleanup},
{"grace", s.Grace},
} {
if f.value != nil && *f.value < 0 {
negative = append(negative, fmt.Sprintf("%s: %d", f.name, *f.value))
}
}
if len(negative) > 0 {
return ShutdownBudget{}, fmt.Errorf(
"extend.shutdown was given a negative number of seconds (%s); "+
"a wait cannot be negative, and 0 is how to say \"do not wait\"",
strings.Join(negative, ", "))
}
return ShutdownBudget{
Drain: budgetSeconds(s.Drain, DefaultDrainSeconds),
Server: budgetSeconds(s.Server, DefaultServerSeconds),
Cleanup: budgetSeconds(s.Cleanup, DefaultCleanupSeconds),
Grace: budgetSeconds(s.Grace, 0),
}, nil
}
func budgetSeconds(configured *int, fallback int) int {
if configured != nil {
return *configured
}
return fallback
}
// Total is the whole of the shutdown, since the three stages run one after the
// other.
func (b ShutdownBudget) Total() int { return b.Drain + b.Server + b.Cleanup }
// Overrun reports how many seconds have to be found for the budget to fit
// inside the configured grace period. It is zero when no grace period was
// configured and when the budget already fits.
//
// Fitting means strictly less: the grace period is when SIGKILL is sent, so a
// budget that ends exactly then leaves the last callback no time to return.
func (b ShutdownBudget) Overrun() int {
if b.Grace <= 0 || b.Total() < b.Grace {
return 0
}
return b.Total() - b.Grace + 1
}
+167 -1
View File
@@ -1,6 +1,9 @@
package config
import "testing"
import (
"strings"
"testing"
)
func TestObjectStoreConfigured(t *testing.T) {
if (ObjectStore{}).Configured() {
@@ -32,3 +35,166 @@ func TestRateLimitThreshold(t *testing.T) {
t.Errorf("configured limit = %v, want %v", got, custom)
}
}
func ptr(v int) *int { return &v }
// The zero-value rule is the same for all four fields, and it is the one the
// section would otherwise need a paragraph of documentation to survive: nil
// takes the default, a number that was written down is spent literally. A
// `server: 0` that quietly became five seconds would be the same class of
// failure this whole batch is about - configuration accepted and not applied.
func TestShutdownBudgetFallbacks(t *testing.T) {
for _, tc := range []struct {
name string
in Shutdown
want ShutdownBudget
}{
{
// What an existing settings.yml hits after an upgrade: no
// extend.shutdown section at all, and therefore the shutdown it
// already had.
name: "nothing configured",
in: Shutdown{},
want: ShutdownBudget{Drain: 0, Server: 5, Cleanup: 3},
},
{
name: "all four configured",
in: Shutdown{Drain: ptr(10), Server: ptr(8), Cleanup: ptr(4), Grace: ptr(30)},
want: ShutdownBudget{Drain: 10, Server: 8, Cleanup: 4, Grace: 30},
},
{
// The case a plain int could not express: do not wait for
// in-flight requests, which is a reasonable thing to ask for when
// the grace period is very short.
name: "explicit zeros are spent, not replaced",
in: Shutdown{Drain: ptr(0), Server: ptr(0), Cleanup: ptr(0)},
want: ShutdownBudget{Drain: 0, Server: 0, Cleanup: 0},
},
{
name: "one field configured, the rest default",
in: Shutdown{Drain: ptr(15)},
want: ShutdownBudget{Drain: 15, Server: 5, Cleanup: 3},
},
} {
t.Run(tc.name, func(t *testing.T) {
got, err := tc.in.Budget()
if err != nil {
t.Fatalf("Budget() = %v", err)
}
if got != tc.want {
t.Errorf("Budget() = %+v, want %+v", got, tc.want)
}
})
}
}
// A negative is refused, not corrected. Turning it into zero would be the
// failure this section exists to remove - written down, accepted, and not what
// happens - and there is no reading of a negative wait to honour.
//
// The last row is what makes the other four mean anything: an implementation
// that refused every value would pass them all.
func TestShutdownBudgetRefusesNegativeSeconds(t *testing.T) {
for _, tc := range []struct {
name string
in Shutdown
wantErr bool
}{
{name: "negative drain", in: Shutdown{Drain: ptr(-1)}, wantErr: true},
{name: "negative server", in: Shutdown{Server: ptr(-1)}, wantErr: true},
{name: "negative cleanup", in: Shutdown{Cleanup: ptr(-1)}, wantErr: true},
{name: "negative grace", in: Shutdown{Grace: ptr(-1)}, wantErr: true},
{name: "explicit zeros are not negative", in: Shutdown{Drain: ptr(0), Server: ptr(0), Cleanup: ptr(0)}},
} {
t.Run(tc.name, func(t *testing.T) {
_, err := tc.in.Budget()
if tc.wantErr && err == nil {
t.Fatal("Budget() accepted a negative number of seconds")
}
if !tc.wantErr && err != nil {
t.Fatalf("Budget() = %v, want the zeros taken literally", err)
}
})
}
}
// The message has to name every field that is wrong, not the first one: a
// caller who fixes one and gets the same error back learns to distrust it.
func TestShutdownBudgetNamesEveryNegativeField(t *testing.T) {
_, err := Shutdown{Drain: ptr(-1), Server: ptr(-30), Cleanup: ptr(-3), Grace: ptr(-9)}.Budget()
if err == nil {
t.Fatal("Budget() accepted four negative values")
}
for _, name := range []string{"drain", "server", "cleanup", "grace"} {
if !strings.Contains(err.Error(), name) {
t.Errorf("%q does not name %s", err, name)
}
}
}
// The sum is what has to fit inside the orchestrator's grace period, and the
// verdict is only reached when a grace period was configured. A fixed
// threshold instead would warn about the manifest this repository ships.
func TestShutdownBudgetOverrun(t *testing.T) {
resolved := func(s Shutdown) ShutdownBudget {
b, err := s.Budget()
if err != nil {
t.Fatalf("Budget() = %v", err)
}
return b
}
for _, tc := range []struct {
name string
budget ShutdownBudget
wantTotal int
wantOverrun int
}{
{
name: "defaults, no grace period to judge against",
budget: resolved(Shutdown{}),
wantTotal: 8,
},
{
name: "fits with room to spare",
budget: resolved(Shutdown{Drain: ptr(10), Grace: ptr(30)}),
wantTotal: 18,
},
{
// Equal is not a fit. The grace period is when SIGKILL is sent, so
// a budget that ends exactly then leaves the last callback no time
// to return.
name: "exactly equal still overruns",
budget: resolved(Shutdown{Drain: ptr(22), Grace: ptr(30)}),
wantTotal: 30,
wantOverrun: 1,
},
{
name: "over by five",
budget: resolved(Shutdown{Drain: ptr(26), Grace: ptr(30)}),
wantTotal: 34,
wantOverrun: 5,
},
{
// The reason the threshold is a configured value rather than a
// constant: the same budget is wrong under `docker stop` and right
// under a Kubernetes default.
name: "the docker default is the tighter one",
budget: resolved(Shutdown{Drain: ptr(10), Grace: ptr(10)}),
wantTotal: 18,
wantOverrun: 9,
},
} {
t.Run(tc.name, func(t *testing.T) {
if got := tc.budget.Total(); got != tc.wantTotal {
t.Errorf("Total() = %d, want %d", got, tc.wantTotal)
}
if got := tc.budget.Overrun(); got != tc.wantOverrun {
t.Errorf("Overrun() = %d, want %d", got, tc.wantOverrun)
}
if over := tc.budget.Overrun(); over > 0 && tc.budget.Total()-over >= tc.budget.Grace {
t.Errorf("Overrun() = %d does not bring %d under the %d grace period",
over, tc.budget.Total(), tc.budget.Grace)
}
})
}
}
+34
View File
@@ -82,6 +82,40 @@ settings:
# 会被负载均衡、监控和压测统计成成功)。
rateLimit:
inboundQPS: 200
# shutdown budgets, in seconds. The three are spent one after the other,
# and their sum has to stay inside the stop grace period the orchestrator
# allows - once it is up, SIGKILL arrives part-way through the cleanup
# callbacks, which is worse than not draining at all.
shutdown:
# How long to keep serving normally after a stop signal arrives. For that
# long /ready answers 503 and keep-alive is switched off, which is what
# gives a load balancer time to take this instance out of rotation before
# the listener closes.
#
# What to set depends on who removes this instance and on what basis. A
# load balancer that polls /ready itself needs at least "check interval x
# failure threshold + however long removal takes to apply". A Kubernetes
# Service removes the endpoint when the Pod is deleted, concurrently with
# SIGTERM and regardless of what the probe returns, so here this covers
# the delay in that removal reaching every node.
#
# 0 by default: a deployment that leaves this alone shuts down exactly as
# it did before this section existed. It also means /ready never reports
# draining - the flip and the closed listener are microseconds apart, and
# no poller reads anything in between.
drain: 0
# How long to wait for in-flight requests once the listener is closed.
server: 5
# How long the BeforeExit cleanup callbacks get after that.
cleanup: 3
# The stop grace period the orchestrator gives this process - `docker stop
# --timeout`, or terminationGracePeriodSeconds. Nothing reads it during a
# shutdown; start-up uses it to say whether the three budgets above fit
# inside it, and warns when they do not. Left out, nothing is compared:
# the reference values are 10s for docker and 30s for Kubernetes, three
# times apart, and a fixed threshold would warn about correct
# configurations.
#grace: 30
# fileStore 对象存储。上传接口的 source 参数决定走哪一家:
# source=1 只存本地,source=2 阿里云 OSS,source=3 七牛 Kodo
# 没有填的那一家在被请求时会返回明确错误,不会静默存到别处。
+34
View File
@@ -66,6 +66,40 @@ settings:
# 会被负载均衡、监控和压测统计成成功)。
rateLimit:
inboundQPS: 200
# shutdown budgets, in seconds. The three are spent one after the other,
# and their sum has to stay inside the stop grace period the orchestrator
# allows - once it is up, SIGKILL arrives part-way through the cleanup
# callbacks, which is worse than not draining at all.
shutdown:
# How long to keep serving normally after a stop signal arrives. For that
# long /ready answers 503 and keep-alive is switched off, which is what
# gives a load balancer time to take this instance out of rotation before
# the listener closes.
#
# What to set depends on who removes this instance and on what basis. A
# load balancer that polls /ready itself needs at least "check interval x
# failure threshold + however long removal takes to apply". A Kubernetes
# Service removes the endpoint when the Pod is deleted, concurrently with
# SIGTERM and regardless of what the probe returns, so here this covers
# the delay in that removal reaching every node.
#
# 0 by default: a deployment that leaves this alone shuts down exactly as
# it did before this section existed. It also means /ready never reports
# draining - the flip and the closed listener are microseconds apart, and
# no poller reads anything in between.
drain: 0
# How long to wait for in-flight requests once the listener is closed.
server: 5
# How long the BeforeExit cleanup callbacks get after that.
cleanup: 3
# The stop grace period the orchestrator gives this process - `docker stop
# --timeout`, or terminationGracePeriodSeconds. Nothing reads it during a
# shutdown; start-up uses it to say whether the three budgets above fit
# inside it, and warns when they do not. Left out, nothing is compared:
# the reference values are 10s for docker and 30s for Kubernetes, three
# times apart, and a fixed threshold would warn about correct
# configurations.
#grace: 30
cache:
# redis:
# addr: 127.0.0.1:6379
+68
View File
@@ -0,0 +1,68 @@
package config
import (
"testing"
coreconfig "github.com/go-admin-team/go-admin-core/v2/config"
"github.com/go-admin-team/go-admin-core/v2/config/source/file"
)
// shippedSettings is the shape the loader fills in, cut down to the part under
// test. The reader is JSON-based, so the keys are matched against field names
// case-insensitively - which is exactly the matching that silently drops a
// section the struct has no field for.
type shippedSettings struct {
Settings struct {
Extend Extend
}
}
func (*shippedSettings) OnChange() {}
// The shutdown section has to arrive where it is read from, and with the
// values the documentation claims.
//
// This is the failure this batch exists to remove, one level up: the loader
// discards keys no field matches, without an error and without a log line, so
// a section put in the wrong place is written, accepted, and never applied.
// Nothing but loading the shipped file through the real loader can tell the
// two apart - the struct compiles either way.
//
// The values are asserted as well as the arrival. A settings.yml that shipped
// a different default from config.Default*Seconds would give two answers to
// "what does an unconfigured deployment spend", and the file is the one people
// read.
func TestTheShippedSettingsReachTheShutdownStruct(t *testing.T) {
for _, name := range []string{"settings.yml", "settings.full.yml"} {
t.Run(name, func(t *testing.T) {
var loaded shippedSettings
c, err := coreconfig.NewConfig(
coreconfig.WithSource(file.NewSource(file.WithPath(name))),
coreconfig.WithEntity(&loaded),
)
if err != nil {
t.Fatalf("load %s: %v", name, err)
}
t.Cleanup(func() { _ = c.Close() })
s := loaded.Settings.Extend.Shutdown
if s.Drain == nil || s.Server == nil || s.Cleanup == nil {
t.Fatalf("%s left extend.shutdown unfilled (%+v); the section is written but nothing reads it",
name, s)
}
want := ShutdownBudget{
Drain: DefaultDrainSeconds,
Server: DefaultServerSeconds,
Cleanup: DefaultCleanupSeconds,
}
got, err := s.Budget()
if err != nil {
t.Fatalf("%s does not resolve: %v", name, err)
}
if got != want {
t.Errorf("%s ships %+v, want the documented defaults %+v", name, got, want)
}
})
}
}
+7
View File
@@ -7,6 +7,13 @@ services:
restart: always
ports:
- 8000:8000
# Compose allows 10 seconds by default, and this process spends
# drain + server + cleanup from extend.shutdown before it exits - 8 out of
# the box, more for anyone who configures a drain window. Past the deadline
# it is sent SIGKILL and the cleanup callbacks are cut off part-way
# through. checksilent's docker-stop-cuts-shutdown-short check compares
# this against config/settings.yml.
stop_grace_period: 30s
volumes:
- ./config/:/go-admin-api/config/
- ./static/:/go-admin-api/static/
+89 -3
View File
@@ -377,7 +377,7 @@ if res.RowsAffected == 0 { return ErrAlreadyPaid } // 别人先改了
| 应用间调用 | 零定义。A 应用要调 B 应用只能直接 import 对方的包,循环依赖就回来了 |
| 领域事件 / EventBus | 无 |
| 缓存的租户隔离 | `service.Service` 有 `Cache` 字段,**是否按租户隔离未验证**。当作没隔离来写 |
| 异步任务 | 有队列,但热更新后消费者会丢(issue #892) |
| 生命周期钩子之外的时点 | 只有下面那四个。没有「路由装好之后、开始监听之前」这一档 |
这几条留给后续批次,按真实需求补——现在凭空设计只会设计错。
如果你的应用卡在这里,在 issue 里说一声,那正是我们要的输入。
@@ -672,6 +672,91 @@ if sdk.Runtime.AppRoutersSealed() { /* RunAppRouters 已经跑过了 */ }
---
## 生命周期挂载点
除了注册路由和迁移,应用还可以把工作挂在进程生命的四个时点上,不必等宿主按名字来调自己。
契约本身在 core,见
[go-admin-core `docs/contract.md`](https://github.com/go-admin-team/go-admin-core/blob/main/docs/contract.md)
的「Life-cycle phases」一节。这里只写**在本仓里它们分别落在哪一行**。
| 阶段 | 在 `cmd/api/server.go` 的位置 | 此时可用 |
|---|---|---|
| `AfterResource` | `bootstrap.SetupConfig` 跑完 `database.Setup` / `storage.Setup` 之后 | 配置、库、缓存、队列、casbin |
| `BeforeRouter` | `initRouter()` **之前** | 以上,加引擎尚未构建这一事实 |
| `AfterListen` | `startServing()` 里,`net.Listen` 返回之后 | 全部,端口**已绑定**、连接进得来 |
| `BeforeExit` | `srv.Shutdown` 返回之后(无论它是否报错) | 全部,正在被拆掉 |
`AfterListen` 承诺的是**端口已绑定**,不是「`Serve` 已经在 accept 循环里」——
`srv.Serve` 在另一个 goroutine 上。这个区别是真实的:绑定成功之后内核就会把连接排进
backlog,所以钩子里去连自己的端口不会被拒;但此刻 `Serve` 可能还没跑到第一次 `Accept`。
绑定失败则**根本不会有这个阶段**:`net.Listen` 的错误直接从 `run()` 返回,
横幅不打印,进程非零退出。
```go
sdk.Runtime.SetPhase(runtime.AfterResource, func() { /* ... */ })
sdk.Runtime.SetShutdown(func(ctx context.Context) { /* ... */ })
```
### `BeforeRouter` 不等于 `before` 注册表
**这两个不是同一个时点,文档里别混着写。** `SetBefore` 的回调由
`runStartupHooks()` 执行,而那是在 `initRouter()` **之后**——引擎已经建好了。
`BeforeRouter` 在它之前。
顺带:`BeforeRouter` 是「硬约束:注册要赶在启动钩子之前」那一节所说的合法注册窗口之一。
它早于 `runStartupHooks()`,所以在这里调 `sdk.Runtime.SetAppRouters` 仍然来得及。
### `AfterResource` 会跑很多次,回调必须扛得住
它在**每次配置热更新之后**都会再跑一遍,因为热更新会重建它所命名的那些资源。
所以这里的回调要求是**「对同一个资源幂等」,不是「第二次什么都不做」**。
本仓自己的队列消费者就是这条规则的样板,也是它存在的理由
(`cmd/api/server.go` 的 `attachQueueConsumers`):
- 热更新重建了队列适配器,挂在旧适配器上的消费者连着一个**再没人往里发消息**的队列,
登录日志和操作日志就此停写且不出声。所以新适配器**必须**重新注册。
- 但同一个适配器不能注册两次,否则每条消息有两个消费者,每行日志写两遍。
**身份不能从访问器取。** `sdk.Runtime.GetQueueAdapter()` 与 `GetQueuePrefix()`
每次调用都新造一个 `runtime.Queue` 包装,比较两次返回等于比较两个包装,
**底层适配器换过多少次都不相等**。要在**创建资源的地方**记身份——
本仓是 `common/storage.QueueGeneration()`。
### 注册消费者要赶在队列启动之前
走哪条实现,取决于配置里有没有 `redis:` 段(`config.QueueConfig.Setup()`):
| 配置 | 实际类型 | 启动后还能注册吗 |
|---|---|---|
| `queue: memory:` | `queue.NewMemory` | **能**。它的 `Register` 每次起一个消费 goroutine,不看是否已 `Run` |
| `queue: redis:` | `storage.LegacyQueueAdapter` 包着新契约实现 | **不能**。`Register` 内部调 `Subscribe`,启动后返回 `storage.ErrQueueAlreadyStarted` |
而 `LegacyQueueAdapter.Register` **没有返回值**——它只能把这个错误写进 slog,
core 里那行注释自己写着「The interface has no way to report this to the caller」。
**静默的是注册这一步,不是之后。** 没有建立消费组,redis 会用
`storage.ErrNoHandler` 拒绝**之后的每一次投递**,而本仓两个调用点
(`common/middleware/logger.go`、`common/middleware/handler/auth.go`)
都把它记为 error——于是日志行一条都不落库,同时每个请求刷一条错误日志。
所以顺序是硬的:**先 `Register` 完,再由注册方 `Run()`。**
`common/storage` 的 `setupQueue` 有意不启动队列。
默认配置选的是 memory 后端,它不在乎顺序——**这个缺陷在默认部署里看不见,
只在配了 redis 的部署上发作**,而丢掉的正是登录日志、操作日志和 api 检查。
### `BeforeExit` 反序执行,预算约束的是等待
清理按**注册的逆序**执行。`SetShutdown` 拿到宿主剩余的预算,
但**它约束的是等待,不是工作**:预算用尽时 `RunShutdown` 停止等待并返回,
而不检查 context 的回调会一直跑到进程退出。Go 没法取消一个不检查取消的函数。
本仓的样板是 cron(`app/jobs/jobbase.go` 的 `startCrontab`):
`cron.Stop()` 返回一个在**已经在跑的任务结束时**关闭的 context,
钩子在它和预算之间二选一。
---
## 安全边界:装一个应用等于信任它
**这一层划不出安全边界,本文不假装划得出。**
@@ -718,5 +803,6 @@ if sdk.Runtime.AppRoutersSealed() { /* RunAppRouters 已经跑过了 */ }
`checksilent` 一个文件都看不到。所以它保的是**这个仓库和它的 fork**,
不是你的应用——你的应用要自己跑自己的检查。
`checksilent` 还检查另外五类"不出声的失败",写模块时值得先看一眼
`go run ./tools/checksilent -h`。
`checksilent` 还检查其他几类"不出声的失败",写模块时值得先看一眼
`AGENTS.md` 的「静默失败校验」一节,或者 `tools/checksilent/checks.go` 里的
`runChecks`(`-h` 只打印命令行参数,不列检查)。
+1 -1
View File
@@ -5,7 +5,7 @@ go 1.25.13
require (
github.com/gin-gonic/gin v1.12.0
github.com/glebarez/sqlite v1.11.0
github.com/go-admin-team/go-admin-core/v2 v2.5.0
github.com/go-admin-team/go-admin-core/v2 v2.8.0
gorm.io/gorm v1.31.2
)
+2 -2
View File
@@ -58,8 +58,8 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-admin-team/go-admin-core/v2 v2.5.0 h1:aD1SALklBxizGB9u8cOgm4OT8z656FM83F4fD6dMz9g=
github.com/go-admin-team/go-admin-core/v2 v2.5.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-admin-team/go-admin-core/v2 v2.8.0 h1:ZTw5Z/UT1/7OltbGPEaEVerRk4z3koB6O8nDbb84tPM=
github.com/go-admin-team/go-admin-core/v2 v2.8.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
+39
View File
@@ -0,0 +1,39 @@
package migration
import (
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/app"
)
// Version is what this application calls itself. A host's installer records
// it, compares it against what is already installed to tell an upgrade from a
// downgrade, and shows it in `migrate status`.
//
// It is not the same thing as the migration version above, and the two move
// independently: adding a migration file without changing what the
// application is called is normal, and so is a release that changes no
// schema. The migration versions decide what runs; this decides what the
// installed row says.
const Version = "1.0.0"
// The manifest is registered from this package rather than one of its own
// because this is the package a host has to import for the application to
// exist at all - the migrations register here too. A second package would be
// a second thing to remember to import, and forgetting it would leave an
// application whose migrations run and which no installer can name.
func init() {
app.Register(app.Manifest{
Code: AppCode,
Name: "Order example",
Version: Version,
Description: "A worked example of an application that ships its own tables, menus and APIs.",
Author: "go-admin",
// Nothing yet. When an application does declare dependencies, a host
// refuses to install it until they are installed - it does not
// install them for you, because the blast radius of installing one
// application should not be "and everything it happens to name".
Requires: nil,
// Reserved. A host stores both and interprets neither.
Pricing: "",
License: "MIT",
})
}
+2 -2
View File
@@ -11,7 +11,7 @@ require (
github.com/casbin/casbin/v3 v3.8.1
github.com/gin-gonic/gin v1.12.0
github.com/glebarez/sqlite v1.11.0
github.com/go-admin-team/go-admin-core/v2 v2.5.0
github.com/go-admin-team/go-admin-core/v2 v2.8.0
github.com/google/uuid v1.6.0
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible
github.com/mssola/user_agent v0.6.0
@@ -26,6 +26,7 @@ require (
github.com/swaggo/gin-swagger v1.6.1
github.com/swaggo/swag v1.16.6
github.com/unrolled/secure v1.17.0
go.yaml.in/yaml/v3 v3.0.5
golang.org/x/crypto v0.54.0
gorm.io/driver/mysql v1.6.0
gorm.io/driver/postgres v1.6.2
@@ -126,7 +127,6 @@ require (
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.1 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/arch v0.30.0 // indirect
golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93 // indirect
golang.org/x/image v0.41.0 // indirect
+2 -2
View File
@@ -145,8 +145,8 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-admin-team/go-admin-core/v2 v2.5.0 h1:aD1SALklBxizGB9u8cOgm4OT8z656FM83F4fD6dMz9g=
github.com/go-admin-team/go-admin-core/v2 v2.5.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-admin-team/go-admin-core/v2 v2.8.0 h1:ZTw5Z/UT1/7OltbGPEaEVerRk4z3koB6O8nDbb84tPM=
github.com/go-admin-team/go-admin-core/v2 v2.8.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o=
+61
View File
@@ -22,6 +22,21 @@ metadata:
app: go-admin
version: v1
spec:
# One replica, and the drain window below buys nothing at one replica: there
# is nowhere to send the traffic this pod stops taking. Raising it needs two
# changes that are not this number:
#
# The volume below is shared by every replica, and the log path in
# settings.yml lives on it, so a second pod would append to the same
# rotating file.
#
# The job scheduler is per process while its handle on a job is one shared
# column. Startup runs `UPDATE sys_job SET entry_id = 0 WHERE entry_id > 0`
# across the whole table (app/jobs/jobbase.go), so a second pod erases the
# first pod's ids and writes its own, and every pod registers the whole
# enabled list in its own scheduler. Neither symptom logs anything: an
# enabled job fires once per pod, and stopping one from the UI removes an
# entry from the wrong process and still answers 200. See #915.
replicas: 1
selector:
matchLabels:
@@ -39,6 +54,40 @@ spec:
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8000
# Readiness answers "send me requests". It fails while the database or
# the cache is unreachable, so this pod stays out of the Service until
# the datastore settings.yml names is really there - which is a change
# from having no probe at all, where a pod with an unreachable database
# was still sent traffic.
#
# timeoutSeconds is 3 rather than the default 1 because the handler
# allows its checks 2 seconds (readyTimeout in
# app/other/router/monitor.go). At the default, a database that answers
# in 1.2s is recorded as a failed check while the handler is returning
# 200.
readinessProbe:
httpGet:
path: /api/v1/ready
port: 8000
periodSeconds: 5
timeoutSeconds: 3
failureThreshold: 3
# Liveness answers "restart me", which is a different question: a
# process whose database is unreachable does not want restarting, so
# this points at /health, which is a bare 200. Both probes skip the
# rate limiter - see exemptProbes in cmd/api/server.go - because a
# liveness probe that collects 429s under load gets the container
# restarted at the moment the deployment can least afford to lose it.
#
# initialDelaySeconds covers the migrations, which run before the
# listener opens.
livenessProbe:
httpGet:
path: /api/v1/health
port: 8000
initialDelaySeconds: 15
periodSeconds: 10
failureThreshold: 3
volumeMounts:
- name: go-admin
mountPath: /temp
@@ -47,6 +96,18 @@ spec:
- name: go-admin-config
mountPath: /config/
readOnly: true
# SIGKILL arrives when this is up, so it has to be longer than what the
# process spends shutting down: extend.shutdown's drain + server +
# cleanup, which settings.yml ships as 0 + 5 + 3. Raise drain here and
# this number has to follow, or the cleanup callbacks are cut off
# part-way through - checksilent's shutdown-budget-overruns-grace check
# is what notices.
#
# No preStop hook on purpose. A sleep there would be spent before the
# process is told anything, so BeginDraining never runs and /ready
# answers 200 for the whole of it - and it would be added to the budget
# above rather than replacing any of it.
terminationGracePeriodSeconds: 30
volumes:
- name: go-admin
persistentVolumeClaim:
+309
View File
@@ -0,0 +1,309 @@
package main
import (
"database/sql"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"testing"
_ "github.com/glebarez/go-sqlite"
)
// The whole 008 chain, against a binary that was actually built and a
// database that was actually migrated.
//
// Everything below this level is covered by unit tests with an injected
// engine and a hand-built schema, which is where the shapes are pinned down.
// What only this can catch is the wiring: that an application's init()
// reaches both registries, that `install` finds a manifest through
// app.Snapshot, that the seeder writes what the uninstaller looks for, and
// that the command exits non-zero when a migration fails - the last of which
// a deployment reads to decide whether to start the new version.
const settings = `settings:
application:
host: 0.0.0.0
mode: dev
name: e2e
port: 8000
readtimeout: 10000
writertimeout: 20000
database:
driver: sqlite3
source: ./e2e.db
jwt:
secret: e2e
timeout: 3600
logger:
path: temp/logs
stdout: default
level: error
enableddb: false
queue:
memory:
poolSize: 10
`
type env struct {
t *testing.T
dir string
bin string
}
// The binary is built once for the whole package. Every test drives the same
// one against its own directory and its own database, and a binary is
// read-only, so there is nothing to isolate - building it per test was three
// links of the same thing.
var (
buildOnce sync.Once
sharedDir string
sharedBin string
buildErr error
)
func TestMain(m *testing.M) {
code := m.Run()
if sharedDir != "" {
os.RemoveAll(sharedDir)
}
os.Exit(code)
}
// binary builds the go-admin binary with the example application linked in,
// on the first call that needs it.
func binary(t *testing.T) string {
t.Helper()
buildOnce.Do(func() {
sharedDir, buildErr = os.MkdirTemp("", "go-admin-e2e")
if buildErr != nil {
return
}
sharedBin = filepath.Join(sharedDir, "go-admin-e2e")
out, err := exec.Command("go", "build", "-tags", "sqlite3", "-o", sharedBin, ".").CombinedOutput()
if err != nil {
buildErr = fmt.Errorf("building the binary: %v\n%s", err, out)
}
})
if buildErr != nil {
t.Fatal(buildErr)
}
return sharedBin
}
// newEnv lays out a working directory for the binary to run in.
func newEnv(t *testing.T) *env {
t.Helper()
if testing.Short() {
t.Skip("builds a binary and migrates a database; skipped under -short")
}
dir := t.TempDir()
if err := os.MkdirAll(filepath.Join(dir, "config"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.MkdirAll(filepath.Join(dir, "temp", "logs"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "config", "settings.yml"), []byte(settings), 0o644); err != nil {
t.Fatal(err)
}
// The framework's first migration reads this file rather than carrying
// the rows in Go.
seedSQL, err := os.ReadFile(filepath.Join("..", "..", "config", "db.sql"))
if err != nil {
t.Fatalf("reading config/db.sql: %v", err)
}
if err := os.WriteFile(filepath.Join(dir, "config", "db.sql"), seedSQL, 0o644); err != nil {
t.Fatal(err)
}
return &env{t: t, dir: dir, bin: binary(t)}
}
// run executes the binary and returns its combined output and exit code.
func (e *env) run(args ...string) (string, int) {
e.t.Helper()
args = append(args, "-c", "config/settings.yml")
cmd := exec.Command(e.bin, args...)
cmd.Dir = e.dir
out, err := cmd.CombinedOutput()
code := 0
if ee, ok := err.(*exec.ExitError); ok {
code = ee.ExitCode()
} else if err != nil {
e.t.Fatalf("running %v: %v", args, err)
}
return string(out), code
}
func (e *env) mustRun(args ...string) string {
e.t.Helper()
out, code := e.run(args...)
if code != 0 {
e.t.Fatalf("%v exited %d:\n%s", args, code, out)
}
return out
}
// open connects to the database the binary uses.
//
// Per call, and closed again straight away, on purpose: the binary under test
// writes this same file, and a connection the test process holds open across
// a run of it is a second writer for no reason. The cost is a few
// milliseconds against a build measured in seconds.
func (e *env) open() *sql.DB {
e.t.Helper()
db, err := sql.Open("sqlite", filepath.Join(e.dir, "e2e.db"))
if err != nil {
e.t.Fatal(err)
}
return db
}
// exec runs one statement against the database the binary uses.
func (e *env) exec(stmt string) {
e.t.Helper()
db := e.open()
defer db.Close()
if _, err := db.Exec(stmt); err != nil {
e.t.Fatalf("%s: %v", stmt, err)
}
}
func (e *env) count(query string, args ...any) int {
e.t.Helper()
db := e.open()
defer db.Close()
var n int
if err := db.QueryRow(query, args...).Scan(&n); err != nil {
e.t.Fatalf("%s: %v", query, err)
}
return n
}
// seeded is what installing this application writes, with the counts a
// finished install leaves behind. An uninstall wants every one of them at
// zero, and a reinstall wants them back - which is why one list serves all
// three checks instead of three lists drifting apart.
var seeded = []struct {
what string
query string
want int
}{
{"menus", "SELECT COUNT(*) FROM sys_menu WHERE app_code = 'order'", 4},
{"apis", "SELECT COUNT(*) FROM sys_api WHERE app_code = 'order'", 4},
{"ledger", "SELECT COUNT(*) FROM sys_app_casbin_grant WHERE app_code = 'order'", 4},
{"policies", "SELECT COUNT(*) FROM casbin_rule WHERE v1 LIKE '/api/v1/order%'", 4},
{"migration records", "SELECT COUNT(*) FROM sys_migration WHERE app_code = 'order'", 1},
{"sys_app rows", "SELECT COUNT(*) FROM sys_app WHERE app_code = 'order'", 1},
}
// assertSeeded checks every row of seeded. gone flips the expectation to
// zero, which is the whole of what an uninstall has to leave.
func (e *env) assertSeeded(when string, gone bool) {
e.t.Helper()
for _, c := range seeded {
want := c.want
if gone {
want = 0
}
if n := e.count(c.query); n != want {
e.t.Errorf("%s, %s = %d, want %d", when, c.what, n, want)
}
}
}
func TestInstallUninstallReinstall(t *testing.T) {
e := newEnv(t)
// Framework only. The application's migration must not run here, or the
// install below has nothing left to do and the interesting half of it
// goes untested - which is what happens if this uses plain `migrate`.
e.mustRun("migrate", "--app", "core")
if n := e.count("SELECT COUNT(*) FROM sys_migration WHERE app_code = 'order'"); n != 0 {
t.Fatalf("the application's migration ran during the framework's: %d rows", n)
}
if n := e.count("SELECT COUNT(*) FROM sqlite_master WHERE name = 'app_order'"); n != 0 {
t.Fatal("the application's own table exists before it was installed")
}
// A1.
out := e.mustRun("migrate", "install", "order")
if !strings.Contains(out, "order-1793800000000") {
t.Errorf("the install did not report applying the migration:\n%s", out)
}
// A7.
if !strings.Contains(out, "rebuild") {
t.Errorf("the install did not say the code is not running yet:\n%s", out)
}
e.assertSeeded("after install", false)
if n := e.count("SELECT COUNT(*) FROM sqlite_master WHERE name = 'app_order'"); n != 1 {
t.Error("the application's own table was not created")
}
if n := e.count("SELECT COUNT(*) FROM sys_app WHERE app_code = 'order' AND status = 2"); n != 1 {
t.Error("sys_app does not say the install finished")
}
// A2: installing the same version again does nothing and says so.
out = e.mustRun("migrate", "install", "order")
if !strings.Contains(out, "already installed") {
t.Errorf("a second install was not reported as a no-op:\n%s", out)
}
// A3: the application's own data is not the uninstaller's to remove.
e.exec("INSERT INTO app_order (created_at, updated_at) VALUES (datetime('now'), datetime('now'))")
if n := e.count("SELECT COUNT(*) FROM app_order"); n != 1 {
t.Fatalf("the business row was not written: %d", n)
}
out = e.mustRun("migrate", "uninstall", "order")
if !strings.Contains(out, "own tables were not touched") {
t.Errorf("the uninstall did not say what it left alone:\n%s", out)
}
e.assertSeeded("after uninstall", true)
if n := e.count("SELECT COUNT(*) FROM sqlite_master WHERE name = 'app_order'"); n != 1 {
t.Error("the uninstall dropped the application's own table")
}
if n := e.count("SELECT COUNT(*) FROM app_order"); n != 1 {
t.Errorf("the uninstall removed %d business row(s)", 1-n)
}
// A4: the migration records had to go, or this reinstall finds every
// version applied, seeds nothing, and reports success.
e.mustRun("migrate", "install", "order")
e.assertSeeded("after reinstall", false)
if n := e.count("SELECT COUNT(*) FROM app_order"); n != 1 {
t.Error("the business row did not survive an uninstall and reinstall")
}
}
// A deployment decides whether to start the new version on this exit code.
func TestMigrateExitsNonZeroWhenItFails(t *testing.T) {
e := newEnv(t)
// The framework's first migration reads config/db.sql. Without it the
// migration fails, which is the cheapest real failure to arrange.
if err := os.Remove(filepath.Join(e.dir, "config", "db.sql")); err != nil {
t.Fatal(err)
}
out, code := e.run("migrate")
if code == 0 {
t.Errorf("a failed migration exited 0:\n%s", out)
}
}
func TestUnknownAppIsRefused(t *testing.T) {
e := newEnv(t)
e.mustRun("migrate", "--app", "core")
out, code := e.run("migrate", "install", "ordr")
if code == 0 {
t.Errorf("a mistyped code was installed:\n%s", out)
}
if !strings.Contains(out, "order") {
t.Errorf("the refusal does not name what is registered:\n%s", out)
}
}
+156
View File
@@ -0,0 +1,156 @@
module go-admin-e2e-apporder
go 1.26.5
require (
github.com/glebarez/go-sqlite v1.22.0
github.com/go-admin-team/example-app-order v0.0.0
go-admin v0.0.0
)
require (
dario.cat/mergo v1.0.2 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
github.com/BurntSushi/toml v1.5.0 // indirect
github.com/KyleBanks/depth v1.2.1 // indirect
github.com/alex-ant/gomath v0.0.0-20160516115720-89013a210a82 // indirect
github.com/alibaba/sentinel-golang v1.0.4 // indirect
github.com/alibaba/sentinel-golang/pkg/adapters/gin v0.0.0-20241224061304-f4c2c5964666 // indirect
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible // indirect
github.com/andeya/ameda v1.5.3 // indirect
github.com/andeya/goutil v1.1.2 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bitly/go-simplejson v0.5.1 // indirect
github.com/bmatcuk/doublestar/v4 v4.10.0 // indirect
github.com/bytedance/go-tagexpr/v2 v2.9.11 // indirect
github.com/bytedance/gopkg v0.1.4 // indirect
github.com/bytedance/sonic v1.15.2 // indirect
github.com/bytedance/sonic/loader v0.5.2 // indirect
github.com/casbin/casbin/v3 v3.8.1 // indirect
github.com/casbin/gorm-adapter/v3 v3.41.0 // indirect
github.com/casbin/govaluate v1.10.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/chanxuehong/rand v0.0.0-20211009035549-2f07823e8e99 // indirect
github.com/chanxuehong/wechat v0.0.0-20230222024006-36f0325263cd // indirect
github.com/cloudwego/base64x v0.1.7 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.15 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/gin-contrib/sse v1.1.1 // indirect
github.com/gin-gonic/gin v1.12.0 // indirect
github.com/glebarez/sqlite v1.11.0 // indirect
github.com/go-admin-team/go-admin-core/v2 v2.8.0 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/go-openapi/jsonpointer v1.0.0 // indirect
github.com/go-openapi/jsonreference v1.0.0 // indirect
github.com/go-openapi/spec v0.22.9 // indirect
github.com/go-openapi/swag/conv v0.28.0 // indirect
github.com/go-openapi/swag/jsonutils v0.28.0 // indirect
github.com/go-openapi/swag/loading v0.28.0 // indirect
github.com/go-openapi/swag/pools v0.28.0 // indirect
github.com/go-openapi/swag/stringutils v0.28.0 // indirect
github.com/go-openapi/swag/typeutils v0.28.0 // indirect
github.com/go-openapi/swag/yamlutils v0.28.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.30.3 // indirect
github.com/go-sql-driver/mysql v1.10.0 // indirect
github.com/goccy/go-json v0.10.6 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 // indirect
github.com/golang-sql/sqlexp v0.1.0 // indirect
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/pgx/v5 v5.10.0 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.4.0 // indirect
github.com/leodido/go-urn v1.5.0 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/mattn/go-sqlite3 v1.14.49 // indirect
github.com/microsoft/go-mssqldb v1.10.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/mojocn/base64Captcha v1.3.8 // indirect
github.com/mssola/user_agent v0.6.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/nyaruka/phonenumbers v1.2.2 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/pelletier/go-toml/v2 v2.4.3 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/prometheus/client_golang v1.24.1 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.70.1 // indirect
github.com/prometheus/procfs v0.21.1 // indirect
github.com/qiniu/go-sdk/v7 v7.27.0 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/quic-go/quic-go v0.61.0 // indirect
github.com/redis/go-redis/v9 v9.22.0 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
github.com/shamsher31/goimgext v1.0.0 // indirect
github.com/shirou/gopsutil/v3 v3.24.5 // indirect
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/spf13/cobra v1.10.2 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/swaggo/files v1.0.1 // indirect
github.com/swaggo/gin-swagger v1.6.1 // indirect
github.com/swaggo/swag v1.16.6 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.3.2 // indirect
github.com/unrolled/secure v1.17.0 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.mongodb.org/mongo-driver/v2 v2.8.0 // indirect
go.uber.org/atomic v1.11.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.1 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/arch v0.30.0 // indirect
golang.org/x/crypto v0.54.0 // indirect
golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93 // indirect
golang.org/x/image v0.41.0 // indirect
golang.org/x/mod v0.38.0 // indirect
golang.org/x/net v0.57.0 // indirect
golang.org/x/sync v0.22.0 // indirect
golang.org/x/sys v0.47.0 // indirect
golang.org/x/text v0.40.0 // indirect
golang.org/x/time v0.15.0 // indirect
golang.org/x/tools v0.48.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gorm.io/driver/mysql v1.6.0 // indirect
gorm.io/driver/postgres v1.6.2 // indirect
gorm.io/driver/sqlite v1.6.0 // indirect
gorm.io/driver/sqlserver v1.6.4 // indirect
gorm.io/gorm v1.31.2 // indirect
gorm.io/plugin/dbresolver v1.6.2 // indirect
gorm.io/plugin/soft_delete v1.2.1 // indirect
modernc.org/fileutil v1.3.40 // indirect
modernc.org/libc v1.67.4 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
modernc.org/sqlite v1.42.2 // indirect
)
replace go-admin => ../..
replace github.com/go-admin-team/example-app-order => ../../example/app-order
+883
View File
@@ -0,0 +1,883 @@
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.1 h1:jHb/wfvRikGdxMXYV3QG/SzUOPYN9KEUUuC0Yd0/vC0=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.1/go.mod h1:pzBXCYn05zvYIrwLgtK8Ap8QcjRg+0i76tMQdWN6wOk=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1 h1:Hk5QBxZQC1jb2Fwj6mpzme37xbCDdNTxU7O9eb5+LB4=
github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.13.1/go.mod h1:IYus9qsFobWIc2YVwe/WPjcnyCkPKtnHAqUYeebc8z0=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.12.0 h1:fhqpLE3UEXi9lPaBRpQ6XuRW0nU7hgg4zlmZZa+a9q4=
github.com/Azure/azure-sdk-for-go/sdk/internal v1.12.0/go.mod h1:7dCRMLwisfRH3dBupKeNCioWYUZ4SS09Z14H+7i8ZoY=
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys v1.4.0 h1:E4MgwLBGeVB5f2MdcIVD3ELVAWpr+WD6MUe1i+tM/PA=
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/azkeys v1.4.0/go.mod h1:Y2b/1clN4zsAoUd/pgNAQHjLDnTis/6ROkUfyob6psM=
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.2.0 h1:nCYfgcSyHZXJI8J0IWE5MsCGlb2xp9fJiXyxWgmOFg4=
github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.2.0/go.mod h1:ucUjca2JtSZboY8IoUqyQyuuXvwbMBVwFOm0vdQPNhA=
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0 h1:XRzhVemXdgvJqCH0sFfrBUTnUJSBrBf7++ypk+twtRs=
github.com/AzureAD/microsoft-authentication-library-for-go v1.6.0/go.mod h1:HKpQxkWaGLJ+D/5H8QRpyQXA1eKjxkFlOMwck5+33Jk=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/Knetic/govaluate v3.0.1-0.20171022003610-9aa49832a739+incompatible/go.mod h1:r7JcOSlj0wfOMncg0iLm8Leh48TZaKVeNIfJntJ2wa0=
github.com/KyleBanks/depth v1.2.1 h1:5h8fQADFrWtarTdtDudMmGsC7GPbOAu6RVB3ffsVFHc=
github.com/KyleBanks/depth v1.2.1/go.mod h1:jzSb9d0L43HxTQfT+oSA1EEp2q+ne2uh6XgeJcm8brE=
github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo=
github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
github.com/StackExchange/wmi v0.0.0-20190523213315-cbe66965904d/go.mod h1:3eOhrUMpNV+6aFIbp5/iudMxNCF27Vw2OZgy4xEx0Fg=
github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g=
github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5/go.mod h1:SkGFH1ia65gfNATL8TAiHDNxPzPdmEL5uirI2Uyuz6c=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/alex-ant/gomath v0.0.0-20160516115720-89013a210a82 h1:7dONQ3WNZ1zy960TmkxJPuwoolZwL7xKtpcM04MBnt4=
github.com/alex-ant/gomath v0.0.0-20160516115720-89013a210a82/go.mod h1:nLnM0KdK1CmygvjpDUO6m1TjSsiQtL61juhNsvV/JVI=
github.com/alibaba/sentinel-golang v1.0.2/go.mod h1:QsB99f/z35D2AiMrAWwgWE85kDTkBUIkcmPrRt+61NI=
github.com/alibaba/sentinel-golang v1.0.4 h1:i0wtMvNVdy7vM4DdzYrlC4r/Mpk1OKUUBurKKkWhEo8=
github.com/alibaba/sentinel-golang v1.0.4/go.mod h1:Lag5rIYyJiPOylK8Kku2P+a23gdKMMqzQS7wTnjWEpk=
github.com/alibaba/sentinel-golang/pkg/adapters/gin v0.0.0-20241224061304-f4c2c5964666 h1:nLA94XbUtqArHLPtBZuPuIlOOS1rWnWs8ANnMj+TW4Y=
github.com/alibaba/sentinel-golang/pkg/adapters/gin v0.0.0-20241224061304-f4c2c5964666/go.mod h1:RWwrQy9bLKVZyr47l34uImkq7LQEc2ngkKn2O2LV1gE=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible h1:8psS8a+wKfiLt1iVDX79F7Y6wUM49Lcha2FMXt4UM8g=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
github.com/andeya/ameda v1.5.3 h1:SvqnhQPZwwabS8HQTRGfJwWPl2w9ZIPInHAw9aE1Wlk=
github.com/andeya/ameda v1.5.3/go.mod h1:FQDHRe1I995v6GG+8aJ7UIUToEmbdTJn/U26NCPIgXQ=
github.com/andeya/goutil v1.0.1/go.mod h1:jEG5/QnnhG7yGxwFUX6Q+JGMif7sjdHmmNVjn7nhJDo=
github.com/andeya/goutil v1.1.2 h1:RiFWFkL/9yXh2SjQkNWOHqErU1x+RauHmeR23eNUzSg=
github.com/andeya/goutil v1.1.2/go.mod h1:jEG5/QnnhG7yGxwFUX6Q+JGMif7sjdHmmNVjn7nhJDo=
github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/apache/thrift v0.13.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
github.com/armon/circbuf v0.0.0-20150827004946-bbbad097214e/go.mod h1:3U/XgcO3hCbHZ8TKRvWD2dDTCfh9M9ya+I9JpbB7O8o=
github.com/armon/go-metrics v0.0.0-20180917152333-f0300d1749da/go.mod h1:Q73ZrmVTwzkszR9V5SSuryQ31EELlFMUz1kKyl939pY=
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310/go.mod h1:ufUuZ+zHj4x4TnLV4JWEpy2hxWSpsRywHrMgIH9cCH8=
github.com/aryann/difflib v0.0.0-20170710044230-e206f873d14a/go.mod h1:DAHtR1m6lCRdSC2Tm3DSWRPvIPr6xNKyeHdqDQSQT+A=
github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU=
github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bgentry/speakeasy v0.1.0/go.mod h1:+zsyZBPWlz7T6j88CTgSN5bM796AkVf0kBD4zp0CCIs=
github.com/bitly/go-simplejson v0.5.1 h1:xgwPbetQScXt1gh9BmoJ6j9JMr3TElvuIyjR8pgdoow=
github.com/bitly/go-simplejson v0.5.1/go.mod h1:YOPVLzCfwK14b4Sff3oP1AmGhI9T9Vsg84etUnlyp+Q=
github.com/bmatcuk/doublestar/v4 v4.6.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bmatcuk/doublestar/v4 v4.10.0 h1:zU9WiOla1YA122oLM6i4EXvGW62DvKZVxIe6TYWexEs=
github.com/bmatcuk/doublestar/v4 v4.10.0/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/bytedance/go-tagexpr/v2 v2.9.11 h1:jJgmoDKPKacGl0llPYbYL/+/2N+Ng0vV0ipbnVssXHY=
github.com/bytedance/go-tagexpr/v2 v2.9.11/go.mod h1:UAyKh4ZRLBPGsyTRFZoPqTni1TlojMdOJXQnEIPCX84=
github.com/bytedance/gopkg v0.1.4 h1:oZnQwnX82KAIWb7033bEwtxvTqXcYMxDBaQxo5JJHWM=
github.com/bytedance/gopkg v0.1.4/go.mod h1:v1zWfPm21Fb+OsyXN2VAHdL6TBb2L88anLQgdyje6R4=
github.com/bytedance/sonic v1.15.2 h1:90H+rcF/FwLXwfB1cudOLq/je83n683Utf4Cbp0xHCo=
github.com/bytedance/sonic v1.15.2/go.mod h1:mT2NbXunuaEbnZ+mRIX/vYqKISmgEuHFDI4UzmKx2SA=
github.com/bytedance/sonic/loader v0.5.2 h1:0QtP1gevc1OZ6/H8Lb9BRZiCXd1Ftjd3OKuj1T1lBIo=
github.com/bytedance/sonic/loader v0.5.2/go.mod h1:AR4NYCk5DdzZizZ5djGqQ92eEhCCcdf5x77udYiSJRo=
github.com/casbin/casbin/v2 v2.1.2/go.mod h1:YcPU1XXisHhLzuxH9coDNf2FbKpjGlbCg3n9yuLkIJQ=
github.com/casbin/casbin/v3 v3.8.1 h1:D4dEY4knePPR4YgNP5WZtWNaOxD0UK0LpPy9+zxtBwo=
github.com/casbin/casbin/v3 v3.8.1/go.mod h1:5rJbQr2e6AuuDDNxnPc5lQlC9nIgg6nS1zYwKXhpHC8=
github.com/casbin/gorm-adapter/v3 v3.41.0 h1:Xhpi0tfRP9aKPDWDf6dgBxHZ9UM6IophxxPIEGWqCNM=
github.com/casbin/gorm-adapter/v3 v3.41.0/go.mod h1:BQZRJhwUnwMpI+pT2m7/cUJwXxrHfzpBpPcNTyMGeGA=
github.com/casbin/govaluate v1.3.0/go.mod h1:G/UnbIjZk/0uMNaLwZZmFQrR72tYRZWQkO70si/iR7A=
github.com/casbin/govaluate v1.10.0 h1:ffGw51/hYH3w3rZcxO/KcaUIDOLP84w7nsidMVgaDG0=
github.com/casbin/govaluate v1.10.0/go.mod h1:G/UnbIjZk/0uMNaLwZZmFQrR72tYRZWQkO70si/iR7A=
github.com/cenkalti/backoff v2.2.1+incompatible/go.mod h1:90ReRw6GdpyfrHakVjL/QHaoyV4aDUVVkXQJJJ3NXXM=
github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/chanxuehong/rand v0.0.0-20211009035549-2f07823e8e99 h1:K62Lb6bsgLOB++z/VAvRvtiEBdNCuMfmQGTGGWMdPpM=
github.com/chanxuehong/rand v0.0.0-20211009035549-2f07823e8e99/go.mod h1:9+sJ9zvvkXC5sPjPEZM3Jpb9n2Q2VtcrGZly0UHYF5I=
github.com/chanxuehong/util v0.0.0-20200304121633-ca8141845b13/go.mod h1:XEYt99iTxMqkv+gW85JX/DdUINHUe43Sbe5AtqSaDAQ=
github.com/chanxuehong/wechat v0.0.0-20230222024006-36f0325263cd h1:v3JNsFZmplLO/Cmiyr/rGvR7lW1ld9lB+d5h4yR0MTI=
github.com/chanxuehong/wechat v0.0.0-20230222024006-36f0325263cd/go.mod h1:mysjrtCs9MmN8hqDf4/mc4eQ26Rt9s1p5oO+fhJlLB4=
github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec/go.mod h1:jMjuTZXRI4dUb/I5gc9Hdhagfvm9+RyrPryS/auMzxE=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/cloudwego/base64x v0.1.7 h1:NppS+Fgzg5ovhn4NkUXaDT3x9jldgH5ToMCqzBSi2zI=
github.com/cloudwego/base64x v0.1.7/go.mod h1:Cu1PV9zfrSf7ET2tIbWbbEy7jO7HHJ13q4X2SQ8aWYg=
github.com/cockroachdb/datadriven v0.0.0-20190809214429-80d97fb3cbaa/go.mod h1:zn76sxSg3SzpJ0PPJaLDCu+Bu0Lg3sKTORVIj19EIF8=
github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20180511133405-39ca1b05acc7/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/pkg v0.0.0-20160727233714-3ac0863d7acf/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs=
github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1:+020luEh2TKB4/GOp8oxxtq0Daoen/Cii55CzbTV6DU=
github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
github.com/edsrzf/mmap-go v1.0.0/go.mod h1:YO35OhQPt3KJa3ryjFM5Bs14WD66h8eGKpfaBNrHW5M=
github.com/envoyproxy/go-control-plane v0.6.9/go.mod h1:SBwIajubJHhxtWwsL9s8ss4safvEdbitLhGGK48rN6g=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4=
github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/gabriel-vasile/mimetype v1.4.15 h1:05iP/CYtZ/w455R/KZM6rZ5ieAdh99UPtd+d3YzLmaI=
github.com/gabriel-vasile/mimetype v1.4.15/go.mod h1:azpTcoLcDZRNgFou5j+APrqQx9HqVPWa6ijYQIIVswQ=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gin-contrib/gzip v0.0.6 h1:NjcunTcGAj5CO1gn4N8jHOSIeRFHIbn51z6K+xaN4d4=
github.com/gin-contrib/gzip v0.0.6/go.mod h1:QOJlmV2xmayAjkNS2Y8NQsMneuRShOU/kjovCXNuzzk=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
github.com/gin-contrib/sse v1.1.1 h1:uGYpNwTacv5R68bSGMapo62iLTRa9l5zxGCps4hK6ko=
github.com/gin-contrib/sse v1.1.1/go.mod h1:QXzuVkA0YO7o/gun03UI1Q+FTI8ZV/n5t03kIQAI89s=
github.com/gin-gonic/gin v1.7.0/go.mod h1:jD2toBW3GZUr5UMcdrwQA10I7RuaFOl/SGeDjXkfUtY=
github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8=
github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc=
github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec+ruQ=
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-admin-team/go-admin-core/v2 v2.8.0 h1:ZTw5Z/UT1/7OltbGPEaEVerRk4z3koB6O8nDbb84tPM=
github.com/go-admin-team/go-admin-core/v2 v2.8.0/go.mod h1:LG/XvEfOplbuadKrPTPm0Nu5pN06aQUNZZC3ao4B4gs=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
github.com/go-ole/go-ole v1.2.4/go.mod h1:XCwSNxSkXRo4vlyPy93sltvi/qJq0jqQhjqQNIwKuxM=
github.com/go-ole/go-ole v1.2.5/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/go-ole/go-ole v1.2.6 h1:/Fpf6oFPoeFik9ty7siob0G6Ke8QvQEuVcuChpwXzpY=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/go-openapi/jsonpointer v1.0.0 h1:kR9tHqY0CtZaOPVFm622dPVNhrvYpwr4uCxgL3h1H8s=
github.com/go-openapi/jsonpointer v1.0.0/go.mod h1:Z3rw7dWu1p9IgitXCFamSlA5lmDiklEB6vkaxcNZW5Y=
github.com/go-openapi/jsonreference v1.0.0 h1:jlmTr6torcd1YgDQvSfNmRtKzYDO4FGBkrAdlAVWnpY=
github.com/go-openapi/jsonreference v1.0.0/go.mod h1:jtwdyGbJk0Xhe5Y+rwtglQP6Sb1WZST4rT32LWB+sv0=
github.com/go-openapi/spec v0.22.9 h1:/vKIFDcGKp0ktZWGbym/tJEWbk6/XOEmAVU0kqKMH+w=
github.com/go-openapi/spec v0.22.9/go.mod h1:b/mNUYIOQOyIiUzUzXEE8xzyZqf93KvM9hQGP91yfl0=
github.com/go-openapi/swag v0.19.15 h1:D2NRCBzS9/pEY3gP9Nl8aDqGUcPFrwG2p+CNFrLyrCM=
github.com/go-openapi/swag/conv v0.28.0 h1:GtqqbyFe7vR5Y7ehxG9W6/OvrSFdf1OLeTGp40TqxH8=
github.com/go-openapi/swag/conv v0.28.0/go.mod h1:mbUE+mzctnhxi864m0Q07SpN8OowD9JhxmxuYvZZD/k=
github.com/go-openapi/swag/jsonutils v0.28.0 h1:YIch6FwO7RXzeAnbO8Tu7dWBZeUEH+4nA0HXltVTnv4=
github.com/go-openapi/swag/jsonutils v0.28.0/go.mod h1:CYM3WlTUcagR2ZoHdz54di/cbBqt82tuxuXgAjxw+mg=
github.com/go-openapi/swag/jsonutils/fixtures_test v0.28.0 h1:qV+VVUAx5Oro8WjVWpZeql7YReTKhT4smR4zhcOQZr0=
github.com/go-openapi/swag/jsonutils/fixtures_test v0.28.0/go.mod h1:mofwUWx70wvskwESqRJ//k/9kURmCgyJl5m5Ppoh5kY=
github.com/go-openapi/swag/loading v0.28.0 h1:td8QZdZC9MIYGGSnSPKShKiK22I2tU5UQvuUhIBPRLU=
github.com/go-openapi/swag/loading v0.28.0/go.mod h1:rXB0QiQX5mMveXEA7ouM4KiiM9jVJe4K6BVbwhD1M4k=
github.com/go-openapi/swag/pools v0.28.0 h1:HPMZWSAfce3rdVTFcjFiCIBtDg9h4x2QlRrHipwhxeU=
github.com/go-openapi/swag/pools v0.28.0/go.mod h1:kVQefhSK5RWuRe7BXsL8htgBPAMpN7HDGpGEknqugeE=
github.com/go-openapi/swag/stringutils v0.28.0 h1:ixsc9iYgDPubHL/8nSkbnryEHpD2VRlBMLKpQyPXcDU=
github.com/go-openapi/swag/stringutils v0.28.0/go.mod h1:lzRN95CxXmA03XcDWHLOb6nOMcxCqR5rGY0lOgsfRoM=
github.com/go-openapi/swag/typeutils v0.28.0 h1:nRBKSBXjDgf01VDPB3fWeD9nQuhCOVeIYAkUx2tbkyY=
github.com/go-openapi/swag/typeutils v0.28.0/go.mod h1:Srm0xFNRZ1Y+vCxJclo5qzx8aj+1pAKda/YfFPrG0dQ=
github.com/go-openapi/swag/yamlutils v0.28.0 h1:TV3JXH6DS46KUroDtMLAYHGkdWf5VDq3wVWFirmzROY=
github.com/go-openapi/swag/yamlutils v0.28.0/go.mod h1:x0q/yndZHEgk9Rx3DyDqzFUmHy55KTvIZldvF2dTJXs=
github.com/go-openapi/testify/enable/yaml/v2 v2.6.0 h1:gGHwAJ0R/5jU8BEGDbfRNR3hL68dAVi84WuOApp29B0=
github.com/go-openapi/testify/enable/yaml/v2 v2.6.0/go.mod h1:tY+St1SGq4NFl0QIqdTY4aEdbChAHxhyB77XQi9iJCo=
github.com/go-openapi/testify/v2 v2.6.0 h1:5PKH2HE7YJ/LuRPQGvSxBRlFXNQhSetBLlGAgUEu3ug=
github.com/go-openapi/testify/v2 v2.6.0/go.mod h1:SgsVHtfooshd0tublTtJ50FPKhujf47YRqauXXOUxfw=
github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
github.com/go-playground/validator/v10 v10.30.3 h1:4MU6YkEwx7GbcPJOZxrtbu+QfF3pJLJuaYTeAH0DYy8=
github.com/go-playground/validator/v10 v10.30.3/go.mod h1:4Axh7oCNGcoGkqLoE4YWt6n20mcEIsPRlB7vPk3lpyc=
github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
github.com/go-sql-driver/mysql v1.10.0 h1:Q+1LV8DkHJvSYAdR83XzuhDaTykuDx0l6fkXxoWCWfw=
github.com/go-sql-driver/mysql v1.10.0/go.mod h1:M+cqaI7+xxXGG9swrdeUIoPG3Y3KCkF0pZej+SK+nWk=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/goccy/go-json v0.10.6 h1:p8HrPJzOakx/mn/bQtjgNjdTcN+/S6FcG2CTtQOrHVU=
github.com/goccy/go-json v0.10.6/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
github.com/goccy/go-yaml v1.19.2/go.mod h1:XBurs7gK8ATbW4ZPGKgcbrY1Br56PdM69F7LkFRi1kA=
github.com/gofrs/flock v0.8.1 h1:+gYjHKf32LDeiEEFhQaotPbLuUXjY5ZqxKgXy7n59aw=
github.com/gofrs/flock v0.8.1/go.mod h1:F1TvTiK9OcQqauNUHlbJvyl9Qa1QvF/gOUDKA14jxHU=
github.com/gogo/googleapis v1.1.0/go.mod h1:gf4bu3Q80BeJ6H1S1vYPm8/ELATdvryBaNFGgqEef3s=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9 h1:au07oEsX2xN0ktxqI+Sida1w446QrXBRJ0nee3SNZlA=
github.com/golang-sql/civil v0.0.0-20220223132316-b832511892a9/go.mod h1:8vg3r2VgvsThLBIFL93Qb5yWzgyZWhEmBwUJWevAkK0=
github.com/golang-sql/sqlexp v0.1.0 h1:ZCD6MBpcuOVfGVqsEmY5/4FtYiKz6tSyUv9LPEDei6A=
github.com/golang-sql/sqlexp v0.1.0/go.mod h1:J4ad9Vo8ZCWQ2GMrC4UCQy1JpCbwU9m3EOqtpKwwwHI=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0 h1:DACJavvAHhabrF08vX0COfcOBJRhZ8lUbR+ZWIs0Y5g=
github.com/golang/freetype v0.0.0-20170609003504-e2365dfdc4a0/go.mod h1:E/TSTwGwJL78qG/PmXZO1EjYhfJinVAhrmmHX6Z8B9k=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20160516000752-02826c3e7903/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/groupcache v0.0.0-20190702054246-869f871628b6/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.6/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gopherjs/gopherjs v1.17.2 h1:fQnZVsXk8uxXIStYb0N4bGk7jeyTalG/wsZjQ25dO0g=
github.com/gopherjs/gopherjs v1.17.2/go.mod h1:pRRIvn/QzFLrKfvEz3qUuEhtE/zLCWfreZ6J5gM2i+k=
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.8.1 h1:TuBL49tXwgrFYWhqrNgrUNEY92u81SPhu7sTdzQEiWY=
github.com/gorilla/mux v1.8.1/go.mod h1:AKf9I4AEqPTmMytcMc0KkNouC66V3BtZ4qD5fmWSiMQ=
github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/go-grpc-middleware v1.0.1-0.20190118093823-f849b5445de4/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
github.com/grpc-ecosystem/grpc-gateway v1.9.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/hashicorp/consul/api v1.3.0/go.mod h1:MmDNSzIMUjNpY/mQ398R4bk2FnqQLoPndWW5VkKPlCE=
github.com/hashicorp/consul/sdk v0.3.0/go.mod h1:VKf9jXwCTEY1QZP2MOLRhb5i/I/ssyNV1vwHyQBF0x8=
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
github.com/hashicorp/go-cleanhttp v0.5.1/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60=
github.com/hashicorp/go-msgpack v0.5.3/go.mod h1:ahLV/dePpqEmjfWmKiqvPkv/twdG7iPBM1vqhUKIvfM=
github.com/hashicorp/go-multierror v1.0.0/go.mod h1:dHtQlpGsu+cZNNAkkCN/P3hoUDHhCYQXV3UM06sGGrk=
github.com/hashicorp/go-rootcerts v1.0.0/go.mod h1:K6zTfqpRlCUIjkwsN4Z+hiSfzSTQa6eBIzfwKfwNnHU=
github.com/hashicorp/go-sockaddr v1.0.0/go.mod h1:7Xibr9yA9JjQq1JpNB2Vw7kxv8xerXegt+ozgdvDeDU=
github.com/hashicorp/go-syslog v1.0.0/go.mod h1:qPfqrKkXGihmCqbJM2mZgkZGvKG1dFdvsLplgctolz4=
github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-uuid v1.0.1/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
github.com/hashicorp/go-version v1.2.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/hashicorp/go.net v0.0.1/go.mod h1:hjKkEWcCURg++eb33jQU7oqQcI9XDCnUzHA0oac0k90=
github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru v0.5.1 h1:0hERBMJE1eitiLkihrMvRVBYAkpHzc/J3QdDN+dAcgU=
github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
github.com/hashicorp/mdns v1.0.0/go.mod h1:tL+uN++7HEJ6SQLQ2/p+z2pH24WQKWjBPkE0mNTz8vQ=
github.com/hashicorp/memberlist v0.1.3/go.mod h1:ajVTdAv/9Im8oMAAj5G31PhhMCZJV2pPBoIllUwCN7I=
github.com/hashicorp/serf v0.8.2/go.mod h1:6hOLApaqBFA1NXqRQAsxw9QxuDEvNxSQRwA/JwenrHc=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible h1:lX3m9hvP5tSnJ8bFg/TdT2BYHj1nSBulealy5VN9mPU=
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible/go.mod h1:l7VUhRbTKCzdOacdT4oWCwATKyvZqUOlOqr0Ous3k4s=
github.com/hudl/fargo v1.3.0/go.mod h1:y3CKSmjA+wD2gak7sUSXTAoopbhU08POFhmITJgmKTg=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/influxdata/influxdb1-client v0.0.0-20191209144304-8bf82d3c094d/go.mod h1:qj24IKcXYK6Iy9ceXlo3Tc+vtHo9lIhSX5JddghvEPo=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 h1:iCEnooe7UlwOQYpKFhBabPMi4aNAfoODPEFNiAnClxo=
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.10.0 h1:VhSvgU2jSli8o3AqIEOTJr7rZwAEUVo4E4XhR94Zfr0=
github.com/jackc/pgx/v5 v5.10.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD/E=
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
github.com/jinzhu/now v1.1.1/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/jinzhu/now v1.1.4/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.8/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
github.com/leodido/go-urn v1.5.0 h1:pLqT2kq1zpHW/1D18QMjMpdtX7cekxqtJJjg5ANyWw0=
github.com/leodido/go-urn v1.5.0/go.mod h1:9BORnCDhdPBJNDEX+w1bJisa8yOKYi116VeO96s4ifE=
github.com/lib/pq v1.10.2 h1:AqzbZs4ZoCBp+GtejcpCpcxM3zlSMx29dXbUSeVtJb8=
github.com/lib/pq v1.10.2/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-bc2310a04743/go.mod h1:qklhhLq1aX+mtWk9cPHPzaBjWImj5ULL6C7HFJtXQMM=
github.com/lightstep/lightstep-tracer-go v0.18.1/go.mod h1:jlF1pusYV4pidLvZ+XD0UBX0ZE6WURAspgAczcDHrL4=
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 h1:6E+4a0GO5zZEnZ81pIr0yLvtUWk2if982qA3F3QD6H4=
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I=
github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-sqlite3 v1.14.3/go.mod h1:WVKg1VTActs4Qso6iwGbiFih2UIHo0ENGwNd0Lj+XmI=
github.com/mattn/go-sqlite3 v1.14.49 h1:B8jBHC3xhxZgxztrgruTuLucebnULQnx4W7cF7SAE9w=
github.com/mattn/go-sqlite3 v1.14.49/go.mod h1:6JTjA44L93a0QCyJef5YvlPoKXntQPjzWv5gtm9sB6w=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/microsoft/go-mssqldb v1.10.0 h1:pHEt+Qz6YFPWqREq10mqSE524QQo+/QremwTCQht7TY=
github.com/microsoft/go-mssqldb v1.10.0/go.mod h1:mnG7lGa9iYJbzJqGCXyuQCegStKMr3kogDLD6+bmggg=
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
github.com/mitchellh/go-homedir v1.0.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/go-testing-interface v1.0.0/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
github.com/mitchellh/gox v0.4.0/go.mod h1:Sd9lOJ0+aimLBi73mGofS1ycjY8lL3uZM3JPS42BGNg=
github.com/mitchellh/iochan v1.0.0/go.mod h1:JwYml1nuB7xOzsp52dPpHFffvOCDupsG0QubkSMEySY=
github.com/mitchellh/mapstructure v0.0.0-20160808181253-ca63d7c062ee/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/mojocn/base64Captcha v1.3.8 h1:rrN9BhCwXKS8ht1e21kvR3iTaMgf4qPC9sRoV52bqEg=
github.com/mojocn/base64Captcha v1.3.8/go.mod h1:QFZy927L8HVP3+VV5z2b1EAEiv1KxVJKZbAucVgLUy4=
github.com/mssola/user_agent v0.6.0 h1:uwPR4rtWlCHRFyyP9u2KOV0u8iQXmS7Z7feTrstQwk4=
github.com/mssola/user_agent v0.6.0/go.mod h1:TTPno8LPY3wAIEKRpAtkdMT0f8SE24pLRGPahjCH4uw=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/nats-io/jwt v0.3.0/go.mod h1:fRYCDE99xlTsqUzISS1Bi75UBJ6ljOJQOAAu5VglpSg=
github.com/nats-io/jwt v0.3.2/go.mod h1:/euKqTS1ZD+zzjYrY7pseZrTtWQSjujC7xjPc8wL6eU=
github.com/nats-io/nats-server/v2 v2.1.2/go.mod h1:Afk+wRZqkMQs/p45uXdrVLuab3gwv3Z8C4HTBu8GD/k=
github.com/nats-io/nats.go v1.9.1/go.mod h1:ZjDU1L/7fJ09jvUSRVBR2e7+RnLiiIQyqyzEE/Zbp4w=
github.com/nats-io/nkeys v0.1.0/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
github.com/nats-io/nkeys v0.1.3/go.mod h1:xpnFELMwJABBLVhffcfd1MZx6VsNRFpEugbxziKVo7w=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/nyaruka/phonenumbers v1.0.55/go.mod h1:sDaTZ/KPX5f8qyV9qN+hIm+4ZBARJrupC6LuhshJq1U=
github.com/nyaruka/phonenumbers v1.2.2 h1:OwVjf7Y4uHoK9VJUrA8ebR0ha2yc6sEYbfrwkq0asCY=
github.com/nyaruka/phonenumbers v1.2.2/go.mod h1:wzk2qq7qwsaBKrfbkWKdgHYOOH+QFTesSpIq53ELw8M=
github.com/oklog/oklog v0.3.2/go.mod h1:FCV+B7mhrz4o+ueLpx+KqkyXRGMWOYEvfiXtdGtbWGs=
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
github.com/olekukonko/tablewriter v0.0.0-20170122224234-a0225b3f23b5/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492/go.mod h1:Ngi6UdF0k5OKD5t5wlmGhe/EDKPoUM3BXZSSfIuJbis=
github.com/opentracing/basictracer-go v1.0.0/go.mod h1:QfBfYuafItcjQuMwinw9GhYKwFXS9KnPs5lxoYwgW74=
github.com/opentracing/opentracing-go v1.0.2/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/opentracing/opentracing-go v1.1.0/go.mod h1:UkNAQd3GIcIGf0SeVgPpRdFStlNbqXla1AfSYxPUl2o=
github.com/opentracing/opentracing-go v1.2.0 h1:uEJPy/1a5RIPAJ0Ov+OIO8OxWu77jEv+1B0VhjKrZUs=
github.com/opentracing/opentracing-go v1.2.0/go.mod h1:GxEUsuufX4nBwe+T+Wl9TAgYrxe9dPLANfrWvHYVTgc=
github.com/openzipkin-contrib/zipkin-go-opentracing v0.4.5/go.mod h1:/wsWhb9smxSfWAKL3wpBW7V8scJMt8N8gnaMCS9E/cA=
github.com/openzipkin/zipkin-go v0.1.6/go.mod h1:QgAqvLzwWbR/WpD4A3cGpPtJrZXNIiJc5AZX7/PBEpw=
github.com/openzipkin/zipkin-go v0.2.1/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4=
github.com/openzipkin/zipkin-go v0.2.2/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4=
github.com/pact-foundation/pact-go v1.0.4/go.mod h1:uExwJY4kCzNPcHRj+hCR/HBbOOIwwtUjcrb0b5/5kLM=
github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/pelletier/go-toml/v2 v2.4.3 h1:GTRvJQutkOSftxIFD5xw9aepkYNuPWmVJpffdDPYVpY=
github.com/pelletier/go-toml/v2 v2.4.3/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac=
github.com/pierrec/lz4 v1.0.2-0.20190131084431-473cd7ce01a1/go.mod h1:3/3N9NVKO0jef7pBehbT1qWhCMrIgbYNnFAZCqQ5LRc=
github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c h1:+mdjkGKdHQG3305AYmdv1U2eRNDiU2ErMBj1gwrq8eQ=
github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c/go.mod h1:7rwL4CYBLnjLxUqIJNnCWiEdr3bn6IUYi15bNlnbCCU=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/profile v1.2.1/go.mod h1:hJw3o1OdXxsrSjjVksARp5W95eeEaEfptyVZyv6JUPA=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c h1:ncq/mPwQF4JjgDlrVEn3C11VoGHZN7m8qihwgMEtzYw=
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.3.0/go.mod h1:hJaj2vgQTGQmVCsAACORcieXFeDPbaTKGT+JTgUa3og=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
github.com/prometheus/client_golang v1.9.0/go.mod h1:FqZLKOZnGdFAhOK4nqGHa7D66IdsO+O441Eve7ptJDU=
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.1.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.7.0/go.mod h1:DjGbpBbp5NYNiECxcL/VnbXCCaQpKd3tt26CguLLsqA=
github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
github.com/prometheus/common v0.15.0/go.mod h1:U+gB1OBLb1lF3O42bTCL+FK18tX9Oar16Clt/msog/s=
github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi/PY=
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A=
github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/qiniu/go-sdk/v7 v7.27.0 h1:n+2U0S5fhbmG/lN/agO8KcYJaQDqROUVShtnp56Mkw8=
github.com/qiniu/go-sdk/v7 v7.27.0/go.mod h1:pTwVR1B+8SXcPLhDzBUasiKFTD9F7jRglRDR553BW3k=
github.com/quic-go/go-ossfuzz-seeds v0.1.0 h1:APacT+iIaNF6fd8AGEiN3bT/Jtkd2jz4v4TzM7MFjy0=
github.com/quic-go/go-ossfuzz-seeds v0.1.0/go.mod h1:3IOHRbJIc+L6YKMwfDtJAM9Vj9k0YY4muhuyUYk5tbk=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.61.0 h1:ui88A53s8MSVYLC56en0KQ17HARk+9986Dn0SBfKNvA=
github.com/quic-go/quic-go v0.61.0/go.mod h1:9So2anK4Tp22URSQq00k+Vo2PNkle96ycDPDHL4s9vs=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/redis/go-redis/v9 v9.22.0 h1:laDvpYXTJtZLloinw1fA5Kqd6HAEH2XKxOkG/PDq2F0=
github.com/redis/go-redis/v9 v9.22.0/go.mod h1:y2g0Wj8rQvuK0ELM+oxSudcLtC09JScs98I/X9gRWY4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/ryanuber/columnize v0.0.0-20160712163229-9b3edd62028f/go.mod h1:sm1tb6uqfes/u+d4ooFouqFdy9/2g9QGwK3SQygK0Ts=
github.com/samuel/go-zookeeper v0.0.0-20190923202752-2cc03de413da/go.mod h1:gi+0XIa01GRL2eRQVjQkKGqKF3SF9vZR/HnPullcV2E=
github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc=
github.com/shamsher31/goimgext v1.0.0 h1:wFKf9GeeE0Xr6UQtliaPgYYgTju2izobM7XpCEgUCC8=
github.com/shamsher31/goimgext v1.0.0/go.mod h1:rYLKgXuTGBIaH49z+jUVSWz7gUWIZmqvYUsdvJbNNOc=
github.com/shirou/gopsutil v3.20.11+incompatible/go.mod h1:5b4v6he4MtMOwMlS0TUMTu2PcXUg8+E1lC7eC3UO/RA=
github.com/shirou/gopsutil/v3 v3.21.6/go.mod h1:JfVbDpIBLVzT8oKbvMg9P3wEIMDDpVn+LwHTKj0ST88=
github.com/shirou/gopsutil/v3 v3.24.5 h1:i0t8kL+kQTvpAYToeuiVk3TgDeKOFioZO3Ztz/iZ9pI=
github.com/shirou/gopsutil/v3 v3.24.5/go.mod h1:bsoOS1aStSs9ErQ1WWfxllSeS1K5D+U30r2NfcubMVk=
github.com/shoenig/go-m1cpu v0.1.6 h1:nxdKQNcEB6vzgA2E2bvzKIYRuNj7XNJ4S/aRSwKzFtM=
github.com/shoenig/go-m1cpu v0.1.6/go.mod h1:1JJMcUBvfNwpq05QDQVAnx3gUHr9IYF7GNg9SUEw2VQ=
github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU=
github.com/shoenig/test v0.6.4/go.mod h1:byHiCGXqrVaflBLAMq/srcZIHynQPQgeyvkvXnjqq0k=
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w=
github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g=
github.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY=
github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
github.com/smartystreets/goconvey v1.6.4/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=
github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=
github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
github.com/sony/gobreaker v0.4.1/go.mod h1:ZKptC7FHNvhBz7dN2LGjPVBz2sZJmc0/PkyDJOjmxWY=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.1/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/streadway/amqp v0.0.0-20190404075320-75d898a42a94/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=
github.com/streadway/amqp v0.0.0-20190827072141-edfb9018d271/go.mod h1:AZpEONHx3DKn8O/DFsRAY58/XVQiIPMTMB1SddzLXVw=
github.com/streadway/handy v0.0.0-20190108123426-d5acb3125c2a/go.mod h1:qNTQ5P5JnDBl6z3cMAg/SywNDC5ABu5ApDIw6lUbRmI=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.5/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/swaggo/files v1.0.1 h1:J1bVJ4XHZNq0I46UU90611i9/YzdrF7x92oX1ig5IdE=
github.com/swaggo/files v1.0.1/go.mod h1:0qXmMNH6sXNf+73t65aKeB+ApmgxdnkQzVTAj2uaMUg=
github.com/swaggo/gin-swagger v1.6.1 h1:Ri06G4gc9N4t4k8hekMigJ9zKTFSlqj/9paAQCQs7cY=
github.com/swaggo/gin-swagger v1.6.1/go.mod h1:LQ+hJStHakCWRiK/YNYtJOu4mR2FP+pxLnILT/qNiTw=
github.com/swaggo/swag v1.16.6 h1:qBNcx53ZaX+M5dxVyTrgQ0PJ/ACK+NzhwcbieTt+9yI=
github.com/swaggo/swag v1.16.6/go.mod h1:ngP2etMK5a0P3QBizic5MEwpRmluJZPHjXcMoj4Xesg=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tklauser/go-sysconf v0.3.6/go.mod h1:MkWzOF4RMCshBAMXuhXJs64Rte09mITnppBXY/rYEFI=
github.com/tklauser/go-sysconf v0.3.12 h1:0QaGUFOdQaIVdPgfITYzaTegZvdCjmYO52cSFAEVmqU=
github.com/tklauser/go-sysconf v0.3.12/go.mod h1:Ho14jnntGE1fpdOqQEEaiKRpvIavV0hSfmBq8nJbHYI=
github.com/tklauser/numcpus v0.2.2/go.mod h1:x3qojaO3uyYt0i56EW/VUYs7uBvdl2fkfZFu0T9wgjM=
github.com/tklauser/numcpus v0.6.1 h1:ng9scYS7az0Bk4OZLvrNXNSAO2Pxr1XXRAPyjhIx+Fk=
github.com/tklauser/numcpus v0.6.1/go.mod h1:1XfjsgE2zo8GVw7POkMbHENHzVg3GzmoZ9fESEdAacY=
github.com/tmc/grpc-websocket-proxy v0.0.0-20170815181823-89b8d40f7ca8/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw=
github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY=
github.com/ugorji/go/codec v1.3.2 h1:zkEASHHyEClGeURfgNT9PJZVfAbs9oEX9QXggwWNJbc=
github.com/ugorji/go/codec v1.3.2/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
github.com/unrolled/secure v1.17.0 h1:Io7ifFgo99Bnh0J7+Q+qcMzWM6kaDPCA5FroFZEdbWU=
github.com/unrolled/secure v1.17.0/go.mod h1:BmF5hyM6tXczk3MpQkFf1hpKSRqCyhqcbiQtiAF7+40=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
github.com/zeebo/xxh3 v1.1.0 h1:s7DLGDK45Dyfg7++yxI0khrfwq9661w9EN78eP/UZVs=
github.com/zeebo/xxh3 v1.1.0/go.mod h1:IisAie1LELR4xhVinxWS5+zf1lA4p0MW4T+w+W07F5s=
go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg=
go.mongodb.org/mongo-driver/v2 v2.8.0 h1:CxWDGQYY8QQwNjAl/aq2sfWakdnWZynnqJ9F4DhHbP8=
go.mongodb.org/mongo-driver/v2 v2.8.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.opencensus.io v0.20.1/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk=
go.opencensus.io v0.20.2/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk=
go.opencensus.io v0.22.2/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw=
go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/atomic v1.5.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=
go.uber.org/atomic v1.6.0/go.mod h1:sABNBOSYdrvTF6hTgEIbc7YasKWGhgEQZyfxyTvoXHQ=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/multierr v1.3.0/go.mod h1:VgVr7evmIr6uPjLBxg28wmKNXyqE9akIJ5XnfpiKl+4=
go.uber.org/multierr v1.5.0/go.mod h1:FeouvMocqHpRaaGuG9EjoKcStLC43Zu/fmqdUMPcKYU=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/tools v0.0.0-20190618225709-2cfd321de3ee/go.mod h1:vJERXedbb3MVM5f9Ejo0C68/HhF8uaILCdgjnY+goOA=
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
go.uber.org/zap v1.13.0/go.mod h1:zwrFLgMcdUuIBviXEYEH1YKNaOBnKXsx2IPda5bBwHM=
go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
go.yaml.in/yaml/v2 v2.4.4 h1:tuyd0P+2Ont/d6e2rl3be67goVK4R6deVxCUX5vyPaQ=
go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/arch v0.30.0 h1:sB9h+1gRGa2+LauFSV0tm8bK1J2yo1bx6/Uyi/P6DTU=
golang.org/x/arch v0.30.0/go.mod h1:0X+GdSIP+kL5wPmpK7sdkEVTt2XoYP0cSjQSbZBwOi8=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181029021203-45a5f77698d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190510104115-cbcb75029529/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93 h1:fQsdNF2N+/YewlRZiricy4P1iimyPKZ/xwniHj8Q2a0=
golang.org/x/exp v0.0.0-20251219203646-944ab1f22d93/go.mod h1:EPRbTFwzwjXj9NpYyyrvenVh9Y+GFeEvMNh7Xuz7xgU=
golang.org/x/image v0.23.0/go.mod h1:wJJBTdLfCCf3tiHa1fNxpZmUI4mmoZvwMCPP0ddoNKY=
golang.org/x/image v0.41.0 h1:8wS72eGJMJaBxK6okTzd4WaXumUlTVlb753MlsSvTCo=
golang.org/x/image v0.41.0/go.mod h1:uIc348UZMSvS5Z65CVZ7iDPaNobNFEPeJ4kbqTOszmA=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20190930215403-16217165b5de/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/mod v0.0.0-20190513183733-4bf6d317e70e/go.mod h1:mXi4GBBbnImb6dmsKGUJ2LatrhH/nqhxcFungHvyanc=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk=
golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181023162649-9b4f9f5ad519/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190125091013-d26f9f9a57f3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190603091049-60506f45cf65/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.7.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.22.0 h1:SZjpbeLmrCk4xhRSZFNZW5gFUeCeFgjekvI/+gfScek=
golang.org/x/sync v0.22.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181026203630-95b1ffbd15a5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190502145724-3ef323f4f1fd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190726091711-fc99dfbffb4e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190826190057-c7b8b68b1456/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191220142924-d4481acd189f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201214210602-f9fddec55a1e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210316164454-77fc1eacc6aa/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/time v0.0.0-20180412165947-fbb02b2291d2/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20190621195816-6e04913cbbac/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
golang.org/x/tools v0.0.0-20191029041327-9cc4af7d6b2c/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191029190741-b9c20aec41a5/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.0.0-20200103221440-774c71fcf114/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=
golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/api v0.3.1/go.mod h1:6wY9I6uQWHQ8EM57III9mq/AjF+i8G65rmVagqKMtkk=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/appengine v1.2.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190425155659-357c62f0e4bb/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
google.golang.org/genproto v0.0.0-20190530194941-fb225487d101/go.mod h1:z3L6/3dTEVtUr6QSP8miRzeRqwQOioJ9I66odjN4I7s=
google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc=
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.0/go.mod h1:chYK+tFQF0nDUGJgXMSgLCQk3phJEuONr2DCgLDdAQM=
google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
google.golang.org/grpc v1.22.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.23.1/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.26.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.27.1/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/cheggaaa/pb.v1 v1.0.25/go.mod h1:V/YB90LKu/1FcN3WVnfiiE5oMCibMjukxqG/qStrOgw=
gopkg.in/errgo.v2 v2.1.0/go.mod h1:hNsd1EY+bozCKY1Ytp96fpM3vjJbqLJn88ws8XvfDNI=
gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
gopkg.in/gcfg.v1 v1.2.3/go.mod h1:yesOnuUOFQAhST5vPY4nbZsb/huCgGGXlipJsBn0b3o=
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.8/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gorm.io/driver/mysql v1.6.0 h1:eNbLmNTpPpTOVZi8MMxCi2aaIm0ZpInbORNXDwyLGvg=
gorm.io/driver/mysql v1.6.0/go.mod h1:D/oCC2GWK3M/dqoLxnOlaNKmXz8WNTfcS9y5ovaSqKo=
gorm.io/driver/postgres v1.6.2 h1:BvXQ/cNUg63q5TFNg672DmDcowZSFrNLkkA3Xe6GXq4=
gorm.io/driver/postgres v1.6.2/go.mod h1:0c4fQA44XhOklXDkgtuKqysHCycTa5i9e3EIpDGCwXk=
gorm.io/driver/sqlite v1.1.3/go.mod h1:AKDgRWk8lcSQSw+9kxCJnX/yySj8G3rdwYlU57cB45c=
gorm.io/driver/sqlite v1.6.0 h1:WHRRrIiulaPiPFmDcod6prc4l2VGVWHz80KspNsxSfQ=
gorm.io/driver/sqlite v1.6.0/go.mod h1:AO9V1qIQddBESngQUKWL9yoH93HIeA1X6V633rBwyT8=
gorm.io/driver/sqlserver v1.6.4 h1:kGA9Z0D7dnIz7yVvWp18qLBSPFpUQWGqMA4rnxkScdQ=
gorm.io/driver/sqlserver v1.6.4/go.mod h1:oRtXDKFRYj8MqyMq+JFEdaA+StSQKC4zupU6blIdB0s=
gorm.io/gorm v1.20.1/go.mod h1:0HFTzE/SqkGTzK6TlDPPQbAYCluiVvhzoA1+aVyzenw=
gorm.io/gorm v1.23.0/go.mod h1:l2lP/RyAtc1ynaTjFksBde/O8v9oOGIApu2/xRitmZk=
gorm.io/gorm v1.31.2 h1:3o8FXNo9v9S858gil+3LlZA1LkCOzgb4g5BL64FgaCo=
gorm.io/gorm v1.31.2/go.mod h1:XyQVbO2k6YkOis7C2437jSit3SsDK72s7n7rsSHd+Gs=
gorm.io/plugin/dbresolver v1.6.2 h1:F4b85TenghUeITqe3+epPSUtHH7RIk3fXr5l83DF8Pc=
gorm.io/plugin/dbresolver v1.6.2/go.mod h1:tctw63jdrOezFR9HmrKnPkmig3m5Edem9fdxk9bQSzM=
gorm.io/plugin/soft_delete v1.2.1 h1:qx9D/c4Xu6w5KT8LviX8DgLcB9hkKl6JC9f44Tj7cGU=
gorm.io/plugin/soft_delete v1.2.1/go.mod h1:Zv7vQctOJTGOsJ/bWgrN1n3od0GBAZgnLjEx+cApLGk=
honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=
modernc.org/cc/v4 v4.27.1 h1:9W30zRlYrefrDV2JE2O8VDtJ1yPGownxciz5rrbQZis=
modernc.org/cc/v4 v4.27.1/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccgo/v4 v4.30.1 h1:4r4U1J6Fhj98NKfSjnPUN7Ze2c6MnAdL0hWw6+LrJpc=
modernc.org/ccgo/v4 v4.30.1/go.mod h1:bIOeI1JL54Utlxn+LwrFyjCx2n2RDiYEaJVSrgdrRfM=
modernc.org/fileutil v1.3.40 h1:ZGMswMNc9JOCrcrakF1HrvmergNLAmxOPjizirpfqBA=
modernc.org/fileutil v1.3.40/go.mod h1:HxmghZSZVAz/LXcMNwZPA/DRrQZEVP9VX0V4LQGQFOc=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.1 h1:k8T3gkXWY9sEiytKhcgyiZ2L0DTyCQ/nvX+LoCljoRE=
modernc.org/gc/v3 v3.1.1/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.67.4 h1:zZGmCMUVPORtKv95c2ReQN5VDjvkoRm9GWPTEPuvlWg=
modernc.org/libc v1.67.4/go.mod h1:QvvnnJ5P7aitu0ReNpVIEyesuhmDLQ8kaEoyMjIFZJA=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.1.4 h1:2kNGMRiUjrp4LcaPuLY2PzUfqM/w9N23quVwhKt5Qm8=
modernc.org/opt v0.1.4/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.42.2 h1:7hkZUNJvJFN2PgfUdjni9Kbvd4ef4mNLOu0B9FGxM74=
modernc.org/sqlite v1.42.2/go.mod h1:+VkC6v3pLOAE0A0uVucQEcbVW0I5nHCeDaBf+DpsQT8=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
sigs.k8s.io/yaml v1.1.0/go.mod h1:UJmg0vDUVViEyp3mgSv9WPwZCDxu4rQW1olrI1uml+o=
sourcegraph.com/sourcegraph/appdash v0.0.0-20190731080439-ebfcffb1b5c0/go.mod h1:hI742Nqp5OhwiqlzhgfbWU4mW4yO10fP+LoT9WOswdU=
+28
View File
@@ -0,0 +1,28 @@
// Command e2e-apporder is a go-admin binary with the example application
// linked into it.
//
// It exists because there is otherwise nothing to install. `migrate install
// order` looks the code up in the registries an application fills from its
// own init(), and an application the binary was not built with registers
// nothing - so every path through the installer past its first check went
// untested, and so did the seeding, the ledger and the uninstall against
// anything but a hand-built fixture.
//
// This lives in its own module, not behind a build tag in the main one. A
// tagged import there would still be resolved by `go mod tidy`, which
// considers every build tag and would go looking for
// github.com/go-admin-team/example-app-order on the network - a repository
// that does not exist, because the example is a directory inside this one.
// A separate module with replace directives is invisible to the main
// module's tidy, its build and its tests, and needs no go.work.
package main
import (
_ "github.com/go-admin-team/example-app-order/migration"
"go-admin/cmd"
)
func main() {
cmd.Execute()
}
+15
View File
@@ -20,6 +20,9 @@ const (
checkMenuIDConflict = "menu-id-collision"
checkImportBoundary = "contract-import-boundary"
checkShimAlias = "contract-shim-alias"
checkDataScopeRoute = "datascope-route-unguarded"
checkShutdownGrace = "shutdown-budget-overruns-grace"
checkDockerStop = "docker-stop-cuts-shutdown-short"
)
// Package paths, relative to the module. Spelled once so a module rename
@@ -47,6 +50,18 @@ func runChecks(s *snapshot, opt options) ([]Finding, error) {
out = append(out, checkMenuIDCollisions(s)...)
out = append(out, checkContractImportBoundary(s)...)
out = append(out, checkContractShimAlias(s)...)
out = append(out, checkDataScopeRoutes(s)...)
for _, run := range []func(*snapshot) ([]Finding, error){
checkShutdownBudgetFitsGrace,
checkDockerStopGrace,
} {
fs, err := run(s)
if err != nil {
return nil, err
}
out = append(out, fs...)
}
if opt.UIDir != "" {
fs, err := checkMenuNames(s, opt.UIDir)
+390
View File
@@ -0,0 +1,390 @@
package main
import (
"go/ast"
"strings"
)
// ---------------------------------------------------------------------------
// check 8: a handler that reads the data permission, on a route that never
// installs the middleware which puts one there
// ---------------------------------------------------------------------------
// permissionGetter is the function a handler calls to obtain the caller's data
// scope, and permissionMiddleware is the middleware that puts one in the
// context. Matched by name rather than by resolved symbol: the tool parses
// without type checking, and both names are distinctive enough that a
// same-named function from somewhere else would still be worth a look.
const (
permissionGetter = "GetPermissionFromContext"
permissionMiddleware = "PermissionAction"
)
// actionsPkgSuffix identifies the package the two names above live in - this
// repository's common/actions shim and core's sdk/contract/actions both end
// this way, and a module rename changes neither.
const actionsPkgSuffix = "/actions"
// handlerKey identifies one handler method uniquely across packages, so that
// two types named SysUser in different packages are not confused.
type handlerKey struct {
Pkg string
Type string
Func string
}
// checkDataScopeRoutes reports a route whose handler asks for the caller's data
// permission while the group it is registered on never installs the middleware
// that supplies one.
//
// GetPermissionFromContext cannot fail. When nothing put a *DataPermission in
// the context it hands back a zero value, whose DataScope is the empty string -
// and the empty string is not one of the five scopes Permission recognises, so
// it takes the default branch. That branch fails closed: the query is given
// `1 = 0` and matches nothing.
//
// The result is an endpoint that answers "not found" or "no permission" for
// rows that plainly exist, and only on deployments that set enabledp: true -
// with data permissions off, Permission returns the query untouched and the
// missing middleware costs nothing. That is the shape this check exists for: a
// default configuration where the mistake is invisible, and a test suite that
// runs on it.
//
// It happened. /api/v1/getinfo read the permission on a group carrying only the
// JWT middleware, so every login on a deployment with data permissions enabled
// ended in a 401 from the endpoint the browser calls immediately after signing
// in - and went back to the login page.
//
// Either half is a fix, and which one depends on the route. A handler that
// reads somebody else's rows wants the middleware. A handler reading the
// caller's own row - where the id comes from the token - wants no scope at all,
// because a scope has nothing left to restrict there and DataScopeSelf, which
// matches on create_by, would reject every user who did not create their own
// account. The check reports the mismatch and leaves the choice.
func checkDataScopeRoutes(s *snapshot) []Finding {
handlers := permissionReadingHandlers(s)
if len(handlers) == 0 {
return nil
}
var out []Finding
for _, sf := range s.Files {
if sf.isTest() {
continue
}
for _, decl := range sf.Syntax.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok || fn.Body == nil {
continue
}
out = append(out, s.routeFindings(sf, fn, handlers)...)
}
}
return out
}
// permissionReadingHandlers collects every method whose body calls the getter.
//
// Test files are included deliberately: a handler is a handler wherever it is
// declared, and skipping them would let a route registered from a test fixture
// go unchecked while the fixture is exactly where a new one gets written first.
func permissionReadingHandlers(s *snapshot) map[handlerKey]bool {
out := map[handlerKey]bool{}
for _, sf := range s.Files {
for _, decl := range sf.Syntax.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok || fn.Body == nil || fn.Recv == nil || len(fn.Recv.List) == 0 {
continue
}
recv := receiverTypeName(fn.Recv.List[0].Type)
if recv == "" {
continue
}
if callsPackageFunc(sf, fn.Body, permissionGetter) {
out[handlerKey{Pkg: sf.Pkg, Type: recv, Func: fn.Name.Name}] = true
}
}
}
return out
}
// routeFindings walks one function looking for group definitions and the routes
// registered on them.
func (s *snapshot) routeFindings(sf *sourceFile, fn *ast.FuncDecl, handlers map[handlerKey]bool) []Finding {
// Local variable bindings for the whole function. The first pass below
// fills these and the second reads them, so a registration sees every
// binding in the function rather than only the ones written above it -
// deliberately, because a `.Use` can be written below a route and still be
// part of the chain. The cost is that a name reused for two different
// things in one function resolves to whichever assignment came last.
apiVars := map[string]handlerKey{} // var -> the type it holds
guarded := map[string]bool{} // group var -> middleware installed
known := map[string]bool{} // group var -> is a router group at all
prefix := map[string]string{} // group var -> the path it was declared with
var out []Finding
ast.Inspect(fn.Body, func(n ast.Node) bool {
switch stmt := n.(type) {
case *ast.AssignStmt:
for i, lhs := range stmt.Lhs {
id, ok := lhs.(*ast.Ident)
if !ok || i >= len(stmt.Rhs) {
continue
}
rhs := stmt.Rhs[i]
if key, ok := apiTypeOf(sf, rhs); ok {
apiVars[id.Name] = key
continue
}
if parent, isGroup := groupSource(rhs); isGroup {
known[id.Name] = true
prefix[id.Name] = prefix[parent] + groupPath(rhs)
// A subgroup inherits whatever its parent already had:
// gin copies the parent's handler chain into the child.
guarded[id.Name] = guarded[parent] || containsCallNamed(rhs, permissionMiddleware)
}
}
case *ast.ExprStmt:
// A separate `g.Use(...)` after the group was defined.
call, ok := stmt.X.(*ast.CallExpr)
if !ok {
return true
}
if target, ok := receiverIdentOf(call, "Use"); ok && known[target] {
if containsCallNamed(call, permissionMiddleware) {
guarded[target] = true
}
}
}
return true
})
// Second pass for the registrations, so that a `.Use` written below a route
// still counts - the middleware chain is assembled before any request is
// served, not in source order.
ast.Inspect(fn.Body, func(n ast.Node) bool {
call, ok := n.(*ast.CallExpr)
if !ok {
return true
}
gvar, method, ok := routeRegistration(call)
if !ok || !known[gvar] || guarded[gvar] {
return true
}
route, handlerVar, handlerName, ok := routeArgs(call)
if !ok {
return true
}
key, ok := apiVars[handlerVar]
if !ok {
return true
}
key.Func = handlerName
if !handlers[key] {
return true
}
out = append(out, s.finding(Error, checkDataScopeRoute, sf, call,
"%s %q is handled by %s.%s, which reads the caller's data permission,\n"+
" but the group it is registered on never installs %s.\n"+
" GetPermissionFromContext then returns the zero value, whose empty DataScope is not a\n"+
" recognised scope, so Permission fails closed and the query matches nothing - on any\n"+
" deployment with enabledp: true. With data permissions off the route works, which is\n"+
" why this does not show up in the default configuration or in CI.\n"+
" Add %s() to the group, or stop scoping a query that is already limited to the caller.",
method, prefix[gvar]+route, key.Type, handlerName, permissionMiddleware, permissionMiddleware))
return true
})
return out
}
// receiverTypeName returns the bare type name of a method receiver, for both
// `(e SysUser)` and `(e *SysUser)`.
func receiverTypeName(expr ast.Expr) string {
if star, ok := expr.(*ast.StarExpr); ok {
expr = star.X
}
if id, ok := expr.(*ast.Ident); ok {
return id.Name
}
return ""
}
// callsPackageFunc reports whether body calls name on a package whose import
// path ends in actionsPkgSuffix.
func callsPackageFunc(sf *sourceFile, body ast.Node, name string) bool {
found := false
ast.Inspect(body, func(n ast.Node) bool {
if found {
return false
}
call, ok := n.(*ast.CallExpr)
if !ok {
return true
}
sel, ok := call.Fun.(*ast.SelectorExpr)
if !ok || sel.Sel.Name != name {
return true
}
pkg, ok := sel.X.(*ast.Ident)
if !ok {
return true
}
if path, ok := sf.imports[pkg.Name]; ok && strings.HasSuffix(path, actionsPkgSuffix) {
found = true
return false
}
return true
})
return found
}
// apiTypeOf recognises `apis.SysUser{}` and returns the package path and type.
func apiTypeOf(sf *sourceFile, expr ast.Expr) (handlerKey, bool) {
lit, ok := expr.(*ast.CompositeLit)
if !ok {
return handlerKey{}, false
}
sel, ok := lit.Type.(*ast.SelectorExpr)
if !ok {
return handlerKey{}, false
}
pkg, ok := sel.X.(*ast.Ident)
if !ok {
return handlerKey{}, false
}
path, ok := sf.imports[pkg.Name]
if !ok {
return handlerKey{}, false
}
return handlerKey{Pkg: path, Type: sel.Sel.Name}, true
}
// groupSource reports whether expr builds a router group, and names the
// variable it was built from when there is one.
func groupSource(expr ast.Expr) (string, bool) {
parent := ""
isGroup := false
ast.Inspect(expr, func(n ast.Node) bool {
call, ok := n.(*ast.CallExpr)
if !ok {
return true
}
sel, ok := call.Fun.(*ast.SelectorExpr)
if !ok || sel.Sel.Name != "Group" {
return true
}
isGroup = true
if id, ok := sel.X.(*ast.Ident); ok {
parent = id.Name
}
return true
})
return parent, isGroup
}
// groupPath returns the literal path a group was declared with, or "" when it
// is not a literal - a computed prefix is left out of the message rather than
// printed as something it is not.
func groupPath(expr ast.Expr) string {
out := ""
ast.Inspect(expr, func(n ast.Node) bool {
call, ok := n.(*ast.CallExpr)
if !ok {
return true
}
sel, ok := call.Fun.(*ast.SelectorExpr)
if !ok || sel.Sel.Name != "Group" || len(call.Args) == 0 {
return true
}
if lit, ok := call.Args[0].(*ast.BasicLit); ok {
out = strings.Trim(lit.Value, `"`)
}
return true
})
return out
}
// containsCallNamed reports whether expr contains a call to a function with
// this name, at any depth of a method chain or argument list.
func containsCallNamed(expr ast.Node, name string) bool {
found := false
ast.Inspect(expr, func(n ast.Node) bool {
if found {
return false
}
call, ok := n.(*ast.CallExpr)
if !ok {
return true
}
switch fun := call.Fun.(type) {
case *ast.SelectorExpr:
if fun.Sel.Name == name {
found = true
return false
}
case *ast.Ident:
if fun.Name == name {
found = true
return false
}
}
return true
})
return found
}
// receiverIdentOf returns the variable a `x.method(...)` call was made on.
func receiverIdentOf(call *ast.CallExpr, method string) (string, bool) {
sel, ok := call.Fun.(*ast.SelectorExpr)
if !ok || sel.Sel.Name != method {
return "", false
}
id, ok := sel.X.(*ast.Ident)
if !ok {
return "", false
}
return id.Name, true
}
// httpMethods are the registration calls this check understands. Any and Match
// are absent on purpose: they take the method as data, and a check that half
// understands a registration is worse than one that says nothing about it.
var httpMethods = map[string]bool{
"GET": true, "POST": true, "PUT": true, "DELETE": true, "PATCH": true, "HEAD": true, "OPTIONS": true,
}
// routeRegistration recognises `g.GET(...)` and names the group and method.
func routeRegistration(call *ast.CallExpr) (string, string, bool) {
sel, ok := call.Fun.(*ast.SelectorExpr)
if !ok || !httpMethods[sel.Sel.Name] {
return "", "", false
}
id, ok := sel.X.(*ast.Ident)
if !ok {
return "", "", false
}
return id.Name, sel.Sel.Name, true
}
// routeArgs pulls the path and the `api.Handler` argument out of a
// registration, ignoring any middleware written between them.
func routeArgs(call *ast.CallExpr) (route, handlerVar, handlerName string, ok bool) {
if len(call.Args) < 2 {
return "", "", "", false
}
lit, isLit := call.Args[0].(*ast.BasicLit)
if !isLit {
return "", "", "", false
}
route = strings.Trim(lit.Value, `"`)
// The handler is the last argument; anything before it is middleware.
sel, isSel := call.Args[len(call.Args)-1].(*ast.SelectorExpr)
if !isSel {
return "", "", "", false
}
id, isIdent := sel.X.(*ast.Ident)
if !isIdent {
return "", "", "", false
}
return route, id.Name, sel.Sel.Name, true
}
+163
View File
@@ -0,0 +1,163 @@
package main
import (
"strings"
"testing"
)
// apisFile is a handler package with two methods: one that reads the caller's
// data permission and one that does not.
const apisFile = `package apis
import (
"github.com/gin-gonic/gin"
"go-admin/common/actions"
)
type SysUser struct{}
func (e SysUser) Scoped(c *gin.Context) {
p := actions.GetPermissionFromContext(c)
_ = p
}
func (e SysUser) Unscoped(c *gin.Context) {}
`
func routerFile(uses string) string {
return `package router
import (
"github.com/gin-gonic/gin"
"go-admin/app/admin/apis"
"go-admin/common/actions"
)
var _ = actions.PermissionAction
func register(v1 *gin.RouterGroup) {
api := apis.SysUser{}
r := v1.Group("/sys-user")` + uses + `
{
r.GET("/:id", api.Scoped)
}
}
`
}
// The mistake itself: a handler that reads the permission, on a group that
// never installs the middleware which puts one there.
func TestDataScopeRouteWithoutTheMiddlewareIsReported(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/apis/sys_user.go": apisFile,
"app/admin/router/sys_user.go": routerFile(`.Use(gin.Logger())`),
})
f := requireOne(t, check(t, root, options{}), checkDataScopeRoute)
for _, want := range []string{`GET "/sys-user/:id"`, "SysUser.Scoped", "PermissionAction"} {
if !strings.Contains(f.Message, want) {
t.Errorf("message does not mention %q:\n%s", want, f.Message)
}
}
}
// The middleware installed in the chain is the fix, and must silence it.
func TestDataScopeRouteWithTheMiddlewareIsQuiet(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/apis/sys_user.go": apisFile,
"app/admin/router/sys_user.go": routerFile(`.Use(gin.Logger()).Use(actions.PermissionAction())`),
})
if got := only(t, check(t, root, options{}), checkDataScopeRoute); len(got) != 0 {
t.Errorf("reported %d findings for a guarded group:\n%v", len(got), got)
}
}
// The other fix - the handler stops reading the permission - must silence it
// too. Reporting a route whose handler needs no scope would push people to
// install middleware they do not want, which is how /getinfo would have been
// "fixed" into rejecting every user who did not create their own account.
func TestARouteWhoseHandlerReadsNoPermissionIsQuiet(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/apis/sys_user.go": apisFile,
"app/admin/router/sys_user.go": `package router
import (
"github.com/gin-gonic/gin"
"go-admin/app/admin/apis"
)
func register(v1 *gin.RouterGroup) {
api := apis.SysUser{}
r := v1.Group("")
{
r.GET("/getinfo", api.Unscoped)
}
}
`,
})
if got := only(t, check(t, root, options{}), checkDataScopeRoute); len(got) != 0 {
t.Errorf("reported %d findings for a handler that reads no permission:\n%v", len(got), got)
}
}
// gin copies the parent's handler chain into a subgroup, so a group carved out
// of a guarded one is guarded. Reporting it would be a false positive, and a
// check that cries wolf is one people switch off.
func TestASubgroupInheritsTheMiddleware(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/apis/sys_user.go": apisFile,
"app/admin/router/sys_user.go": `package router
import (
"github.com/gin-gonic/gin"
"go-admin/app/admin/apis"
"go-admin/common/actions"
)
func register(v1 *gin.RouterGroup) {
api := apis.SysUser{}
parent := v1.Group("/sys").Use(actions.PermissionAction())
child := parent.Group("/user")
{
child.GET("/:id", api.Scoped)
}
}
`,
})
if got := only(t, check(t, root, options{}), checkDataScopeRoute); len(got) != 0 {
t.Errorf("reported %d findings for a subgroup of a guarded group:\n%v", len(got), got)
}
}
// Two packages can both declare a SysUser. Only the one whose method reads the
// permission may be reported, or the check becomes a name search.
func TestAHandlerIsMatchedByPackageNotJustName(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/apis/sys_user.go": apisFile,
"app/other/apis/sys_user.go": `package apis
import "github.com/gin-gonic/gin"
type SysUser struct{}
func (e SysUser) Scoped(c *gin.Context) {}
`,
"app/other/router/sys_user.go": `package router
import (
"github.com/gin-gonic/gin"
"go-admin/app/other/apis"
)
func register(v1 *gin.RouterGroup) {
api := apis.SysUser{}
r := v1.Group("/other")
{
r.GET("/:id", api.Scoped)
}
}
`,
})
if got := only(t, check(t, root, options{}), checkDataScopeRoute); len(got) != 0 {
t.Errorf("reported %d findings for a same-named handler in another package:\n%v", len(got), got)
}
}
+17 -7
View File
@@ -1,13 +1,23 @@
// Command checksilent reports the failures in this repository that do not
// announce themselves: no error, no log line, behaviour quietly wrong.
//
// Seven checks, six of them ERROR and one WARN. An ERROR fails the run; a WARN
// prints and does not. The split is not about how bad the consequence is - all
// seven are bad - but about how certain the detection is. Everything reported as
// an ERROR is decided from this repository's own syntax. The one WARN compares
// against a second repository through a regular expression, and a check that
// can be wrong must not be able to stop a build, or the first response to it
// will be an ignore comment.
// An ERROR fails the run; a WARN prints and does not. The split is not about
// how bad the consequence is - every one of these is bad - but about how much
// room is left to act.
//
// Most of them report only ERROR: each is decided from this repository's own
// files and is either true or not. The menu-name check reports only WARN,
// because it compares against a second repository through a regular
// expression, and a check that can be wrong must not be able to stop a build
// or the first response to it will be an ignore comment. The two
// shutdown-budget checks report at both levels from one arithmetic: a budget
// that already overruns is an ERROR, and one that fits with no headroom left
// is a WARN - it works today, so failing the build on it would be failing a
// correct configuration.
//
// The list of checks is runChecks in checks.go. It is deliberately not
// repeated here as a count: the two places that carried one were both wrong by
// the time anybody looked.
//
// Usage:
//
+651
View File
@@ -0,0 +1,651 @@
package main
import (
"bytes"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
yaml "go.yaml.in/yaml/v3"
)
// The files this check compares, and the package the fallbacks come from.
const (
settingsFile = "config/settings.yml"
k8sDeployFile = "scripts/k8s/deploy.yml"
pkgHostConfig = "config"
drainConstName = "DefaultDrainSeconds"
serverConstName = "DefaultServerSeconds"
cleanupConstName = "DefaultCleanupSeconds"
)
// graceMarginSeconds is the headroom a shutdown budget needs beyond itself.
//
// Spelled once and used by both checks below, because they fail the same way:
// somebody raises a budget in config/settings.yml and does not go looking for
// the two other places that have to allow room for it. Two margins would
// eventually be two different numbers.
const graceMarginSeconds = 5
// checkShutdownBudgetFitsGrace compares the shutdown budget this repository
// ships against the stop grace period its own Kubernetes manifest allows.
//
// The two are not merely adjacent examples. scripts/k8s/prerun.sh builds the
// settings-admin ConfigMap out of config/settings.yml, and the Deployment
// mounts that ConfigMap - so the manifest deploys that file.
//
// The budgets are spent one after the other, and when their sum reaches
// terminationGracePeriodSeconds the kubelet sends SIGKILL while the cleanup
// callbacks are still running. Nothing reports it: the pod disappears
// mid-shutdown and it reads as a crash rather than as a number that was raised
// in one file and not the other. Which is how it would be raised - drain is
// the interesting knob and the grace period is in a different directory.
//
// Two levels, and an overrun is not also reported as a shortage of headroom:
// every Error satisfies the Warn condition too, and an Error that always drags
// a duplicate Warn behind it teaches people to skip Warns.
//
// A preStop hook counts, even though the shipped manifest has none. It is
// spent before the process is told anything, so it is added to the budget
// rather than overlapping it - and a self-check that cannot see it would
// understate the real cost by however long somebody set it to, which is worse
// than not checking.
//
// It reports nothing when either file is absent and when the manifest sets no
// grace period, because there is then no second number to disagree with.
func checkShutdownBudgetFitsGrace(s *snapshot) ([]Finding, error) {
budget, ok, err := shippedShutdownBudget(s)
if err != nil || !ok {
return nil, err
}
m, ok, err := readManifest(s)
if err != nil || !ok {
return nil, err
}
if m.grace == nil {
return nil, nil
}
var out []Finding
if m.preStopUnreadable {
out = append(out, Finding{
Check: checkShutdownGrace,
Severity: Warn.String(),
File: k8sDeployFile,
Line: m.preStopLine,
Col: 1,
Message: "this preStop hook is not a sleep, so how long it takes cannot be read here " +
"and is not in the sum below; it is spent before the process is told anything, " +
"so whatever it costs has to fit inside terminationGracePeriodSeconds as well.",
severity: Warn,
})
}
total := m.preStop + budget.drain + budget.server + budget.cleanup
grace := *m.grace
spelled := fmt.Sprintf("preStop %d + drain %d + server %d + cleanup %d",
m.preStop, budget.drain, budget.server, budget.cleanup)
switch {
case total >= grace:
out = append(out, Finding{
Check: checkShutdownGrace,
Severity: Error.String(),
File: k8sDeployFile,
Line: m.graceLine,
Col: 1,
Message: fmt.Sprintf(
"terminationGracePeriodSeconds is %d and the shutdown takes %d (%s, from %s); "+
"SIGKILL would arrive while the cleanup callbacks are still running. "+
"Raise it to %d, or take %d off the budget.",
grace, total, spelled, settingsFile,
total+graceMarginSeconds, total+graceMarginSeconds-grace),
severity: Error,
})
case total+graceMarginSeconds > grace:
out = append(out, Finding{
Check: checkShutdownGrace,
Severity: Warn.String(),
File: k8sDeployFile,
Line: m.graceLine,
Col: 1,
Message: fmt.Sprintf(
"terminationGracePeriodSeconds is %d and the shutdown takes %d (%s, from %s), "+
"which leaves under %ds of headroom; a callback that runs slightly long is "+
"cut off. Raise it to %d.",
grace, total, spelled, settingsFile, graceMarginSeconds, total+graceMarginSeconds),
severity: Warn,
})
}
return out, nil
}
// dockerStopArgs matches a stop command in a script or a workflow.
var (
dockerStopArgs = regexp.MustCompile(`\bdocker\s+stop\b`)
// --timeout is the current name, --time its deprecated spelling and -t the
// short form; docker still accepts all three, so all three are read. The
// long name comes first because --time is a prefix of it, and a flag that
// the check cannot read is reported as no deadline at all - which would
// have this tool pressing people towards the deprecated spelling.
dockerStopTime = regexp.MustCompile(`(--timeout|--time|-t)[=\s]*(\d+)`)
)
// checkDockerStopGrace reports a stop path that does not allow this process
// the time it spends shutting down.
//
// docker allows ten seconds unless told otherwise, and that number is nowhere
// near the command - so a budget raised in config/settings.yml passes every
// test, deploys, and then has its cleanup callbacks killed on the next
// release. Same failure as the manifest's grace period, same arithmetic, same
// margin; only the file it lives in is different.
//
// Both ways of stopping this repository's container are covered, because
// covering one of two identical paths is what produces a clean run that means
// nothing: `docker stop` in a workflow or a script, and stop_grace_period in
// the compose file the Makefile's own `make run` uses.
//
// An absent deadline is reported rather than assumed to be ten: the value that
// applies is then invisible at the call site and cannot follow the budget.
func checkDockerStopGrace(s *snapshot) ([]Finding, error) {
budget, ok, err := shippedShutdownBudget(s)
if err != nil || !ok {
return nil, err
}
total := budget.drain + budget.server + budget.cleanup
spelled := fmt.Sprintf("drain %d + server %d + cleanup %d",
budget.drain, budget.server, budget.cleanup)
sites, err := findStopDeadlines(s)
if err != nil {
return nil, err
}
var out []Finding
for _, site := range sites {
finding := Finding{
Check: checkDockerStop,
File: site.file,
Line: site.line,
Col: 1,
}
switch {
case !site.set:
finding.Severity, finding.severity = Error.String(), Error
finding.Message = fmt.Sprintf(
"%s allows the default %d seconds, and this process spends %d shutting down "+
"(%s, from %s). %s.",
site.what, dockerDefaultGraceSeconds, total, spelled, settingsFile,
site.fix(total+graceMarginSeconds))
case site.seconds <= total:
finding.Severity, finding.severity = Error.String(), Error
finding.Message = fmt.Sprintf(
"%s allows %d seconds and this shutdown takes %d (%s, from %s); the cleanup "+
"callbacks are killed part-way through. %s.",
site.what, site.seconds, total, spelled, settingsFile,
site.fix(total+graceMarginSeconds))
case site.seconds < total+graceMarginSeconds:
finding.Severity, finding.severity = Warn.String(), Warn
finding.Message = fmt.Sprintf(
"%s allows %d seconds over a shutdown that takes %d (%s, from %s), which leaves "+
"under %ds of headroom. %s.",
site.what, site.seconds, total, spelled, settingsFile, graceMarginSeconds,
site.fix(total+graceMarginSeconds))
default:
continue
}
out = append(out, finding)
}
return out, nil
}
// dockerDefaultGraceSeconds is what docker allows a container to stop in when
// nothing says otherwise. It applies to `docker stop` and to compose alike.
const dockerDefaultGraceSeconds = 10
// stopSite is one place this repository decides how long a container gets.
type stopSite struct {
file string
line int
// what names the setting in the finding, in the spelling of the file it
// was found in.
what string
// compose says which of the two fixes to suggest.
compose bool
seconds int
set bool
}
func (s stopSite) fix(seconds int) string {
if s.compose {
return fmt.Sprintf("Set stop_grace_period: %ds", seconds)
}
return fmt.Sprintf("Pass --timeout %d", seconds)
}
func findStopDeadlines(s *snapshot) ([]stopSite, error) {
sites, err := findDockerStops(s.Root)
if err != nil {
return nil, err
}
compose, err := findComposeServices(s)
if err != nil {
return nil, err
}
return append(sites, compose...), nil
}
// dockerStopExtensions and dockerStopNames are where a stop command can be
// written in this repository: workflows, shell scripts and the Makefile.
var (
dockerStopExtensions = map[string]bool{".yml": true, ".yaml": true, ".sh": true, ".bash": true}
dockerStopNames = map[string]bool{"Makefile": true, "makefile": true}
)
func findDockerStops(root string) ([]stopSite, error) {
var out []stopSite
err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
if path != root && skippedDirs[info.Name()] {
return filepath.SkipDir
}
return nil
}
if !dockerStopExtensions[filepath.Ext(path)] && !dockerStopNames[info.Name()] {
return nil
}
b, err := os.ReadFile(path)
if err != nil {
return err
}
rel, err := filepath.Rel(root, path)
if err != nil {
return err
}
for i, line := range strings.Split(string(b), "\n") {
// A commented-out command is not one that runs, and the settings
// file describes `docker stop` in prose right beside the budget
// this check reads.
if trimmed := strings.TrimSpace(line); strings.HasPrefix(trimmed, "#") {
continue
}
if !dockerStopArgs.MatchString(line) {
continue
}
site := stopSite{
file: filepath.ToSlash(rel),
line: i + 1,
what: "`docker stop` with no --timeout",
}
if m := dockerStopTime.FindStringSubmatch(line); m != nil {
seconds, err := strconv.Atoi(m[2])
if err != nil {
continue
}
// Quoted back in the spelling it was written in, so the
// message cannot misreport what the line says.
site.what = fmt.Sprintf("`docker stop %s %d`", m[1], seconds)
site.seconds, site.set = seconds, true
}
out = append(out, site)
}
return nil
})
return out, err
}
type shutdownSeconds struct{ drain, server, cleanup int }
// shippedShutdownBudget reads extend.shutdown out of the settings file this
// repository ships, filling in whatever it leaves out from the Go constants
// that do the same at run time.
//
// Taking the fallbacks from the snapshot rather than repeating 0/5/3 here is
// what keeps this honest when the defaults move: a tool that carries its own
// copy of the number it is checking eventually checks the wrong one.
//
// A negative value is left alone. config.Shutdown.Budget refuses it and the
// server does not start, so it is not a failure that passes unnoticed - and
// adding a negative into the sums above would understate them.
func shippedShutdownBudget(s *snapshot) (shutdownSeconds, bool, error) {
raw, ok, err := readRepoFile(s, settingsFile)
if err != nil || !ok {
return shutdownSeconds{}, false, err
}
var doc struct {
Settings struct {
Extend struct {
Shutdown *struct {
Drain *int `yaml:"drain"`
Server *int `yaml:"server"`
Cleanup *int `yaml:"cleanup"`
} `yaml:"shutdown"`
} `yaml:"extend"`
} `yaml:"settings"`
}
if err := yaml.Unmarshal(raw, &doc); err != nil {
return shutdownSeconds{}, false, fmt.Errorf("%s: %w", settingsFile, err)
}
section := doc.Settings.Extend.Shutdown
if section == nil {
return shutdownSeconds{}, false, nil
}
defaults, ok := s.hostConfigDefaults()
if !ok {
// The constants moved or were renamed. Reporting nothing would let the
// check go quiet, which is the failure it exists to catch, so this
// stops the run instead.
return shutdownSeconds{}, false, fmt.Errorf(
"%s has extend.shutdown but package %s declares no %s/%s/%s to fall back on",
settingsFile, pkgHostConfig, drainConstName, serverConstName, cleanupConstName)
}
budget := shutdownSeconds{
drain: orDefault(section.Drain, defaults.drain),
server: orDefault(section.Server, defaults.server),
cleanup: orDefault(section.Cleanup, defaults.cleanup),
}
if budget.drain < 0 || budget.server < 0 || budget.cleanup < 0 {
return shutdownSeconds{}, false, nil
}
return budget, true, nil
}
func orDefault(configured *int, fallback int) int {
if configured != nil {
return *configured
}
return fallback
}
// hostConfigDefaults reads the three fallback constants out of the parsed tree.
func (s *snapshot) hostConfigDefaults() (shutdownSeconds, bool) {
for _, sf := range s.Files {
if sf.Pkg != s.pkg(pkgHostConfig) {
continue
}
drain, okDrain := sf.consts[drainConstName]
server, okServer := sf.consts[serverConstName]
cleanup, okCleanup := sf.consts[cleanupConstName]
if okDrain && okServer && okCleanup {
return shutdownSeconds{int(drain), int(server), int(cleanup)}, true
}
}
return shutdownSeconds{}, false
}
// manifest is what the shipped Deployment says about how long it will wait.
type manifest struct {
grace *int
graceLine int
// preStop is the longest sleep any container's hook performs, since the
// hooks of several containers run at the same time.
preStop int
preStopLine int
preStopUnreadable bool
}
var preStopSleep = regexp.MustCompile(`\bsleep\s+(\d+)s?\b`)
// readManifest finds the grace period and the preStop hooks in the shipped
// manifest, with the lines they are on so a finding can be opened at them.
//
// The file holds several documents and only the Deployment carries a pod
// template, so every document is decoded and the first one with a grace period
// wins.
func readManifest(s *snapshot) (manifest, bool, error) {
raw, ok, err := readRepoFile(s, k8sDeployFile)
if err != nil || !ok {
return manifest{}, false, err
}
dec := yaml.NewDecoder(bytes.NewReader(raw))
for {
var doc struct {
Spec struct {
Template struct {
Spec struct {
Grace *int `yaml:"terminationGracePeriodSeconds"`
Containers []struct {
Lifecycle struct {
// A value, not a pointer: yaml.v3 only hands
// the raw node to a field of type yaml.Node,
// and a *yaml.Node field is allocated and left
// empty - which reads as "the hook is there but
// unreadable" for every manifest that has one.
PreStop yaml.Node `yaml:"preStop"`
} `yaml:"lifecycle"`
} `yaml:"containers"`
} `yaml:"spec"`
} `yaml:"template"`
} `yaml:"spec"`
}
switch err := dec.Decode(&doc); {
case errors.Is(err, io.EOF):
return manifest{}, false, nil
case err != nil:
return manifest{}, false, fmt.Errorf("%s: %w", k8sDeployFile, err)
}
pod := doc.Spec.Template.Spec
if pod.Grace == nil && len(pod.Containers) == 0 {
continue
}
m := manifest{
grace: pod.Grace,
graceLine: lineOf(raw, "terminationGracePeriodSeconds:"),
}
for _, c := range pod.Containers {
hook := c.Lifecycle.PreStop
if hook.Kind == 0 {
continue
}
m.preStopLine = hook.Line
if seconds, ok := preStopSeconds(&hook); ok {
// The longest one, not the sum: the hooks of several
// containers run at the same time.
if seconds > m.preStop {
m.preStop = seconds
}
continue
}
m.preStopUnreadable = true
}
if m.grace == nil {
continue
}
return m, true, nil
}
}
// preStopSeconds reads how long a hook sleeps for.
//
// Every scalar under the hook is joined and searched, because the sleep can be
// written as one argument or as several: ["sh","-c","sleep 10"] and
// ["sleep","10"] both wait ten seconds.
func preStopSeconds(node *yaml.Node) (int, bool) {
var words []string
var walk func(*yaml.Node)
walk = func(n *yaml.Node) {
if n == nil {
return
}
if n.Kind == yaml.ScalarNode {
words = append(words, n.Value)
}
for _, child := range n.Content {
walk(child)
}
}
walk(node)
m := preStopSleep.FindStringSubmatch(strings.Join(words, " "))
if m == nil {
return 0, false
}
seconds, err := strconv.Atoi(m[1])
if err != nil {
return 0, false
}
return seconds, true
}
// lineOf locates a key for a finding's position. A miss reports line 1 rather
// than failing: the position is where to look, and the message is the finding.
func lineOf(content []byte, key string) int {
for i, l := range strings.Split(string(content), "\n") {
if strings.Contains(l, key) && !strings.HasPrefix(strings.TrimSpace(l), "#") {
return i + 1
}
}
return 1
}
// readRepoFile reads a file relative to the scanned root, reporting absence
// rather than failing on it: the checks run over fixtures that carry only what
// the check under test needs.
func readRepoFile(s *snapshot, rel string) ([]byte, bool, error) {
b, err := os.ReadFile(filepath.Join(s.Root, filepath.FromSlash(rel)))
switch {
case errors.Is(err, os.ErrNotExist):
return nil, false, nil
case err != nil:
return nil, false, err
}
return b, true, nil
}
// composeFiles are the names Docker Compose looks for, in its own order of
// preference.
var composeFiles = []string{"compose.yaml", "compose.yml", "docker-compose.yaml", "docker-compose.yml"}
// composeDuration matches the durations compose accepts for
// stop_grace_period: a bare number of seconds, or hours, minutes and seconds
// in that order.
var composeDuration = regexp.MustCompile(`^(?:(\d+)h)?(?:(\d+)m)?(?:(\d+)s?)?$`)
// findComposeServices reports the stop_grace_period of every compose service
// that runs this repository's own image.
//
// Only those services. The grace period of a database or a cache alongside it
// is not this process's shutdown budget, and reporting one against the other
// would be arithmetic about two unrelated things.
func findComposeServices(s *snapshot) ([]stopSite, error) {
var out []stopSite
for _, name := range composeFiles {
raw, ok, err := readRepoFile(s, name)
if err != nil {
return nil, err
}
if !ok {
continue
}
var root yaml.Node
if err := yaml.Unmarshal(raw, &root); err != nil {
return nil, fmt.Errorf("%s: %w", name, err)
}
if len(root.Content) == 0 {
continue
}
services := mapValue(root.Content[0], "services")
if services == nil {
continue
}
for i := 0; i+1 < len(services.Content); i += 2 {
key, service := services.Content[i], services.Content[i+1]
if !runsThisRepo(service, s.ModulePath) {
continue
}
site := stopSite{
file: name,
line: key.Line,
what: fmt.Sprintf("service %s, which sets no stop_grace_period,", key.Value),
compose: true,
}
if grace := mapValue(service, "stop_grace_period"); grace != nil {
seconds, ok := composeSeconds(grace.Value)
if !ok {
// A duration this cannot read is left alone rather than
// guessed at: compose knows what it means, and inventing a
// number here would report against a value nobody wrote.
continue
}
site.line = grace.Line
site.what = fmt.Sprintf("stop_grace_period on service %s", key.Value)
site.seconds, site.set = seconds, true
}
out = append(out, site)
}
}
return out, nil
}
// runsThisRepo reports whether a compose service starts the image this
// repository builds - by building it, or by naming it.
func runsThisRepo(service *yaml.Node, modulePath string) bool {
if mapValue(service, "build") != nil {
return true
}
image := mapValue(service, "image")
if image == nil {
return false
}
repository := image.Value
if i := strings.LastIndex(repository, ":"); i > strings.LastIndex(repository, "/") {
repository = repository[:i]
}
return baseName(repository) == baseName(modulePath)
}
func baseName(path string) string {
if i := strings.LastIndex(path, "/"); i >= 0 {
return path[i+1:]
}
return path
}
func composeSeconds(value string) (int, bool) {
m := composeDuration.FindStringSubmatch(strings.TrimSpace(value))
if m == nil || m[1]+m[2]+m[3] == "" {
return 0, false
}
var total int
for i, unit := range []int{3600, 60, 1} {
if m[i+1] == "" {
continue
}
n, err := strconv.Atoi(m[i+1])
if err != nil {
return 0, false
}
total += n * unit
}
return total, true
}
// mapValue returns the value a mapping node holds for key.
func mapValue(node *yaml.Node, key string) *yaml.Node {
if node == nil || node.Kind != yaml.MappingNode {
return nil
}
for i := 0; i+1 < len(node.Content); i += 2 {
if node.Content[i].Value == key {
return node.Content[i+1]
}
}
return nil
}
+484
View File
@@ -0,0 +1,484 @@
package main
import (
"strings"
"testing"
)
// hostConfigSource is the part of config/extend.go this check reads: the
// fallbacks it applies to whatever the settings file leaves out.
const hostConfigSource = `package config
const (
DefaultDrainSeconds = 0
DefaultServerSeconds = 5
DefaultCleanupSeconds = 3
)
`
// factorySettings is what this repository ships: 0 + 5 + 3.
const factorySettings = "settings:\n extend:\n shutdown:\n drain: 0\n server: 5\n cleanup: 3\n"
func settingsWith(shutdown string) string {
return "settings:\n extend:\n" + shutdown
}
// deployWith builds a manifest with the given container extras and pod-level
// lines, in the shape the shipped one has.
func deployWith(containerExtra, podExtra string) string {
return `---
apiVersion: v1
kind: Service
metadata:
name: go-admin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: go-admin-v1
spec:
replicas: 1
template:
spec:
containers:
- name: go-admin
image: go-admin
` + containerExtra + podExtra
}
func graceOf(seconds string) string {
return " terminationGracePeriodSeconds: " + seconds + "\n"
}
const preStopSleep25 = ` lifecycle:
preStop:
exec:
command: ["sh", "-c", "sleep 25"]
`
// The six scenarios worked through in the technical plan, plus the one that
// only fails when preStop is left out of the sum.
//
// The values matter. "raise server to 25" gives 28, which is under a grace
// period of 30 and reaches only the WARN level - it would not show that the
// ERROR level works at all.
func TestShutdownBudgetAgainstTheGracePeriod(t *testing.T) {
for _, tc := range []struct {
name string
settings string
deploy string
want Severity
contains string
}{
{
name: "the shipped defaults, with headroom",
settings: factorySettings,
deploy: deployWith("", graceOf("30")),
want: -1,
},
{
// The one this check exists for: drain is the interesting knob and
// the grace period is in another directory, so raising one and not
// the other is the natural mistake.
name: "the budget was raised and the manifest was not",
settings: settingsWith(" shutdown:\n drain: 0\n server: 30\n cleanup: 3\n"),
deploy: deployWith("", graceOf("30")),
want: Error,
contains: "preStop 0 + drain 0 + server 30 + cleanup 3",
},
{
// Equal is not a fit: the grace period is when SIGKILL is sent, so
// a budget that ends exactly then leaves nothing time to return.
name: "the grace period was lowered to the budget",
settings: factorySettings,
deploy: deployWith("", graceOf("8")),
want: Error,
},
{
name: "fits, but with nothing to spare",
settings: factorySettings,
deploy: deployWith("", graceOf("12")),
want: Warn,
contains: "leaves under 5s of headroom",
},
{
// The hook is spent before the process is told anything, so it is
// added to the budget rather than overlapping it.
name: "a preStop hook is part of the budget",
settings: factorySettings,
deploy: deployWith(preStopSleep25, graceOf("30")),
want: Error,
contains: "preStop 25 + drain 0 + server 5 + cleanup 3",
},
{
// The example from the review: 10 + 10 + 5 + 3 against 30.
name: "preStop and a drain window together, just fitting",
settings: settingsWith(" shutdown:\n drain: 10\n server: 5\n cleanup: 3\n"),
deploy: deployWith(` lifecycle:
preStop:
exec:
command: ["sleep", "10"]
`, graceOf("30")),
want: Warn,
},
{
name: "no shutdown section",
settings: settingsWith(" rateLimit:\n inboundQPS: 200\n"),
deploy: deployWith("", graceOf("30")),
want: -1,
},
{
// Nothing to disagree with. A manifest without a grace period gets
// the Kubernetes default, which this file cannot see, and guessing
// at it would make the check wrong rather than quiet.
name: "the manifest sets no grace period",
settings: settingsWith(" shutdown:\n drain: 300\n"),
deploy: deployWith("", ""),
want: -1,
},
{
// config.Shutdown.Budget refuses this and the server does not
// start, so it is not a failure that passes unnoticed - and adding
// a negative into the sum would understate it.
name: "a negative budget is left to the run-time refusal",
settings: settingsWith(" shutdown:\n drain: -100\n server: 5\n cleanup: 3\n"),
deploy: deployWith("", graceOf("5")),
want: -1,
},
} {
t.Run(tc.name, func(t *testing.T) {
root := fixture(t, map[string]string{
"config/extend.go": hostConfigSource,
"config/settings.yml": tc.settings,
"scripts/k8s/deploy.yml": tc.deploy,
})
got := only(t, check(t, root, options{}), checkShutdownGrace)
if tc.want < 0 {
if len(got) != 0 {
t.Fatalf("reported %d findings, want none:\n%v", len(got), got)
}
return
}
if len(got) != 1 {
// An ERROR also satisfies the WARN condition, so a second
// finding here means the two levels were not made exclusive -
// and an ERROR that always drags a duplicate WARN behind it
// teaches people to skip WARNs.
t.Fatalf("reported %d findings, want exactly 1:\n%v", len(got), got)
}
if got[0].severity != tc.want {
t.Errorf("reported %s, want %s: %s", got[0].Severity, tc.want, got[0].Message)
}
if tc.contains != "" && !strings.Contains(got[0].Message, tc.contains) {
t.Errorf("message %q does not contain %q", got[0].Message, tc.contains)
}
if got[0].File != k8sDeployFile {
t.Errorf("reported against %s, want %s", got[0].File, k8sDeployFile)
}
if want := lineOf([]byte(tc.deploy), "terminationGracePeriodSeconds:"); got[0].Line != want {
t.Errorf("reported line %d, want %d", got[0].Line, want)
}
})
}
}
// A hook whose duration cannot be read is said out loud rather than counted as
// nothing. It is still spent inside the grace period, and a self-check that
// silently valued it at zero would be the understatement this check exists to
// prevent.
func TestAnUnreadablePreStopIsReported(t *testing.T) {
root := fixture(t, map[string]string{
"config/extend.go": hostConfigSource,
"config/settings.yml": factorySettings,
"scripts/k8s/deploy.yml": deployWith(` lifecycle:
preStop:
httpGet:
path: /drain
port: 8000
`, graceOf("30")),
})
got := only(t, check(t, root, options{}), checkShutdownGrace)
if len(got) != 1 {
t.Fatalf("reported %d findings, want 1:\n%v", len(got), got)
}
if got[0].severity != Warn {
t.Errorf("reported %s, want WARN", got[0].Severity)
}
if !strings.Contains(got[0].Message, "not a sleep") {
t.Errorf("message %q does not say why the hook could not be read", got[0].Message)
}
}
// Either file missing means there is nothing to compare, which is the state
// every other check's fixture is in.
func TestShutdownBudgetIsSkippedWithoutBothFiles(t *testing.T) {
for _, files := range []map[string]string{
{"config/extend.go": hostConfigSource},
{"config/extend.go": hostConfigSource, "config/settings.yml": settingsWith(" shutdown:\n drain: 300\n")},
{"config/extend.go": hostConfigSource, "scripts/k8s/deploy.yml": deployWith("", graceOf("30"))},
} {
root := fixture(t, files)
if got := only(t, check(t, root, options{}), checkShutdownGrace); len(got) != 0 {
t.Errorf("reported %d findings with only %d file(s):\n%v", len(got), len(files), got)
}
}
}
// A tool that cannot find the defaults it is meant to apply has to say so.
// Reporting nothing would be the failure this whole tool is about: a check
// that stops checking and goes on printing a clean run.
func TestShutdownBudgetStopsWhenTheFallbacksAreGone(t *testing.T) {
root := fixture(t, map[string]string{
"config/extend.go": "package config\n\nconst DefaultDrainSeconds = 0\n",
"config/settings.yml": settingsWith(" shutdown:\n drain: 1\n"),
"scripts/k8s/deploy.yml": deployWith("", graceOf("30")),
})
s, err := load(root)
if err != nil {
t.Fatalf("load: %v", err)
}
if _, err := runChecks(s, options{}); err == nil {
t.Fatal("runChecks succeeded with the fallback constants renamed away")
} else if !strings.Contains(err.Error(), serverConstName) {
t.Errorf("error %q does not name the missing constant", err)
}
}
// The same arithmetic and the same margin as the manifest check, against the
// other place a shutdown gets cut short.
func TestDockerStopAgainstTheShutdownBudget(t *testing.T) {
for _, tc := range []struct {
name string
script string
want Severity
contains string
}{
{
name: "explicit and generous",
script: "sudo docker stop --timeout 30 \"$PREV\"\n",
want: -1,
},
{
// --time is the deprecated spelling of the same flag and docker
// still honours it. A check that could not read it would report a
// deadline that exists as missing, and push whoever fixed that
// towards a flag that is on its way out.
name: "the deprecated spelling still counts",
script: "docker stop --time 30 go-admin\n",
want: -1,
},
{
name: "the short form counts too",
script: "docker stop -t 30 go-admin\n",
want: -1,
},
{
// docker's default is 10 and this process spends 8, so it happens
// to work today - and would stop working the first time anybody
// configures a drain window, without the command changing.
name: "no deadline at all",
script: "sudo docker stop \"$PREV\" >/dev/null\n",
want: Error,
contains: "Pass --timeout 13",
},
{
name: "shorter than the shutdown",
script: "docker stop --timeout 5 go-admin\n",
want: Error,
contains: "allows 5 seconds and this shutdown takes 8",
},
{
// Quoted back in the spelling that was written, so the message
// cannot misreport the line it is pointing at.
name: "the message quotes the flag that was used",
script: "docker stop -t 5 go-admin\n",
want: Error,
contains: "`docker stop -t 5`",
},
{
name: "longer than the shutdown but inside the margin",
script: "docker stop --timeout=10 go-admin\n",
want: Warn,
},
{
name: "exactly the margin",
script: "docker stop --timeout 13 go-admin\n",
want: -1,
},
{
// The settings file describes `docker stop` in prose right beside
// the budget this check reads.
name: "a commented-out command is not one that runs",
script: "# docker stop go-admin\n",
want: -1,
},
} {
t.Run(tc.name, func(t *testing.T) {
root := fixture(t, map[string]string{
"config/extend.go": hostConfigSource,
"config/settings.yml": factorySettings,
"scripts/deploy.sh": "#!/bin/sh\n" + tc.script,
})
got := only(t, check(t, root, options{}), checkDockerStop)
if tc.want < 0 {
if len(got) != 0 {
t.Fatalf("reported %d findings, want none:\n%v", len(got), got)
}
return
}
if len(got) != 1 {
t.Fatalf("reported %d findings, want 1:\n%v", len(got), got)
}
if got[0].severity != tc.want {
t.Errorf("reported %s, want %s: %s", got[0].Severity, tc.want, got[0].Message)
}
if tc.contains != "" && !strings.Contains(got[0].Message, tc.contains) {
t.Errorf("message %q does not contain %q", got[0].Message, tc.contains)
}
if got[0].File != "scripts/deploy.sh" || got[0].Line != 2 {
t.Errorf("reported %s:%d, want scripts/deploy.sh:2", got[0].File, got[0].Line)
}
})
}
}
func composeWith(service string) string {
return "version: '3.8'\nservices:\n" + service
}
// The compose file is the other way this repository's container is stopped -
// `make run` starts it that way - and it fails identically: the default is ten
// seconds and it is nowhere near the budget it has to cover.
func TestComposeStopGraceAgainstTheShutdownBudget(t *testing.T) {
for _, tc := range []struct {
name string
service string
want Severity
contains string
}{
{
name: "generous",
service: " api:\n image: go-admin:latest\n stop_grace_period: 30s\n",
want: -1,
},
{
name: "not set at all",
service: " api:\n image: go-admin:latest\n",
want: Error,
contains: "Set stop_grace_period: 13s",
},
{
name: "shorter than the shutdown",
service: " api:\n image: go-admin:latest\n stop_grace_period: 5s\n",
want: Error,
contains: "stop_grace_period on service api allows 5 seconds",
},
{
name: "longer than the shutdown but inside the margin",
service: " api:\n image: go-admin:latest\n stop_grace_period: 10s\n",
want: Warn,
},
{
// Compose takes hours and minutes as well as seconds, and a check
// that only read the digits would call 1m30s ninety times too
// short.
name: "minutes and seconds",
service: " api:\n image: go-admin:latest\n stop_grace_period: 1m30s\n",
want: -1,
},
{
// A service running something else is not this process, and its
// grace period has nothing to do with this budget.
name: "another image is not this application",
service: " db:\n image: mysql:8\n",
want: -1,
},
{
// Built from this repository, so it is this application whatever
// the image ends up being called.
name: "built here rather than named",
service: " api:\n build: .\n",
want: Error,
contains: "service api, which sets no stop_grace_period",
},
} {
t.Run(tc.name, func(t *testing.T) {
root := fixture(t, map[string]string{
"config/extend.go": hostConfigSource,
"config/settings.yml": factorySettings,
"docker-compose.yml": composeWith(tc.service),
})
got := only(t, check(t, root, options{}), checkDockerStop)
if tc.want < 0 {
if len(got) != 0 {
t.Fatalf("reported %d findings, want none:\n%v", len(got), got)
}
return
}
if len(got) != 1 {
t.Fatalf("reported %d findings, want 1:\n%v", len(got), got)
}
if got[0].severity != tc.want {
t.Errorf("reported %s, want %s: %s", got[0].Severity, tc.want, got[0].Message)
}
if tc.contains != "" && !strings.Contains(got[0].Message, tc.contains) {
t.Errorf("message %q does not contain %q", got[0].Message, tc.contains)
}
if got[0].File != "docker-compose.yml" {
t.Errorf("reported against %s, want docker-compose.yml", got[0].File)
}
})
}
}
func TestComposeDurations(t *testing.T) {
for _, tc := range []struct {
in string
want int
wantOK bool
}{
{in: "30s", want: 30, wantOK: true},
{in: "30", want: 30, wantOK: true},
{in: "1m30s", want: 90, wantOK: true},
{in: "2m", want: 120, wantOK: true},
{in: "1h", want: 3600, wantOK: true},
{in: "1h0m30s", want: 3630, wantOK: true},
{in: "", wantOK: false},
{in: "forever", wantOK: false},
{in: "500ms", wantOK: false},
} {
t.Run(tc.in, func(t *testing.T) {
got, ok := composeSeconds(tc.in)
if ok != tc.wantOK {
t.Fatalf("composeSeconds(%q) ok = %v, want %v", tc.in, ok, tc.wantOK)
}
if ok && got != tc.want {
t.Errorf("composeSeconds(%q) = %d, want %d", tc.in, got, tc.want)
}
})
}
}
// The command can be written in a workflow or in the Makefile as easily as in
// a shell script, and a check that only looked at one of them would be quiet
// about the others.
func TestDockerStopIsFoundInEveryKindOfFile(t *testing.T) {
root := fixture(t, map[string]string{
"config/extend.go": hostConfigSource,
"config/settings.yml": factorySettings,
".github/workflows/ship.yml": "jobs:\n deploy:\n steps:\n - run: docker stop app\n",
"Makefile": "stop:\n\tdocker stop app\n",
"scripts/deploy.sh": "docker stop app\n",
})
got := only(t, check(t, root, options{}), checkDockerStop)
if len(got) != 3 {
t.Fatalf("found %d commands, want 3:\n%v", len(got), got)
}
}
func TestLineOfIgnoresComments(t *testing.T) {
content := []byte("a: 1\n # terminationGracePeriodSeconds: 99\n terminationGracePeriodSeconds: 30\n")
if got := lineOf(content, "terminationGracePeriodSeconds:"); got != 3 {
t.Errorf("lineOf = %d, want 3 - a commented-out key is not the setting", got)
}
}