The five /sysjob routes use the generic actions. The detail route uses
ViewAs to keep answering SysJobItem, whose entryId the edit form reads
where the model marshals entry_id. GenerateM becomes ToModel, and the
Generate methods go.
Recorded before the change and replayed after it, the same 14 requests
get byte-identical responses; answering the detail with the model
instead of SysJobItem makes exactly the two detail requests differ.
With a real ToModel on both modules, a test now builds a package that
pairs the demo model with the jobs request and requires the build to
fail on that pairing, next to the same package with the pair corrected.
The five routes use actions.Index, View, Create, Update and Delete. The
control request's GenerateM becomes ToModel, returning the model type
itself, and the Generate methods on the model and the three request
types go, along with the tests that held them to returning copies:
nothing calls them any more.
Recorded before the change and replayed after it, the same 18 requests
- creates, lists with filters and paging, details, updates and deletes,
found and missing - get byte-identical responses.
Every generator route was reachable by any account that could log in:
the /gen and /db routes were in CasbinExclude, and the /sys/tables
routes were mounted without AuthCheckRole. Any user could read and
change any table's configuration and, in dev mode, generate files
from it.
All thirteen now go through AuthCheckRole. admin is let through as
before; another role needs the generator's menus, whose APIs the
previous commit binds and grants. The captcha, registered alongside
them, stays public.
The package, table and business names a table's configuration carries
become directories and file names, and nothing on the server checked
them; the config page's patterns were the only check.
Each is now checked against what it becomes: a package name is
lowercase letters and digits, a table name letters, digits and
underscores, a business name an identifier starting with a lowercase
letter. None allows a dot or a separator, and none is stricter than
the config page. The check runs on import, where a list holding one
bad name is refused whole, on save, and again before generating, so a
configuration saved before this check existed is refused rather than
trusted.
The generator joins a table's package, table and business names into
the paths it writes, and wrote them with os.MkdirAll and os.WriteFile.
A name carrying "..", or a symlink under the target directory pointing
elsewhere, took the write wherever it led.
Every file is now written through os.Root: backend files under the
working directory, frontend files under gen.frontpath. A path that
resolves outside its root is refused, symlinks included, and the
request reports the write failure without writing anything outside.
A table whose columns are all its key and ones the framework fills in
got an add button and an edit link opening an empty form. The page now
leaves out both, the dialog, useForm and the API calls only the form
used. A page with anything to enter is generated byte for byte as
before.
The config page hid both columns along with the ones the framework
fills in, so a generated page could not list or filter by when a row
was created or last changed without editing sys_columns by hand. They
are now shown; id, create_by, update_by and deleted_at stay hidden.
The templates already keep both out of the create/edit form.
The test imports a table, ticks list and query on created_at through
the config page's own endpoints, generates it, and filters the
generated list with the value the generated date picker sends.
dropFixture soft-deleted the sys_tables rows it cleaned up. The config
page's business-name check counts soft-deleted rows, so a second test
importing the same table could not save its configuration. The delete
endpoint removes these rows with Unscoped; the fixture now does too.
v2.11.0 makes pkg.FileCreate return its error instead of ending the
process when the file cannot be created. Two comments in the code
generator described the old behaviour in the present tense.
Preview, GenCode, GenApiToFile and GenMenuAndApi read a table's
configuration with tab, _ := table.Get(...), so an id that matched nothing,
or a failed query, carried on with an empty table. GenCode and Preview then
refused it as having no primary key, and GenMenuAndApi went on to seed menus
and APIs for it.
Each now answers with the lookup's error.
Preview, NOActionsGen and genApiToFile assigned each template's Execute
error to err and never read it - 19 times - so a template failing part-way
left a truncated file behind it and the request reported success. The
files were written through pkg.FileCreate, which returns no error at all:
when it cannot create a file it closes a nil *os.File and calls
log.Fatalln, ending the server process. Four of the directories it writes
into were created with _ = pkg.PathCreate(...).
Every template now renders into memory first; if any fails, nothing is
written and the response names the template. Files are then written
through writeGenerated, which creates the directory and returns any error
instead of exiting. NOActionsGen and genApiToFile report whether they
succeeded, so GenCode and GenApiToFile no longer follow an error response
with "Code generated successfully".
SysTables.Create and Update wrote each column with _, _ = …, so a column
the database rejected was dropped while the request reported success. And
because neither ran in a transaction, returning that error alone would not
have been enough: the table row and every column before the failing one
were already committed, leaving a table the config page lists with columns
missing and code generated from fewer columns than the table has.
Both now run the table and its columns in one transaction and return the
first column's error, named. Create rolls back to nothing; Update leaves the
table and every column as they were. Both callers already answer an error
with a 500.
The generator had no test that compiled what it writes; test/gen_test.go is
commented out. This one goes through the real path end to end: fixture
tables in MySQL, the import handler, GenCode writing its files, the files
compiled into this module's own packages through a -overlay, and a CRUD
test run over HTTP against the generated handlers.
The shapes are an int key named id, an int key named otherwise, a string
key, no key and a composite key. The last two must be refused with nothing
written. For the rest: read and update through the path parameter, a
missing key neither found nor created by an update, a key of "1=1" read as
a value, and delete by key.
The importer only reads MySQL, so the test needs GO_ADMIN_TEST_MYSQL_DSN;
without it the test skips locally and fails under CI. It also asserts each
generated CRUD test actually ran, since a -run pattern that matches nothing
exits 0.
The templates address a row by a single key: one field on the model, one
path parameter, one value per row in a delete. A table with no primary key
rendered a model whose GetId returned an empty field name, and a composite
key was silently narrowed to whichever of its columns the importer read
last. Both wrote files that break the build of the whole module.
Preview and NOActionsGen now check first and return an error naming the
table and, for a composite key, its columns. Nothing is written.
log.Error is the unformatted entry point, so "find dept list error, %s"
went out with the %s intact and the error text glued to it:
find dept list error, %sconnection refused
Errorf is what the line meant. go vet reports this once go-admin-core's
logger is recognised as a print-style wrapper, which it is from v2.10.0.
A second instance pointed at the same database did not divide the work, it
overwrote it. Every instance registered the whole enabled list into its own
cron.Cron, and startup ran `UPDATE sys_job SET entry_id = 0 WHERE entry_id
> 0` across the entire table, so the newest process erased the ids the
previous one wrote and put its own over the top. Neither symptom logged
anything: an enabled job fired once per instance, and stopping one from the
UI removed an entry from the wrong process and answered 200 either way.
A supervisor per tenant now keeps the scheduler in step with the lease.
Holding it is not decided once at startup, because both of the other
answers are wrong for longer than a moment:
- an instance that never got the lease keeps asking, so the death of the
holder does not stop the jobs until somebody restarts a process by hand;
- an instance that holds it stops scheduling as soon as its lease has
lapsed, because a holder still scheduling after the lease has gone
elsewhere is the two-schedulers defect reached from the other side.
A failed renewal is not a lost lease. The scheduler keeps running until the
lease could actually have expired: a database that is briefly unreachable
must not stop the jobs, and cannot have handed them to anyone else, because
nobody else can reach it to take the lease either.
Stopping is no longer arranged by startCrontab. A scheduler now stops for
two different reasons and only the supervisor knows which - and since a
start happens every time the lease is taken, registering a shutdown
callback there would add one per leadership change for the life of the
process, because SetShutdown appends.
sys_job.entry_id keeps its meaning. This does not distribute the jobs and is
not meant to: the HTTP side scales, the scheduler stays single-writer.
Fixes#915.
One row per database, taken and renewed by single UPDATE statements whose
RowsAffected the database decides. Nothing uses it yet; the scheduler is
wired to it next.
The two timestamps are epoch milliseconds in a BIGINT rather than timestamp
columns, which is the one decision here worth explaining. A timestamp does
not survive the trip through a driver unchanged: read over go-admin's own
`parseTime=True&loc=Local` DSN, MySQL's UTC_TIMESTAMP arrives relabelled as
local time, and on a UTC+8 host every lease is eight hours out. It is
invisible to any test that compares the lease against itself, because each
instance's own arithmetic stays self-consistent - only the comparison
between two instances is wrong, which is the only comparison that matters.
An integer has no timezone for a driver to apply.
The migration seeds the row free. There is no insert path at runtime, so two
instances starting together cannot race to create the row they are both
trying to claim, and neither has to tell a duplicate-key error apart from a
real one in whichever driver it is running against. A missing row is
therefore reported rather than recovered from: silently never scheduling
anywhere is the worse failure.
The current-time expression differs per dialect and all four are covered:
MySQL, PostgreSQL and SQL Server against real servers, SQLite by default.
jsonFieldPattern copied businessName's rule (^[a-z][A-Za-z]+$: at least
two letters, no digits) on the theory that jsonField should tighten to
the same identifier shape. That does not hold: businessName is typed
by a person on genInfoForm.vue, so a strict pattern is a reasonable
guardrail on human input. jsonField is computed by the importer from
the column name (sys_tables.go's namelist/JsonField loop) - nobody
types it, so the same pattern only rejected names the importer
legitimately produces: a single-letter column ("x") or one whose last
segment ends in a digit ("address2", "a1") both collapse to a single
camelCase word with nothing left to re-capitalize, and both failed the
old check.
The blast radius is wider than "this one column can't be edited":
validateAndSanitizeColumns runs over every column on every Update, so
a table that merely contains one such column could not save any
config change at all, including edits with nothing to do with that
column.
Relaxed to ^[a-z][A-Za-z0-9]*$ - any legal JS/TS identifier starting
with a lowercase letter. Still rejects what has to be rejected: empty,
whitespace/punctuation, and leading-digit names, since those cannot be
unquoted object keys in the generated interface/lang file at all.
Uniqueness and the expression-content check on defaultValue are
unchanged - defaultValue is genuinely user-typed (F6's config page),
so tightening it was the right call to begin with; this was the only
place a human-input rule had been copied onto machine-generated data.
Verified against the real import path, not hand-typed jsonField values:
built a table with columns id/x/address2/a1 in a fake information_schema,
ran it through the real SysTable.Insert, confirmed the importer computes
exactly jsonField x/address2/a1, then submitted an update through the
real SysTable.Update changing only tableComment (nothing about those
columns). Confirmed red first - 500, "jsonField 格式不合法：\"x\"" - a
change unrelated to any of the three columns was rejected solely because
they existed on the table. Restored the fix - 200, "修改成功".
MLTBName (table_name with underscores turned to dashes, e.g. "user_profile"
-> "user-profile") is a gorm:"-" field - table.Get never fills it in, the
caller has to. NOActionsGen has done so since it existed; Preview never
did, so every template's import path that reads it
(from '@/api/{PackageName}/{MLTBName}' in vue.go.template, present in
both the pre-Vue3 template and F4's rewrite) rendered with the module
segment missing - "from '@/api/admin/'" - in the preview dialog only.
The real generated file was always correct.
Pre-existing, not introduced by this PRD, but worth fixing now: F3/F9
added two more Preview panes (the language packs) on top of an assumption
- that Preview's output stands in for what NOActionsGen actually writes -
that was never true for this field. Checked the rest of gen.go for the
same "only set on the write path" shape; MLTBName is the only one -
every other field Preview's templates read comes straight off the
sys_tables/sys_columns rows table.Get already loads.
Verified with a throwaway harness driving Gen.Preview through a real gin
Context and sqlite-backed db, parsing the JSON response and reading back
template/vue.go.template's import line. Confirmed red first (temporarily
removed the added line): "from '@/api/verify010/'". Restored it: "from
'@/api/verify010/verify-widget'".
vue.go.template (F4) only had a flat fallback for an unconfigured
colWidth: 110 for datetime, 120 for everything else. R2 asks for a
precise per-type inference instead - varchar(n) tiered by n, tinyint(1)
narrower than a general integer, text/blob wide - sized so a typical
table's list columns land inside go-admin-ui's ~580px text-column
budget. text/template cannot parse "varchar(255)" itself, so this has
to run in gen.go before the template executes, not in the template.
Judgment is columnType, not goType, same as F1/F2's earlier reversal
(API契约.md §1.1): sys_tables.go:323-338 gives every non-primary-key
int/tinyint/bigint/decimal column goType "string", so goType alone
cannot tell a boolean flag from a bigint from a name column.
InferColumnWidth(columnType string) int is exported and pure per
测试用例.md §2.5's own request, so it can be pinned with an exact
input/output table rather than only asserting "the rendered page
happens not to overflow" - see column_width_test.go, including the
tinyint(1)-vs-tinyint(4) and the varchar tier-boundary cases.
applyInferredColumnWidths only touches a column already at the 0
sentinel, wired into both Preview and NOActionsGen ahead of the
template execute calls; a column already configured (by the user or F6)
is left untouched, also pinned by a test.
Verified against the real NOActionsGen call path (not just the unit
test): generated a table mixing every branch (tinyint(1), datetime,
decimal, varchar at three tiers, text, and one pre-configured column)
and printed the resulting tab.Columns[i].ColWidth after generation -
every value matched InferColumnWidth's own table, and the
pre-configured column's 333 was left untouched.
That bind-and-save path had zero field-level validation (API契约.md
§1.2/§2.1, decision D6), which left four ways to silently corrupt the
generator's own metadata or the language packs it writes:
- jsonField had no format rule at all, unlike tableName/className/
businessName, which all have a pattern check. A jsonField with a
dot, a leading digit, or anything not matching lower camelCase
still saved, and would land verbatim as a key segment in
gen/{PackageName}/{BusinessName}.ts.
- jsonField had no per-table uniqueness check. Two columns retitled
to the same jsonField overwrite each other's generated i18n key
with no warning.
- businessName had no cross-table uniqueness check within a
packageName. Two tables sharing one write the same language pack
path and the second's generation silently clobbers the first's.
- defaultValue is spliced into the generated defaultModel() as a
literal, never evaluated (API契约.md §2.1) - so content shaped
like a function call or block does not do what it looks like it
does, and previously saved without complaint either way.
colWidth is handled differently on purpose: API契约.md §2.1 says an
out-of-range value should fall back to the inferred width rather than
be rejected, so it is reset to the 0 sentinel in place instead of
failing the request - same outcome as if it had never been set.
G10 (colliding with the built-in admin/* i18n namespace) is not one of
the four: D9 already moved generated keys into their own gen/
namespace, so that collision no longer exists.
All four checks and colWidth's sanitize-in-place path are covered by
sys_tables_validate_test.go. Confirmed red first: swapped in a no-op
stand-in for both validators and reran - 13 sub-tests that should now
be rejected or sanitized passed straight through instead (jsonField
format x5, jsonField uniqueness x1, colWidth range x2, defaultValue
expression x4, businessName uniqueness x1). Restored the real
implementation and reran - all green.
Updates(&e) uses GORM's struct form, which skips zero-value fields -
but 0/"" is exactly the sentinel PRD 010 F1/F2 chose for "unconfigured"
(数据库变更.md §1.1). A caller resetting colWidth or defaultValue back
to that sentinel was therefore silently ignored: the row kept its old
value, the API reported success, and reopening the edit form showed the
stale number/string again.
Not fixed by switching to Select("*") - that would also start writing
this struct's other zero-valued fields (Sort, the Pk/Required/... bools),
none of which are part of this change. A second, map-form Updates
scoped to just these two columns writes the sentinel without touching
anything else; map-form Updates does not skip zero values.
TestSysColumnsUpdateClearsSentinelFields pins the regression: seed
150/"active", update to 0/"", read back and assert it stuck. Confirmed
red before this change (got 150/"active" back) and green after.
Add lang-zh.go.template and lang-en.go.template, one key per column
keyed by JsonField, valued from ColumnComment (R3) with a fallback to
JsonField when ColumnComment is blank (G11) - both locales resolve to
the same zh-CN text today since D3 rules out machine translation and
go-admin-ui's parity test rejects blank values.
NOActionsGen writes both files to lang/{locale}/gen/{PackageName}/
{BusinessName}.ts, two levels deep so go-admin-ui's gen-namespace.ts
glob (./*/*.ts under each locale's gen/) picks them up; nesting under
PackageName (rather than a flat gen/{BusinessName}.ts) avoids two
tables in different packages silently overwriting each other's
translations, since BusinessName has no uniqueness check.
Output has to be single-quoted with no trailing comma to satisfy
go-admin-ui's eslint config, which text/template's builtin `printf "%q"`
does not produce, hence the small parseGenTemplate/singleQuote helper
shared by the two templates.
Verified end to end: generated a real table's output (including a
blank-ColumnComment column, to exercise the G11 fallback), copied the
two language packs into go-admin-ui and ran vue-tsc --noEmit and eslint
against them - both clean, comma-dangle/quotes included.
Rewrite template/v4/js.go.template (renamed to ts.go.template) to emit
TypeScript matching go-admin-ui's src/api/demo/product.ts: an interface
per row and per query params, typed CRUD functions, and a del{Class}
that takes Id[] like useRemove expects. GoType maps to number for
int/int64/float32/float64 and string otherwise, since go-admin-ui's
request() types the {code,data,msg} envelope rather than the payload.
Preview's response map key changes from template/js.go.template to
template/api.ts.template so its Tab label matches the new content;
NOActionsGen writes the file with a .ts extension.
Verified end to end: generated a real table's output, copied it into
go-admin-ui and ran vue-tsc --noEmit and eslint against it - both clean.
Back the code generator's Vue 3 template migration: col_width lets R2's
column-width inference be overridden per field, and default_value lets
R1/A6's "unconfigured rows still generate a usable page" guarantee hold
for generated forms. Both use a sentinel default (0 / "") rather than
NULL so "unconfigured" has exactly one representation - see
docs-prd/010-代码生成器前端模板迁移Vue3/数据库变更.md §1.1.
The migration and the model change land together: AddColumn reads the
column definition off tools.SysColumns's own gorm tag, so splitting them
across commits would leave one of them failing to compile.
`gofmt -l` listed 26 files. Seventeen of them were missing the newline at the
end of the file; the rest are indentation that used spaces where the file uses
tabs, a handful of call sites written `f(a,b)`, and the doc comment spacing
gofmt has rewritten since 1.19 (`//X` to `// X`).
Nothing here changes behaviour: `go build ./...` and `go vet ./...` are clean
and `go test ./common/...` passes, which is the half of the tree these files
are concentrated in.
Only the files gofmt named are touched, so the diff reads line by line rather
than as a reflow of the whole repository. `gofmt -l` is now empty, which is the
precondition for gating it in CI -- worth doing, but a separate change.
The generator's import reads its comma-separated table list with
c.Request.FormValue("tables"), which on a request declaring itself as JSON reads
the URL query and nothing else. go-admin-ui v3.2.0 began sending that list in
the body, so the handler saw an empty string, asked information_schema for a
table named "", and every import failed with "table name cannot be empty！" —
on a fresh installation that is the first thing the generator is asked to do.
tablesToImport reads the query first and falls back to the body, so a front end
sending either works against this server. It also drops blank entries:
splitting "" yields one empty name rather than nothing at all, which is why the
old code reached a database query at all before failing.
The front end sends the list in the query again on its side; this half is what
lets an installation already running v3.2.0 recover without changing it.
The string was spelled out at each site that raises it, and once more in the
test file that asserts on it. A test holding its own copy cannot tell the
difference between the handler answering something else and the message having
been reworded: it goes on asserting a string the server no longer sends, and
goes on passing.
The three copies in app/other/models/tools are left alone; they are raised from
a different layer and nothing asserts on them.
newEngine takes the method, path and handler, so a second test file does not
have to restate the sqlite connection, the driver override and its cleanup, the
CustomError middleware and the two context keys. serveJSON does the same for
running one request and decoding the envelope.
Nothing about what is asserted changes; newColumnListEngine and columnListMsg
keep their names and their callers.
#924 buffered the channel Remove returns, so the goroutine it starts can
finish when the caller has already timed out and walked away. That half of
the patch was not covered: reverting the buffer on its own left the whole
suite green, while the timeout assertion it shipped with stayed red only for
the error-return half.
This test abandons twenty channels and then requires every goroutine to be
gone. It waits for the scheduler to empty first, so a goroutine that has not
yet reached its send cannot make the test pass while proving nothing.
Without the buffer: 20 of 20 parked on `chan send` at jobbase.go:216.
1786700008000 added sys_menu.seed_code and left it NULL on every row that was
already there. That is right for the host's own hand-placed menus: there is
nothing to derive one from.
An application's rows are in that population too, and for those it is
derivable - menu_name is what identified them before the column existed. The
natural-key lookup missed them, so a reseed inserted a second copy beside each
one, and the new unique index could not object, because NULL never collides on
MySQL, PostgreSQL or SQLite and is filtered out of the index on SQL Server.
Claimed when the application is seeded rather than by a backfill migration.
The value is only derivable where the spec's own Code is in hand: menuName
concatenates two pascalCase strings and does not reverse, so a migration
looking at menu_name alone would be guessing. For the same reason more than
one match is refused and named rather than picked from - attaching an
application's menu to whichever row the database returned first is the failure
this is meant to prevent, not a smaller version of it.
The match is scoped to the application's own app_code, so a row belonging to
another application, or to the host, is not claimed.
An adopted row then goes through the ordinary repair, so it comes out carrying
what the spec says rather than what it held from before.
Three degradations turn the new assertions red: not adopting at all, picking a
row when there is more than one, and dropping the app_code from the match.
repairExistingMenu reconciled a row's paths and its api bindings and left
every other column as an earlier run had written it. That cost three different
things, and none of them announced itself.
A menu whose parent was removed and seeded again kept parent_id pointing at
the dead row while its paths named the new one. The tree is built from
parent_id - SysMenu.GetPage walks down from ParentId == 0 - so the menu was
gone from the sidebar, with the migration reporting success.
A menu somebody added by hand under a seeded one kept the old prefix when its
ancestor moved. It is in no spec, so nothing else would ever rewrite it;
SysMenu.Update already does this cascade for the same column when a menu is
moved through the UI.
An application that renamed a menu, or moved its component, in a new version
had the change ignored: the row was found by its natural key and returned
untouched.
The row a spec describes now has one definition, and both the insert and the
repair use it - they cannot drift into disagreeing about what a spec decides.
The repair writes every column on that list.
Visible and IsFrame are deliberately not on it. They are seeding defaults the
application never expressed, so an administrator who hid a seeded menu keeps
it hidden; there is a test that hides one and reseeds.
The cascade matches the row itself or a row strictly underneath it, rather
than `paths LIKE old || '%'`, which also catches /0/1/20 when old is /0/1/2.
An empty old path takes the single-row branch instead: there is no subtree
under one, and the LIKE would have matched the whole table.
Five degradations turn the new assertions red: not writing the spec columns at
all, leaving ParentId off the list, putting Visible on it, not cascading, and
cascading on the loose prefix. The last one did not, at first - the decoy rows
were built against the path of the menu whose parent moved rather than the
path that actually gets rewritten, so the prefix they collided with was never
the one passed to the query.
Two comments added in this branch cite docs-prd/008-.../数据库变更.md by path.
That directory is not tracked here, so the citation reads as a file the reader
can open and cannot. The reasoning it pointed at is short enough to state in
place.
Three comments from the previous batch cite the same path and are left alone;
they belong to a different change.
uk_sys_menu_app_seed_code_del covers (app_code, seed_code, deleted_at) and is
created by 1786700008000 with explicit SQL. The struct tag named the same
index on SeedCode alone, and a named uniqueIndex tag collects only the fields
carrying that name - so AutoMigrate on this model would build a unique index
on seed_code by itself: stricter than the real one, and forbidding two
applications from both having a "dir" node, which the composite key exists to
allow.
Worse than being stricter, it would win. The migration only creates its index
when HasIndex says the name is free, so a schema built by AutoMigrate first
keeps the wrong index and the migration steps over it without a word.
The tag cannot express the real index: deleted_at comes from the ModelTime
embed shared by every table, which no single model can add a tag to. So the
tag goes and the migration is the only thing that creates it.
No database is affected. The initial table migration AutoMigrates a frozen
snapshot of this model that has neither app_code nor seed_code, and nothing
else in the repository AutoMigrates the live one - which is why this stayed
invisible until a test built the schema from the live model and seeded two
applications, and got a unique-constraint failure on a seed code they are
supposed to be able to share.
sys_app_casbin_grant has existed since the registry tables were added and
nothing ever wrote to it. An uninstaller reading it would have found it empty,
deleted no policy at all, and reported every one of them as an unattributable
leftover - which is what "report and skip" looks like when the ledger was
simply never written, and is indistinguishable from it working.
grantToAdminRole now writes an entry for each policy it creates. The entry
carries the tuple casbin_rule is unique on rather than a foreign key into it,
because casbin_rule is not this project's table: the gorm adapter's SavePolicy
truncates it and writes it back from memory, and SysRole.Update replaces a
role's policy rows wholesale. Both rebuild the same tuple from the same
sys_menu/sys_api data, so a match on the tuple survives what a row id does
not.
Only policies this install actually created are recorded - the insert is
conditional and its RowsAffected says which. A policy that was already there
was granted by somebody else and is not this app's to take away.
The two ways that can be wrong are not equally bad, which is what settles it.
Under-recording leaves a policy behind and the uninstall says so, because a
policy naming this app's own path with no ledger entry is exactly what it
reports as an orphan. Over-recording deletes somebody's authorization,
silently. Between a visible leftover and an invisible deletion, take the
leftover.
The ledger insert is itself conditional, for a case the obvious retry test
does not reach: on a plain re-run the policy still exists, so the insert is
skipped before the ledger is touched. It is reached when the policy row was
removed while its entry stayed, and a plain insert would then abort the whole
seed on the ledger's unique index. There is a test for that specific shape,
and replacing the insert with a plain one turns it red - which the plain retry
test does not.
Ordering: the ledger table is created by a framework migration, and version
strings sort bare digits ahead of any app-prefixed one, so it exists before
any application's seed runs. Nothing in the framework's own migrations calls
SeedMenus.
`migrate install <code>` brings one application up to the version its manifest
declares: it runs that application's outstanding migrations and records what
it did in sys_app.
It goes under migrate rather than under the existing `app` command, which
already means "generate the skeleton of a new application" - a directory that
does not exist yet, not an application already compiled into this binary.
Installing one is running its migrations, which is what this command is, so
--domain, resolveDB and the guard that refuses a mistyped code instead of
reporting a successful no-op are all already here.
Three phases, each committing on its own, and they are not one transaction.
An application's versions are separate migration files, and on MySQL a DDL
statement commits the transaction around it - destroying an outer transaction
and every savepoint taken from it. So this does not promise that a
half-installed application cannot happen. It promises one is visible when it
does: phase A writes "installing" before anything that can fail, phase B runs
the migrations, phase C turns that into "installed" or into "failed" with the
version it stopped on.
What is left to apply comes from sys_migration, never from sys_app. sys_app
is a derived view - a summary, and the answer to "which version does this app
think it is at". If it were the authority, an operator who deleted
sys_migration rows by hand would be told an application is installed while its
schema is not, which is worse than not knowing. So "already installed, nothing
to do" needs all three: nothing outstanding, recorded as installed, and the
same version. A row stuck at "installing" - what it reads as after the process
was killed partway - is not installed, and retrying is just running the
command again.
An upgrade is in place and keeps the first install's time; a downgrade is
refused, and refused before phase A writes anything, so a refusal cannot cost
the operator the row that told them what they had. An unparseable recorded
version is refused the same way, while it is still readable.
The report ends by saying the code is not running yet. That is not a
pleasantry: Go links at build time and Vite resolves its import globs at build
time, so installing an application writes its menus, its APIs and its
permissions and cannot make one line of its code run - and the menus appearing
is exactly what makes an operator believe otherwise.
Ten degradations were applied one at a time to check the tests name the
behaviour rather than the shape: deciding the no-op from sys_app alone,
always writing installed_at, allowing the downgrade, keeping the previous
attempt's diagnostics on a row that now says installed, treating "installing"
as installed, truncating last_error by bytes so a Chinese message is cut
mid-rune, not recording the failure at all, skipping code normalization, and
writing phase A before either the downgrade or the version-parse refusal.
Each went red on the assertion it was aimed at. An eleventh was discarded
rather than counted: it failed in the first install's setup, not on the claim.
The idempotency check added in the previous commit stopped a retry
inserting a second copy, and introduced a quieter failure in its place:
a retry that found an existing sys_menu row skipped everything after the
insert. Those are the sys_menu_api_rule bindings and the materialized
path, and neither is written by the statement that writes the menu -
paths is a separate UPDATE, and on MySQL an earlier DDL has already
committed the transaction that was supposed to hold them together.
So an install interrupted between those steps left a menu that exists,
sits outside the tree with an empty path, and is bound to no API. What
core's contract.md says about such a menu is that it is invisible to
every role and its apis are authorized for no one - while the installer
reports success.
Reusing now repairs. paths is compared before it is written, so a row
that is already right is not touched. Bindings are inserted with WHERE
NOT EXISTS rather than deleted and rebuilt: an administrator can bind an
api to a menu from the menu screen, and delete-then-rebuild would take
that with it on the next retry - the same accident as sys_role.go's
Association.Delete, pointing the other way.
Confirmed as a defect before it was fixed, by building the half-written
state and watching the assertions fail:
dir.Paths = "", want "/0/1"
binding count for list = 0, want 1
Four paths through the repair, each with a degradation that reds its own
test and leaves the others green: missing bindings only, missing paths
only, both, and neither. The fourth asserts no UPDATE is issued for a
row already correct.
A fifth covers what the repair must not do. Rebuilding bindings instead
of inserting them leaves every other test green while silently deleting
a binding this code did not create; that one now fails with "a retry
silently deleted a binding it does not own".
Bindings an older version of a manifest created and a newer one no
longer lists are left alone. Removing them is a delete, and a delete
needs the same certainty about ownership that uninstall does - this
function cannot tell a stale binding from one somebody added by hand.
seedApis and seedMenuTree were bare tx.Create calls. A migration that
failed partway and was run again re-inserted everything it had already
written - which is not hypothetical: the demo site collected eighteen
duplicate sys_menu rows this way, and three duplicate menus were visible
in its sidebar.
Both now look for a live row already holding the natural key and reuse
it. Only live rows count: a row an earlier soft-delete retired does not
stand in the way of a fresh insert under the same key, which is also
what the unique indexes allow.
The app_code half of each key has a test of its own. Without it the
lookups still passed every existing test while quietly letting one
application adopt another's rows - and an uninstall would then delete
rows the other application believed were its own, on both sides without
an error. Removing app_code from either lookup now fails with
"has 1 row(s) ... want 2 - one per app".
Seeding needs something to look for before it inserts, or a retry writes
a second copy of everything it already wrote. sys_api already had one in
(app_code, path, action). sys_menu had nothing usable: menu_name is
pascalCase(appCode) + pascalCase(code), which is not injective -
"list-all", "listAll" and "list_all" all become "ListAll" - so the
original code cannot be recovered from it. Hence a new column.
seed_code is nullable, against this repository's habit of NOT NULL
DEFAULT '' for a new column, and deliberately. Every row that predates
it has no meaningful value, and under a unique index an empty string
collides with every other empty string while NULL collides with nothing.
The convention exists because deleted_at's nullability broke a unique
index; here nullability is what makes one possible.
The unique index on sys_api cannot simply be created: a live database is
known to hold historical duplicates - the demo site had eighteen. The
migration looks first and refuses while naming the offending rows,
rather than letting CREATE UNIQUE INDEX fail with a constraint error
that names none. Same shape as 1786700003000's refuseOnDuplicates.
Run against SQLite, MySQL 8.0 and PostgreSQL 15, including the CONCAT
duplicate check, which had only ever been executed by SQLite's driver.
sys_app is one row per installed application. It is physically deleted
on uninstall rather than following the millisecond soft-delete marker
the other sys_ tables use: an installed-app registry has no "deleted by
accident, needs recovering" case, and a physical delete is what lets the
same code be installed again afterwards.
status is installing/installed/failed rather than a boolean, because an
install spanning several migration files is not atomic on MySQL - DDL
commits implicitly, so a run can stop in the middle. failed_version and
last_error are diagnostic snapshots for a person to read; nothing may
decide anything from them, and the field comments say so. Where to
resume is answered by sys_migration, which cannot drift from what was
actually applied.
sys_app_casbin_grant records which casbin_rule rows an install created,
keyed by casbin_rule's own natural key. That table is not extended
instead: gorm-adapter's SavePolicy truncates and reloads it from an
in-memory model, which would drop any column added here without a word.
Built against SQLite, MySQL 8.0 and PostgreSQL 15.
The warning said to set application.mode and stopped there. Following
that on a running process does not close anything: buildRouter has one
call site, in run(), and route registration is on no phase and no reload
callback, so a configuration reload moves the mode and leaves the routes
exactly where they were.
The reader is then worse off than before they acted. The mode now says
prod, GenWriteRoutesEnabled agrees, and the endpoints are still served -
so the one thing they could check to confirm the fix reports success
while the exposure is untouched, until something restarts the process.
A test pins the gap rather than the prose: build under dev, move the
mode to prod, and the routes are still in the engine. It fails if
registration ever becomes dynamic, which is the change that would make
the new sentence wrong.
That test degrades differently from the others - making it fail means
rewriting registration, not weakening it - so what was checked instead
is that it cannot go vacuous. Both of its premises are guarded: with the
gate always refusing it reports building under dev without the writing
routes, and with the gate always allowing it reports the predicate still
allowing prod. Neither failure can be mistaken for the assertion passing.
Raised by Copilot on #917.
registeredRoutes set config.ApplicationConfig.Mode and gave it back with
t.Cleanup, which runs at the end of the test rather than at the end of
the helper. Everything the caller did after the call therefore ran under
the mode the helper had been asked about, not one the caller chose.
Nothing was wrong yet: the one caller that reads the mode afterwards
sets it itself, and the two cleanups happen to unwind in an order that
leaves the right value. Both of those are accidents, and neither is
visible at the call site.
A defer inside the helper makes the borrowing end where it starts. The
doc comment said the mode was put back before returning while the code
did not, so that is now true rather than aspirational.
The counter-proof is the reason this has a test of its own: with
t.Cleanup back in place TestRegisteredRoutesRestoresTheModeBeforeReturning
fails and nothing else does, which is what a leak this quiet looks like
when something is actually watching for it.
Raised by Copilot on #917.
Three of the code generator's endpoints do not read. /gen/toproject
writes seven Go and Vue source files onto the host, one of them under
the path gen.frontpath names; /gen/apitofile writes a migration;
/gen/todb inserts menus and APIs. All three are GET, and all three are
listed in CasbinExclude - which AuthCheckRole skips - so Enforce never
runs for them. Any account that can log in could call them, on every
deployment.
They are now registered only where application.mode is dev or demo. dev
is the shipped default and is where the generator is meant to be used.
demo keeps them because demo mode already has a better answer than a
404: DemoEvn refuses these three by name and explains itself, which is
what the demo is for. test and prod get nothing, and so does a process
whose mode was never set.
This does not make the endpoints safe where they exist; it stops them
existing where nobody should be calling them. A host left on the shipped
dev is still open, which is why the next commit says so at start-up.
CasbinExclude is left alone on purpose. Taking the three off that list
would make them require a permission no existing deployment has granted,
so every non-admin user would start getting 403 from a tool that worked
yesterday. That is a migration, not a guard, and it belongs with a
release that can carry one.
Four counter-proofs, each red on the test that names the behaviour and
green everywhere else: a gate that always allows fails test/prod/unset
only; a gate that always refuses fails dev/demo and takes
TestEveryRouteDemoModeRefusesStillExists with it; moving a read-only
route inside the gate fails the reading test; and spelling the condition
at the registration site instead of calling the predicate fails the
agreement test, which is what keeps that test from being a tautology.
Two pre-existing deviations: a space before the comma in
sysNoCheckRoleRouter's parameter list, and no newline at end of file.
Separated from the change that follows so its diff is only the change.
zhangwenjian