Compare commits

...
Author SHA1 Message Date
zhangwenjian f406ca0160 test✅: fail loudly instead of skipping when the sqlite setup breaks
The privilege-escalation tests skipped themselves when opening the in-memory
database or running AutoMigrate failed. Both depend on nothing outside the
process, so a failure there means the environment is genuinely broken - and a
security regression that quietly does not run is worse than one that is
missing, because CI stays green either way.

Claude-Session: https://claude.ai/code/session_01HPTAw8b8tAdFNFn8rKdPYx
2026-09-04 19:10:53 +08:00
zhangwenjian 4d6456a588 test✅: cover vertical privilege escalation on sys-user update
Two directions, because the fix has to hold both: an attacker with no policy on
this route cannot raise another user's role, and a self-edit cannot raise its
own. The second one is what keeps the fix from being "just remove the route
from CasbinExclude", which would break the profile page.

The tests drive the handler directly rather than through the router, because
the middleware is exactly what does not run for this route - the defence lives
in the handler, so that is where it has to be proven.

Claude-Session: https://claude.ai/code/session_01HPTAw8b8tAdFNFn8rKdPYx
2026-09-04 17:23:07 +08:00
zhangwenjian 07ff92aa55 fix🐛: lock privileged fields on self-edit
The profile page posts the whole user object back, including roleId, deptId and
status, because it renders from a full SysUser it fetched earlier. A caller
editing their own record can therefore hand back a tampered roleId.

Self-edits now reload those three fields from the database and ignore whatever
the request carried. For an honest client this is a no-op - the values it sends
are already its own - so the profile page keeps working unchanged.

Claude-Session: https://claude.ai/code/session_01HPTAw8b8tAdFNFn8rKdPYx
2026-09-04 17:23:07 +08:00
zhangwenjian 4156387eb9 fix🐛: enforce Casbin when editing another user
PUT /api/v1/sys-user sits in CasbinExclude so the profile page can reach it,
which means AuthCheckRole never runs for this route. The handler took the
target user id from the request body, so any authenticated caller could edit
another user's record - including their roleId.

The route has to stay excluded: the profile page and the admin user list share
this one endpoint, so removing the exclusion would break self-service editing
for every non-admin role. The check therefore moves into the handler: when the
target is not the caller, the request is put through Casbin explicitly.

EnforceRoleFor carries the same admin short-circuit and enforcement AuthCheckRole
uses, so a route that opts out of the middleware can still ask the same question.

Claude-Session: https://claude.ai/code/session_01HPTAw8b8tAdFNFn8rKdPYx
2026-09-04 17:23:01 +08:00
wenjianzhang 34773a0a81 Merge pull request #894 from go-admin-team/chore/core-v2.4.1
Bump go-admin-core to v2.4.1
2026-09-01 20:47:11 +08:00
zhangwenjian 36a018400b chore🔧(deps): bump go-admin-core to v2.4.1
Documentation wording only; no code change between the two.
2026-09-01 20:42:36 +08:00
wenjianzhang 7bb02c5f1d Merge pull request #893 from go-admin-team/docs/contract-wording
Describe the rules rather than who follows them
2026-09-01 20:38:34 +08:00
zhangwenjian 15fb128236 docs📝: describe the rules rather than who follows them
The warning on Authorizator matters to anyone keeping a copy of that file, not
to one particular consumer, and it reads better addressed to all of them: check
what reads those context keys before taking this change.
2026-09-01 19:56:27 +08:00
wenjianzhang 3581e060ec Merge pull request #891 from go-admin-team/feat/003-app-prep
Groundwork for installable applications
2026-09-01 19:38:23 +08:00
zhangwenjian 0604a29596 feat✨(server): run the startup hooks through core
The package-level AppRouters slice keeps working and keeps running first, so
a fork that only ever appended to it sees no change. What is new is that the
core registry runs too, and that before callbacks run at all - this server
never had a loop for them.

Both go through core RunAppRouters / RunBefore, which brings the panic guard
and the seal with them.
2026-09-01 18:18:38 +08:00
zhangwenjian d8a2958797 chore🔧(deps): bump go-admin-core to v2.4.0 2026-09-01 18:18:38 +08:00
zhangwenjian ab28fa7bed docs📝: write down what a third-party app may depend on 2026-09-01 17:45:41 +08:00
zhangwenjian e88d751039 chore🔧: run the silent-failure checks in CI 2026-09-01 17:45:41 +08:00
zhangwenjian b836945eea feat✨: add checksilent, for the failures that do not report themselves
Six checks, five at ERROR and one - the cross-repository menu-name comparison -
at WARN, because it can only match by regular expression across two modules and
a false positive that fails CI teaches people to silence the tool.

The summary names which contract roots were actually scanned: core/ is a
separate module with no directory here, and a check that quietly covers less
than it claims is worse than no check.
2026-09-01 17:45:41 +08:00
zhangwenjian d7a8e66753 feat✨: add migrate status, --dry-run and --app
--app rejects a code nothing was registered under, on all three paths. It used
to take a typo as "nothing matched" and report success: migrate said the app
was unknown and still exited 0, while --dry-run and status printed the same
words an up-to-date database produces.
2026-09-01 17:45:40 +08:00
zhangwenjian 68780a845c feat✨: register migrations under an app code with ForApp 2026-09-01 17:45:40 +08:00
zhangwenjian 487dc94a2e feat✨: record on sys_migration which app a migration belongs to 2026-09-01 17:45:40 +08:00
zhangwenjian 016e977776 refactor🎨: drop Authorizator assertions that never matched
The map Authorizator receives is built by IdentityHandler in the same file and
carries IdentityKey / UserName / RoleKey / UserId / RoleIds / DataScope - not
user and role. Both assertions failed on every request, and because the ok
result was discarded the five c.Set calls stored zero values and the function
returned true anyway. Nothing in this repository or in core reads those keys.

go-admin-pro has its own copy of this file and does read them; this change
must not be carried over there verbatim.
2026-09-01 17:45:40 +08:00
zhangwenjian dcfe512204 refactor🎨: move the operation log status constants out of app/admin
common/middleware imported app/admin/service/dto for two string constants,
which made a package apps are told to build on depend on one particular app.
2026-09-01 17:45:40 +08:00
zhangwenjian fe6ebfd47c chore🔧: ignore the Go workspace files and the checksilent binary
go.work points this module at a local checkout of go-admin-core while the
two are developed together. It is a local tool and must never be committed:
CI resolves core from go.mod.

checksilent is where `go build ./tools/checksilent` drops its binary - four
megabytes beside the server one, which was already ignored by name.
2026-09-01 17:45:40 +08:00
wenjianzhang eba5fba3da Merge pull request #889 from go-admin-team/fix/password-hook-and-body-buffering
fix: a password hook that could destroy credentials, and a body copy on every request
2026-09-01 14:22:13 +08:00
zhangwenjian b7e9a79225 fix🐛: refuse an out-of-scope API update with the permission message
The previous commit added a data-permission scope to SysApi.Update and
returned early on db.Error, which left the RowsAffected check below it
unreachable: First reports a row the scope excluded as ErrRecordNotFound,
so the caller got "record not found" where the code meant to say
"无权更新该数据".

Map that one error to the permission message and drop the check it made
dead. The two cases - the row does not exist, and the row exists but is
not yours - have to look the same from outside, and now do.

Found by Copilot's review of #889.
2026-09-01 14:17:20 +08:00
wenjianzhang deffb19fd8 Merge pull request #888 from go-admin-team/ci/run-tests
Run the test suite in CI
2026-09-01 11:38:10 +08:00
zhangwenjian c858b322bd fix🐛: apply the data permission when updating an API
SysApi.Update took a DataPermission and never used it, so with data
permission enabled the update reached rows the caller could not read
through GetPage, Get or Remove, which all scope the query. It also
reported "无权更新该数据" for a row that simply did not exist, a message
that only becomes true once the scope is applied.

Drops the Debug() left on the query, which logged the statement for
every call.
2026-09-01 11:35:51 +08:00
zhangwenjian 1b5b52f0f1 perf👌: only read the request body when the operation log will store it
LoggerToFile is registered on the engine, so every POST, PUT, GET and
DELETE had its body copied into memory - through a bytes.Buffer, a
ReadAll and a string conversion - before any handler ran. The only
consumer is operParam on the operation-log row, which is written when
logger.enableddb is on, and that is off in the shipped configuration.

There was no size limit either, and a file upload is a POST like any
other: a 1MB request allocated 4.3MB here and a 16MB upload allocated
about 67MB, to build a value nobody stored.

The body is now read only when the operation log will use it, and at
most 32KB of it. The handler still receives the whole request: it reads
the copied part from memory and the rest from the connection, so what
this holds is bounded however large the request is. 32KB also keeps the
value inside the TEXT column it is written to.

The bufio.Writer this replaces was never flushed. Nothing was truncated
only because bytes.Buffer implements io.ReaderFrom, so io.Copy bypassed
the buffer entirely - a different destination would have dropped the
tail of every request body.
2026-09-01 11:35:45 +08:00
zhangwenjian ecb31a158b fix🐛: stop re-hashing a password that is already hashed
BeforeCreate and BeforeUpdate run Encrypt on whatever is in the struct,
and a user read from the database carries the stored hash in Password.
Hashing it again produces a hash of a hash: the password that user knows
stops matching, they cannot log in, and nothing reports an error.

Only the Omit("password") on SysUser.Update stood between that and the
stored credential. Any other write to this model - a profile update
written the way every other model here is written - destroys the
password, permanently and silently.

Encrypt now returns early when Password already parses as a bcrypt hash.
That also removes the round SysUser.Update was paying and discarding:
306ns where it was 54.7ms, on a route reachable without the permission
check, since PUT /api/v1/sys-user is in CasbinExclude.

The cost of deciding from the value is that a password which is itself a
well-formed bcrypt hash would be stored unchanged. That is a
60-character string beginning "$2a$", and it grants whoever set it no
access they did not already have.
2026-09-01 11:35:33 +08:00
zhangwenjian 1aecc140dc ci🔧: run the test suite on every push and pull request
The repository has 19 test files and nothing was running any of them. Both
workflows build with go build, which does not compile _test.go, the Makefile's
test target was commented out, and there is no pre-commit hook. Every test in
the tree, including the schema guards that exist precisely to catch a silent
breakage, only ran when someone remembered to type go test.

Enables the commented-out target and calls it from go.yml, the one workflow
that fires on every push and pull request. build.yml is left alone: it skips
documentation-only changes and deploys on master, so it is the wrong place for
a gate that should never be skipped.

Runs with -race. common/actions reuses model instances across concurrent
requests, so a Generate() that returns in place rather than a copy leaks data
between them, which a single-threaded run cannot see.

Verified locally: the suite passes under CGO_ENABLED=0 and under -race, and
make test exits non-zero when a test fails, so the step actually gates.

Claude-Session: https://claude.ai/code/session_01DJhM6LvhkNPej35wy9F7Aq
2026-09-01 11:33:08 +08:00
wenjianzhang e464a4aedd Merge pull request #887 from go-admin-team/fix/migration-model-soft-delete-drift
Guard new migrations against the frozen seed models
2026-08-31 15:06:05 +08:00
zhangwenjian d115c5299c docs📝: say which models package a new migration may seed through
The hazard had no signal at its point of contact. Someone adding a business
module is told to copy 1786700001000_demo_menu.go, which imports the frozen
seed models - correct for that file, wrong for anything ordered after the
soft-delete conversion. The frozen ModelTime carried no comment at all, so
opening it taught the reader nothing.

Documents the boundary in three places the author actually passes through:
the frozen type itself, the contributor guide, and the module-scaffolding
skill, which previously said to copy the reference file verbatim and now
says to copy its structure but not its imports.

Claude-Session: https://claude.ai/code/session_01DJhM6LvhkNPej35wy9F7Aq
2026-08-31 15:01:26 +08:00
zhangwenjian 9bd542bb59 test✅: guard post-conversion migrations against the frozen seed models
Migrations ordered after 1786700003000 must not seed through
cmd/migrate/migration/models. That package's ModelTime declares a nullable
gorm.DeletedAt, which is the shape the columns had until that migration
converted deleted_at to a NOT NULL millisecond marker.

Afterwards it breaks in both directions. Writes put NULL into a NOT NULL
column and fail on the first insert. Reads are scoped "WHERE deleted_at IS
NULL" while live rows hold 0, so they match nothing - and 1786700001000
looks the admin role up that way and treats ErrRecordNotFound as "roles are
not seeded yet, skip authorisation", which would leave a module seeded with
no permissions and the migration still recorded as applied.

A fresh database does not surface either one: every migration using that
package today is ordered before the conversion, so it runs while the column
is still nullable. Only a migration added afterwards hits it.

Also pulls the import scan out of importsRuntimeModels so both checks share
one implementation, and derives the version through migration.GetFilename
rather than a second filename-parsing rule.

Verified by adding a violating migration and confirming the test fails with
an actionable message, then removing it and confirming the suite passes.

Claude-Session: https://claude.ai/code/session_01DJhM6LvhkNPej35wy9F7Aq
2026-08-31 15:01:11 +08:00
42 changed files with 4298 additions and 87 deletions
+12 -2
View File
@@ -49,8 +49,18 @@ model、dto、router 三个文件,完整写法照抄 `app/demo/` 的结构。
### 4. 写菜单、接口与权限种子数据
这一步最容易被漏掉——代码能编译、接口能测通,但界面上看不到菜单、点了按钮说
没权限,往往就是漏了这一步。**完整参照 `cmd/migrate/migration/version/1786700001000_demo_menu.go`**
——那是可运行、幂等(用 `upsert`,重复跑不会报错)的真实例子,逐字照抄结构,只换 ID 和业务字段。
没权限,往往就是漏了这一步。结构参照 `cmd/migrate/migration/version/1786700001000_demo_menu.go`
——它是可运行、幂等(用 `upsert`,重复跑不会报错)的真实例子。
:::danger
**但不要照抄它的 import。** 那个文件用的是 `cmd/migrate/migration/models`,
只因为它的版本号排在软删除转换(`1786700003000`)之前才是安全的。
**你新写的迁移版本号在转换之后,必须改用 `app/` 下的运行时模型**
(`app/admin/models.SysApi`、`SysMenu`),否则第一条 insert 就会
`NOT NULL constraint failed: sys_api.deleted_at`。
`TestPostConversionMigrationsAvoidFrozenSeedModels` 会拦住这个错误。
:::
一个模块要在界面上可用,需要四类数据,缺一样都不行:
+14
View File
@@ -28,9 +28,23 @@ jobs:
- name: Get dependencies
run: go mod tidy
# go build does not compile _test.go, so building alone never ran a single
# test. This is the only workflow that fires on every push and pull request,
# which makes it the one place a test gate belongs.
- name: Test
run: make test
- name: Build
run: make build
# Fails the build on the silent-failure classes listed in
# tools/checksilent, one of which is the contract boundary: nothing under
# common/ may import app/. A boundary that is only written down erodes; this
# is what keeps it true.
- name: Silent-failure checks
run: make checksilent
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
if: startsWith(github.ref, 'refs/tags/')
+11
View File
@@ -6,6 +6,10 @@ main.exe
*.exe
go-admin
go-admin.exe
# `go build ./tools/checksilent` drops the binary here, next to the one for the
# server. Anchored with a leading slash: unanchored, the same pattern matches
# tools/checksilent/ as well and the tool's own source never gets committed.
/checksilent
temp/
!temp
vendor
@@ -29,3 +33,10 @@ CLAUDE.md
.claude/skills/*
!.claude/skills/new-business-module/
config/settings.local.dev.yml
# Go workspace files. They exist to point this module at a local checkout of
# go-admin-core while the two are developed together, which is a private
# arrangement between one machine's directories - committing one would break
# the build for everyone else.
go.work
go.work.sum
+51
View File
@@ -117,6 +117,18 @@ func (SysPost) TableName() string { return "sys_post" }
`TableName()` 必须显式声明(GORM 配置了 `SingularTable`,不会自动推导复数)。
## 公共契约面
第三方应用(`app/` 下的业务模块)可以稳定依赖哪些包、路由与迁移怎么注册、
哪些约束是硬的,见 `docs/contract.md`。
两条与主仓贡献者直接相关的:
- **`common/`、`core/` 不得 import `app/`** —— `make checksilent` 在 CI 里守着,违反即红。
- **注册类 API(`AppRouters` / `sdk.Runtime.SetAppRouters` / `migration.ForApp`)
只允许在 `init()` 中调用** —— 注册期靠 Go 的包初始化顺序保证无并发写,
core 侧的 setter 没有加锁。
## 路由注册
通过 `init()` 自注册,不在中心文件手工添加:
@@ -193,6 +205,45 @@ go run -tags sqlite3 . server -c config/settings.sqlite.yml
`git status` 看不到,PR 里也不会出现。两个目录的包名分别是 `version` 与
`version_local`(后者与目录名不一致,因为标识符不能含连字符)。
### 写种子数据用哪个 models 包
`1786700003000` 之后新增的迁移,**种子数据要用 `app/` 下的运行时模型**
(如 `app/admin/models.SysApi`、`SysMenu`),**不要用 `cmd/migrate/migration/models`**。
后者的 `ModelTime` 声明的是可空的 `gorm.DeletedAt`,这对它之前的迁移是对的(那正是
当时列的形状),转换之后就不再成立,两个方向都会出问题:
- **写**:往 NOT NULL 列里塞 NULL,第一条 insert 就 `NOT NULL constraint failed`
- **读**:GORM 拼 `WHERE deleted_at IS NULL`,而活跃行存的是 `0`,静默查不到——
照抄 `demo_menu.go` 的授权段落会因此跳过授权,菜单建好、权限没授、迁移仍记为成功
干净库跑不出这个问题,今天所有用该包的迁移都排在转换之前。完整推导见
`schema_coverage_test.go` 里 `TestPostConversionMigrationsAvoidFrozenSeedModels`
的注释,那个测试也守着这条边界。
## 静默失败校验
`make checksilent` 检查六类**不报错、不记日志、行为悄悄变得不对**的问题,
CI 会跑,命中 ERROR 即失败:
| 检查 | 级别 | 静默后果 |
|---|---|---|
| `modeltime-mix` | ERROR | 两个 `ModelTime` 混用,整张表查不到数据 |
| `menu-sort-overflow` | ERROR | 菜单 `sort` 超 127,MySQL tinyint 拒绝写入,迁移中断 |
| `config-value-truncation` | ERROR | `sys_config.config_value` 超 255 字符被静默截断 |
| `menu-id-collision` | ERROR | 两个模块硬编码同一菜单 ID,互相覆盖 |
| `contract-import-boundary` | ERROR | 契约包 import `app/`,应用无法独立编译 |
| `menu-name-mismatch` | WARN | 菜单名与前端组件 `name` 不一致,keep-alive 缓存静默失效 |
最后一条要跨仓库比对,只能做正则启发式,因此是 WARN,**不影响退出码**,
且默认跳过;要跑它得指定前端目录:
```bash
make checksilent UI_DIR=../go-admin-ui/src
```
升级门槛:连续 2 个发版周期零误报后转为 ERROR。
## 提交规范
格式 `type+emoji: 描述`:
+21 -3
View File
@@ -37,9 +37,27 @@ stop:
#@echo "go-admin stop success"
#.PHONY: test
#test:
# go test -v ./... -cover
# -race is worth the extra minute here: common/actions reuses model instances
# across concurrent requests, so a Generate() that returns in place instead of
# a copy leaks data between them - and that is invisible to a single-threaded
# test run.
.PHONY: test
test:
go test -race -cover ./...
# Reports the failures that do not announce themselves - see
# tools/checksilent. Exits non-zero on an ERROR; the one WARN-level check
# prints and does not fail the build.
#
# Pass UI_DIR to enable the cross-repository menu-name check, which is skipped
# without it: make checksilent UI_DIR=../go-admin-ui/src
.PHONY: checksilent
checksilent:
ifdef UI_DIR
go run ./tools/checksilent -ui-dir $(UI_DIR)
else
go run ./tools/checksilent
endif
#.PHONY: docker
#docker:
+26 -2
View File
@@ -1,6 +1,7 @@
package apis
import (
"errors"
"github.com/gin-gonic/gin/binding"
"go-admin/app/admin/models"
"golang.org/x/crypto/bcrypt"
@@ -15,6 +16,7 @@ import (
"go-admin/app/admin/service"
"go-admin/app/admin/service/dto"
"go-admin/common/actions"
"go-admin/common/middleware"
)
type SysUser struct {
@@ -149,12 +151,34 @@ func (e SysUser) Update(c *gin.Context) {
return
}
req.SetUpdateBy(user.GetUserId(c))
callerId := user.GetUserId(c)
// This route is in CasbinExclude so the personal-center screen can edit
// the caller's own record without a policy grant (see settings.go). That
// exclusion covers the whole route, not just the caller's own record, and
// the request carries the target userId in the body - so without this
// check here, any authenticated caller could edit any other user, up to
// and including their roleId. When the target is someone else, ask Casbin
// directly for the permission AuthCheckRole skipped.
if req.UserId != callerId {
allowed, err := middleware.EnforceRoleFor(c, c.Request.URL.Path, c.Request.Method)
if err != nil {
e.Logger.Error(err)
e.Error(500, err, err.Error())
return
}
if !allowed {
e.Error(http.StatusForbidden, errors.New("无权更新其他用户数据"), "对不起,您没有该接口访问权限,请联系管理员")
return
}
}
req.SetUpdateBy(callerId)
//数据权限检查
p := actions.GetPermissionFromContext(c)
err = s.Update(&req, p)
err = s.Update(&req, p, callerId)
if err != nil {
e.Logger.Error(err)
return
+175
View File
@@ -0,0 +1,175 @@
package apis
import (
"bytes"
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"github.com/gin-gonic/gin"
"github.com/glebarez/sqlite"
mycasbin "github.com/go-admin-team/go-admin-core/v2/casbin"
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
"github.com/go-admin-team/go-admin-core/v2/logger"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
"gorm.io/gorm"
"go-admin/app/admin/models"
)
// PUT /api/v1/sys-user is in settings.go's CasbinExclude so the
// personal-center screen (go-admin-ui's userInfo.vue) can edit the caller's
// own record without holding a policy grant on this route. AuthCheckRole
// skips Enforce entirely for an excluded route, so this file's job is to pin
// what the handler itself now has to hold shut: the target userId comes from
// the request body, and nothing upstream of the handler ever checked it
// against the caller.
// setupPrivescDB wires an in-memory database and a Casbin enforcer with an
// empty policy - the state of a fresh install for any role but admin - under
// a tenant unique to the calling test, so mycasbin's process-wide enforcer
// cache can't hand one test's database to another.
func setupPrivescDB(t *testing.T) (*gorm.DB, string) {
t.Helper()
// Fatalf, not Skipf: this database is in-memory sqlite with no external
// dependency, so failing to open or migrate it means the environment is
// actually broken. Skipping here would let these two anti-privesc
// regression tests silently stop running while CI stays green - a
// standing assertion that never fires is worse than no assertion.
db, err := gorm.Open(sqlite.Open("file::memory:"), &gorm.Config{})
if err != nil {
t.Fatalf("sqlite unavailable: %v", err)
}
if err := db.AutoMigrate(&models.SysUser{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
tenant := "sys-user-privesc-" + t.Name()
previousInterval := mycasbin.ReloadInterval
mycasbin.ReloadInterval = 0 // opt out of the background reload goroutine; the test never writes a policy
t.Cleanup(func() { mycasbin.ReloadInterval = previousInterval })
e := mycasbin.Setup(db, tenant)
previousEnforcer := sdk.Runtime.GetCasbinByTenant(tenant)
sdk.Runtime.SetCasbinByTenant(tenant, e)
t.Cleanup(func() { sdk.Runtime.SetCasbinByTenant(tenant, previousEnforcer) })
return db, tenant
}
// callUpdate drives SysUser.Update the way the router does for an
// authenticated, non-admin caller: JWT claims already decoded into the
// context (that is jwtauth's job, not this handler's) and a database - but
// without AuthCheckRole, since that middleware never runs Enforce for this
// route at all.
func callUpdate(t *testing.T, db *gorm.DB, tenant string, callerId int, body map[string]interface{}) *httptest.ResponseRecorder {
t.Helper()
gin.SetMode(gin.TestMode)
raw, err := json.Marshal(body)
if err != nil {
t.Fatalf("marshal request body: %v", err)
}
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Request = httptest.NewRequest(http.MethodPut, "/api/v1/sys-user", bytes.NewReader(raw))
c.Request.Host = tenant
c.Request.Header.Set("Content-Type", "application/json")
c.Set("db", db)
c.Set(pkg.LoggerKey, logger.NewHelper(logger.DefaultLogger))
c.Set(jwt.JwtPayloadKey, jwt.MapClaims{
"identity": float64(callerId),
"rolekey": "ordinary-role", // holds no Casbin policy anywhere in this test
})
SysUser{}.Update(c)
return w
}
// TestUpdate_CannotEscalatePrivilegeThroughAnotherUsersRecord is the
// regression for H6. Before the fix, an ordinary authenticated user could PUT
// a body naming another user's id and change that user's roleId - the route
// being Casbin-excluded meant no permission check ever ran, and the data
// permission scope that would otherwise gate this is off by default.
func TestUpdate_CannotEscalatePrivilegeThroughAnotherUsersRecord(t *testing.T) {
db, tenant := setupPrivescDB(t)
victim := models.SysUser{Username: "bob", NickName: "Bob", RoleId: 2, DeptId: 1, Status: "1"}
if err := db.Create(&victim).Error; err != nil {
t.Fatal(err)
}
attacker := models.SysUser{Username: "alice", NickName: "Alice", RoleId: 2, DeptId: 1, Status: "1"}
if err := db.Create(&attacker).Error; err != nil {
t.Fatal(err)
}
const elevatedRoleId = 1 // a role the attacker does not hold and has no policy for
callUpdate(t, db, tenant, attacker.UserId, map[string]interface{}{
"userId": victim.UserId,
"username": victim.Username,
"nickName": "pwned",
"phone": "13800000000",
"email": "bob@example.com",
"roleId": elevatedRoleId,
"deptId": victim.DeptId,
"status": victim.Status,
})
var after models.SysUser
if err := db.First(&after, victim.UserId).Error; err != nil {
t.Fatal(err)
}
if after.RoleId == elevatedRoleId {
t.Fatalf("an attacker with no Casbin permission on this route escalated the victim's roleId to %d", after.RoleId)
}
if after.NickName == "pwned" {
t.Fatalf("an attacker with no Casbin permission on this route modified another user's record: %+v", after)
}
}
// TestUpdate_SelfEditCannotChangePrivilegedFields covers the case the
// CasbinExclude entry exists for: the personal-center screen has to keep
// working for the caller's own record. The fields that screen exposes
// (nickName/phone/email/sex) must still save, while roleId/deptId/status stay
// whatever the database already had even if the request carries something
// else - a compromised or hand-crafted client is the only way that request
// would ever differ from what the honest form sends.
func TestUpdate_SelfEditCannotChangePrivilegedFields(t *testing.T) {
db, tenant := setupPrivescDB(t)
self := models.SysUser{Username: "carol", NickName: "Carol", RoleId: 2, DeptId: 1, Status: "1"}
if err := db.Create(&self).Error; err != nil {
t.Fatal(err)
}
const elevatedRoleId = 1
callUpdate(t, db, tenant, self.UserId, map[string]interface{}{
"userId": self.UserId,
"username": self.Username,
"nickName": "Carol Updated",
"phone": "13900000000",
"email": "carol@example.com",
"roleId": elevatedRoleId, // tampered; must not take effect
"deptId": self.DeptId,
"status": self.Status,
})
var after models.SysUser
if err := db.First(&after, self.UserId).Error; err != nil {
t.Fatal(err)
}
if after.RoleId == elevatedRoleId {
t.Fatalf("a self-edit changed the caller's own roleId to %d", after.RoleId)
}
if after.NickName != "Carol Updated" {
t.Fatalf("the legitimate personal-center edit did not go through: %+v", after)
}
}
+26 -14
View File
@@ -42,19 +42,35 @@ func (e *SysUser) GetId() interface{} {
return e.UserId
}
// Encrypt 加密
func (e *SysUser) Encrypt() (err error) {
// Encrypt hashes Password, unless it already holds a hash.
//
// The hooks below run on whatever is in the struct, and a user read from the
// database carries the stored hash in that field. Hashing it again produces a
// hash of a hash, and the password that user knows no longer matches anything:
// they cannot log in, and nothing reports an error. The only thing preventing
// that today is an Omit("password") on the one update that loads a user first,
// which makes every other write to this model one line away from destroying
// credentials.
//
// bcrypt.Cost parses a hash and fails on anything else, so it distinguishes
// the two cases without the call site having to say which it is. The cost is
// that a password which is itself a well-formed bcrypt hash would be stored
// unchanged - a 60-character string beginning "$2a$", not something a person
// types, and it grants whoever set it no access they did not already have.
func (e *SysUser) Encrypt() error {
if e.Password == "" {
return
return nil
}
if _, err := bcrypt.Cost([]byte(e.Password)); err == nil {
return nil
}
var hash []byte
if hash, err = bcrypt.GenerateFromPassword([]byte(e.Password), bcrypt.DefaultCost); err != nil {
return
} else {
e.Password = string(hash)
return
hash, err := bcrypt.GenerateFromPassword([]byte(e.Password), bcrypt.DefaultCost)
if err != nil {
return err
}
e.Password = string(hash)
return nil
}
func (e *SysUser) BeforeCreate(_ *gorm.DB) error {
@@ -62,11 +78,7 @@ func (e *SysUser) BeforeCreate(_ *gorm.DB) error {
}
func (e *SysUser) BeforeUpdate(_ *gorm.DB) error {
var err error
if e.Password != "" {
err = e.Encrypt()
}
return err
return e.Encrypt()
}
func (e *SysUser) AfterFind(_ *gorm.DB) error {
+106
View File
@@ -0,0 +1,106 @@
package models
import (
"testing"
"golang.org/x/crypto/bcrypt"
)
const knownPassword = "correct-horse-battery-staple"
// A user loaded from the database carries the stored hash in Password, and the
// hooks run on whatever is in the struct. Hashing it a second time produces a
// hash of a hash: the password the user knows stops matching, they cannot log
// in, and nothing reports an error.
//
// Only an Omit("password") on one call site stood between this and every write
// to the model. This is the test that removes the need for it.
func TestEncryptLeavesAnAlreadyHashedPasswordAlone(t *testing.T) {
fresh := SysUser{Password: knownPassword}
if err := fresh.Encrypt(); err != nil {
t.Fatalf("Encrypt: %v", err)
}
stored := fresh.Password
if err := bcrypt.CompareHashAndPassword([]byte(stored), []byte(knownPassword)); err != nil {
t.Fatalf("setup failed: the password was not hashed: %v", err)
}
// What a query puts in the struct, and what an update then hands the hook.
loaded := SysUser{Password: stored}
if err := loaded.Encrypt(); err != nil {
t.Fatalf("Encrypt on a loaded user: %v", err)
}
if loaded.Password != stored {
t.Error("Encrypt re-hashed a stored hash; the user can no longer log in")
}
if err := bcrypt.CompareHashAndPassword([]byte(loaded.Password), []byte(knownPassword)); err != nil {
t.Errorf("the user can no longer log in with their password: %v", err)
}
}
// The other half: a password that is not a hash still gets hashed, on create
// and on update alike.
func TestEncryptHashesAPlaintextPassword(t *testing.T) {
for _, c := range []struct {
name string
hook func(*SysUser) error
}{
{"BeforeCreate", func(u *SysUser) error { return u.BeforeCreate(nil) }},
{"BeforeUpdate", func(u *SysUser) error { return u.BeforeUpdate(nil) }},
} {
t.Run(c.name, func(t *testing.T) {
u := SysUser{Password: knownPassword}
if err := c.hook(&u); err != nil {
t.Fatal(err)
}
if u.Password == knownPassword {
t.Fatal("the password was stored as it was typed")
}
if err := bcrypt.CompareHashAndPassword([]byte(u.Password), []byte(knownPassword)); err != nil {
t.Errorf("the stored value does not verify the password: %v", err)
}
})
}
}
// An empty Password means "not being set", and must not become a hash of "".
func TestEncryptIgnoresAnEmptyPassword(t *testing.T) {
u := SysUser{}
if err := u.Encrypt(); err != nil {
t.Fatal(err)
}
if u.Password != "" {
t.Errorf("an unset password became %q", u.Password)
}
}
// Encrypt runs on every update of this model, including the ones that change
// something else entirely. What it costs when there is nothing to do is the
// difference between a profile update and a bcrypt round; the correctness test
// above is what catches a regression, this reports the size of it.
func BenchmarkEncrypt(b *testing.B) {
fresh := SysUser{Password: knownPassword}
if err := fresh.Encrypt(); err != nil {
b.Fatal(err)
}
b.Run("already hashed", func(b *testing.B) {
u := SysUser{Password: fresh.Password}
b.ReportAllocs()
for i := 0; i < b.N; i++ {
if err := u.Encrypt(); err != nil {
b.Fatal(err)
}
}
})
b.Run("plaintext", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
u := SysUser{Password: knownPassword}
if err := u.Encrypt(); err != nil {
b.Fatal(err)
}
}
})
}
+7 -2
View File
@@ -5,12 +5,17 @@ import (
"go-admin/app/admin/models"
"go-admin/common/dto"
"go-admin/common/global"
common "go-admin/common/models"
)
// Deprecated: use global.OperaStatusEnabled / global.OperaStatusDisabled.
// These two names are kept - misspelling and all - because forks import them;
// the values moved to common/global so common/middleware no longer has to
// import this package. See docs/contract.md.
const (
OperaStatusEnabel = "1" // 状态-正常
OperaStatusDisable = "2" // 状态-关闭
OperaStatusEnabel = global.OperaStatusEnabled // 状态-正常
OperaStatusDisable = global.OperaStatusDisabled // 状态-关闭
)
type SysOperaLogGetPageReq struct {
@@ -0,0 +1,24 @@
package dto
import (
"testing"
"go-admin/common/global"
)
// The values moved to common/global so common/middleware would stop importing
// this package; these two names stayed behind as aliases, misspelling and all,
// because forks import them.
//
// If they ever drift apart, rows written through the two spellings land in
// different buckets and the operation-log filter silently misses half of them.
func TestDeprecatedStatusAliasesStillMatch(t *testing.T) {
if OperaStatusEnabel != global.OperaStatusEnabled {
t.Errorf("OperaStatusEnabel = %q, global.OperaStatusEnabled = %q",
OperaStatusEnabel, global.OperaStatusEnabled)
}
if OperaStatusDisable != global.OperaStatusDisabled {
t.Errorf("OperaStatusDisable = %q, global.OperaStatusDisabled = %q",
OperaStatusDisable, global.OperaStatusDisabled)
}
}
+14 -3
View File
@@ -6,6 +6,8 @@ import (
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
"github.com/go-admin-team/go-admin-core/v2/sdk/service"
"gorm.io/gorm"
"go-admin/app/admin/models"
"go-admin/app/admin/service/dto"
"go-admin/common/actions"
@@ -74,9 +76,18 @@ func (e *SysApi) Get(d *dto.SysApiGetReq, p *actions.DataPermission, model *mode
// Update 修改SysApi对象
func (e *SysApi) Update(c *dto.SysApiUpdateReq, p *actions.DataPermission) error {
var model = models.SysApi{}
db := e.Orm.Debug().First(&model, c.GetId())
if db.RowsAffected == 0 {
return errors.New("无权更新该数据")
db := e.Orm.Scopes(
actions.Permission(model.TableName(), p),
).First(&model, c.GetId())
if err := db.Error; err != nil {
// First reports a row the data permission excluded exactly as it
// reports one that does not exist, and the caller should not be able
// to tell those apart either.
if errors.Is(err, gorm.ErrRecordNotFound) {
return errors.New("无权更新该数据")
}
e.Log.Errorf("Service UpdateSysApi error:%s", err)
return err
}
c.Generate(&model)
db = e.Orm.Save(&model)
@@ -0,0 +1,70 @@
package service
import (
"strings"
"testing"
"github.com/glebarez/sqlite"
"github.com/go-admin-team/go-admin-core/v2/logger"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
"github.com/go-admin-team/go-admin-core/v2/sdk/service"
"gorm.io/gorm"
"go-admin/app/admin/models"
"go-admin/app/admin/service/dto"
"go-admin/common/actions"
)
// An update the data permission excludes has to be refused, and refused in a
// way that does not tell the caller whether the row exists. First reports both
// cases the same way - no rows - so the message has to come from there rather
// than from a RowsAffected check the error return has already skipped past.
func TestSysApiUpdateRefusesARowOutsideTheDataPermission(t *testing.T) {
db, err := gorm.Open(sqlite.Open("file:sysapi-perm?mode=memory&cache=shared"), &gorm.Config{})
if err != nil {
t.Skipf("sqlite unavailable: %v", err)
}
if err := db.AutoMigrate(&models.SysApi{}); err != nil {
t.Skipf("automigrate: %v", err)
}
prev := config.ApplicationConfig.EnableDP
config.ApplicationConfig.EnableDP = true
t.Cleanup(func() { config.ApplicationConfig.EnableDP = prev })
// Owned by user 1.
row := models.SysApi{Handle: "h", Title: "t", Path: "/api/v1/probe", Type: "BUS", Action: "GET"}
row.CreateBy = 1
if err := db.Create(&row).Error; err != nil {
t.Fatal(err)
}
e := &SysApi{Service: service.Service{Orm: db, Log: logger.NewHelper(logger.DefaultLogger)}}
req := &dto.SysApiUpdateReq{Id: row.Id, Title: "changed"}
// User 2, scope 5: only rows they created.
outsider := &actions.DataPermission{DataScope: "5", UserId: 2, DeptId: 1, RoleId: 2}
err = e.Update(req, outsider)
if err == nil {
t.Fatal("the update was allowed on a row the data permission excludes")
}
if !strings.Contains(err.Error(), "无权更新该数据") {
t.Errorf("refused with %q, want the permission message; a raw database error tells the "+
"caller the row exists", err)
}
var after models.SysApi
if err := db.First(&after, row.Id).Error; err != nil {
t.Fatal(err)
}
if after.Title != "t" {
t.Errorf("the row was modified: title is now %q", after.Title)
}
// The owner still gets through, so the scope is refusing rather than
// everything failing.
owner := &actions.DataPermission{DataScope: "5", UserId: 1, DeptId: 1, RoleId: 1}
if err := e.Update(&dto.SysApiUpdateReq{Id: row.Id, Title: "by owner"}, owner); err != nil {
t.Fatalf("the owner could not update their own row: %v", err)
}
}
+15 -1
View File
@@ -84,7 +84,16 @@ func (e *SysUser) Insert(c *dto.SysUserInsertReq) error {
}
// Update 修改SysUser对象
func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission) error {
//
// callerId is who is asking, not who SetUpdateBy recorded - that field only
// says who to blame, it never constrained who could be edited. When the
// target is the caller themselves, roleId/deptId/status are kept at whatever
// the database already has no matter what the request body carries: this is
// the personal-center screen's route (see CasbinExclude in settings.go, and
// the check in the API handler ahead of this call), and letting a caller
// grant themselves a different role or department through it would be a
// privilege escalation the exclusion was never meant to open.
func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission, callerId int) error {
var err error
var model models.SysUser
db := e.Orm.Scopes(
@@ -98,6 +107,11 @@ func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission) err
return errors.New("无权更新该数据")
}
if model.UserId == callerId {
c.RoleId = model.RoleId
c.DeptId = model.DeptId
c.Status = model.Status
}
c.Generate(&model)
update := e.Orm.Model(&model).Where("user_id = ?", &model.UserId).Omit("password", "salt").Updates(&model)
if err = update.Error; err != nil {
+23 -3
View File
@@ -82,9 +82,7 @@ func run() error {
}
initRouter()
for _, f := range AppRouters {
f()
}
runStartupHooks()
srv := &http.Server{
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
@@ -154,6 +152,28 @@ func run() error {
return nil
}
// runStartupHooks runs the router registries and then the before callbacks.
//
// The package-level slice runs first and in its existing order, so a fork that
// only ever appended to AppRouters sees no change at all. The core registry
// runs second, through RunAppRouters: a module can register through
// sdk.Runtime.SetAppRouters and no longer has to import this command package -
// which is a main package's plumbing - just to be routed.
//
// The loop over the core registry now lives in core, which is what brings the
// panic guard and the registration seal with it. RunBefore closes a gap rather
// than moving one: the open-source edition never executed the before callbacks
// at all, so SetBefore was accepted and silently ignored. It has to stay ahead
// of ListenAndServe, because a callback registered WithFatal exits the process
// and that must not happen to one that is already serving.
func runStartupHooks() {
for _, f := range AppRouters {
f()
}
sdk.Runtime.RunAppRouters()
sdk.Runtime.RunBefore()
}
//var Router runtime.Router
func tip() {
+116
View File
@@ -0,0 +1,116 @@
package api
import (
"strings"
"testing"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
)
// freshRuntime hands the test its own Runtime and puts the old one back.
//
// Both registries close permanently the first time they are run, and
// sdk.Runtime is a process-wide singleton, so a test that runs the startup
// hooks would otherwise leave every later test in this binary registering into
// a closed registry - which is only an ERROR log, not a failure. The symptom
// is a test that passes alone and loses its routes when run with the others.
func freshRuntime(t *testing.T) {
t.Helper()
previous := sdk.Runtime
t.Cleanup(func() { sdk.Runtime = previous })
sdk.Runtime = runtime.NewConfig()
}
// Acceptance 1 and 2 together: the package-level slice a fork appends to and
// the core registry a module registers through both run, package-level first,
// registration order preserved inside each.
//
// The order matters beyond neatness. A module that appends to AppRouters has to
// import go-admin/cmd/api, which is why every module used to need a seven-line
// file in the command package; SetAppRouters is the way out of that. Running
// the old registry first is what makes the change invisible to anyone who never
// takes it.
func TestRunStartupHooksRunsBothRegistriesInOrder(t *testing.T) {
freshRuntime(t)
savedPackage := AppRouters
t.Cleanup(func() { AppRouters = savedPackage })
var order []string
AppRouters = []func(){
func() { order = append(order, "package-1") },
func() { order = append(order, "package-2") },
}
sdk.Runtime.SetAppRouters(func() { order = append(order, "runtime-1") })
sdk.Runtime.SetAppRouters(func() { order = append(order, "runtime-2") })
runStartupHooks()
const want = "package-1,package-2,runtime-1,runtime-2"
if got := strings.Join(order, ","); got != want {
t.Errorf("ran %q, want %q", got, want)
}
}
// Acceptance 17: a before callback registered through core actually runs.
//
// It did not, ever: core stored the callbacks and nothing executed them, so
// SetBefore was accepted and silently did nothing. The gap survived because
// core offered the registry without ever running it, leaving each consumer to
// write - or forget - its own loop.
func TestBeforeCallbacksRun(t *testing.T) {
freshRuntime(t)
savedPackage := AppRouters
t.Cleanup(func() { AppRouters = savedPackage })
AppRouters = nil
var order []string
sdk.Runtime.SetBefore(func() { order = append(order, "before-1") })
sdk.Runtime.SetBefore(func() { order = append(order, "before-2") })
sdk.Runtime.SetAppRouters(func() { order = append(order, "router") })
runStartupHooks()
// Routers first, then before: both happen ahead of ListenAndServe, and a
// router callback is what puts the engine in place for anything that comes
// after it.
const want = "router,before-1,before-2"
if got := strings.Join(order, ","); got != want {
t.Errorf("ran %q, want %q", got, want)
}
}
// A panicking module must not take the server down with it. The guard lives in
// core; this asserts that go-admin actually goes through it rather than around
// it with a loop of its own.
func TestAPanickingRouterDoesNotStopStartup(t *testing.T) {
freshRuntime(t)
savedPackage := AppRouters
t.Cleanup(func() { AppRouters = savedPackage })
AppRouters = nil
var order []string
sdk.Runtime.SetAppRouters(func() { order = append(order, "first") })
sdk.Runtime.SetAppRouters(func() { panic("a third-party module blew up") })
sdk.Runtime.SetAppRouters(func() { order = append(order, "third") })
sdk.Runtime.SetBefore(func() { order = append(order, "before") })
runStartupHooks()
const want = "first,third,before"
if got := strings.Join(order, ","); got != want {
t.Errorf("ran %q, want %q", got, want)
}
}
// The default AppRouters must keep the admin routes on it. Emptying the slice
// would not fail to compile anywhere - it would just serve a server with no
// admin API and no error.
func TestAdminRouterIsRegisteredOnThePackageSlice(t *testing.T) {
if len(AppRouters) == 0 {
t.Fatal("AppRouters is empty; the admin router is registered in init()")
}
}
+49
View File
@@ -0,0 +1,49 @@
package migrate
import (
"strings"
"testing"
"go-admin/cmd/migrate/migration"
)
// A mistyped --app used to be indistinguishable from an up-to-date database on
// all three paths: `migrate` printed that the app was unknown and exited 0,
// while `--dry-run` and `status` printed "nothing to apply" and "none
// recorded" - the same words a database with nothing pending produces. An
// operator scripting `migrate --app crmm && deploy` therefore deployed against
// a database the migrations never touched.
func TestAppRegistrationErrorRejectsAnUnknownCode(t *testing.T) {
restore := appCode
t.Cleanup(func() { appCode = restore })
appCode = "doesnotexist"
err := appRegistrationError()
if err == nil {
t.Fatal("an unregistered app code must be an error, not an empty run")
}
if !strings.Contains(err.Error(), `"doesnotexist"`) {
t.Errorf("the message must quote what was typed; got %q", err)
}
// Listing what is registered is what turns the error into a fix: the typo
// is usually one letter away from something in this list.
if !strings.Contains(err.Error(), migration.FrameworkAppCode) {
t.Errorf("the message must list the registered codes; got %q", err)
}
}
func TestAppRegistrationErrorAcceptsWhatIsRegistered(t *testing.T) {
restore := appCode
t.Cleanup(func() { appCode = restore })
for _, code := range []string{
"", // no --app at all: every migration runs
migration.FrameworkAppCode, // "core", the framework's own
strings.ToUpper(migration.FrameworkAppCode), // codes normalize to lower case
} {
appCode = code
if err := appRegistrationError(); err != nil {
t.Errorf("appCode %q must be accepted; got %v", code, err)
}
}
}
+259 -12
View File
@@ -1,21 +1,37 @@
package migration
import (
"fmt"
"log"
"path/filepath"
"sort"
"strings"
"sync"
"time"
"gorm.io/gorm"
common "go-admin/common/models"
)
var Migrate = &Migration{
version: make(map[string]func(db *gorm.DB, version string) error),
var Migrate = newMigration()
func newMigration() *Migration {
return &Migration{version: make(map[string]versionEntry)}
}
// versionEntry is one registered migration plus the app it belongs to. The
// empty app code means the framework itself, which is also what the
// sys_migration.app_code column defaults to, so history written before this
// field existed reads back correctly with no backfill.
type versionEntry struct {
appCode string
fn func(db *gorm.DB, version string) error
}
type Migration struct {
db *gorm.DB
version map[string]func(db *gorm.DB, version string) error
version map[string]versionEntry
mutex sync.Mutex
}
@@ -27,20 +43,247 @@ func (e *Migration) SetDb(db *gorm.DB) {
e.db = db
}
// SetVersion registers a migration owned by the framework. Signature and
// behaviour are unchanged: every existing call site in version/*.go keeps
// compiling and keeps writing common.Migration{Version: version} with no app
// code, which is the correct meaning of "framework".
func (e *Migration) SetVersion(k string, f func(db *gorm.DB, version string) error) {
e.mutex.Lock()
defer e.mutex.Unlock()
e.version[k] = f
e.setVersion(k, "", f)
}
func (e *Migration) Migrate() {
versions := make([]string, 0)
for k := range e.version {
func (e *Migration) setVersion(k, appCode string, f func(db *gorm.DB, version string) error) {
e.mutex.Lock()
defer e.mutex.Unlock()
e.version[k] = versionEntry{appCode: appCode, fn: f}
}
// AppMigrationFunc is the signature of a migration registered through ForApp.
//
// It receives appCode explicitly because the migration - not the framework -
// writes its own completion row, normally as the last statement inside its own
// transaction. That is what makes "the schema change and the record of it
// commit together" true, and the framework cannot insert the row on the
// migration's behalf without giving that up. Handing the code to the function
// is what stops an app's migrations from silently recording themselves as the
// framework's.
type AppMigrationFunc func(db *gorm.DB, version, appCode string) error
// AppRegistrar is a per-app view over a registry.
type AppRegistrar struct {
m *Migration
appCode string
}
// FrameworkAppCode is the name migrate status prints for migrations that belong
// to the framework rather than to an app, and the name --app accepts to select
// them. The stored app code for those is the empty string; this is only the
// spelling humans use. It is reserved - ForApp rejects it - so that every group
// heading status prints is also a value --app understands.
const FrameworkAppCode = "core"
// ForApp returns a registrar that records migrations under code.
//
// The code is lower-cased: sys_migration.version sorts as ASCII, so mixed case
// would order MyApp before crm for no reason a reader could guess, and the two
// spellings would group as two different apps in migrate status.
//
// An empty or reserved code panics rather than falling back to the framework.
// Registration happens in init(), so this fires the first time the binary runs
// anywhere, which is the point: an app whose migrations quietly file themselves
// under the framework is exactly the class of silent failure this work is meant
// to remove. Framework migrations call Migrate.SetVersion directly.
func ForApp(code string) *AppRegistrar { return Migrate.ForApp(code) }
// ForApp is the same on an explicit registry, which is what tests use.
func (e *Migration) ForApp(code string) *AppRegistrar {
code = NormalizeAppCode(code)
switch code {
case "":
panic("migration.ForApp: empty app code; framework migrations use Migrate.SetVersion")
case FrameworkAppCode:
panic("migration.ForApp: app code " + FrameworkAppCode + " is reserved for the framework")
}
return &AppRegistrar{m: e, appCode: code}
}
// AppCode reports the code this registrar files migrations under, after
// normalisation.
func (r *AppRegistrar) AppCode() string { return r.appCode }
// SetVersion registers an app-owned migration under k, which is the bare
// timestamp taken from the file name exactly as framework migrations do.
//
// What reaches sys_migration.version is the namespaced form; the version string
// handed to f is that same namespaced string, so a migration that writes
// common.Migration{Version: version, AppCode: appCode} records the key the
// registry will look for next time.
func (r *AppRegistrar) SetVersion(k string, f AppMigrationFunc) {
key := namespacedKey(r.appCode, k)
r.m.setVersion(key, r.appCode, func(db *gorm.DB, version string) error {
return f(db, version, r.appCode)
})
}
// namespacedKey scopes k to appCode so two apps cannot collide on the
// sys_migration.version primary key by minting the same millisecond timestamp.
// Framework migrations (appCode == "") stay bare, matching every version string
// already in production.
func namespacedKey(appCode, k string) string {
if appCode == "" {
return k
}
return appCode + "-" + k
}
// StatusEntry is one row of migrate status.
type StatusEntry struct {
AppCode string
Version string
Registered bool
Applied bool
ApplyTime *time.Time
}
// Status merges the in-process registry with sys_migration, so it reports all
// three shapes at once: registered but not applied, registered and applied, and
// applied while nothing registers it any more - a row left behind by a
// migration file that was deleted, or by an app that was uninstalled.
//
// It only reads. Nothing here creates or alters a table, which is what lets
// both `status` and `--dry-run` run against a database without touching it.
func (e *Migration) Status() ([]StatusEntry, error) {
if e.db == nil {
return nil, fmt.Errorf("migration: no database configured")
}
e.mutex.Lock()
registered := make(map[string]string, len(e.version))
for k, v := range e.version {
registered[k] = v.appCode
}
e.mutex.Unlock()
applied := make(map[string]common.Migration)
// A database that has never been migrated has no sys_migration table.
// Reporting everything as pending is the honest answer there; erroring out
// would make status useless in exactly the case it is most wanted.
if e.db.Migrator().HasTable(&common.Migration{}) {
var rows []common.Migration
if err := e.db.Find(&rows).Error; err != nil {
return nil, err
}
for _, r := range rows {
applied[r.Version] = r
}
}
versions := make(map[string]struct{}, len(registered)+len(applied))
for k := range registered {
versions[k] = struct{}{}
}
for k := range applied {
versions[k] = struct{}{}
}
list := make([]string, 0, len(versions))
for k := range versions {
list = append(list, k)
}
sort.Strings(list)
out := make([]StatusEntry, 0, len(list))
for _, v := range list {
entry := StatusEntry{Version: v}
if code, ok := registered[v]; ok {
entry.Registered = true
entry.AppCode = code
}
if row, ok := applied[v]; ok {
entry.Applied = true
t := row.ApplyTime
entry.ApplyTime = &t
if !entry.Registered {
// Nothing registers this version any more, so the database is
// the only source left for what it belonged to.
entry.AppCode = row.AppCode
}
}
out = append(out, entry)
}
return out, nil
}
// Migrate applies every registered migration that has not been applied yet,
// across all apps. Existing callers are unaffected.
func (e *Migration) Migrate() { e.run(allApps) }
// MigrateApp applies only the migrations registered under appCode. Pass
// FrameworkAppCode for the framework's own migrations.
func (e *Migration) MigrateApp(appCode string) { e.run(AppFilter(appCode)) }
// NormalizeAppCode applies the same rule ForApp does, so a code typed on the
// command line matches one written in an init().
func NormalizeAppCode(code string) string {
return strings.ToLower(strings.TrimSpace(code))
}
// AppFilter turns a code as typed into the code stored in the registry, so
// "core" selects the framework's migrations, whose stored code is empty.
func AppFilter(code string) string {
code = NormalizeAppCode(code)
if code == FrameworkAppCode {
return ""
}
return code
}
// DisplayAppCode is the inverse: what to print for a stored code.
func DisplayAppCode(code string) string {
if code == "" {
return FrameworkAppCode
}
return code
}
// AppCodes lists the app codes with at least one registered migration, framework
// included under its display name, sorted.
func (e *Migration) AppCodes() []string {
e.mutex.Lock()
seen := map[string]struct{}{}
for _, v := range e.version {
seen[DisplayAppCode(v.appCode)] = struct{}{}
}
e.mutex.Unlock()
out := make([]string, 0, len(seen))
for code := range seen {
out = append(out, code)
}
sort.Strings(out)
return out
}
func (e *Migration) run(appCode string) {
e.mutex.Lock()
versions := make([]string, 0, len(e.version))
entries := make(map[string]versionEntry, len(e.version))
for k, v := range e.version {
if appCode != allApps && v.appCode != appCode {
continue
}
versions = append(versions, k)
entries[k] = v
}
if !sort.StringsAreSorted(versions) {
sort.Strings(versions)
e.mutex.Unlock()
sort.Strings(versions)
// A mistyped --app would otherwise select nothing and report "no
// migrations to apply", which reads exactly like "already up to date".
if appCode != allApps && len(versions) == 0 {
log.Printf("no migrations are registered for app %q; registered: %s",
DisplayAppCode(appCode), strings.Join(e.AppCodes(), ", "))
return
}
var err error
var count int64
applied := 0
@@ -56,7 +299,7 @@ func (e *Migration) Migrate() {
continue
}
log.Printf("applying migration %s", v)
if err = (e.version[v])(e.db.Debug(), v); err != nil {
if err = entries[v].fn(e.db.Debug(), v); err != nil {
log.Fatalf("migration %s failed: %v", v, err)
}
applied++
@@ -68,6 +311,10 @@ func (e *Migration) Migrate() {
}
}
// allApps is the sentinel run() takes to mean "do not filter". It is distinct
// from the empty app code, which selects the framework's own migrations.
const allApps = "\x00all"
func GetFilename(s string) string {
s = filepath.Base(s)
return s[:13]
+390
View File
@@ -0,0 +1,390 @@
package migration
import (
"bytes"
"log"
"os"
"strings"
"testing"
"time"
"github.com/glebarez/sqlite"
"gorm.io/gorm"
"gorm.io/gorm/logger"
common "go-admin/common/models"
)
func newTestDB(t *testing.T) *gorm.DB {
t.Helper()
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
t.Fatalf("open sqlite: %v", err)
}
if err = db.AutoMigrate(&common.Migration{}); err != nil {
t.Fatalf("automigrate: %v", err)
}
return db
}
// recordFor is what an app's migration is expected to do: write its own
// completion row, with the version it was handed and the app code it was told
// it belongs to.
func recordFor(db *gorm.DB, version, appCode string) error {
return db.Create(&common.Migration{Version: version, AppCode: appCode}).Error
}
func rowsByVersion(t *testing.T, db *gorm.DB) map[string]common.Migration {
t.Helper()
var rows []common.Migration
if err := db.Find(&rows).Error; err != nil {
t.Fatalf("read sys_migration: %v", err)
}
out := make(map[string]common.Migration, len(rows))
for _, r := range rows {
out[r.Version] = r
}
return out
}
// Acceptance 9: a migration registered through ForApp("x") lands in
// sys_migration with app_code "x".
//
// The registry cannot write that row for the migration, because the row is the
// migration's own last statement inside its own transaction. So the only thing
// that can make this true is handing the code to the function - which is why
// AppMigrationFunc takes three parameters.
func TestForAppRecordsItsAppCode(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
return recordFor(db, version, appCode)
})
m.Migrate()
rows := rowsByVersion(t, db)
row, ok := rows["x-1786800001000"]
if !ok {
t.Fatalf("no row for x-1786800001000; got %v", rows)
}
if row.AppCode != "x" {
t.Errorf("app_code = %q, want %q", row.AppCode, "x")
}
}
// The framework path is untouched: same signature, and an empty app code, which
// is what the column defaults to and what every row written before this field
// existed reads back as.
func TestSetVersionStillRecordsTheFrameworkAsEmpty(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
return db.Create(&common.Migration{Version: version}).Error
})
m.Migrate()
rows := rowsByVersion(t, db)
row, ok := rows["1786700009000"]
if !ok {
t.Fatalf("no row for 1786700009000; got %v", rows)
}
if row.AppCode != "" {
t.Errorf("app_code = %q, want empty (framework)", row.AppCode)
}
}
// Acceptance 12: --app x runs x's migrations and touches nothing else.
func TestMigrateAppRunsOnlyThatApp(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
ran := map[string]bool{}
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
ran["core"] = true
return db.Create(&common.Migration{Version: version}).Error
})
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
ran["x"] = true
return recordFor(db, version, appCode)
})
m.ForApp("y").SetVersion("1786800002000", func(db *gorm.DB, version, appCode string) error {
ran["y"] = true
return recordFor(db, version, appCode)
})
m.MigrateApp("x")
if !ran["x"] {
t.Error("x did not run")
}
if ran["y"] || ran["core"] {
t.Errorf("MigrateApp(x) also ran %v", ran)
}
rows := rowsByVersion(t, db)
if len(rows) != 1 {
t.Fatalf("sys_migration has %d rows, want 1: %v", len(rows), rows)
}
}
// "core" is what status prints for the framework, so --app core has to select
// it. The stored code is the empty string; AppFilter is the translation.
func TestMigrateAppCoreSelectsTheFramework(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
ran := map[string]bool{}
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
ran["core"] = true
return db.Create(&common.Migration{Version: version}).Error
})
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
ran["x"] = true
return recordFor(db, version, appCode)
})
m.MigrateApp(FrameworkAppCode)
if !ran["core"] {
t.Error("framework migration did not run")
}
if ran["x"] {
t.Error("--app core also ran x")
}
}
// Zero-argument Migrate keeps meaning "everything", which is what every
// existing caller relies on.
func TestMigrateRunsEveryApp(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
var order []string
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
order = append(order, version)
return db.Create(&common.Migration{Version: version}).Error
})
m.ForApp("bbb").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
order = append(order, version)
return recordFor(db, version, appCode)
})
m.ForApp("aaa").SetVersion("1786800002000", func(db *gorm.DB, version, appCode string) error {
order = append(order, version)
return recordFor(db, version, appCode)
})
m.Migrate()
// Namespacing puts every framework migration - bare digits - ahead of every
// app migration, and orders apps by code rather than by whose timestamp
// happened to be smaller. aaa's file is the newer of the two and still runs
// first. Cross-app order is not promised, but this is the order, and it is
// the one to notice changed.
want := []string{"1786700009000", "aaa-1786800002000", "bbb-1786800001000"}
if len(order) != len(want) {
t.Fatalf("ran %v, want %v", order, want)
}
for i := range want {
if order[i] != want[i] {
t.Fatalf("ran %v, want %v", order, want)
}
}
}
// Two apps minting the same millisecond timestamp used to mean one of them was
// read as already applied and silently skipped. The namespace prefix is what
// makes that impossible without changing the primary key.
func TestNamespacingKeepsTwoAppsWithTheSameTimestampApart(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
const sameTimestamp = "1786800001000"
ran := 0
for _, app := range []string{"crm", "oms"} {
m.ForApp(app).SetVersion(sameTimestamp, func(db *gorm.DB, version, appCode string) error {
ran++
return recordFor(db, version, appCode)
})
}
m.Migrate()
if ran != 2 {
t.Errorf("ran %d migrations, want 2", ran)
}
rows := rowsByVersion(t, db)
for _, want := range []string{"crm-" + sameTimestamp, "oms-" + sameTimestamp} {
if _, ok := rows[want]; !ok {
t.Errorf("missing %s; got %v", want, rows)
}
}
}
func TestNamespacedKeyLeavesFrameworkVersionsBare(t *testing.T) {
if got := namespacedKey("", "1786700009000"); got != "1786700009000" {
t.Errorf("framework version was rewritten to %q", got)
}
if got := namespacedKey("crm", "1786800001000"); got != "crm-1786800001000" {
t.Errorf("namespacedKey = %q", got)
}
}
// An app code differing only in case would group as two apps in status and sort
// before every lower-case one, for no reason a reader could guess.
func TestForAppNormalisesTheCode(t *testing.T) {
m := newMigration()
if got := m.ForApp(" CRM ").AppCode(); got != "crm" {
t.Errorf("AppCode = %q, want crm", got)
}
}
func TestForAppRejectsReservedCodes(t *testing.T) {
for _, code := range []string{"", " ", FrameworkAppCode, "CORE"} {
t.Run("code="+code, func(t *testing.T) {
defer func() {
if recover() == nil {
t.Errorf("ForApp(%q) did not panic", code)
}
}()
newMigration().ForApp(code)
})
}
}
func TestStatusReportsPendingAppliedAndOrphaned(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
applied := time.Date(2026, 8, 25, 14, 3, 11, 0, time.UTC)
if err := db.Create(&common.Migration{Version: "1786700009000", ApplyTime: applied}).Error; err != nil {
t.Fatal(err)
}
// Recorded, but nothing registers it any more.
if err := db.Create(&common.Migration{Version: "gone-1786800000000", ApplyTime: applied, AppCode: "gone"}).Error; err != nil {
t.Fatal(err)
}
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error { return nil })
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error { return nil })
entries, err := m.Status()
if err != nil {
t.Fatal(err)
}
byVersion := map[string]StatusEntry{}
for _, e := range entries {
byVersion[e.Version] = e
}
if e := byVersion["1786700009000"]; !e.Applied || !e.Registered || e.AppCode != "" {
t.Errorf("framework entry = %+v", e)
} else if e.ApplyTime == nil || !e.ApplyTime.Equal(applied) {
t.Errorf("framework apply time = %v, want %v", e.ApplyTime, applied)
}
if e := byVersion["crm-1786800001000"]; e.Applied || !e.Registered || e.AppCode != "crm" {
t.Errorf("crm entry = %+v", e)
}
if e := byVersion["gone-1786800000000"]; !e.Applied || e.Registered || e.AppCode != "gone" {
t.Errorf("orphaned entry = %+v", e)
}
}
// Acceptance 11 rests on this: status and --dry-run both go through Status, and
// Status must not create the table it reads.
func TestStatusDoesNotCreateItsTable(t *testing.T) {
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
Logger: logger.Default.LogMode(logger.Silent),
})
if err != nil {
t.Fatal(err)
}
m := newMigration()
m.SetDb(db)
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error { return nil })
entries, err := m.Status()
if err != nil {
t.Fatalf("Status on a database with no sys_migration: %v", err)
}
if len(entries) != 1 || entries[0].Applied {
t.Errorf("entries = %+v, want one pending", entries)
}
if db.Migrator().HasTable(&common.Migration{}) {
t.Error("Status created sys_migration; it must only read")
}
}
// The completion row is the migration's own last statement, inside its own
// transaction. A migration that fails must leave no record of having run, or
// the next run skips it and the schema stays half-changed with nothing to say
// so.
func TestFailedMigrationLeavesNoRecord(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
return db.Transaction(func(tx *gorm.DB) error {
if err := recordFor(tx, version, appCode); err != nil {
return err
}
return errTestMigrationFailed
})
})
// run() calls log.Fatal on failure, which would take the test binary with
// it, so drive the registered function directly - the point here is the
// transaction boundary, not the scheduler.
entry := m.version["crm-1786800001000"]
if err := entry.fn(db, "crm-1786800001000"); err == nil {
t.Fatal("migration reported success")
}
if rows := rowsByVersion(t, db); len(rows) != 0 {
t.Errorf("sys_migration has %v after a failed migration", rows)
}
}
var errTestMigrationFailed = &testError{"boom"}
type testError struct{ s string }
func (e *testError) Error() string { return e.s }
// A mistyped --app used to select nothing and print "no migrations to apply",
// which reads as "already up to date" - the command reports success and does
// nothing, which is the failure mode this whole batch exists to remove.
func TestMigrateAppOnAnUnknownCodeSaysSo(t *testing.T) {
db := newTestDB(t)
m := newMigration()
m.SetDb(db)
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
return db.Create(&common.Migration{Version: version}).Error
})
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
return recordFor(db, version, appCode)
})
var buf bytes.Buffer
log.SetOutput(&buf)
t.Cleanup(func() { log.SetOutput(os.Stderr) })
m.MigrateApp("crmm")
if !strings.Contains(buf.String(), `no migrations are registered for app "crmm"`) {
t.Errorf("output = %q", buf.String())
}
if !strings.Contains(buf.String(), "registered: core, crm") {
t.Errorf("the message must list what is registered; got %q", buf.String())
}
if rows := rowsByVersion(t, db); len(rows) != 0 {
t.Errorf("a typo ran %v", rows)
}
}
+8
View File
@@ -15,6 +15,14 @@ type Model struct {
Id int `json:"id" gorm:"primaryKey;autoIncrement;comment:主键编码"`
}
// ModelTime is frozen at the schema shape these tables had before
// 1786700003000 converted deleted_at to a NOT NULL millisecond marker. That is
// correct for the migrations ordered before the conversion, and wrong for any
// added after it: writes put NULL into a NOT NULL column, and reads are scoped
// "WHERE deleted_at IS NULL" and match nothing.
//
// Migrations after that version seed through the runtime models in app/.
// TestPostConversionMigrationsAvoidFrozenSeedModels enforces this.
type ModelTime struct {
CreatedAt time.Time `json:"createdAt" gorm:"comment:创建时间"`
UpdatedAt time.Time `json:"updatedAt" gorm:"comment:最后更新时间"`
@@ -9,6 +9,8 @@ import (
"strconv"
"strings"
"testing"
"go-admin/cmd/migrate/migration"
)
// The repository carries two ModelTime types. The one in
@@ -79,9 +81,13 @@ func runtimeSoftDeleteTables(t *testing.T) map[string]string {
}
func importsRuntimeModels(f *ast.File) bool {
return importsPackage(f, "go-admin/common/models")
}
func importsPackage(f *ast.File, pkg string) bool {
for _, imp := range f.Imports {
p, err := strconv.Unquote(imp.Path.Value)
if err == nil && p == "go-admin/common/models" {
if err == nil && p == pkg {
return true
}
}
@@ -171,3 +177,89 @@ func repoRoot(t *testing.T) string {
t.Fatal("go.mod not found above the test directory")
return ""
}
// softDeleteConversion is the version at which sys_api, sys_menu and the rest
// stop storing deleted_at as a nullable timestamp and start storing the NOT
// NULL millisecond marker.
const softDeleteConversion = 1786700003000
// versionPrefixLen is the width migration.GetFilename slices off a filename.
const versionPrefixLen = 13
// Migrations ordered after the conversion must not seed rows through
// cmd/migrate/migration/models.
//
// That package's ModelTime still declares a nullable gorm.DeletedAt, which is
// correct for the migrations that predate the conversion - it is the shape the
// column had when they ran. Reusing it afterwards writes NULL into a NOT NULL
// column and the migration fails on its first insert:
//
// NOT NULL constraint failed: sys_api.deleted_at
//
// A fresh database never catches this, because every migration using that
// package today is ordered before the conversion and so runs while the column
// is still nullable. Only a migration added afterwards hits it, which in
// practice means the next person adding a business module - the reference
// they copy, 1786700001000_demo_menu.go, is itself one of the safe ones.
//
// Reads through that package are worse than writes, which is why the whole
// import is banned rather than just the inserts. gorm scopes a nullable
// DeletedAt as "WHERE deleted_at IS NULL", and after the conversion live rows
// hold 0, so the row is simply not there:
//
// frozen SysRole -> record not found
// runtime SysRole -> roleId=1
//
// 1786700001000_demo_menu.go looks the admin role up that way and treats
// ErrRecordNotFound as "roles are not seeded yet, skip authorisation". A
// post-conversion copy that switched its inserts to the runtime models but
// kept this lookup would seed the menu, grant nothing, and still record the
// migration as applied - the menu appears, its buttons do nothing, and no
// error is reported anywhere.
func TestPostConversionMigrationsAvoidFrozenSeedModels(t *testing.T) {
const frozenModels = "go-admin/cmd/migrate/migration/models"
dir, err := os.Getwd()
if err != nil {
t.Fatal(err)
}
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatal(err)
}
checked := 0
for _, e := range entries {
name := e.Name()
if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
continue
}
// GetFilename is what every migration uses to derive its own version,
// so the two stay in step if the filename convention ever changes.
if len(name) < versionPrefixLen {
continue
}
version, err := strconv.ParseInt(migration.GetFilename(name), 10, 64)
if err != nil || version <= softDeleteConversion {
continue // not a versioned migration, or one that predates the change
}
checked++
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, filepath.Join(dir, name), nil, parser.ImportsOnly)
if err != nil {
t.Fatalf("parse %s: %v", name, err)
}
if importsPackage(f, frozenModels) {
t.Errorf("%s is ordered after the soft-delete conversion but seeds through %s;\n"+
" that package writes a nullable deleted_at and will fail with\n"+
" \"NOT NULL constraint failed\" on its first insert.\n"+
" Use the runtime models under app/ instead - they carry the marker.",
name, frozenModels)
}
}
if checked == 0 {
t.Fatal("no post-conversion migrations found; the scan is broken, not the code")
}
}
+141 -8
View File
@@ -3,12 +3,16 @@ package migrate
import (
"bytes"
"fmt"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
"os"
"strconv"
"strings"
"text/template"
"time"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
"gorm.io/gorm"
"github.com/go-admin-team/go-admin-core/v2/config/source/file"
"github.com/spf13/cobra"
@@ -25,6 +29,8 @@ var (
generate bool
goAdmin bool
host string
appCode string
dryRun bool
StartCmd = &cobra.Command{
Use: "migrate",
Short: "Initialize the database",
@@ -33,14 +39,31 @@ var (
run()
},
}
statusCmd = &cobra.Command{
Use: "status",
Short: "List applied and pending migrations, grouped by app",
Example: "go-admin migrate status -c config/settings.yml",
Run: func(cmd *cobra.Command, args []string) {
runStatus()
},
}
)
// fixme 在您看不见代码的时候运行迁移,我觉得是不安全的,所以编译后最好不要去执行迁移
func init() {
StartCmd.PersistentFlags().StringVarP(&configYml, "config", "c", "config/settings.yml", "Start server with provided configuration file")
StartCmd.PersistentFlags().BoolVarP(&generate, "generate", "g", false, "generate migration file")
StartCmd.PersistentFlags().BoolVarP(&goAdmin, "goAdmin", "a", false, "generate go-admin migration file")
StartCmd.PersistentFlags().BoolVarP(&goAdmin, "goAdmin", "a", false, "with -g, write the generated file to version/ instead of version-local/ (does not affect which migrations run)")
StartCmd.PersistentFlags().StringVarP(&host, "domain", "d", "*", "select tenant host")
// --app is deliberately long-only. -a already means "generate into
// version/ rather than version-local/", which is about writing a template
// file, not about which migrations run; giving the two the same letter
// would be a trap.
StartCmd.PersistentFlags().StringVar(&appCode, "app", "", "limit to the migrations of one app (\""+migration.FrameworkAppCode+"\" for the framework's own)")
StartCmd.Flags().BoolVar(&dryRun, "dry-run", false, "list what would be applied, in order, and write nothing")
StartCmd.AddCommand(statusCmd)
}
func run() {
@@ -58,7 +81,12 @@ func run() {
}
}
func migrateModel() error {
// resolveDB picks the tenant database and hands it to the registry.
//
// It creates and alters nothing, which is what lets status and --dry-run share
// it: those two must be able to run against a production database without
// leaving a trace.
func resolveDB() (*gorm.DB, error) {
if host == "" {
host = "*"
}
@@ -73,29 +101,134 @@ func migrateModel() error {
}
}
if db == nil {
return fmt.Errorf("未找到数据库配置")
return nil, fmt.Errorf("未找到数据库配置")
}
if config.DatabasesConfig[host].Driver == "mysql" {
//初始化数据库时候用
db.Set("gorm:table_options", "ENGINE=InnoDB CHARSET=utf8mb4")
}
err := db.Debug().AutoMigrate(&models.Migration{})
return db, nil
}
// exitUnlessAppRegistered ends the command when --app names something no
// migration was registered under.
//
// Every path took a typo as "nothing matched" and reported success: `migrate`
// printed that the app was unknown and still exited 0, while `--dry-run` and
// `status` said "nothing to apply" and "none recorded" - which is what an
// up-to-date database says too, so the output does not even hint at the typo.
// An operator running `go-admin migrate --app crmm && deploy` gets the deploy.
//
// Checked against the registry, which init() has already filled, so this runs
// before any database work and costs nothing. It lives in the command layer
// because the exit code does: the migration package stays callable from a test
// without taking the process down with it.
func exitUnlessAppRegistered() {
if err := appRegistrationError(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
// appRegistrationError carries the decision on its own so it can be tested;
// exitUnlessAppRegistered is only the os.Exit around it. Nil means --app was
// either empty or names a registered app.
func appRegistrationError() error {
if appCode == "" {
return nil
}
want := migration.DisplayAppCode(migration.AppFilter(appCode))
registered := migration.Migrate.AppCodes()
for _, c := range registered {
if c == want {
return nil
}
}
return fmt.Errorf("no migrations are registered for app %q; registered: %s",
want, strings.Join(registered, ", "))
}
func migrateModel() error {
db, err := resolveDB()
if err != nil {
return err
}
// sys_migration is the one table that never goes through a versioned
// migration - it is the table that records them. AutoMigrate realigns it
// on every run, which is how the app_code column reaches an existing
// database without anyone writing a migration for it.
if err = db.Debug().AutoMigrate(&models.Migration{}); err != nil {
return err
}
migration.Migrate.SetDb(db.Debug())
if appCode != "" {
migration.Migrate.MigrateApp(appCode)
return nil
}
migration.Migrate.Migrate()
return err
return nil
}
func initDB() {
// Before the database is touched, so a typo cannot get as far as looking
// like a successful no-op on either path below.
exitUnlessAppRegistered()
//3. 初始化数据库链接
database.Setup()
if dryRun {
db, err := resolveDB()
if err != nil {
fmt.Println(err)
return
}
migration.Migrate.SetDb(db)
entries, err := migration.Migrate.Status()
if err != nil {
fmt.Println(err)
return
}
if err = printPending(os.Stdout, entries, appCode); err != nil {
fmt.Println(err)
}
return
}
//4. 数据库迁移
fmt.Println("数据库迁移开始")
_ = migrateModel()
if err := migrateModel(); err != nil {
fmt.Println(err)
return
}
fmt.Println(`数据库基础数据初始化成功`)
}
func runStatus() {
config.Setup(
file.NewSource(file.WithPath(configYml)),
func() {
exitUnlessAppRegistered()
database.Setup()
db, err := resolveDB()
if err != nil {
fmt.Println(err)
return
}
migration.Migrate.SetDb(db)
entries, err := migration.Migrate.Status()
if err != nil {
fmt.Println(err)
return
}
if err = printStatus(os.Stdout, entries, appCode); err != nil {
fmt.Println(err)
}
},
)
}
func genFile() error {
t1, err := template.ParseFiles("template/migrate.template")
if err != nil {
+160
View File
@@ -0,0 +1,160 @@
package migrate
import (
"fmt"
"io"
"sort"
"strings"
"time"
"go-admin/cmd/migrate/migration"
)
const applyTimeLayout = "2006-01-02 15:04:05"
// printStatus lists every migration this binary knows about together with every
// row already in sys_migration, grouped by app.
//
// filter is an app code as typed on the command line; empty means every app.
func printStatus(w io.Writer, entries []migration.StatusEntry, filter string) error {
entries = filterByApp(entries, filter)
groups, order := groupByApp(entries)
if len(order) == 0 {
_, err := fmt.Fprintln(w, "no migrations registered and none recorded")
return err
}
// One width for the whole listing rather than one per group: the versions
// of two apps line up, so a long list can be read down the column.
width := versionWidth(entries)
var applied, pending, orphaned int
for i, app := range order {
if i > 0 {
fmt.Fprintln(w)
}
fmt.Fprintf(w, "[%s]\n", app)
for _, e := range groups[app] {
state := "pending"
switch {
case e.Applied && !e.Registered:
state = "orphaned"
orphaned++
case e.Applied:
state = "applied"
applied++
default:
pending++
}
fmt.Fprintln(w, strings.TrimRight(
fmt.Sprintf(" %-*s%-*s%s", stateWidth, state, width, e.Version, formatApplyTime(e.ApplyTime)), " "))
}
}
fmt.Fprintf(w, "\n%d applied, %d pending across %d app(s)\n", applied, pending, len(order))
if orphaned > 0 {
fmt.Fprintf(w, "%d orphaned: recorded in sys_migration, but nothing in this binary registers them.\n"+
"Expected after a migration file is removed or an app is uninstalled; they will not run again.\n", orphaned)
}
return nil
}
// printPending is --dry-run: the same data as status, narrowed to what an
// actual run would do and printed in the order it would do it.
//
// It reads and prints. Every write path - AutoMigrate on sys_migration
// included - is on the other branch in initDB, so a dry run leaves the database
// byte for byte as it found it.
func printPending(w io.Writer, entries []migration.StatusEntry, filter string) error {
entries = filterByApp(entries, filter)
fmt.Fprintln(w, "dry-run: nothing will be written")
pending := make([]migration.StatusEntry, 0, len(entries))
for _, e := range entries {
// An orphaned row is recorded and unregistered; a real run cannot
// apply it, so a dry run must not offer to.
if !e.Applied && e.Registered {
pending = append(pending, e)
}
}
if len(pending) == 0 {
_, err := fmt.Fprintln(w, "nothing to apply")
return err
}
appWidth := 0
for _, e := range pending {
if n := len(migration.DisplayAppCode(e.AppCode)) + 2; n > appWidth {
appWidth = n
}
}
fmt.Fprintln(w, "would apply, in this order:")
for _, e := range pending {
fmt.Fprintf(w, " %-*s%s\n", appWidth+2, "["+migration.DisplayAppCode(e.AppCode)+"]", e.Version)
}
fmt.Fprintf(w, "\n%d migration(s) pending\n", len(pending))
return nil
}
// stateWidth is the width of the applied/pending/orphaned column, sized to the
// longest of the three plus a gap.
const stateWidth = len("orphaned") + 2
func versionWidth(entries []migration.StatusEntry) int {
width := 0
for _, e := range entries {
if n := len(e.Version) + 2; n > width {
width = n
}
}
return width
}
// filterByApp keeps the entries of one app. The filter is matched after the
// same normalisation ForApp applies, so --app CRM finds crm.
func filterByApp(entries []migration.StatusEntry, filter string) []migration.StatusEntry {
if filter == "" {
return entries
}
want := migration.AppFilter(filter)
out := make([]migration.StatusEntry, 0, len(entries))
for _, e := range entries {
if e.AppCode == want {
out = append(out, e)
}
}
return out
}
// groupByApp buckets entries by display name and returns the buckets plus the
// order to print them in: the framework first, then apps alphabetically. That
// is also the order a full run executes them in, because version strings sort
// as ASCII and the framework's are bare digits.
func groupByApp(entries []migration.StatusEntry) (map[string][]migration.StatusEntry, []string) {
groups := make(map[string][]migration.StatusEntry)
for _, e := range entries {
app := migration.DisplayAppCode(e.AppCode)
groups[app] = append(groups[app], e)
}
order := make([]string, 0, len(groups))
for app := range groups {
order = append(order, app)
}
sort.Slice(order, func(i, j int) bool {
if (order[i] == migration.FrameworkAppCode) != (order[j] == migration.FrameworkAppCode) {
return order[i] == migration.FrameworkAppCode
}
return order[i] < order[j]
})
return groups, order
}
func formatApplyTime(t *time.Time) string {
if t == nil {
return ""
}
return t.Format(applyTimeLayout)
}
+170
View File
@@ -0,0 +1,170 @@
package migrate
import (
"bytes"
"strings"
"testing"
"time"
"go-admin/cmd/migrate/migration"
)
func at(s string) *time.Time {
t, err := time.Parse(applyTimeLayout, s)
if err != nil {
panic(err)
}
return &t
}
// The order Status returns: version strings sorted as ASCII.
func sampleEntries() []migration.StatusEntry {
return []migration.StatusEntry{
{Version: "1786700001000", AppCode: "", Registered: true, Applied: true, ApplyTime: at("2026-08-20 10:00:00")},
{Version: "1786700005000", AppCode: "", Registered: true},
{Version: "crm-1786800001000", AppCode: "crm", Registered: true, Applied: true, ApplyTime: at("2026-08-25 14:03:11")},
{Version: "crm-1786800002000", AppCode: "crm", Registered: true},
}
}
func TestPrintStatusGroupsByApp(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), ""); err != nil {
t.Fatal(err)
}
got := buf.String()
for _, want := range []string{
"[core]",
"[crm]",
"applied 1786700001000 2026-08-20 10:00:00",
"pending 1786700005000",
"applied crm-1786800001000 2026-08-25 14:03:11",
"pending crm-1786800002000",
"2 applied, 2 pending across 2 app(s)",
} {
if !strings.Contains(got, want) {
t.Errorf("output missing %q:\n%s", want, got)
}
}
// The framework heads the list, because that is the order a full run
// executes in.
if strings.Index(got, "[core]") > strings.Index(got, "[crm]") {
t.Errorf("core is not listed first:\n%s", got)
}
}
// A row nobody registers any more is neither applied-and-current nor pending.
// Calling it applied would say the migration is in this binary, which is what
// sends someone looking for a file that was deleted.
func TestPrintStatusMarksOrphanedRows(t *testing.T) {
entries := append(sampleEntries(), migration.StatusEntry{
Version: "gone-1786800000000", AppCode: "gone", Applied: true, ApplyTime: at("2026-08-01 09:00:00"),
})
var buf bytes.Buffer
if err := printStatus(&buf, entries, ""); err != nil {
t.Fatal(err)
}
got := buf.String()
if !strings.Contains(got, "orphaned gone-1786800000000") {
t.Errorf("orphaned row not marked:\n%s", got)
}
if !strings.Contains(got, "nothing in this binary registers them") {
t.Errorf("orphaned rows need an explanation:\n%s", got)
}
if !strings.Contains(got, "2 applied, 2 pending") {
t.Errorf("orphaned rows must not be counted as applied:\n%s", got)
}
}
func TestPrintStatusFiltersByApp(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), "crm"); err != nil {
t.Fatal(err)
}
got := buf.String()
if strings.Contains(got, "[core]") {
t.Errorf("--app crm listed the framework:\n%s", got)
}
if !strings.Contains(got, "across 1 app(s)") {
t.Errorf("output = %s", got)
}
}
// status prints [core]; --app core has to mean the same thing.
func TestPrintStatusAppCoreSelectsTheFramework(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, sampleEntries(), migration.FrameworkAppCode); err != nil {
t.Fatal(err)
}
got := buf.String()
if strings.Contains(got, "[crm]") {
t.Errorf("--app core listed crm:\n%s", got)
}
if !strings.Contains(got, "[core]") {
t.Errorf("--app core listed nothing:\n%s", got)
}
}
func TestPrintStatusOnAnEmptyRegistry(t *testing.T) {
var buf bytes.Buffer
if err := printStatus(&buf, nil, ""); err != nil {
t.Fatal(err)
}
if !strings.Contains(buf.String(), "no migrations registered and none recorded") {
t.Errorf("output = %s", buf.String())
}
}
func TestPrintPendingListsOnlyPendingInOrder(t *testing.T) {
var buf bytes.Buffer
if err := printPending(&buf, sampleEntries(), ""); err != nil {
t.Fatal(err)
}
got := buf.String()
if !strings.Contains(got, "dry-run: nothing will be written") {
t.Errorf("dry-run must say it writes nothing:\n%s", got)
}
if strings.Contains(got, "1786700001000\n") || strings.Contains(got, "crm-1786800001000") {
t.Errorf("dry-run listed already applied migrations:\n%s", got)
}
if !strings.Contains(got, "[core] 1786700005000") || !strings.Contains(got, "[crm] crm-1786800002000") {
t.Errorf("dry-run is missing pending migrations:\n%s", got)
}
if !strings.Contains(got, "2 migration(s) pending") {
t.Errorf("output = %s", got)
}
if strings.Index(got, "1786700005000") > strings.Index(got, "crm-1786800002000") {
t.Errorf("dry-run order does not match run order:\n%s", got)
}
}
// An orphaned row is applied and unregistered; a dry run must not offer to
// apply it, because a real run cannot.
func TestPrintPendingSkipsOrphanedRows(t *testing.T) {
entries := []migration.StatusEntry{
{Version: "gone-1786800000000", AppCode: "gone", Applied: true, ApplyTime: at("2026-08-01 09:00:00")},
}
var buf bytes.Buffer
if err := printPending(&buf, entries, ""); err != nil {
t.Fatal(err)
}
if !strings.Contains(buf.String(), "nothing to apply") {
t.Errorf("output = %s", buf.String())
}
}
func TestPrintPendingFiltersByApp(t *testing.T) {
var buf bytes.Buffer
if err := printPending(&buf, sampleEntries(), "CRM"); err != nil {
t.Fatal(err)
}
got := buf.String()
if strings.Contains(got, "[core]") {
t.Errorf("--app CRM listed the framework:\n%s", got)
}
if !strings.Contains(got, "1 migration(s) pending") {
t.Errorf("output = %s", got)
}
}
+14
View File
@@ -0,0 +1,14 @@
package global
// Status values written to sys_opera_log.status.
//
// They live here rather than in app/admin/service/dto because
// common/middleware/logger.go writes the operation-log message and needs them.
// A package promised as a stable contract must not compile-depend on a
// business module: a fork that replaces or drops app/admin would otherwise
// stop compiling common/middleware, which is not something a contract package
// is allowed to do. See docs/contract.md.
const (
OperaStatusEnabled = "1"
OperaStatusDisabled = "2"
)
+24 -13
View File
@@ -1,7 +1,6 @@
package handler
import (
"go-admin/app/admin/models"
"go-admin/common"
"net/http"
@@ -163,19 +162,31 @@ func LogOut(c *gin.Context) {
}
// Authorizator decides whether a parsed identity may proceed. It authorizes
// every identity IdentityHandler was able to build, which is what it has always
// done.
//
// It used to also assert data["user"] and data["role"] into app/admin/models
// types and copy five fields onto the context. Those two keys are not in the
// map: IdentityHandler builds it from the token claims and puts in
// IdentityKey / UserName / RoleKey / UserId / RoleIds / DataScope. Both
// assertions therefore failed on every request, and because the ok result was
// discarded, the five c.Set calls stored zero values and the function returned
// true regardless.
//
// Nothing in this repository or in go-admin-core reads role / roleIds /
// userId / userName / dataScope off the context - the open-source data
// permission path reads the JWT claims through
// common/actions.Permission -> user.GetUserIdStr(c). Dropping the block
// therefore removes five zero values nobody read, and with them the last
// import of app/admin from a contract package.
//
// Anything maintaining its own copy of this file must check its own consumers
// before taking this change: a codebase that does read those keys off the
// context needs Authorizator to keep setting them.
func Authorizator(data interface{}, c *gin.Context) bool {
if v, ok := data.(map[string]interface{}); ok {
u, _ := v["user"].(models.SysUser)
r, _ := v["role"].(models.SysRole)
c.Set("role", r.RoleName)
c.Set("roleIds", r.RoleId)
c.Set("userId", u.UserId)
c.Set("userName", u.Username)
c.Set("dataScope", r.DataScope)
return true
}
return false
_, ok := data.(map[string]interface{})
return ok
}
func Unauthorized(c *gin.Context, code int, message string) {
+67 -20
View File
@@ -1,19 +1,18 @@
package middleware
import (
"bufio"
"bytes"
"encoding/json"
"go-admin/app/admin/service/dto"
"errors"
"go-admin/common"
"io"
"io/ioutil"
"net/http"
"strings"
"time"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/jwtauth/user"
"github.com/go-admin-team/go-admin-core/v2/logger"
"github.com/go-admin-team/go-admin-core/v2/sdk"
"github.com/go-admin-team/go-admin-core/v2/sdk/api"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
@@ -28,19 +27,14 @@ func LoggerToFile() gin.HandlerFunc {
// 开始时间
startTime := time.Now()
// 处理请求
//
// The body is only read when it has a destination. operParam below is
// the only consumer, and it is written when logger.enableddb is on -
// off in the shipped configuration, where reading the body was a copy
// of every request made and discarded.
var body string
switch c.Request.Method {
case http.MethodPost, http.MethodPut, http.MethodGet, http.MethodDelete:
bf := bytes.NewBuffer(nil)
wt := bufio.NewWriter(bf)
_, err := io.Copy(wt, c.Request.Body)
if err != nil {
log.Warnf("copy body error, %s", err.Error())
err = nil
}
rb, _ := ioutil.ReadAll(bf)
c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(rb))
body = string(rb)
if config.LoggerConfig.EnabledDB {
body = readOperParam(c, log)
}
c.Next()
@@ -100,10 +94,55 @@ func LoggerToFile() gin.HandlerFunc {
}
}
// SetDBOperLog 写入操作日志表 fixme 该方法后续即将弃用
func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) {
// operParamLimit caps what is copied out of a request body for the operation
// log. A file upload is a POST like any other and reaches this middleware
// before any handler, so without a limit the whole upload is held in memory to
// write a log row - a 16MB upload allocated about 67MB. The limit also keeps
// the value inside the column, which is TEXT.
const operParamLimit = 32 << 10
log := api.GetRequestLogger(c)
// readOperParam copies the start of the request body for the operation log and
// leaves the request readable by the handler.
//
// The body is not buffered whole: the handler reads the part copied here from
// memory and the rest straight from the connection, so what this holds is
// bounded by operParamLimit however large the request is.
func readOperParam(c *gin.Context, log *logger.Helper) string {
switch c.Request.Method {
case http.MethodPost, http.MethodPut, http.MethodGet, http.MethodDelete:
default:
return ""
}
if c.Request.Body == nil {
return ""
}
rest := c.Request.Body
head := make([]byte, operParamLimit)
n, err := io.ReadFull(rest, head)
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
log.Warnf("read body for the operation log: %s", err)
}
head = head[:n]
c.Request.Body = readCloser{
Reader: io.MultiReader(bytes.NewReader(head), rest),
Closer: rest,
}
return string(head)
}
type readCloser struct {
io.Reader
io.Closer
}
// operaLogFields builds the message written to the operation log queue.
//
// Split out of SetDBOperLog so the field set can be asserted in a test: the
// consumer on the other end of the queue reads these keys by name, so a
// dropped or renamed key costs a column in sys_opera_log and reports nothing.
func operaLogFields(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) map[string]interface{} {
l := make(map[string]interface{})
l["_fullPath"] = c.FullPath()
l["operUrl"] = reqUri
@@ -120,10 +159,18 @@ func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string
l["createBy"] = user.GetUserId(c)
l["updateBy"] = user.GetUserId(c)
if status == http.StatusOK {
l["status"] = dto.OperaStatusEnabel
l["status"] = global.OperaStatusEnabled
} else {
l["status"] = dto.OperaStatusDisable
l["status"] = global.OperaStatusDisabled
}
return l
}
// SetDBOperLog 写入操作日志表 fixme 该方法后续即将弃用
func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) {
log := api.GetRequestLogger(c)
l := operaLogFields(c, clientIP, statusCode, reqUri, reqMethod, latencyTime, body, result, status)
q := sdk.Runtime.GetQueuePrefix(c.Request.Host)
message, err := sdk.Runtime.GetStreamMessage("", global.OperateLog, l)
if err != nil {
+124
View File
@@ -0,0 +1,124 @@
package middleware
import (
"bytes"
"io"
"net/http"
"net/http/httptest"
"runtime"
"strings"
"testing"
"github.com/gin-gonic/gin"
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
)
// serveWithLogger runs one request through the logger middleware and returns
// what the handler saw, with logger.enableddb set as given.
func serveWithLogger(t testing.TB, enabledDB bool, method, body string) string {
t.Helper()
prev := config.LoggerConfig.EnabledDB
config.LoggerConfig.EnabledDB = enabledDB
t.Cleanup(func() { config.LoggerConfig.EnabledDB = prev })
gin.SetMode(gin.ReleaseMode)
r := gin.New()
r.Use(LoggerToFile())
var seen string
handler := func(c *gin.Context) {
b, err := io.ReadAll(c.Request.Body)
if err != nil {
t.Errorf("handler could not read the body: %v", err)
}
seen = string(b)
c.Status(http.StatusOK)
}
r.Handle(method, "/probe", handler)
req := httptest.NewRequest(method, "/probe", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
r.ServeHTTP(httptest.NewRecorder(), req)
return seen
}
// The middleware rewrites Request.Body so it can log the parameters. Whatever
// else it does, the handler has to receive the request the client sent - all
// of it, whether or not the operation log is on, and whether or not the body
// is longer than what gets logged.
func TestHandlerStillSeesTheWholeBody(t *testing.T) {
cases := []struct {
name string
enabledDB bool
body string
}{
{"log off, short body", false, `{"username":"admin"}`},
{"log on, short body", true, `{"username":"admin"}`},
{"log off, empty body", false, ""},
{"log on, empty body", true, ""},
// Longer than operParamLimit: the logged copy is truncated, the body is not.
{"log on, body past the limit", true, strings.Repeat("x", operParamLimit+4096)},
{"log off, body past the limit", false, strings.Repeat("y", operParamLimit+4096)},
// Exactly at the boundary, where a fencepost error would show.
{"log on, body exactly at the limit", true, strings.Repeat("z", operParamLimit)},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
for _, method := range []string{http.MethodPost, http.MethodPut, http.MethodDelete} {
if got := serveWithLogger(t, c.enabledDB, method, c.body); got != c.body {
t.Errorf("%s: handler saw %d bytes, the client sent %d",
method, len(got), len(c.body))
}
}
})
}
}
// The body is read for one reason - operParam on the operation log row - and
// that row is only written when logger.enableddb is on. With it off, reading
// the body is a copy of every request made and thrown away, and a file upload
// is a POST like any other: 16MB of upload allocated about 67MB here.
//
// Allocation counts are deterministic across machines; wall-clock is not.
func TestBodyIsNotCopiedWhenTheOperationLogIsOff(t *testing.T) {
const size = 1 << 20
body := strings.Repeat("x", size)
prev := config.LoggerConfig.EnabledDB
config.LoggerConfig.EnabledDB = false
t.Cleanup(func() { config.LoggerConfig.EnabledDB = prev })
gin.SetMode(gin.ReleaseMode)
r := gin.New()
r.Use(LoggerToFile())
r.POST("/probe", func(c *gin.Context) { c.Status(http.StatusOK) })
payload := []byte(body)
run := func() {
req := httptest.NewRequest(http.MethodPost, "/probe", bytes.NewReader(payload))
req.Header.Set("Content-Type", "application/json")
r.ServeHTTP(httptest.NewRecorder(), req)
}
var before, after uint64
before = heapAllocs()
run()
after = heapAllocs()
// The handler never reads the body, so a request that does not copy it
// should allocate far less than the body's size. The old middleware
// allocated about four times the body.
if grew := after - before; grew > size/2 {
t.Errorf("a %d-byte request allocated %d bytes with the operation log off; "+
"the body should not be read when nothing consumes it", size, grew)
}
}
func heapAllocs() uint64 {
var m runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&m)
return m.TotalAlloc
}
+73
View File
@@ -0,0 +1,73 @@
package middleware
import (
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/gin-gonic/gin"
"go-admin/common/global"
)
// The operation-log consumer reads these keys by name off the queue message.
// Losing one costs a column in sys_opera_log and reports nothing - the request
// still succeeds, the log row is just wrong.
//
// This locks the set down across the move of the status constants out of
// app/admin/service/dto, which touched every request path.
var operaLogKeys = []string{
"_fullPath", "operUrl", "operIp", "operLocation", "operName",
"requestMethod", "operParam", "operTime", "jsonResult", "latencyTime",
"statusCode", "userAgent", "createBy", "updateBy", "status",
}
func TestOperaLogFieldsAreComplete(t *testing.T) {
gin.SetMode(gin.TestMode)
c, _ := gin.CreateTestContext(httptest.NewRecorder())
c.Request = httptest.NewRequest(http.MethodPost, "/api/v1/sys-user", nil)
c.Request.Header.Set("User-Agent", "go-test")
l := operaLogFields(c, "127.0.0.1", http.StatusOK, "/api/v1/sys-user", http.MethodPost,
12*time.Millisecond, `{"a":1}`, `{"code":200}`, http.StatusOK)
for _, k := range operaLogKeys {
if _, ok := l[k]; !ok {
t.Errorf("operation log is missing %q", k)
}
}
if len(l) != len(operaLogKeys) {
t.Errorf("operation log has %d fields, expected %d; update operaLogKeys deliberately, not to make this pass",
len(l), len(operaLogKeys))
}
if got := l["operUrl"]; got != "/api/v1/sys-user" {
t.Errorf("operUrl = %v", got)
}
if got := l["userAgent"]; got != "go-test" {
t.Errorf("userAgent = %v", got)
}
}
// status is what tells a failed request from a successful one in the log table.
// It is a string, and it is the one field whose source package changed.
func TestOperaLogStatusMapping(t *testing.T) {
gin.SetMode(gin.TestMode)
for _, tc := range []struct {
name string
status int
want string
}{
{"ok", http.StatusOK, global.OperaStatusEnabled},
{"error", http.StatusInternalServerError, global.OperaStatusDisabled},
} {
t.Run(tc.name, func(t *testing.T) {
c, _ := gin.CreateTestContext(httptest.NewRecorder())
c.Request = httptest.NewRequest(http.MethodGet, "/", nil)
l := operaLogFields(c, "127.0.0.1", tc.status, "/", http.MethodGet, 0, "", "", tc.status)
if l["status"] != tc.want {
t.Fatalf("status = %v, want %v", l["status"], tc.want)
}
})
}
}
+27
View File
@@ -59,6 +59,33 @@ func AuthCheckRole() gin.HandlerFunc {
}
}
// EnforceRoleFor reports whether the caller's role has explicit Casbin
// permission to act on path with method.
//
// AuthCheckRole never calls Enforce for a route CasbinExclude lists - that
// is the whole point of the list. A handler on such a route can still need
// the real answer for part of what it does: sys_user.go's Update shares its
// excluded route between the personal-center screen editing the caller's own
// record (which is why the route is excluded at all) and an admin editing
// someone else's, and only the second case is meant to require a policy
// grant. That handler asks here instead of assuming the middleware already
// checked.
func EnforceRoleFor(c *gin.Context, path, method string) (bool, error) {
data, ok := c.Get(jwtauth.JwtPayloadKey)
if !ok {
return false, nil
}
v, ok := data.(jwtauth.MapClaims)
if !ok {
return false, nil
}
if v["rolekey"] == "admin" {
return true, nil
}
e := sdk.Runtime.GetCasbinByTenant(c.Request.Host)
return e.Enforce(v["rolekey"], path, method)
}
// excludedFromCasbin reports whether the route skips the permission check.
//
// It runs for every non-admin request, so the order matters: the method rules
+11
View File
@@ -5,6 +5,17 @@ import "time"
type Migration struct {
Version string `gorm:"primaryKey"`
ApplyTime time.Time `gorm:"autoCreateTime"`
// AppCode identifies which app registered this migration. The empty string
// means the framework itself.
//
// NOT NULL DEFAULT '' rather than a nullable column, and the difference is
// not cosmetic: on a nullable column the rows that already exist when
// AutoMigrate adds it hold NULL, and the first SELECT scanning one into
// this string field fails with "converting NULL to string is unsupported".
// The default is what makes "existing history belongs to the framework"
// true without a backfill script anyone could forget to run.
AppCode string `gorm:"type:varchar(64);not null;default:'';index:idx_sys_migration_app_code;comment:AppCode"`
}
func (Migration) TableName() string {
+199
View File
@@ -0,0 +1,199 @@
# 公共契约面
> 本文写给**第三方应用作者**:你写一个装进 go-admin 的业务模块,可以依赖什么、
> 怎么注册进来、哪些东西随时可能变。
>
> 主仓贡献者的编码约定见根目录 `AGENTS.md`,设计取舍见 `docs/architecture.md`。
---
## 承诺稳定的包
| 包 | 用途 |
|---|---|
| `common/actions` | 通用 CRUD Action(Index / View / Create / Update / Delete / Permission) |
| `common/dto` | 分页、`search` tag 解析、`Control` / `Index` 接口 |
| `common/models` | `ActiveRecord`、`ControlBy`、`ModelTime`、`Model` |
| `common/middleware` | `AuthCheckRole`、`InitMiddleware` 等 |
**依据不是拍脑袋列的**:`app/demo` 是一个可编译、有测试、CI 会跑的标准 CRUD 模块,
把它的 `go-admin/` 前缀 import 全部去重之后,恰好就是这四个包 —— 它代表
"写一个标准模块所需要的最小依赖面"。你的模块如果需要第五个包,先在 issue 里说一声,
那多半意味着契约面缺了什么。
"稳定"的含义:**在 `2.x` 内不做破坏性变更**。新增导出符号不算破坏;改签名、
改语义、删除导出符号算,会走 major 版本并在 release note 里单列。
### 没有已知例外
这四个包**不 import `app/` 下的任何东西**,2026-08-31 起由 CI 强制
(见下方「边界由 CI 守着」)。在此之前有两处反向依赖,都已根治:
| 原位置 | 反向依赖 | 处理 |
|---|---|---|
| `common/middleware/logger.go` | `app/admin/service/dto` 的两个操作日志状态常量 | 常量下沉到 `common/global`,`dto` 侧保留同名常量作为 deprecated 别名,fork 不受影响 |
| `common/middleware/handler/auth.go` | `app/admin/models` 的 `SysUser` / `SysRole` | 该段断言恒失败、设的是零值且开源版无人读取,属死代码,已删除 |
之所以不把它们记成"已知例外":这份文档的作用就是告诉你哪些包可以依赖,
如果第一条下面就挂着例外脚注,后来人会照着例外抄,边界从第一天起就是脏的。
---
## 其余包不保证稳定
`common/` 下没有出现在上表里的包(`common/global`、`common/storage`、
`common/database`、`common/file_store`、`common/response`、`common/service`、
`common/apis`、`common/middleware/handler`、根 `common` 包……)以及
`app/admin` 的内部实现,**均不承诺稳定**。
其中 `common/global`、`common/middleware/handler`、根 `common` 包是
`common/middleware` 的编译期依赖 —— 它们会被一起拉进你的依赖图,但这不代表
它们的 API 稳定。**不要因为"都在 `common/` 目录下"就认为是契约面。**
规划中的 001(模块路径改名)会把非契约包移进 `internal/`,由编译器强制这条边界。
届时上表之外的包对外部模块直接不可见 —— 现在就照上表写,那次改动对你零成本。
---
## 注册路由
一个应用模块要注册自己的路由,写一个 `func()` 签名的 `InitRouter`
(照抄 `app/demo/router/router.go`),然后二选一接进来:
```go
// 方式一(历史写法,仍然有效):在主仓 cmd/api/<name>.go 里
AppRouters = append(AppRouters, router.InitRouter)
// 方式二(推荐):不需要 import go-admin/cmd/api
sdk.Runtime.SetAppRouters(router.InitRouter)
```
方式二是本次新接上的。差别只有一个但很关键:方式一要求你的模块
`import "go-admin/cmd/api"` —— 那是主程序的命令包,让业务模块依赖它很别扭,
也正是"主仓要为每个模块加一个七行文件"的根源。
**执行顺序**:先跑完包级 `AppRouters`,再由 core 的 `sdk.Runtime.RunAppRouters()`
跑它自己的注册表,各自内部保持注册顺序。别依赖跨来源的相对顺序,各模块的
`RouterGroup` 前缀互不相同,本来就不该有顺序依赖。
走方式二还多拿到两样东西,都在 core 那边实现(见
[core 的 `docs/contract.md`](https://github.com/go-admin-team/go-admin-core/blob/main/docs/contract.md)):
**panic 护栏**——你的 `InitRouter` panic 了,其余模块照常注册、进程不退出,日志里会写明
是哪一行注册的;**失败分级**——`sdk.Runtime.SetAppRoutersWith(f, runtime.WithFatal())`
声明「我起不来就别启动」。方式一(包级 `AppRouters`)没有护栏,panic 直接掀桌。
`InitRouter()` 内部的约定:自己拿 `sdk.Runtime.GetEngine()`,按需建
`gin.RouterGroup`,通过 `init()` 自注册到你自己包内的
`routerCheckRole` / `routerNoCheckRole` 列表,不在任何中心文件手工列举
(与 `AGENTS.md`「路由注册」一节一致)。
---
## 注册数据库迁移
框架自身的迁移不变:
```go
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700001000DemoMenu)
```
应用的迁移走 `ForApp`:
```go
func init() {
_, fileName, _, _ := runtime.Caller(0)
migration.ForApp("crm").SetVersion(migration.GetFilename(fileName), initCrmTables)
}
func initCrmTables(db *gorm.DB, version, appCode string) error {
return db.Transaction(func(tx *gorm.DB) error {
// ... schema / data changes ...
return tx.Create(&common.Migration{Version: version, AppCode: appCode}).Error
})
}
```
四条必须知道的规则:
1. **完成记录由迁移函数自己写**,而且要写在自己的事务里。框架的调度循环只做
"这个 version 在 `sys_migration` 里有没有" 的判断,从不代你插入 —— 这样
"数据改完了"和"标记成已完成"才是同一个事务,不会出现改了一半却被记成成功。
2. **`AppCode` 必须写进去**。签名多带一个 `appCode` 参数就是为此 —— 忘了写,
schema 上那一列等于白加,你的迁移会被记成框架的。
3. **落库的 `version` 是加了前缀的**。`ForApp("crm")` 注册 `1786800001000`,
实际写进 `sys_migration.version` 的是 `crm-1786800001000`,函数收到的
`version` 参数已经是这个带前缀的值,照抄进 `common.Migration{Version: version}`
即可。前缀的意义是:两个来源不同的应用哪怕碰巧生成同一个毫秒时间戳,也不会撞主键、
不会有一方被误判为"已应用"。
4. **应用 code 一律小写**,`ForApp` 会自己 `strings.ToLower` 一遍。`core` 是保留字
(`migrate status` 用它表示框架自身,`--app core` 选中框架),`ForApp("core")`
会 panic。
顺序保证:**同一应用内按版本号严格有序**。跨应用顺序不做承诺 —— 由于前缀的存在,
今天的实际顺序是"先跑完全部框架迁移,再按 appCode 字母序逐个应用跑完",
但这是实现细节,不要依赖它。跨应用依赖(应用 A 的迁移要求应用 B 先跑完)
需要依赖拓扑排序,属于后续阶段。
看当前状态、看这次会跑什么,不用猜:
```bash
go-admin migrate status -c config/settings.yml # 按应用分组列出已应用 / 待应用
go-admin migrate --dry-run -c config/settings.yml # 列出会执行什么、什么顺序,不写库
go-admin migrate --app crm -c config/settings.yml # 只跑 crm 的迁移
```
`status` 与 `--dry-run` 是纯只读的,不建表、不改表结构,可以直接对生产库执行。
---
## 硬约束:注册要赶在启动钩子之前
三个注册入口——`AppRouters`、`sdk.Runtime.SetAppRouters`、`migration.ForApp`——
都必须在 `cmd/api/server.go` 的 `runStartupHooks()` 执行之前调用完。
`init()` 是最省事的位置:Go 规范保证包级变量初始化与 `init()` 在 `main()` 之前
**单 goroutine 顺序执行**,注册期天然没有并发写。但它不是唯一合法位置——
在 `run()` 之类早于启动钩子的地方注册同样成立。这条规则约束的是**顺序**,
不是你写在哪个函数里。
`sdk.Runtime.SetAppRouters` 的准确语义以 core 为准:
> [go-admin-core `docs/contract.md`](https://github.com/go-admin-team/go-admin-core/blob/main/docs/contract.md)
那份文档写明了注册类与资源类的划分、封闭时刻、护栏边界(**只覆盖同步 panic,
你自己 `go func()` 出去的 panic 框架够不着**)、以及配置热更新会在运行期
重新执行 setup 回调这件事。
主仓这边只补三条它管不着的:
1. **`RunAppRouters()` 跑过之后,core 的注册表就封闭了**,再调
`sdk.Runtime.SetAppRouters` 会被丢弃并记一条 ERROR 日志。包级 `AppRouters`
没有这个机制——它就是一个普通 slice,什么时候 append 都"成功",
但 `runStartupHooks()` 之后 append 的那些永远不会被执行,且不出声。
这是继续推荐方式二的理由之一。
2. **封闭是黏性的,而 `sdk.Runtime` 是包级单例。** 写测试时若会触发启动钩子,
必须换掉它再还原,否则同一个测试二进制里后面的测试会静默丢注册:
```go
previous := sdk.Runtime
t.Cleanup(func() { sdk.Runtime = previous })
sdk.Runtime = runtime.NewConfig()
```
`cmd/api/server_test.go` 里的 `freshRuntime` 就是这个。
3. **`migration.ForApp` 是主仓的东西**,core 不认识它,上面那份文档不覆盖它。
它的约束仍然是"注册要在迁移调度循环跑起来之前",实践上就是 `init()`。
---
## 边界由 CI 守着
`common/`、`core/` 不得 import `app/`,这条由 `tools/checksilent` 的
`contract-import-boundary` 检查固化,`make checksilent` 在 CI 里跑,违反即失败
(测试文件同样算 —— 一个删掉 `app/admin` 的 fork 也应该能跑 `go test ./...`)。
靠人工评审列契约面会漏。上面那两处反向依赖里,第二处就是评审没发现、
靠机器全量扫描才找出来的。
`tools/checksilent` 还检查另外五类"不出声的失败",写模块时值得先看一眼
`go run ./tools/checksilent -h`。
+1 -1
View File
@@ -11,7 +11,7 @@ require (
github.com/casbin/casbin/v3 v3.8.1
github.com/gin-gonic/gin v1.12.0
github.com/glebarez/sqlite v1.11.0
github.com/go-admin-team/go-admin-core/v2 v2.3.0
github.com/go-admin-team/go-admin-core/v2 v2.4.1
github.com/google/uuid v1.6.0
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible
github.com/mssola/user_agent v0.6.0
+2 -2
View File
@@ -145,8 +145,8 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
github.com/go-admin-team/go-admin-core/v2 v2.3.0 h1:P2Po+jsJByzv1y2cN8eEGfFOutWSYNzQTJxLGAFmHkE=
github.com/go-admin-team/go-admin-core/v2 v2.3.0/go.mod h1:YiJr2+vqC9qV5AoGeL+1W55h3XZ99CB5xWnP3Wo8c5g=
github.com/go-admin-team/go-admin-core/v2 v2.4.1 h1:69QprBVMcQzjVP0UksCwi//A0qh8gwcIHcwHmABPp2U=
github.com/go-admin-team/go-admin-core/v2 v2.4.1/go.mod h1:YiJr2+vqC9qV5AoGeL+1W55h3XZ99CB5xWnP3Wo8c5g=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o=
+257
View File
@@ -0,0 +1,257 @@
package main
import (
"go/ast"
"go/token"
"strconv"
"strings"
)
// structLiteral is a composite literal whose type resolved to a named struct.
type structLiteral struct {
PkgPath string
Name string
Lit *ast.CompositeLit
}
// forEachStructLiteral visits every composite literal in the file whose type
// resolves to a named type, including the ones written with the type elided.
//
// The elided form is the one that matters: seed data is written as
// []models.SysMenu{{MenuId: 9000}, {MenuId: 9001}}, and the inner literals carry
// no type of their own. A walker that only looked at CompositeLit.Type would
// silently skip every seed in the repository and report nothing, which for a
// tool about silent failure would be its own punchline.
func forEachStructLiteral(sf *sourceFile, fn func(structLiteral)) {
// The type each type-less literal inherits from the literal containing it.
elided := map[*ast.CompositeLit]ast.Expr{}
var propagate func(lit *ast.CompositeLit, typ ast.Expr)
propagate = func(lit *ast.CompositeLit, typ ast.Expr) {
child := elementType(typ)
if child == nil {
return
}
for _, elt := range lit.Elts {
v := elt
if kv, ok := elt.(*ast.KeyValueExpr); ok {
v = kv.Value
}
if cl, ok := v.(*ast.CompositeLit); ok && cl.Type == nil {
elided[cl] = child
propagate(cl, child)
}
}
}
// ast.Inspect visits a node before its children, so every typed literal
// fills in its descendants before the reporting pass reaches them.
ast.Inspect(sf.Syntax, func(n ast.Node) bool {
cl, ok := n.(*ast.CompositeLit)
if !ok {
return true
}
if typ := litType(cl, elided); typ != nil {
propagate(cl, typ)
}
return true
})
ast.Inspect(sf.Syntax, func(n ast.Node) bool {
cl, ok := n.(*ast.CompositeLit)
if !ok {
return true
}
typ := litType(cl, elided)
if typ == nil {
return true
}
if pkg, name, ok := resolveNamed(sf, typ); ok {
fn(structLiteral{PkgPath: pkg, Name: name, Lit: cl})
}
return true
})
}
func litType(cl *ast.CompositeLit, elided map[*ast.CompositeLit]ast.Expr) ast.Expr {
if cl.Type != nil {
return cl.Type
}
return elided[cl]
}
// resolveNamed maps a type expression to (import path, type name). A bare
// identifier means a type declared in this file's own package.
func resolveNamed(sf *sourceFile, typ ast.Expr) (string, string, bool) {
switch t := typ.(type) {
case *ast.StarExpr:
return resolveNamed(sf, t.X)
case *ast.Ident:
return sf.Pkg, t.Name, true
case *ast.SelectorExpr:
pkgIdent, ok := t.X.(*ast.Ident)
if !ok {
return "", "", false
}
path, ok := sf.imports[pkgIdent.Name]
if !ok {
return "", "", false
}
return path, t.Sel.Name, true
}
return "", "", false
}
// elementType is the type the children of a composite literal take when they
// leave theirs out.
func elementType(typ ast.Expr) ast.Expr {
switch t := typ.(type) {
case *ast.ArrayType:
return t.Elt
case *ast.MapType:
return t.Value
case *ast.StarExpr:
return elementType(t.X)
}
return nil
}
// field returns the value written for a named field of a struct literal.
func field(lit *ast.CompositeLit, name string) (ast.Expr, bool) {
for _, elt := range lit.Elts {
kv, ok := elt.(*ast.KeyValueExpr)
if !ok {
continue
}
if key, ok := kv.Key.(*ast.Ident); ok && key.Name == name {
return kv.Value, true
}
}
return nil, false
}
// intValue evaluates an integer field: a literal, a negated literal, or an
// identifier naming a constant in the same package.
//
// Anything computed at run time is skipped rather than guessed at. That is the
// one thing these checks miss, and missing is the right way to be wrong here -
// a false positive teaches people to add ignore comments, and then the tool is
// finished.
func intValue(sf *sourceFile, expr ast.Expr) (int64, bool) {
switch e := expr.(type) {
case *ast.Ident:
v, ok := sf.consts[e.Name]
return v, ok
case *ast.UnaryExpr:
if e.Op == token.SUB {
if v, ok := intValue(sf, e.X); ok {
return -v, true
}
}
}
return intLiteral(expr)
}
func intLiteral(expr ast.Expr) (int64, bool) {
lit, ok := expr.(*ast.BasicLit)
if !ok || lit.Kind != token.INT {
return 0, false
}
v, err := strconv.ParseInt(strings.ReplaceAll(lit.Value, "_", ""), 0, 64)
if err != nil {
return 0, false
}
return v, true
}
// stringValue evaluates a string field: a literal, a concatenation of literals,
// or an identifier naming a string constant in the same package.
func stringValue(sf *sourceFile, expr ast.Expr) (string, bool) {
switch e := expr.(type) {
case *ast.BasicLit:
if e.Kind != token.STRING {
return "", false
}
s, err := strconv.Unquote(e.Value)
if err != nil {
return "", false
}
return s, true
case *ast.BinaryExpr:
if e.Op != token.ADD {
return "", false
}
l, lok := stringValue(sf, e.X)
r, rok := stringValue(sf, e.Y)
if lok && rok {
return l + r, true
}
}
return "", false
}
// embeddedTypes returns the types a struct embeds, as (import path, name).
func embeddedTypes(sf *sourceFile, st *ast.StructType) [][2]string {
var out [][2]string
for _, f := range st.Fields.List {
if len(f.Names) != 0 {
continue // a named field, not an embed
}
if pkg, name, ok := resolveNamed(sf, f.Type); ok {
out = append(out, [2]string{pkg, name})
}
}
return out
}
// tableNames maps struct name to the literal its TableName method returns.
func tableNames(sf *sourceFile) map[string]string {
out := map[string]string{}
for _, decl := range sf.Syntax.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok || fn.Name.Name != "TableName" || fn.Recv == nil || len(fn.Recv.List) != 1 || fn.Body == nil {
continue
}
recv := receiverName(fn.Recv.List[0].Type)
if recv == "" {
continue
}
ast.Inspect(fn.Body, func(n ast.Node) bool {
ret, ok := n.(*ast.ReturnStmt)
if !ok || len(ret.Results) != 1 {
return true
}
if s, ok := stringValue(sf, ret.Results[0]); ok && out[recv] == "" {
out[recv] = s
}
return true
})
}
return out
}
func receiverName(expr ast.Expr) string {
switch t := expr.(type) {
case *ast.Ident:
return t.Name
case *ast.StarExpr:
return receiverName(t.X)
}
return ""
}
// structTypes maps struct name to its declaration.
func structTypes(sf *sourceFile) map[string]*ast.StructType {
out := map[string]*ast.StructType{}
ast.Inspect(sf.Syntax, func(n ast.Node) bool {
ts, ok := n.(*ast.TypeSpec)
if !ok {
return true
}
if st, ok := ts.Type.(*ast.StructType); ok {
out[ts.Name.Name] = st
}
return true
})
return out
}
+409
View File
@@ -0,0 +1,409 @@
package main
import (
"fmt"
"go/token"
"path"
"sort"
"strconv"
"strings"
)
// Check names. They appear in every message and in the CI log, so they are
// what people will search for.
const (
checkModelTimeMix = "modeltime-mix"
checkMenuSort = "menu-sort-overflow"
checkMenuName = "menu-name-mismatch"
checkConfigValue = "config-value-truncation"
checkMenuIDConflict = "menu-id-collision"
checkImportBoundary = "contract-import-boundary"
)
// Package paths, relative to the module. Spelled once so a module rename
// touches one place.
const (
pkgFrozenModels = "cmd/migrate/migration/models"
pkgRuntimeModel = "common/models"
pkgAdminModels = "app/admin/models"
)
// options are the run-time knobs. Only the frontend directory is one: every
// other check either applies or does not, with nothing to configure.
type options struct {
// UIDir is the go-admin-ui src directory. Empty disables checkMenuName,
// which is the only check that needs a second repository.
UIDir string
}
// runChecks runs every check over one parse of the tree.
func runChecks(s *snapshot, opt options) ([]Finding, error) {
var out []Finding
out = append(out, checkModelTimeMixing(s)...)
out = append(out, checkMenuSortOverflow(s)...)
out = append(out, checkConfigValueLength(s)...)
out = append(out, checkMenuIDCollisions(s)...)
out = append(out, checkContractImportBoundary(s)...)
if opt.UIDir != "" {
fs, err := checkMenuNames(s, opt.UIDir)
if err != nil {
return nil, err
}
out = append(out, fs...)
}
sortFindings(out)
return out, nil
}
func (s *snapshot) pkg(rel string) string { return s.ModulePath + "/" + rel }
func (s *snapshot) finding(sev Severity, check string, sf *sourceFile, pos posLike, format string, args ...interface{}) Finding {
file, line, col := s.Pos(sf, pos.Pos())
return Finding{
Check: check,
Severity: sev.String(),
File: file,
Line: line,
Col: col,
Message: fmt.Sprintf(format, args...),
severity: sev,
}
}
type posLike interface{ Pos() token.Pos }
// ---------------------------------------------------------------------------
// check 1: the two ModelTime flavours
// ---------------------------------------------------------------------------
// checkModelTimeMixing reports code that mixes the repository's two soft-delete
// shapes.
//
// cmd/migrate/migration/models.ModelTime declares a nullable gorm.DeletedAt;
// common/models.ModelTime declares the NOT NULL millisecond marker. Mixing them
// on one table is not a compile error and not a run-time error either: gorm
// scopes the nullable flavour as "WHERE deleted_at IS NULL" while live rows hold
// 0, so every row of the table becomes invisible and the feature reading it
// simply returns nothing. sys_columns and sys_tables sat in that state until
// 1786700004000 - the code generator listed no tables at all and reported no
// error.
//
// Two shapes are reported, and both are unambiguous:
//
// 1. a runtime model under app/ that embeds the frozen package's time struct -
// always wrong, that package is the shape the columns had before the
// conversion;
// 2. a migration ordered after the conversion that imports the frozen package
// - AGENTS.md states this rule, and the version/ directory already has a
// test for it; this extends it to version-local/, where third-party and
// downstream migrations live and where no test was watching.
//
// Not reported: that two model packages describe the same table with different
// flavours. That is true of a dozen tables on purpose - the frozen package is
// correct for the migrations that predate the conversion - so reporting it
// would be reporting the design.
func checkModelTimeMixing(s *snapshot) []Finding {
var out []Finding
frozen := s.pkg(pkgFrozenModels)
for _, sf := range s.Files {
if strings.HasPrefix(sf.Path, "app/") && sf.Imports(frozen) {
tables := tableNames(sf)
for name, st := range structTypes(sf) {
table, isModel := tables[name]
if !isModel {
continue
}
for _, emb := range embeddedTypes(sf, st) {
if emb[0] != frozen {
continue
}
out = append(out, s.finding(Error, checkModelTimeMix, sf, st,
"runtime model %s (table %s) embeds %s.%s, whose DeletedAt is the nullable pre-conversion shape;\n"+
" gorm will query this table with deleted_at IS NULL while live rows hold 0, and it will return nothing.\n"+
" Embed %s.ModelTime instead.",
name, table, pkgFrozenModels, emb[1], pkgRuntimeModel))
}
}
}
version, isMigration := migrationVersion(sf.Path)
if !isMigration || version <= softDeleteConversion || !sf.Imports(frozen) {
continue
}
spec := sf.ImportSpec(frozen)
out = append(out, s.finding(Error, checkModelTimeMix, sf, spec,
"migration %d is ordered after the soft-delete conversion (%d) but seeds through %s;\n"+
" that package writes a nullable deleted_at into a NOT NULL column, and reads through it match no rows.\n"+
" Use the runtime models under app/ instead.",
version, softDeleteConversion, pkgFrozenModels))
}
return out
}
// softDeleteConversion is the version at which deleted_at stopped being a
// nullable timestamp and became the NOT NULL millisecond marker.
const softDeleteConversion = 1786700003000
// ---------------------------------------------------------------------------
// check 2: menu sort overflows a tinyint
// ---------------------------------------------------------------------------
// checkMenuSortOverflow reports a seeded menu sort outside a tinyint.
//
// sys_menu.sort is `gorm:"size:4"`, which MySQL builds as a tinyint holding
// -128..127. sqlite ignores the width, so an overflowing value passes every
// local test and fails on a real install - with Error 1264, partway through a
// migration that is not transactional, leaving every later migration unapplied.
// That is how a seeded Sort: 900 once stopped the run before the soft-delete
// conversion and left nobody able to log in.
func checkMenuSortOverflow(s *snapshot) []Finding {
const (
min = -128
max = 127
)
var out []Finding
for _, sf := range s.Files {
forEachStructLiteral(sf, func(lit structLiteral) {
if !s.isMenuModel(lit) {
return
}
expr, ok := field(lit.Lit, "Sort")
if !ok {
return
}
v, ok := intValue(sf, expr)
if !ok || (v >= min && v <= max) {
return
}
out = append(out, s.finding(Error, checkMenuSort, sf, expr,
"menu sort %d does not fit a tinyint (%d..%d);\n"+
" MySQL rejects it with Error 1264 and the migration stops there, leaving later migrations unapplied.",
v, min, max))
})
}
return out
}
// ---------------------------------------------------------------------------
// check 4: sys_config value truncation
// ---------------------------------------------------------------------------
// checkConfigValueLength reports a seeded sys_config value longer than the
// column.
//
// config_value is varchar(255). MySQL outside strict mode truncates rather than
// refusing, so the migration succeeds, the row is written, and the setting is
// silently half of what was intended.
//
// Counted in runes, not bytes, because varchar(255) counts characters - byte
// counting would flag Chinese values that fit.
func checkConfigValueLength(s *snapshot) []Finding {
const limit = 255
var out []Finding
for _, sf := range s.Files {
forEachStructLiteral(sf, func(lit structLiteral) {
if lit.Name != "SysConfig" || !s.isModelPackage(lit.PkgPath) {
return
}
expr, ok := field(lit.Lit, "ConfigValue")
if !ok {
return
}
v, ok := stringValue(sf, expr)
if !ok {
return
}
if n := len([]rune(v)); n > limit {
out = append(out, s.finding(Error, checkConfigValue, sf, expr,
"sys_config.config_value is %d characters, over the varchar(%d) column;\n"+
" MySQL outside strict mode truncates instead of failing, so the migration succeeds with half the value.",
n, limit))
}
})
}
return out
}
// ---------------------------------------------------------------------------
// check 5: hard-coded menu ids colliding
// ---------------------------------------------------------------------------
// checkMenuIDCollisions reports the same menu id seeded from more than one file.
//
// menu_id is the primary key and every seed is an upsert, so two modules that
// pick the same id do not collide loudly - the second overwrites the first, and
// which one wins depends on migration order. One module's menu quietly becomes
// the other's.
//
// Only across files. Inside one file the same id appearing twice is the same
// menu being written and then referenced, which is how the seeds are written
// today and not a mistake.
func checkMenuIDCollisions(s *snapshot) []Finding {
type site struct {
sf *sourceFile
expr posLike
file string
line int
}
sites := map[int64][]site{}
for _, sf := range s.Files {
forEachStructLiteral(sf, func(lit structLiteral) {
if !s.isMenuModel(lit) {
return
}
expr, ok := field(lit.Lit, "MenuId")
if !ok {
return
}
v, ok := intValue(sf, expr)
if !ok {
return
}
file, line, _ := s.Pos(sf, expr.Pos())
sites[v] = append(sites[v], site{sf: sf, expr: expr, file: file, line: line})
})
}
ids := make([]int64, 0, len(sites))
for id := range sites {
ids = append(ids, id)
}
sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] })
var out []Finding
for _, id := range ids {
group := sites[id]
files := map[string]bool{}
for _, st := range group {
files[st.file] = true
}
if len(files) < 2 {
continue
}
sort.Slice(group, func(i, j int) bool {
if group[i].file != group[j].file {
return group[i].file < group[j].file
}
return group[i].line < group[j].line
})
related := make([]string, 0, len(group)-1)
for _, st := range group[1:] {
related = append(related, fmt.Sprintf("also at %s:%d", st.file, st.line))
}
f := s.finding(Error, checkMenuIDConflict, group[0].sf, group[0].expr,
"menu id %d is seeded from %d files;\n"+
" menu_id is the primary key and the seeds upsert, so whichever migration runs last overwrites the other's menu.",
id, len(files))
f.Related = related
out = append(out, f)
}
return out
}
// ---------------------------------------------------------------------------
// check 6: the contract packages must not import app/
// ---------------------------------------------------------------------------
// contractRoots are the trees that may not depend on a business module. core/
// does not exist in this repository yet and is listed because the boundary is
// declared for both in docs/contract.md; naming it here means the check is
// already in place the day the directory appears.
//
// Which of them actually exist is reported by ScannedContractRoots, because a
// root that is absent contributes nothing and a check that silently covers less
// than it claims is worse than no check: it leaves people believing a boundary
// is guarded when nothing is guarding it.
var contractRoots = []string{"common/", "core/"}
// ScannedContractRoots splits contractRoots by whether the snapshot actually
// holds files under them, so the summary can name what was covered.
func ScannedContractRoots(s *snapshot) (scanned, absent []string) {
for _, root := range contractRoots {
found := false
for _, sf := range s.Files {
if strings.HasPrefix(sf.Path, root) {
found = true
break
}
}
if found {
scanned = append(scanned, strings.TrimSuffix(root, "/"))
} else {
absent = append(absent, strings.TrimSuffix(root, "/"))
}
}
return scanned, absent
}
// checkContractImportBoundary reports a contract package importing app/.
//
// docs/contract.md promises four packages under common/ as the surface an app
// may build on. A promise like that stops being true the moment the surface
// imports one particular app: a fork that replaces app/admin then cannot
// compile common/middleware, and an app can no longer be built against the
// contract alone. Nothing about it fails visibly - it fails when somebody tries
// to take the framework apart, which is the whole point of the exercise.
//
// Test files count. A fork that drops app/admin should be able to run go test
// ./... too.
func checkContractImportBoundary(s *snapshot) []Finding {
appPrefix := s.ModulePath + "/app/"
var out []Finding
for _, sf := range s.Files {
inContract := false
for _, root := range contractRoots {
if strings.HasPrefix(sf.Path, root) {
inContract = true
break
}
}
if !inContract {
continue
}
for _, spec := range sf.Syntax.Imports {
path, err := strconv.Unquote(spec.Path.Value)
if err != nil || !strings.HasPrefix(path, appPrefix) {
continue
}
out = append(out, s.finding(Error, checkImportBoundary, sf, spec,
"%s is a contract package and imports %s;\n"+
" a fork that replaces or drops that app can then no longer compile the contract surface it was told to build on.\n"+
" Move what is shared down into common/, or out of the contract package.",
sf.Path, path))
}
}
return out
}
// migrationVersion reads the 13-digit timestamp a migration file name starts
// with. Files outside the two migration directories are not migrations, however
// they are named.
func migrationVersion(rel string) (int64, bool) {
dir := path.Dir(rel)
if dir != "cmd/migrate/migration/version" && dir != "cmd/migrate/migration/version-local" {
return 0, false
}
name := path.Base(rel)
if len(name) < 13 {
return 0, false
}
v, err := strconv.ParseInt(name[:13], 10, 64)
if err != nil {
return 0, false
}
return v, true
}
// isMenuModel reports whether a literal is one of the SysMenu models rather
// than, say, the SysMenu service struct that shares the name.
func (s *snapshot) isMenuModel(lit structLiteral) bool {
return lit.Name == "SysMenu" && s.isModelPackage(lit.PkgPath)
}
func (s *snapshot) isModelPackage(path string) bool {
return path == s.pkg(pkgFrozenModels) || path == s.pkg(pkgAdminModels)
}
+454
View File
@@ -0,0 +1,454 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
// fixture writes a miniature repository and returns its root. Every check gets
// one of these carrying the exact mistake it exists to find, so a check that
// stops working fails a test rather than going quiet - which would be the same
// failure mode the tool is about.
func fixture(t *testing.T, files map[string]string) string {
t.Helper()
dir := t.TempDir()
files["go.mod"] = "module go-admin\n\ngo 1.26\n"
for name, content := range files {
path := filepath.Join(dir, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
}
return dir
}
func check(t *testing.T, root string, opt options) []Finding {
t.Helper()
s, err := load(root)
if err != nil {
t.Fatalf("load: %v", err)
}
findings, err := runChecks(s, opt)
if err != nil {
t.Fatalf("run: %v", err)
}
return findings
}
func only(t *testing.T, findings []Finding, name string) []Finding {
t.Helper()
var out []Finding
for _, f := range findings {
if f.Check == name {
out = append(out, f)
}
}
return out
}
func requireOne(t *testing.T, findings []Finding, name string) Finding {
t.Helper()
got := only(t, findings, name)
if len(got) != 1 {
t.Fatalf("%s produced %d findings, want 1:\n%v", name, len(got), findings)
}
return got[0]
}
const frozenModelsPkg = `package models
import (
"time"
"gorm.io/gorm"
)
type ModelTime struct {
CreatedAt time.Time
DeletedAt gorm.DeletedAt
}
type SysMenu struct {
MenuId int
MenuName string
Component string
MenuType string
Sort int
ModelTime
}
func (SysMenu) TableName() string { return "sys_menu" }
type SysConfig struct {
ConfigKey string
ConfigValue string
ModelTime
}
func (SysConfig) TableName() string { return "sys_config" }
`
// ---------------------------------------------------------------------------
func TestModelTimeMixDetectsARuntimeModelOnTheFrozenShape(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"app/shop/models/product.go": `package models
import frozen "go-admin/cmd/migrate/migration/models"
type Product struct {
Id int
frozen.ModelTime
}
func (Product) TableName() string { return "shop_product" }
`,
})
f := requireOne(t, check(t, root, options{}), checkModelTimeMix)
if f.Severity != "ERROR" {
t.Errorf("severity = %s", f.Severity)
}
if !strings.Contains(f.Message, "shop_product") {
t.Errorf("message = %s", f.Message)
}
if f.File != "app/shop/models/product.go" {
t.Errorf("file = %s", f.File)
}
}
func TestModelTimeMixDetectsAPostConversionMigrationOnTheFrozenPackage(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version-local/1786700009000_seed.go": `package version_local
import "go-admin/cmd/migrate/migration/models"
func seed() interface{} { return &models.SysMenu{} }
`,
})
f := requireOne(t, check(t, root, options{}), checkModelTimeMix)
if !strings.Contains(f.Message, "1786700009000") {
t.Errorf("message = %s", f.Message)
}
if f.Line != 3 {
t.Errorf("expected the import line, got line %d", f.Line)
}
}
// A migration ordered before the conversion is right to use that package - the
// nullable column is the shape it had at the time.
func TestModelTimeMixLeavesPreConversionMigrationsAlone(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
import "go-admin/cmd/migrate/migration/models"
func seed() interface{} { return &models.SysMenu{} }
`,
})
if got := only(t, check(t, root, options{}), checkModelTimeMix); len(got) != 0 {
t.Errorf("reported %v", got)
}
}
// ---------------------------------------------------------------------------
func TestMenuSortOverflowIsDetectedThroughAnElidedSliceLiteral(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
import "go-admin/cmd/migrate/migration/models"
func seed() []models.SysMenu {
return []models.SysMenu{
{MenuId: 9000, Sort: 100},
{MenuId: 9001, Sort: 900},
}
}
`,
})
f := requireOne(t, check(t, root, options{}), checkMenuSort)
if f.Severity != "ERROR" || !strings.Contains(f.Message, "900") {
t.Errorf("finding = %+v", f)
}
if f.Line != 8 {
t.Errorf("line = %d, want the overflowing element", f.Line)
}
}
// The SysMenu of the service layer shares the name and has nothing to do with
// the column. Reporting it would be the false positive that gets the tool
// switched off.
func TestMenuSortIgnoresSameNamedTypesFromOtherPackages(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"app/admin/service/sys_menu.go": `package service
type SysMenu struct{ Sort int }
`,
"app/admin/apis/sys_menu.go": `package apis
import "go-admin/app/admin/service"
func handler() interface{} { return service.SysMenu{Sort: 9000} }
`,
})
if got := only(t, check(t, root, options{}), checkMenuSort); len(got) != 0 {
t.Errorf("reported %v", got)
}
}
// ---------------------------------------------------------------------------
func TestConfigValueTruncationIsDetected(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
import "go-admin/cmd/migrate/migration/models"
func seed() models.SysConfig {
return models.SysConfig{ConfigKey: "k", ConfigValue: "` + strings.Repeat("a", 300) + `"}
}
`,
})
f := requireOne(t, check(t, root, options{}), checkConfigValue)
if f.Severity != "ERROR" || !strings.Contains(f.Message, "300 characters") {
t.Errorf("finding = %+v", f)
}
}
// varchar(255) counts characters. Counting bytes would report a Chinese value
// that fits perfectly well.
func TestConfigValueCountsRunesNotBytes(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
import "go-admin/cmd/migrate/migration/models"
func seed() models.SysConfig {
return models.SysConfig{ConfigValue: "` + strings.Repeat("中", 200) + `"}
}
`,
})
if got := only(t, check(t, root, options{}), checkConfigValue); len(got) != 0 {
t.Errorf("200 Chinese characters fit in varchar(255) but were reported: %v", got)
}
}
// ---------------------------------------------------------------------------
func TestMenuIDCollisionAcrossFilesIsDetectedThroughConstants(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version/1786700001000_crm.go": `package version
import "go-admin/cmd/migrate/migration/models"
const crmMenuId = 9000
func seedCrm() models.SysMenu { return models.SysMenu{MenuId: crmMenuId, MenuName: "CRM"} }
`,
"cmd/migrate/migration/version/1786700002000_oms.go": `package version
import "go-admin/cmd/migrate/migration/models"
func seedOms() models.SysMenu { return models.SysMenu{MenuId: 9000, MenuName: "OMS"} }
`,
})
f := requireOne(t, check(t, root, options{}), checkMenuIDConflict)
if f.Severity != "ERROR" || !strings.Contains(f.Message, "9000") {
t.Errorf("finding = %+v", f)
}
// The second site has to be printed too, or the report says a collision
// happened without saying with what.
if len(f.Related) != 1 || !strings.Contains(f.Related[0], "1786700002000_oms.go") {
t.Errorf("related = %v", f.Related)
}
}
// The seeds write a menu and then refer to it again to attach permissions. That
// is one menu, not two modules fighting over an id.
func TestMenuIDRepeatedWithinOneFileIsNotACollision(t *testing.T) {
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version/1786700001000_crm.go": `package version
import "go-admin/cmd/migrate/migration/models"
func seed() []models.SysMenu {
dir := models.SysMenu{MenuId: 9000, MenuName: "CRM"}
again := models.SysMenu{MenuId: 9000, MenuName: "CRM"}
return []models.SysMenu{dir, again}
}
`,
})
if got := only(t, check(t, root, options{}), checkMenuIDConflict); len(got) != 0 {
t.Errorf("reported %v", got)
}
}
// ---------------------------------------------------------------------------
func TestContractImportBoundaryIsDetected(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/service/dto/log.go": `package dto
const OperaStatusEnabel = "1"
`,
"common/middleware/logger.go": `package middleware
import "go-admin/app/admin/service/dto"
func status() string { return dto.OperaStatusEnabel }
`,
})
f := requireOne(t, check(t, root, options{}), checkImportBoundary)
if f.Severity != "ERROR" {
t.Errorf("severity = %s", f.Severity)
}
if !strings.Contains(f.Message, "go-admin/app/admin/service/dto") {
t.Errorf("message = %s", f.Message)
}
if f.File != "common/middleware/logger.go" || f.Line != 3 {
t.Errorf("position = %s:%d", f.File, f.Line)
}
}
// A fork that drops app/admin should be able to run the tests too, so a
// test-only import is the same violation.
func TestContractImportBoundaryCoversTestFiles(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
"common/actions/permission_test.go": `package actions
import "go-admin/app/admin/models"
var _ = models.SysUser{}
`,
})
if got := only(t, check(t, root, options{}), checkImportBoundary); len(got) != 1 {
t.Errorf("findings = %v", got)
}
}
func TestContractImportBoundaryAllowsAppToImportCommon(t *testing.T) {
root := fixture(t, map[string]string{
"common/models/by.go": "package models\n\ntype ControlBy struct{}\n",
"app/demo/models/product.go": `package models
import "go-admin/common/models"
type Product struct{ models.ControlBy }
`,
})
if got := only(t, check(t, root, options{}), checkImportBoundary); len(got) != 0 {
t.Errorf("the dependency direction that is allowed was reported: %v", got)
}
}
// ---------------------------------------------------------------------------
func menuNameFixture(t *testing.T, menuName, componentName string) (string, string) {
t.Helper()
root := fixture(t, map[string]string{
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
import "go-admin/cmd/migrate/migration/models"
func seed() models.SysMenu {
return models.SysMenu{MenuId: 9001, MenuName: "` + menuName + `", MenuType: "C", Component: "/demo/product/index"}
}
`,
})
ui := t.TempDir()
path := filepath.Join(ui, "views", "demo", "product", "index.vue")
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
t.Fatal(err)
}
vue := "<script setup>\ndefineOptions({ name: '" + componentName + "' })\n</script>\n<template><div/></template>\n"
if err := os.WriteFile(path, []byte(vue), 0o644); err != nil {
t.Fatal(err)
}
return root, ui
}
func TestMenuNameMismatchIsDetectedAsAWarning(t *testing.T) {
root, ui := menuNameFixture(t, "DemoProduct", "Product")
f := requireOne(t, check(t, root, options{UIDir: ui}), checkMenuName)
if f.Severity != "WARN" {
t.Errorf("severity = %s; this check must not decide an exit code yet", f.Severity)
}
if !strings.Contains(f.Message, "DemoProduct") || !strings.Contains(f.Message, "Product") {
t.Errorf("message = %s", f.Message)
}
// Acceptance 14c.
if !strings.Contains(f.Message, "heuristic") || !strings.Contains(f.Message, "false positives are possible") {
t.Errorf("the warning must say it is a heuristic:\n%s", f.Message)
}
}
func TestMenuNameMatchIsSilent(t *testing.T) {
root, ui := menuNameFixture(t, "DemoProduct", "DemoProduct")
if got := only(t, check(t, root, options{UIDir: ui}), checkMenuName); len(got) != 0 {
t.Errorf("reported %v", got)
}
}
// Without the frontend repository there is nothing to compare against, and the
// check has to disappear rather than guess.
func TestMenuNameIsSkippedWithoutTheUIDirectory(t *testing.T) {
root, _ := menuNameFixture(t, "DemoProduct", "Product")
if got := only(t, check(t, root, options{}), checkMenuName); len(got) != 0 {
t.Errorf("reported %v without -ui-dir", got)
}
}
// A component the frontend does not carry is the "app not installed" case F2
// handles with a placeholder; this check has nothing to say about it.
func TestMenuNameIsSilentWhenTheComponentIsMissing(t *testing.T) {
root, _ := menuNameFixture(t, "DemoProduct", "Product")
empty := t.TempDir()
if got := only(t, check(t, root, options{UIDir: empty}), checkMenuName); len(got) != 0 {
t.Errorf("reported %v", got)
}
}
func TestComponentNameParsesBothVueStyles(t *testing.T) {
for _, tc := range []struct {
name string
src string
want string
}{
{"script setup", "<script setup>\ndefineOptions({ name: 'Product' })\n</script>", "Product"},
{"options api", "<script>\nexport default {\n name: 'Product',\n data() {}\n}\n</script>", "Product"},
{"defineComponent", "<script>\nexport default defineComponent({\n name: 'Product'\n})\n</script>", "Product"},
} {
t.Run(tc.name, func(t *testing.T) {
got, ok := componentName(tc.src)
if !ok || got != tc.want {
t.Errorf("componentName = %q, %v", got, ok)
}
})
}
if _, ok := componentName("<script setup>\nconst a = 1\n</script>"); ok {
t.Error("a component with no declared name must not be compared")
}
}
+78
View File
@@ -0,0 +1,78 @@
package main
import (
"fmt"
"sort"
)
// Severity decides whether a finding stops CI.
//
// Two levels and no more. A third would immediately be used to park findings
// nobody intends to fix, and the point of this tool is that everything it
// reports is something that fails without saying so.
type Severity int
const (
// Warn prints and does not affect the exit code.
Warn Severity = iota
// Error prints and makes the run fail.
Error
)
func (s Severity) String() string {
if s == Error {
return "ERROR"
}
return "WARN"
}
// Finding is one problem, located precisely enough to open the file at it.
type Finding struct {
Check string `json:"check"`
Severity string `json:"severity"`
File string `json:"file"`
Line int `json:"line"`
Col int `json:"col"`
Message string `json:"message"`
Related []string `json:"related,omitempty"`
severity Severity
}
func (f Finding) String() string {
s := fmt.Sprintf("%s:%d:%d: [%s] %s: %s", f.File, f.Line, f.Col, f.Severity, f.Check, f.Message)
for _, r := range f.Related {
s += "\n " + r
}
return s
}
// sortFindings orders by file, then position, then check, so two runs over the
// same tree print the same thing and a diff of the output is meaningful.
func sortFindings(fs []Finding) {
sort.Slice(fs, func(i, j int) bool {
a, b := fs[i], fs[j]
if a.File != b.File {
return a.File < b.File
}
if a.Line != b.Line {
return a.Line < b.Line
}
if a.Col != b.Col {
return a.Col < b.Col
}
if a.Check != b.Check {
return a.Check < b.Check
}
return a.Message < b.Message
})
}
func hasError(fs []Finding) bool {
for _, f := range fs {
if f.severity == Error {
return true
}
}
return false
}
+105
View File
@@ -0,0 +1,105 @@
// Command checksilent reports the failures in this repository that do not
// announce themselves: no error, no log line, behaviour quietly wrong.
//
// Six checks, five of them ERROR and one WARN. An ERROR fails the run; a WARN
// prints and does not. The split is not about how bad the consequence is - all
// six are bad - but about how certain the detection is. Everything reported as
// an ERROR is decided from this repository's own syntax. The one WARN compares
// against a second repository through a regular expression, and a check that
// can be wrong must not be able to stop a build, or the first response to it
// will be an ignore comment.
//
// Usage:
//
// go run ./tools/checksilent
// go run ./tools/checksilent -ui-dir ../go-admin-ui/src
// go run ./tools/checksilent -json
//
// Or through the Makefile, which is what CI runs:
//
// make checksilent
package main
import (
"encoding/json"
"flag"
"fmt"
"io"
"os"
"strings"
)
func main() {
var (
root = flag.String("root", ".", "repository root to scan")
uiDir = flag.String("ui-dir", "", "go-admin-ui src directory; enables the menu-name check, which is skipped without it")
asJSON = flag.Bool("json", false, "print findings as JSON")
)
flag.Parse()
code, err := run(os.Stdout, *root, options{UIDir: *uiDir}, *asJSON)
if err != nil {
fmt.Fprintln(os.Stderr, "checksilent:", err)
os.Exit(2)
}
os.Exit(code)
}
// run returns the process exit code: 0 when nothing or only warnings were
// found, 1 when at least one error was.
func run(w io.Writer, root string, opt options, asJSON bool) (int, error) {
s, err := load(root)
if err != nil {
return 0, err
}
findings, err := runChecks(s, opt)
if err != nil {
return 0, err
}
if asJSON {
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
if err = enc.Encode(findings); err != nil {
return 0, err
}
} else {
for _, f := range findings {
fmt.Fprintln(w, f)
}
printSummary(w, findings, opt, s)
}
if hasError(findings) {
return 1, nil
}
return 0, nil
}
func printSummary(w io.Writer, findings []Finding, opt options, s *snapshot) {
var errors, warnings int
for _, f := range findings {
if f.severity == Error {
errors++
} else {
warnings++
}
}
if len(findings) > 0 {
fmt.Fprintln(w)
}
fmt.Fprintf(w, "checksilent: %d error(s), %d warning(s)\n", errors, warnings)
if warnings > 0 {
fmt.Fprintln(w, "Warnings do not affect the exit code.")
}
if opt.UIDir == "" {
fmt.Fprintf(w, "The %s check was skipped: pass -ui-dir <go-admin-ui>/src to run it.\n", checkMenuName)
}
// Said out loud so nobody reads a clean run as "the boundary holds
// everywhere it was declared". core/ is a separate module and has no
// directory here, so this repository's copy of the check cannot cover it.
if scanned, absent := ScannedContractRoots(s); len(absent) > 0 {
fmt.Fprintf(w, "The %s check covered %s; %s does not exist here and was not scanned.\n",
checkImportBoundary, strings.Join(scanned, ", "), strings.Join(absent, ", "))
}
}
+138
View File
@@ -0,0 +1,138 @@
package main
import (
"bytes"
"encoding/json"
"strings"
"testing"
)
// Acceptance 14b, second half: the WARN check on its own leaves the exit code
// at zero. If it did not, the first person it misjudged would silence it, and a
// silenced check is worse than none - it looks like coverage.
func TestOnlyWarningsExitZero(t *testing.T) {
root, ui := menuNameFixture(t, "DemoProduct", "Product")
var buf bytes.Buffer
code, err := run(&buf, root, options{UIDir: ui}, false)
if err != nil {
t.Fatal(err)
}
if code != 0 {
t.Errorf("exit code = %d, want 0:\n%s", code, buf.String())
}
out := buf.String()
if !strings.Contains(out, "[WARN]") {
t.Errorf("the warning was not printed:\n%s", out)
}
if !strings.Contains(out, "0 error(s), 1 warning(s)") {
t.Errorf("summary = %s", out)
}
if !strings.Contains(out, "Warnings do not affect the exit code.") {
t.Errorf("output must say warnings are not fatal:\n%s", out)
}
}
// Acceptance 14b, first half: any of the other five fails the run.
func TestAnyErrorExitsNonZero(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
"common/middleware/auth.go": `package middleware
import "go-admin/app/admin/models"
var _ = models.SysUser{}
`,
})
var buf bytes.Buffer
code, err := run(&buf, root, options{}, false)
if err != nil {
t.Fatal(err)
}
if code != 1 {
t.Errorf("exit code = %d, want 1:\n%s", code, buf.String())
}
if !strings.Contains(buf.String(), "[ERROR]") {
t.Errorf("output = %s", buf.String())
}
}
func TestCleanTreeExitsZero(t *testing.T) {
root := fixture(t, map[string]string{
"common/models/by.go": "package models\n\ntype ControlBy struct{}\n",
})
var buf bytes.Buffer
code, err := run(&buf, root, options{}, false)
if err != nil {
t.Fatal(err)
}
if code != 0 {
t.Errorf("exit code = %d, want 0:\n%s", code, buf.String())
}
if !strings.Contains(buf.String(), "0 error(s), 0 warning(s)") {
t.Errorf("output = %s", buf.String())
}
}
// Every message has to name a file and a line, or the report is a puzzle rather
// than a finding.
func TestTextOutputLocatesEveryFinding(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
"common/middleware/auth.go": `package middleware
import "go-admin/app/admin/models"
var _ = models.SysUser{}
`,
})
var buf bytes.Buffer
if _, err := run(&buf, root, options{}, false); err != nil {
t.Fatal(err)
}
line := strings.SplitN(buf.String(), "\n", 2)[0]
if !strings.HasPrefix(line, "common/middleware/auth.go:3:") {
t.Errorf("first line does not locate the finding: %q", line)
}
}
func TestJSONOutput(t *testing.T) {
root := fixture(t, map[string]string{
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
"common/middleware/auth.go": `package middleware
import "go-admin/app/admin/models"
var _ = models.SysUser{}
`,
})
var buf bytes.Buffer
code, err := run(&buf, root, options{}, true)
if err != nil {
t.Fatal(err)
}
if code != 1 {
t.Errorf("exit code = %d", code)
}
var findings []Finding
if err = json.Unmarshal(buf.Bytes(), &findings); err != nil {
t.Fatalf("output is not JSON: %v\n%s", err, buf.String())
}
if len(findings) != 1 || findings[0].Check != checkImportBoundary || findings[0].Severity != "ERROR" {
t.Errorf("findings = %+v", findings)
}
}
// The tool runs on this repository in CI, so it has to be clean here. This also
// covers acceptance 13b: nothing under common/ imports app/ any more.
func TestThisRepositoryIsClean(t *testing.T) {
var buf bytes.Buffer
code, err := run(&buf, "../..", options{}, false)
if err != nil {
t.Fatal(err)
}
if code != 0 {
t.Errorf("checksilent reports problems in this repository:\n%s", buf.String())
}
}
+131
View File
@@ -0,0 +1,131 @@
package main
import (
"fmt"
"os"
"path/filepath"
"regexp"
"strings"
)
// heuristicNote is appended to every menu-name finding.
//
// Required by the acceptance criteria, and for a reason worth restating: this
// is the one check that compares two repositories through a regular expression,
// so it will occasionally be wrong. Reporting it as a certainty is how a tool
// gets a reputation and then an ignore comment on every line.
const heuristicNote = "\n (heuristic: matched across repositories by regular expression - false positives are possible, verify before changing anything)"
var (
// Vue 3 <script setup>, which is the prevailing style in go-admin-ui.
defineOptionsName = regexp.MustCompile(`defineOptions\(\s*\{[^}]*?name:\s*['"` + "`" + `]([^'"` + "`" + `]+)['"` + "`" + `]`)
// Options API, still present in older views.
exportDefaultName = regexp.MustCompile(`export default\s*(?:defineComponent\(\s*)?\{[^}]*?name:\s*['"` + "`" + `]([^'"` + "`" + `]+)['"` + "`" + `]`)
)
// checkMenuNames compares the menu_name a migration seeds against the name the
// component declares.
//
// keep-alive caches by component name, and the page cache is configured by menu
// name. When they disagree nothing breaks and nothing is logged - the page just
// stops being cached, or the wrong page is evicted, and it looks like a
// performance quirk.
//
// WARN, not ERROR, and deliberately so. The two sides are in different
// repositories and different languages, so this is a regular expression against
// a .vue file, not a parse. It is scheduled to become an ERROR after two release
// cycles with no false positive; until then it must not decide an exit code.
func checkMenuNames(s *snapshot, uiDir string) ([]Finding, error) {
info, err := os.Stat(uiDir)
if err != nil {
return nil, fmt.Errorf("ui dir %s: %w", uiDir, err)
}
if !info.IsDir() {
return nil, fmt.Errorf("ui dir %s is not a directory", uiDir)
}
var out []Finding
for _, sf := range s.Files {
forEachStructLiteral(sf, func(lit structLiteral) {
if !s.isMenuModel(lit) {
return
}
menuType, _ := stringField(sf, lit, "MenuType")
// A directory has no page of its own; its component is the layout.
if menuType == "M" || menuType == "F" {
return
}
component, ok := stringField(sf, lit, "Component")
if !ok || component == "" || component == "Layout" {
return
}
menuName, ok := stringField(sf, lit, "MenuName")
if !ok || menuName == "" {
return
}
path := componentFile(uiDir, component)
if path == "" {
return
}
b, err := os.ReadFile(path)
if err != nil {
// A component the frontend does not carry is F2's business -
// the placeholder that says the app is not installed - not this
// check's.
return
}
declared, ok := componentName(string(b))
if !ok {
// No literal name at all: the build may derive one from the
// file path. Nothing to compare, so nothing to say.
return
}
if declared == menuName {
return
}
expr, _ := field(lit.Lit, "MenuName")
rel := path
if r, err := filepath.Rel(uiDir, path); err == nil {
rel = r
}
out = append(out, s.finding(Warn, checkMenuName, sf, expr,
"menu_name %q does not match the component name %q declared in %s;\n"+
" keep-alive caches by component name and the cache is configured by menu name, so the page silently stops being cached.%s",
menuName, declared, rel, heuristicNote))
})
}
return out, nil
}
func stringField(sf *sourceFile, lit structLiteral, name string) (string, bool) {
expr, ok := field(lit.Lit, name)
if !ok {
return "", false
}
return stringValue(sf, expr)
}
// componentFile maps a menu component path to a file in the frontend tree.
//
// Two roots, because F2 adds a second: anything under apps/ is an installed
// app's view, everything else is a view of the main repository.
func componentFile(uiDir, component string) string {
c := strings.TrimPrefix(filepath.ToSlash(component), "/")
if c == "" {
return ""
}
if strings.HasPrefix(c, "apps/") {
return filepath.Join(uiDir, filepath.FromSlash(c)+".vue")
}
return filepath.Join(uiDir, "views", filepath.FromSlash(c)+".vue")
}
func componentName(src string) (string, bool) {
for _, re := range []*regexp.Regexp{defineOptionsName, exportDefaultName} {
if m := re.FindStringSubmatch(src); m != nil {
return m[1], true
}
}
return "", false
}
+203
View File
@@ -0,0 +1,203 @@
package main
import (
"fmt"
"go/ast"
"go/parser"
"go/token"
"os"
"path/filepath"
"strconv"
"strings"
)
// skippedDirs are never walked. Generated and vendored trees would produce
// findings nobody can act on, and node_modules would make the walk the slowest
// part of CI.
var skippedDirs = map[string]bool{
".git": true,
".idea": true,
"node_modules": true,
"vendor": true,
"testdata": true,
"dist": true,
"temp": true,
}
// sourceFile is one parsed Go file plus the mapping from the local names it
// uses for packages to the paths those names stand for.
//
// Resolving through the file's own import block, rather than matching on the
// identifier, is what keeps service.SysMenu apart from models.SysMenu.
type sourceFile struct {
// Path is relative to the scanned root and is what gets printed.
Path string
Pkg string // import path of the package this file belongs to
Syntax *ast.File
imports map[string]string // local name -> import path
consts map[string]int64 // package-level integer constants, filled per package
}
// snapshot is every Go file under the root, parsed once and shared by all the
// checks.
type snapshot struct {
Root string
ModulePath string
Fset *token.FileSet
Files []*sourceFile
}
// load parses every Go file under root.
//
// Type checking is deliberately not done. Every rule here is about a literal
// written into a seed or an import that should not be there, and both are in
// the syntax; a type checker would drag in a resolvable build of the whole
// module, which is exactly what a check meant to run on a half-broken tree
// cannot depend on.
func load(root string) (*snapshot, error) {
modulePath, err := readModulePath(root)
if err != nil {
return nil, err
}
s := &snapshot{Root: root, ModulePath: modulePath, Fset: token.NewFileSet()}
err = filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
if path != root && skippedDirs[info.Name()] {
return filepath.SkipDir
}
return nil
}
if !strings.HasSuffix(path, ".go") {
return nil
}
rel, err := filepath.Rel(root, path)
if err != nil {
return err
}
rel = filepath.ToSlash(rel)
f, err := parser.ParseFile(s.Fset, path, nil, 0)
if err != nil {
// A file that does not parse is someone's work in progress, not a
// silent failure. Reporting it here would only repeat what the
// compiler already says louder.
return nil
}
dir := filepath.ToSlash(filepath.Dir(rel))
pkg := modulePath
if dir != "." {
pkg = modulePath + "/" + dir
}
s.Files = append(s.Files, &sourceFile{
Path: rel,
Pkg: pkg,
Syntax: f,
imports: fileImports(f),
})
return nil
})
if err != nil {
return nil, err
}
s.resolveConstants()
return s, nil
}
func readModulePath(root string) (string, error) {
b, err := os.ReadFile(filepath.Join(root, "go.mod"))
if err != nil {
return "", fmt.Errorf("read go.mod: %w", err)
}
for _, line := range strings.Split(string(b), "\n") {
line = strings.TrimSpace(line)
if rest, ok := strings.CutPrefix(line, "module "); ok {
return strings.TrimSpace(rest), nil
}
}
return "", fmt.Errorf("%s has no module line", filepath.Join(root, "go.mod"))
}
func fileImports(f *ast.File) map[string]string {
out := make(map[string]string, len(f.Imports))
for _, spec := range f.Imports {
path, err := strconv.Unquote(spec.Path.Value)
if err != nil {
continue
}
name := path[strings.LastIndex(path, "/")+1:]
if spec.Name != nil {
name = spec.Name.Name
}
out[name] = path
}
return out
}
// resolveConstants collects package-level integer constants per directory, so a
// seed written as MenuId: demoMenuId can be compared against one written as
// MenuId: 9000. Without this the menu-id check would see the well-written code
// and miss exactly the collisions it exists to find.
func (s *snapshot) resolveConstants() {
byPkg := map[string]map[string]int64{}
for _, sf := range s.Files {
consts, ok := byPkg[sf.Pkg]
if !ok {
consts = map[string]int64{}
byPkg[sf.Pkg] = consts
}
for _, decl := range sf.Syntax.Decls {
gen, ok := decl.(*ast.GenDecl)
if !ok || gen.Tok != token.CONST {
continue
}
for _, spec := range gen.Specs {
vs, ok := spec.(*ast.ValueSpec)
if !ok {
continue
}
for i, name := range vs.Names {
if i >= len(vs.Values) {
continue
}
if v, ok := intLiteral(vs.Values[i]); ok {
consts[name.Name] = v
}
}
}
}
}
for _, sf := range s.Files {
sf.consts = byPkg[sf.Pkg]
}
}
// Pos turns a token position into the file/line/col a Finding carries.
func (s *snapshot) Pos(sf *sourceFile, p token.Pos) (string, int, int) {
pos := s.Fset.Position(p)
return sf.Path, pos.Line, pos.Column
}
// Imports reports whether the file imports path.
func (sf *sourceFile) Imports(path string) bool {
for _, p := range sf.imports {
if p == path {
return true
}
}
return false
}
// ImportSpec returns the import declaration for path, for reporting a position
// on the import line itself.
func (sf *sourceFile) ImportSpec(path string) *ast.ImportSpec {
for _, spec := range sf.Syntax.Imports {
if p, err := strconv.Unquote(spec.Path.Value); err == nil && p == path {
return spec
}
}
return nil
}