mirror of
https://github.com/go-admin-team/go-admin.git
synced 2026-09-24 19:17:43 +00:00
Compare commits
30
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f406ca0160 | ||
|
|
4d6456a588 | ||
|
|
07ff92aa55 | ||
|
|
4156387eb9 | ||
|
|
34773a0a81 | ||
|
|
36a018400b | ||
|
|
7bb02c5f1d | ||
|
|
15fb128236 | ||
|
|
3581e060ec | ||
|
|
0604a29596 | ||
|
|
d8a2958797 | ||
|
|
ab28fa7bed | ||
|
|
e88d751039 | ||
|
|
b836945eea | ||
|
|
d7a8e66753 | ||
|
|
68780a845c | ||
|
|
487dc94a2e | ||
|
|
016e977776 | ||
|
|
dcfe512204 | ||
|
|
fe6ebfd47c | ||
|
|
eba5fba3da | ||
|
|
b7e9a79225 | ||
|
|
deffb19fd8 | ||
|
|
c858b322bd | ||
|
|
1b5b52f0f1 | ||
|
|
ecb31a158b | ||
|
|
1aecc140dc | ||
|
|
e464a4aedd | ||
|
|
d115c5299c | ||
|
|
9bd542bb59 |
@@ -49,8 +49,18 @@ model、dto、router 三个文件,完整写法照抄 `app/demo/` 的结构。
|
||||
### 4. 写菜单、接口与权限种子数据
|
||||
|
||||
这一步最容易被漏掉——代码能编译、接口能测通,但界面上看不到菜单、点了按钮说
|
||||
没权限,往往就是漏了这一步。**完整参照 `cmd/migrate/migration/version/1786700001000_demo_menu.go`**
|
||||
——那是可运行、幂等(用 `upsert`,重复跑不会报错)的真实例子,逐字照抄结构,只换 ID 和业务字段。
|
||||
没权限,往往就是漏了这一步。结构参照 `cmd/migrate/migration/version/1786700001000_demo_menu.go`
|
||||
——它是可运行、幂等(用 `upsert`,重复跑不会报错)的真实例子。
|
||||
|
||||
:::danger
|
||||
**但不要照抄它的 import。** 那个文件用的是 `cmd/migrate/migration/models`,
|
||||
只因为它的版本号排在软删除转换(`1786700003000`)之前才是安全的。
|
||||
|
||||
**你新写的迁移版本号在转换之后,必须改用 `app/` 下的运行时模型**
|
||||
(`app/admin/models.SysApi`、`SysMenu`),否则第一条 insert 就会
|
||||
`NOT NULL constraint failed: sys_api.deleted_at`。
|
||||
`TestPostConversionMigrationsAvoidFrozenSeedModels` 会拦住这个错误。
|
||||
:::
|
||||
|
||||
一个模块要在界面上可用,需要四类数据,缺一样都不行:
|
||||
|
||||
|
||||
@@ -28,9 +28,23 @@ jobs:
|
||||
|
||||
- name: Get dependencies
|
||||
run: go mod tidy
|
||||
|
||||
# go build does not compile _test.go, so building alone never ran a single
|
||||
# test. This is the only workflow that fires on every push and pull request,
|
||||
# which makes it the one place a test gate belongs.
|
||||
- name: Test
|
||||
run: make test
|
||||
|
||||
- name: Build
|
||||
run: make build
|
||||
|
||||
# Fails the build on the silent-failure classes listed in
|
||||
# tools/checksilent, one of which is the contract boundary: nothing under
|
||||
# common/ may import app/. A boundary that is only written down erodes; this
|
||||
# is what keeps it true.
|
||||
- name: Silent-failure checks
|
||||
run: make checksilent
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
|
||||
if: startsWith(github.ref, 'refs/tags/')
|
||||
|
||||
+11
@@ -6,6 +6,10 @@ main.exe
|
||||
*.exe
|
||||
go-admin
|
||||
go-admin.exe
|
||||
# `go build ./tools/checksilent` drops the binary here, next to the one for the
|
||||
# server. Anchored with a leading slash: unanchored, the same pattern matches
|
||||
# tools/checksilent/ as well and the tool's own source never gets committed.
|
||||
/checksilent
|
||||
temp/
|
||||
!temp
|
||||
vendor
|
||||
@@ -29,3 +33,10 @@ CLAUDE.md
|
||||
.claude/skills/*
|
||||
!.claude/skills/new-business-module/
|
||||
config/settings.local.dev.yml
|
||||
|
||||
# Go workspace files. They exist to point this module at a local checkout of
|
||||
# go-admin-core while the two are developed together, which is a private
|
||||
# arrangement between one machine's directories - committing one would break
|
||||
# the build for everyone else.
|
||||
go.work
|
||||
go.work.sum
|
||||
|
||||
@@ -117,6 +117,18 @@ func (SysPost) TableName() string { return "sys_post" }
|
||||
|
||||
`TableName()` 必须显式声明(GORM 配置了 `SingularTable`,不会自动推导复数)。
|
||||
|
||||
## 公共契约面
|
||||
|
||||
第三方应用(`app/` 下的业务模块)可以稳定依赖哪些包、路由与迁移怎么注册、
|
||||
哪些约束是硬的,见 `docs/contract.md`。
|
||||
|
||||
两条与主仓贡献者直接相关的:
|
||||
|
||||
- **`common/`、`core/` 不得 import `app/`** —— `make checksilent` 在 CI 里守着,违反即红。
|
||||
- **注册类 API(`AppRouters` / `sdk.Runtime.SetAppRouters` / `migration.ForApp`)
|
||||
只允许在 `init()` 中调用** —— 注册期靠 Go 的包初始化顺序保证无并发写,
|
||||
core 侧的 setter 没有加锁。
|
||||
|
||||
## 路由注册
|
||||
|
||||
通过 `init()` 自注册,不在中心文件手工添加:
|
||||
@@ -193,6 +205,45 @@ go run -tags sqlite3 . server -c config/settings.sqlite.yml
|
||||
`git status` 看不到,PR 里也不会出现。两个目录的包名分别是 `version` 与
|
||||
`version_local`(后者与目录名不一致,因为标识符不能含连字符)。
|
||||
|
||||
### 写种子数据用哪个 models 包
|
||||
|
||||
`1786700003000` 之后新增的迁移,**种子数据要用 `app/` 下的运行时模型**
|
||||
(如 `app/admin/models.SysApi`、`SysMenu`),**不要用 `cmd/migrate/migration/models`**。
|
||||
|
||||
后者的 `ModelTime` 声明的是可空的 `gorm.DeletedAt`,这对它之前的迁移是对的(那正是
|
||||
当时列的形状),转换之后就不再成立,两个方向都会出问题:
|
||||
|
||||
- **写**:往 NOT NULL 列里塞 NULL,第一条 insert 就 `NOT NULL constraint failed`
|
||||
- **读**:GORM 拼 `WHERE deleted_at IS NULL`,而活跃行存的是 `0`,静默查不到——
|
||||
照抄 `demo_menu.go` 的授权段落会因此跳过授权,菜单建好、权限没授、迁移仍记为成功
|
||||
|
||||
干净库跑不出这个问题,今天所有用该包的迁移都排在转换之前。完整推导见
|
||||
`schema_coverage_test.go` 里 `TestPostConversionMigrationsAvoidFrozenSeedModels`
|
||||
的注释,那个测试也守着这条边界。
|
||||
|
||||
## 静默失败校验
|
||||
|
||||
`make checksilent` 检查六类**不报错、不记日志、行为悄悄变得不对**的问题,
|
||||
CI 会跑,命中 ERROR 即失败:
|
||||
|
||||
| 检查 | 级别 | 静默后果 |
|
||||
|---|---|---|
|
||||
| `modeltime-mix` | ERROR | 两个 `ModelTime` 混用,整张表查不到数据 |
|
||||
| `menu-sort-overflow` | ERROR | 菜单 `sort` 超 127,MySQL tinyint 拒绝写入,迁移中断 |
|
||||
| `config-value-truncation` | ERROR | `sys_config.config_value` 超 255 字符被静默截断 |
|
||||
| `menu-id-collision` | ERROR | 两个模块硬编码同一菜单 ID,互相覆盖 |
|
||||
| `contract-import-boundary` | ERROR | 契约包 import `app/`,应用无法独立编译 |
|
||||
| `menu-name-mismatch` | WARN | 菜单名与前端组件 `name` 不一致,keep-alive 缓存静默失效 |
|
||||
|
||||
最后一条要跨仓库比对,只能做正则启发式,因此是 WARN,**不影响退出码**,
|
||||
且默认跳过;要跑它得指定前端目录:
|
||||
|
||||
```bash
|
||||
make checksilent UI_DIR=../go-admin-ui/src
|
||||
```
|
||||
|
||||
升级门槛:连续 2 个发版周期零误报后转为 ERROR。
|
||||
|
||||
## 提交规范
|
||||
|
||||
格式 `type+emoji: 描述`:
|
||||
|
||||
@@ -37,9 +37,27 @@ stop:
|
||||
#@echo "go-admin stop success"
|
||||
|
||||
|
||||
#.PHONY: test
|
||||
#test:
|
||||
# go test -v ./... -cover
|
||||
# -race is worth the extra minute here: common/actions reuses model instances
|
||||
# across concurrent requests, so a Generate() that returns in place instead of
|
||||
# a copy leaks data between them - and that is invisible to a single-threaded
|
||||
# test run.
|
||||
.PHONY: test
|
||||
test:
|
||||
go test -race -cover ./...
|
||||
|
||||
# Reports the failures that do not announce themselves - see
|
||||
# tools/checksilent. Exits non-zero on an ERROR; the one WARN-level check
|
||||
# prints and does not fail the build.
|
||||
#
|
||||
# Pass UI_DIR to enable the cross-repository menu-name check, which is skipped
|
||||
# without it: make checksilent UI_DIR=../go-admin-ui/src
|
||||
.PHONY: checksilent
|
||||
checksilent:
|
||||
ifdef UI_DIR
|
||||
go run ./tools/checksilent -ui-dir $(UI_DIR)
|
||||
else
|
||||
go run ./tools/checksilent
|
||||
endif
|
||||
|
||||
#.PHONY: docker
|
||||
#docker:
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package apis
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"github.com/gin-gonic/gin/binding"
|
||||
"go-admin/app/admin/models"
|
||||
"golang.org/x/crypto/bcrypt"
|
||||
@@ -15,6 +16,7 @@ import (
|
||||
"go-admin/app/admin/service"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"go-admin/common/actions"
|
||||
"go-admin/common/middleware"
|
||||
)
|
||||
|
||||
type SysUser struct {
|
||||
@@ -149,12 +151,34 @@ func (e SysUser) Update(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
req.SetUpdateBy(user.GetUserId(c))
|
||||
callerId := user.GetUserId(c)
|
||||
|
||||
// This route is in CasbinExclude so the personal-center screen can edit
|
||||
// the caller's own record without a policy grant (see settings.go). That
|
||||
// exclusion covers the whole route, not just the caller's own record, and
|
||||
// the request carries the target userId in the body - so without this
|
||||
// check here, any authenticated caller could edit any other user, up to
|
||||
// and including their roleId. When the target is someone else, ask Casbin
|
||||
// directly for the permission AuthCheckRole skipped.
|
||||
if req.UserId != callerId {
|
||||
allowed, err := middleware.EnforceRoleFor(c, c.Request.URL.Path, c.Request.Method)
|
||||
if err != nil {
|
||||
e.Logger.Error(err)
|
||||
e.Error(500, err, err.Error())
|
||||
return
|
||||
}
|
||||
if !allowed {
|
||||
e.Error(http.StatusForbidden, errors.New("无权更新其他用户数据"), "对不起,您没有该接口访问权限,请联系管理员")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
req.SetUpdateBy(callerId)
|
||||
|
||||
//数据权限检查
|
||||
p := actions.GetPermissionFromContext(c)
|
||||
|
||||
err = s.Update(&req, p)
|
||||
err = s.Update(&req, p, callerId)
|
||||
if err != nil {
|
||||
e.Logger.Error(err)
|
||||
return
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
package apis
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/glebarez/sqlite"
|
||||
mycasbin "github.com/go-admin-team/go-admin-core/v2/casbin"
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
"github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
)
|
||||
|
||||
// PUT /api/v1/sys-user is in settings.go's CasbinExclude so the
|
||||
// personal-center screen (go-admin-ui's userInfo.vue) can edit the caller's
|
||||
// own record without holding a policy grant on this route. AuthCheckRole
|
||||
// skips Enforce entirely for an excluded route, so this file's job is to pin
|
||||
// what the handler itself now has to hold shut: the target userId comes from
|
||||
// the request body, and nothing upstream of the handler ever checked it
|
||||
// against the caller.
|
||||
|
||||
// setupPrivescDB wires an in-memory database and a Casbin enforcer with an
|
||||
// empty policy - the state of a fresh install for any role but admin - under
|
||||
// a tenant unique to the calling test, so mycasbin's process-wide enforcer
|
||||
// cache can't hand one test's database to another.
|
||||
func setupPrivescDB(t *testing.T) (*gorm.DB, string) {
|
||||
t.Helper()
|
||||
|
||||
// Fatalf, not Skipf: this database is in-memory sqlite with no external
|
||||
// dependency, so failing to open or migrate it means the environment is
|
||||
// actually broken. Skipping here would let these two anti-privesc
|
||||
// regression tests silently stop running while CI stays green - a
|
||||
// standing assertion that never fires is worse than no assertion.
|
||||
db, err := gorm.Open(sqlite.Open("file::memory:"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatalf("sqlite unavailable: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(&models.SysUser{}); err != nil {
|
||||
t.Fatalf("automigrate: %v", err)
|
||||
}
|
||||
|
||||
tenant := "sys-user-privesc-" + t.Name()
|
||||
|
||||
previousInterval := mycasbin.ReloadInterval
|
||||
mycasbin.ReloadInterval = 0 // opt out of the background reload goroutine; the test never writes a policy
|
||||
t.Cleanup(func() { mycasbin.ReloadInterval = previousInterval })
|
||||
|
||||
e := mycasbin.Setup(db, tenant)
|
||||
previousEnforcer := sdk.Runtime.GetCasbinByTenant(tenant)
|
||||
sdk.Runtime.SetCasbinByTenant(tenant, e)
|
||||
t.Cleanup(func() { sdk.Runtime.SetCasbinByTenant(tenant, previousEnforcer) })
|
||||
|
||||
return db, tenant
|
||||
}
|
||||
|
||||
// callUpdate drives SysUser.Update the way the router does for an
|
||||
// authenticated, non-admin caller: JWT claims already decoded into the
|
||||
// context (that is jwtauth's job, not this handler's) and a database - but
|
||||
// without AuthCheckRole, since that middleware never runs Enforce for this
|
||||
// route at all.
|
||||
func callUpdate(t *testing.T, db *gorm.DB, tenant string, callerId int, body map[string]interface{}) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
raw, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal request body: %v", err)
|
||||
}
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest(http.MethodPut, "/api/v1/sys-user", bytes.NewReader(raw))
|
||||
c.Request.Host = tenant
|
||||
c.Request.Header.Set("Content-Type", "application/json")
|
||||
|
||||
c.Set("db", db)
|
||||
c.Set(pkg.LoggerKey, logger.NewHelper(logger.DefaultLogger))
|
||||
c.Set(jwt.JwtPayloadKey, jwt.MapClaims{
|
||||
"identity": float64(callerId),
|
||||
"rolekey": "ordinary-role", // holds no Casbin policy anywhere in this test
|
||||
})
|
||||
|
||||
SysUser{}.Update(c)
|
||||
return w
|
||||
}
|
||||
|
||||
// TestUpdate_CannotEscalatePrivilegeThroughAnotherUsersRecord is the
|
||||
// regression for H6. Before the fix, an ordinary authenticated user could PUT
|
||||
// a body naming another user's id and change that user's roleId - the route
|
||||
// being Casbin-excluded meant no permission check ever ran, and the data
|
||||
// permission scope that would otherwise gate this is off by default.
|
||||
func TestUpdate_CannotEscalatePrivilegeThroughAnotherUsersRecord(t *testing.T) {
|
||||
db, tenant := setupPrivescDB(t)
|
||||
|
||||
victim := models.SysUser{Username: "bob", NickName: "Bob", RoleId: 2, DeptId: 1, Status: "1"}
|
||||
if err := db.Create(&victim).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
attacker := models.SysUser{Username: "alice", NickName: "Alice", RoleId: 2, DeptId: 1, Status: "1"}
|
||||
if err := db.Create(&attacker).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
const elevatedRoleId = 1 // a role the attacker does not hold and has no policy for
|
||||
|
||||
callUpdate(t, db, tenant, attacker.UserId, map[string]interface{}{
|
||||
"userId": victim.UserId,
|
||||
"username": victim.Username,
|
||||
"nickName": "pwned",
|
||||
"phone": "13800000000",
|
||||
"email": "bob@example.com",
|
||||
"roleId": elevatedRoleId,
|
||||
"deptId": victim.DeptId,
|
||||
"status": victim.Status,
|
||||
})
|
||||
|
||||
var after models.SysUser
|
||||
if err := db.First(&after, victim.UserId).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after.RoleId == elevatedRoleId {
|
||||
t.Fatalf("an attacker with no Casbin permission on this route escalated the victim's roleId to %d", after.RoleId)
|
||||
}
|
||||
if after.NickName == "pwned" {
|
||||
t.Fatalf("an attacker with no Casbin permission on this route modified another user's record: %+v", after)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUpdate_SelfEditCannotChangePrivilegedFields covers the case the
|
||||
// CasbinExclude entry exists for: the personal-center screen has to keep
|
||||
// working for the caller's own record. The fields that screen exposes
|
||||
// (nickName/phone/email/sex) must still save, while roleId/deptId/status stay
|
||||
// whatever the database already had even if the request carries something
|
||||
// else - a compromised or hand-crafted client is the only way that request
|
||||
// would ever differ from what the honest form sends.
|
||||
func TestUpdate_SelfEditCannotChangePrivilegedFields(t *testing.T) {
|
||||
db, tenant := setupPrivescDB(t)
|
||||
|
||||
self := models.SysUser{Username: "carol", NickName: "Carol", RoleId: 2, DeptId: 1, Status: "1"}
|
||||
if err := db.Create(&self).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
const elevatedRoleId = 1
|
||||
|
||||
callUpdate(t, db, tenant, self.UserId, map[string]interface{}{
|
||||
"userId": self.UserId,
|
||||
"username": self.Username,
|
||||
"nickName": "Carol Updated",
|
||||
"phone": "13900000000",
|
||||
"email": "carol@example.com",
|
||||
"roleId": elevatedRoleId, // tampered; must not take effect
|
||||
"deptId": self.DeptId,
|
||||
"status": self.Status,
|
||||
})
|
||||
|
||||
var after models.SysUser
|
||||
if err := db.First(&after, self.UserId).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after.RoleId == elevatedRoleId {
|
||||
t.Fatalf("a self-edit changed the caller's own roleId to %d", after.RoleId)
|
||||
}
|
||||
if after.NickName != "Carol Updated" {
|
||||
t.Fatalf("the legitimate personal-center edit did not go through: %+v", after)
|
||||
}
|
||||
}
|
||||
@@ -42,19 +42,35 @@ func (e *SysUser) GetId() interface{} {
|
||||
return e.UserId
|
||||
}
|
||||
|
||||
// Encrypt 加密
|
||||
func (e *SysUser) Encrypt() (err error) {
|
||||
// Encrypt hashes Password, unless it already holds a hash.
|
||||
//
|
||||
// The hooks below run on whatever is in the struct, and a user read from the
|
||||
// database carries the stored hash in that field. Hashing it again produces a
|
||||
// hash of a hash, and the password that user knows no longer matches anything:
|
||||
// they cannot log in, and nothing reports an error. The only thing preventing
|
||||
// that today is an Omit("password") on the one update that loads a user first,
|
||||
// which makes every other write to this model one line away from destroying
|
||||
// credentials.
|
||||
//
|
||||
// bcrypt.Cost parses a hash and fails on anything else, so it distinguishes
|
||||
// the two cases without the call site having to say which it is. The cost is
|
||||
// that a password which is itself a well-formed bcrypt hash would be stored
|
||||
// unchanged - a 60-character string beginning "$2a$", not something a person
|
||||
// types, and it grants whoever set it no access they did not already have.
|
||||
func (e *SysUser) Encrypt() error {
|
||||
if e.Password == "" {
|
||||
return
|
||||
return nil
|
||||
}
|
||||
if _, err := bcrypt.Cost([]byte(e.Password)); err == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
var hash []byte
|
||||
if hash, err = bcrypt.GenerateFromPassword([]byte(e.Password), bcrypt.DefaultCost); err != nil {
|
||||
return
|
||||
} else {
|
||||
e.Password = string(hash)
|
||||
return
|
||||
hash, err := bcrypt.GenerateFromPassword([]byte(e.Password), bcrypt.DefaultCost)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
e.Password = string(hash)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (e *SysUser) BeforeCreate(_ *gorm.DB) error {
|
||||
@@ -62,11 +78,7 @@ func (e *SysUser) BeforeCreate(_ *gorm.DB) error {
|
||||
}
|
||||
|
||||
func (e *SysUser) BeforeUpdate(_ *gorm.DB) error {
|
||||
var err error
|
||||
if e.Password != "" {
|
||||
err = e.Encrypt()
|
||||
}
|
||||
return err
|
||||
return e.Encrypt()
|
||||
}
|
||||
|
||||
func (e *SysUser) AfterFind(_ *gorm.DB) error {
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"golang.org/x/crypto/bcrypt"
|
||||
)
|
||||
|
||||
const knownPassword = "correct-horse-battery-staple"
|
||||
|
||||
// A user loaded from the database carries the stored hash in Password, and the
|
||||
// hooks run on whatever is in the struct. Hashing it a second time produces a
|
||||
// hash of a hash: the password the user knows stops matching, they cannot log
|
||||
// in, and nothing reports an error.
|
||||
//
|
||||
// Only an Omit("password") on one call site stood between this and every write
|
||||
// to the model. This is the test that removes the need for it.
|
||||
func TestEncryptLeavesAnAlreadyHashedPasswordAlone(t *testing.T) {
|
||||
fresh := SysUser{Password: knownPassword}
|
||||
if err := fresh.Encrypt(); err != nil {
|
||||
t.Fatalf("Encrypt: %v", err)
|
||||
}
|
||||
stored := fresh.Password
|
||||
if err := bcrypt.CompareHashAndPassword([]byte(stored), []byte(knownPassword)); err != nil {
|
||||
t.Fatalf("setup failed: the password was not hashed: %v", err)
|
||||
}
|
||||
|
||||
// What a query puts in the struct, and what an update then hands the hook.
|
||||
loaded := SysUser{Password: stored}
|
||||
if err := loaded.Encrypt(); err != nil {
|
||||
t.Fatalf("Encrypt on a loaded user: %v", err)
|
||||
}
|
||||
if loaded.Password != stored {
|
||||
t.Error("Encrypt re-hashed a stored hash; the user can no longer log in")
|
||||
}
|
||||
if err := bcrypt.CompareHashAndPassword([]byte(loaded.Password), []byte(knownPassword)); err != nil {
|
||||
t.Errorf("the user can no longer log in with their password: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The other half: a password that is not a hash still gets hashed, on create
|
||||
// and on update alike.
|
||||
func TestEncryptHashesAPlaintextPassword(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
hook func(*SysUser) error
|
||||
}{
|
||||
{"BeforeCreate", func(u *SysUser) error { return u.BeforeCreate(nil) }},
|
||||
{"BeforeUpdate", func(u *SysUser) error { return u.BeforeUpdate(nil) }},
|
||||
} {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
u := SysUser{Password: knownPassword}
|
||||
if err := c.hook(&u); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if u.Password == knownPassword {
|
||||
t.Fatal("the password was stored as it was typed")
|
||||
}
|
||||
if err := bcrypt.CompareHashAndPassword([]byte(u.Password), []byte(knownPassword)); err != nil {
|
||||
t.Errorf("the stored value does not verify the password: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// An empty Password means "not being set", and must not become a hash of "".
|
||||
func TestEncryptIgnoresAnEmptyPassword(t *testing.T) {
|
||||
u := SysUser{}
|
||||
if err := u.Encrypt(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if u.Password != "" {
|
||||
t.Errorf("an unset password became %q", u.Password)
|
||||
}
|
||||
}
|
||||
|
||||
// Encrypt runs on every update of this model, including the ones that change
|
||||
// something else entirely. What it costs when there is nothing to do is the
|
||||
// difference between a profile update and a bcrypt round; the correctness test
|
||||
// above is what catches a regression, this reports the size of it.
|
||||
func BenchmarkEncrypt(b *testing.B) {
|
||||
fresh := SysUser{Password: knownPassword}
|
||||
if err := fresh.Encrypt(); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
|
||||
b.Run("already hashed", func(b *testing.B) {
|
||||
u := SysUser{Password: fresh.Password}
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
if err := u.Encrypt(); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
b.Run("plaintext", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
u := SysUser{Password: knownPassword}
|
||||
if err := u.Encrypt(); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -5,12 +5,17 @@ import (
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/common/dto"
|
||||
"go-admin/common/global"
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
// Deprecated: use global.OperaStatusEnabled / global.OperaStatusDisabled.
|
||||
// These two names are kept - misspelling and all - because forks import them;
|
||||
// the values moved to common/global so common/middleware no longer has to
|
||||
// import this package. See docs/contract.md.
|
||||
const (
|
||||
OperaStatusEnabel = "1" // 状态-正常
|
||||
OperaStatusDisable = "2" // 状态-关闭
|
||||
OperaStatusEnabel = global.OperaStatusEnabled // 状态-正常
|
||||
OperaStatusDisable = global.OperaStatusDisabled // 状态-关闭
|
||||
)
|
||||
|
||||
type SysOperaLogGetPageReq struct {
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
package dto
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"go-admin/common/global"
|
||||
)
|
||||
|
||||
// The values moved to common/global so common/middleware would stop importing
|
||||
// this package; these two names stayed behind as aliases, misspelling and all,
|
||||
// because forks import them.
|
||||
//
|
||||
// If they ever drift apart, rows written through the two spellings land in
|
||||
// different buckets and the operation-log filter silently misses half of them.
|
||||
func TestDeprecatedStatusAliasesStillMatch(t *testing.T) {
|
||||
if OperaStatusEnabel != global.OperaStatusEnabled {
|
||||
t.Errorf("OperaStatusEnabel = %q, global.OperaStatusEnabled = %q",
|
||||
OperaStatusEnabel, global.OperaStatusEnabled)
|
||||
}
|
||||
if OperaStatusDisable != global.OperaStatusDisabled {
|
||||
t.Errorf("OperaStatusDisable = %q, global.OperaStatusDisabled = %q",
|
||||
OperaStatusDisable, global.OperaStatusDisabled)
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,8 @@ import (
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/service"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"go-admin/common/actions"
|
||||
@@ -74,9 +76,18 @@ func (e *SysApi) Get(d *dto.SysApiGetReq, p *actions.DataPermission, model *mode
|
||||
// Update 修改SysApi对象
|
||||
func (e *SysApi) Update(c *dto.SysApiUpdateReq, p *actions.DataPermission) error {
|
||||
var model = models.SysApi{}
|
||||
db := e.Orm.Debug().First(&model, c.GetId())
|
||||
if db.RowsAffected == 0 {
|
||||
return errors.New("无权更新该数据")
|
||||
db := e.Orm.Scopes(
|
||||
actions.Permission(model.TableName(), p),
|
||||
).First(&model, c.GetId())
|
||||
if err := db.Error; err != nil {
|
||||
// First reports a row the data permission excluded exactly as it
|
||||
// reports one that does not exist, and the caller should not be able
|
||||
// to tell those apart either.
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return errors.New("无权更新该数据")
|
||||
}
|
||||
e.Log.Errorf("Service UpdateSysApi error:%s", err)
|
||||
return err
|
||||
}
|
||||
c.Generate(&model)
|
||||
db = e.Orm.Save(&model)
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/service"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"go-admin/common/actions"
|
||||
)
|
||||
|
||||
// An update the data permission excludes has to be refused, and refused in a
|
||||
// way that does not tell the caller whether the row exists. First reports both
|
||||
// cases the same way - no rows - so the message has to come from there rather
|
||||
// than from a RowsAffected check the error return has already skipped past.
|
||||
func TestSysApiUpdateRefusesARowOutsideTheDataPermission(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:sysapi-perm?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Skipf("sqlite unavailable: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(&models.SysApi{}); err != nil {
|
||||
t.Skipf("automigrate: %v", err)
|
||||
}
|
||||
|
||||
prev := config.ApplicationConfig.EnableDP
|
||||
config.ApplicationConfig.EnableDP = true
|
||||
t.Cleanup(func() { config.ApplicationConfig.EnableDP = prev })
|
||||
|
||||
// Owned by user 1.
|
||||
row := models.SysApi{Handle: "h", Title: "t", Path: "/api/v1/probe", Type: "BUS", Action: "GET"}
|
||||
row.CreateBy = 1
|
||||
if err := db.Create(&row).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
e := &SysApi{Service: service.Service{Orm: db, Log: logger.NewHelper(logger.DefaultLogger)}}
|
||||
req := &dto.SysApiUpdateReq{Id: row.Id, Title: "changed"}
|
||||
|
||||
// User 2, scope 5: only rows they created.
|
||||
outsider := &actions.DataPermission{DataScope: "5", UserId: 2, DeptId: 1, RoleId: 2}
|
||||
err = e.Update(req, outsider)
|
||||
if err == nil {
|
||||
t.Fatal("the update was allowed on a row the data permission excludes")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "无权更新该数据") {
|
||||
t.Errorf("refused with %q, want the permission message; a raw database error tells the "+
|
||||
"caller the row exists", err)
|
||||
}
|
||||
|
||||
var after models.SysApi
|
||||
if err := db.First(&after, row.Id).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after.Title != "t" {
|
||||
t.Errorf("the row was modified: title is now %q", after.Title)
|
||||
}
|
||||
|
||||
// The owner still gets through, so the scope is refusing rather than
|
||||
// everything failing.
|
||||
owner := &actions.DataPermission{DataScope: "5", UserId: 1, DeptId: 1, RoleId: 1}
|
||||
if err := e.Update(&dto.SysApiUpdateReq{Id: row.Id, Title: "by owner"}, owner); err != nil {
|
||||
t.Fatalf("the owner could not update their own row: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -84,7 +84,16 @@ func (e *SysUser) Insert(c *dto.SysUserInsertReq) error {
|
||||
}
|
||||
|
||||
// Update 修改SysUser对象
|
||||
func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission) error {
|
||||
//
|
||||
// callerId is who is asking, not who SetUpdateBy recorded - that field only
|
||||
// says who to blame, it never constrained who could be edited. When the
|
||||
// target is the caller themselves, roleId/deptId/status are kept at whatever
|
||||
// the database already has no matter what the request body carries: this is
|
||||
// the personal-center screen's route (see CasbinExclude in settings.go, and
|
||||
// the check in the API handler ahead of this call), and letting a caller
|
||||
// grant themselves a different role or department through it would be a
|
||||
// privilege escalation the exclusion was never meant to open.
|
||||
func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission, callerId int) error {
|
||||
var err error
|
||||
var model models.SysUser
|
||||
db := e.Orm.Scopes(
|
||||
@@ -98,6 +107,11 @@ func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission) err
|
||||
return errors.New("无权更新该数据")
|
||||
|
||||
}
|
||||
if model.UserId == callerId {
|
||||
c.RoleId = model.RoleId
|
||||
c.DeptId = model.DeptId
|
||||
c.Status = model.Status
|
||||
}
|
||||
c.Generate(&model)
|
||||
update := e.Orm.Model(&model).Where("user_id = ?", &model.UserId).Omit("password", "salt").Updates(&model)
|
||||
if err = update.Error; err != nil {
|
||||
|
||||
+23
-3
@@ -82,9 +82,7 @@ func run() error {
|
||||
}
|
||||
initRouter()
|
||||
|
||||
for _, f := range AppRouters {
|
||||
f()
|
||||
}
|
||||
runStartupHooks()
|
||||
|
||||
srv := &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
|
||||
@@ -154,6 +152,28 @@ func run() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// runStartupHooks runs the router registries and then the before callbacks.
|
||||
//
|
||||
// The package-level slice runs first and in its existing order, so a fork that
|
||||
// only ever appended to AppRouters sees no change at all. The core registry
|
||||
// runs second, through RunAppRouters: a module can register through
|
||||
// sdk.Runtime.SetAppRouters and no longer has to import this command package -
|
||||
// which is a main package's plumbing - just to be routed.
|
||||
//
|
||||
// The loop over the core registry now lives in core, which is what brings the
|
||||
// panic guard and the registration seal with it. RunBefore closes a gap rather
|
||||
// than moving one: the open-source edition never executed the before callbacks
|
||||
// at all, so SetBefore was accepted and silently ignored. It has to stay ahead
|
||||
// of ListenAndServe, because a callback registered WithFatal exits the process
|
||||
// and that must not happen to one that is already serving.
|
||||
func runStartupHooks() {
|
||||
for _, f := range AppRouters {
|
||||
f()
|
||||
}
|
||||
sdk.Runtime.RunAppRouters()
|
||||
sdk.Runtime.RunBefore()
|
||||
}
|
||||
|
||||
//var Router runtime.Router
|
||||
|
||||
func tip() {
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
)
|
||||
|
||||
// freshRuntime hands the test its own Runtime and puts the old one back.
|
||||
//
|
||||
// Both registries close permanently the first time they are run, and
|
||||
// sdk.Runtime is a process-wide singleton, so a test that runs the startup
|
||||
// hooks would otherwise leave every later test in this binary registering into
|
||||
// a closed registry - which is only an ERROR log, not a failure. The symptom
|
||||
// is a test that passes alone and loses its routes when run with the others.
|
||||
func freshRuntime(t *testing.T) {
|
||||
t.Helper()
|
||||
previous := sdk.Runtime
|
||||
t.Cleanup(func() { sdk.Runtime = previous })
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
}
|
||||
|
||||
// Acceptance 1 and 2 together: the package-level slice a fork appends to and
|
||||
// the core registry a module registers through both run, package-level first,
|
||||
// registration order preserved inside each.
|
||||
//
|
||||
// The order matters beyond neatness. A module that appends to AppRouters has to
|
||||
// import go-admin/cmd/api, which is why every module used to need a seven-line
|
||||
// file in the command package; SetAppRouters is the way out of that. Running
|
||||
// the old registry first is what makes the change invisible to anyone who never
|
||||
// takes it.
|
||||
func TestRunStartupHooksRunsBothRegistriesInOrder(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
savedPackage := AppRouters
|
||||
t.Cleanup(func() { AppRouters = savedPackage })
|
||||
|
||||
var order []string
|
||||
AppRouters = []func(){
|
||||
func() { order = append(order, "package-1") },
|
||||
func() { order = append(order, "package-2") },
|
||||
}
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "runtime-1") })
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "runtime-2") })
|
||||
|
||||
runStartupHooks()
|
||||
|
||||
const want = "package-1,package-2,runtime-1,runtime-2"
|
||||
if got := strings.Join(order, ","); got != want {
|
||||
t.Errorf("ran %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 17: a before callback registered through core actually runs.
|
||||
//
|
||||
// It did not, ever: core stored the callbacks and nothing executed them, so
|
||||
// SetBefore was accepted and silently did nothing. The gap survived because
|
||||
// core offered the registry without ever running it, leaving each consumer to
|
||||
// write - or forget - its own loop.
|
||||
func TestBeforeCallbacksRun(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
savedPackage := AppRouters
|
||||
t.Cleanup(func() { AppRouters = savedPackage })
|
||||
AppRouters = nil
|
||||
|
||||
var order []string
|
||||
sdk.Runtime.SetBefore(func() { order = append(order, "before-1") })
|
||||
sdk.Runtime.SetBefore(func() { order = append(order, "before-2") })
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "router") })
|
||||
|
||||
runStartupHooks()
|
||||
|
||||
// Routers first, then before: both happen ahead of ListenAndServe, and a
|
||||
// router callback is what puts the engine in place for anything that comes
|
||||
// after it.
|
||||
const want = "router,before-1,before-2"
|
||||
if got := strings.Join(order, ","); got != want {
|
||||
t.Errorf("ran %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A panicking module must not take the server down with it. The guard lives in
|
||||
// core; this asserts that go-admin actually goes through it rather than around
|
||||
// it with a loop of its own.
|
||||
func TestAPanickingRouterDoesNotStopStartup(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
savedPackage := AppRouters
|
||||
t.Cleanup(func() { AppRouters = savedPackage })
|
||||
AppRouters = nil
|
||||
|
||||
var order []string
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "first") })
|
||||
sdk.Runtime.SetAppRouters(func() { panic("a third-party module blew up") })
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "third") })
|
||||
sdk.Runtime.SetBefore(func() { order = append(order, "before") })
|
||||
|
||||
runStartupHooks()
|
||||
|
||||
const want = "first,third,before"
|
||||
if got := strings.Join(order, ","); got != want {
|
||||
t.Errorf("ran %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The default AppRouters must keep the admin routes on it. Emptying the slice
|
||||
// would not fail to compile anywhere - it would just serve a server with no
|
||||
// admin API and no error.
|
||||
func TestAdminRouterIsRegisteredOnThePackageSlice(t *testing.T) {
|
||||
if len(AppRouters) == 0 {
|
||||
t.Fatal("AppRouters is empty; the admin router is registered in init()")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
// A mistyped --app used to be indistinguishable from an up-to-date database on
|
||||
// all three paths: `migrate` printed that the app was unknown and exited 0,
|
||||
// while `--dry-run` and `status` printed "nothing to apply" and "none
|
||||
// recorded" - the same words a database with nothing pending produces. An
|
||||
// operator scripting `migrate --app crmm && deploy` therefore deployed against
|
||||
// a database the migrations never touched.
|
||||
func TestAppRegistrationErrorRejectsAnUnknownCode(t *testing.T) {
|
||||
restore := appCode
|
||||
t.Cleanup(func() { appCode = restore })
|
||||
|
||||
appCode = "doesnotexist"
|
||||
err := appRegistrationError()
|
||||
if err == nil {
|
||||
t.Fatal("an unregistered app code must be an error, not an empty run")
|
||||
}
|
||||
if !strings.Contains(err.Error(), `"doesnotexist"`) {
|
||||
t.Errorf("the message must quote what was typed; got %q", err)
|
||||
}
|
||||
// Listing what is registered is what turns the error into a fix: the typo
|
||||
// is usually one letter away from something in this list.
|
||||
if !strings.Contains(err.Error(), migration.FrameworkAppCode) {
|
||||
t.Errorf("the message must list the registered codes; got %q", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppRegistrationErrorAcceptsWhatIsRegistered(t *testing.T) {
|
||||
restore := appCode
|
||||
t.Cleanup(func() { appCode = restore })
|
||||
|
||||
for _, code := range []string{
|
||||
"", // no --app at all: every migration runs
|
||||
migration.FrameworkAppCode, // "core", the framework's own
|
||||
strings.ToUpper(migration.FrameworkAppCode), // codes normalize to lower case
|
||||
} {
|
||||
appCode = code
|
||||
if err := appRegistrationError(); err != nil {
|
||||
t.Errorf("appCode %q must be accepted; got %v", code, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
+259
-12
@@ -1,21 +1,37 @@
|
||||
package migration
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
var Migrate = &Migration{
|
||||
version: make(map[string]func(db *gorm.DB, version string) error),
|
||||
var Migrate = newMigration()
|
||||
|
||||
func newMigration() *Migration {
|
||||
return &Migration{version: make(map[string]versionEntry)}
|
||||
}
|
||||
|
||||
// versionEntry is one registered migration plus the app it belongs to. The
|
||||
// empty app code means the framework itself, which is also what the
|
||||
// sys_migration.app_code column defaults to, so history written before this
|
||||
// field existed reads back correctly with no backfill.
|
||||
type versionEntry struct {
|
||||
appCode string
|
||||
fn func(db *gorm.DB, version string) error
|
||||
}
|
||||
|
||||
type Migration struct {
|
||||
db *gorm.DB
|
||||
version map[string]func(db *gorm.DB, version string) error
|
||||
version map[string]versionEntry
|
||||
mutex sync.Mutex
|
||||
}
|
||||
|
||||
@@ -27,20 +43,247 @@ func (e *Migration) SetDb(db *gorm.DB) {
|
||||
e.db = db
|
||||
}
|
||||
|
||||
// SetVersion registers a migration owned by the framework. Signature and
|
||||
// behaviour are unchanged: every existing call site in version/*.go keeps
|
||||
// compiling and keeps writing common.Migration{Version: version} with no app
|
||||
// code, which is the correct meaning of "framework".
|
||||
func (e *Migration) SetVersion(k string, f func(db *gorm.DB, version string) error) {
|
||||
e.mutex.Lock()
|
||||
defer e.mutex.Unlock()
|
||||
e.version[k] = f
|
||||
e.setVersion(k, "", f)
|
||||
}
|
||||
|
||||
func (e *Migration) Migrate() {
|
||||
versions := make([]string, 0)
|
||||
for k := range e.version {
|
||||
func (e *Migration) setVersion(k, appCode string, f func(db *gorm.DB, version string) error) {
|
||||
e.mutex.Lock()
|
||||
defer e.mutex.Unlock()
|
||||
e.version[k] = versionEntry{appCode: appCode, fn: f}
|
||||
}
|
||||
|
||||
// AppMigrationFunc is the signature of a migration registered through ForApp.
|
||||
//
|
||||
// It receives appCode explicitly because the migration - not the framework -
|
||||
// writes its own completion row, normally as the last statement inside its own
|
||||
// transaction. That is what makes "the schema change and the record of it
|
||||
// commit together" true, and the framework cannot insert the row on the
|
||||
// migration's behalf without giving that up. Handing the code to the function
|
||||
// is what stops an app's migrations from silently recording themselves as the
|
||||
// framework's.
|
||||
type AppMigrationFunc func(db *gorm.DB, version, appCode string) error
|
||||
|
||||
// AppRegistrar is a per-app view over a registry.
|
||||
type AppRegistrar struct {
|
||||
m *Migration
|
||||
appCode string
|
||||
}
|
||||
|
||||
// FrameworkAppCode is the name migrate status prints for migrations that belong
|
||||
// to the framework rather than to an app, and the name --app accepts to select
|
||||
// them. The stored app code for those is the empty string; this is only the
|
||||
// spelling humans use. It is reserved - ForApp rejects it - so that every group
|
||||
// heading status prints is also a value --app understands.
|
||||
const FrameworkAppCode = "core"
|
||||
|
||||
// ForApp returns a registrar that records migrations under code.
|
||||
//
|
||||
// The code is lower-cased: sys_migration.version sorts as ASCII, so mixed case
|
||||
// would order MyApp before crm for no reason a reader could guess, and the two
|
||||
// spellings would group as two different apps in migrate status.
|
||||
//
|
||||
// An empty or reserved code panics rather than falling back to the framework.
|
||||
// Registration happens in init(), so this fires the first time the binary runs
|
||||
// anywhere, which is the point: an app whose migrations quietly file themselves
|
||||
// under the framework is exactly the class of silent failure this work is meant
|
||||
// to remove. Framework migrations call Migrate.SetVersion directly.
|
||||
func ForApp(code string) *AppRegistrar { return Migrate.ForApp(code) }
|
||||
|
||||
// ForApp is the same on an explicit registry, which is what tests use.
|
||||
func (e *Migration) ForApp(code string) *AppRegistrar {
|
||||
code = NormalizeAppCode(code)
|
||||
switch code {
|
||||
case "":
|
||||
panic("migration.ForApp: empty app code; framework migrations use Migrate.SetVersion")
|
||||
case FrameworkAppCode:
|
||||
panic("migration.ForApp: app code " + FrameworkAppCode + " is reserved for the framework")
|
||||
}
|
||||
return &AppRegistrar{m: e, appCode: code}
|
||||
}
|
||||
|
||||
// AppCode reports the code this registrar files migrations under, after
|
||||
// normalisation.
|
||||
func (r *AppRegistrar) AppCode() string { return r.appCode }
|
||||
|
||||
// SetVersion registers an app-owned migration under k, which is the bare
|
||||
// timestamp taken from the file name exactly as framework migrations do.
|
||||
//
|
||||
// What reaches sys_migration.version is the namespaced form; the version string
|
||||
// handed to f is that same namespaced string, so a migration that writes
|
||||
// common.Migration{Version: version, AppCode: appCode} records the key the
|
||||
// registry will look for next time.
|
||||
func (r *AppRegistrar) SetVersion(k string, f AppMigrationFunc) {
|
||||
key := namespacedKey(r.appCode, k)
|
||||
r.m.setVersion(key, r.appCode, func(db *gorm.DB, version string) error {
|
||||
return f(db, version, r.appCode)
|
||||
})
|
||||
}
|
||||
|
||||
// namespacedKey scopes k to appCode so two apps cannot collide on the
|
||||
// sys_migration.version primary key by minting the same millisecond timestamp.
|
||||
// Framework migrations (appCode == "") stay bare, matching every version string
|
||||
// already in production.
|
||||
func namespacedKey(appCode, k string) string {
|
||||
if appCode == "" {
|
||||
return k
|
||||
}
|
||||
return appCode + "-" + k
|
||||
}
|
||||
|
||||
// StatusEntry is one row of migrate status.
|
||||
type StatusEntry struct {
|
||||
AppCode string
|
||||
Version string
|
||||
Registered bool
|
||||
Applied bool
|
||||
ApplyTime *time.Time
|
||||
}
|
||||
|
||||
// Status merges the in-process registry with sys_migration, so it reports all
|
||||
// three shapes at once: registered but not applied, registered and applied, and
|
||||
// applied while nothing registers it any more - a row left behind by a
|
||||
// migration file that was deleted, or by an app that was uninstalled.
|
||||
//
|
||||
// It only reads. Nothing here creates or alters a table, which is what lets
|
||||
// both `status` and `--dry-run` run against a database without touching it.
|
||||
func (e *Migration) Status() ([]StatusEntry, error) {
|
||||
if e.db == nil {
|
||||
return nil, fmt.Errorf("migration: no database configured")
|
||||
}
|
||||
|
||||
e.mutex.Lock()
|
||||
registered := make(map[string]string, len(e.version))
|
||||
for k, v := range e.version {
|
||||
registered[k] = v.appCode
|
||||
}
|
||||
e.mutex.Unlock()
|
||||
|
||||
applied := make(map[string]common.Migration)
|
||||
// A database that has never been migrated has no sys_migration table.
|
||||
// Reporting everything as pending is the honest answer there; erroring out
|
||||
// would make status useless in exactly the case it is most wanted.
|
||||
if e.db.Migrator().HasTable(&common.Migration{}) {
|
||||
var rows []common.Migration
|
||||
if err := e.db.Find(&rows).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, r := range rows {
|
||||
applied[r.Version] = r
|
||||
}
|
||||
}
|
||||
|
||||
versions := make(map[string]struct{}, len(registered)+len(applied))
|
||||
for k := range registered {
|
||||
versions[k] = struct{}{}
|
||||
}
|
||||
for k := range applied {
|
||||
versions[k] = struct{}{}
|
||||
}
|
||||
list := make([]string, 0, len(versions))
|
||||
for k := range versions {
|
||||
list = append(list, k)
|
||||
}
|
||||
sort.Strings(list)
|
||||
|
||||
out := make([]StatusEntry, 0, len(list))
|
||||
for _, v := range list {
|
||||
entry := StatusEntry{Version: v}
|
||||
if code, ok := registered[v]; ok {
|
||||
entry.Registered = true
|
||||
entry.AppCode = code
|
||||
}
|
||||
if row, ok := applied[v]; ok {
|
||||
entry.Applied = true
|
||||
t := row.ApplyTime
|
||||
entry.ApplyTime = &t
|
||||
if !entry.Registered {
|
||||
// Nothing registers this version any more, so the database is
|
||||
// the only source left for what it belonged to.
|
||||
entry.AppCode = row.AppCode
|
||||
}
|
||||
}
|
||||
out = append(out, entry)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Migrate applies every registered migration that has not been applied yet,
|
||||
// across all apps. Existing callers are unaffected.
|
||||
func (e *Migration) Migrate() { e.run(allApps) }
|
||||
|
||||
// MigrateApp applies only the migrations registered under appCode. Pass
|
||||
// FrameworkAppCode for the framework's own migrations.
|
||||
func (e *Migration) MigrateApp(appCode string) { e.run(AppFilter(appCode)) }
|
||||
|
||||
// NormalizeAppCode applies the same rule ForApp does, so a code typed on the
|
||||
// command line matches one written in an init().
|
||||
func NormalizeAppCode(code string) string {
|
||||
return strings.ToLower(strings.TrimSpace(code))
|
||||
}
|
||||
|
||||
// AppFilter turns a code as typed into the code stored in the registry, so
|
||||
// "core" selects the framework's migrations, whose stored code is empty.
|
||||
func AppFilter(code string) string {
|
||||
code = NormalizeAppCode(code)
|
||||
if code == FrameworkAppCode {
|
||||
return ""
|
||||
}
|
||||
return code
|
||||
}
|
||||
|
||||
// DisplayAppCode is the inverse: what to print for a stored code.
|
||||
func DisplayAppCode(code string) string {
|
||||
if code == "" {
|
||||
return FrameworkAppCode
|
||||
}
|
||||
return code
|
||||
}
|
||||
|
||||
// AppCodes lists the app codes with at least one registered migration, framework
|
||||
// included under its display name, sorted.
|
||||
func (e *Migration) AppCodes() []string {
|
||||
e.mutex.Lock()
|
||||
seen := map[string]struct{}{}
|
||||
for _, v := range e.version {
|
||||
seen[DisplayAppCode(v.appCode)] = struct{}{}
|
||||
}
|
||||
e.mutex.Unlock()
|
||||
|
||||
out := make([]string, 0, len(seen))
|
||||
for code := range seen {
|
||||
out = append(out, code)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func (e *Migration) run(appCode string) {
|
||||
e.mutex.Lock()
|
||||
versions := make([]string, 0, len(e.version))
|
||||
entries := make(map[string]versionEntry, len(e.version))
|
||||
for k, v := range e.version {
|
||||
if appCode != allApps && v.appCode != appCode {
|
||||
continue
|
||||
}
|
||||
versions = append(versions, k)
|
||||
entries[k] = v
|
||||
}
|
||||
if !sort.StringsAreSorted(versions) {
|
||||
sort.Strings(versions)
|
||||
e.mutex.Unlock()
|
||||
sort.Strings(versions)
|
||||
|
||||
// A mistyped --app would otherwise select nothing and report "no
|
||||
// migrations to apply", which reads exactly like "already up to date".
|
||||
if appCode != allApps && len(versions) == 0 {
|
||||
log.Printf("no migrations are registered for app %q; registered: %s",
|
||||
DisplayAppCode(appCode), strings.Join(e.AppCodes(), ", "))
|
||||
return
|
||||
}
|
||||
|
||||
var err error
|
||||
var count int64
|
||||
applied := 0
|
||||
@@ -56,7 +299,7 @@ func (e *Migration) Migrate() {
|
||||
continue
|
||||
}
|
||||
log.Printf("applying migration %s", v)
|
||||
if err = (e.version[v])(e.db.Debug(), v); err != nil {
|
||||
if err = entries[v].fn(e.db.Debug(), v); err != nil {
|
||||
log.Fatalf("migration %s failed: %v", v, err)
|
||||
}
|
||||
applied++
|
||||
@@ -68,6 +311,10 @@ func (e *Migration) Migrate() {
|
||||
}
|
||||
}
|
||||
|
||||
// allApps is the sentinel run() takes to mean "do not filter". It is distinct
|
||||
// from the empty app code, which selects the framework's own migrations.
|
||||
const allApps = "\x00all"
|
||||
|
||||
func GetFilename(s string) string {
|
||||
s = filepath.Base(s)
|
||||
return s[:13]
|
||||
|
||||
@@ -0,0 +1,390 @@
|
||||
package migration
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
func newTestDB(t *testing.T) *gorm.DB {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("open sqlite: %v", err)
|
||||
}
|
||||
if err = db.AutoMigrate(&common.Migration{}); err != nil {
|
||||
t.Fatalf("automigrate: %v", err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
// recordFor is what an app's migration is expected to do: write its own
|
||||
// completion row, with the version it was handed and the app code it was told
|
||||
// it belongs to.
|
||||
func recordFor(db *gorm.DB, version, appCode string) error {
|
||||
return db.Create(&common.Migration{Version: version, AppCode: appCode}).Error
|
||||
}
|
||||
|
||||
func rowsByVersion(t *testing.T, db *gorm.DB) map[string]common.Migration {
|
||||
t.Helper()
|
||||
var rows []common.Migration
|
||||
if err := db.Find(&rows).Error; err != nil {
|
||||
t.Fatalf("read sys_migration: %v", err)
|
||||
}
|
||||
out := make(map[string]common.Migration, len(rows))
|
||||
for _, r := range rows {
|
||||
out[r.Version] = r
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Acceptance 9: a migration registered through ForApp("x") lands in
|
||||
// sys_migration with app_code "x".
|
||||
//
|
||||
// The registry cannot write that row for the migration, because the row is the
|
||||
// migration's own last statement inside its own transaction. So the only thing
|
||||
// that can make this true is handing the code to the function - which is why
|
||||
// AppMigrationFunc takes three parameters.
|
||||
func TestForAppRecordsItsAppCode(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
m.Migrate()
|
||||
|
||||
rows := rowsByVersion(t, db)
|
||||
row, ok := rows["x-1786800001000"]
|
||||
if !ok {
|
||||
t.Fatalf("no row for x-1786800001000; got %v", rows)
|
||||
}
|
||||
if row.AppCode != "x" {
|
||||
t.Errorf("app_code = %q, want %q", row.AppCode, "x")
|
||||
}
|
||||
}
|
||||
|
||||
// The framework path is untouched: same signature, and an empty app code, which
|
||||
// is what the column defaults to and what every row written before this field
|
||||
// existed reads back as.
|
||||
func TestSetVersionStillRecordsTheFrameworkAsEmpty(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.Migrate()
|
||||
|
||||
rows := rowsByVersion(t, db)
|
||||
row, ok := rows["1786700009000"]
|
||||
if !ok {
|
||||
t.Fatalf("no row for 1786700009000; got %v", rows)
|
||||
}
|
||||
if row.AppCode != "" {
|
||||
t.Errorf("app_code = %q, want empty (framework)", row.AppCode)
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 12: --app x runs x's migrations and touches nothing else.
|
||||
func TestMigrateAppRunsOnlyThatApp(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
ran := map[string]bool{}
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
ran["core"] = true
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran["x"] = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
m.ForApp("y").SetVersion("1786800002000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran["y"] = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.MigrateApp("x")
|
||||
|
||||
if !ran["x"] {
|
||||
t.Error("x did not run")
|
||||
}
|
||||
if ran["y"] || ran["core"] {
|
||||
t.Errorf("MigrateApp(x) also ran %v", ran)
|
||||
}
|
||||
rows := rowsByVersion(t, db)
|
||||
if len(rows) != 1 {
|
||||
t.Fatalf("sys_migration has %d rows, want 1: %v", len(rows), rows)
|
||||
}
|
||||
}
|
||||
|
||||
// "core" is what status prints for the framework, so --app core has to select
|
||||
// it. The stored code is the empty string; AppFilter is the translation.
|
||||
func TestMigrateAppCoreSelectsTheFramework(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
ran := map[string]bool{}
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
ran["core"] = true
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran["x"] = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.MigrateApp(FrameworkAppCode)
|
||||
|
||||
if !ran["core"] {
|
||||
t.Error("framework migration did not run")
|
||||
}
|
||||
if ran["x"] {
|
||||
t.Error("--app core also ran x")
|
||||
}
|
||||
}
|
||||
|
||||
// Zero-argument Migrate keeps meaning "everything", which is what every
|
||||
// existing caller relies on.
|
||||
func TestMigrateRunsEveryApp(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
var order []string
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
order = append(order, version)
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("bbb").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
order = append(order, version)
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
m.ForApp("aaa").SetVersion("1786800002000", func(db *gorm.DB, version, appCode string) error {
|
||||
order = append(order, version)
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.Migrate()
|
||||
|
||||
// Namespacing puts every framework migration - bare digits - ahead of every
|
||||
// app migration, and orders apps by code rather than by whose timestamp
|
||||
// happened to be smaller. aaa's file is the newer of the two and still runs
|
||||
// first. Cross-app order is not promised, but this is the order, and it is
|
||||
// the one to notice changed.
|
||||
want := []string{"1786700009000", "aaa-1786800002000", "bbb-1786800001000"}
|
||||
if len(order) != len(want) {
|
||||
t.Fatalf("ran %v, want %v", order, want)
|
||||
}
|
||||
for i := range want {
|
||||
if order[i] != want[i] {
|
||||
t.Fatalf("ran %v, want %v", order, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Two apps minting the same millisecond timestamp used to mean one of them was
|
||||
// read as already applied and silently skipped. The namespace prefix is what
|
||||
// makes that impossible without changing the primary key.
|
||||
func TestNamespacingKeepsTwoAppsWithTheSameTimestampApart(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
const sameTimestamp = "1786800001000"
|
||||
ran := 0
|
||||
for _, app := range []string{"crm", "oms"} {
|
||||
m.ForApp(app).SetVersion(sameTimestamp, func(db *gorm.DB, version, appCode string) error {
|
||||
ran++
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
}
|
||||
m.Migrate()
|
||||
|
||||
if ran != 2 {
|
||||
t.Errorf("ran %d migrations, want 2", ran)
|
||||
}
|
||||
rows := rowsByVersion(t, db)
|
||||
for _, want := range []string{"crm-" + sameTimestamp, "oms-" + sameTimestamp} {
|
||||
if _, ok := rows[want]; !ok {
|
||||
t.Errorf("missing %s; got %v", want, rows)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNamespacedKeyLeavesFrameworkVersionsBare(t *testing.T) {
|
||||
if got := namespacedKey("", "1786700009000"); got != "1786700009000" {
|
||||
t.Errorf("framework version was rewritten to %q", got)
|
||||
}
|
||||
if got := namespacedKey("crm", "1786800001000"); got != "crm-1786800001000" {
|
||||
t.Errorf("namespacedKey = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// An app code differing only in case would group as two apps in status and sort
|
||||
// before every lower-case one, for no reason a reader could guess.
|
||||
func TestForAppNormalisesTheCode(t *testing.T) {
|
||||
m := newMigration()
|
||||
if got := m.ForApp(" CRM ").AppCode(); got != "crm" {
|
||||
t.Errorf("AppCode = %q, want crm", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestForAppRejectsReservedCodes(t *testing.T) {
|
||||
for _, code := range []string{"", " ", FrameworkAppCode, "CORE"} {
|
||||
t.Run("code="+code, func(t *testing.T) {
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Errorf("ForApp(%q) did not panic", code)
|
||||
}
|
||||
}()
|
||||
newMigration().ForApp(code)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatusReportsPendingAppliedAndOrphaned(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
applied := time.Date(2026, 8, 25, 14, 3, 11, 0, time.UTC)
|
||||
if err := db.Create(&common.Migration{Version: "1786700009000", ApplyTime: applied}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Recorded, but nothing registers it any more.
|
||||
if err := db.Create(&common.Migration{Version: "gone-1786800000000", ApplyTime: applied, AppCode: "gone"}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error { return nil })
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error { return nil })
|
||||
|
||||
entries, err := m.Status()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
byVersion := map[string]StatusEntry{}
|
||||
for _, e := range entries {
|
||||
byVersion[e.Version] = e
|
||||
}
|
||||
|
||||
if e := byVersion["1786700009000"]; !e.Applied || !e.Registered || e.AppCode != "" {
|
||||
t.Errorf("framework entry = %+v", e)
|
||||
} else if e.ApplyTime == nil || !e.ApplyTime.Equal(applied) {
|
||||
t.Errorf("framework apply time = %v, want %v", e.ApplyTime, applied)
|
||||
}
|
||||
if e := byVersion["crm-1786800001000"]; e.Applied || !e.Registered || e.AppCode != "crm" {
|
||||
t.Errorf("crm entry = %+v", e)
|
||||
}
|
||||
if e := byVersion["gone-1786800000000"]; !e.Applied || e.Registered || e.AppCode != "gone" {
|
||||
t.Errorf("orphaned entry = %+v", e)
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 11 rests on this: status and --dry-run both go through Status, and
|
||||
// Status must not create the table it reads.
|
||||
func TestStatusDoesNotCreateItsTable(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error { return nil })
|
||||
|
||||
entries, err := m.Status()
|
||||
if err != nil {
|
||||
t.Fatalf("Status on a database with no sys_migration: %v", err)
|
||||
}
|
||||
if len(entries) != 1 || entries[0].Applied {
|
||||
t.Errorf("entries = %+v, want one pending", entries)
|
||||
}
|
||||
if db.Migrator().HasTable(&common.Migration{}) {
|
||||
t.Error("Status created sys_migration; it must only read")
|
||||
}
|
||||
}
|
||||
|
||||
// The completion row is the migration's own last statement, inside its own
|
||||
// transaction. A migration that fails must leave no record of having run, or
|
||||
// the next run skips it and the schema stays half-changed with nothing to say
|
||||
// so.
|
||||
func TestFailedMigrationLeavesNoRecord(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := recordFor(tx, version, appCode); err != nil {
|
||||
return err
|
||||
}
|
||||
return errTestMigrationFailed
|
||||
})
|
||||
})
|
||||
|
||||
// run() calls log.Fatal on failure, which would take the test binary with
|
||||
// it, so drive the registered function directly - the point here is the
|
||||
// transaction boundary, not the scheduler.
|
||||
entry := m.version["crm-1786800001000"]
|
||||
if err := entry.fn(db, "crm-1786800001000"); err == nil {
|
||||
t.Fatal("migration reported success")
|
||||
}
|
||||
if rows := rowsByVersion(t, db); len(rows) != 0 {
|
||||
t.Errorf("sys_migration has %v after a failed migration", rows)
|
||||
}
|
||||
}
|
||||
|
||||
var errTestMigrationFailed = &testError{"boom"}
|
||||
|
||||
type testError struct{ s string }
|
||||
|
||||
func (e *testError) Error() string { return e.s }
|
||||
|
||||
// A mistyped --app used to select nothing and print "no migrations to apply",
|
||||
// which reads as "already up to date" - the command reports success and does
|
||||
// nothing, which is the failure mode this whole batch exists to remove.
|
||||
func TestMigrateAppOnAnUnknownCodeSaysSo(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
var buf bytes.Buffer
|
||||
log.SetOutput(&buf)
|
||||
t.Cleanup(func() { log.SetOutput(os.Stderr) })
|
||||
|
||||
m.MigrateApp("crmm")
|
||||
|
||||
if !strings.Contains(buf.String(), `no migrations are registered for app "crmm"`) {
|
||||
t.Errorf("output = %q", buf.String())
|
||||
}
|
||||
if !strings.Contains(buf.String(), "registered: core, crm") {
|
||||
t.Errorf("the message must list what is registered; got %q", buf.String())
|
||||
}
|
||||
if rows := rowsByVersion(t, db); len(rows) != 0 {
|
||||
t.Errorf("a typo ran %v", rows)
|
||||
}
|
||||
}
|
||||
@@ -15,6 +15,14 @@ type Model struct {
|
||||
Id int `json:"id" gorm:"primaryKey;autoIncrement;comment:主键编码"`
|
||||
}
|
||||
|
||||
// ModelTime is frozen at the schema shape these tables had before
|
||||
// 1786700003000 converted deleted_at to a NOT NULL millisecond marker. That is
|
||||
// correct for the migrations ordered before the conversion, and wrong for any
|
||||
// added after it: writes put NULL into a NOT NULL column, and reads are scoped
|
||||
// "WHERE deleted_at IS NULL" and match nothing.
|
||||
//
|
||||
// Migrations after that version seed through the runtime models in app/.
|
||||
// TestPostConversionMigrationsAvoidFrozenSeedModels enforces this.
|
||||
type ModelTime struct {
|
||||
CreatedAt time.Time `json:"createdAt" gorm:"comment:创建时间"`
|
||||
UpdatedAt time.Time `json:"updatedAt" gorm:"comment:最后更新时间"`
|
||||
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
// The repository carries two ModelTime types. The one in
|
||||
@@ -79,9 +81,13 @@ func runtimeSoftDeleteTables(t *testing.T) map[string]string {
|
||||
}
|
||||
|
||||
func importsRuntimeModels(f *ast.File) bool {
|
||||
return importsPackage(f, "go-admin/common/models")
|
||||
}
|
||||
|
||||
func importsPackage(f *ast.File, pkg string) bool {
|
||||
for _, imp := range f.Imports {
|
||||
p, err := strconv.Unquote(imp.Path.Value)
|
||||
if err == nil && p == "go-admin/common/models" {
|
||||
if err == nil && p == pkg {
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -171,3 +177,89 @@ func repoRoot(t *testing.T) string {
|
||||
t.Fatal("go.mod not found above the test directory")
|
||||
return ""
|
||||
}
|
||||
|
||||
// softDeleteConversion is the version at which sys_api, sys_menu and the rest
|
||||
// stop storing deleted_at as a nullable timestamp and start storing the NOT
|
||||
// NULL millisecond marker.
|
||||
const softDeleteConversion = 1786700003000
|
||||
|
||||
// versionPrefixLen is the width migration.GetFilename slices off a filename.
|
||||
const versionPrefixLen = 13
|
||||
|
||||
// Migrations ordered after the conversion must not seed rows through
|
||||
// cmd/migrate/migration/models.
|
||||
//
|
||||
// That package's ModelTime still declares a nullable gorm.DeletedAt, which is
|
||||
// correct for the migrations that predate the conversion - it is the shape the
|
||||
// column had when they ran. Reusing it afterwards writes NULL into a NOT NULL
|
||||
// column and the migration fails on its first insert:
|
||||
//
|
||||
// NOT NULL constraint failed: sys_api.deleted_at
|
||||
//
|
||||
// A fresh database never catches this, because every migration using that
|
||||
// package today is ordered before the conversion and so runs while the column
|
||||
// is still nullable. Only a migration added afterwards hits it, which in
|
||||
// practice means the next person adding a business module - the reference
|
||||
// they copy, 1786700001000_demo_menu.go, is itself one of the safe ones.
|
||||
//
|
||||
// Reads through that package are worse than writes, which is why the whole
|
||||
// import is banned rather than just the inserts. gorm scopes a nullable
|
||||
// DeletedAt as "WHERE deleted_at IS NULL", and after the conversion live rows
|
||||
// hold 0, so the row is simply not there:
|
||||
//
|
||||
// frozen SysRole -> record not found
|
||||
// runtime SysRole -> roleId=1
|
||||
//
|
||||
// 1786700001000_demo_menu.go looks the admin role up that way and treats
|
||||
// ErrRecordNotFound as "roles are not seeded yet, skip authorisation". A
|
||||
// post-conversion copy that switched its inserts to the runtime models but
|
||||
// kept this lookup would seed the menu, grant nothing, and still record the
|
||||
// migration as applied - the menu appears, its buttons do nothing, and no
|
||||
// error is reported anywhere.
|
||||
func TestPostConversionMigrationsAvoidFrozenSeedModels(t *testing.T) {
|
||||
const frozenModels = "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
dir, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
checked := 0
|
||||
for _, e := range entries {
|
||||
name := e.Name()
|
||||
if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
|
||||
continue
|
||||
}
|
||||
// GetFilename is what every migration uses to derive its own version,
|
||||
// so the two stay in step if the filename convention ever changes.
|
||||
if len(name) < versionPrefixLen {
|
||||
continue
|
||||
}
|
||||
version, err := strconv.ParseInt(migration.GetFilename(name), 10, 64)
|
||||
if err != nil || version <= softDeleteConversion {
|
||||
continue // not a versioned migration, or one that predates the change
|
||||
}
|
||||
checked++
|
||||
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, filepath.Join(dir, name), nil, parser.ImportsOnly)
|
||||
if err != nil {
|
||||
t.Fatalf("parse %s: %v", name, err)
|
||||
}
|
||||
if importsPackage(f, frozenModels) {
|
||||
t.Errorf("%s is ordered after the soft-delete conversion but seeds through %s;\n"+
|
||||
" that package writes a nullable deleted_at and will fail with\n"+
|
||||
" \"NOT NULL constraint failed\" on its first insert.\n"+
|
||||
" Use the runtime models under app/ instead - they carry the marker.",
|
||||
name, frozenModels)
|
||||
}
|
||||
}
|
||||
|
||||
if checked == 0 {
|
||||
t.Fatal("no post-conversion migrations found; the scan is broken, not the code")
|
||||
}
|
||||
}
|
||||
|
||||
+141
-8
@@ -3,12 +3,16 @@ package migrate
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"text/template"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/config/source/file"
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
@@ -25,6 +29,8 @@ var (
|
||||
generate bool
|
||||
goAdmin bool
|
||||
host string
|
||||
appCode string
|
||||
dryRun bool
|
||||
StartCmd = &cobra.Command{
|
||||
Use: "migrate",
|
||||
Short: "Initialize the database",
|
||||
@@ -33,14 +39,31 @@ var (
|
||||
run()
|
||||
},
|
||||
}
|
||||
statusCmd = &cobra.Command{
|
||||
Use: "status",
|
||||
Short: "List applied and pending migrations, grouped by app",
|
||||
Example: "go-admin migrate status -c config/settings.yml",
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
runStatus()
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
// fixme 在您看不见代码的时候运行迁移,我觉得是不安全的,所以编译后最好不要去执行迁移
|
||||
func init() {
|
||||
StartCmd.PersistentFlags().StringVarP(&configYml, "config", "c", "config/settings.yml", "Start server with provided configuration file")
|
||||
StartCmd.PersistentFlags().BoolVarP(&generate, "generate", "g", false, "generate migration file")
|
||||
StartCmd.PersistentFlags().BoolVarP(&goAdmin, "goAdmin", "a", false, "generate go-admin migration file")
|
||||
StartCmd.PersistentFlags().BoolVarP(&goAdmin, "goAdmin", "a", false, "with -g, write the generated file to version/ instead of version-local/ (does not affect which migrations run)")
|
||||
StartCmd.PersistentFlags().StringVarP(&host, "domain", "d", "*", "select tenant host")
|
||||
|
||||
// --app is deliberately long-only. -a already means "generate into
|
||||
// version/ rather than version-local/", which is about writing a template
|
||||
// file, not about which migrations run; giving the two the same letter
|
||||
// would be a trap.
|
||||
StartCmd.PersistentFlags().StringVar(&appCode, "app", "", "limit to the migrations of one app (\""+migration.FrameworkAppCode+"\" for the framework's own)")
|
||||
StartCmd.Flags().BoolVar(&dryRun, "dry-run", false, "list what would be applied, in order, and write nothing")
|
||||
|
||||
StartCmd.AddCommand(statusCmd)
|
||||
}
|
||||
|
||||
func run() {
|
||||
@@ -58,7 +81,12 @@ func run() {
|
||||
}
|
||||
}
|
||||
|
||||
func migrateModel() error {
|
||||
// resolveDB picks the tenant database and hands it to the registry.
|
||||
//
|
||||
// It creates and alters nothing, which is what lets status and --dry-run share
|
||||
// it: those two must be able to run against a production database without
|
||||
// leaving a trace.
|
||||
func resolveDB() (*gorm.DB, error) {
|
||||
if host == "" {
|
||||
host = "*"
|
||||
}
|
||||
@@ -73,29 +101,134 @@ func migrateModel() error {
|
||||
}
|
||||
}
|
||||
if db == nil {
|
||||
return fmt.Errorf("未找到数据库配置")
|
||||
return nil, fmt.Errorf("未找到数据库配置")
|
||||
}
|
||||
if config.DatabasesConfig[host].Driver == "mysql" {
|
||||
//初始化数据库时候用
|
||||
db.Set("gorm:table_options", "ENGINE=InnoDB CHARSET=utf8mb4")
|
||||
}
|
||||
err := db.Debug().AutoMigrate(&models.Migration{})
|
||||
return db, nil
|
||||
}
|
||||
|
||||
// exitUnlessAppRegistered ends the command when --app names something no
|
||||
// migration was registered under.
|
||||
//
|
||||
// Every path took a typo as "nothing matched" and reported success: `migrate`
|
||||
// printed that the app was unknown and still exited 0, while `--dry-run` and
|
||||
// `status` said "nothing to apply" and "none recorded" - which is what an
|
||||
// up-to-date database says too, so the output does not even hint at the typo.
|
||||
// An operator running `go-admin migrate --app crmm && deploy` gets the deploy.
|
||||
//
|
||||
// Checked against the registry, which init() has already filled, so this runs
|
||||
// before any database work and costs nothing. It lives in the command layer
|
||||
// because the exit code does: the migration package stays callable from a test
|
||||
// without taking the process down with it.
|
||||
func exitUnlessAppRegistered() {
|
||||
if err := appRegistrationError(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// appRegistrationError carries the decision on its own so it can be tested;
|
||||
// exitUnlessAppRegistered is only the os.Exit around it. Nil means --app was
|
||||
// either empty or names a registered app.
|
||||
func appRegistrationError() error {
|
||||
if appCode == "" {
|
||||
return nil
|
||||
}
|
||||
want := migration.DisplayAppCode(migration.AppFilter(appCode))
|
||||
registered := migration.Migrate.AppCodes()
|
||||
for _, c := range registered {
|
||||
if c == want {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("no migrations are registered for app %q; registered: %s",
|
||||
want, strings.Join(registered, ", "))
|
||||
}
|
||||
|
||||
func migrateModel() error {
|
||||
db, err := resolveDB()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// sys_migration is the one table that never goes through a versioned
|
||||
// migration - it is the table that records them. AutoMigrate realigns it
|
||||
// on every run, which is how the app_code column reaches an existing
|
||||
// database without anyone writing a migration for it.
|
||||
if err = db.Debug().AutoMigrate(&models.Migration{}); err != nil {
|
||||
return err
|
||||
}
|
||||
migration.Migrate.SetDb(db.Debug())
|
||||
if appCode != "" {
|
||||
migration.Migrate.MigrateApp(appCode)
|
||||
return nil
|
||||
}
|
||||
migration.Migrate.Migrate()
|
||||
return err
|
||||
return nil
|
||||
}
|
||||
|
||||
func initDB() {
|
||||
// Before the database is touched, so a typo cannot get as far as looking
|
||||
// like a successful no-op on either path below.
|
||||
exitUnlessAppRegistered()
|
||||
|
||||
//3. 初始化数据库链接
|
||||
database.Setup()
|
||||
|
||||
if dryRun {
|
||||
db, err := resolveDB()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
migration.Migrate.SetDb(db)
|
||||
entries, err := migration.Migrate.Status()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
if err = printPending(os.Stdout, entries, appCode); err != nil {
|
||||
fmt.Println(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
//4. 数据库迁移
|
||||
fmt.Println("数据库迁移开始")
|
||||
_ = migrateModel()
|
||||
if err := migrateModel(); err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
fmt.Println(`数据库基础数据初始化成功`)
|
||||
}
|
||||
|
||||
func runStatus() {
|
||||
config.Setup(
|
||||
file.NewSource(file.WithPath(configYml)),
|
||||
func() {
|
||||
exitUnlessAppRegistered()
|
||||
|
||||
database.Setup()
|
||||
db, err := resolveDB()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
migration.Migrate.SetDb(db)
|
||||
entries, err := migration.Migrate.Status()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
if err = printStatus(os.Stdout, entries, appCode); err != nil {
|
||||
fmt.Println(err)
|
||||
}
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
func genFile() error {
|
||||
t1, err := template.ParseFiles("template/migrate.template")
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
const applyTimeLayout = "2006-01-02 15:04:05"
|
||||
|
||||
// printStatus lists every migration this binary knows about together with every
|
||||
// row already in sys_migration, grouped by app.
|
||||
//
|
||||
// filter is an app code as typed on the command line; empty means every app.
|
||||
func printStatus(w io.Writer, entries []migration.StatusEntry, filter string) error {
|
||||
entries = filterByApp(entries, filter)
|
||||
|
||||
groups, order := groupByApp(entries)
|
||||
if len(order) == 0 {
|
||||
_, err := fmt.Fprintln(w, "no migrations registered and none recorded")
|
||||
return err
|
||||
}
|
||||
|
||||
// One width for the whole listing rather than one per group: the versions
|
||||
// of two apps line up, so a long list can be read down the column.
|
||||
width := versionWidth(entries)
|
||||
|
||||
var applied, pending, orphaned int
|
||||
for i, app := range order {
|
||||
if i > 0 {
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
fmt.Fprintf(w, "[%s]\n", app)
|
||||
for _, e := range groups[app] {
|
||||
state := "pending"
|
||||
switch {
|
||||
case e.Applied && !e.Registered:
|
||||
state = "orphaned"
|
||||
orphaned++
|
||||
case e.Applied:
|
||||
state = "applied"
|
||||
applied++
|
||||
default:
|
||||
pending++
|
||||
}
|
||||
fmt.Fprintln(w, strings.TrimRight(
|
||||
fmt.Sprintf(" %-*s%-*s%s", stateWidth, state, width, e.Version, formatApplyTime(e.ApplyTime)), " "))
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Fprintf(w, "\n%d applied, %d pending across %d app(s)\n", applied, pending, len(order))
|
||||
if orphaned > 0 {
|
||||
fmt.Fprintf(w, "%d orphaned: recorded in sys_migration, but nothing in this binary registers them.\n"+
|
||||
"Expected after a migration file is removed or an app is uninstalled; they will not run again.\n", orphaned)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// printPending is --dry-run: the same data as status, narrowed to what an
|
||||
// actual run would do and printed in the order it would do it.
|
||||
//
|
||||
// It reads and prints. Every write path - AutoMigrate on sys_migration
|
||||
// included - is on the other branch in initDB, so a dry run leaves the database
|
||||
// byte for byte as it found it.
|
||||
func printPending(w io.Writer, entries []migration.StatusEntry, filter string) error {
|
||||
entries = filterByApp(entries, filter)
|
||||
|
||||
fmt.Fprintln(w, "dry-run: nothing will be written")
|
||||
|
||||
pending := make([]migration.StatusEntry, 0, len(entries))
|
||||
for _, e := range entries {
|
||||
// An orphaned row is recorded and unregistered; a real run cannot
|
||||
// apply it, so a dry run must not offer to.
|
||||
if !e.Applied && e.Registered {
|
||||
pending = append(pending, e)
|
||||
}
|
||||
}
|
||||
if len(pending) == 0 {
|
||||
_, err := fmt.Fprintln(w, "nothing to apply")
|
||||
return err
|
||||
}
|
||||
|
||||
appWidth := 0
|
||||
for _, e := range pending {
|
||||
if n := len(migration.DisplayAppCode(e.AppCode)) + 2; n > appWidth {
|
||||
appWidth = n
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "would apply, in this order:")
|
||||
for _, e := range pending {
|
||||
fmt.Fprintf(w, " %-*s%s\n", appWidth+2, "["+migration.DisplayAppCode(e.AppCode)+"]", e.Version)
|
||||
}
|
||||
fmt.Fprintf(w, "\n%d migration(s) pending\n", len(pending))
|
||||
return nil
|
||||
}
|
||||
|
||||
// stateWidth is the width of the applied/pending/orphaned column, sized to the
|
||||
// longest of the three plus a gap.
|
||||
const stateWidth = len("orphaned") + 2
|
||||
|
||||
func versionWidth(entries []migration.StatusEntry) int {
|
||||
width := 0
|
||||
for _, e := range entries {
|
||||
if n := len(e.Version) + 2; n > width {
|
||||
width = n
|
||||
}
|
||||
}
|
||||
return width
|
||||
}
|
||||
|
||||
// filterByApp keeps the entries of one app. The filter is matched after the
|
||||
// same normalisation ForApp applies, so --app CRM finds crm.
|
||||
func filterByApp(entries []migration.StatusEntry, filter string) []migration.StatusEntry {
|
||||
if filter == "" {
|
||||
return entries
|
||||
}
|
||||
want := migration.AppFilter(filter)
|
||||
out := make([]migration.StatusEntry, 0, len(entries))
|
||||
for _, e := range entries {
|
||||
if e.AppCode == want {
|
||||
out = append(out, e)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// groupByApp buckets entries by display name and returns the buckets plus the
|
||||
// order to print them in: the framework first, then apps alphabetically. That
|
||||
// is also the order a full run executes them in, because version strings sort
|
||||
// as ASCII and the framework's are bare digits.
|
||||
func groupByApp(entries []migration.StatusEntry) (map[string][]migration.StatusEntry, []string) {
|
||||
groups := make(map[string][]migration.StatusEntry)
|
||||
for _, e := range entries {
|
||||
app := migration.DisplayAppCode(e.AppCode)
|
||||
groups[app] = append(groups[app], e)
|
||||
}
|
||||
order := make([]string, 0, len(groups))
|
||||
for app := range groups {
|
||||
order = append(order, app)
|
||||
}
|
||||
sort.Slice(order, func(i, j int) bool {
|
||||
if (order[i] == migration.FrameworkAppCode) != (order[j] == migration.FrameworkAppCode) {
|
||||
return order[i] == migration.FrameworkAppCode
|
||||
}
|
||||
return order[i] < order[j]
|
||||
})
|
||||
return groups, order
|
||||
}
|
||||
|
||||
func formatApplyTime(t *time.Time) string {
|
||||
if t == nil {
|
||||
return ""
|
||||
}
|
||||
return t.Format(applyTimeLayout)
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
func at(s string) *time.Time {
|
||||
t, err := time.Parse(applyTimeLayout, s)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return &t
|
||||
}
|
||||
|
||||
// The order Status returns: version strings sorted as ASCII.
|
||||
func sampleEntries() []migration.StatusEntry {
|
||||
return []migration.StatusEntry{
|
||||
{Version: "1786700001000", AppCode: "", Registered: true, Applied: true, ApplyTime: at("2026-08-20 10:00:00")},
|
||||
{Version: "1786700005000", AppCode: "", Registered: true},
|
||||
{Version: "crm-1786800001000", AppCode: "crm", Registered: true, Applied: true, ApplyTime: at("2026-08-25 14:03:11")},
|
||||
{Version: "crm-1786800002000", AppCode: "crm", Registered: true},
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintStatusGroupsByApp(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, sampleEntries(), ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
|
||||
for _, want := range []string{
|
||||
"[core]",
|
||||
"[crm]",
|
||||
"applied 1786700001000 2026-08-20 10:00:00",
|
||||
"pending 1786700005000",
|
||||
"applied crm-1786800001000 2026-08-25 14:03:11",
|
||||
"pending crm-1786800002000",
|
||||
"2 applied, 2 pending across 2 app(s)",
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("output missing %q:\n%s", want, got)
|
||||
}
|
||||
}
|
||||
// The framework heads the list, because that is the order a full run
|
||||
// executes in.
|
||||
if strings.Index(got, "[core]") > strings.Index(got, "[crm]") {
|
||||
t.Errorf("core is not listed first:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A row nobody registers any more is neither applied-and-current nor pending.
|
||||
// Calling it applied would say the migration is in this binary, which is what
|
||||
// sends someone looking for a file that was deleted.
|
||||
func TestPrintStatusMarksOrphanedRows(t *testing.T) {
|
||||
entries := append(sampleEntries(), migration.StatusEntry{
|
||||
Version: "gone-1786800000000", AppCode: "gone", Applied: true, ApplyTime: at("2026-08-01 09:00:00"),
|
||||
})
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, entries, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if !strings.Contains(got, "orphaned gone-1786800000000") {
|
||||
t.Errorf("orphaned row not marked:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "nothing in this binary registers them") {
|
||||
t.Errorf("orphaned rows need an explanation:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "2 applied, 2 pending") {
|
||||
t.Errorf("orphaned rows must not be counted as applied:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintStatusFiltersByApp(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, sampleEntries(), "crm"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if strings.Contains(got, "[core]") {
|
||||
t.Errorf("--app crm listed the framework:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "across 1 app(s)") {
|
||||
t.Errorf("output = %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// status prints [core]; --app core has to mean the same thing.
|
||||
func TestPrintStatusAppCoreSelectsTheFramework(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, sampleEntries(), migration.FrameworkAppCode); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if strings.Contains(got, "[crm]") {
|
||||
t.Errorf("--app core listed crm:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "[core]") {
|
||||
t.Errorf("--app core listed nothing:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintStatusOnAnEmptyRegistry(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, nil, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(buf.String(), "no migrations registered and none recorded") {
|
||||
t.Errorf("output = %s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintPendingListsOnlyPendingInOrder(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printPending(&buf, sampleEntries(), ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
|
||||
if !strings.Contains(got, "dry-run: nothing will be written") {
|
||||
t.Errorf("dry-run must say it writes nothing:\n%s", got)
|
||||
}
|
||||
if strings.Contains(got, "1786700001000\n") || strings.Contains(got, "crm-1786800001000") {
|
||||
t.Errorf("dry-run listed already applied migrations:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "[core] 1786700005000") || !strings.Contains(got, "[crm] crm-1786800002000") {
|
||||
t.Errorf("dry-run is missing pending migrations:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "2 migration(s) pending") {
|
||||
t.Errorf("output = %s", got)
|
||||
}
|
||||
if strings.Index(got, "1786700005000") > strings.Index(got, "crm-1786800002000") {
|
||||
t.Errorf("dry-run order does not match run order:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// An orphaned row is applied and unregistered; a dry run must not offer to
|
||||
// apply it, because a real run cannot.
|
||||
func TestPrintPendingSkipsOrphanedRows(t *testing.T) {
|
||||
entries := []migration.StatusEntry{
|
||||
{Version: "gone-1786800000000", AppCode: "gone", Applied: true, ApplyTime: at("2026-08-01 09:00:00")},
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := printPending(&buf, entries, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(buf.String(), "nothing to apply") {
|
||||
t.Errorf("output = %s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintPendingFiltersByApp(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printPending(&buf, sampleEntries(), "CRM"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if strings.Contains(got, "[core]") {
|
||||
t.Errorf("--app CRM listed the framework:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "1 migration(s) pending") {
|
||||
t.Errorf("output = %s", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package global
|
||||
|
||||
// Status values written to sys_opera_log.status.
|
||||
//
|
||||
// They live here rather than in app/admin/service/dto because
|
||||
// common/middleware/logger.go writes the operation-log message and needs them.
|
||||
// A package promised as a stable contract must not compile-depend on a
|
||||
// business module: a fork that replaces or drops app/admin would otherwise
|
||||
// stop compiling common/middleware, which is not something a contract package
|
||||
// is allowed to do. See docs/contract.md.
|
||||
const (
|
||||
OperaStatusEnabled = "1"
|
||||
OperaStatusDisabled = "2"
|
||||
)
|
||||
@@ -1,7 +1,6 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/common"
|
||||
"net/http"
|
||||
|
||||
@@ -163,19 +162,31 @@ func LogOut(c *gin.Context) {
|
||||
|
||||
}
|
||||
|
||||
// Authorizator decides whether a parsed identity may proceed. It authorizes
|
||||
// every identity IdentityHandler was able to build, which is what it has always
|
||||
// done.
|
||||
//
|
||||
// It used to also assert data["user"] and data["role"] into app/admin/models
|
||||
// types and copy five fields onto the context. Those two keys are not in the
|
||||
// map: IdentityHandler builds it from the token claims and puts in
|
||||
// IdentityKey / UserName / RoleKey / UserId / RoleIds / DataScope. Both
|
||||
// assertions therefore failed on every request, and because the ok result was
|
||||
// discarded, the five c.Set calls stored zero values and the function returned
|
||||
// true regardless.
|
||||
//
|
||||
// Nothing in this repository or in go-admin-core reads role / roleIds /
|
||||
// userId / userName / dataScope off the context - the open-source data
|
||||
// permission path reads the JWT claims through
|
||||
// common/actions.Permission -> user.GetUserIdStr(c). Dropping the block
|
||||
// therefore removes five zero values nobody read, and with them the last
|
||||
// import of app/admin from a contract package.
|
||||
//
|
||||
// Anything maintaining its own copy of this file must check its own consumers
|
||||
// before taking this change: a codebase that does read those keys off the
|
||||
// context needs Authorizator to keep setting them.
|
||||
func Authorizator(data interface{}, c *gin.Context) bool {
|
||||
|
||||
if v, ok := data.(map[string]interface{}); ok {
|
||||
u, _ := v["user"].(models.SysUser)
|
||||
r, _ := v["role"].(models.SysRole)
|
||||
c.Set("role", r.RoleName)
|
||||
c.Set("roleIds", r.RoleId)
|
||||
c.Set("userId", u.UserId)
|
||||
c.Set("userName", u.Username)
|
||||
c.Set("dataScope", r.DataScope)
|
||||
return true
|
||||
}
|
||||
return false
|
||||
_, ok := data.(map[string]interface{})
|
||||
return ok
|
||||
}
|
||||
|
||||
func Unauthorized(c *gin.Context, code int, message string) {
|
||||
|
||||
+67
-20
@@ -1,19 +1,18 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"errors"
|
||||
"go-admin/common"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/jwtauth/user"
|
||||
"github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/api"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
@@ -28,19 +27,14 @@ func LoggerToFile() gin.HandlerFunc {
|
||||
// 开始时间
|
||||
startTime := time.Now()
|
||||
// 处理请求
|
||||
//
|
||||
// The body is only read when it has a destination. operParam below is
|
||||
// the only consumer, and it is written when logger.enableddb is on -
|
||||
// off in the shipped configuration, where reading the body was a copy
|
||||
// of every request made and discarded.
|
||||
var body string
|
||||
switch c.Request.Method {
|
||||
case http.MethodPost, http.MethodPut, http.MethodGet, http.MethodDelete:
|
||||
bf := bytes.NewBuffer(nil)
|
||||
wt := bufio.NewWriter(bf)
|
||||
_, err := io.Copy(wt, c.Request.Body)
|
||||
if err != nil {
|
||||
log.Warnf("copy body error, %s", err.Error())
|
||||
err = nil
|
||||
}
|
||||
rb, _ := ioutil.ReadAll(bf)
|
||||
c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(rb))
|
||||
body = string(rb)
|
||||
if config.LoggerConfig.EnabledDB {
|
||||
body = readOperParam(c, log)
|
||||
}
|
||||
|
||||
c.Next()
|
||||
@@ -100,10 +94,55 @@ func LoggerToFile() gin.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// SetDBOperLog 写入操作日志表 fixme 该方法后续即将弃用
|
||||
func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) {
|
||||
// operParamLimit caps what is copied out of a request body for the operation
|
||||
// log. A file upload is a POST like any other and reaches this middleware
|
||||
// before any handler, so without a limit the whole upload is held in memory to
|
||||
// write a log row - a 16MB upload allocated about 67MB. The limit also keeps
|
||||
// the value inside the column, which is TEXT.
|
||||
const operParamLimit = 32 << 10
|
||||
|
||||
log := api.GetRequestLogger(c)
|
||||
// readOperParam copies the start of the request body for the operation log and
|
||||
// leaves the request readable by the handler.
|
||||
//
|
||||
// The body is not buffered whole: the handler reads the part copied here from
|
||||
// memory and the rest straight from the connection, so what this holds is
|
||||
// bounded by operParamLimit however large the request is.
|
||||
func readOperParam(c *gin.Context, log *logger.Helper) string {
|
||||
switch c.Request.Method {
|
||||
case http.MethodPost, http.MethodPut, http.MethodGet, http.MethodDelete:
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
if c.Request.Body == nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
rest := c.Request.Body
|
||||
head := make([]byte, operParamLimit)
|
||||
n, err := io.ReadFull(rest, head)
|
||||
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
|
||||
log.Warnf("read body for the operation log: %s", err)
|
||||
}
|
||||
head = head[:n]
|
||||
|
||||
c.Request.Body = readCloser{
|
||||
Reader: io.MultiReader(bytes.NewReader(head), rest),
|
||||
Closer: rest,
|
||||
}
|
||||
return string(head)
|
||||
}
|
||||
|
||||
type readCloser struct {
|
||||
io.Reader
|
||||
io.Closer
|
||||
}
|
||||
|
||||
// operaLogFields builds the message written to the operation log queue.
|
||||
//
|
||||
// Split out of SetDBOperLog so the field set can be asserted in a test: the
|
||||
// consumer on the other end of the queue reads these keys by name, so a
|
||||
// dropped or renamed key costs a column in sys_opera_log and reports nothing.
|
||||
func operaLogFields(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) map[string]interface{} {
|
||||
l := make(map[string]interface{})
|
||||
l["_fullPath"] = c.FullPath()
|
||||
l["operUrl"] = reqUri
|
||||
@@ -120,10 +159,18 @@ func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string
|
||||
l["createBy"] = user.GetUserId(c)
|
||||
l["updateBy"] = user.GetUserId(c)
|
||||
if status == http.StatusOK {
|
||||
l["status"] = dto.OperaStatusEnabel
|
||||
l["status"] = global.OperaStatusEnabled
|
||||
} else {
|
||||
l["status"] = dto.OperaStatusDisable
|
||||
l["status"] = global.OperaStatusDisabled
|
||||
}
|
||||
return l
|
||||
}
|
||||
|
||||
// SetDBOperLog 写入操作日志表 fixme 该方法后续即将弃用
|
||||
func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) {
|
||||
|
||||
log := api.GetRequestLogger(c)
|
||||
l := operaLogFields(c, clientIP, statusCode, reqUri, reqMethod, latencyTime, body, result, status)
|
||||
q := sdk.Runtime.GetQueuePrefix(c.Request.Host)
|
||||
message, err := sdk.Runtime.GetStreamMessage("", global.OperateLog, l)
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
)
|
||||
|
||||
// serveWithLogger runs one request through the logger middleware and returns
|
||||
// what the handler saw, with logger.enableddb set as given.
|
||||
func serveWithLogger(t testing.TB, enabledDB bool, method, body string) string {
|
||||
t.Helper()
|
||||
|
||||
prev := config.LoggerConfig.EnabledDB
|
||||
config.LoggerConfig.EnabledDB = enabledDB
|
||||
t.Cleanup(func() { config.LoggerConfig.EnabledDB = prev })
|
||||
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
r := gin.New()
|
||||
r.Use(LoggerToFile())
|
||||
|
||||
var seen string
|
||||
handler := func(c *gin.Context) {
|
||||
b, err := io.ReadAll(c.Request.Body)
|
||||
if err != nil {
|
||||
t.Errorf("handler could not read the body: %v", err)
|
||||
}
|
||||
seen = string(b)
|
||||
c.Status(http.StatusOK)
|
||||
}
|
||||
r.Handle(method, "/probe", handler)
|
||||
|
||||
req := httptest.NewRequest(method, "/probe", strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
r.ServeHTTP(httptest.NewRecorder(), req)
|
||||
return seen
|
||||
}
|
||||
|
||||
// The middleware rewrites Request.Body so it can log the parameters. Whatever
|
||||
// else it does, the handler has to receive the request the client sent - all
|
||||
// of it, whether or not the operation log is on, and whether or not the body
|
||||
// is longer than what gets logged.
|
||||
func TestHandlerStillSeesTheWholeBody(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
enabledDB bool
|
||||
body string
|
||||
}{
|
||||
{"log off, short body", false, `{"username":"admin"}`},
|
||||
{"log on, short body", true, `{"username":"admin"}`},
|
||||
{"log off, empty body", false, ""},
|
||||
{"log on, empty body", true, ""},
|
||||
// Longer than operParamLimit: the logged copy is truncated, the body is not.
|
||||
{"log on, body past the limit", true, strings.Repeat("x", operParamLimit+4096)},
|
||||
{"log off, body past the limit", false, strings.Repeat("y", operParamLimit+4096)},
|
||||
// Exactly at the boundary, where a fencepost error would show.
|
||||
{"log on, body exactly at the limit", true, strings.Repeat("z", operParamLimit)},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
for _, method := range []string{http.MethodPost, http.MethodPut, http.MethodDelete} {
|
||||
if got := serveWithLogger(t, c.enabledDB, method, c.body); got != c.body {
|
||||
t.Errorf("%s: handler saw %d bytes, the client sent %d",
|
||||
method, len(got), len(c.body))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The body is read for one reason - operParam on the operation log row - and
|
||||
// that row is only written when logger.enableddb is on. With it off, reading
|
||||
// the body is a copy of every request made and thrown away, and a file upload
|
||||
// is a POST like any other: 16MB of upload allocated about 67MB here.
|
||||
//
|
||||
// Allocation counts are deterministic across machines; wall-clock is not.
|
||||
func TestBodyIsNotCopiedWhenTheOperationLogIsOff(t *testing.T) {
|
||||
const size = 1 << 20
|
||||
body := strings.Repeat("x", size)
|
||||
|
||||
prev := config.LoggerConfig.EnabledDB
|
||||
config.LoggerConfig.EnabledDB = false
|
||||
t.Cleanup(func() { config.LoggerConfig.EnabledDB = prev })
|
||||
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
r := gin.New()
|
||||
r.Use(LoggerToFile())
|
||||
r.POST("/probe", func(c *gin.Context) { c.Status(http.StatusOK) })
|
||||
|
||||
payload := []byte(body)
|
||||
run := func() {
|
||||
req := httptest.NewRequest(http.MethodPost, "/probe", bytes.NewReader(payload))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
r.ServeHTTP(httptest.NewRecorder(), req)
|
||||
}
|
||||
|
||||
var before, after uint64
|
||||
before = heapAllocs()
|
||||
run()
|
||||
after = heapAllocs()
|
||||
|
||||
// The handler never reads the body, so a request that does not copy it
|
||||
// should allocate far less than the body's size. The old middleware
|
||||
// allocated about four times the body.
|
||||
if grew := after - before; grew > size/2 {
|
||||
t.Errorf("a %d-byte request allocated %d bytes with the operation log off; "+
|
||||
"the body should not be read when nothing consumes it", size, grew)
|
||||
}
|
||||
}
|
||||
|
||||
func heapAllocs() uint64 {
|
||||
var m runtime.MemStats
|
||||
runtime.GC()
|
||||
runtime.ReadMemStats(&m)
|
||||
return m.TotalAlloc
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"go-admin/common/global"
|
||||
)
|
||||
|
||||
// The operation-log consumer reads these keys by name off the queue message.
|
||||
// Losing one costs a column in sys_opera_log and reports nothing - the request
|
||||
// still succeeds, the log row is just wrong.
|
||||
//
|
||||
// This locks the set down across the move of the status constants out of
|
||||
// app/admin/service/dto, which touched every request path.
|
||||
var operaLogKeys = []string{
|
||||
"_fullPath", "operUrl", "operIp", "operLocation", "operName",
|
||||
"requestMethod", "operParam", "operTime", "jsonResult", "latencyTime",
|
||||
"statusCode", "userAgent", "createBy", "updateBy", "status",
|
||||
}
|
||||
|
||||
func TestOperaLogFieldsAreComplete(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodPost, "/api/v1/sys-user", nil)
|
||||
c.Request.Header.Set("User-Agent", "go-test")
|
||||
|
||||
l := operaLogFields(c, "127.0.0.1", http.StatusOK, "/api/v1/sys-user", http.MethodPost,
|
||||
12*time.Millisecond, `{"a":1}`, `{"code":200}`, http.StatusOK)
|
||||
|
||||
for _, k := range operaLogKeys {
|
||||
if _, ok := l[k]; !ok {
|
||||
t.Errorf("operation log is missing %q", k)
|
||||
}
|
||||
}
|
||||
if len(l) != len(operaLogKeys) {
|
||||
t.Errorf("operation log has %d fields, expected %d; update operaLogKeys deliberately, not to make this pass",
|
||||
len(l), len(operaLogKeys))
|
||||
}
|
||||
if got := l["operUrl"]; got != "/api/v1/sys-user" {
|
||||
t.Errorf("operUrl = %v", got)
|
||||
}
|
||||
if got := l["userAgent"]; got != "go-test" {
|
||||
t.Errorf("userAgent = %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// status is what tells a failed request from a successful one in the log table.
|
||||
// It is a string, and it is the one field whose source package changed.
|
||||
func TestOperaLogStatusMapping(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
status int
|
||||
want string
|
||||
}{
|
||||
{"ok", http.StatusOK, global.OperaStatusEnabled},
|
||||
{"error", http.StatusInternalServerError, global.OperaStatusDisabled},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
l := operaLogFields(c, "127.0.0.1", tc.status, "/", http.MethodGet, 0, "", "", tc.status)
|
||||
if l["status"] != tc.want {
|
||||
t.Fatalf("status = %v, want %v", l["status"], tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -59,6 +59,33 @@ func AuthCheckRole() gin.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// EnforceRoleFor reports whether the caller's role has explicit Casbin
|
||||
// permission to act on path with method.
|
||||
//
|
||||
// AuthCheckRole never calls Enforce for a route CasbinExclude lists - that
|
||||
// is the whole point of the list. A handler on such a route can still need
|
||||
// the real answer for part of what it does: sys_user.go's Update shares its
|
||||
// excluded route between the personal-center screen editing the caller's own
|
||||
// record (which is why the route is excluded at all) and an admin editing
|
||||
// someone else's, and only the second case is meant to require a policy
|
||||
// grant. That handler asks here instead of assuming the middleware already
|
||||
// checked.
|
||||
func EnforceRoleFor(c *gin.Context, path, method string) (bool, error) {
|
||||
data, ok := c.Get(jwtauth.JwtPayloadKey)
|
||||
if !ok {
|
||||
return false, nil
|
||||
}
|
||||
v, ok := data.(jwtauth.MapClaims)
|
||||
if !ok {
|
||||
return false, nil
|
||||
}
|
||||
if v["rolekey"] == "admin" {
|
||||
return true, nil
|
||||
}
|
||||
e := sdk.Runtime.GetCasbinByTenant(c.Request.Host)
|
||||
return e.Enforce(v["rolekey"], path, method)
|
||||
}
|
||||
|
||||
// excludedFromCasbin reports whether the route skips the permission check.
|
||||
//
|
||||
// It runs for every non-admin request, so the order matters: the method rules
|
||||
|
||||
@@ -5,6 +5,17 @@ import "time"
|
||||
type Migration struct {
|
||||
Version string `gorm:"primaryKey"`
|
||||
ApplyTime time.Time `gorm:"autoCreateTime"`
|
||||
|
||||
// AppCode identifies which app registered this migration. The empty string
|
||||
// means the framework itself.
|
||||
//
|
||||
// NOT NULL DEFAULT '' rather than a nullable column, and the difference is
|
||||
// not cosmetic: on a nullable column the rows that already exist when
|
||||
// AutoMigrate adds it hold NULL, and the first SELECT scanning one into
|
||||
// this string field fails with "converting NULL to string is unsupported".
|
||||
// The default is what makes "existing history belongs to the framework"
|
||||
// true without a backfill script anyone could forget to run.
|
||||
AppCode string `gorm:"type:varchar(64);not null;default:'';index:idx_sys_migration_app_code;comment:AppCode"`
|
||||
}
|
||||
|
||||
func (Migration) TableName() string {
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
# 公共契约面
|
||||
|
||||
> 本文写给**第三方应用作者**:你写一个装进 go-admin 的业务模块,可以依赖什么、
|
||||
> 怎么注册进来、哪些东西随时可能变。
|
||||
>
|
||||
> 主仓贡献者的编码约定见根目录 `AGENTS.md`,设计取舍见 `docs/architecture.md`。
|
||||
|
||||
---
|
||||
|
||||
## 承诺稳定的包
|
||||
|
||||
| 包 | 用途 |
|
||||
|---|---|
|
||||
| `common/actions` | 通用 CRUD Action(Index / View / Create / Update / Delete / Permission) |
|
||||
| `common/dto` | 分页、`search` tag 解析、`Control` / `Index` 接口 |
|
||||
| `common/models` | `ActiveRecord`、`ControlBy`、`ModelTime`、`Model` |
|
||||
| `common/middleware` | `AuthCheckRole`、`InitMiddleware` 等 |
|
||||
|
||||
**依据不是拍脑袋列的**:`app/demo` 是一个可编译、有测试、CI 会跑的标准 CRUD 模块,
|
||||
把它的 `go-admin/` 前缀 import 全部去重之后,恰好就是这四个包 —— 它代表
|
||||
"写一个标准模块所需要的最小依赖面"。你的模块如果需要第五个包,先在 issue 里说一声,
|
||||
那多半意味着契约面缺了什么。
|
||||
|
||||
"稳定"的含义:**在 `2.x` 内不做破坏性变更**。新增导出符号不算破坏;改签名、
|
||||
改语义、删除导出符号算,会走 major 版本并在 release note 里单列。
|
||||
|
||||
### 没有已知例外
|
||||
|
||||
这四个包**不 import `app/` 下的任何东西**,2026-08-31 起由 CI 强制
|
||||
(见下方「边界由 CI 守着」)。在此之前有两处反向依赖,都已根治:
|
||||
|
||||
| 原位置 | 反向依赖 | 处理 |
|
||||
|---|---|---|
|
||||
| `common/middleware/logger.go` | `app/admin/service/dto` 的两个操作日志状态常量 | 常量下沉到 `common/global`,`dto` 侧保留同名常量作为 deprecated 别名,fork 不受影响 |
|
||||
| `common/middleware/handler/auth.go` | `app/admin/models` 的 `SysUser` / `SysRole` | 该段断言恒失败、设的是零值且开源版无人读取,属死代码,已删除 |
|
||||
|
||||
之所以不把它们记成"已知例外":这份文档的作用就是告诉你哪些包可以依赖,
|
||||
如果第一条下面就挂着例外脚注,后来人会照着例外抄,边界从第一天起就是脏的。
|
||||
|
||||
---
|
||||
|
||||
## 其余包不保证稳定
|
||||
|
||||
`common/` 下没有出现在上表里的包(`common/global`、`common/storage`、
|
||||
`common/database`、`common/file_store`、`common/response`、`common/service`、
|
||||
`common/apis`、`common/middleware/handler`、根 `common` 包……)以及
|
||||
`app/admin` 的内部实现,**均不承诺稳定**。
|
||||
|
||||
其中 `common/global`、`common/middleware/handler`、根 `common` 包是
|
||||
`common/middleware` 的编译期依赖 —— 它们会被一起拉进你的依赖图,但这不代表
|
||||
它们的 API 稳定。**不要因为"都在 `common/` 目录下"就认为是契约面。**
|
||||
|
||||
规划中的 001(模块路径改名)会把非契约包移进 `internal/`,由编译器强制这条边界。
|
||||
届时上表之外的包对外部模块直接不可见 —— 现在就照上表写,那次改动对你零成本。
|
||||
|
||||
---
|
||||
|
||||
## 注册路由
|
||||
|
||||
一个应用模块要注册自己的路由,写一个 `func()` 签名的 `InitRouter`
|
||||
(照抄 `app/demo/router/router.go`),然后二选一接进来:
|
||||
|
||||
```go
|
||||
// 方式一(历史写法,仍然有效):在主仓 cmd/api/<name>.go 里
|
||||
AppRouters = append(AppRouters, router.InitRouter)
|
||||
|
||||
// 方式二(推荐):不需要 import go-admin/cmd/api
|
||||
sdk.Runtime.SetAppRouters(router.InitRouter)
|
||||
```
|
||||
|
||||
方式二是本次新接上的。差别只有一个但很关键:方式一要求你的模块
|
||||
`import "go-admin/cmd/api"` —— 那是主程序的命令包,让业务模块依赖它很别扭,
|
||||
也正是"主仓要为每个模块加一个七行文件"的根源。
|
||||
|
||||
**执行顺序**:先跑完包级 `AppRouters`,再由 core 的 `sdk.Runtime.RunAppRouters()`
|
||||
跑它自己的注册表,各自内部保持注册顺序。别依赖跨来源的相对顺序,各模块的
|
||||
`RouterGroup` 前缀互不相同,本来就不该有顺序依赖。
|
||||
|
||||
走方式二还多拿到两样东西,都在 core 那边实现(见
|
||||
[core 的 `docs/contract.md`](https://github.com/go-admin-team/go-admin-core/blob/main/docs/contract.md)):
|
||||
**panic 护栏**——你的 `InitRouter` panic 了,其余模块照常注册、进程不退出,日志里会写明
|
||||
是哪一行注册的;**失败分级**——`sdk.Runtime.SetAppRoutersWith(f, runtime.WithFatal())`
|
||||
声明「我起不来就别启动」。方式一(包级 `AppRouters`)没有护栏,panic 直接掀桌。
|
||||
|
||||
`InitRouter()` 内部的约定:自己拿 `sdk.Runtime.GetEngine()`,按需建
|
||||
`gin.RouterGroup`,通过 `init()` 自注册到你自己包内的
|
||||
`routerCheckRole` / `routerNoCheckRole` 列表,不在任何中心文件手工列举
|
||||
(与 `AGENTS.md`「路由注册」一节一致)。
|
||||
|
||||
---
|
||||
|
||||
## 注册数据库迁移
|
||||
|
||||
框架自身的迁移不变:
|
||||
|
||||
```go
|
||||
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700001000DemoMenu)
|
||||
```
|
||||
|
||||
应用的迁移走 `ForApp`:
|
||||
|
||||
```go
|
||||
func init() {
|
||||
_, fileName, _, _ := runtime.Caller(0)
|
||||
migration.ForApp("crm").SetVersion(migration.GetFilename(fileName), initCrmTables)
|
||||
}
|
||||
|
||||
func initCrmTables(db *gorm.DB, version, appCode string) error {
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
// ... schema / data changes ...
|
||||
return tx.Create(&common.Migration{Version: version, AppCode: appCode}).Error
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
四条必须知道的规则:
|
||||
|
||||
1. **完成记录由迁移函数自己写**,而且要写在自己的事务里。框架的调度循环只做
|
||||
"这个 version 在 `sys_migration` 里有没有" 的判断,从不代你插入 —— 这样
|
||||
"数据改完了"和"标记成已完成"才是同一个事务,不会出现改了一半却被记成成功。
|
||||
2. **`AppCode` 必须写进去**。签名多带一个 `appCode` 参数就是为此 —— 忘了写,
|
||||
schema 上那一列等于白加,你的迁移会被记成框架的。
|
||||
3. **落库的 `version` 是加了前缀的**。`ForApp("crm")` 注册 `1786800001000`,
|
||||
实际写进 `sys_migration.version` 的是 `crm-1786800001000`,函数收到的
|
||||
`version` 参数已经是这个带前缀的值,照抄进 `common.Migration{Version: version}`
|
||||
即可。前缀的意义是:两个来源不同的应用哪怕碰巧生成同一个毫秒时间戳,也不会撞主键、
|
||||
不会有一方被误判为"已应用"。
|
||||
4. **应用 code 一律小写**,`ForApp` 会自己 `strings.ToLower` 一遍。`core` 是保留字
|
||||
(`migrate status` 用它表示框架自身,`--app core` 选中框架),`ForApp("core")`
|
||||
会 panic。
|
||||
|
||||
顺序保证:**同一应用内按版本号严格有序**。跨应用顺序不做承诺 —— 由于前缀的存在,
|
||||
今天的实际顺序是"先跑完全部框架迁移,再按 appCode 字母序逐个应用跑完",
|
||||
但这是实现细节,不要依赖它。跨应用依赖(应用 A 的迁移要求应用 B 先跑完)
|
||||
需要依赖拓扑排序,属于后续阶段。
|
||||
|
||||
看当前状态、看这次会跑什么,不用猜:
|
||||
|
||||
```bash
|
||||
go-admin migrate status -c config/settings.yml # 按应用分组列出已应用 / 待应用
|
||||
go-admin migrate --dry-run -c config/settings.yml # 列出会执行什么、什么顺序,不写库
|
||||
go-admin migrate --app crm -c config/settings.yml # 只跑 crm 的迁移
|
||||
```
|
||||
|
||||
`status` 与 `--dry-run` 是纯只读的,不建表、不改表结构,可以直接对生产库执行。
|
||||
|
||||
---
|
||||
|
||||
## 硬约束:注册要赶在启动钩子之前
|
||||
|
||||
三个注册入口——`AppRouters`、`sdk.Runtime.SetAppRouters`、`migration.ForApp`——
|
||||
都必须在 `cmd/api/server.go` 的 `runStartupHooks()` 执行之前调用完。
|
||||
|
||||
`init()` 是最省事的位置:Go 规范保证包级变量初始化与 `init()` 在 `main()` 之前
|
||||
**单 goroutine 顺序执行**,注册期天然没有并发写。但它不是唯一合法位置——
|
||||
在 `run()` 之类早于启动钩子的地方注册同样成立。这条规则约束的是**顺序**,
|
||||
不是你写在哪个函数里。
|
||||
|
||||
`sdk.Runtime.SetAppRouters` 的准确语义以 core 为准:
|
||||
|
||||
> [go-admin-core `docs/contract.md`](https://github.com/go-admin-team/go-admin-core/blob/main/docs/contract.md)
|
||||
|
||||
那份文档写明了注册类与资源类的划分、封闭时刻、护栏边界(**只覆盖同步 panic,
|
||||
你自己 `go func()` 出去的 panic 框架够不着**)、以及配置热更新会在运行期
|
||||
重新执行 setup 回调这件事。
|
||||
|
||||
主仓这边只补三条它管不着的:
|
||||
|
||||
1. **`RunAppRouters()` 跑过之后,core 的注册表就封闭了**,再调
|
||||
`sdk.Runtime.SetAppRouters` 会被丢弃并记一条 ERROR 日志。包级 `AppRouters`
|
||||
没有这个机制——它就是一个普通 slice,什么时候 append 都"成功",
|
||||
但 `runStartupHooks()` 之后 append 的那些永远不会被执行,且不出声。
|
||||
这是继续推荐方式二的理由之一。
|
||||
2. **封闭是黏性的,而 `sdk.Runtime` 是包级单例。** 写测试时若会触发启动钩子,
|
||||
必须换掉它再还原,否则同一个测试二进制里后面的测试会静默丢注册:
|
||||
|
||||
```go
|
||||
previous := sdk.Runtime
|
||||
t.Cleanup(func() { sdk.Runtime = previous })
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
```
|
||||
|
||||
`cmd/api/server_test.go` 里的 `freshRuntime` 就是这个。
|
||||
3. **`migration.ForApp` 是主仓的东西**,core 不认识它,上面那份文档不覆盖它。
|
||||
它的约束仍然是"注册要在迁移调度循环跑起来之前",实践上就是 `init()`。
|
||||
|
||||
---
|
||||
|
||||
## 边界由 CI 守着
|
||||
|
||||
`common/`、`core/` 不得 import `app/`,这条由 `tools/checksilent` 的
|
||||
`contract-import-boundary` 检查固化,`make checksilent` 在 CI 里跑,违反即失败
|
||||
(测试文件同样算 —— 一个删掉 `app/admin` 的 fork 也应该能跑 `go test ./...`)。
|
||||
|
||||
靠人工评审列契约面会漏。上面那两处反向依赖里,第二处就是评审没发现、
|
||||
靠机器全量扫描才找出来的。
|
||||
|
||||
`tools/checksilent` 还检查另外五类"不出声的失败",写模块时值得先看一眼
|
||||
`go run ./tools/checksilent -h`。
|
||||
@@ -11,7 +11,7 @@ require (
|
||||
github.com/casbin/casbin/v3 v3.8.1
|
||||
github.com/gin-gonic/gin v1.12.0
|
||||
github.com/glebarez/sqlite v1.11.0
|
||||
github.com/go-admin-team/go-admin-core/v2 v2.3.0
|
||||
github.com/go-admin-team/go-admin-core/v2 v2.4.1
|
||||
github.com/google/uuid v1.6.0
|
||||
github.com/huaweicloud/huaweicloud-sdk-go-obs v3.26.6+incompatible
|
||||
github.com/mssola/user_agent v0.6.0
|
||||
|
||||
@@ -145,8 +145,8 @@ github.com/glebarez/go-sqlite v1.22.0 h1:uAcMJhaA6r3LHMTFgP0SifzgXg46yJkgxqyuyec
|
||||
github.com/glebarez/go-sqlite v1.22.0/go.mod h1:PlBIdHe0+aUEFn+r2/uthrWq4FxbzugL0L8Li6yQJbc=
|
||||
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
|
||||
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
|
||||
github.com/go-admin-team/go-admin-core/v2 v2.3.0 h1:P2Po+jsJByzv1y2cN8eEGfFOutWSYNzQTJxLGAFmHkE=
|
||||
github.com/go-admin-team/go-admin-core/v2 v2.3.0/go.mod h1:YiJr2+vqC9qV5AoGeL+1W55h3XZ99CB5xWnP3Wo8c5g=
|
||||
github.com/go-admin-team/go-admin-core/v2 v2.4.1 h1:69QprBVMcQzjVP0UksCwi//A0qh8gwcIHcwHmABPp2U=
|
||||
github.com/go-admin-team/go-admin-core/v2 v2.4.1/go.mod h1:YiJr2+vqC9qV5AoGeL+1W55h3XZ99CB5xWnP3Wo8c5g=
|
||||
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
|
||||
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
|
||||
github.com/go-kit/kit v0.10.0/go.mod h1:xUsJbQ/Fp4kEt7AFgCuvyX4a71u8h9jB8tj/ORgOZ7o=
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"go/ast"
|
||||
"go/token"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// structLiteral is a composite literal whose type resolved to a named struct.
|
||||
type structLiteral struct {
|
||||
PkgPath string
|
||||
Name string
|
||||
Lit *ast.CompositeLit
|
||||
}
|
||||
|
||||
// forEachStructLiteral visits every composite literal in the file whose type
|
||||
// resolves to a named type, including the ones written with the type elided.
|
||||
//
|
||||
// The elided form is the one that matters: seed data is written as
|
||||
// []models.SysMenu{{MenuId: 9000}, {MenuId: 9001}}, and the inner literals carry
|
||||
// no type of their own. A walker that only looked at CompositeLit.Type would
|
||||
// silently skip every seed in the repository and report nothing, which for a
|
||||
// tool about silent failure would be its own punchline.
|
||||
func forEachStructLiteral(sf *sourceFile, fn func(structLiteral)) {
|
||||
// The type each type-less literal inherits from the literal containing it.
|
||||
elided := map[*ast.CompositeLit]ast.Expr{}
|
||||
|
||||
var propagate func(lit *ast.CompositeLit, typ ast.Expr)
|
||||
propagate = func(lit *ast.CompositeLit, typ ast.Expr) {
|
||||
child := elementType(typ)
|
||||
if child == nil {
|
||||
return
|
||||
}
|
||||
for _, elt := range lit.Elts {
|
||||
v := elt
|
||||
if kv, ok := elt.(*ast.KeyValueExpr); ok {
|
||||
v = kv.Value
|
||||
}
|
||||
if cl, ok := v.(*ast.CompositeLit); ok && cl.Type == nil {
|
||||
elided[cl] = child
|
||||
propagate(cl, child)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ast.Inspect visits a node before its children, so every typed literal
|
||||
// fills in its descendants before the reporting pass reaches them.
|
||||
ast.Inspect(sf.Syntax, func(n ast.Node) bool {
|
||||
cl, ok := n.(*ast.CompositeLit)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
if typ := litType(cl, elided); typ != nil {
|
||||
propagate(cl, typ)
|
||||
}
|
||||
return true
|
||||
})
|
||||
|
||||
ast.Inspect(sf.Syntax, func(n ast.Node) bool {
|
||||
cl, ok := n.(*ast.CompositeLit)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
typ := litType(cl, elided)
|
||||
if typ == nil {
|
||||
return true
|
||||
}
|
||||
if pkg, name, ok := resolveNamed(sf, typ); ok {
|
||||
fn(structLiteral{PkgPath: pkg, Name: name, Lit: cl})
|
||||
}
|
||||
return true
|
||||
})
|
||||
}
|
||||
|
||||
func litType(cl *ast.CompositeLit, elided map[*ast.CompositeLit]ast.Expr) ast.Expr {
|
||||
if cl.Type != nil {
|
||||
return cl.Type
|
||||
}
|
||||
return elided[cl]
|
||||
}
|
||||
|
||||
// resolveNamed maps a type expression to (import path, type name). A bare
|
||||
// identifier means a type declared in this file's own package.
|
||||
func resolveNamed(sf *sourceFile, typ ast.Expr) (string, string, bool) {
|
||||
switch t := typ.(type) {
|
||||
case *ast.StarExpr:
|
||||
return resolveNamed(sf, t.X)
|
||||
case *ast.Ident:
|
||||
return sf.Pkg, t.Name, true
|
||||
case *ast.SelectorExpr:
|
||||
pkgIdent, ok := t.X.(*ast.Ident)
|
||||
if !ok {
|
||||
return "", "", false
|
||||
}
|
||||
path, ok := sf.imports[pkgIdent.Name]
|
||||
if !ok {
|
||||
return "", "", false
|
||||
}
|
||||
return path, t.Sel.Name, true
|
||||
}
|
||||
return "", "", false
|
||||
}
|
||||
|
||||
// elementType is the type the children of a composite literal take when they
|
||||
// leave theirs out.
|
||||
func elementType(typ ast.Expr) ast.Expr {
|
||||
switch t := typ.(type) {
|
||||
case *ast.ArrayType:
|
||||
return t.Elt
|
||||
case *ast.MapType:
|
||||
return t.Value
|
||||
case *ast.StarExpr:
|
||||
return elementType(t.X)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// field returns the value written for a named field of a struct literal.
|
||||
func field(lit *ast.CompositeLit, name string) (ast.Expr, bool) {
|
||||
for _, elt := range lit.Elts {
|
||||
kv, ok := elt.(*ast.KeyValueExpr)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if key, ok := kv.Key.(*ast.Ident); ok && key.Name == name {
|
||||
return kv.Value, true
|
||||
}
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// intValue evaluates an integer field: a literal, a negated literal, or an
|
||||
// identifier naming a constant in the same package.
|
||||
//
|
||||
// Anything computed at run time is skipped rather than guessed at. That is the
|
||||
// one thing these checks miss, and missing is the right way to be wrong here -
|
||||
// a false positive teaches people to add ignore comments, and then the tool is
|
||||
// finished.
|
||||
func intValue(sf *sourceFile, expr ast.Expr) (int64, bool) {
|
||||
switch e := expr.(type) {
|
||||
case *ast.Ident:
|
||||
v, ok := sf.consts[e.Name]
|
||||
return v, ok
|
||||
case *ast.UnaryExpr:
|
||||
if e.Op == token.SUB {
|
||||
if v, ok := intValue(sf, e.X); ok {
|
||||
return -v, true
|
||||
}
|
||||
}
|
||||
}
|
||||
return intLiteral(expr)
|
||||
}
|
||||
|
||||
func intLiteral(expr ast.Expr) (int64, bool) {
|
||||
lit, ok := expr.(*ast.BasicLit)
|
||||
if !ok || lit.Kind != token.INT {
|
||||
return 0, false
|
||||
}
|
||||
v, err := strconv.ParseInt(strings.ReplaceAll(lit.Value, "_", ""), 0, 64)
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
return v, true
|
||||
}
|
||||
|
||||
// stringValue evaluates a string field: a literal, a concatenation of literals,
|
||||
// or an identifier naming a string constant in the same package.
|
||||
func stringValue(sf *sourceFile, expr ast.Expr) (string, bool) {
|
||||
switch e := expr.(type) {
|
||||
case *ast.BasicLit:
|
||||
if e.Kind != token.STRING {
|
||||
return "", false
|
||||
}
|
||||
s, err := strconv.Unquote(e.Value)
|
||||
if err != nil {
|
||||
return "", false
|
||||
}
|
||||
return s, true
|
||||
case *ast.BinaryExpr:
|
||||
if e.Op != token.ADD {
|
||||
return "", false
|
||||
}
|
||||
l, lok := stringValue(sf, e.X)
|
||||
r, rok := stringValue(sf, e.Y)
|
||||
if lok && rok {
|
||||
return l + r, true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// embeddedTypes returns the types a struct embeds, as (import path, name).
|
||||
func embeddedTypes(sf *sourceFile, st *ast.StructType) [][2]string {
|
||||
var out [][2]string
|
||||
for _, f := range st.Fields.List {
|
||||
if len(f.Names) != 0 {
|
||||
continue // a named field, not an embed
|
||||
}
|
||||
if pkg, name, ok := resolveNamed(sf, f.Type); ok {
|
||||
out = append(out, [2]string{pkg, name})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// tableNames maps struct name to the literal its TableName method returns.
|
||||
func tableNames(sf *sourceFile) map[string]string {
|
||||
out := map[string]string{}
|
||||
for _, decl := range sf.Syntax.Decls {
|
||||
fn, ok := decl.(*ast.FuncDecl)
|
||||
if !ok || fn.Name.Name != "TableName" || fn.Recv == nil || len(fn.Recv.List) != 1 || fn.Body == nil {
|
||||
continue
|
||||
}
|
||||
recv := receiverName(fn.Recv.List[0].Type)
|
||||
if recv == "" {
|
||||
continue
|
||||
}
|
||||
ast.Inspect(fn.Body, func(n ast.Node) bool {
|
||||
ret, ok := n.(*ast.ReturnStmt)
|
||||
if !ok || len(ret.Results) != 1 {
|
||||
return true
|
||||
}
|
||||
if s, ok := stringValue(sf, ret.Results[0]); ok && out[recv] == "" {
|
||||
out[recv] = s
|
||||
}
|
||||
return true
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func receiverName(expr ast.Expr) string {
|
||||
switch t := expr.(type) {
|
||||
case *ast.Ident:
|
||||
return t.Name
|
||||
case *ast.StarExpr:
|
||||
return receiverName(t.X)
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
// structTypes maps struct name to its declaration.
|
||||
func structTypes(sf *sourceFile) map[string]*ast.StructType {
|
||||
out := map[string]*ast.StructType{}
|
||||
ast.Inspect(sf.Syntax, func(n ast.Node) bool {
|
||||
ts, ok := n.(*ast.TypeSpec)
|
||||
if !ok {
|
||||
return true
|
||||
}
|
||||
if st, ok := ts.Type.(*ast.StructType); ok {
|
||||
out[ts.Name.Name] = st
|
||||
}
|
||||
return true
|
||||
})
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,409 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"go/token"
|
||||
"path"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Check names. They appear in every message and in the CI log, so they are
|
||||
// what people will search for.
|
||||
const (
|
||||
checkModelTimeMix = "modeltime-mix"
|
||||
checkMenuSort = "menu-sort-overflow"
|
||||
checkMenuName = "menu-name-mismatch"
|
||||
checkConfigValue = "config-value-truncation"
|
||||
checkMenuIDConflict = "menu-id-collision"
|
||||
checkImportBoundary = "contract-import-boundary"
|
||||
)
|
||||
|
||||
// Package paths, relative to the module. Spelled once so a module rename
|
||||
// touches one place.
|
||||
const (
|
||||
pkgFrozenModels = "cmd/migrate/migration/models"
|
||||
pkgRuntimeModel = "common/models"
|
||||
pkgAdminModels = "app/admin/models"
|
||||
)
|
||||
|
||||
// options are the run-time knobs. Only the frontend directory is one: every
|
||||
// other check either applies or does not, with nothing to configure.
|
||||
type options struct {
|
||||
// UIDir is the go-admin-ui src directory. Empty disables checkMenuName,
|
||||
// which is the only check that needs a second repository.
|
||||
UIDir string
|
||||
}
|
||||
|
||||
// runChecks runs every check over one parse of the tree.
|
||||
func runChecks(s *snapshot, opt options) ([]Finding, error) {
|
||||
var out []Finding
|
||||
out = append(out, checkModelTimeMixing(s)...)
|
||||
out = append(out, checkMenuSortOverflow(s)...)
|
||||
out = append(out, checkConfigValueLength(s)...)
|
||||
out = append(out, checkMenuIDCollisions(s)...)
|
||||
out = append(out, checkContractImportBoundary(s)...)
|
||||
|
||||
if opt.UIDir != "" {
|
||||
fs, err := checkMenuNames(s, opt.UIDir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, fs...)
|
||||
}
|
||||
|
||||
sortFindings(out)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (s *snapshot) pkg(rel string) string { return s.ModulePath + "/" + rel }
|
||||
|
||||
func (s *snapshot) finding(sev Severity, check string, sf *sourceFile, pos posLike, format string, args ...interface{}) Finding {
|
||||
file, line, col := s.Pos(sf, pos.Pos())
|
||||
return Finding{
|
||||
Check: check,
|
||||
Severity: sev.String(),
|
||||
File: file,
|
||||
Line: line,
|
||||
Col: col,
|
||||
Message: fmt.Sprintf(format, args...),
|
||||
severity: sev,
|
||||
}
|
||||
}
|
||||
|
||||
type posLike interface{ Pos() token.Pos }
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// check 1: the two ModelTime flavours
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// checkModelTimeMixing reports code that mixes the repository's two soft-delete
|
||||
// shapes.
|
||||
//
|
||||
// cmd/migrate/migration/models.ModelTime declares a nullable gorm.DeletedAt;
|
||||
// common/models.ModelTime declares the NOT NULL millisecond marker. Mixing them
|
||||
// on one table is not a compile error and not a run-time error either: gorm
|
||||
// scopes the nullable flavour as "WHERE deleted_at IS NULL" while live rows hold
|
||||
// 0, so every row of the table becomes invisible and the feature reading it
|
||||
// simply returns nothing. sys_columns and sys_tables sat in that state until
|
||||
// 1786700004000 - the code generator listed no tables at all and reported no
|
||||
// error.
|
||||
//
|
||||
// Two shapes are reported, and both are unambiguous:
|
||||
//
|
||||
// 1. a runtime model under app/ that embeds the frozen package's time struct -
|
||||
// always wrong, that package is the shape the columns had before the
|
||||
// conversion;
|
||||
// 2. a migration ordered after the conversion that imports the frozen package
|
||||
// - AGENTS.md states this rule, and the version/ directory already has a
|
||||
// test for it; this extends it to version-local/, where third-party and
|
||||
// downstream migrations live and where no test was watching.
|
||||
//
|
||||
// Not reported: that two model packages describe the same table with different
|
||||
// flavours. That is true of a dozen tables on purpose - the frozen package is
|
||||
// correct for the migrations that predate the conversion - so reporting it
|
||||
// would be reporting the design.
|
||||
func checkModelTimeMixing(s *snapshot) []Finding {
|
||||
var out []Finding
|
||||
frozen := s.pkg(pkgFrozenModels)
|
||||
|
||||
for _, sf := range s.Files {
|
||||
if strings.HasPrefix(sf.Path, "app/") && sf.Imports(frozen) {
|
||||
tables := tableNames(sf)
|
||||
for name, st := range structTypes(sf) {
|
||||
table, isModel := tables[name]
|
||||
if !isModel {
|
||||
continue
|
||||
}
|
||||
for _, emb := range embeddedTypes(sf, st) {
|
||||
if emb[0] != frozen {
|
||||
continue
|
||||
}
|
||||
out = append(out, s.finding(Error, checkModelTimeMix, sf, st,
|
||||
"runtime model %s (table %s) embeds %s.%s, whose DeletedAt is the nullable pre-conversion shape;\n"+
|
||||
" gorm will query this table with deleted_at IS NULL while live rows hold 0, and it will return nothing.\n"+
|
||||
" Embed %s.ModelTime instead.",
|
||||
name, table, pkgFrozenModels, emb[1], pkgRuntimeModel))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
version, isMigration := migrationVersion(sf.Path)
|
||||
if !isMigration || version <= softDeleteConversion || !sf.Imports(frozen) {
|
||||
continue
|
||||
}
|
||||
spec := sf.ImportSpec(frozen)
|
||||
out = append(out, s.finding(Error, checkModelTimeMix, sf, spec,
|
||||
"migration %d is ordered after the soft-delete conversion (%d) but seeds through %s;\n"+
|
||||
" that package writes a nullable deleted_at into a NOT NULL column, and reads through it match no rows.\n"+
|
||||
" Use the runtime models under app/ instead.",
|
||||
version, softDeleteConversion, pkgFrozenModels))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// softDeleteConversion is the version at which deleted_at stopped being a
|
||||
// nullable timestamp and became the NOT NULL millisecond marker.
|
||||
const softDeleteConversion = 1786700003000
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// check 2: menu sort overflows a tinyint
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// checkMenuSortOverflow reports a seeded menu sort outside a tinyint.
|
||||
//
|
||||
// sys_menu.sort is `gorm:"size:4"`, which MySQL builds as a tinyint holding
|
||||
// -128..127. sqlite ignores the width, so an overflowing value passes every
|
||||
// local test and fails on a real install - with Error 1264, partway through a
|
||||
// migration that is not transactional, leaving every later migration unapplied.
|
||||
// That is how a seeded Sort: 900 once stopped the run before the soft-delete
|
||||
// conversion and left nobody able to log in.
|
||||
func checkMenuSortOverflow(s *snapshot) []Finding {
|
||||
const (
|
||||
min = -128
|
||||
max = 127
|
||||
)
|
||||
var out []Finding
|
||||
for _, sf := range s.Files {
|
||||
forEachStructLiteral(sf, func(lit structLiteral) {
|
||||
if !s.isMenuModel(lit) {
|
||||
return
|
||||
}
|
||||
expr, ok := field(lit.Lit, "Sort")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
v, ok := intValue(sf, expr)
|
||||
if !ok || (v >= min && v <= max) {
|
||||
return
|
||||
}
|
||||
out = append(out, s.finding(Error, checkMenuSort, sf, expr,
|
||||
"menu sort %d does not fit a tinyint (%d..%d);\n"+
|
||||
" MySQL rejects it with Error 1264 and the migration stops there, leaving later migrations unapplied.",
|
||||
v, min, max))
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// check 4: sys_config value truncation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// checkConfigValueLength reports a seeded sys_config value longer than the
|
||||
// column.
|
||||
//
|
||||
// config_value is varchar(255). MySQL outside strict mode truncates rather than
|
||||
// refusing, so the migration succeeds, the row is written, and the setting is
|
||||
// silently half of what was intended.
|
||||
//
|
||||
// Counted in runes, not bytes, because varchar(255) counts characters - byte
|
||||
// counting would flag Chinese values that fit.
|
||||
func checkConfigValueLength(s *snapshot) []Finding {
|
||||
const limit = 255
|
||||
var out []Finding
|
||||
for _, sf := range s.Files {
|
||||
forEachStructLiteral(sf, func(lit structLiteral) {
|
||||
if lit.Name != "SysConfig" || !s.isModelPackage(lit.PkgPath) {
|
||||
return
|
||||
}
|
||||
expr, ok := field(lit.Lit, "ConfigValue")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
v, ok := stringValue(sf, expr)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if n := len([]rune(v)); n > limit {
|
||||
out = append(out, s.finding(Error, checkConfigValue, sf, expr,
|
||||
"sys_config.config_value is %d characters, over the varchar(%d) column;\n"+
|
||||
" MySQL outside strict mode truncates instead of failing, so the migration succeeds with half the value.",
|
||||
n, limit))
|
||||
}
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// check 5: hard-coded menu ids colliding
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// checkMenuIDCollisions reports the same menu id seeded from more than one file.
|
||||
//
|
||||
// menu_id is the primary key and every seed is an upsert, so two modules that
|
||||
// pick the same id do not collide loudly - the second overwrites the first, and
|
||||
// which one wins depends on migration order. One module's menu quietly becomes
|
||||
// the other's.
|
||||
//
|
||||
// Only across files. Inside one file the same id appearing twice is the same
|
||||
// menu being written and then referenced, which is how the seeds are written
|
||||
// today and not a mistake.
|
||||
func checkMenuIDCollisions(s *snapshot) []Finding {
|
||||
type site struct {
|
||||
sf *sourceFile
|
||||
expr posLike
|
||||
file string
|
||||
line int
|
||||
}
|
||||
sites := map[int64][]site{}
|
||||
|
||||
for _, sf := range s.Files {
|
||||
forEachStructLiteral(sf, func(lit structLiteral) {
|
||||
if !s.isMenuModel(lit) {
|
||||
return
|
||||
}
|
||||
expr, ok := field(lit.Lit, "MenuId")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
v, ok := intValue(sf, expr)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
file, line, _ := s.Pos(sf, expr.Pos())
|
||||
sites[v] = append(sites[v], site{sf: sf, expr: expr, file: file, line: line})
|
||||
})
|
||||
}
|
||||
|
||||
ids := make([]int64, 0, len(sites))
|
||||
for id := range sites {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
sort.Slice(ids, func(i, j int) bool { return ids[i] < ids[j] })
|
||||
|
||||
var out []Finding
|
||||
for _, id := range ids {
|
||||
group := sites[id]
|
||||
files := map[string]bool{}
|
||||
for _, st := range group {
|
||||
files[st.file] = true
|
||||
}
|
||||
if len(files) < 2 {
|
||||
continue
|
||||
}
|
||||
sort.Slice(group, func(i, j int) bool {
|
||||
if group[i].file != group[j].file {
|
||||
return group[i].file < group[j].file
|
||||
}
|
||||
return group[i].line < group[j].line
|
||||
})
|
||||
related := make([]string, 0, len(group)-1)
|
||||
for _, st := range group[1:] {
|
||||
related = append(related, fmt.Sprintf("also at %s:%d", st.file, st.line))
|
||||
}
|
||||
f := s.finding(Error, checkMenuIDConflict, group[0].sf, group[0].expr,
|
||||
"menu id %d is seeded from %d files;\n"+
|
||||
" menu_id is the primary key and the seeds upsert, so whichever migration runs last overwrites the other's menu.",
|
||||
id, len(files))
|
||||
f.Related = related
|
||||
out = append(out, f)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// check 6: the contract packages must not import app/
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// contractRoots are the trees that may not depend on a business module. core/
|
||||
// does not exist in this repository yet and is listed because the boundary is
|
||||
// declared for both in docs/contract.md; naming it here means the check is
|
||||
// already in place the day the directory appears.
|
||||
//
|
||||
// Which of them actually exist is reported by ScannedContractRoots, because a
|
||||
// root that is absent contributes nothing and a check that silently covers less
|
||||
// than it claims is worse than no check: it leaves people believing a boundary
|
||||
// is guarded when nothing is guarding it.
|
||||
var contractRoots = []string{"common/", "core/"}
|
||||
|
||||
// ScannedContractRoots splits contractRoots by whether the snapshot actually
|
||||
// holds files under them, so the summary can name what was covered.
|
||||
func ScannedContractRoots(s *snapshot) (scanned, absent []string) {
|
||||
for _, root := range contractRoots {
|
||||
found := false
|
||||
for _, sf := range s.Files {
|
||||
if strings.HasPrefix(sf.Path, root) {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if found {
|
||||
scanned = append(scanned, strings.TrimSuffix(root, "/"))
|
||||
} else {
|
||||
absent = append(absent, strings.TrimSuffix(root, "/"))
|
||||
}
|
||||
}
|
||||
return scanned, absent
|
||||
}
|
||||
|
||||
// checkContractImportBoundary reports a contract package importing app/.
|
||||
//
|
||||
// docs/contract.md promises four packages under common/ as the surface an app
|
||||
// may build on. A promise like that stops being true the moment the surface
|
||||
// imports one particular app: a fork that replaces app/admin then cannot
|
||||
// compile common/middleware, and an app can no longer be built against the
|
||||
// contract alone. Nothing about it fails visibly - it fails when somebody tries
|
||||
// to take the framework apart, which is the whole point of the exercise.
|
||||
//
|
||||
// Test files count. A fork that drops app/admin should be able to run go test
|
||||
// ./... too.
|
||||
func checkContractImportBoundary(s *snapshot) []Finding {
|
||||
appPrefix := s.ModulePath + "/app/"
|
||||
var out []Finding
|
||||
for _, sf := range s.Files {
|
||||
inContract := false
|
||||
for _, root := range contractRoots {
|
||||
if strings.HasPrefix(sf.Path, root) {
|
||||
inContract = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !inContract {
|
||||
continue
|
||||
}
|
||||
for _, spec := range sf.Syntax.Imports {
|
||||
path, err := strconv.Unquote(spec.Path.Value)
|
||||
if err != nil || !strings.HasPrefix(path, appPrefix) {
|
||||
continue
|
||||
}
|
||||
out = append(out, s.finding(Error, checkImportBoundary, sf, spec,
|
||||
"%s is a contract package and imports %s;\n"+
|
||||
" a fork that replaces or drops that app can then no longer compile the contract surface it was told to build on.\n"+
|
||||
" Move what is shared down into common/, or out of the contract package.",
|
||||
sf.Path, path))
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// migrationVersion reads the 13-digit timestamp a migration file name starts
|
||||
// with. Files outside the two migration directories are not migrations, however
|
||||
// they are named.
|
||||
func migrationVersion(rel string) (int64, bool) {
|
||||
dir := path.Dir(rel)
|
||||
if dir != "cmd/migrate/migration/version" && dir != "cmd/migrate/migration/version-local" {
|
||||
return 0, false
|
||||
}
|
||||
name := path.Base(rel)
|
||||
if len(name) < 13 {
|
||||
return 0, false
|
||||
}
|
||||
v, err := strconv.ParseInt(name[:13], 10, 64)
|
||||
if err != nil {
|
||||
return 0, false
|
||||
}
|
||||
return v, true
|
||||
}
|
||||
|
||||
// isMenuModel reports whether a literal is one of the SysMenu models rather
|
||||
// than, say, the SysMenu service struct that shares the name.
|
||||
func (s *snapshot) isMenuModel(lit structLiteral) bool {
|
||||
return lit.Name == "SysMenu" && s.isModelPackage(lit.PkgPath)
|
||||
}
|
||||
|
||||
func (s *snapshot) isModelPackage(path string) bool {
|
||||
return path == s.pkg(pkgFrozenModels) || path == s.pkg(pkgAdminModels)
|
||||
}
|
||||
@@ -0,0 +1,454 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// fixture writes a miniature repository and returns its root. Every check gets
|
||||
// one of these carrying the exact mistake it exists to find, so a check that
|
||||
// stops working fails a test rather than going quiet - which would be the same
|
||||
// failure mode the tool is about.
|
||||
func fixture(t *testing.T, files map[string]string) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
files["go.mod"] = "module go-admin\n\ngo 1.26\n"
|
||||
for name, content := range files {
|
||||
path := filepath.Join(dir, filepath.FromSlash(name))
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
return dir
|
||||
}
|
||||
|
||||
func check(t *testing.T, root string, opt options) []Finding {
|
||||
t.Helper()
|
||||
s, err := load(root)
|
||||
if err != nil {
|
||||
t.Fatalf("load: %v", err)
|
||||
}
|
||||
findings, err := runChecks(s, opt)
|
||||
if err != nil {
|
||||
t.Fatalf("run: %v", err)
|
||||
}
|
||||
return findings
|
||||
}
|
||||
|
||||
func only(t *testing.T, findings []Finding, name string) []Finding {
|
||||
t.Helper()
|
||||
var out []Finding
|
||||
for _, f := range findings {
|
||||
if f.Check == name {
|
||||
out = append(out, f)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func requireOne(t *testing.T, findings []Finding, name string) Finding {
|
||||
t.Helper()
|
||||
got := only(t, findings, name)
|
||||
if len(got) != 1 {
|
||||
t.Fatalf("%s produced %d findings, want 1:\n%v", name, len(got), findings)
|
||||
}
|
||||
return got[0]
|
||||
}
|
||||
|
||||
const frozenModelsPkg = `package models
|
||||
|
||||
import (
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type ModelTime struct {
|
||||
CreatedAt time.Time
|
||||
DeletedAt gorm.DeletedAt
|
||||
}
|
||||
|
||||
type SysMenu struct {
|
||||
MenuId int
|
||||
MenuName string
|
||||
Component string
|
||||
MenuType string
|
||||
Sort int
|
||||
ModelTime
|
||||
}
|
||||
|
||||
func (SysMenu) TableName() string { return "sys_menu" }
|
||||
|
||||
type SysConfig struct {
|
||||
ConfigKey string
|
||||
ConfigValue string
|
||||
ModelTime
|
||||
}
|
||||
|
||||
func (SysConfig) TableName() string { return "sys_config" }
|
||||
`
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestModelTimeMixDetectsARuntimeModelOnTheFrozenShape(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"app/shop/models/product.go": `package models
|
||||
|
||||
import frozen "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
type Product struct {
|
||||
Id int
|
||||
frozen.ModelTime
|
||||
}
|
||||
|
||||
func (Product) TableName() string { return "shop_product" }
|
||||
`,
|
||||
})
|
||||
|
||||
f := requireOne(t, check(t, root, options{}), checkModelTimeMix)
|
||||
if f.Severity != "ERROR" {
|
||||
t.Errorf("severity = %s", f.Severity)
|
||||
}
|
||||
if !strings.Contains(f.Message, "shop_product") {
|
||||
t.Errorf("message = %s", f.Message)
|
||||
}
|
||||
if f.File != "app/shop/models/product.go" {
|
||||
t.Errorf("file = %s", f.File)
|
||||
}
|
||||
}
|
||||
|
||||
func TestModelTimeMixDetectsAPostConversionMigrationOnTheFrozenPackage(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version-local/1786700009000_seed.go": `package version_local
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seed() interface{} { return &models.SysMenu{} }
|
||||
`,
|
||||
})
|
||||
|
||||
f := requireOne(t, check(t, root, options{}), checkModelTimeMix)
|
||||
if !strings.Contains(f.Message, "1786700009000") {
|
||||
t.Errorf("message = %s", f.Message)
|
||||
}
|
||||
if f.Line != 3 {
|
||||
t.Errorf("expected the import line, got line %d", f.Line)
|
||||
}
|
||||
}
|
||||
|
||||
// A migration ordered before the conversion is right to use that package - the
|
||||
// nullable column is the shape it had at the time.
|
||||
func TestModelTimeMixLeavesPreConversionMigrationsAlone(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seed() interface{} { return &models.SysMenu{} }
|
||||
`,
|
||||
})
|
||||
if got := only(t, check(t, root, options{}), checkModelTimeMix); len(got) != 0 {
|
||||
t.Errorf("reported %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestMenuSortOverflowIsDetectedThroughAnElidedSliceLiteral(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seed() []models.SysMenu {
|
||||
return []models.SysMenu{
|
||||
{MenuId: 9000, Sort: 100},
|
||||
{MenuId: 9001, Sort: 900},
|
||||
}
|
||||
}
|
||||
`,
|
||||
})
|
||||
|
||||
f := requireOne(t, check(t, root, options{}), checkMenuSort)
|
||||
if f.Severity != "ERROR" || !strings.Contains(f.Message, "900") {
|
||||
t.Errorf("finding = %+v", f)
|
||||
}
|
||||
if f.Line != 8 {
|
||||
t.Errorf("line = %d, want the overflowing element", f.Line)
|
||||
}
|
||||
}
|
||||
|
||||
// The SysMenu of the service layer shares the name and has nothing to do with
|
||||
// the column. Reporting it would be the false positive that gets the tool
|
||||
// switched off.
|
||||
func TestMenuSortIgnoresSameNamedTypesFromOtherPackages(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"app/admin/service/sys_menu.go": `package service
|
||||
|
||||
type SysMenu struct{ Sort int }
|
||||
`,
|
||||
"app/admin/apis/sys_menu.go": `package apis
|
||||
|
||||
import "go-admin/app/admin/service"
|
||||
|
||||
func handler() interface{} { return service.SysMenu{Sort: 9000} }
|
||||
`,
|
||||
})
|
||||
if got := only(t, check(t, root, options{}), checkMenuSort); len(got) != 0 {
|
||||
t.Errorf("reported %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestConfigValueTruncationIsDetected(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seed() models.SysConfig {
|
||||
return models.SysConfig{ConfigKey: "k", ConfigValue: "` + strings.Repeat("a", 300) + `"}
|
||||
}
|
||||
`,
|
||||
})
|
||||
|
||||
f := requireOne(t, check(t, root, options{}), checkConfigValue)
|
||||
if f.Severity != "ERROR" || !strings.Contains(f.Message, "300 characters") {
|
||||
t.Errorf("finding = %+v", f)
|
||||
}
|
||||
}
|
||||
|
||||
// varchar(255) counts characters. Counting bytes would report a Chinese value
|
||||
// that fits perfectly well.
|
||||
func TestConfigValueCountsRunesNotBytes(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seed() models.SysConfig {
|
||||
return models.SysConfig{ConfigValue: "` + strings.Repeat("中", 200) + `"}
|
||||
}
|
||||
`,
|
||||
})
|
||||
if got := only(t, check(t, root, options{}), checkConfigValue); len(got) != 0 {
|
||||
t.Errorf("200 Chinese characters fit in varchar(255) but were reported: %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestMenuIDCollisionAcrossFilesIsDetectedThroughConstants(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version/1786700001000_crm.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
const crmMenuId = 9000
|
||||
|
||||
func seedCrm() models.SysMenu { return models.SysMenu{MenuId: crmMenuId, MenuName: "CRM"} }
|
||||
`,
|
||||
"cmd/migrate/migration/version/1786700002000_oms.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seedOms() models.SysMenu { return models.SysMenu{MenuId: 9000, MenuName: "OMS"} }
|
||||
`,
|
||||
})
|
||||
|
||||
f := requireOne(t, check(t, root, options{}), checkMenuIDConflict)
|
||||
if f.Severity != "ERROR" || !strings.Contains(f.Message, "9000") {
|
||||
t.Errorf("finding = %+v", f)
|
||||
}
|
||||
// The second site has to be printed too, or the report says a collision
|
||||
// happened without saying with what.
|
||||
if len(f.Related) != 1 || !strings.Contains(f.Related[0], "1786700002000_oms.go") {
|
||||
t.Errorf("related = %v", f.Related)
|
||||
}
|
||||
}
|
||||
|
||||
// The seeds write a menu and then refer to it again to attach permissions. That
|
||||
// is one menu, not two modules fighting over an id.
|
||||
func TestMenuIDRepeatedWithinOneFileIsNotACollision(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version/1786700001000_crm.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seed() []models.SysMenu {
|
||||
dir := models.SysMenu{MenuId: 9000, MenuName: "CRM"}
|
||||
again := models.SysMenu{MenuId: 9000, MenuName: "CRM"}
|
||||
return []models.SysMenu{dir, again}
|
||||
}
|
||||
`,
|
||||
})
|
||||
if got := only(t, check(t, root, options{}), checkMenuIDConflict); len(got) != 0 {
|
||||
t.Errorf("reported %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func TestContractImportBoundaryIsDetected(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"app/admin/service/dto/log.go": `package dto
|
||||
|
||||
const OperaStatusEnabel = "1"
|
||||
`,
|
||||
"common/middleware/logger.go": `package middleware
|
||||
|
||||
import "go-admin/app/admin/service/dto"
|
||||
|
||||
func status() string { return dto.OperaStatusEnabel }
|
||||
`,
|
||||
})
|
||||
|
||||
f := requireOne(t, check(t, root, options{}), checkImportBoundary)
|
||||
if f.Severity != "ERROR" {
|
||||
t.Errorf("severity = %s", f.Severity)
|
||||
}
|
||||
if !strings.Contains(f.Message, "go-admin/app/admin/service/dto") {
|
||||
t.Errorf("message = %s", f.Message)
|
||||
}
|
||||
if f.File != "common/middleware/logger.go" || f.Line != 3 {
|
||||
t.Errorf("position = %s:%d", f.File, f.Line)
|
||||
}
|
||||
}
|
||||
|
||||
// A fork that drops app/admin should be able to run the tests too, so a
|
||||
// test-only import is the same violation.
|
||||
func TestContractImportBoundaryCoversTestFiles(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
|
||||
"common/actions/permission_test.go": `package actions
|
||||
|
||||
import "go-admin/app/admin/models"
|
||||
|
||||
var _ = models.SysUser{}
|
||||
`,
|
||||
})
|
||||
if got := only(t, check(t, root, options{}), checkImportBoundary); len(got) != 1 {
|
||||
t.Errorf("findings = %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestContractImportBoundaryAllowsAppToImportCommon(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"common/models/by.go": "package models\n\ntype ControlBy struct{}\n",
|
||||
"app/demo/models/product.go": `package models
|
||||
|
||||
import "go-admin/common/models"
|
||||
|
||||
type Product struct{ models.ControlBy }
|
||||
`,
|
||||
})
|
||||
if got := only(t, check(t, root, options{}), checkImportBoundary); len(got) != 0 {
|
||||
t.Errorf("the dependency direction that is allowed was reported: %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
func menuNameFixture(t *testing.T, menuName, componentName string) (string, string) {
|
||||
t.Helper()
|
||||
root := fixture(t, map[string]string{
|
||||
"cmd/migrate/migration/models/models.go": frozenModelsPkg,
|
||||
"cmd/migrate/migration/version/1786700001000_seed.go": `package version
|
||||
|
||||
import "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
func seed() models.SysMenu {
|
||||
return models.SysMenu{MenuId: 9001, MenuName: "` + menuName + `", MenuType: "C", Component: "/demo/product/index"}
|
||||
}
|
||||
`,
|
||||
})
|
||||
ui := t.TempDir()
|
||||
path := filepath.Join(ui, "views", "demo", "product", "index.vue")
|
||||
if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
vue := "<script setup>\ndefineOptions({ name: '" + componentName + "' })\n</script>\n<template><div/></template>\n"
|
||||
if err := os.WriteFile(path, []byte(vue), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return root, ui
|
||||
}
|
||||
|
||||
func TestMenuNameMismatchIsDetectedAsAWarning(t *testing.T) {
|
||||
root, ui := menuNameFixture(t, "DemoProduct", "Product")
|
||||
|
||||
f := requireOne(t, check(t, root, options{UIDir: ui}), checkMenuName)
|
||||
if f.Severity != "WARN" {
|
||||
t.Errorf("severity = %s; this check must not decide an exit code yet", f.Severity)
|
||||
}
|
||||
if !strings.Contains(f.Message, "DemoProduct") || !strings.Contains(f.Message, "Product") {
|
||||
t.Errorf("message = %s", f.Message)
|
||||
}
|
||||
// Acceptance 14c.
|
||||
if !strings.Contains(f.Message, "heuristic") || !strings.Contains(f.Message, "false positives are possible") {
|
||||
t.Errorf("the warning must say it is a heuristic:\n%s", f.Message)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMenuNameMatchIsSilent(t *testing.T) {
|
||||
root, ui := menuNameFixture(t, "DemoProduct", "DemoProduct")
|
||||
if got := only(t, check(t, root, options{UIDir: ui}), checkMenuName); len(got) != 0 {
|
||||
t.Errorf("reported %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Without the frontend repository there is nothing to compare against, and the
|
||||
// check has to disappear rather than guess.
|
||||
func TestMenuNameIsSkippedWithoutTheUIDirectory(t *testing.T) {
|
||||
root, _ := menuNameFixture(t, "DemoProduct", "Product")
|
||||
if got := only(t, check(t, root, options{}), checkMenuName); len(got) != 0 {
|
||||
t.Errorf("reported %v without -ui-dir", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A component the frontend does not carry is the "app not installed" case F2
|
||||
// handles with a placeholder; this check has nothing to say about it.
|
||||
func TestMenuNameIsSilentWhenTheComponentIsMissing(t *testing.T) {
|
||||
root, _ := menuNameFixture(t, "DemoProduct", "Product")
|
||||
empty := t.TempDir()
|
||||
if got := only(t, check(t, root, options{UIDir: empty}), checkMenuName); len(got) != 0 {
|
||||
t.Errorf("reported %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestComponentNameParsesBothVueStyles(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
src string
|
||||
want string
|
||||
}{
|
||||
{"script setup", "<script setup>\ndefineOptions({ name: 'Product' })\n</script>", "Product"},
|
||||
{"options api", "<script>\nexport default {\n name: 'Product',\n data() {}\n}\n</script>", "Product"},
|
||||
{"defineComponent", "<script>\nexport default defineComponent({\n name: 'Product'\n})\n</script>", "Product"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, ok := componentName(tc.src)
|
||||
if !ok || got != tc.want {
|
||||
t.Errorf("componentName = %q, %v", got, ok)
|
||||
}
|
||||
})
|
||||
}
|
||||
if _, ok := componentName("<script setup>\nconst a = 1\n</script>"); ok {
|
||||
t.Error("a component with no declared name must not be compared")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// Severity decides whether a finding stops CI.
|
||||
//
|
||||
// Two levels and no more. A third would immediately be used to park findings
|
||||
// nobody intends to fix, and the point of this tool is that everything it
|
||||
// reports is something that fails without saying so.
|
||||
type Severity int
|
||||
|
||||
const (
|
||||
// Warn prints and does not affect the exit code.
|
||||
Warn Severity = iota
|
||||
// Error prints and makes the run fail.
|
||||
Error
|
||||
)
|
||||
|
||||
func (s Severity) String() string {
|
||||
if s == Error {
|
||||
return "ERROR"
|
||||
}
|
||||
return "WARN"
|
||||
}
|
||||
|
||||
// Finding is one problem, located precisely enough to open the file at it.
|
||||
type Finding struct {
|
||||
Check string `json:"check"`
|
||||
Severity string `json:"severity"`
|
||||
File string `json:"file"`
|
||||
Line int `json:"line"`
|
||||
Col int `json:"col"`
|
||||
Message string `json:"message"`
|
||||
Related []string `json:"related,omitempty"`
|
||||
|
||||
severity Severity
|
||||
}
|
||||
|
||||
func (f Finding) String() string {
|
||||
s := fmt.Sprintf("%s:%d:%d: [%s] %s: %s", f.File, f.Line, f.Col, f.Severity, f.Check, f.Message)
|
||||
for _, r := range f.Related {
|
||||
s += "\n " + r
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// sortFindings orders by file, then position, then check, so two runs over the
|
||||
// same tree print the same thing and a diff of the output is meaningful.
|
||||
func sortFindings(fs []Finding) {
|
||||
sort.Slice(fs, func(i, j int) bool {
|
||||
a, b := fs[i], fs[j]
|
||||
if a.File != b.File {
|
||||
return a.File < b.File
|
||||
}
|
||||
if a.Line != b.Line {
|
||||
return a.Line < b.Line
|
||||
}
|
||||
if a.Col != b.Col {
|
||||
return a.Col < b.Col
|
||||
}
|
||||
if a.Check != b.Check {
|
||||
return a.Check < b.Check
|
||||
}
|
||||
return a.Message < b.Message
|
||||
})
|
||||
}
|
||||
|
||||
func hasError(fs []Finding) bool {
|
||||
for _, f := range fs {
|
||||
if f.severity == Error {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
// Command checksilent reports the failures in this repository that do not
|
||||
// announce themselves: no error, no log line, behaviour quietly wrong.
|
||||
//
|
||||
// Six checks, five of them ERROR and one WARN. An ERROR fails the run; a WARN
|
||||
// prints and does not. The split is not about how bad the consequence is - all
|
||||
// six are bad - but about how certain the detection is. Everything reported as
|
||||
// an ERROR is decided from this repository's own syntax. The one WARN compares
|
||||
// against a second repository through a regular expression, and a check that
|
||||
// can be wrong must not be able to stop a build, or the first response to it
|
||||
// will be an ignore comment.
|
||||
//
|
||||
// Usage:
|
||||
//
|
||||
// go run ./tools/checksilent
|
||||
// go run ./tools/checksilent -ui-dir ../go-admin-ui/src
|
||||
// go run ./tools/checksilent -json
|
||||
//
|
||||
// Or through the Makefile, which is what CI runs:
|
||||
//
|
||||
// make checksilent
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func main() {
|
||||
var (
|
||||
root = flag.String("root", ".", "repository root to scan")
|
||||
uiDir = flag.String("ui-dir", "", "go-admin-ui src directory; enables the menu-name check, which is skipped without it")
|
||||
asJSON = flag.Bool("json", false, "print findings as JSON")
|
||||
)
|
||||
flag.Parse()
|
||||
|
||||
code, err := run(os.Stdout, *root, options{UIDir: *uiDir}, *asJSON)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "checksilent:", err)
|
||||
os.Exit(2)
|
||||
}
|
||||
os.Exit(code)
|
||||
}
|
||||
|
||||
// run returns the process exit code: 0 when nothing or only warnings were
|
||||
// found, 1 when at least one error was.
|
||||
func run(w io.Writer, root string, opt options, asJSON bool) (int, error) {
|
||||
s, err := load(root)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
findings, err := runChecks(s, opt)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if asJSON {
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetIndent("", " ")
|
||||
if err = enc.Encode(findings); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
} else {
|
||||
for _, f := range findings {
|
||||
fmt.Fprintln(w, f)
|
||||
}
|
||||
printSummary(w, findings, opt, s)
|
||||
}
|
||||
|
||||
if hasError(findings) {
|
||||
return 1, nil
|
||||
}
|
||||
return 0, nil
|
||||
}
|
||||
|
||||
func printSummary(w io.Writer, findings []Finding, opt options, s *snapshot) {
|
||||
var errors, warnings int
|
||||
for _, f := range findings {
|
||||
if f.severity == Error {
|
||||
errors++
|
||||
} else {
|
||||
warnings++
|
||||
}
|
||||
}
|
||||
if len(findings) > 0 {
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
fmt.Fprintf(w, "checksilent: %d error(s), %d warning(s)\n", errors, warnings)
|
||||
if warnings > 0 {
|
||||
fmt.Fprintln(w, "Warnings do not affect the exit code.")
|
||||
}
|
||||
if opt.UIDir == "" {
|
||||
fmt.Fprintf(w, "The %s check was skipped: pass -ui-dir <go-admin-ui>/src to run it.\n", checkMenuName)
|
||||
}
|
||||
// Said out loud so nobody reads a clean run as "the boundary holds
|
||||
// everywhere it was declared". core/ is a separate module and has no
|
||||
// directory here, so this repository's copy of the check cannot cover it.
|
||||
if scanned, absent := ScannedContractRoots(s); len(absent) > 0 {
|
||||
fmt.Fprintf(w, "The %s check covered %s; %s does not exist here and was not scanned.\n",
|
||||
checkImportBoundary, strings.Join(scanned, ", "), strings.Join(absent, ", "))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Acceptance 14b, second half: the WARN check on its own leaves the exit code
|
||||
// at zero. If it did not, the first person it misjudged would silence it, and a
|
||||
// silenced check is worse than none - it looks like coverage.
|
||||
func TestOnlyWarningsExitZero(t *testing.T) {
|
||||
root, ui := menuNameFixture(t, "DemoProduct", "Product")
|
||||
|
||||
var buf bytes.Buffer
|
||||
code, err := run(&buf, root, options{UIDir: ui}, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code != 0 {
|
||||
t.Errorf("exit code = %d, want 0:\n%s", code, buf.String())
|
||||
}
|
||||
out := buf.String()
|
||||
if !strings.Contains(out, "[WARN]") {
|
||||
t.Errorf("the warning was not printed:\n%s", out)
|
||||
}
|
||||
if !strings.Contains(out, "0 error(s), 1 warning(s)") {
|
||||
t.Errorf("summary = %s", out)
|
||||
}
|
||||
if !strings.Contains(out, "Warnings do not affect the exit code.") {
|
||||
t.Errorf("output must say warnings are not fatal:\n%s", out)
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 14b, first half: any of the other five fails the run.
|
||||
func TestAnyErrorExitsNonZero(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
|
||||
"common/middleware/auth.go": `package middleware
|
||||
|
||||
import "go-admin/app/admin/models"
|
||||
|
||||
var _ = models.SysUser{}
|
||||
`,
|
||||
})
|
||||
|
||||
var buf bytes.Buffer
|
||||
code, err := run(&buf, root, options{}, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code != 1 {
|
||||
t.Errorf("exit code = %d, want 1:\n%s", code, buf.String())
|
||||
}
|
||||
if !strings.Contains(buf.String(), "[ERROR]") {
|
||||
t.Errorf("output = %s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCleanTreeExitsZero(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"common/models/by.go": "package models\n\ntype ControlBy struct{}\n",
|
||||
})
|
||||
var buf bytes.Buffer
|
||||
code, err := run(&buf, root, options{}, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code != 0 {
|
||||
t.Errorf("exit code = %d, want 0:\n%s", code, buf.String())
|
||||
}
|
||||
if !strings.Contains(buf.String(), "0 error(s), 0 warning(s)") {
|
||||
t.Errorf("output = %s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
// Every message has to name a file and a line, or the report is a puzzle rather
|
||||
// than a finding.
|
||||
func TestTextOutputLocatesEveryFinding(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
|
||||
"common/middleware/auth.go": `package middleware
|
||||
|
||||
import "go-admin/app/admin/models"
|
||||
|
||||
var _ = models.SysUser{}
|
||||
`,
|
||||
})
|
||||
var buf bytes.Buffer
|
||||
if _, err := run(&buf, root, options{}, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
line := strings.SplitN(buf.String(), "\n", 2)[0]
|
||||
if !strings.HasPrefix(line, "common/middleware/auth.go:3:") {
|
||||
t.Errorf("first line does not locate the finding: %q", line)
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONOutput(t *testing.T) {
|
||||
root := fixture(t, map[string]string{
|
||||
"app/admin/models/user.go": "package models\n\ntype SysUser struct{}\n",
|
||||
"common/middleware/auth.go": `package middleware
|
||||
|
||||
import "go-admin/app/admin/models"
|
||||
|
||||
var _ = models.SysUser{}
|
||||
`,
|
||||
})
|
||||
var buf bytes.Buffer
|
||||
code, err := run(&buf, root, options{}, true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code != 1 {
|
||||
t.Errorf("exit code = %d", code)
|
||||
}
|
||||
var findings []Finding
|
||||
if err = json.Unmarshal(buf.Bytes(), &findings); err != nil {
|
||||
t.Fatalf("output is not JSON: %v\n%s", err, buf.String())
|
||||
}
|
||||
if len(findings) != 1 || findings[0].Check != checkImportBoundary || findings[0].Severity != "ERROR" {
|
||||
t.Errorf("findings = %+v", findings)
|
||||
}
|
||||
}
|
||||
|
||||
// The tool runs on this repository in CI, so it has to be clean here. This also
|
||||
// covers acceptance 13b: nothing under common/ imports app/ any more.
|
||||
func TestThisRepositoryIsClean(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
code, err := run(&buf, "../..", options{}, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if code != 0 {
|
||||
t.Errorf("checksilent reports problems in this repository:\n%s", buf.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// heuristicNote is appended to every menu-name finding.
|
||||
//
|
||||
// Required by the acceptance criteria, and for a reason worth restating: this
|
||||
// is the one check that compares two repositories through a regular expression,
|
||||
// so it will occasionally be wrong. Reporting it as a certainty is how a tool
|
||||
// gets a reputation and then an ignore comment on every line.
|
||||
const heuristicNote = "\n (heuristic: matched across repositories by regular expression - false positives are possible, verify before changing anything)"
|
||||
|
||||
var (
|
||||
// Vue 3 <script setup>, which is the prevailing style in go-admin-ui.
|
||||
defineOptionsName = regexp.MustCompile(`defineOptions\(\s*\{[^}]*?name:\s*['"` + "`" + `]([^'"` + "`" + `]+)['"` + "`" + `]`)
|
||||
// Options API, still present in older views.
|
||||
exportDefaultName = regexp.MustCompile(`export default\s*(?:defineComponent\(\s*)?\{[^}]*?name:\s*['"` + "`" + `]([^'"` + "`" + `]+)['"` + "`" + `]`)
|
||||
)
|
||||
|
||||
// checkMenuNames compares the menu_name a migration seeds against the name the
|
||||
// component declares.
|
||||
//
|
||||
// keep-alive caches by component name, and the page cache is configured by menu
|
||||
// name. When they disagree nothing breaks and nothing is logged - the page just
|
||||
// stops being cached, or the wrong page is evicted, and it looks like a
|
||||
// performance quirk.
|
||||
//
|
||||
// WARN, not ERROR, and deliberately so. The two sides are in different
|
||||
// repositories and different languages, so this is a regular expression against
|
||||
// a .vue file, not a parse. It is scheduled to become an ERROR after two release
|
||||
// cycles with no false positive; until then it must not decide an exit code.
|
||||
func checkMenuNames(s *snapshot, uiDir string) ([]Finding, error) {
|
||||
info, err := os.Stat(uiDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("ui dir %s: %w", uiDir, err)
|
||||
}
|
||||
if !info.IsDir() {
|
||||
return nil, fmt.Errorf("ui dir %s is not a directory", uiDir)
|
||||
}
|
||||
|
||||
var out []Finding
|
||||
for _, sf := range s.Files {
|
||||
forEachStructLiteral(sf, func(lit structLiteral) {
|
||||
if !s.isMenuModel(lit) {
|
||||
return
|
||||
}
|
||||
menuType, _ := stringField(sf, lit, "MenuType")
|
||||
// A directory has no page of its own; its component is the layout.
|
||||
if menuType == "M" || menuType == "F" {
|
||||
return
|
||||
}
|
||||
component, ok := stringField(sf, lit, "Component")
|
||||
if !ok || component == "" || component == "Layout" {
|
||||
return
|
||||
}
|
||||
menuName, ok := stringField(sf, lit, "MenuName")
|
||||
if !ok || menuName == "" {
|
||||
return
|
||||
}
|
||||
|
||||
path := componentFile(uiDir, component)
|
||||
if path == "" {
|
||||
return
|
||||
}
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
// A component the frontend does not carry is F2's business -
|
||||
// the placeholder that says the app is not installed - not this
|
||||
// check's.
|
||||
return
|
||||
}
|
||||
declared, ok := componentName(string(b))
|
||||
if !ok {
|
||||
// No literal name at all: the build may derive one from the
|
||||
// file path. Nothing to compare, so nothing to say.
|
||||
return
|
||||
}
|
||||
if declared == menuName {
|
||||
return
|
||||
}
|
||||
expr, _ := field(lit.Lit, "MenuName")
|
||||
rel := path
|
||||
if r, err := filepath.Rel(uiDir, path); err == nil {
|
||||
rel = r
|
||||
}
|
||||
out = append(out, s.finding(Warn, checkMenuName, sf, expr,
|
||||
"menu_name %q does not match the component name %q declared in %s;\n"+
|
||||
" keep-alive caches by component name and the cache is configured by menu name, so the page silently stops being cached.%s",
|
||||
menuName, declared, rel, heuristicNote))
|
||||
})
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func stringField(sf *sourceFile, lit structLiteral, name string) (string, bool) {
|
||||
expr, ok := field(lit.Lit, name)
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
return stringValue(sf, expr)
|
||||
}
|
||||
|
||||
// componentFile maps a menu component path to a file in the frontend tree.
|
||||
//
|
||||
// Two roots, because F2 adds a second: anything under apps/ is an installed
|
||||
// app's view, everything else is a view of the main repository.
|
||||
func componentFile(uiDir, component string) string {
|
||||
c := strings.TrimPrefix(filepath.ToSlash(component), "/")
|
||||
if c == "" {
|
||||
return ""
|
||||
}
|
||||
if strings.HasPrefix(c, "apps/") {
|
||||
return filepath.Join(uiDir, filepath.FromSlash(c)+".vue")
|
||||
}
|
||||
return filepath.Join(uiDir, "views", filepath.FromSlash(c)+".vue")
|
||||
}
|
||||
|
||||
func componentName(src string) (string, bool) {
|
||||
for _, re := range []*regexp.Regexp{defineOptionsName, exportDefaultName} {
|
||||
if m := re.FindStringSubmatch(src); m != nil {
|
||||
return m[1], true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
@@ -0,0 +1,203 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// skippedDirs are never walked. Generated and vendored trees would produce
|
||||
// findings nobody can act on, and node_modules would make the walk the slowest
|
||||
// part of CI.
|
||||
var skippedDirs = map[string]bool{
|
||||
".git": true,
|
||||
".idea": true,
|
||||
"node_modules": true,
|
||||
"vendor": true,
|
||||
"testdata": true,
|
||||
"dist": true,
|
||||
"temp": true,
|
||||
}
|
||||
|
||||
// sourceFile is one parsed Go file plus the mapping from the local names it
|
||||
// uses for packages to the paths those names stand for.
|
||||
//
|
||||
// Resolving through the file's own import block, rather than matching on the
|
||||
// identifier, is what keeps service.SysMenu apart from models.SysMenu.
|
||||
type sourceFile struct {
|
||||
// Path is relative to the scanned root and is what gets printed.
|
||||
Path string
|
||||
Pkg string // import path of the package this file belongs to
|
||||
Syntax *ast.File
|
||||
imports map[string]string // local name -> import path
|
||||
consts map[string]int64 // package-level integer constants, filled per package
|
||||
}
|
||||
|
||||
// snapshot is every Go file under the root, parsed once and shared by all the
|
||||
// checks.
|
||||
type snapshot struct {
|
||||
Root string
|
||||
ModulePath string
|
||||
Fset *token.FileSet
|
||||
Files []*sourceFile
|
||||
}
|
||||
|
||||
// load parses every Go file under root.
|
||||
//
|
||||
// Type checking is deliberately not done. Every rule here is about a literal
|
||||
// written into a seed or an import that should not be there, and both are in
|
||||
// the syntax; a type checker would drag in a resolvable build of the whole
|
||||
// module, which is exactly what a check meant to run on a half-broken tree
|
||||
// cannot depend on.
|
||||
func load(root string) (*snapshot, error) {
|
||||
modulePath, err := readModulePath(root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
s := &snapshot{Root: root, ModulePath: modulePath, Fset: token.NewFileSet()}
|
||||
err = filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if info.IsDir() {
|
||||
if path != root && skippedDirs[info.Name()] {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if !strings.HasSuffix(path, ".go") {
|
||||
return nil
|
||||
}
|
||||
rel, err := filepath.Rel(root, path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel = filepath.ToSlash(rel)
|
||||
|
||||
f, err := parser.ParseFile(s.Fset, path, nil, 0)
|
||||
if err != nil {
|
||||
// A file that does not parse is someone's work in progress, not a
|
||||
// silent failure. Reporting it here would only repeat what the
|
||||
// compiler already says louder.
|
||||
return nil
|
||||
}
|
||||
dir := filepath.ToSlash(filepath.Dir(rel))
|
||||
pkg := modulePath
|
||||
if dir != "." {
|
||||
pkg = modulePath + "/" + dir
|
||||
}
|
||||
s.Files = append(s.Files, &sourceFile{
|
||||
Path: rel,
|
||||
Pkg: pkg,
|
||||
Syntax: f,
|
||||
imports: fileImports(f),
|
||||
})
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.resolveConstants()
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func readModulePath(root string) (string, error) {
|
||||
b, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("read go.mod: %w", err)
|
||||
}
|
||||
for _, line := range strings.Split(string(b), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if rest, ok := strings.CutPrefix(line, "module "); ok {
|
||||
return strings.TrimSpace(rest), nil
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("%s has no module line", filepath.Join(root, "go.mod"))
|
||||
}
|
||||
|
||||
func fileImports(f *ast.File) map[string]string {
|
||||
out := make(map[string]string, len(f.Imports))
|
||||
for _, spec := range f.Imports {
|
||||
path, err := strconv.Unquote(spec.Path.Value)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
name := path[strings.LastIndex(path, "/")+1:]
|
||||
if spec.Name != nil {
|
||||
name = spec.Name.Name
|
||||
}
|
||||
out[name] = path
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// resolveConstants collects package-level integer constants per directory, so a
|
||||
// seed written as MenuId: demoMenuId can be compared against one written as
|
||||
// MenuId: 9000. Without this the menu-id check would see the well-written code
|
||||
// and miss exactly the collisions it exists to find.
|
||||
func (s *snapshot) resolveConstants() {
|
||||
byPkg := map[string]map[string]int64{}
|
||||
for _, sf := range s.Files {
|
||||
consts, ok := byPkg[sf.Pkg]
|
||||
if !ok {
|
||||
consts = map[string]int64{}
|
||||
byPkg[sf.Pkg] = consts
|
||||
}
|
||||
for _, decl := range sf.Syntax.Decls {
|
||||
gen, ok := decl.(*ast.GenDecl)
|
||||
if !ok || gen.Tok != token.CONST {
|
||||
continue
|
||||
}
|
||||
for _, spec := range gen.Specs {
|
||||
vs, ok := spec.(*ast.ValueSpec)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
for i, name := range vs.Names {
|
||||
if i >= len(vs.Values) {
|
||||
continue
|
||||
}
|
||||
if v, ok := intLiteral(vs.Values[i]); ok {
|
||||
consts[name.Name] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, sf := range s.Files {
|
||||
sf.consts = byPkg[sf.Pkg]
|
||||
}
|
||||
}
|
||||
|
||||
// Pos turns a token position into the file/line/col a Finding carries.
|
||||
func (s *snapshot) Pos(sf *sourceFile, p token.Pos) (string, int, int) {
|
||||
pos := s.Fset.Position(p)
|
||||
return sf.Path, pos.Line, pos.Column
|
||||
}
|
||||
|
||||
// Imports reports whether the file imports path.
|
||||
func (sf *sourceFile) Imports(path string) bool {
|
||||
for _, p := range sf.imports {
|
||||
if p == path {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ImportSpec returns the import declaration for path, for reporting a position
|
||||
// on the import line itself.
|
||||
func (sf *sourceFile) ImportSpec(path string) *ast.ImportSpec {
|
||||
for _, spec := range sf.Syntax.Imports {
|
||||
if p, err := strconv.Unquote(spec.Path.Value); err == nil && p == path {
|
||||
return spec
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
Reference in New Issue
Block a user