mirror of
https://github.com/go-admin-team/go-admin.git
synced 2026-09-24 19:17:43 +00:00
Compare commits
149
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
047b23872c | ||
|
|
84bd87dcc9 | ||
|
|
0e7a13aeba | ||
|
|
85d50da494 | ||
|
|
1d9def4314 | ||
|
|
ed9bbd01e2 | ||
|
|
523d6a3649 | ||
|
|
6326962862 | ||
|
|
cd7c8375c0 | ||
|
|
63bcc912ef | ||
|
|
adcdd2edcd | ||
|
|
29406f839e | ||
|
|
a43133ab7b | ||
|
|
7002cd4065 | ||
|
|
8faa8d2aed | ||
|
|
705427178d | ||
|
|
8f10d202e6 | ||
|
|
5648bd1dcf | ||
|
|
a442eadb96 | ||
|
|
f3b67e9abc | ||
|
|
4e51f56623 | ||
|
|
7e4e17bbcf | ||
|
|
799e892a68 | ||
|
|
0e2adb3165 | ||
|
|
249e044ded | ||
|
|
d6309c75be | ||
|
|
211ae85a4e | ||
|
|
3c3d94ca76 | ||
|
|
6138d2d74c | ||
|
|
d5de79f75b | ||
|
|
46e793972c | ||
|
|
46f4092b43 | ||
|
|
196195357b | ||
|
|
c579c5f84c | ||
|
|
241c27358b | ||
|
|
aa3c9866cb | ||
|
|
2ac01ea584 | ||
|
|
7f9cc1e435 | ||
|
|
4fb0529d2d | ||
|
|
8ee4141af6 | ||
|
|
36f2549172 | ||
|
|
dff0e64f51 | ||
|
|
0b78bc1e2e | ||
|
|
94163f9afb | ||
|
|
4510b06959 | ||
|
|
750c7c744e | ||
|
|
d52dca1cb6 | ||
|
|
71413a4248 | ||
|
|
4fede43254 | ||
|
|
0a629e2f3f | ||
|
|
37065fb089 | ||
|
|
6966f14dd4 | ||
|
|
22716e90c1 | ||
|
|
73cce7fc2f | ||
|
|
b59c7f0d46 | ||
|
|
d3a44a2a6b | ||
|
|
5c3c3907d5 | ||
|
|
f2215e132e | ||
|
|
a69afab34f | ||
|
|
e0132db1b9 | ||
|
|
7c3f55a873 | ||
|
|
f7c0247394 | ||
|
|
ac23556029 | ||
|
|
f64115e03a | ||
|
|
a2524c31bf | ||
|
|
71d6211c61 | ||
|
|
c67760bc39 | ||
|
|
d3e7f46a46 | ||
|
|
55866682ae | ||
|
|
550e95ff43 | ||
|
|
060b6cfd64 | ||
|
|
89a4738394 | ||
|
|
d8529289cf | ||
|
|
4a8f97b1ee | ||
|
|
d54ac844ef | ||
|
|
379fba515f | ||
|
|
58105cb478 | ||
|
|
47af6f4306 | ||
|
|
7d29c9953a | ||
|
|
0729624c2f | ||
|
|
b3a740ab2a | ||
|
|
adf617f5d0 | ||
|
|
049a20cd04 | ||
|
|
be3c4452e3 | ||
|
|
5b01c9ada8 | ||
|
|
ffd82a6a10 | ||
|
|
dd8d89a990 | ||
|
|
2e5b23565e | ||
|
|
f4e3f04d30 | ||
|
|
954ebdc9eb | ||
|
|
2840010dfd | ||
|
|
b147d9b833 | ||
|
|
b3ecb81614 | ||
|
|
ce4581bb99 | ||
|
|
f406ca0160 | ||
|
|
39ea1f6aef | ||
|
|
b2053f507a | ||
|
|
9520117914 | ||
|
|
7a5fc7d440 | ||
|
|
bd5e83d464 | ||
|
|
63dd40a8d7 | ||
|
|
32bd88504d | ||
|
|
d70818a9db | ||
|
|
9d4a425fc0 | ||
|
|
4d6456a588 | ||
|
|
07ff92aa55 | ||
|
|
4156387eb9 | ||
|
|
34773a0a81 | ||
|
|
36a018400b | ||
|
|
7bb02c5f1d | ||
|
|
15fb128236 | ||
|
|
3581e060ec | ||
|
|
0604a29596 | ||
|
|
d8a2958797 | ||
|
|
ab28fa7bed | ||
|
|
e88d751039 | ||
|
|
b836945eea | ||
|
|
d7a8e66753 | ||
|
|
68780a845c | ||
|
|
487dc94a2e | ||
|
|
016e977776 | ||
|
|
dcfe512204 | ||
|
|
fe6ebfd47c | ||
|
|
eba5fba3da | ||
|
|
b7e9a79225 | ||
|
|
deffb19fd8 | ||
|
|
c858b322bd | ||
|
|
1b5b52f0f1 | ||
|
|
ecb31a158b | ||
|
|
1aecc140dc | ||
|
|
e464a4aedd | ||
|
|
595c4a6be5 | ||
|
|
d115c5299c | ||
|
|
9bd542bb59 | ||
|
|
0fa015b6d0 | ||
|
|
ec7d838ebd | ||
|
|
26e116c16c | ||
|
|
90d98893f5 | ||
|
|
10f162bf5d | ||
|
|
19909746f5 | ||
|
|
205febdb8a | ||
|
|
5aec4ba32b | ||
|
|
8f1ea50dfe | ||
|
|
1483ca401d | ||
|
|
ed74623a73 | ||
|
|
1bc2e22833 | ||
|
|
cd8edfa5d4 | ||
|
|
dcc2c8e175 | ||
|
|
d991a285ba |
@@ -49,8 +49,18 @@ model、dto、router 三个文件,完整写法照抄 `app/demo/` 的结构。
|
||||
### 4. 写菜单、接口与权限种子数据
|
||||
|
||||
这一步最容易被漏掉——代码能编译、接口能测通,但界面上看不到菜单、点了按钮说
|
||||
没权限,往往就是漏了这一步。**完整参照 `cmd/migrate/migration/version/1786700001000_demo_menu.go`**
|
||||
——那是可运行、幂等(用 `upsert`,重复跑不会报错)的真实例子,逐字照抄结构,只换 ID 和业务字段。
|
||||
没权限,往往就是漏了这一步。结构参照 `cmd/migrate/migration/version/1786700001000_demo_menu.go`
|
||||
——它是可运行、幂等(用 `upsert`,重复跑不会报错)的真实例子。
|
||||
|
||||
:::danger
|
||||
**但不要照抄它的 import。** 那个文件用的是 `cmd/migrate/migration/models`,
|
||||
只因为它的版本号排在软删除转换(`1786700003000`)之前才是安全的。
|
||||
|
||||
**你新写的迁移版本号在转换之后,必须改用 `app/` 下的运行时模型**
|
||||
(`app/admin/models.SysApi`、`SysMenu`),否则第一条 insert 就会
|
||||
`NOT NULL constraint failed: sys_api.deleted_at`。
|
||||
`TestPostConversionMigrationsAvoidFrozenSeedModels` 会拦住这个错误。
|
||||
:::
|
||||
|
||||
一个模块要在界面上可用,需要四类数据,缺一样都不行:
|
||||
|
||||
|
||||
@@ -1,10 +1,42 @@
|
||||
name: Build
|
||||
|
||||
# Documentation-only changes, and changes confined to the Kubernetes
|
||||
# manifests, skip this workflow entirely.
|
||||
#
|
||||
# A push to master here does not just build - it pushes an image, runs the
|
||||
# migrations and restarts the demo container, so the site takes a short outage.
|
||||
# Paying that for a README edit is waste at best; at worst a deploy fails for a
|
||||
# reason unrelated to anything in the change. Code coverage is unaffected,
|
||||
# because go.yml still builds every push and pull request.
|
||||
#
|
||||
# scripts/k8s holds deploy.yml, storage.yml and prerun.sh, and the deploy below
|
||||
# reads none of them - it is an ssh into one host that runs docker, building the
|
||||
# Dockerfile at the repository root. Those manifests are for people deploying to
|
||||
# a cluster of their own. The pattern is scripts/k8s/** rather than scripts/**
|
||||
# because scripts/Dockerfile is a build input: go.yml builds the release image
|
||||
# from it on a tag.
|
||||
#
|
||||
# A file outside these patterns still runs the workflow even when the rest of
|
||||
# the change is ignorable: paths-ignore skips only when every changed path
|
||||
# matches. Editing this file is one such case, on purpose - a deploy script
|
||||
# that is never exercised by the change that broke it is worse than an outage.
|
||||
on:
|
||||
push:
|
||||
branches: [ master ]
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'docs/**'
|
||||
- 'LICENSE*'
|
||||
- '.github/ISSUE_TEMPLATE/**'
|
||||
- 'scripts/k8s/**'
|
||||
pull_request:
|
||||
branches: [ master ]
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'docs/**'
|
||||
- 'LICENSE*'
|
||||
- '.github/ISSUE_TEMPLATE/**'
|
||||
- 'scripts/k8s/**'
|
||||
|
||||
# One deploy at a time. Two merges seconds apart raced here: both runs did
|
||||
# docker rm -f then docker run, the second removed the container the first had
|
||||
@@ -98,7 +130,14 @@ jobs:
|
||||
if sudo docker ps -a --format '{{.Names}}' | grep -qx "$NAME"; then
|
||||
sudo docker rm -f "$PREV" >/dev/null 2>&1 || true
|
||||
sudo docker rename "$NAME" "$PREV"
|
||||
sudo docker stop "$PREV" >/dev/null
|
||||
# --timeout, because the default is 10 seconds and the process
|
||||
# spends drain + server + cleanup from extend.shutdown before it
|
||||
# exits - 8 seconds out of the box, and more for anyone who
|
||||
# configures a drain window. Past the deadline docker sends
|
||||
# SIGKILL and the cleanup callbacks are cut off part-way through.
|
||||
# checksilent's docker-stop-cuts-shutdown-short check compares
|
||||
# this number against config/settings.yml.
|
||||
sudo docker stop --timeout 30 "$PREV" >/dev/null
|
||||
fi
|
||||
|
||||
sudo docker run -d -p 8000:8000 \
|
||||
|
||||
@@ -15,6 +15,27 @@ jobs:
|
||||
build:
|
||||
name: Build
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
# The queue's ordering rule - consumers registered before the queue is
|
||||
# started - is invisible on the memory backend, which is the default and
|
||||
# therefore what every other test runs on: queue.Memory's Register starts a
|
||||
# consumer goroutine whatever the state. Only redis refuses a late
|
||||
# registration, so without a server here the tests that cover it would skip
|
||||
# and the suite would report success for a queue that accepts no consumers.
|
||||
services:
|
||||
redis:
|
||||
image: redis:7-alpine
|
||||
ports:
|
||||
- 6379:6379
|
||||
options: >-
|
||||
--health-cmd "redis-cli ping"
|
||||
--health-interval 5s
|
||||
--health-timeout 3s
|
||||
--health-retries 10
|
||||
|
||||
env:
|
||||
GO_ADMIN_TEST_REDIS_ADDR: 127.0.0.1:6379
|
||||
|
||||
steps:
|
||||
|
||||
- name: Set up Go 1.26
|
||||
@@ -28,9 +49,23 @@ jobs:
|
||||
|
||||
- name: Get dependencies
|
||||
run: go mod tidy
|
||||
|
||||
# go build does not compile _test.go, so building alone never ran a single
|
||||
# test. This is the only workflow that fires on every push and pull request,
|
||||
# which makes it the one place a test gate belongs.
|
||||
- name: Test
|
||||
run: make test
|
||||
|
||||
- name: Build
|
||||
run: make build
|
||||
|
||||
# Fails the build on the silent-failure classes listed in
|
||||
# tools/checksilent, one of which is the contract boundary: nothing under
|
||||
# common/ may import app/. A boundary that is only written down erodes; this
|
||||
# is what keeps it true.
|
||||
- name: Silent-failure checks
|
||||
run: make checksilent
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
|
||||
if: startsWith(github.ref, 'refs/tags/')
|
||||
|
||||
+11
@@ -6,6 +6,10 @@ main.exe
|
||||
*.exe
|
||||
go-admin
|
||||
go-admin.exe
|
||||
# `go build ./tools/checksilent` drops the binary here, next to the one for the
|
||||
# server. Anchored with a leading slash: unanchored, the same pattern matches
|
||||
# tools/checksilent/ as well and the tool's own source never gets committed.
|
||||
/checksilent
|
||||
temp/
|
||||
!temp
|
||||
vendor
|
||||
@@ -29,3 +33,10 @@ CLAUDE.md
|
||||
.claude/skills/*
|
||||
!.claude/skills/new-business-module/
|
||||
config/settings.local.dev.yml
|
||||
|
||||
# Go workspace files. They exist to point this module at a local checkout of
|
||||
# go-admin-core while the two are developed together, which is a private
|
||||
# arrangement between one machine's directories - committing one would break
|
||||
# the build for everyone else.
|
||||
go.work
|
||||
go.work.sum
|
||||
|
||||
@@ -117,6 +117,21 @@ func (SysPost) TableName() string { return "sys_post" }
|
||||
|
||||
`TableName()` 必须显式声明(GORM 配置了 `SingularTable`,不会自动推导复数)。
|
||||
|
||||
## 公共契约面
|
||||
|
||||
第三方应用(`app/` 下的业务模块)可以稳定依赖哪些包、路由与迁移怎么注册、
|
||||
哪些约束是硬的,见 `docs/contract.md`。
|
||||
|
||||
两条与主仓贡献者直接相关的:
|
||||
|
||||
- **`common/`、`core/` 不得 import `app/`** —— `make checksilent` 在 CI 里守着,违反即红。
|
||||
- **从 core 契约包声明出来的类型必须写成别名**(`type X = pkg.Y`,不是 `type X pkg.Y`)
|
||||
—— `contract-shim-alias` 检查守着。defined type 会丢掉整个方法集,
|
||||
而且**不一定在本仓编译失败**,理由见 `docs/contract.md` 末节。
|
||||
- **注册类 API(`AppRouters` / `sdk.Runtime.SetAppRouters` / `migration.ForApp`)
|
||||
必须在 `runStartupHooks()` 之前调用完** —— `init()` 是最省事的位置,
|
||||
但约束的是**顺序**,不是写在哪个函数里;晚到的注册会被丢弃并只记一条 ERROR。
|
||||
|
||||
## 路由注册
|
||||
|
||||
通过 `init()` 自注册,不在中心文件手工添加:
|
||||
@@ -179,7 +194,8 @@ go run -tags sqlite3 . server -c config/settings.sqlite.yml
|
||||
|
||||
## 数据库迁移
|
||||
|
||||
文件名前 13 位为时间戳版本号。**已执行过的迁移文件不可修改** ——
|
||||
文件名前 13 位为毫秒时间戳版本号,不合规的名字会在启动时 panic 并报出该文件名。
|
||||
**已执行过的迁移文件不可修改** ——
|
||||
`sys_migration` 表按版本号去重,改动不会重跑,只能新增一个迁移来修正。
|
||||
|
||||
放哪个目录取决于身份:
|
||||
@@ -193,6 +209,54 @@ go run -tags sqlite3 . server -c config/settings.sqlite.yml
|
||||
`git status` 看不到,PR 里也不会出现。两个目录的包名分别是 `version` 与
|
||||
`version_local`(后者与目录名不一致,因为标识符不能含连字符)。
|
||||
|
||||
### 写种子数据用哪个 models 包
|
||||
|
||||
`1786700003000` 之后新增的迁移,**种子数据要用 `app/` 下的运行时模型**
|
||||
(如 `app/admin/models.SysApi`、`SysMenu`),**不要用 `cmd/migrate/migration/models`**。
|
||||
|
||||
后者的 `ModelTime` 声明的是可空的 `gorm.DeletedAt`,这对它之前的迁移是对的(那正是
|
||||
当时列的形状),转换之后就不再成立,两个方向都会出问题:
|
||||
|
||||
- **写**:往 NOT NULL 列里塞 NULL,第一条 insert 就 `NOT NULL constraint failed`
|
||||
- **读**:GORM 拼 `WHERE deleted_at IS NULL`,而活跃行存的是 `0`,静默查不到——
|
||||
照抄 `demo_menu.go` 的授权段落会因此跳过授权,菜单建好、权限没授、迁移仍记为成功
|
||||
|
||||
干净库跑不出这个问题,今天所有用该包的迁移都排在转换之前。完整推导见
|
||||
`schema_coverage_test.go` 里 `TestPostConversionMigrationsAvoidFrozenSeedModels`
|
||||
的注释,那个测试也守着这条边界。
|
||||
|
||||
## 静默失败校验
|
||||
|
||||
`make checksilent` 逐条检查那些**不报错、不记日志、行为悄悄变得不对**的问题,
|
||||
CI 会跑,命中 ERROR 即失败。这里不写条数——写死的数字会悄悄过时,
|
||||
真正的清单是 `tools/checksilent/checks.go` 里 `runChecks` 跑的那几个:
|
||||
|
||||
| 检查 | 级别 | 静默后果 |
|
||||
|---|---|---|
|
||||
| `modeltime-mix` | ERROR | 两个 `ModelTime` 混用,整张表查不到数据 |
|
||||
| `menu-sort-overflow` | ERROR | 菜单 `sort` 超 127,MySQL tinyint 拒绝写入,迁移中断 |
|
||||
| `config-value-truncation` | ERROR | `sys_config.config_value` 超 255 字符被静默截断 |
|
||||
| `menu-id-collision` | ERROR | 两个模块硬编码同一菜单 ID,互相覆盖 |
|
||||
| `contract-import-boundary` | ERROR | 契约包 import `app/`,应用无法独立编译 |
|
||||
| `contract-shim-alias` | ERROR | 契约薄壳写成 defined type 而非别名,方法集丢失,本仓可能照常编译、第三方应用编译不过 |
|
||||
| `datascope-route-unguarded` | ERROR | handler 读调用方的数据权限,而注册它的路由组没装提供权限的中间件。取不到时拿到零值、走 fail-closed 分支,查询被塞进 `1 = 0`:接口对确实存在的行返回「查不到」,且只在 `enabledp: true` 的部署上出现 |
|
||||
| `shutdown-budget-overruns-grace` | ERROR / WARN | `settings.yml` 的 `extend.shutdown` 预算(含清单里的 `preStop`)放不进自带 k8s 清单的 `terminationGracePeriodSeconds`,SIGKILL 在清理回调跑到一半时到达 |
|
||||
| `docker-stop-cuts-shutdown-short` | ERROR / WARN | 停止容器的两条路径——脚本/工作流里的 `docker stop`,和 `docker-compose.yml` 的 `stop_grace_period`——没写或写得不够关闭预算用。两边默认都是 10 秒,而这个数字离命令很远,调大预算的人不会想起它 |
|
||||
| `menu-name-mismatch` | WARN | 菜单名与前端组件 `name` 不一致,keep-alive 缓存静默失效 |
|
||||
|
||||
两条关闭预算检查分两级,用的是同一条算术和同一个 5 秒边际:真的超限报 ERROR,
|
||||
放得进但余量不足 5 秒报 WARN。余量不足做 WARN 不做 ERROR,是因为那是个技术上
|
||||
跑得通的配置——**一条在正确配置下也会响的 ERROR,训练的是忽略它**。
|
||||
|
||||
最后一条要跨仓库比对,只能做正则启发式,因此是 WARN,**不影响退出码**,
|
||||
且默认跳过;要跑它得指定前端目录:
|
||||
|
||||
```bash
|
||||
make checksilent UI_DIR=../go-admin-ui/src
|
||||
```
|
||||
|
||||
升级门槛:连续 2 个发版周期零误报后转为 ERROR。
|
||||
|
||||
## 提交规范
|
||||
|
||||
格式 `type+emoji: 描述`:
|
||||
|
||||
@@ -15,7 +15,16 @@ build-sqlite:
|
||||
# make run
|
||||
run:
|
||||
# delete go-admin-api container
|
||||
@if [ $(shell docker ps -aq --filter name=go-admin --filter publish=8000) ]; then docker rm -f go-admin; fi
|
||||
#
|
||||
# stop then rm, rather than `rm -f`. The force flag kills a running
|
||||
# container with SIGKILL and no grace at all, so restarting locally cut
|
||||
# short every shutdown this application does - the drain window was never
|
||||
# once reached on a developer's machine. --timeout has to cover
|
||||
# extend.shutdown's drain + server + cleanup; checksilent's
|
||||
# docker-stop-cuts-shutdown-short check compares it against
|
||||
# config/settings.yml. On a container that has already stopped, stop is a
|
||||
# no-op and the removal is unchanged.
|
||||
@if [ $(shell docker ps -aq --filter name=go-admin --filter publish=8000) ]; then docker stop --timeout 30 go-admin && docker rm go-admin; fi
|
||||
|
||||
# 启动方法一 run go-admin-api container docker-compose 启动方式
|
||||
# 进入到项目根目录 执行 make run 命令
|
||||
@@ -37,9 +46,27 @@ stop:
|
||||
#@echo "go-admin stop success"
|
||||
|
||||
|
||||
#.PHONY: test
|
||||
#test:
|
||||
# go test -v ./... -cover
|
||||
# -race is worth the extra minute here: common/actions reuses model instances
|
||||
# across concurrent requests, so a Generate() that returns in place instead of
|
||||
# a copy leaks data between them - and that is invisible to a single-threaded
|
||||
# test run.
|
||||
.PHONY: test
|
||||
test:
|
||||
go test -race -cover ./...
|
||||
|
||||
# Reports the failures that do not announce themselves - see
|
||||
# tools/checksilent. Exits non-zero on an ERROR; the one WARN-level check
|
||||
# prints and does not fail the build.
|
||||
#
|
||||
# Pass UI_DIR to enable the cross-repository menu-name check, which is skipped
|
||||
# without it: make checksilent UI_DIR=../go-admin-ui/src
|
||||
.PHONY: checksilent
|
||||
checksilent:
|
||||
ifdef UI_DIR
|
||||
go run ./tools/checksilent -ui-dir $(UI_DIR)
|
||||
else
|
||||
go run ./tools/checksilent
|
||||
endif
|
||||
|
||||
#.PHONY: docker
|
||||
#docker:
|
||||
|
||||
+7
-7
@@ -3,11 +3,11 @@
|
||||
<img align="right" width="320" src="https://doc-image.zhangwj.com/img/go-admin.svg">
|
||||
|
||||
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin/releases)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
|
||||
[English](https://github.com/go-admin-team/go-admin/blob/master/README.md) | 简体中文
|
||||
[English](https://github.com/go-admin-team/go-admin/blob/master/README.md) | 简体中文 | [繁體中文](https://github.com/go-admin-team/go-admin/blob/master/README.zh-TW.md) | [日本語](https://github.com/go-admin-team/go-admin/blob/master/README.ja-JP.md)
|
||||
|
||||
基于Gin + Vue + Element UI OR Arco Design OR Ant Design的前后端分离权限管理系统,系统初始化极度简单,只需要配置文件中,修改数据库连接,系统支持多指令操作,迁移指令可以让初始化数据库信息变得更简单,服务指令可以很简单的启动api服务
|
||||
|
||||
@@ -78,9 +78,9 @@ antd 体验(go-admin-pro):[https://antd.go-admin.pro](https://antd.go-admi
|
||||
|
||||
### 轻松实现go-admin写出第一个应用 - 文档教程
|
||||
|
||||
[步骤一 - 基础内容介绍](https://doc.go-admin.dev/guide/intro/tutorial01.html)
|
||||
[步骤一 - 基础内容介绍](https://www.go-admin.pro/guide/intro/tutorial01.html)
|
||||
|
||||
[步骤二 - 实际应用 - 编写增删改查](https://doc.go-admin.dev/guide/intro/tutorial02.html)
|
||||
[步骤二 - 实际应用 - 编写增删改查](https://www.go-admin.pro/guide/intro/tutorial02.html)
|
||||
|
||||
### 手把手教你从入门到放弃 - 视频教程
|
||||
|
||||
@@ -173,7 +173,7 @@ D:\Code\go-admin>go build
|
||||
cgo: exec gcc: exec: "gcc": executable file not found in %PATH%
|
||||
```
|
||||
|
||||
[解决cgo问题进入](https://doc.go-admin.dev/zh-CN/guide/faq#cgo-%E7%9A%84%E9%97%AE%E9%A2%98)
|
||||
[解决cgo问题进入](https://www.go-admin.pro/zh-CN/guide/faq#cgo-%E7%9A%84%E9%97%AE%E9%A2%98)
|
||||
|
||||
|
||||
#### 初始化数据库,以及服务启动
|
||||
@@ -327,7 +327,7 @@ pnpm dev
|
||||
4. [gin](https://github.com/gin-gonic/gin)
|
||||
5. [casbin](https://github.com/casbin/casbin)
|
||||
6. [spf13/viper](https://github.com/spf13/viper)
|
||||
7. [gorm](https://github.com/jinzhu/gorm)
|
||||
7. [gorm](https://github.com/go-gorm/gorm)
|
||||
8. [gin-swagger](https://github.com/swaggo/gin-swagger)
|
||||
9. [golang-jwt](https://github.com/golang-jwt/jwt)
|
||||
10. [vue-element-admin](https://github.com/PanJiaChen/vue-element-admin)
|
||||
|
||||
+350
@@ -0,0 +1,350 @@
|
||||
# go-admin
|
||||
|
||||
<img align="right" width="320" src="https://doc-image.zhangwj.com/img/go-admin.svg">
|
||||
|
||||
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin/releases)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
|
||||
[English](https://github.com/go-admin-team/go-admin/blob/master/README.md) | [简体中文](https://github.com/go-admin-team/go-admin/blob/master/README.Zh-cn.md) | [繁體中文](https://github.com/go-admin-team/go-admin/blob/master/README.zh-TW.md) | 日本語
|
||||
|
||||
Gin + Vue + Element UI / Arco Design / Ant Design による、フロントエンドとバックエンドを分離した権限管理システムです。初期化は非常に簡単で、設定ファイルのデータベース接続情報を変更するだけで動作します。複数のコマンドに対応しており、マイグレーションコマンドでデータベースの初期化が容易になり、サーバーコマンドで API を手軽に起動できます。
|
||||
|
||||
[オンラインドキュメント](https://www.go-admin.pro)
|
||||
|
||||
[フロントエンドプロジェクト](https://github.com/go-admin-team/go-admin-ui)
|
||||
|
||||
[動画チュートリアル](https://space.bilibili.com/565616721/channel/detail?cid=125737)
|
||||
|
||||
## 🎬 オンラインデモ
|
||||
|
||||
Element Plus vue3 デモ:[https://vue.go-admin.pro](https://vue.go-admin.pro/#/login)
|
||||
> ⚠️⚠️⚠️ アカウント / パスワード: admin / 123456
|
||||
|
||||
antd デモ(go-admin-pro):[https://antd.go-admin.pro](https://antd.go-admin.pro/)
|
||||
> ⚠️⚠️⚠️ アカウント / パスワード: admin / 123456
|
||||
|
||||
## ✨ 特徴
|
||||
|
||||
- RESTful API の設計規約に準拠
|
||||
|
||||
- GIN WEB API フレームワークをベースに、豊富なミドルウェアを提供(ユーザー認証、CORS、アクセスログ、トレース ID など)
|
||||
|
||||
- Casbin による RBAC アクセス制御モデル
|
||||
|
||||
- JWT 認証
|
||||
|
||||
- Swagger ドキュメントに対応(swaggo ベース)
|
||||
|
||||
- GORM によるデータベース永続化、複数種類のデータベースに拡張可能
|
||||
|
||||
- 設定ファイルからモデルへの単純なマッピングで、必要な設定をすぐに取得
|
||||
|
||||
- コード生成ツール
|
||||
|
||||
- フォームビルダー
|
||||
|
||||
- マルチコマンド方式
|
||||
|
||||
- マルチテナント対応
|
||||
|
||||
- TODO: ユニットテスト
|
||||
|
||||
## 🎁 標準機能
|
||||
|
||||
1. マルチテナント:デフォルトで対応。データベース単位で分離し、1 データベースにつき 1 テナント。
|
||||
1. ユーザー管理:システムの操作者であるユーザーの設定を行います。
|
||||
2. 部門管理:組織構造(会社・部門・グループ)を設定します。ツリー構造で表示し、データ権限に対応します。
|
||||
3. 役職管理:ユーザーが担当する職務を設定します。
|
||||
4. メニュー管理:メニュー、操作権限、ボタン権限識別子、API 権限などを設定します。
|
||||
5. ロール管理:ロールへのメニュー権限の割り当て、および組織単位でのデータ範囲権限の設定を行います。
|
||||
6. 辞書管理:システム内で頻繁に使う固定的なデータを管理します。
|
||||
7. パラメータ管理:よく使うパラメータを動的に設定します。
|
||||
8. 操作ログ:正常系の操作ログと異常情報のログを記録・検索します。
|
||||
9. ログインログ:ログイン履歴を記録・検索します。ログイン異常も含みます。
|
||||
1. API ドキュメント:業務コードから API ドキュメントを自動生成します。
|
||||
1. コード生成:テーブル定義から CRUD 業務を生成します。すべて画面上で操作でき、基本的な業務をコードなしで実現できます。
|
||||
1. フォームビルダー:ページのスタイルをカスタマイズし、ドラッグ&ドロップでレイアウトを作成します。
|
||||
1. サービス監視:サーバーの基本情報を確認します。
|
||||
1. コンテンツ管理:デモ機能。カテゴリ管理とコンテンツ管理を含み、入門用の参考実装として利用できます。
|
||||
1. スケジュールタスク:自動実行タスク。現在は API 呼び出しと関数呼び出しに対応しています。
|
||||
|
||||
## 事前準備
|
||||
|
||||
ローカルに [go] [gin] [node](http://nodejs.org/) と [git](https://git-scm.com/) をインストールしてください。
|
||||
|
||||
ダウンロードから使いこなすまでを解説した動画とドキュメントのチュートリアルを用意しています。本プロジェクトを試す前に、まずこれらに目を通すことを強くおすすめします。
|
||||
|
||||
### go-admin で最初のアプリケーションを作る - ドキュメント
|
||||
|
||||
[ステップ 1 - 基礎の紹介](https://www.go-admin.pro/guide/intro/tutorial01.html)
|
||||
|
||||
[ステップ 2 - 実践 - CRUD を書く](https://www.go-admin.pro/guide/intro/tutorial02.html)
|
||||
|
||||
### 動画チュートリアル
|
||||
|
||||
[go-admin の起動方法](https://www.bilibili.com/video/BV1z5411x7JG)
|
||||
|
||||
[生成ツールで業務を手軽に実装する](https://www.bilibili.com/video/BV1Dg4y1i79D)
|
||||
|
||||
[v1.1.0 のコード生成ツール](https://www.bilibili.com/video/BV1N54y1i71P) [応用]
|
||||
|
||||
[マルチコマンドでの起動方法と IDE 設定](https://www.bilibili.com/video/BV1Fg4y1q7ph)
|
||||
|
||||
[go-admin のメニュー設定](https://www.bilibili.com/video/BV1Wp4y1D715) [必見]
|
||||
|
||||
[メニュー情報と API 情報の設定方法](https://www.bilibili.com/video/BV1zv411B7nG) [必見]
|
||||
|
||||
[go-admin の権限設定](https://www.bilibili.com/video/BV1rt4y197d3) [必見]
|
||||
|
||||
[go-admin のデータ権限](https://www.bilibili.com/video/BV1LK4y1s71e) [必見]
|
||||
|
||||
**不明点はまず上記のドキュメントと記事をご確認ください。解決しない場合は issue や pr をお寄せください。動画とドキュメントは継続的に更新しています**
|
||||
|
||||
## 📦 ローカル開発
|
||||
|
||||
### 動作要件
|
||||
|
||||
go 1.26.5
|
||||
|
||||
node バージョン: v22 以上(v24 LTS 推奨)
|
||||
|
||||
パッケージマネージャー: pnpm v9 以上(UI プロジェクトは pnpm を使用)
|
||||
|
||||
### 開発ディレクトリの作成
|
||||
|
||||
```bash
|
||||
|
||||
# 開発ディレクトリを作成
|
||||
mkdir goadmin
|
||||
cd goadmin
|
||||
```
|
||||
|
||||
### コードの取得
|
||||
|
||||
> 重要:2 つのプロジェクトは同じディレクトリに配置してください。
|
||||
|
||||
```bash
|
||||
# バックエンドのコードを取得
|
||||
git clone https://github.com/go-admin-team/go-admin.git
|
||||
|
||||
# フロントエンドのコードを取得
|
||||
git clone https://github.com/go-admin-team/go-admin-ui.git
|
||||
|
||||
```
|
||||
|
||||
### 起動方法
|
||||
|
||||
#### サーバーの起動
|
||||
|
||||
```bash
|
||||
# go-admin バックエンドプロジェクトへ移動
|
||||
cd ./go-admin
|
||||
|
||||
# 依存関係を整理
|
||||
go mod tidy
|
||||
|
||||
# ビルド
|
||||
go build
|
||||
|
||||
# 設定を変更
|
||||
# ファイルパス go-admin/config/settings.yml
|
||||
vi ./config/settings.yml
|
||||
|
||||
# 1. 設定ファイル内のデータベース情報を変更
|
||||
# 注意: settings.database 配下の設定項目
|
||||
# 2. log のパスを確認
|
||||
```
|
||||
|
||||
⚠️注意 Windows 環境で CGO が未導入の場合、次のエラーが発生します。
|
||||
|
||||
```bash
|
||||
E:\go-admin>go build
|
||||
# github.com/mattn/go-sqlite3
|
||||
cgo: exec /missing-cc: exec: "/missing-cc": file does not exist
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```bash
|
||||
D:\Code\go-admin>go build
|
||||
# github.com/mattn/go-sqlite3
|
||||
cgo: exec gcc: exec: "gcc": executable file not found in %PATH%
|
||||
```
|
||||
|
||||
[cgo の問題の解決方法はこちら](https://www.go-admin.pro/zh-CN/guide/faq#cgo-%E7%9A%84%E9%97%AE%E9%A2%98)
|
||||
|
||||
|
||||
#### データベースの初期化とサービス起動
|
||||
|
||||
``` bash
|
||||
# 初回はデータベースのリソース情報を初期化する必要があります
|
||||
# macOS または linux の場合
|
||||
$ ./go-admin migrate -c config/settings.dev.yml
|
||||
|
||||
# ⚠️注意: windows の場合
|
||||
$ go-admin.exe migrate -c config/settings.dev.yml
|
||||
|
||||
|
||||
# プロジェクトを起動します。IDE からデバッグ実行することもできます
|
||||
# macOS または linux の場合
|
||||
$ ./go-admin server -c config/settings.yml
|
||||
|
||||
|
||||
# ⚠️注意: windows の場合
|
||||
$ go-admin.exe server -c config/settings.yml
|
||||
```
|
||||
|
||||
#### sys_api テーブルへのデータ追加方法
|
||||
|
||||
起動時に `-a true` を付けると、不足している API データが自動的に追加されます。
|
||||
```bash
|
||||
./go-admin server -c config/settings.yml -a true
|
||||
```
|
||||
|
||||
#### docker でのビルドと起動
|
||||
|
||||
```shell
|
||||
# イメージをビルド
|
||||
docker build -t go-admin .
|
||||
|
||||
# コンテナを起動します。1 つ目の go-admin はコンテナ名、2 つ目はイメージ名です
|
||||
# -v は設定ファイルのマウント ローカルパス:コンテナ内パス
|
||||
docker run --name go-admin -p 8000:8000 -v /config/settings.yml:/config/settings.yml -d go-admin-server
|
||||
```
|
||||
|
||||
#### ドキュメント生成
|
||||
|
||||
```bash
|
||||
go generate
|
||||
```
|
||||
|
||||
#### クロスコンパイル
|
||||
|
||||
```bash
|
||||
# windows
|
||||
env GOOS=windows GOARCH=amd64 go build main.go
|
||||
|
||||
# or
|
||||
# linux
|
||||
env GOOS=linux GOARCH=amd64 go build main.go
|
||||
```
|
||||
|
||||
### UI 側の起動方法
|
||||
|
||||
```bash
|
||||
# pnpm をインストール(未導入の場合)
|
||||
npm install -g pnpm
|
||||
|
||||
# 依存関係をインストール
|
||||
pnpm install
|
||||
|
||||
# 中国本土のネットワークではミラーを指定すると高速化できます
|
||||
pnpm install --registry=https://registry.npmmirror.com
|
||||
|
||||
# 開発サーバーを起動
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
## 📨 コミュニティ
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td><img src="https://raw.githubusercontent.com/wenjianzhang/image/master/img/wx.png" width="180px"></td>
|
||||
<td><img src="https://doc-image.zhangwj.com/img/qrcode_for_gh_b798dc7db30c_258.jpg" width="180px"></td>
|
||||
<td><img src="https://raw.githubusercontent.com/wenjianzhang/image/master/img/qq2.png" width="200px"></td>
|
||||
<td><a href="https://space.bilibili.com/565616721">wenjianzhang</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>微信</td>
|
||||
<td>公众号🔥🔥🔥</td>
|
||||
<td><a target="_blank" href="https://shang.qq.com/wpa/qunwpa?idkey=0f2bf59f5f2edec6a4550c364242c0641f870aa328e468c4ee4b7dbfb392627b"><img border="0" src="https://pub.idqqimg.com/wpa/images/group.png" alt="go-admin技术交流乙号" title="go-admin技术交流乙号"></a></td>
|
||||
<td>哔哩哔哩🔥🔥🔥</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 💎 コントリビューター
|
||||
|
||||
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wenjianzhang" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/3890175?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/G-Akiraka" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45746659?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/lwnmengjing" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/12806223?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/bing127" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31166183?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/chengxiao" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/1379545?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/NightFire0307" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/19854086?v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/appleboy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/21979?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/ninstein" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/580303?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/kikiyou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/17959053?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/horizonzy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/22524871?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Cassuis" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/48005724?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/hqcchina" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/5179057?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/nodece" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16235121?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/stephenzhang0713" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/18169290?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/zhouxixi-dev" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/100399679?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Jalins" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31172582?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wkf928592" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6063351?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wxxiong6" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6983441?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Silicon-He" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/52478309?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/GizmoOAO" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20385106?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/bestgopher" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/36840497?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wxb1207" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20775558?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/misakichan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16569274?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/zhuxuyang" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/19301024?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/mss-boot" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/109259065?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/AuroraV" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/37330199?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Vingurzhou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/57127283?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/haimait" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/40926384?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/zyd" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/3446278?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/infnan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/38274826?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/d1y" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45585937?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/qlijin" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/515900?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/logtous
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/88697234?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/stepway
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/9927079?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/NaturalGao
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/43291304?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/DemoLiang
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/23476007?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/jfcg
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/1410597?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Nicole0724
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/10487328?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
## JetBrains のオープンソースライセンス支援
|
||||
|
||||
`go-admin` は一貫して JetBrains 社の GoLand 統合開発環境で開発されています。**free JetBrains Open Source license(s)** による正規の無償ライセンス提供に、この場を借りて感謝を申し上げます。
|
||||
|
||||
<a href="https://www.jetbrains.com/?from=kubeadm-ha" target="_blank"><img src="https://raw.githubusercontent.com/panjf2000/illustrations/master/jetbrains/jetbrains-variant-4.png" width="250" align="middle"/></a>
|
||||
|
||||
## 🤝 謝辞
|
||||
|
||||
1. [ant-design](https://github.com/ant-design/ant-design)
|
||||
2. [ant-design-pro](https://github.com/ant-design/ant-design-pro)
|
||||
2. [arco-design](https://github.com/arco-design/arco-design)
|
||||
2. [arco-design-pro](https://github.com/arco-design/arco-design-pro)
|
||||
4. [gin](https://github.com/gin-gonic/gin)
|
||||
5. [casbin](https://github.com/casbin/casbin)
|
||||
6. [spf13/viper](https://github.com/spf13/viper)
|
||||
7. [gorm](https://github.com/go-gorm/gorm)
|
||||
8. [gin-swagger](https://github.com/swaggo/gin-swagger)
|
||||
9. [golang-jwt](https://github.com/golang-jwt/jwt)
|
||||
10. [vue-element-admin](https://github.com/PanJiaChen/vue-element-admin)
|
||||
11. [ruoyi-vue](https://gitee.com/y_project/RuoYi-Vue)
|
||||
12. [form-generator](https://github.com/JakHuang/form-generator)
|
||||
|
||||
## 🤟 支援
|
||||
|
||||
> このプロジェクトがお役に立ちましたら、作者にジュースを一杯おごる形で応援いただけます :tropical_drink:
|
||||
|
||||
<img class="no-margin" src="https://raw.githubusercontent.com/wenjianzhang/image/master/img/pay.png" height="200px" >
|
||||
|
||||
## 🤝 関連リンク
|
||||
|
||||
- [mss-boot-io](https://docs.mss-boot-io.top/)
|
||||
|
||||
## 🔑 License
|
||||
|
||||
[MIT](https://github.com/go-admin-team/go-admin/blob/master/LICENSE.md)
|
||||
|
||||
Copyright (c) 2026 wenjianzhang
|
||||
@@ -4,15 +4,15 @@
|
||||
<img align="right" width="320" src="https://raw.githubusercontent.com/wenjianzhang/image/203c5930b9ed08d5cf2fcb4516b85e412f8e0e60/img/go-admin.svg">
|
||||
|
||||
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin/releases)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
|
||||
English | [简体中文](https://github.com/go-admin-team/go-admin/blob/master/README.Zh-cn.md)
|
||||
English | [简体中文](https://github.com/go-admin-team/go-admin/blob/master/README.Zh-cn.md) | [繁體中文](https://github.com/go-admin-team/go-admin/blob/master/README.zh-TW.md) | [日本語](https://github.com/go-admin-team/go-admin/blob/master/README.ja-JP.md)
|
||||
|
||||
The front-end and back-end separation authority management system based on Gin + Vue + Element UI OR Arco Design is extremely simple to initialize the system. You only need to modify the database connection in the configuration file. The system supports multi-instruction operations. Migration instructions can make it easier to initialize database information. Service instructions It's easy to start the api service.
|
||||
The front-end and back-end separation authority management system based on Gin + Vue + Element UI OR Arco Design OR Ant Design is extremely simple to initialize the system. You only need to modify the database connection in the configuration file. The system supports multi-instruction operations. Migration instructions can make it easier to initialize database information. Service instructions It's easy to start the api service.
|
||||
|
||||
[documentation](https://www.go-admin.dev)
|
||||
[documentation](https://www.go-admin.pro)
|
||||
|
||||
[Front-end project](https://github.com/go-admin-team/go-admin-ui)
|
||||
|
||||
@@ -76,9 +76,9 @@ At the same time, a series of tutorials including videos and documents are provi
|
||||
|
||||
### Easily implement go-admin to write the first application-documentation tutorial
|
||||
|
||||
[Step 1 - basic content introduction](https://doc.go-admin.dev/guide/intro/tutorial01.html)
|
||||
[Step 1 - basic content introduction](https://www.go-admin.pro/guide/intro/tutorial01.html)
|
||||
|
||||
[Step 2 - Practical application - writing database operations](https://doc.go-admin.dev/guide/intro/tutorial02.html)
|
||||
[Step 2 - Practical application - writing database operations](https://www.go-admin.pro/guide/intro/tutorial02.html)
|
||||
|
||||
### Teach you from getting started to giving up-video tutorial
|
||||
|
||||
@@ -155,7 +155,7 @@ vi ./config/settings.yml
|
||||
# 2. Confirm the log path
|
||||
```
|
||||
|
||||
:::tip ⚠️Note that this problem will occur if CGO is not installed in the windows10+ environment;
|
||||
⚠️ Note that this problem will occur if CGO is not installed in the windows10+ environment;
|
||||
|
||||
```bash
|
||||
E:\go-admin>go build
|
||||
@@ -171,9 +171,7 @@ D:\Code\go-admin>go build
|
||||
cgo: exec gcc: exec: "gcc": executable file not found in %PATH%
|
||||
```
|
||||
|
||||
[Solve the cgo problem and enter](https://doc.go-admin.dev/guide/faq#cgo-%E7%9A%84%E9%97%AE%E9%A2%98)
|
||||
|
||||
:::
|
||||
[Solve the cgo problem and enter](https://www.go-admin.pro/guide/faq#cgo-%E7%9A%84%E9%97%AE%E9%A2%98)
|
||||
|
||||
#### Initialize the database, and start the service
|
||||
|
||||
@@ -318,7 +316,7 @@ The `go-admin` project has always been developed in the GoLand integrated develo
|
||||
2. [gin](https://github.com/gin-gonic/gin)
|
||||
2. [casbin](https://github.com/casbin/casbin)
|
||||
2. [spf13/viper](https://github.com/spf13/viper)
|
||||
2. [gorm](https://github.com/jinzhu/gorm)
|
||||
2. [gorm](https://github.com/go-gorm/gorm)
|
||||
2. [gin-swagger](https://github.com/swaggo/gin-swagger)
|
||||
2. [golang-jwt](https://github.com/golang-jwt/jwt)
|
||||
2. [vue-element-admin](https://github.com/PanJiaChen/vue-element-admin)
|
||||
|
||||
+350
@@ -0,0 +1,350 @@
|
||||
# go-admin
|
||||
|
||||
<img align="right" width="320" src="https://doc-image.zhangwj.com/img/go-admin.svg">
|
||||
|
||||
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
[](https://github.com/go-admin-team/go-admin/releases)
|
||||
[](https://github.com/go-admin-team/go-admin)
|
||||
|
||||
[English](https://github.com/go-admin-team/go-admin/blob/master/README.md) | [简体中文](https://github.com/go-admin-team/go-admin/blob/master/README.Zh-cn.md) | 繁體中文 | [日本語](https://github.com/go-admin-team/go-admin/blob/master/README.ja-JP.md)
|
||||
|
||||
基於 Gin + Vue + Element UI OR Arco Design OR Ant Design 的前後端分離權限管理系統。系統初始化極為簡單,只需在設定檔中修改資料庫連線資訊即可。系統支援多指令操作:遷移指令讓資料庫初始化變得更簡單,服務指令則能輕鬆啟動 API 服務。
|
||||
|
||||
[線上文件](https://www.go-admin.pro)
|
||||
|
||||
[前端專案](https://github.com/go-admin-team/go-admin-ui)
|
||||
|
||||
[影片教學](https://space.bilibili.com/565616721/channel/detail?cid=125737)
|
||||
|
||||
## 🎬 線上體驗
|
||||
|
||||
Element Plus vue3 體驗:[https://vue.go-admin.pro](https://vue.go-admin.pro/#/login)
|
||||
> ⚠️⚠️⚠️ 帳號 / 密碼: admin / 123456
|
||||
|
||||
antd 體驗(go-admin-pro):[https://antd.go-admin.pro](https://antd.go-admin.pro/)
|
||||
> ⚠️⚠️⚠️ 帳號 / 密碼: admin / 123456
|
||||
|
||||
## ✨ 特性
|
||||
|
||||
- 遵循 RESTful API 設計規範
|
||||
|
||||
- 基於 GIN WEB API 框架,提供豐富的中介軟體支援(使用者認證、跨域、存取日誌、追蹤 ID 等)
|
||||
|
||||
- 基於 Casbin 的 RBAC 存取控制模型
|
||||
|
||||
- JWT 認證
|
||||
|
||||
- 支援 Swagger 文件(基於 swaggo)
|
||||
|
||||
- 基於 GORM 的資料庫儲存,可擴充多種類型資料庫
|
||||
|
||||
- 設定檔簡單的模型映射,快速取得所需設定
|
||||
|
||||
- 程式碼產生工具
|
||||
|
||||
- 表單建構工具
|
||||
|
||||
- 多指令模式
|
||||
|
||||
- 多租戶的支援
|
||||
|
||||
- TODO: 單元測試
|
||||
|
||||
## 🎁 內建
|
||||
|
||||
1. 多租戶:系統預設支援多租戶,按資料庫分離,一個資料庫一個租戶。
|
||||
1. 使用者管理:使用者是系統操作者,該功能主要完成系統使用者設定。
|
||||
2. 部門管理:設定系統組織架構(公司、部門、小組),以樹狀結構呈現並支援資料權限。
|
||||
3. 職位管理:設定系統使用者所擔任的職務。
|
||||
4. 選單管理:設定系統選單、操作權限、按鈕權限標識、介面權限等。
|
||||
5. 角色管理:角色選單權限分配、設定角色按機構進行資料範圍權限劃分。
|
||||
6. 字典管理:對系統中經常使用且較為固定的資料進行維護。
|
||||
7. 參數管理:對系統動態設定常用參數。
|
||||
8. 操作日誌:系統正常操作的日誌記錄與查詢;系統異常資訊的日誌記錄與查詢。
|
||||
9. 登入日誌:系統登入日誌記錄查詢,包含登入異常。
|
||||
1. 介面文件:根據業務程式碼自動產生相關的 API 介面文件。
|
||||
1. 程式碼產生:根據資料表結構產生對應的增刪改查業務,全程視覺化操作,讓基本業務可以零程式碼實現。
|
||||
1. 表單建構:自訂頁面樣式,拖拉放實現頁面佈局。
|
||||
1. 服務監控:檢視伺服器的基本資訊。
|
||||
1. 內容管理:demo 功能,下設分類管理、內容管理,可參考使用以快速入門。
|
||||
1. 排程任務:自動化任務,目前支援介面呼叫與函式呼叫。
|
||||
|
||||
## 準備工作
|
||||
|
||||
你需要在本機安裝 [go] [gin] [node](http://nodejs.org/) 和 [git](https://git-scm.com/)
|
||||
|
||||
同時配套了系列教學(含影片與文件),說明如何從下載到熟練使用。強烈建議先看完這些教學再來實作本專案!!!
|
||||
|
||||
### 輕鬆用 go-admin 寫出第一個應用 - 文件教學
|
||||
|
||||
[步驟一 - 基礎內容介紹](https://www.go-admin.pro/guide/intro/tutorial01.html)
|
||||
|
||||
[步驟二 - 實際應用 - 撰寫增刪改查](https://www.go-admin.pro/guide/intro/tutorial02.html)
|
||||
|
||||
### 手把手教你從入門到放棄 - 影片教學
|
||||
|
||||
[如何啟動 go-admin](https://www.bilibili.com/video/BV1z5411x7JG)
|
||||
|
||||
[使用產生工具輕鬆實現業務](https://www.bilibili.com/video/BV1Dg4y1i79D)
|
||||
|
||||
[v1.1.0 版本程式碼產生工具 - 釋放雙手](https://www.bilibili.com/video/BV1N54y1i71P) [進階]
|
||||
|
||||
[多指令啟動方式講解以及 IDE 設定](https://www.bilibili.com/video/BV1Fg4y1q7ph)
|
||||
|
||||
[go-admin 選單的設定說明](https://www.bilibili.com/video/BV1Wp4y1D715) [必看]
|
||||
|
||||
[如何設定選單資訊以及介面資訊](https://www.bilibili.com/video/BV1zv411B7nG) [必看]
|
||||
|
||||
[go-admin 權限設定使用說明](https://www.bilibili.com/video/BV1rt4y197d3) [必看]
|
||||
|
||||
[go-admin 資料權限使用說明](https://www.bilibili.com/video/BV1LK4y1s71e) [必看]
|
||||
|
||||
**如有問題請先參閱上述文件與文章,若仍無法解決,歡迎提出 issue 與 pr。影片教學與文件持續更新中**
|
||||
|
||||
## 📦 本機開發
|
||||
|
||||
### 環境需求
|
||||
|
||||
go 1.26.5
|
||||
|
||||
node 版本: v22+(建議 v24 LTS)
|
||||
|
||||
套件管理器: pnpm v9+(UI 專案使用 pnpm)
|
||||
|
||||
### 建立開發目錄
|
||||
|
||||
```bash
|
||||
|
||||
# 建立開發目錄
|
||||
mkdir goadmin
|
||||
cd goadmin
|
||||
```
|
||||
|
||||
### 取得程式碼
|
||||
|
||||
> 重點注意:兩個專案必須放在同一資料夾下;
|
||||
|
||||
```bash
|
||||
# 取得後端程式碼
|
||||
git clone https://github.com/go-admin-team/go-admin.git
|
||||
|
||||
# 取得前端程式碼
|
||||
git clone https://github.com/go-admin-team/go-admin-ui.git
|
||||
|
||||
```
|
||||
|
||||
### 啟動說明
|
||||
|
||||
#### 伺服器端啟動說明
|
||||
|
||||
```bash
|
||||
# 進入 go-admin 後端專案
|
||||
cd ./go-admin
|
||||
|
||||
# 更新整理相依套件
|
||||
go mod tidy
|
||||
|
||||
# 編譯專案
|
||||
go build
|
||||
|
||||
# 修改設定
|
||||
# 檔案路徑 go-admin/config/settings.yml
|
||||
vi ./config/settings.yml
|
||||
|
||||
# 1. 在設定檔中修改資料庫資訊
|
||||
# 注意: settings.database 下對應的設定資料
|
||||
# 2. 確認 log 路徑
|
||||
```
|
||||
|
||||
⚠️注意 在 Windows 環境若未安裝 CGO,會出現這個問題;
|
||||
|
||||
```bash
|
||||
E:\go-admin>go build
|
||||
# github.com/mattn/go-sqlite3
|
||||
cgo: exec /missing-cc: exec: "/missing-cc": file does not exist
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```bash
|
||||
D:\Code\go-admin>go build
|
||||
# github.com/mattn/go-sqlite3
|
||||
cgo: exec gcc: exec: "gcc": executable file not found in %PATH%
|
||||
```
|
||||
|
||||
[解決 cgo 問題請進入](https://www.go-admin.pro/zh-CN/guide/faq#cgo-%E7%9A%84%E9%97%AE%E9%A2%98)
|
||||
|
||||
|
||||
#### 初始化資料庫,以及服務啟動
|
||||
|
||||
``` bash
|
||||
# 首次設定需要初始化資料庫資源資訊
|
||||
# macOS or linux 下使用
|
||||
$ ./go-admin migrate -c config/settings.dev.yml
|
||||
|
||||
# ⚠️注意:windows 下使用
|
||||
$ go-admin.exe migrate -c config/settings.dev.yml
|
||||
|
||||
|
||||
# 啟動專案,也可以用 IDE 進行除錯
|
||||
# macOS or linux 下使用
|
||||
$ ./go-admin server -c config/settings.yml
|
||||
|
||||
|
||||
# ⚠️注意:windows 下使用
|
||||
$ go-admin.exe server -c config/settings.yml
|
||||
```
|
||||
|
||||
#### sys_api 表的資料如何新增
|
||||
|
||||
在專案啟動時,使用 `-a true` 系統會自動新增缺少的介面資料
|
||||
```bash
|
||||
./go-admin server -c config/settings.yml -a true
|
||||
```
|
||||
|
||||
#### 使用 docker 編譯啟動
|
||||
|
||||
```shell
|
||||
# 編譯映像檔
|
||||
docker build -t go-admin .
|
||||
|
||||
# 啟動容器,第一個 go-admin 是容器名稱,第二個 go-admin 是映像檔名稱
|
||||
# -v 映射設定檔 本機路徑:容器路徑
|
||||
docker run --name go-admin -p 8000:8000 -v /config/settings.yml:/config/settings.yml -d go-admin-server
|
||||
```
|
||||
|
||||
#### 文件產生
|
||||
|
||||
```bash
|
||||
go generate
|
||||
```
|
||||
|
||||
#### 交叉編譯
|
||||
|
||||
```bash
|
||||
# windows
|
||||
env GOOS=windows GOARCH=amd64 go build main.go
|
||||
|
||||
# or
|
||||
# linux
|
||||
env GOOS=linux GOARCH=amd64 go build main.go
|
||||
```
|
||||
|
||||
### UI 互動端啟動說明
|
||||
|
||||
```bash
|
||||
# 安裝 pnpm(若未安裝)
|
||||
npm install -g pnpm
|
||||
|
||||
# 安裝相依套件
|
||||
pnpm install
|
||||
|
||||
# 中國大陸網路可指定鏡像來源加速
|
||||
pnpm install --registry=https://registry.npmmirror.com
|
||||
|
||||
# 啟動服務
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
## 📨 互動
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td><img src="https://raw.githubusercontent.com/wenjianzhang/image/master/img/wx.png" width="180px"></td>
|
||||
<td><img src="https://doc-image.zhangwj.com/img/qrcode_for_gh_b798dc7db30c_258.jpg" width="180px"></td>
|
||||
<td><img src="https://raw.githubusercontent.com/wenjianzhang/image/master/img/qq2.png" width="200px"></td>
|
||||
<td><a href="https://space.bilibili.com/565616721">wenjianzhang</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>微信</td>
|
||||
<td>公众号🔥🔥🔥</td>
|
||||
<td><a target="_blank" href="https://shang.qq.com/wpa/qunwpa?idkey=0f2bf59f5f2edec6a4550c364242c0641f870aa328e468c4ee4b7dbfb392627b"><img border="0" src="https://pub.idqqimg.com/wpa/images/group.png" alt="go-admin技术交流乙号" title="go-admin技术交流乙号"></a></td>
|
||||
<td>哔哩哔哩🔥🔥🔥</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 💎 貢獻者
|
||||
|
||||
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wenjianzhang" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/3890175?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/G-Akiraka" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45746659?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/lwnmengjing" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/12806223?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/bing127" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31166183?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/chengxiao" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/1379545?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/NightFire0307" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/19854086?v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/appleboy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/21979?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/ninstein" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/580303?v=4&h=60&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/kikiyou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/17959053?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/horizonzy" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/22524871?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Cassuis" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/48005724?s=64&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/hqcchina" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/5179057?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/nodece" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16235121?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/stephenzhang0713" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/18169290?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/zhouxixi-dev" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/100399679?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Jalins" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/31172582?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wkf928592" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6063351?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wxxiong6" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/6983441?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Silicon-He" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/52478309?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/GizmoOAO" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20385106?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/bestgopher" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/36840497?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/wxb1207" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/20775558?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/misakichan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/16569274?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/zhuxuyang" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/19301024?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/mss-boot" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/109259065?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/AuroraV" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/37330199?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Vingurzhou" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/57127283?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/haimait" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/40926384?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/zyd" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/3446278?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/infnan" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/38274826?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/d1y" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/45585937?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/qlijin" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/515900?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/logtous
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/88697234?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/stepway
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/9927079?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/NaturalGao
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/43291304?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/DemoLiang
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/23476007?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/jfcg
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/1410597?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
<span style="margin: 0 5px;" ><a href="https://github.com/Nicole0724
|
||||
" ><img src="https://images.weserv.nl/?url=avatars.githubusercontent.com/u/10487328?s=60&v=4&w=60&fit=cover&mask=circle&maxage=7d" /></a></span>
|
||||
## JetBrains 開源證書支援
|
||||
|
||||
`go-admin` 專案一直以來都是在 JetBrains 公司旗下的 GoLand 整合開發環境中進行開發,基於 **free JetBrains Open Source license(s)** 正版免費授權,在此表達我的謝意。
|
||||
|
||||
<a href="https://www.jetbrains.com/?from=kubeadm-ha" target="_blank"><img src="https://raw.githubusercontent.com/panjf2000/illustrations/master/jetbrains/jetbrains-variant-4.png" width="250" align="middle"/></a>
|
||||
|
||||
## 🤝 特別感謝
|
||||
|
||||
1. [ant-design](https://github.com/ant-design/ant-design)
|
||||
2. [ant-design-pro](https://github.com/ant-design/ant-design-pro)
|
||||
2. [arco-design](https://github.com/arco-design/arco-design)
|
||||
2. [arco-design-pro](https://github.com/arco-design/arco-design-pro)
|
||||
4. [gin](https://github.com/gin-gonic/gin)
|
||||
5. [casbin](https://github.com/casbin/casbin)
|
||||
6. [spf13/viper](https://github.com/spf13/viper)
|
||||
7. [gorm](https://github.com/go-gorm/gorm)
|
||||
8. [gin-swagger](https://github.com/swaggo/gin-swagger)
|
||||
9. [golang-jwt](https://github.com/golang-jwt/jwt)
|
||||
10. [vue-element-admin](https://github.com/PanJiaChen/vue-element-admin)
|
||||
11. [ruoyi-vue](https://gitee.com/y_project/RuoYi-Vue)
|
||||
12. [form-generator](https://github.com/JakHuang/form-generator)
|
||||
|
||||
## 🤟 贊助
|
||||
|
||||
> 如果你覺得這個專案幫助到了你,可以幫作者買一杯果汁表示鼓勵 :tropical_drink:
|
||||
|
||||
<img class="no-margin" src="https://raw.githubusercontent.com/wenjianzhang/image/master/img/pay.png" height="200px" >
|
||||
|
||||
## 🤝 連結
|
||||
|
||||
- [mss-boot-io](https://docs.mss-boot-io.top/)
|
||||
|
||||
## 🔑 License
|
||||
|
||||
[MIT](https://github.com/go-admin-team/go-admin/blob/master/LICENSE.md)
|
||||
|
||||
Copyright (c) 2026 wenjianzhang
|
||||
@@ -21,13 +21,16 @@ func (e System) GenerateCaptchaHandler(c *gin.Context) {
|
||||
e.Error(500, err, "服务初始化失败!")
|
||||
return
|
||||
}
|
||||
id, b64s, answer, err := captcha.DriverDigitFunc()
|
||||
// The answer is deliberately discarded rather than logged. It used to be
|
||||
// written at info level, which put a currently valid captcha answer in the
|
||||
// application log - anyone able to read the log could bypass the check the
|
||||
// captcha exists to enforce.
|
||||
id, b64s, _, err := captcha.DriverDigitFunc()
|
||||
if err != nil {
|
||||
e.Logger.Errorf("DriverDigitFunc error, %s", err.Error())
|
||||
e.Error(500, err, "验证码获取失败")
|
||||
return
|
||||
}
|
||||
e.Logger.Infof("DriverDigitFunc answer: %s", answer)
|
||||
e.Custom(gin.H{
|
||||
"code": 200,
|
||||
"data": b64s,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package apis
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"github.com/gin-gonic/gin/binding"
|
||||
"go-admin/app/admin/models"
|
||||
"golang.org/x/crypto/bcrypt"
|
||||
@@ -15,6 +16,7 @@ import (
|
||||
"go-admin/app/admin/service"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"go-admin/common/actions"
|
||||
"go-admin/common/middleware"
|
||||
)
|
||||
|
||||
type SysUser struct {
|
||||
@@ -149,12 +151,34 @@ func (e SysUser) Update(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
req.SetUpdateBy(user.GetUserId(c))
|
||||
callerId := user.GetUserId(c)
|
||||
|
||||
// This route is in CasbinExclude so the personal-center screen can edit
|
||||
// the caller's own record without a policy grant (see settings.go). That
|
||||
// exclusion covers the whole route, not just the caller's own record, and
|
||||
// the request carries the target userId in the body - so without this
|
||||
// check here, any authenticated caller could edit any other user, up to
|
||||
// and including their roleId. When the target is someone else, ask Casbin
|
||||
// directly for the permission AuthCheckRole skipped.
|
||||
if req.UserId != callerId {
|
||||
allowed, err := middleware.EnforceRoleFor(c, c.Request.URL.Path, c.Request.Method)
|
||||
if err != nil {
|
||||
e.Logger.Error(err)
|
||||
e.Error(500, err, err.Error())
|
||||
return
|
||||
}
|
||||
if !allowed {
|
||||
e.Error(http.StatusForbidden, errors.New("无权更新其他用户数据"), "对不起,您没有该接口访问权限,请联系管理员")
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
req.SetUpdateBy(callerId)
|
||||
|
||||
//数据权限检查
|
||||
p := actions.GetPermissionFromContext(c)
|
||||
|
||||
err = s.Update(&req, p)
|
||||
err = s.Update(&req, p, callerId)
|
||||
if err != nil {
|
||||
e.Logger.Error(err)
|
||||
return
|
||||
@@ -420,7 +444,6 @@ func (e SysUser) GetInfo(c *gin.Context) {
|
||||
e.Error(500, err, err.Error())
|
||||
return
|
||||
}
|
||||
p := actions.GetPermissionFromContext(c)
|
||||
var roles = make([]string, 1)
|
||||
roles[0] = user.GetRoleName(c)
|
||||
var permissions = make([]string, 1)
|
||||
@@ -440,7 +463,14 @@ func (e SysUser) GetInfo(c *gin.Context) {
|
||||
}
|
||||
sysUser := models.SysUser{}
|
||||
req.Id = user.GetUserId(c)
|
||||
err = s.Get(&req, p, &sysUser)
|
||||
// Unscoped on purpose: the id is the caller's own, taken from the token.
|
||||
// This used to go through Get with whatever GetPermissionFromContext
|
||||
// returned - and this route installs no PermissionAction, so that was the
|
||||
// zero value. An unset scope is not a recognised one, so once unknown
|
||||
// scopes started failing closed rather than silently matching everything,
|
||||
// every login on a deployment with enabledp: true ended here with a 401
|
||||
// and the browser went straight back to the login page.
|
||||
err = s.GetSelf(&req, &sysUser)
|
||||
if err != nil {
|
||||
e.Error(http.StatusUnauthorized, err, "登录失败")
|
||||
return
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
package apis
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/glebarez/sqlite"
|
||||
mycasbin "github.com/go-admin-team/go-admin-core/v2/casbin"
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
"github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
)
|
||||
|
||||
// PUT /api/v1/sys-user is in settings.go's CasbinExclude so the
|
||||
// personal-center screen (go-admin-ui's userInfo.vue) can edit the caller's
|
||||
// own record without holding a policy grant on this route. AuthCheckRole
|
||||
// skips Enforce entirely for an excluded route, so this file's job is to pin
|
||||
// what the handler itself now has to hold shut: the target userId comes from
|
||||
// the request body, and nothing upstream of the handler ever checked it
|
||||
// against the caller.
|
||||
|
||||
// setupPrivescDB wires an in-memory database and a Casbin enforcer with an
|
||||
// empty policy - the state of a fresh install for any role but admin - under
|
||||
// a tenant unique to the calling test, so mycasbin's process-wide enforcer
|
||||
// cache can't hand one test's database to another.
|
||||
func setupPrivescDB(t *testing.T) (*gorm.DB, string) {
|
||||
t.Helper()
|
||||
|
||||
// Fatalf, not Skipf: this database is in-memory sqlite with no external
|
||||
// dependency, so failing to open or migrate it means the environment is
|
||||
// actually broken. Skipping here would let these two anti-privesc
|
||||
// regression tests silently stop running while CI stays green - a
|
||||
// standing assertion that never fires is worse than no assertion.
|
||||
db, err := gorm.Open(sqlite.Open("file::memory:"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatalf("sqlite unavailable: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(&models.SysUser{}); err != nil {
|
||||
t.Fatalf("automigrate: %v", err)
|
||||
}
|
||||
|
||||
tenant := "sys-user-privesc-" + t.Name()
|
||||
|
||||
previousInterval := mycasbin.ReloadInterval
|
||||
mycasbin.ReloadInterval = 0 // opt out of the background reload goroutine; the test never writes a policy
|
||||
t.Cleanup(func() { mycasbin.ReloadInterval = previousInterval })
|
||||
|
||||
e := mycasbin.Setup(db, tenant)
|
||||
previousEnforcer := sdk.Runtime.GetCasbinByTenant(tenant)
|
||||
sdk.Runtime.SetCasbinByTenant(tenant, e)
|
||||
t.Cleanup(func() { sdk.Runtime.SetCasbinByTenant(tenant, previousEnforcer) })
|
||||
|
||||
return db, tenant
|
||||
}
|
||||
|
||||
// callUpdate drives SysUser.Update the way the router does for an
|
||||
// authenticated, non-admin caller: JWT claims already decoded into the
|
||||
// context (that is jwtauth's job, not this handler's) and a database - but
|
||||
// without AuthCheckRole, since that middleware never runs Enforce for this
|
||||
// route at all.
|
||||
func callUpdate(t *testing.T, db *gorm.DB, tenant string, callerId int, body map[string]interface{}) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
raw, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
t.Fatalf("marshal request body: %v", err)
|
||||
}
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest(http.MethodPut, "/api/v1/sys-user", bytes.NewReader(raw))
|
||||
c.Request.Host = tenant
|
||||
c.Request.Header.Set("Content-Type", "application/json")
|
||||
|
||||
c.Set("db", db)
|
||||
c.Set(pkg.LoggerKey, logger.NewHelper(logger.DefaultLogger))
|
||||
c.Set(jwt.JwtPayloadKey, jwt.MapClaims{
|
||||
"identity": float64(callerId),
|
||||
"rolekey": "ordinary-role", // holds no Casbin policy anywhere in this test
|
||||
})
|
||||
|
||||
SysUser{}.Update(c)
|
||||
return w
|
||||
}
|
||||
|
||||
// TestUpdate_CannotEscalatePrivilegeThroughAnotherUsersRecord is the
|
||||
// regression for H6. Before the fix, an ordinary authenticated user could PUT
|
||||
// a body naming another user's id and change that user's roleId - the route
|
||||
// being Casbin-excluded meant no permission check ever ran, and the data
|
||||
// permission scope that would otherwise gate this is off by default.
|
||||
func TestUpdate_CannotEscalatePrivilegeThroughAnotherUsersRecord(t *testing.T) {
|
||||
db, tenant := setupPrivescDB(t)
|
||||
|
||||
victim := models.SysUser{Username: "bob", NickName: "Bob", RoleId: 2, DeptId: 1, Status: "1"}
|
||||
if err := db.Create(&victim).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
attacker := models.SysUser{Username: "alice", NickName: "Alice", RoleId: 2, DeptId: 1, Status: "1"}
|
||||
if err := db.Create(&attacker).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
const elevatedRoleId = 1 // a role the attacker does not hold and has no policy for
|
||||
|
||||
callUpdate(t, db, tenant, attacker.UserId, map[string]interface{}{
|
||||
"userId": victim.UserId,
|
||||
"username": victim.Username,
|
||||
"nickName": "pwned",
|
||||
"phone": "13800000000",
|
||||
"email": "bob@example.com",
|
||||
"roleId": elevatedRoleId,
|
||||
"deptId": victim.DeptId,
|
||||
"status": victim.Status,
|
||||
})
|
||||
|
||||
var after models.SysUser
|
||||
if err := db.First(&after, victim.UserId).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after.RoleId == elevatedRoleId {
|
||||
t.Fatalf("an attacker with no Casbin permission on this route escalated the victim's roleId to %d", after.RoleId)
|
||||
}
|
||||
if after.NickName == "pwned" {
|
||||
t.Fatalf("an attacker with no Casbin permission on this route modified another user's record: %+v", after)
|
||||
}
|
||||
}
|
||||
|
||||
// TestUpdate_SelfEditCannotChangePrivilegedFields covers the case the
|
||||
// CasbinExclude entry exists for: the personal-center screen has to keep
|
||||
// working for the caller's own record. The fields that screen exposes
|
||||
// (nickName/phone/email/sex) must still save, while roleId/deptId/status stay
|
||||
// whatever the database already had even if the request carries something
|
||||
// else - a compromised or hand-crafted client is the only way that request
|
||||
// would ever differ from what the honest form sends.
|
||||
func TestUpdate_SelfEditCannotChangePrivilegedFields(t *testing.T) {
|
||||
db, tenant := setupPrivescDB(t)
|
||||
|
||||
self := models.SysUser{Username: "carol", NickName: "Carol", RoleId: 2, DeptId: 1, Status: "1"}
|
||||
if err := db.Create(&self).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
const elevatedRoleId = 1
|
||||
|
||||
callUpdate(t, db, tenant, self.UserId, map[string]interface{}{
|
||||
"userId": self.UserId,
|
||||
"username": self.Username,
|
||||
"nickName": "Carol Updated",
|
||||
"phone": "13900000000",
|
||||
"email": "carol@example.com",
|
||||
"roleId": elevatedRoleId, // tampered; must not take effect
|
||||
"deptId": self.DeptId,
|
||||
"status": self.Status,
|
||||
})
|
||||
|
||||
var after models.SysUser
|
||||
if err := db.First(&after, self.UserId).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after.RoleId == elevatedRoleId {
|
||||
t.Fatalf("a self-edit changed the caller's own roleId to %d", after.RoleId)
|
||||
}
|
||||
if after.NickName != "Carol Updated" {
|
||||
t.Fatalf("the legitimate personal-center edit did not go through: %+v", after)
|
||||
}
|
||||
}
|
||||
@@ -1,81 +0,0 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"gorm.io/gorm"
|
||||
|
||||
log "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
)
|
||||
|
||||
type DataPermission struct {
|
||||
DataScope string
|
||||
UserId int
|
||||
DeptId int
|
||||
RoleId int
|
||||
}
|
||||
|
||||
func (e *DataPermission) GetDataScope(tableName string, db *gorm.DB) (*gorm.DB, error) {
|
||||
|
||||
if !config.ApplicationConfig.EnableDP {
|
||||
usageStr := `数据权限已经为您` + pkg.Green(`关闭`) + `,如需开启请参考配置文件字段说明`
|
||||
log.Debug("%s\n", usageStr)
|
||||
return db, nil
|
||||
}
|
||||
user := new(SysUser)
|
||||
role := new(SysRole)
|
||||
err := db.Find(user, e.UserId).Error
|
||||
if err != nil {
|
||||
return nil, errors.New("获取用户数据出错 msg:" + err.Error())
|
||||
}
|
||||
err = db.Find(role, user.RoleId).Error
|
||||
if err != nil {
|
||||
return nil, errors.New("获取用户数据出错 msg:" + err.Error())
|
||||
}
|
||||
if role.DataScope == "2" {
|
||||
db = db.Where(tableName+".create_by in (select sys_user.user_id from sys_role_dept left join sys_user on sys_user.dept_id=sys_role_dept.dept_id where sys_role_dept.role_id = ?)", user.RoleId)
|
||||
}
|
||||
if role.DataScope == "3" {
|
||||
db = db.Where(tableName+".create_by in (SELECT user_id from sys_user where dept_id = ? )", user.DeptId)
|
||||
}
|
||||
if role.DataScope == "4" {
|
||||
db = db.Where(tableName+".create_by in (SELECT user_id from sys_user where sys_user.dept_id in(select dept_id from sys_dept where dept_path like ? ))", "%"+pkg.IntToString(user.DeptId)+"%")
|
||||
}
|
||||
if role.DataScope == "5" || role.DataScope == "" {
|
||||
db = db.Where(tableName+".create_by = ?", e.UserId)
|
||||
}
|
||||
|
||||
return db, nil
|
||||
}
|
||||
|
||||
//func DataScopes(tableName string, userId int) func(db *gorm.DB) *gorm.DB {
|
||||
// return func(db *gorm.DB) *gorm.DB {
|
||||
// user := new(SysUser)
|
||||
// role := new(SysRole)
|
||||
// user.UserId = userId
|
||||
// err := db.Find(user, userId).Error
|
||||
// if err != nil {
|
||||
// db.Error = errors.New("获取用户数据出错 msg:" + err.Error())
|
||||
// return db
|
||||
// }
|
||||
// err = db.Find(role, user.RoleId).Error
|
||||
// if err != nil {
|
||||
// db.Error = errors.New("获取用户数据出错 msg:" + err.Error())
|
||||
// return db
|
||||
// }
|
||||
// if role.DataScope == "2" {
|
||||
// return db.Where(tableName+".create_by in (select sys_user.user_id from sys_role_dept left join sys_user on sys_user.dept_id=sys_role_dept.dept_id where sys_role_dept.role_id = ?)", user.RoleId)
|
||||
// }
|
||||
// if role.DataScope == "3" {
|
||||
// return db.Where(tableName+".create_by in (SELECT user_id from sys_user where dept_id = ? )", user.DeptId)
|
||||
// }
|
||||
// if role.DataScope == "4" {
|
||||
// return db.Where(tableName+".create_by in (SELECT user_id from sys_user where sys_user.dept_id in(select dept_id from sys_dept where dept_path like ? ))", "%"+pkg.IntToString(user.DeptId)+"%")
|
||||
// }
|
||||
// if role.DataScope == "5" || role.DataScope == "" {
|
||||
// return db.Where(tableName+".create_by = ?", userId)
|
||||
// }
|
||||
// return db
|
||||
// }
|
||||
//}
|
||||
@@ -23,6 +23,10 @@ type SysApi struct {
|
||||
Path string `json:"path" gorm:"size:128;comment:地址"`
|
||||
Action string `json:"action" gorm:"size:16;comment:请求类型"`
|
||||
Type string `json:"type" gorm:"size:16;comment:接口类型"`
|
||||
// AppCode identifies which application's seed.SeedMenus call wrote this
|
||||
// row; empty for the host's own built-in APIs. Same NOT NULL DEFAULT ''
|
||||
// reasoning as SysMenu.AppCode.
|
||||
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;default:'';index:idx_sys_api_app_code;comment:AppCode"`
|
||||
models.ModelTime
|
||||
models.ControlBy
|
||||
}
|
||||
|
||||
@@ -26,6 +26,12 @@ type SysMenu struct {
|
||||
RoleId int `gorm:"-"`
|
||||
Children []SysMenu `json:"children,omitempty" gorm:"-"`
|
||||
IsSelect bool `json:"is_select" gorm:"-"`
|
||||
// AppCode identifies which application's seed.SeedMenus call wrote this
|
||||
// row; empty for the host's own built-in menus. NOT NULL DEFAULT '' for
|
||||
// the same reason sys_migration.app_code is (see contract/models.Migration):
|
||||
// AutoMigrate adding this column to an existing table leaves every
|
||||
// pre-existing row reading back as "" rather than NULL.
|
||||
AppCode string `json:"appCode" gorm:"type:varchar(64);not null;default:'';index:idx_sys_menu_app_code;comment:AppCode"`
|
||||
models.ControlBy
|
||||
models.ModelTime
|
||||
}
|
||||
|
||||
@@ -42,19 +42,35 @@ func (e *SysUser) GetId() interface{} {
|
||||
return e.UserId
|
||||
}
|
||||
|
||||
// Encrypt 加密
|
||||
func (e *SysUser) Encrypt() (err error) {
|
||||
// Encrypt hashes Password, unless it already holds a hash.
|
||||
//
|
||||
// The hooks below run on whatever is in the struct, and a user read from the
|
||||
// database carries the stored hash in that field. Hashing it again produces a
|
||||
// hash of a hash, and the password that user knows no longer matches anything:
|
||||
// they cannot log in, and nothing reports an error. The only thing preventing
|
||||
// that today is an Omit("password") on the one update that loads a user first,
|
||||
// which makes every other write to this model one line away from destroying
|
||||
// credentials.
|
||||
//
|
||||
// bcrypt.Cost parses a hash and fails on anything else, so it distinguishes
|
||||
// the two cases without the call site having to say which it is. The cost is
|
||||
// that a password which is itself a well-formed bcrypt hash would be stored
|
||||
// unchanged - a 60-character string beginning "$2a$", not something a person
|
||||
// types, and it grants whoever set it no access they did not already have.
|
||||
func (e *SysUser) Encrypt() error {
|
||||
if e.Password == "" {
|
||||
return
|
||||
return nil
|
||||
}
|
||||
if _, err := bcrypt.Cost([]byte(e.Password)); err == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
var hash []byte
|
||||
if hash, err = bcrypt.GenerateFromPassword([]byte(e.Password), bcrypt.DefaultCost); err != nil {
|
||||
return
|
||||
} else {
|
||||
e.Password = string(hash)
|
||||
return
|
||||
hash, err := bcrypt.GenerateFromPassword([]byte(e.Password), bcrypt.DefaultCost)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
e.Password = string(hash)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (e *SysUser) BeforeCreate(_ *gorm.DB) error {
|
||||
@@ -62,11 +78,7 @@ func (e *SysUser) BeforeCreate(_ *gorm.DB) error {
|
||||
}
|
||||
|
||||
func (e *SysUser) BeforeUpdate(_ *gorm.DB) error {
|
||||
var err error
|
||||
if e.Password != "" {
|
||||
err = e.Encrypt()
|
||||
}
|
||||
return err
|
||||
return e.Encrypt()
|
||||
}
|
||||
|
||||
func (e *SysUser) AfterFind(_ *gorm.DB) error {
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"golang.org/x/crypto/bcrypt"
|
||||
)
|
||||
|
||||
const knownPassword = "correct-horse-battery-staple"
|
||||
|
||||
// A user loaded from the database carries the stored hash in Password, and the
|
||||
// hooks run on whatever is in the struct. Hashing it a second time produces a
|
||||
// hash of a hash: the password the user knows stops matching, they cannot log
|
||||
// in, and nothing reports an error.
|
||||
//
|
||||
// Only an Omit("password") on one call site stood between this and every write
|
||||
// to the model. This is the test that removes the need for it.
|
||||
func TestEncryptLeavesAnAlreadyHashedPasswordAlone(t *testing.T) {
|
||||
fresh := SysUser{Password: knownPassword}
|
||||
if err := fresh.Encrypt(); err != nil {
|
||||
t.Fatalf("Encrypt: %v", err)
|
||||
}
|
||||
stored := fresh.Password
|
||||
if err := bcrypt.CompareHashAndPassword([]byte(stored), []byte(knownPassword)); err != nil {
|
||||
t.Fatalf("setup failed: the password was not hashed: %v", err)
|
||||
}
|
||||
|
||||
// What a query puts in the struct, and what an update then hands the hook.
|
||||
loaded := SysUser{Password: stored}
|
||||
if err := loaded.Encrypt(); err != nil {
|
||||
t.Fatalf("Encrypt on a loaded user: %v", err)
|
||||
}
|
||||
if loaded.Password != stored {
|
||||
t.Error("Encrypt re-hashed a stored hash; the user can no longer log in")
|
||||
}
|
||||
if err := bcrypt.CompareHashAndPassword([]byte(loaded.Password), []byte(knownPassword)); err != nil {
|
||||
t.Errorf("the user can no longer log in with their password: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The other half: a password that is not a hash still gets hashed, on create
|
||||
// and on update alike.
|
||||
func TestEncryptHashesAPlaintextPassword(t *testing.T) {
|
||||
for _, c := range []struct {
|
||||
name string
|
||||
hook func(*SysUser) error
|
||||
}{
|
||||
{"BeforeCreate", func(u *SysUser) error { return u.BeforeCreate(nil) }},
|
||||
{"BeforeUpdate", func(u *SysUser) error { return u.BeforeUpdate(nil) }},
|
||||
} {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
u := SysUser{Password: knownPassword}
|
||||
if err := c.hook(&u); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if u.Password == knownPassword {
|
||||
t.Fatal("the password was stored as it was typed")
|
||||
}
|
||||
if err := bcrypt.CompareHashAndPassword([]byte(u.Password), []byte(knownPassword)); err != nil {
|
||||
t.Errorf("the stored value does not verify the password: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// An empty Password means "not being set", and must not become a hash of "".
|
||||
func TestEncryptIgnoresAnEmptyPassword(t *testing.T) {
|
||||
u := SysUser{}
|
||||
if err := u.Encrypt(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if u.Password != "" {
|
||||
t.Errorf("an unset password became %q", u.Password)
|
||||
}
|
||||
}
|
||||
|
||||
// Encrypt runs on every update of this model, including the ones that change
|
||||
// something else entirely. What it costs when there is nothing to do is the
|
||||
// difference between a profile update and a bcrypt round; the correctness test
|
||||
// above is what catches a regression, this reports the size of it.
|
||||
func BenchmarkEncrypt(b *testing.B) {
|
||||
fresh := SysUser{Password: knownPassword}
|
||||
if err := fresh.Encrypt(); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
|
||||
b.Run("already hashed", func(b *testing.B) {
|
||||
u := SysUser{Password: fresh.Password}
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
if err := u.Encrypt(); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
b.Run("plaintext", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
u := SysUser{Password: knownPassword}
|
||||
if err := u.Encrypt(); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -25,11 +25,9 @@ func InitRouter() {
|
||||
os.Exit(-1)
|
||||
}
|
||||
|
||||
// the jwt middleware
|
||||
authMiddleware, err := common.AuthInit()
|
||||
if err != nil {
|
||||
log.Fatalf("JWT Init Error, %s", err.Error())
|
||||
}
|
||||
// the jwt middleware: shared instance InitMiddleware built at startup,
|
||||
// not one built here per module (see common/middleware.GetAuthMiddleware).
|
||||
authMiddleware := common.GetAuthMiddleware()
|
||||
|
||||
// 注册系统路由
|
||||
InitSysRouter(r, authMiddleware)
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
|
||||
"go-admin/app/admin/apis"
|
||||
"go-admin/common/actions"
|
||||
"go-admin/common/middleware"
|
||||
)
|
||||
|
||||
@@ -15,7 +16,10 @@ func init() {
|
||||
// registerSysApiRouter
|
||||
func registerSysApiRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddleware) {
|
||||
api := apis.SysApi{}
|
||||
r := v1.Group("/sys-api").Use(authMiddleware.MiddlewareFunc()).Use(middleware.AuthCheckRole())
|
||||
// PermissionAction is not optional here: all three handlers below read the
|
||||
// data permission out of the context, and without it they read the zero
|
||||
// value - an unset scope, which Permission now fails closed on.
|
||||
r := v1.Group("/sys-api").Use(authMiddleware.MiddlewareFunc()).Use(middleware.AuthCheckRole()).Use(actions.PermissionAction())
|
||||
{
|
||||
r.GET("", api.GetPage)
|
||||
r.GET("/:id", api.Get)
|
||||
|
||||
@@ -5,12 +5,17 @@ import (
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/common/dto"
|
||||
"go-admin/common/global"
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
// Deprecated: use global.OperaStatusEnabled / global.OperaStatusDisabled.
|
||||
// These two names are kept - misspelling and all - because forks import them;
|
||||
// the values moved to common/global so common/middleware no longer has to
|
||||
// import this package. See docs/contract.md.
|
||||
const (
|
||||
OperaStatusEnabel = "1" // 状态-正常
|
||||
OperaStatusDisable = "2" // 状态-关闭
|
||||
OperaStatusEnabel = global.OperaStatusEnabled // 状态-正常
|
||||
OperaStatusDisable = global.OperaStatusDisabled // 状态-关闭
|
||||
)
|
||||
|
||||
type SysOperaLogGetPageReq struct {
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
package dto
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"go-admin/common/global"
|
||||
)
|
||||
|
||||
// The values moved to common/global so common/middleware would stop importing
|
||||
// this package; these two names stayed behind as aliases, misspelling and all,
|
||||
// because forks import them.
|
||||
//
|
||||
// If they ever drift apart, rows written through the two spellings land in
|
||||
// different buckets and the operation-log filter silently misses half of them.
|
||||
func TestDeprecatedStatusAliasesStillMatch(t *testing.T) {
|
||||
if OperaStatusEnabel != global.OperaStatusEnabled {
|
||||
t.Errorf("OperaStatusEnabel = %q, global.OperaStatusEnabled = %q",
|
||||
OperaStatusEnabel, global.OperaStatusEnabled)
|
||||
}
|
||||
if OperaStatusDisable != global.OperaStatusDisabled {
|
||||
t.Errorf("OperaStatusDisable = %q, global.OperaStatusDisabled = %q",
|
||||
OperaStatusDisable, global.OperaStatusDisabled)
|
||||
}
|
||||
}
|
||||
@@ -42,7 +42,7 @@ type SysRoleInsertReq struct {
|
||||
Flag string `form:"flag" comment:"标记"` // 标记
|
||||
Remark string `form:"remark" comment:"备注"` // 备注
|
||||
Admin bool `form:"admin" comment:"是否管理员"`
|
||||
DataScope string `form:"dataScope"`
|
||||
DataScope string `form:"dataScope" vd:"$=='1'||$=='2'||$=='3'||$=='4'||$=='5'"` // must be one of actions.DataScope{All,Custom,Dept,DeptTree,Self}; PRD 006 F14/H2
|
||||
SysMenu []models.SysMenu `form:"sysMenu"`
|
||||
MenuIds []int `form:"menuIds"`
|
||||
SysDept []models.SysDept `form:"sysDept"`
|
||||
@@ -79,7 +79,7 @@ type SysRoleUpdateReq struct {
|
||||
Flag string `form:"flag" comment:"标记"` // 标记
|
||||
Remark string `form:"remark" comment:"备注"` // 备注
|
||||
Admin bool `form:"admin" comment:"是否管理员"`
|
||||
DataScope string `form:"dataScope"`
|
||||
DataScope string `form:"dataScope" vd:"$=='1'||$=='2'||$=='3'||$=='4'||$=='5'"` // must be one of actions.DataScope{All,Custom,Dept,DeptTree,Self}; PRD 006 F14/H2
|
||||
SysMenu []models.SysMenu `form:"sysMenu"`
|
||||
MenuIds []int `form:"menuIds"`
|
||||
SysDept []models.SysDept `form:"sysDept"`
|
||||
@@ -147,7 +147,7 @@ func (s *SysRoleDeleteReq) GetId() interface{} {
|
||||
// RoleDataScopeReq 角色数据权限修改
|
||||
type RoleDataScopeReq struct {
|
||||
RoleId int `json:"roleId" binding:"required"`
|
||||
DataScope string `json:"dataScope" binding:"required"`
|
||||
DataScope string `json:"dataScope" binding:"required" vd:"$=='1'||$=='2'||$=='3'||$=='4'||$=='5'"` // must be one of actions.DataScope{All,Custom,Dept,DeptTree,Self}; PRD 006 F14/H2
|
||||
DeptIds []int `json:"deptIds"`
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
package dto
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
vd "github.com/bytedance/go-tagexpr/v2/validator"
|
||||
)
|
||||
|
||||
// api.Bind calls vd.Validate unconditionally on every request, regardless of
|
||||
// which binding stage ran, so a vd tag on DataScope is enough to reject
|
||||
// anything actions.Permission's fail-closed default would otherwise have to
|
||||
// deal with. PRD 006 F14/H2 named this the real trigger for the default
|
||||
// branch: SysRoleInsertReq.DataScope had no validation at all, so leaving
|
||||
// dataScope out of a create-role request wrote an empty string straight to
|
||||
// sys_role.
|
||||
func TestDataScopeRejectsWhatPermissionCannotRecognize(t *testing.T) {
|
||||
invalid := []string{"", "0", "6", "all", " 1", "1 "}
|
||||
valid := []string{"1", "2", "3", "4", "5"}
|
||||
|
||||
t.Run("SysRoleInsertReq", func(t *testing.T) {
|
||||
for _, s := range invalid {
|
||||
req := SysRoleInsertReq{RoleName: "r", RoleKey: "r", DataScope: s}
|
||||
if err := vd.Validate(&req); err == nil {
|
||||
t.Errorf("DataScope %q was accepted, want rejected", s)
|
||||
}
|
||||
}
|
||||
for _, s := range valid {
|
||||
req := SysRoleInsertReq{RoleName: "r", RoleKey: "r", DataScope: s}
|
||||
if err := vd.Validate(&req); err != nil {
|
||||
t.Errorf("DataScope %q was rejected: %v", s, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("SysRoleUpdateReq", func(t *testing.T) {
|
||||
for _, s := range invalid {
|
||||
req := SysRoleUpdateReq{RoleName: "r", RoleKey: "r", DataScope: s}
|
||||
if err := vd.Validate(&req); err == nil {
|
||||
t.Errorf("DataScope %q was accepted, want rejected", s)
|
||||
}
|
||||
}
|
||||
for _, s := range valid {
|
||||
req := SysRoleUpdateReq{RoleName: "r", RoleKey: "r", DataScope: s}
|
||||
if err := vd.Validate(&req); err != nil {
|
||||
t.Errorf("DataScope %q was rejected: %v", s, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("RoleDataScopeReq", func(t *testing.T) {
|
||||
for _, s := range invalid {
|
||||
req := RoleDataScopeReq{RoleId: 1, DataScope: s}
|
||||
if err := vd.Validate(&req); err == nil {
|
||||
t.Errorf("DataScope %q was accepted, want rejected", s)
|
||||
}
|
||||
}
|
||||
for _, s := range valid {
|
||||
req := RoleDataScopeReq{RoleId: 1, DataScope: s}
|
||||
if err := vd.Validate(&req); err != nil {
|
||||
t.Errorf("DataScope %q was rejected: %v", s, err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,308 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/seed"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
)
|
||||
|
||||
// adminSeeder is go-admin's own implementation of seed.Seeder: it turns the
|
||||
// MenuSpec/ApiSpec values a third-party application asks for into rows
|
||||
// across the four tables a visible, working menu entry needs - sys_api,
|
||||
// sys_menu, sys_menu_api_rule, and sys_role_menu/casbin_rule - following the
|
||||
// same shape cmd/migrate/migration/version/1786700001000_demo_menu.go
|
||||
// already hand-writes for the host's own demo module.
|
||||
//
|
||||
// See go-admin-core's docs/contract.md, "Application-supplied menu and API
|
||||
// entries", for the requirements this satisfies, and the security note on
|
||||
// seed.Seeder for what this boundary does and does not protect against: an
|
||||
// application already holds the same *gorm.DB this receives and could write
|
||||
// sys_menu/sys_api/casbin_rule directly, bypassing this entirely.
|
||||
type adminSeeder struct{}
|
||||
|
||||
func init() {
|
||||
seed.RegisterSeeder(adminSeeder{})
|
||||
}
|
||||
|
||||
// adminRoleKey is the role every seeded menu is granted to. This mirrors
|
||||
// 1786700001000_demo_menu.go's own convention rather than inventing a
|
||||
// second one: MenuSpec carries no "which roles should see this" field for a
|
||||
// Seeder to consult instead, and admin is the one role guaranteed to exist
|
||||
// once the framework's own seed data has run.
|
||||
const adminRoleKey = "admin"
|
||||
|
||||
// menuSortRange is what sys_menu.sort's column type actually holds.
|
||||
//
|
||||
// sort is `gorm:"size:4"`, which MySQL builds as a tinyint (-128..127);
|
||||
// sqlite ignores the width and accepts anything, so this only ever surfaces
|
||||
// on a real install, mid-migration, as Error 1264 - by which point the
|
||||
// migration has already run other, non-transactional DDL that will not be
|
||||
// retried. tools/checksilent's menu-sort-overflow check catches this for
|
||||
// every MenuSpec-shaped literal committed to this repository, but it walks
|
||||
// the repository's own source tree: a third-party application living in the
|
||||
// module cache is invisible to it. This is the equivalent check for that
|
||||
// application, run when its migration actually calls SeedMenus rather than
|
||||
// never.
|
||||
const (
|
||||
menuSortMin = -128
|
||||
menuSortMax = 127
|
||||
)
|
||||
|
||||
func (adminSeeder) SeedMenus(tx *gorm.DB, appCode string, menus []seed.MenuSpec, apis []seed.ApiSpec) error {
|
||||
apiRows, err := seedApis(tx, appCode, apis)
|
||||
if err != nil {
|
||||
return fmt.Errorf("seed: app %q: apis: %w", appCode, err)
|
||||
}
|
||||
|
||||
menuIDs, err := seedMenuTree(tx, appCode, menus, apiRows)
|
||||
if err != nil {
|
||||
return fmt.Errorf("seed: app %q: menus: %w", appCode, err)
|
||||
}
|
||||
|
||||
// Not `len(menuIDs) == 0`: grantToAdminRole grants two independent
|
||||
// things, and an application is free to register apis without menus -
|
||||
// endpoints another service calls, or a UI mounted somewhere else.
|
||||
// Skipping the whole call on an empty menu list wrote the sys_api rows
|
||||
// and then no casbin rule for them, so those endpoints were denied to
|
||||
// everyone, admin included, with a migration that reported success.
|
||||
if len(menuIDs) == 0 && len(apiRows) == 0 {
|
||||
return nil
|
||||
}
|
||||
return grantToAdminRole(tx, menuIDs, apiRows)
|
||||
}
|
||||
|
||||
// seedApis writes one sys_api row per ApiSpec and returns them keyed by
|
||||
// ApiSpec.Code, so seedMenuTree can resolve a MenuSpec's ApiCodes into the
|
||||
// rows sys_menu_api_rule needs to reference.
|
||||
//
|
||||
// sys_api.id is left to autoincrement rather than assigned by the caller,
|
||||
// unlike 1786700001000_demo_menu.go's hand-picked ids: that migration is
|
||||
// the one file tools/checksilent's menu-id-collision check can see, because
|
||||
// it lives in this repository; nothing plays that role for a third-party
|
||||
// application's ids in the module cache. Never accepting a caller-chosen id
|
||||
// here removes the collision this Seeder has no way to detect instead of
|
||||
// trying to detect it after the fact.
|
||||
func seedApis(tx *gorm.DB, appCode string, apis []seed.ApiSpec) (map[string]models.SysApi, error) {
|
||||
seen := make(map[string]bool, len(apis))
|
||||
rows := make(map[string]models.SysApi, len(apis))
|
||||
for _, a := range apis {
|
||||
if a.Code == "" {
|
||||
return nil, errors.New("ApiSpec.Code must not be empty")
|
||||
}
|
||||
if seen[a.Code] {
|
||||
return nil, fmt.Errorf("duplicate ApiSpec.Code %q", a.Code)
|
||||
}
|
||||
seen[a.Code] = true
|
||||
|
||||
row := models.SysApi{
|
||||
Handle: a.Handle,
|
||||
Title: a.Title,
|
||||
Path: a.Path,
|
||||
Action: a.Method,
|
||||
Type: "SYS",
|
||||
AppCode: appCode,
|
||||
}
|
||||
if err := tx.Create(&row).Error; err != nil {
|
||||
return nil, fmt.Errorf("api %q: %w", a.Code, err)
|
||||
}
|
||||
rows[a.Code] = row
|
||||
}
|
||||
return rows, nil
|
||||
}
|
||||
|
||||
// seedMenuTree writes one sys_menu row per MenuSpec, resolving Parent/Code
|
||||
// references into parent_id/paths, and returns every menu id created so the
|
||||
// caller can grant them to a role.
|
||||
//
|
||||
// Specs do not have to be given in parent-before-child order: this makes
|
||||
// repeated passes over the remaining specs, creating whichever ones have
|
||||
// their Parent (if any) already created, until every spec is placed. A
|
||||
// spec whose Parent never resolves - naming a Code missing from this call,
|
||||
// or only reachable through a cycle - stops making progress and is reported
|
||||
// rather than looping forever.
|
||||
func seedMenuTree(tx *gorm.DB, appCode string, specs []seed.MenuSpec, apiRows map[string]models.SysApi) ([]int, error) {
|
||||
byCode := make(map[string]seed.MenuSpec, len(specs))
|
||||
for _, s := range specs {
|
||||
if s.Code == "" {
|
||||
return nil, errors.New("MenuSpec.Code must not be empty")
|
||||
}
|
||||
if _, dup := byCode[s.Code]; dup {
|
||||
return nil, fmt.Errorf("duplicate MenuSpec.Code %q", s.Code)
|
||||
}
|
||||
if err := validateMenuSpec(s); err != nil {
|
||||
return nil, fmt.Errorf("%q: %w", s.Code, err)
|
||||
}
|
||||
byCode[s.Code] = s
|
||||
}
|
||||
|
||||
created := make(map[string]models.SysMenu, len(specs))
|
||||
ids := make([]int, 0, len(specs))
|
||||
|
||||
for len(created) < len(specs) {
|
||||
progressed := false
|
||||
for _, s := range specs {
|
||||
if _, done := created[s.Code]; done {
|
||||
continue
|
||||
}
|
||||
|
||||
var parentRow models.SysMenu
|
||||
if s.Parent != "" {
|
||||
parent, ok := created[s.Parent]
|
||||
if !ok {
|
||||
if _, exists := byCode[s.Parent]; !exists {
|
||||
return nil, fmt.Errorf("%q: Parent %q is not a Code in this call", s.Code, s.Parent)
|
||||
}
|
||||
continue // s.Parent exists but has not been created yet; retry next pass
|
||||
}
|
||||
parentRow = parent
|
||||
}
|
||||
|
||||
row := models.SysMenu{
|
||||
MenuName: menuName(appCode, s.Code),
|
||||
Title: s.Title,
|
||||
Icon: s.Icon,
|
||||
Path: s.Path,
|
||||
MenuType: s.Kind,
|
||||
Permission: s.Permission,
|
||||
ParentId: parentRow.MenuId,
|
||||
Component: s.Component,
|
||||
Sort: s.Sort,
|
||||
// Visible "0" is shown, not hidden - the same defaults
|
||||
// 1786700001000_demo_menu.go seeds its own menu with. A
|
||||
// freshly installed application's menu should not need an
|
||||
// administrator to first find and unhide it.
|
||||
Visible: "0",
|
||||
IsFrame: "1",
|
||||
AppCode: appCode,
|
||||
}
|
||||
for _, code := range s.ApiCodes {
|
||||
api, ok := apiRows[code]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%q: ApiCodes references %q, which is not an ApiSpec.Code in this call", s.Code, code)
|
||||
}
|
||||
// The full row, not just {Id: api.Id}: gorm's many2many
|
||||
// association save upserts an associated row whose primary
|
||||
// key is already set, so a stub carrying only Id would
|
||||
// overwrite every other column of an sys_api row this same
|
||||
// call just wrote with zero values.
|
||||
row.SysApi = append(row.SysApi, api)
|
||||
}
|
||||
|
||||
if err := tx.Create(&row).Error; err != nil {
|
||||
return nil, fmt.Errorf("%q: %w", s.Code, err)
|
||||
}
|
||||
|
||||
// paths is a materialized path from the root ("/0"), built from
|
||||
// ids that only exist once the row above is created - the same
|
||||
// two-step create-then-update 1786700001000_demo_menu.go's
|
||||
// hand-assigned ids let it do in one literal, sequenced here
|
||||
// instead.
|
||||
if s.Parent == "" {
|
||||
row.Paths = "/0/" + strconv.Itoa(row.MenuId)
|
||||
} else {
|
||||
row.Paths = parentRow.Paths + "/" + strconv.Itoa(row.MenuId)
|
||||
}
|
||||
if err := tx.Model(&models.SysMenu{}).Where("menu_id = ?", row.MenuId).
|
||||
Update("paths", row.Paths).Error; err != nil {
|
||||
return nil, fmt.Errorf("%q: writing paths: %w", s.Code, err)
|
||||
}
|
||||
|
||||
created[s.Code] = row
|
||||
ids = append(ids, row.MenuId)
|
||||
progressed = true
|
||||
}
|
||||
if !progressed {
|
||||
return nil, fmt.Errorf("unresolved Parent reference(s) among %d remaining spec(s); check for a cycle", len(specs)-len(created))
|
||||
}
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// validateMenuSpec rejects the malformed input tools/checksilent's
|
||||
// menu-sort-overflow and Kind-adjacent checks would catch for an in-tree
|
||||
// seed but cannot for a third-party application's - see menuSortRange's doc
|
||||
// comment.
|
||||
func validateMenuSpec(s seed.MenuSpec) error {
|
||||
switch s.Kind {
|
||||
case contractmodels.Directory, contractmodels.Menu, contractmodels.Button:
|
||||
default:
|
||||
return fmt.Errorf("Kind %q is not one of Directory/Menu/Button", s.Kind)
|
||||
}
|
||||
if s.Sort < menuSortMin || s.Sort > menuSortMax {
|
||||
return fmt.Errorf("Sort %d does not fit sys_menu.sort's tinyint column (%d..%d)", s.Sort, menuSortMin, menuSortMax)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// menuName synthesizes sys_menu.menu_name from appCode and the spec's Code,
|
||||
// since MenuSpec carries no field of its own for it - contract/seed's
|
||||
// package doc says a MenuSpec is what rendering a menu and checking a
|
||||
// button permission need, not a mirror of sys_menu's columns.
|
||||
//
|
||||
// PascalCasing both and concatenating them, rather than using Code alone,
|
||||
// is what keeps two applications that both picked the plain word "list" as
|
||||
// a Code from producing the identical menu_name: the frontend's keep-alive
|
||||
// cache matches a route by this exact string, not by (appCode, Code), so a
|
||||
// collision there is a UI bug, not a database error, and nothing else here
|
||||
// would ever surface it.
|
||||
func menuName(appCode, code string) string {
|
||||
return pascalCase(appCode) + pascalCase(code)
|
||||
}
|
||||
|
||||
func pascalCase(s string) string {
|
||||
var b strings.Builder
|
||||
for _, part := range strings.FieldsFunc(s, func(r rune) bool { return r == '-' || r == '_' }) {
|
||||
b.WriteString(strings.ToUpper(part[:1]))
|
||||
b.WriteString(part[1:])
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// grantToAdminRole is sys_role_menu and casbin_rule: the two tables
|
||||
// go-admin-core's contract.md requires alongside sys_menu/sys_api, without
|
||||
// which a seeded menu is invisible to every role and its apis are
|
||||
// authorized for no one.
|
||||
//
|
||||
// It follows 1786700001000_demo_menu.go's exact pattern, including
|
||||
// tolerating a missing admin role: a database that has not yet run the
|
||||
// framework's own seed data (config/db.sql, inside 1599190683659_tables.go)
|
||||
// has nothing to grant to yet, and namespacedKey's ordering guarantee - every
|
||||
// framework migration sorts before every app-prefixed one - means that
|
||||
// should not happen in practice, but failing this call over it would be
|
||||
// worse than a menu with no grant yet.
|
||||
func grantToAdminRole(tx *gorm.DB, menuIDs []int, apiRows map[string]models.SysApi) error {
|
||||
var role models.SysRole
|
||||
if err := tx.Where("role_key = ?", adminRoleKey).First(&role).Error; err != nil {
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
for _, id := range menuIDs {
|
||||
if err := tx.Exec(
|
||||
"INSERT INTO sys_role_menu (role_id, menu_id) SELECT ?, ? WHERE NOT EXISTS (SELECT 1 FROM sys_role_menu WHERE role_id = ? AND menu_id = ?)",
|
||||
role.RoleId, id, role.RoleId, id,
|
||||
).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
for _, a := range apiRows {
|
||||
if err := tx.Exec(
|
||||
"INSERT INTO casbin_rule (ptype, v0, v1, v2, v3, v4, v5) SELECT 'p', ?, ?, ?, '', '', '' WHERE NOT EXISTS (SELECT 1 FROM casbin_rule WHERE ptype='p' AND v0=? AND v1=? AND v2=?)",
|
||||
role.RoleKey, a.Path, a.Action, role.RoleKey, a.Path, a.Action,
|
||||
).Error; err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"gorm.io/gorm"
|
||||
|
||||
contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/contract/seed"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
)
|
||||
|
||||
// newSeedTestDB builds the tables adminSeeder.SeedMenus writes to. sys_menu,
|
||||
// sys_api, sys_role and sys_role_menu (GORM's own join table for
|
||||
// SysRole.SysMenu) come from AutoMigrate; casbin_rule does not have a GORM
|
||||
// model anywhere in this codebase - see 1786700001000_demo_menu.go's own
|
||||
// comment on why models.CasbinRule (-> sys_casbin_rule) is the wrong table -
|
||||
// so it is created directly, matching the columns grantToAdminRole's INSERT
|
||||
// addresses.
|
||||
func newSeedTestDB(t *testing.T) *gorm.DB {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(&models.SysMenu{}, &models.SysApi{}, &models.SysRole{}); err != nil {
|
||||
t.Fatalf("automigrate: %v", err)
|
||||
}
|
||||
if err := db.Exec(`CREATE TABLE casbin_rule (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
ptype TEXT, v0 TEXT, v1 TEXT, v2 TEXT, v3 TEXT, v4 TEXT, v5 TEXT
|
||||
)`).Error; err != nil {
|
||||
t.Fatalf("create casbin_rule: %v", err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
func seedAdminRole(t *testing.T, db *gorm.DB) models.SysRole {
|
||||
t.Helper()
|
||||
role := models.SysRole{RoleName: "Administrator", RoleKey: adminRoleKey}
|
||||
if err := db.Create(&role).Error; err != nil {
|
||||
t.Fatalf("seed admin role: %v", err)
|
||||
}
|
||||
return role
|
||||
}
|
||||
|
||||
// This is the acceptance case go-admin-core's docs/contract.md requires: one
|
||||
// SeedMenus call populates all four tables a visible, working menu entry
|
||||
// needs, every row tagged with the appCode it was called with, and the
|
||||
// parent/child tree resolved into sys_menu's parent_id/paths.
|
||||
func TestSeedMenusPopulatesAllFourTables(t *testing.T) {
|
||||
db := newSeedTestDB(t)
|
||||
seedAdminRole(t, db)
|
||||
|
||||
menus := []seed.MenuSpec{
|
||||
{Code: "dir", Kind: contractmodels.Directory, Title: "Order Example", Path: "/apps/order", Component: "Layout", Sort: 10},
|
||||
{Code: "list", Parent: "dir", Kind: contractmodels.Menu, Title: "Orders", Path: "list", Component: "apps/order/order/index", Sort: 1, ApiCodes: []string{"list"}},
|
||||
{Code: "btn-create", Parent: "list", Kind: contractmodels.Button, Title: "Create", Permission: "order:order:create", Sort: 1},
|
||||
}
|
||||
apis := []seed.ApiSpec{
|
||||
{Code: "list", Title: "Order list", Path: "/api/v1/order", Method: "GET", Handle: "apis.Order.GetPage-fm"},
|
||||
}
|
||||
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
return adminSeeder{}.SeedMenus(tx, "order", menus, apis)
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("SeedMenus: %v", err)
|
||||
}
|
||||
|
||||
var apiRows []models.SysApi
|
||||
if err := db.Find(&apiRows).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(apiRows) != 1 || apiRows[0].AppCode != "order" || apiRows[0].Path != "/api/v1/order" {
|
||||
t.Fatalf("sys_api = %+v", apiRows)
|
||||
}
|
||||
|
||||
var menuRows []models.SysMenu
|
||||
if err := db.Order("sort").Find(&menuRows).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(menuRows) != 3 {
|
||||
t.Fatalf("sys_menu has %d rows, want 3: %+v", len(menuRows), menuRows)
|
||||
}
|
||||
byName := map[string]models.SysMenu{}
|
||||
for _, m := range menuRows {
|
||||
if m.AppCode != "order" {
|
||||
t.Errorf("menu %q app_code = %q, want order", m.MenuName, m.AppCode)
|
||||
}
|
||||
byName[m.MenuName] = m
|
||||
}
|
||||
dir, ok := byName[menuName("order", "dir")]
|
||||
if !ok || dir.ParentId != 0 || dir.Paths != "/0/"+strconv.Itoa(dir.MenuId) {
|
||||
t.Fatalf("dir menu = %+v", dir)
|
||||
}
|
||||
list, ok := byName[menuName("order", "list")]
|
||||
if !ok || list.ParentId != dir.MenuId || list.Paths != dir.Paths+"/"+strconv.Itoa(list.MenuId) {
|
||||
t.Fatalf("list menu = %+v (dir=%+v)", list, dir)
|
||||
}
|
||||
btn, ok := byName[menuName("order", "btn-create")]
|
||||
if !ok || btn.ParentId != list.MenuId {
|
||||
t.Fatalf("btn menu = %+v (list=%+v)", btn, list)
|
||||
}
|
||||
|
||||
// sys_menu_api_rule: gorm's own many2many join table for SysMenu.SysApi.
|
||||
var joinCount int64
|
||||
if err := db.Table("sys_menu_api_rule").
|
||||
Where("sys_menu_menu_id = ? AND sys_api_id = ?", list.MenuId, apiRows[0].Id).
|
||||
Count(&joinCount).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if joinCount != 1 {
|
||||
t.Errorf("sys_menu_api_rule has %d row(s) linking list to its api, want 1", joinCount)
|
||||
}
|
||||
|
||||
// sys_role_menu: every seeded menu granted to the admin role.
|
||||
var roleMenuCount int64
|
||||
if err := db.Table("sys_role_menu").Count(&roleMenuCount).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if roleMenuCount != 3 {
|
||||
t.Errorf("sys_role_menu has %d row(s), want 3 (one per seeded menu)", roleMenuCount)
|
||||
}
|
||||
|
||||
// casbin_rule: the api's path/method granted to the admin role.
|
||||
var casbinCount int64
|
||||
if err := db.Table("casbin_rule").
|
||||
Where("ptype = 'p' AND v0 = ? AND v1 = ? AND v2 = ?", adminRoleKey, "/api/v1/order", "GET").
|
||||
Count(&casbinCount).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if casbinCount != 1 {
|
||||
t.Errorf("casbin_rule has %d matching row(s), want 1", casbinCount)
|
||||
}
|
||||
}
|
||||
|
||||
// A database that has not run the framework's own seed data yet (no admin
|
||||
// role) must not fail SeedMenus - 1786700001000_demo_menu.go tolerates
|
||||
// exactly the same condition for the host's own demo module.
|
||||
func TestSeedMenusToleratesMissingAdminRole(t *testing.T) {
|
||||
db := newSeedTestDB(t)
|
||||
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
return adminSeeder{}.SeedMenus(tx, "order", []seed.MenuSpec{
|
||||
{Code: "dir", Kind: contractmodels.Directory, Title: "Order"},
|
||||
}, nil)
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("SeedMenus: %v", err)
|
||||
}
|
||||
|
||||
var roleMenuCount int64
|
||||
if err := db.Table("sys_role_menu").Count(&roleMenuCount).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if roleMenuCount != 0 {
|
||||
t.Errorf("sys_role_menu has %d row(s) with no role to grant to", roleMenuCount)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSeedMenusRejectsMalformedSpecs(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
menus []seed.MenuSpec
|
||||
apis []seed.ApiSpec
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "duplicate menu code",
|
||||
menus: []seed.MenuSpec{{Code: "a", Kind: contractmodels.Directory}, {Code: "a", Kind: contractmodels.Directory}},
|
||||
want: `duplicate MenuSpec.Code "a"`,
|
||||
},
|
||||
{
|
||||
name: "unresolved parent",
|
||||
menus: []seed.MenuSpec{{Code: "a", Parent: "missing", Kind: contractmodels.Menu}},
|
||||
want: `Parent "missing" is not a Code in this call`,
|
||||
},
|
||||
{
|
||||
name: "unresolved api code",
|
||||
menus: []seed.MenuSpec{{Code: "a", Kind: contractmodels.Menu, ApiCodes: []string{"missing"}}},
|
||||
want: `ApiCodes references "missing"`,
|
||||
},
|
||||
{
|
||||
name: "unknown kind",
|
||||
menus: []seed.MenuSpec{{Code: "a", Kind: "X"}},
|
||||
want: `Kind "X" is not one of Directory/Menu/Button`,
|
||||
},
|
||||
{
|
||||
name: "sort overflows a tinyint",
|
||||
menus: []seed.MenuSpec{{Code: "a", Kind: contractmodels.Directory, Sort: 900}},
|
||||
want: `Sort 900 does not fit sys_menu.sort's tinyint column`,
|
||||
},
|
||||
{
|
||||
name: "duplicate api code",
|
||||
apis: []seed.ApiSpec{{Code: "x"}, {Code: "x"}},
|
||||
want: `duplicate ApiSpec.Code "x"`,
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
db := newSeedTestDB(t)
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
return adminSeeder{}.SeedMenus(tx, "order", tc.menus, tc.apis)
|
||||
})
|
||||
if err == nil || !strings.Contains(err.Error(), tc.want) {
|
||||
t.Fatalf("err = %v, want it to contain %q", err, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestSeederIsRegistered pins the registration itself, not the behaviour.
|
||||
//
|
||||
// Every other test here calls adminSeeder{}.SeedMenus directly, which proves
|
||||
// the implementation is right and proves nothing about whether anything ever
|
||||
// reaches it: delete the RegisterSeeder call in init() and they all stay
|
||||
// green, while a real migrate fails with ErrNoSeeder and no menu is written.
|
||||
// Going through the package-level SeedMenus is what closes that gap - it is
|
||||
// the door an application actually knocks on.
|
||||
func TestSeederIsRegistered(t *testing.T) {
|
||||
db := newSeedTestDB(t)
|
||||
err := db.Transaction(func(tx *gorm.DB) error {
|
||||
return seed.SeedMenus(tx, "probe", []seed.MenuSpec{{
|
||||
Code: "root", Kind: contractmodels.Directory, Title: "Probe", Sort: 1,
|
||||
}}, nil)
|
||||
})
|
||||
if errors.Is(err, seed.ErrNoSeeder) {
|
||||
t.Fatal("no Seeder is registered: an application's SeedMenus would write no menu at all")
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("SeedMenus through the package-level entry point: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// An application is free to register apis with no menus at all - endpoints
|
||||
// another service calls, or a UI mounted somewhere else. Skipping
|
||||
// grantToAdminRole on an empty menu list wrote the sys_api rows and then no
|
||||
// casbin rule for them, so every one of those endpoints was denied to
|
||||
// everyone including admin, from a migration that reported success.
|
||||
func TestSeedMenusGrantsApisWhenThereAreNoMenus(t *testing.T) {
|
||||
db := newSeedTestDB(t)
|
||||
role := seedAdminRole(t, db)
|
||||
|
||||
apis := []seed.ApiSpec{
|
||||
{Code: "hook", Title: "Inbound hook", Path: "/api/v1/hook", Method: "POST", Handle: "hook.Receive"},
|
||||
{Code: "sync", Title: "Sync", Path: "/api/v1/sync", Method: "GET", Handle: "hook.Sync"},
|
||||
}
|
||||
if err := (adminSeeder{}).SeedMenus(db, "hooks", nil, apis); err != nil {
|
||||
t.Fatalf("SeedMenus: %v", err)
|
||||
}
|
||||
|
||||
var apiCount int64
|
||||
db.Model(&models.SysApi{}).Where("app_code = ?", "hooks").Count(&apiCount)
|
||||
if apiCount != int64(len(apis)) {
|
||||
t.Fatalf("sys_api rows = %d, want %d", apiCount, len(apis))
|
||||
}
|
||||
|
||||
for _, a := range apis {
|
||||
var n int64
|
||||
db.Table("casbin_rule").
|
||||
Where("ptype = 'p' AND v0 = ? AND v1 = ? AND v2 = ?", role.RoleKey, a.Path, a.Method).
|
||||
Count(&n)
|
||||
if n != 1 {
|
||||
t.Errorf("casbin_rule for %s %s = %d rows, want 1: the endpoint is denied to admin", a.Method, a.Path, n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The other half of the same guard: nothing registered at all must stay a
|
||||
// no-op rather than start touching sys_role_menu or casbin_rule.
|
||||
func TestSeedMenusWithNothingRegisteredWritesNothing(t *testing.T) {
|
||||
db := newSeedTestDB(t)
|
||||
seedAdminRole(t, db)
|
||||
|
||||
if err := (adminSeeder{}).SeedMenus(db, "empty", nil, nil); err != nil {
|
||||
t.Fatalf("SeedMenus: %v", err)
|
||||
}
|
||||
for _, table := range []string{"casbin_rule", "sys_role_menu"} {
|
||||
var n int64
|
||||
db.Table(table).Count(&n)
|
||||
if n != 0 {
|
||||
t.Errorf("%s has %d rows, want 0", table, n)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,8 @@ import (
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/service"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"go-admin/common/actions"
|
||||
@@ -74,9 +76,18 @@ func (e *SysApi) Get(d *dto.SysApiGetReq, p *actions.DataPermission, model *mode
|
||||
// Update 修改SysApi对象
|
||||
func (e *SysApi) Update(c *dto.SysApiUpdateReq, p *actions.DataPermission) error {
|
||||
var model = models.SysApi{}
|
||||
db := e.Orm.Debug().First(&model, c.GetId())
|
||||
if db.RowsAffected == 0 {
|
||||
return errors.New("无权更新该数据")
|
||||
db := e.Orm.Scopes(
|
||||
actions.Permission(model.TableName(), p),
|
||||
).First(&model, c.GetId())
|
||||
if err := db.Error; err != nil {
|
||||
// First reports a row the data permission excluded exactly as it
|
||||
// reports one that does not exist, and the caller should not be able
|
||||
// to tell those apart either.
|
||||
if errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
return errors.New("无权更新该数据")
|
||||
}
|
||||
e.Log.Errorf("Service UpdateSysApi error:%s", err)
|
||||
return err
|
||||
}
|
||||
c.Generate(&model)
|
||||
db = e.Orm.Save(&model)
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/service"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"go-admin/common/actions"
|
||||
)
|
||||
|
||||
// An update the data permission excludes has to be refused, and refused in a
|
||||
// way that does not tell the caller whether the row exists. First reports both
|
||||
// cases the same way - no rows - so the message has to come from there rather
|
||||
// than from a RowsAffected check the error return has already skipped past.
|
||||
func TestSysApiUpdateRefusesARowOutsideTheDataPermission(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:sysapi-perm?mode=memory&cache=shared"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Skipf("sqlite unavailable: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(&models.SysApi{}); err != nil {
|
||||
t.Skipf("automigrate: %v", err)
|
||||
}
|
||||
|
||||
prev := config.ApplicationConfig.EnableDP
|
||||
config.ApplicationConfig.EnableDP = true
|
||||
t.Cleanup(func() { config.ApplicationConfig.EnableDP = prev })
|
||||
|
||||
// Owned by user 1.
|
||||
row := models.SysApi{Handle: "h", Title: "t", Path: "/api/v1/probe", Type: "BUS", Action: "GET"}
|
||||
row.CreateBy = 1
|
||||
if err := db.Create(&row).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
e := &SysApi{Service: service.Service{Orm: db, Log: logger.NewHelper(logger.DefaultLogger)}}
|
||||
req := &dto.SysApiUpdateReq{Id: row.Id, Title: "changed"}
|
||||
|
||||
// User 2, scope 5: only rows they created.
|
||||
outsider := &actions.DataPermission{DataScope: "5", UserId: 2, DeptId: 1, RoleId: 2}
|
||||
err = e.Update(req, outsider)
|
||||
if err == nil {
|
||||
t.Fatal("the update was allowed on a row the data permission excludes")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "无权更新该数据") {
|
||||
t.Errorf("refused with %q, want the permission message; a raw database error tells the "+
|
||||
"caller the row exists", err)
|
||||
}
|
||||
|
||||
var after models.SysApi
|
||||
if err := db.First(&after, row.Id).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if after.Title != "t" {
|
||||
t.Errorf("the row was modified: title is now %q", after.Title)
|
||||
}
|
||||
|
||||
// The owner still gets through, so the scope is refusing rather than
|
||||
// everything failing.
|
||||
owner := &actions.DataPermission{DataScope: "5", UserId: 1, DeptId: 1, RoleId: 1}
|
||||
if err := e.Update(&dto.SysApiUpdateReq{Id: row.Id, Title: "by owner"}, owner); err != nil {
|
||||
t.Fatalf("the owner could not update their own row: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -38,6 +38,30 @@ func (e *SysUser) GetPage(c *dto.SysUserGetPageReq, p *actions.DataPermission, l
|
||||
return nil
|
||||
}
|
||||
|
||||
// GetSelf 获取调用者自己的 SysUser 对象,不套数据权限
|
||||
//
|
||||
// The data scope answers "whose rows may this user see"; the caller here is
|
||||
// reading their own, and the id comes from the token, so there is nothing left
|
||||
// for a scope to restrict. Applying one is not a stricter version of this
|
||||
// query - it is a broken one. DataScopeSelf matches on create_by, and a user
|
||||
// account is created by whoever added it, so a scoped self-read would fail for
|
||||
// every user who did not create their own account.
|
||||
//
|
||||
// GetProfile has always read the same row this way, with no scope at all.
|
||||
func (e *SysUser) GetSelf(d *dto.SysUserById, model *models.SysUser) error {
|
||||
err := e.Orm.First(model, d.GetId()).Error
|
||||
if err != nil && errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
err = errors.New("查看对象不存在或无权查看")
|
||||
e.Log.Errorf("db error: %s", err)
|
||||
return err
|
||||
}
|
||||
if err != nil {
|
||||
e.Log.Errorf("db error: %s", err)
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get 获取SysUser对象
|
||||
func (e *SysUser) Get(d *dto.SysUserById, p *actions.DataPermission, model *models.SysUser) error {
|
||||
var data models.SysUser
|
||||
@@ -84,7 +108,16 @@ func (e *SysUser) Insert(c *dto.SysUserInsertReq) error {
|
||||
}
|
||||
|
||||
// Update 修改SysUser对象
|
||||
func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission) error {
|
||||
//
|
||||
// callerId is who is asking, not who SetUpdateBy recorded - that field only
|
||||
// says who to blame, it never constrained who could be edited. When the
|
||||
// target is the caller themselves, roleId/deptId/status are kept at whatever
|
||||
// the database already has no matter what the request body carries: this is
|
||||
// the personal-center screen's route (see CasbinExclude in settings.go, and
|
||||
// the check in the API handler ahead of this call), and letting a caller
|
||||
// grant themselves a different role or department through it would be a
|
||||
// privilege escalation the exclusion was never meant to open.
|
||||
func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission, callerId int) error {
|
||||
var err error
|
||||
var model models.SysUser
|
||||
db := e.Orm.Scopes(
|
||||
@@ -98,6 +131,11 @@ func (e *SysUser) Update(c *dto.SysUserUpdateReq, p *actions.DataPermission) err
|
||||
return errors.New("无权更新该数据")
|
||||
|
||||
}
|
||||
if model.UserId == callerId {
|
||||
c.RoleId = model.RoleId
|
||||
c.DeptId = model.DeptId
|
||||
c.Status = model.Status
|
||||
}
|
||||
c.Generate(&model)
|
||||
update := e.Orm.Model(&model).Where("user_id = ?", &model.UserId).Omit("password", "salt").Updates(&model)
|
||||
if err = update.Error; err != nil {
|
||||
|
||||
@@ -33,11 +33,9 @@ func InitRouter() {
|
||||
os.Exit(-1)
|
||||
}
|
||||
|
||||
// the jwt middleware
|
||||
authMiddleware, err := common.AuthInit()
|
||||
if err != nil {
|
||||
log.Fatalf("JWT Init Error, %s", err.Error())
|
||||
}
|
||||
// the jwt middleware: shared instance InitMiddleware built at startup,
|
||||
// not one built here per module (see common/middleware.GetAuthMiddleware).
|
||||
authMiddleware := common.GetAuthMiddleware()
|
||||
|
||||
// 注册业务路由
|
||||
InitBusinessRouter(r, authMiddleware)
|
||||
|
||||
+29
-3
@@ -1,6 +1,7 @@
|
||||
package jobs
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
log "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
@@ -145,11 +146,36 @@ func setup(key string, db *gorm.DB) {
|
||||
}
|
||||
|
||||
// 其中任务
|
||||
crontab.Start()
|
||||
startCrontab(crontab)
|
||||
}
|
||||
|
||||
// startCrontab starts c and arranges for it to be stopped on the way out.
|
||||
//
|
||||
// The stop used to be `defer crontab.Stop()` followed by `select {}`. The
|
||||
// select never returned, so the defer never ran and the scheduler was never
|
||||
// stopped; and because setup never returned, the loop in Setup never reached
|
||||
// the second tenant - only whichever database came first out of the map ever
|
||||
// got a scheduler at all. cron.Start is itself `go c.run()`, so the select was
|
||||
// blocking for nothing.
|
||||
//
|
||||
// cron.Stop returns a context that closes once the jobs already running have
|
||||
// finished. That is the wait the shutdown budget exists to bound: giving up on
|
||||
// it leaves those jobs running until the process exits, which is better than
|
||||
// holding the whole shutdown open for one job that will not end.
|
||||
func startCrontab(c *cron.Cron) {
|
||||
c.Start()
|
||||
fmt.Println(time.Now().Format(timeFormat), " [INFO] JobCore start success.")
|
||||
|
||||
// 关闭任务
|
||||
defer crontab.Stop()
|
||||
select {}
|
||||
sdk.Runtime.SetShutdown(func(ctx context.Context) {
|
||||
stopped := c.Stop()
|
||||
select {
|
||||
case <-stopped.Done():
|
||||
fmt.Println(time.Now().Format(timeFormat), " [INFO] JobCore stopped.")
|
||||
case <-ctx.Done():
|
||||
fmt.Println(time.Now().Format(timeFormat), " [WARN] JobCore stop gave up waiting for running jobs")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// AddJob 添加任务 AddJob(invokeTarget string, jobId int, jobName string, cronExpression string)
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
package jobs
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg/cronjob"
|
||||
)
|
||||
|
||||
// The scheduler had never been stopped. `defer crontab.Stop()` sat directly
|
||||
// above a `select {}` that never returned, so the deferred call was
|
||||
// unreachable for the life of the process.
|
||||
//
|
||||
// There is one test rather than several because BeforeExit closes to further
|
||||
// registration once it has run: a second RunShutdown in this binary would find
|
||||
// an empty registry and pass while proving nothing.
|
||||
func TestTheSchedulerIsStoppedOnTheWayOut(t *testing.T) {
|
||||
var ticks atomic.Int64
|
||||
|
||||
c := cronjob.NewWithSeconds()
|
||||
if _, err := c.AddFunc("* * * * * *", func() { ticks.Add(1) }); err != nil {
|
||||
t.Fatalf("AddFunc: %v", err)
|
||||
}
|
||||
|
||||
startCrontab(c)
|
||||
|
||||
// It has to be running before stopping it can mean anything.
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for ticks.Load() == 0 && time.Now().Before(deadline) {
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
if ticks.Load() == 0 {
|
||||
t.Fatal("the scheduler never ran the job, so this test cannot show it was stopped")
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
if err := sdk.Runtime.RunShutdown(ctx); err != nil {
|
||||
t.Fatalf("RunShutdown: %v", err)
|
||||
}
|
||||
|
||||
// Two and a half seconds is two more firings of a job that runs every
|
||||
// second, so silence here is the assertion.
|
||||
at := ticks.Load()
|
||||
time.Sleep(2500 * time.Millisecond)
|
||||
if n := ticks.Load() - at; n > 0 {
|
||||
t.Errorf("the job fired %d more times after shutdown: the scheduler is still running", n)
|
||||
}
|
||||
}
|
||||
@@ -26,10 +26,9 @@ func InitRouter() {
|
||||
os.Exit(-1)
|
||||
}
|
||||
|
||||
authMiddleware, err := common.AuthInit()
|
||||
if err != nil {
|
||||
log.Fatalf("JWT Init Error, %s", err.Error())
|
||||
}
|
||||
// the jwt middleware: shared instance InitMiddleware built at startup,
|
||||
// not one built here per module (see common/middleware.GetAuthMiddleware).
|
||||
authMiddleware := common.GetAuthMiddleware()
|
||||
|
||||
// 注册业务路由
|
||||
initRouter(r, authMiddleware)
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
|
||||
"go-admin/common/middleware"
|
||||
)
|
||||
|
||||
// registeredRoutes builds the generator's routes on an engine of its own and
|
||||
// reports the patterns they were registered under.
|
||||
//
|
||||
// The mode has to be given rather than inherited, because it now decides what
|
||||
// gets registered: a test that leaves it at the zero value would be asking
|
||||
// about a mode no deployment runs in, and would pass whether or not the gate
|
||||
// works.
|
||||
//
|
||||
// It is put back before this returns, not at the end of the test. t.Cleanup
|
||||
// would leave the mode set for everything the caller does afterwards, so a
|
||||
// caller that went on to assert something mode-dependent would be reading a
|
||||
// value this helper left behind rather than one it chose. A caller that does
|
||||
// want the mode set has to set it, which is visible where it happens.
|
||||
//
|
||||
// The JWT middleware is a zero value. MiddlewareFunc only closes over the
|
||||
// receiver and is never called here - no request is served, the engine is
|
||||
// asked what it has - so nothing dereferences it.
|
||||
func registeredRoutes(t *testing.T, mode string) map[string]bool {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
previous := config.ApplicationConfig.Mode
|
||||
defer func() { config.ApplicationConfig.Mode = previous }()
|
||||
config.ApplicationConfig.Mode = mode
|
||||
|
||||
r := gin.New()
|
||||
v1 := r.Group("/api/v1")
|
||||
sysNoCheckRoleRouter(v1, &jwt.GinJWTMiddleware{})
|
||||
registerDBRouter(v1, &jwt.GinJWTMiddleware{})
|
||||
|
||||
out := map[string]bool{}
|
||||
for _, route := range r.Routes() {
|
||||
out[route.Path] = true
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// The list of routes demo mode refuses lives in common/middleware, which may
|
||||
// not import app/ and therefore cannot see whether any of them is still a
|
||||
// route. This is the half that can be checked, and it is checked here because
|
||||
// this is where the routes are declared: rename one, and the entry over there
|
||||
// stops matching anything, demo mode silently starts serving it again, and
|
||||
// nothing else would say so.
|
||||
func TestEveryRouteDemoModeRefusesStillExists(t *testing.T) {
|
||||
routes := registeredRoutes(t, "demo")
|
||||
for _, guarded := range middleware.DemoWriteRoutes() {
|
||||
if !routes[guarded] {
|
||||
t.Errorf("demo mode refuses %q, but no route is registered under that pattern - "+
|
||||
"either it was renamed, or it moved to another file; the guard now matches nothing",
|
||||
guarded)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The other direction, and the one the demo host cares about: the generator's
|
||||
// read-only routes have to stay reachable, or a demo deployment cannot show
|
||||
// the feature at all. Refusing too much is as much of a defect as refusing too
|
||||
// little.
|
||||
func TestTheGeneratorsReadOnlyRoutesAreNotRefused(t *testing.T) {
|
||||
refused := map[string]bool{}
|
||||
for _, guarded := range middleware.DemoWriteRoutes() {
|
||||
refused[guarded] = true
|
||||
}
|
||||
|
||||
for _, readOnly := range []string{
|
||||
"/api/v1/gen/preview/:tableId",
|
||||
"/api/v1/gen/tabletree",
|
||||
"/api/v1/db/tables/page",
|
||||
"/api/v1/db/columns/page",
|
||||
} {
|
||||
if !registeredRoutes(t, "demo")[readOnly] {
|
||||
t.Fatalf("%s is not registered, so this test is asserting against nothing", readOnly)
|
||||
}
|
||||
if refused[readOnly] {
|
||||
t.Errorf("demo mode refuses %s, which only reads - the demo host needs it to "+
|
||||
"demonstrate the generator", readOnly)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -3,15 +3,49 @@ package router
|
||||
import (
|
||||
"github.com/gin-gonic/gin"
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
|
||||
"go-admin/app/admin/apis"
|
||||
"go-admin/app/other/apis/tools"
|
||||
)
|
||||
|
||||
// GenWriteRoutesEnabled reports whether the code generator's writing endpoints
|
||||
// are registered in this process.
|
||||
//
|
||||
// Three of the generator's endpoints do not read. /gen/toproject writes seven
|
||||
// Go and Vue source files onto this host, one of them under the path
|
||||
// gen.frontpath names; /gen/apitofile writes a migration; /gen/todb inserts
|
||||
// menus and APIs. All three are GET, all three are listed in CasbinExclude,
|
||||
// and AuthCheckRole skips what is on that list - so Enforce never runs for
|
||||
// them and any account that can log in may call them. That is a bargain a
|
||||
// workstation can make and a deployment cannot.
|
||||
//
|
||||
// dev is the shipped default and is where the generator is meant to be used.
|
||||
// demo keeps them because demo mode already has a better answer than a 404:
|
||||
// DemoEvn refuses these three by name and explains itself, which is the thing
|
||||
// the demo exists to show. test and prod get nothing.
|
||||
//
|
||||
// The mode is read once, while the routes are being built. Changing
|
||||
// application.mode in a running process adds and removes nothing - a
|
||||
// configuration reload rebuilds neither the engine nor its routes.
|
||||
//
|
||||
// core has constants for dev, test and prod but none for demo, which this
|
||||
// repository spells as a literal in common/middleware/demo.go. Both are
|
||||
// literals here so that the two read as one set rather than two conventions.
|
||||
func GenWriteRoutesEnabled() bool {
|
||||
switch config.ApplicationConfig.Mode {
|
||||
case "dev", "demo":
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func init() {
|
||||
routerCheckRole = append(routerCheckRole, sysNoCheckRoleRouter, registerDBRouter, registerSysTableRouter)
|
||||
}
|
||||
|
||||
func sysNoCheckRoleRouter(v1 *gin.RouterGroup ,authMiddleware *jwt.GinJWTMiddleware) {
|
||||
func sysNoCheckRoleRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddleware) {
|
||||
r1 := v1.Group("")
|
||||
{
|
||||
sys := apis.System{}
|
||||
@@ -22,9 +56,11 @@ func sysNoCheckRoleRouter(v1 *gin.RouterGroup ,authMiddleware *jwt.GinJWTMiddlew
|
||||
{
|
||||
gen := tools.Gen{}
|
||||
r.GET("/gen/preview/:tableId", gen.Preview)
|
||||
r.GET("/gen/toproject/:tableId", gen.GenCode)
|
||||
r.GET("/gen/apitofile/:tableId", gen.GenApiToFile)
|
||||
r.GET("/gen/todb/:tableId", gen.GenMenuAndApi)
|
||||
if GenWriteRoutesEnabled() {
|
||||
r.GET("/gen/toproject/:tableId", gen.GenCode)
|
||||
r.GET("/gen/apitofile/:tableId", gen.GenApiToFile)
|
||||
r.GET("/gen/todb/:tableId", gen.GenMenuAndApi)
|
||||
}
|
||||
sysTable := tools.SysTable{}
|
||||
r.GET("/gen/tabletree", sysTable.GetSysTablesTree)
|
||||
}
|
||||
@@ -53,4 +89,4 @@ func registerSysTableRouter(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddl
|
||||
tablesInfo.GET("", sysTable.GetSysTablesInfo)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
)
|
||||
|
||||
// genWritingRoutes are the three that do not read. They write Go and Vue
|
||||
// source onto the host, a migration, and rows in sys_menu.
|
||||
var genWritingRoutes = []string{
|
||||
"/api/v1/gen/toproject/:tableId",
|
||||
"/api/v1/gen/apitofile/:tableId",
|
||||
"/api/v1/gen/todb/:tableId",
|
||||
}
|
||||
|
||||
// genReadingRoutes are the rest of the generator's surface. Gating the three
|
||||
// above must not cost any of these: a deployment that cannot list its tables
|
||||
// or preview a template has lost the feature, not secured it.
|
||||
var genReadingRoutes = []string{
|
||||
"/api/v1/gen/preview/:tableId",
|
||||
"/api/v1/gen/tabletree",
|
||||
"/api/v1/db/tables/page",
|
||||
"/api/v1/db/columns/page",
|
||||
}
|
||||
|
||||
// The endpoints that write are registered where the mode says development and
|
||||
// nowhere else.
|
||||
//
|
||||
// They are in CasbinExclude, so Enforce never runs for them and any account
|
||||
// that can log in may call them. dev is the shipped default and is where the
|
||||
// generator is meant to be used; demo keeps them because DemoEvn refuses these
|
||||
// three by name and saying so is the thing the demo is for. Everything else,
|
||||
// including the empty mode a process gets when nothing set one, is refused by
|
||||
// not existing.
|
||||
func TestGeneratorWritingRoutesExistOnlyWhereTheModeAllowsIt(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
mode string
|
||||
expected bool
|
||||
why string
|
||||
}{
|
||||
{"dev", true, "the shipped default, and where the generator is used"},
|
||||
{"demo", true, "registered so demo mode can refuse them by name"},
|
||||
{"test", false, "a deployment, however much it is called a test"},
|
||||
{"prod", false, "a deployment"},
|
||||
{"", false, "no mode configured is not a reason to trust the caller"},
|
||||
} {
|
||||
t.Run("mode="+tc.mode, func(t *testing.T) {
|
||||
routes := registeredRoutes(t, tc.mode)
|
||||
for _, writing := range genWritingRoutes {
|
||||
if got := routes[writing]; got != tc.expected {
|
||||
t.Errorf("mode %q: %s registered = %v, want %v (%s)",
|
||||
tc.mode, writing, got, tc.expected, tc.why)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The other direction. Refusing too much is as much of a defect as refusing
|
||||
// too little, and the read-only half of the generator is what a demo shows.
|
||||
func TestGeneratorReadingRoutesExistInEveryMode(t *testing.T) {
|
||||
for _, mode := range []string{"dev", "demo", "test", "prod", ""} {
|
||||
t.Run("mode="+mode, func(t *testing.T) {
|
||||
routes := registeredRoutes(t, mode)
|
||||
for _, reading := range genReadingRoutes {
|
||||
if !routes[reading] {
|
||||
t.Errorf("mode %q: %s is not registered - the gate took a route that only reads",
|
||||
mode, reading)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// GenWriteRoutesEnabled is what cmd/api reads to decide whether to warn at
|
||||
// start-up. If it and the registration ever disagree, the log says one thing
|
||||
// and the engine does another, so they are checked against each other rather
|
||||
// than each against a list.
|
||||
func TestGenWriteRoutesEnabledAgreesWithWhatWasRegistered(t *testing.T) {
|
||||
for _, mode := range []string{"dev", "demo", "test", "prod", ""} {
|
||||
t.Run("mode="+mode, func(t *testing.T) {
|
||||
routes := registeredRoutes(t, mode)
|
||||
|
||||
// registeredRoutes puts the mode back before it returns, so ask
|
||||
// the predicate under a mode set here - about the same value the
|
||||
// engine was just built under.
|
||||
previous := config.ApplicationConfig.Mode
|
||||
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
|
||||
config.ApplicationConfig.Mode = mode
|
||||
|
||||
claimed := GenWriteRoutesEnabled()
|
||||
actual := routes["/api/v1/gen/todb/:tableId"]
|
||||
if claimed != actual {
|
||||
t.Errorf("mode %q: GenWriteRoutesEnabled() = %v but the route was registered = %v",
|
||||
mode, claimed, actual)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The helper restores the mode before it returns, so nothing it was asked
|
||||
// about leaks into what the caller does next.
|
||||
//
|
||||
// Worth a test of its own because the failure is silent: a helper that left
|
||||
// the mode set would make every assertion after the call read a value the
|
||||
// caller did not choose, and each of those assertions would still pass for as
|
||||
// long as the leaked value happened to be the right one.
|
||||
func TestRegisteredRoutesRestoresTheModeBeforeReturning(t *testing.T) {
|
||||
const sentinel = "not-a-mode"
|
||||
|
||||
previous := config.ApplicationConfig.Mode
|
||||
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
|
||||
config.ApplicationConfig.Mode = sentinel
|
||||
|
||||
registeredRoutes(t, "prod")
|
||||
|
||||
if got := config.ApplicationConfig.Mode; got != sentinel {
|
||||
t.Errorf("mode after the helper returned = %q, want %q - it was left set to what "+
|
||||
"the helper was asked about", got, sentinel)
|
||||
}
|
||||
}
|
||||
|
||||
// Changing the mode after the routes were built unregisters nothing.
|
||||
//
|
||||
// buildRouter has one call site, in run(), and route registration is not on
|
||||
// any phase or reload callback - so a configuration reload moves
|
||||
// config.ApplicationConfig.Mode without moving the routes. From that moment
|
||||
// GenWriteRoutesEnabled answers about a mode the engine was not built under.
|
||||
//
|
||||
// That gap is why the start-up warning tells the reader to restart rather than
|
||||
// only to change the mode. This pins it: if registration ever becomes dynamic,
|
||||
// this test fails and the message it justifies has to be revisited.
|
||||
func TestChangingTheModeDoesNotUnregisterWhatWasAlreadyBuilt(t *testing.T) {
|
||||
built := registeredRoutes(t, "dev")
|
||||
if !built["/api/v1/gen/todb/:tableId"] {
|
||||
t.Fatal("built under dev without the writing routes, so this test asserts nothing")
|
||||
}
|
||||
|
||||
previous := config.ApplicationConfig.Mode
|
||||
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
|
||||
config.ApplicationConfig.Mode = "prod"
|
||||
|
||||
if GenWriteRoutesEnabled() {
|
||||
t.Fatal("the predicate still allows prod, so the disagreement below is not the one meant")
|
||||
}
|
||||
if !built["/api/v1/gen/todb/:tableId"] {
|
||||
t.Error("the route left the engine when the mode changed - registration has become " +
|
||||
"dynamic, and the start-up warning's advice to restart is now wrong")
|
||||
}
|
||||
}
|
||||
@@ -25,10 +25,9 @@ func InitRouter() {
|
||||
os.Exit(-1)
|
||||
}
|
||||
// the jwt middleware
|
||||
authMiddleware, err := common.AuthInit()
|
||||
if err != nil {
|
||||
log.Fatalf("JWT Init Error, %s", err.Error())
|
||||
}
|
||||
// the jwt middleware: shared instance InitMiddleware built at startup,
|
||||
// not one built here per module (see common/middleware.GetAuthMiddleware).
|
||||
authMiddleware := common.GetAuthMiddleware()
|
||||
|
||||
// 注册业务路由
|
||||
// TODO: 这里可存放业务路由,里边并无实际路由只有演示代码
|
||||
|
||||
@@ -1,23 +1,82 @@
|
||||
package router
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/tools/transfer"
|
||||
"github.com/prometheus/client_golang/prometheus/promhttp"
|
||||
|
||||
"go-admin/common/health"
|
||||
)
|
||||
|
||||
func init() {
|
||||
routerNoCheckRole = append(routerNoCheckRole, registerMonitorRouter)
|
||||
routerNoCheckRole = append(routerNoCheckRole, RegisterMonitorRouter)
|
||||
}
|
||||
|
||||
// 需认证的路由代码
|
||||
func registerMonitorRouter(v1 *gin.RouterGroup) {
|
||||
// readyTimeout bounds the whole probe. What constrains it is the orchestrator's
|
||||
// per-check timeout rather than its polling period: Kubernetes allows a probe
|
||||
// one second by default, so a dependency that answers in 1.2s is recorded as a
|
||||
// failed check however promptly this handler returns. A manifest that mounts
|
||||
// this probe has to raise timeoutSeconds above this value, and
|
||||
// scripts/k8s/deploy.yml does.
|
||||
const readyTimeout = 2 * time.Second
|
||||
|
||||
// HealthPath and ReadyPath are the two probe routes, relative to APIPrefix.
|
||||
//
|
||||
// Exported for the same reason as the prefix: the rate limiter has to be told
|
||||
// to skip them, and it is installed in a package that cannot import this one.
|
||||
const (
|
||||
HealthPath = "/health"
|
||||
ReadyPath = "/ready"
|
||||
)
|
||||
|
||||
// RegisterMonitorRouter mounts the metrics endpoint and the two probes on v1.
|
||||
//
|
||||
// Exported so that a test can put the real probes on a server of its own. The
|
||||
// alternative - a test that re-implements the handler it means to check - is
|
||||
// how a probe comes to be asserted against a copy of itself.
|
||||
//
|
||||
// 无需认证的路由代码
|
||||
func RegisterMonitorRouter(v1 *gin.RouterGroup) {
|
||||
v1.GET("/metrics", transfer.Handler(promhttp.Handler()))
|
||||
//健康检查
|
||||
v1.GET("/health", func(c *gin.Context) {
|
||||
|
||||
// 健康检查(存活)
|
||||
//
|
||||
// Stays a bare 200 on purpose. This is the answer to "should I restart
|
||||
// you", and a process whose database is unreachable does not want
|
||||
// restarting - that turns one outage into a crash loop and throws away the
|
||||
// connection pool, the cache and every in-flight request along the way.
|
||||
v1.GET(HealthPath, func(c *gin.Context) {
|
||||
c.Status(http.StatusOK)
|
||||
})
|
||||
|
||||
}
|
||||
// 就绪检查
|
||||
//
|
||||
// The answer to "should I send you requests". It fails while a dependency
|
||||
// is unreachable, and from the moment shutdown begins - for as long as
|
||||
// extend.shutdown.drain says, which is zero unless it is configured. The
|
||||
// package comment in common/health says what that window is worth, and to
|
||||
// whom.
|
||||
v1.GET(ReadyPath, func(c *gin.Context) {
|
||||
if health.Draining() {
|
||||
c.JSON(http.StatusServiceUnavailable, gin.H{
|
||||
"status": "draining",
|
||||
"checks": []health.Check{},
|
||||
})
|
||||
return
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(c.Request.Context(), readyTimeout)
|
||||
defer cancel()
|
||||
|
||||
checks := health.Ready(ctx)
|
||||
status := http.StatusOK
|
||||
if !health.Healthy(checks) {
|
||||
status = http.StatusServiceUnavailable
|
||||
}
|
||||
c.JSON(status, gin.H{"status": http.StatusText(status), "checks": checks})
|
||||
})
|
||||
|
||||
}
|
||||
|
||||
@@ -10,6 +10,13 @@ var (
|
||||
routerCheckRole = make([]func(v1 *gin.RouterGroup, authMiddleware *jwt.GinJWTMiddleware), 0)
|
||||
)
|
||||
|
||||
// APIPrefix is the group every route below is registered under.
|
||||
//
|
||||
// Exported because the middleware chain in cmd/api has to name two of those
|
||||
// routes in full - the rate limiter is installed on the engine and must skip
|
||||
// the probes - and a prefix spelled in two places is a prefix that drifts.
|
||||
const APIPrefix = "/api/v1"
|
||||
|
||||
// initRouter 路由示例
|
||||
func initRouter(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware) *gin.Engine {
|
||||
|
||||
@@ -24,7 +31,7 @@ func initRouter(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware) *gin.Engine
|
||||
// noCheckRoleRouter 无需认证的路由示例
|
||||
func noCheckRoleRouter(r *gin.Engine) {
|
||||
// 可根据业务需求来设置接口版本
|
||||
v1 := r.Group("/api/v1")
|
||||
v1 := r.Group(APIPrefix)
|
||||
|
||||
for _, f := range routerNoCheckRole {
|
||||
f(v1)
|
||||
@@ -34,7 +41,7 @@ func noCheckRoleRouter(r *gin.Engine) {
|
||||
// checkRoleRouter 需要认证的路由示例
|
||||
func checkRoleRouter(r *gin.Engine, authMiddleware *jwt.GinJWTMiddleware) {
|
||||
// 可根据业务需求来设置接口版本
|
||||
v1 := r.Group("/api/v1")
|
||||
v1 := r.Group(APIPrefix)
|
||||
|
||||
for _, f := range routerCheckRole {
|
||||
f(v1, authMiddleware)
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
)
|
||||
|
||||
// The warning fires exactly where the generator's writing endpoints are served
|
||||
// and nothing else refuses them.
|
||||
//
|
||||
// dev is the case the warning exists for: it is the shipped default, so it is
|
||||
// the mode a deployment that changed nothing is running in. demo serves the
|
||||
// routes too, but DemoEvn refuses all three by name, so warning there would
|
||||
// describe an exposure that is not there.
|
||||
func TestGeneratorWriteRoutesWarningFiresWhereTheExposureIs(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
mode string
|
||||
want bool
|
||||
why string
|
||||
}{
|
||||
{"dev", true, "shipped default, endpoints served and not refused"},
|
||||
{"demo", false, "served, but DemoEvn refuses all three"},
|
||||
{"test", false, "not served"},
|
||||
{"prod", false, "not served"},
|
||||
{"", false, "not served"},
|
||||
} {
|
||||
t.Run("mode="+tc.mode, func(t *testing.T) {
|
||||
previous := config.ApplicationConfig.Mode
|
||||
t.Cleanup(func() { config.ApplicationConfig.Mode = previous })
|
||||
config.ApplicationConfig.Mode = tc.mode
|
||||
|
||||
if got := generatorWriteRoutesNeedWarning(); got != tc.want {
|
||||
t.Errorf("mode %q: warning = %v, want %v (%s)", tc.mode, got, tc.want, tc.why)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,160 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
)
|
||||
|
||||
// freePort returns a port nothing is listening on. It is inherently a guess -
|
||||
// the port is free when it is handed back and could be taken a moment later -
|
||||
// but every alternative needs the caller to hold the listener, which is the one
|
||||
// thing these tests cannot do.
|
||||
func freePort(t *testing.T) int {
|
||||
t.Helper()
|
||||
ln, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatalf("probe listen: %v", err)
|
||||
}
|
||||
port := ln.Addr().(*net.TCPAddr).Port
|
||||
_ = ln.Close()
|
||||
return port
|
||||
}
|
||||
|
||||
// AfterListen promises a hook that the port is reachable. Both halves of that
|
||||
// are asserted here, and in one test rather than two, because the phase seals
|
||||
// itself once it has run: a second test calling RunPhase again would find a
|
||||
// closed registry and pass while proving nothing.
|
||||
//
|
||||
// The failing bind comes first for the same reason. It must leave the phase
|
||||
// unsealed, which is only visible if nothing has sealed it yet.
|
||||
func TestAfterListenIsAnnouncedOnlyOnceThePortIsBound(t *testing.T) {
|
||||
// The pause makes the "announced synchronously" claim testable: if the
|
||||
// announcement were moved onto a goroutine, startServing would return
|
||||
// while the hook was still sleeping and the count below would be zero.
|
||||
var ran int
|
||||
sdk.Runtime.SetPhase(runtime.AfterListen, func() {
|
||||
time.Sleep(50 * time.Millisecond)
|
||||
ran++
|
||||
})
|
||||
|
||||
// Somebody else already has the port. Under ListenAndServe this surfaced
|
||||
// on the serving goroutine, far too late to stop the announcement.
|
||||
taken, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatalf("occupy: %v", err)
|
||||
}
|
||||
defer func() { _ = taken.Close() }()
|
||||
|
||||
blocked := &http.Server{Addr: taken.Addr().String(), Handler: http.NewServeMux()}
|
||||
if err := startServing(blocked, false, "", ""); err == nil {
|
||||
t.Fatal("startServing returned no error for a port that was already taken")
|
||||
}
|
||||
if ran != 0 {
|
||||
t.Errorf("AfterListen ran %d times after a failed bind; a hook there is told the port is reachable", ran)
|
||||
}
|
||||
if sdk.Runtime.PhaseSealed(runtime.AfterListen) {
|
||||
t.Error("a failed bind sealed AfterListen, so the phase could never run for a server that did start")
|
||||
}
|
||||
|
||||
// A certificate that cannot be read is the other way to fail before there
|
||||
// is anything to announce. ServeTLS reads it on the serving goroutine, so
|
||||
// without the check in startServing this would be a hook told the port was
|
||||
// reachable while the server was already on its way down.
|
||||
if err := startServing(&http.Server{Addr: "127.0.0.1:0"}, true, "no-such.pem", "no-such.key"); err == nil {
|
||||
t.Fatal("startServing returned no error for a certificate that does not exist")
|
||||
}
|
||||
if ran != 0 {
|
||||
t.Errorf("AfterListen ran %d times after a certificate failure", ran)
|
||||
}
|
||||
if sdk.Runtime.PhaseSealed(runtime.AfterListen) {
|
||||
t.Error("a certificate failure sealed AfterListen")
|
||||
}
|
||||
|
||||
// And now a bind that works.
|
||||
port := freePort(t)
|
||||
srv := &http.Server{Addr: fmt.Sprintf("127.0.0.1:%d", port), Handler: http.NewServeMux()}
|
||||
if err := startServing(srv, false, "", ""); err != nil {
|
||||
t.Fatalf("startServing on a free port: %v", err)
|
||||
}
|
||||
defer func() { _ = srv.Close() }()
|
||||
|
||||
// Checked the instant startServing returns, so this is also the assertion
|
||||
// that it did not return early: an asynchronous announcement would still
|
||||
// be inside the sleep. Synchrony matters because an announcement that
|
||||
// overlaps the wait below could, on a fast SIGTERM, have the shutdown
|
||||
// callbacks finish before the startup ones.
|
||||
if ran != 1 {
|
||||
t.Fatalf("AfterListen ran %d times, want 1", ran)
|
||||
}
|
||||
|
||||
// The claim is not "Serve was called" but "the port answers". Dial it.
|
||||
c, err := net.DialTimeout("tcp", srv.Addr, 5*time.Second)
|
||||
if err != nil {
|
||||
t.Fatalf("AfterListen ran but the port does not answer: %v", err)
|
||||
}
|
||||
_ = c.Close()
|
||||
}
|
||||
|
||||
// BeforeRouter is the last point at which a module can still affect how routes
|
||||
// are built, so it has to run while there is no engine yet. The before registry
|
||||
// is a different moment despite the name: those callbacks run after initRouter
|
||||
// has built the engine.
|
||||
//
|
||||
// The two are two lines apart in buildRouter, and calling them equivalent is a
|
||||
// mistake this repository has already made in writing. Until this test the
|
||||
// ordering was checked by reading - which is how the stop signals came to be
|
||||
// armed after the readiness banner in the same file.
|
||||
func TestBeforeRouterRunsWhileThereIsNoEngine(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
// AuthInit reads these two package-level values and nothing else. No
|
||||
// database is involved in building a router: the handlers are registered,
|
||||
// not called.
|
||||
config.ApplicationConfig.Mode = "dev"
|
||||
config.JwtConfig.Secret = "test-secret-for-the-router-build"
|
||||
|
||||
type observation struct {
|
||||
ran int
|
||||
engineWas interface{}
|
||||
engineSeen bool
|
||||
}
|
||||
var phase, before observation
|
||||
|
||||
sdk.Runtime.SetPhase(runtime.BeforeRouter, func() {
|
||||
phase.ran++
|
||||
phase.engineWas = sdk.Runtime.GetEngine()
|
||||
phase.engineSeen = true
|
||||
})
|
||||
sdk.Runtime.SetBefore(func() {
|
||||
before.ran++
|
||||
before.engineWas = sdk.Runtime.GetEngine()
|
||||
before.engineSeen = true
|
||||
})
|
||||
|
||||
buildRouter()
|
||||
|
||||
if phase.ran != 1 {
|
||||
t.Fatalf("BeforeRouter ran %d times, want 1", phase.ran)
|
||||
}
|
||||
if !phase.engineSeen || phase.engineWas != nil {
|
||||
t.Errorf("BeforeRouter saw engine %v, want nil: it is meant to run before initRouter builds one", phase.engineWas)
|
||||
}
|
||||
|
||||
if before.ran != 1 {
|
||||
t.Fatalf("the before registry ran %d times, want 1", before.ran)
|
||||
}
|
||||
if before.engineWas == nil {
|
||||
t.Error("a before callback saw no engine; that registry is meant to run after initRouter, and describing it as equivalent to BeforeRouter is the error this asserts against")
|
||||
}
|
||||
|
||||
if sdk.Runtime.GetEngine() == nil {
|
||||
t.Error("buildRouter returned with no engine built")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,167 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
|
||||
)
|
||||
|
||||
// recordingQueue records what was done to it, in order. Register and Run are
|
||||
// the two calls whose order is the point of this file; Append and Shutdown are
|
||||
// here to satisfy the interface.
|
||||
type recordingQueue struct {
|
||||
mu sync.Mutex
|
||||
events []string
|
||||
ran chan struct{}
|
||||
}
|
||||
|
||||
func newRecordingQueue() *recordingQueue {
|
||||
return &recordingQueue{ran: make(chan struct{}, 4)}
|
||||
}
|
||||
|
||||
func (q *recordingQueue) record(e string) {
|
||||
q.mu.Lock()
|
||||
q.events = append(q.events, e)
|
||||
q.mu.Unlock()
|
||||
}
|
||||
|
||||
func (q *recordingQueue) seen() []string {
|
||||
q.mu.Lock()
|
||||
defer q.mu.Unlock()
|
||||
return append([]string(nil), q.events...)
|
||||
}
|
||||
|
||||
func (q *recordingQueue) String() string { return "recording" }
|
||||
func (q *recordingQueue) Append(corestorage.Messager) error { return nil }
|
||||
func (q *recordingQueue) Register(name string, _ corestorage.ConsumerFunc) {
|
||||
q.record("register:" + name)
|
||||
}
|
||||
func (q *recordingQueue) Shutdown() {}
|
||||
|
||||
func (q *recordingQueue) Run() {
|
||||
q.record("run")
|
||||
select {
|
||||
case q.ran <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
|
||||
// waitForRun waits for Run, which is started on a goroutine.
|
||||
func (q *recordingQueue) waitForRun(t *testing.T) {
|
||||
t.Helper()
|
||||
select {
|
||||
case <-q.ran:
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatalf("Run was never called; saw: %v", q.seen())
|
||||
}
|
||||
}
|
||||
|
||||
// The consumers must be registered before the queue is started. A queue that
|
||||
// is already running refuses further registration - the contract
|
||||
// implementations answer storage.ErrQueueAlreadyStarted - and the legacy
|
||||
// adapter this path goes through drops that error, so the wrong order loses
|
||||
// consumers with nothing said about it. The memory backend does not care,
|
||||
// which is exactly why this cannot be left to be noticed in use.
|
||||
func TestConsumersAreRegisteredBeforeTheQueueIsStarted(t *testing.T) {
|
||||
attachedQueue.Store(0)
|
||||
t.Cleanup(func() { attachedQueue.Store(0) })
|
||||
|
||||
q := newRecordingQueue()
|
||||
attachConsumersOnce(1, q)
|
||||
q.waitForRun(t)
|
||||
|
||||
seen := q.seen()
|
||||
runAt := -1
|
||||
registers := 0
|
||||
for i, e := range seen {
|
||||
switch {
|
||||
case e == "run":
|
||||
if runAt < 0 {
|
||||
runAt = i
|
||||
}
|
||||
case strings.HasPrefix(e, "register:"):
|
||||
registers++
|
||||
if runAt >= 0 {
|
||||
t.Errorf("%q came after Run; a running queue refuses registration", e)
|
||||
}
|
||||
}
|
||||
}
|
||||
if registers != 3 {
|
||||
t.Errorf("registered %d consumers, want 3; saw %v", registers, seen)
|
||||
}
|
||||
if runAt < 0 {
|
||||
t.Errorf("the queue was never started; saw %v", seen)
|
||||
}
|
||||
}
|
||||
|
||||
// AfterResource runs again on every configuration reload, so the hook has to
|
||||
// be idempotent with respect to a given queue - not "does nothing the second
|
||||
// time". Registering twice on the same queue would give every message two
|
||||
// consumers and write every log row twice.
|
||||
func TestTheSameQueueIsNotGivenConsumersTwice(t *testing.T) {
|
||||
attachedQueue.Store(0)
|
||||
t.Cleanup(func() { attachedQueue.Store(0) })
|
||||
|
||||
q := newRecordingQueue()
|
||||
attachConsumersOnce(1, q)
|
||||
q.waitForRun(t)
|
||||
attachConsumersOnce(1, q)
|
||||
|
||||
// Nothing to wait for on the second call, so give a wrong implementation
|
||||
// the time it would need to show up.
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
if n := len(q.seen()); n != 4 {
|
||||
t.Errorf("%d calls after attaching twice to the same queue, want 4 (3 registers + 1 run); saw %v", n, q.seen())
|
||||
}
|
||||
}
|
||||
|
||||
// The other half of the same rule: a reload builds a new adapter, and the
|
||||
// consumers on the old one are attached to a queue nobody publishes to any
|
||||
// more. A new generation must get its own set.
|
||||
func TestANewQueueGetsItsOwnConsumers(t *testing.T) {
|
||||
attachedQueue.Store(0)
|
||||
t.Cleanup(func() { attachedQueue.Store(0) })
|
||||
|
||||
first := newRecordingQueue()
|
||||
attachConsumersOnce(1, first)
|
||||
first.waitForRun(t)
|
||||
|
||||
second := newRecordingQueue()
|
||||
attachConsumersOnce(2, second)
|
||||
second.waitForRun(t)
|
||||
|
||||
if n := len(second.seen()); n != 4 {
|
||||
t.Errorf("the queue from the second generation saw %d calls, want 4; saw %v", n, second.seen())
|
||||
}
|
||||
if n := len(first.seen()); n != 4 {
|
||||
t.Errorf("the queue from the first generation saw %d calls, want 4 - it should not have been touched again; saw %v", n, first.seen())
|
||||
}
|
||||
}
|
||||
|
||||
// Generation 0 means the configuration has no queue section at all, so nothing
|
||||
// was installed and the runtime hands back its own memory queue. That case
|
||||
// still has to get consumers - the registration it replaces was unconditional,
|
||||
// and dropping it would stop the login and operation logs for anyone who
|
||||
// commented the section out.
|
||||
func TestAnUnconfiguredQueueStillGetsConsumers(t *testing.T) {
|
||||
attachedQueue.Store(0)
|
||||
t.Cleanup(func() { attachedQueue.Store(0) })
|
||||
|
||||
q := newRecordingQueue()
|
||||
attachConsumersOnce(0, q)
|
||||
q.waitForRun(t)
|
||||
|
||||
if n := len(q.seen()); n != 4 {
|
||||
t.Errorf("an unconfigured queue saw %d calls, want 4; saw %v", n, q.seen())
|
||||
}
|
||||
|
||||
// And still only once.
|
||||
attachConsumersOnce(0, q)
|
||||
time.Sleep(200 * time.Millisecond)
|
||||
if n := len(q.seen()); n != 4 {
|
||||
t.Errorf("generation 0 was attached to twice: %d calls, want 4; saw %v", n, q.seen())
|
||||
}
|
||||
}
|
||||
+498
-44
@@ -2,10 +2,14 @@ package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
@@ -13,16 +17,21 @@ import (
|
||||
log "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/api"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/bootstrap"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
|
||||
"github.com/pkg/errors"
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/app/admin/router"
|
||||
"go-admin/app/jobs"
|
||||
otherrouter "go-admin/app/other/router"
|
||||
"go-admin/common/database"
|
||||
"go-admin/common/global"
|
||||
"go-admin/common/health"
|
||||
common "go-admin/common/middleware"
|
||||
"go-admin/common/middleware/handler"
|
||||
"go-admin/common/storage"
|
||||
@@ -59,46 +68,124 @@ func init() {
|
||||
func setup() {
|
||||
// 注入配置扩展项
|
||||
config.ExtendConfig = &ext.ExtConfig
|
||||
|
||||
// Registered before the configuration is read. SetupConfig announces
|
||||
// AfterResource as soon as the callbacks that build the resources have
|
||||
// run, so a hook added after that call would miss the first round and the
|
||||
// queue would have no consumers until somebody edited the config file.
|
||||
sdk.Runtime.SetPhase(runtime.AfterResource, attachQueueConsumers)
|
||||
|
||||
// On AfterListen rather than on a bare goroutine from run(). Two reasons:
|
||||
// the phase runs behind core's panic guard, which does not reach across a
|
||||
// goroutine boundary - a panic while loading jobs used to take the whole
|
||||
// process down with a stack that named this file - and the jobs it starts
|
||||
// can call the API, which is only true once the socket is accepting.
|
||||
sdk.Runtime.SetPhase(runtime.AfterListen, startCronJobs)
|
||||
|
||||
//1. 读取配置
|
||||
config.Setup(
|
||||
bootstrap.SetupConfig(
|
||||
file.NewSource(file.WithPath(configYml)),
|
||||
database.Setup,
|
||||
storage.Setup,
|
||||
)
|
||||
//注册监听函数
|
||||
queue := sdk.Runtime.GetQueuePrefix("")
|
||||
queue.Register(global.LoginLog, models.SaveLoginLog)
|
||||
queue.Register(global.OperateLog, models.SaveOperaLog)
|
||||
queue.Register(global.ApiCheck, models.SaveSysApi)
|
||||
go queue.Run()
|
||||
|
||||
usageStr := `starting api server...`
|
||||
log.Info(usageStr)
|
||||
}
|
||||
|
||||
// startCronJobs registers the job implementations and starts a scheduler for
|
||||
// every tenant database.
|
||||
//
|
||||
// It is synchronous, like the phase that runs it. jobs.Setup returns now that
|
||||
// the `select {}` at the end of its per-tenant setup is gone, which is what
|
||||
// makes that possible; while it was there this could only be a goroutine, and
|
||||
// a goroutine is outside the panic guard.
|
||||
func startCronJobs() {
|
||||
jobs.InitJob()
|
||||
jobs.Setup(sdk.Runtime.GetAllDb())
|
||||
}
|
||||
|
||||
// attachedQueue is the queue generation the consumers are attached to, plus
|
||||
// one, so that the zero value means "attached to nothing yet". Written from
|
||||
// the goroutine running the phase, read from the next one - rounds never
|
||||
// overlap, but they are not the same goroutine.
|
||||
var attachedQueue atomic.Uint64
|
||||
|
||||
// attachQueueConsumers registers the log consumers against the queue that is
|
||||
// current, and starts it.
|
||||
//
|
||||
// It runs on AfterResource, so it runs again after every configuration reload
|
||||
// - and it has to. A reload rebuilds the queue adapter, and consumers
|
||||
// registered against the one that existed at start-up are attached to an
|
||||
// adapter nobody publishes to any more, so the login and operation logs stop
|
||||
// being written with nothing said about it.
|
||||
//
|
||||
// It is therefore idempotent with respect to a given queue rather than "does
|
||||
// nothing the second time": a new adapter gets a fresh set of consumers, the
|
||||
// same one gets none. Registering twice on the same queue would give every
|
||||
// message two consumers and write every log row twice.
|
||||
//
|
||||
// Generation 0 means the configuration has no queue section, so nothing was
|
||||
// installed and GetQueuePrefix hands back the runtime's own memory queue.
|
||||
// That case still gets consumers - it is what the previous unconditional
|
||||
// registration did, and dropping it would silently stop logging for anyone who
|
||||
// commented the section out - it just never gets them twice.
|
||||
func attachQueueConsumers() {
|
||||
attachConsumersOnce(storage.QueueGeneration(), sdk.Runtime.GetQueuePrefix(""))
|
||||
}
|
||||
|
||||
// attachConsumersOnce puts the log consumers on q and starts it, unless gen
|
||||
// says this queue already has them.
|
||||
//
|
||||
// Split out from attachQueueConsumers so that the order and the once-ness can
|
||||
// be checked against a queue the test controls: the sequence that matters here
|
||||
// cannot be read back out of a real adapter.
|
||||
func attachConsumersOnce(gen uint64, q corestorage.AdapterQueue) {
|
||||
if attachedQueue.Load() == gen+1 {
|
||||
return
|
||||
}
|
||||
attachedQueue.Store(gen + 1)
|
||||
|
||||
//注册监听函数
|
||||
q.Register(global.LoginLog, models.SaveLoginLog)
|
||||
q.Register(global.OperateLog, models.SaveOperaLog)
|
||||
q.Register(global.ApiCheck, models.SaveSysApi)
|
||||
|
||||
// Started only now, and by whoever registered. setupQueue deliberately
|
||||
// leaves it stopped: a queue that is already running refuses further
|
||||
// registration, and the adapter in this path drops that error on the
|
||||
// floor, so starting first loses consumers without a word.
|
||||
go q.Run()
|
||||
}
|
||||
|
||||
func run() error {
|
||||
// Resolved first, and used both for the line it prints and for the
|
||||
// shutdown that spends it. Reading the configuration again at signal time
|
||||
// would let the two disagree, and the sum that gets printed is the whole
|
||||
// point of printing it.
|
||||
//
|
||||
// Refused rather than corrected, and refused before anything is built: a
|
||||
// budget that cannot be spent as written is a configuration error, and the
|
||||
// moment to say so is while nothing depends on this process yet.
|
||||
seconds, err := ext.ExtConfig.Shutdown.Budget()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
reportShutdownBudget(seconds)
|
||||
|
||||
if config.ApplicationConfig.Mode == pkg.ModeProd.String() {
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
}
|
||||
initRouter()
|
||||
|
||||
for _, f := range AppRouters {
|
||||
f()
|
||||
}
|
||||
buildRouter()
|
||||
reportGeneratorWriteRoutes()
|
||||
|
||||
srv := &http.Server{
|
||||
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
|
||||
Handler: sdk.Runtime.GetEngine(),
|
||||
Addr: fmt.Sprintf("%s:%d", config.ApplicationConfig.Host, config.ApplicationConfig.Port),
|
||||
Handler: sdk.Runtime.GetEngine(),
|
||||
ReadTimeout: time.Duration(config.ApplicationConfig.ReadTimeout) * time.Second,
|
||||
WriteTimeout: time.Duration(config.ApplicationConfig.WriterTimeout) * time.Second,
|
||||
}
|
||||
|
||||
go func() {
|
||||
jobs.InitJob()
|
||||
jobs.Setup(sdk.Runtime.GetAllDb())
|
||||
|
||||
}()
|
||||
|
||||
if apiCheck {
|
||||
var routers = sdk.Runtime.GetRouter()
|
||||
q := sdk.Runtime.GetQueuePrefix("")
|
||||
@@ -116,18 +203,17 @@ func run() error {
|
||||
}
|
||||
}
|
||||
|
||||
go func() {
|
||||
// 服务连接
|
||||
if config.SslConfig.Enable {
|
||||
if err := srv.ListenAndServeTLS(config.SslConfig.Pem, config.SslConfig.KeyStr); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Fatal("listen: ", err)
|
||||
}
|
||||
} else {
|
||||
if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
log.Fatal("listen: ", err)
|
||||
}
|
||||
}
|
||||
}()
|
||||
// Armed before the server starts serving, and well before the readiness
|
||||
// banner: a signal arriving between "the process is up" and "the process
|
||||
// is listening for signals" reaches the default handler and kills it
|
||||
// without any of the shutdown below. That window is the whole reason
|
||||
// arming is separate from waiting.
|
||||
quit, disarmStopSignals := armStopSignals()
|
||||
|
||||
if err := startServing(srv, config.SslConfig.Enable, config.SslConfig.Pem, config.SslConfig.KeyStr); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fmt.Println(pkg.Red(string(global.LogoContent)))
|
||||
tip()
|
||||
fmt.Println(pkg.Green("Server run at:"))
|
||||
@@ -137,23 +223,363 @@ func run() error {
|
||||
fmt.Printf("- Local: http://localhost:%d/swagger/admin/index.html \r\n", config.ApplicationConfig.Port)
|
||||
fmt.Printf("- Network: %s://%s:%d/swagger/admin/index.html \r\n", "http", pkg.GetLocalHost(), config.ApplicationConfig.Port)
|
||||
fmt.Printf("%s Enter Control + C Shutdown Server \r\n", pkg.GetCurrentTimeStr())
|
||||
// 等待中断信号以优雅地关闭服务器(设置 5 秒的超时时间)
|
||||
quit := make(chan os.Signal, 1)
|
||||
signal.Notify(quit, os.Interrupt)
|
||||
|
||||
<-quit
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
log.Info("Shutdown Server ... ")
|
||||
|
||||
if err := srv.Shutdown(ctx); err != nil {
|
||||
log.Fatal("Server Shutdown:", err)
|
||||
serverErr, cleanupErr := gracefulShutdown(srv, quit, disarmStopSignals, budgetFrom(seconds))
|
||||
if serverErr != nil {
|
||||
// Not log.Fatal: that is an unconditional os.Exit(1), and Shutdown
|
||||
// reports an error exactly when connections were still in flight -
|
||||
// which is when the cleanup that ran after it mattered most.
|
||||
log.Error("Server Shutdown: ", serverErr)
|
||||
}
|
||||
if cleanupErr != nil {
|
||||
log.Error("Cleanup: ", cleanupErr)
|
||||
}
|
||||
log.Info("Server exiting")
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// budget is the three waits a shutdown spends, in the order it spends them.
|
||||
type budget struct {
|
||||
drain time.Duration
|
||||
server time.Duration
|
||||
cleanup time.Duration
|
||||
}
|
||||
|
||||
// budgetFrom turns the resolved seconds into the durations the sequence waits
|
||||
// on.
|
||||
func budgetFrom(s ext.ShutdownBudget) budget {
|
||||
return budget{
|
||||
drain: time.Duration(s.Drain) * time.Second,
|
||||
server: time.Duration(s.Server) * time.Second,
|
||||
cleanup: time.Duration(s.Cleanup) * time.Second,
|
||||
}
|
||||
}
|
||||
|
||||
// defaultBudget is what a process with no extend.shutdown section spends.
|
||||
func defaultBudget() budget {
|
||||
return budget{drain: drainTimeout, server: shutdownTimeout, cleanup: cleanupTimeout}
|
||||
}
|
||||
|
||||
// gracefulShutdown takes the process down in the order that gives something
|
||||
// else a chance to notice first.
|
||||
//
|
||||
// The whole order lives here, and run() is not the only caller: the signal
|
||||
// tests run this function rather than reproducing it. A test that reproduces a
|
||||
// sequence asserts against its own copy and stays green while the sequence it
|
||||
// was written for regresses.
|
||||
//
|
||||
// The caller has already taken the first signal off quit. quit is handed on
|
||||
// because a second signal during the drain window ends the window early -
|
||||
// somebody sending another kill wants this over with sooner - and because
|
||||
// until the window is over that signal must not reach the default handler and
|
||||
// kill the process outright.
|
||||
//
|
||||
// disarm is therefore called at the end of the window rather than on the first
|
||||
// signal. After it, a second signal is handled by the default disposition
|
||||
// again, which is the only way out of a Shutdown or a cleanup callback that
|
||||
// never returns. Restoring it any earlier would put every ordinary shutdown
|
||||
// inside that escape hatch for the whole length of the drain, where before
|
||||
// this window existed only a hung callback could reach it.
|
||||
//
|
||||
// The two waits' errors are returned separately rather than logged: they fail
|
||||
// for different reasons, and the caller decides what each is worth.
|
||||
func gracefulShutdown(srv *http.Server, quit <-chan os.Signal, disarm func(), b budget) (serverErr, cleanupErr error) {
|
||||
// Said before anything is taken apart. A configuration reload arriving in
|
||||
// this window would otherwise re-run AfterResource - rebuilding the pool
|
||||
// and the queue adapter, and re-registering consumers - on top of cleanup
|
||||
// that has already run.
|
||||
sdk.Runtime.BeginShutdown()
|
||||
|
||||
// Readiness fails from here, which is before the server stops accepting.
|
||||
// That order is necessary and not sufficient: with nothing between this
|
||||
// line and the listener closing, the two are microseconds apart and a
|
||||
// poller on a multi-second interval sees the refused connection instead of
|
||||
// the 503. The window below is what turns the order into something
|
||||
// observable - extend.shutdown.drain, which is zero unless it is
|
||||
// configured.
|
||||
health.BeginDraining()
|
||||
|
||||
// Keep-alive off for the same window, and for the same reason. The server
|
||||
// keeps connections alive while !disableKeepAlives && !shuttingDown(), and
|
||||
// shuttingDown() is only set by Shutdown itself - so without this line
|
||||
// every pooled connection stays open for the whole drain and is cut at the
|
||||
// end of it anyway, which is the cost of the window without its benefit.
|
||||
// This is the switch Shutdown flips, moved earlier by the window's length:
|
||||
// answers now carry Connection: close, and the idle connections a balancer
|
||||
// is holding are closed at once rather than when it next tries to use one.
|
||||
srv.SetKeepAlivesEnabled(false)
|
||||
|
||||
drain(quit, b.drain)
|
||||
|
||||
// Restored here, not on the first signal: from this point a second signal
|
||||
// must reach the default handler, so a shutdown that hangs can still be
|
||||
// interrupted.
|
||||
disarm()
|
||||
|
||||
log.Info("Shutdown Server ... ")
|
||||
serverErr = shutdownServer(srv, b.server)
|
||||
// Runs whether or not the wait above failed, and deliberately so: Shutdown
|
||||
// reports an error exactly when connections were still in flight, which is
|
||||
// when there is most left to clean up after.
|
||||
cleanupErr = runShutdownHooks(b.cleanup)
|
||||
log.Info("Server exiting")
|
||||
|
||||
return serverErr, cleanupErr
|
||||
}
|
||||
|
||||
// drain keeps serving for d, or until another stop signal arrives.
|
||||
//
|
||||
// Requests are answered normally throughout. Refusing them would move the
|
||||
// outage earlier rather than avoid it - the point of the window is that this
|
||||
// instance is still able to work while whoever routes to it stops routing.
|
||||
func drain(quit <-chan os.Signal, d time.Duration) {
|
||||
if d <= 0 {
|
||||
return
|
||||
}
|
||||
log.Infof("Draining for %s: still serving, /ready answers 503 from here", d)
|
||||
|
||||
timer := time.NewTimer(d)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-quit:
|
||||
log.Info("Second signal during the drain window, closing the listener now")
|
||||
case <-timer.C:
|
||||
}
|
||||
}
|
||||
|
||||
// Reference stop grace periods, printed when nothing was configured to compare
|
||||
// against. They are three times apart, which is why the check below needs a
|
||||
// configured value rather than a constant of its own: a budget that overruns
|
||||
// under one of them fits comfortably under the other.
|
||||
const (
|
||||
dockerStopGraceSeconds = 10
|
||||
kubernetesGraceSeconds = 30
|
||||
)
|
||||
|
||||
// reportGeneratorWriteRoutes says whether this process serves the code
|
||||
// generator's writing endpoints, and to whom.
|
||||
//
|
||||
// The endpoints are gated on the mode, and the shipped configuration says dev -
|
||||
// so the deployment most likely to be exposed is the one that changed nothing,
|
||||
// and the one least likely to go looking. Silence there would leave the gate
|
||||
// technically correct and practically useless.
|
||||
//
|
||||
// Nothing is said in demo mode. The routes are registered, but DemoEvn refuses
|
||||
// all three by name, so a warning would describe an exposure that is not there.
|
||||
func reportGeneratorWriteRoutes() {
|
||||
if !generatorWriteRoutesNeedWarning() {
|
||||
return
|
||||
}
|
||||
log.Warnf("the code generator's writing endpoints are served in mode %q: "+
|
||||
"/api/v1/gen/{toproject,apitofile,todb} write Go and Vue source onto this host and rows "+
|
||||
"into this database, and they are in CasbinExclude, so any account that can log in may "+
|
||||
"call them. Set application.mode to prod or test on anything that is not a workstation, "+
|
||||
"then restart: these routes were registered at start-up and a configuration reload does "+
|
||||
"not rebuild them.",
|
||||
config.ApplicationConfig.Mode)
|
||||
}
|
||||
|
||||
// generatorWriteRoutesNeedWarning reports whether there is an exposure to warn
|
||||
// about: the endpoints are served, and nothing else is refusing them.
|
||||
//
|
||||
// Split from the logging so the decision can be tested. A warning nobody can
|
||||
// make fire is indistinguishable from no warning at all, and this one exists
|
||||
// precisely for the case nobody is looking at.
|
||||
func generatorWriteRoutesNeedWarning() bool {
|
||||
return otherrouter.GenWriteRoutesEnabled() && config.ApplicationConfig.Mode != "demo"
|
||||
}
|
||||
|
||||
// reportShutdownBudget states what a shutdown will spend and whether it fits.
|
||||
//
|
||||
// The sum is taken from the resolved values, not from the configuration file:
|
||||
// a field left out of extend.shutdown still costs its default, so adding up
|
||||
// what was written down understates the budget by exactly the fields nobody
|
||||
// wrote.
|
||||
func reportShutdownBudget(s ext.ShutdownBudget) {
|
||||
log.Infof("shutdown budget: drain %ds + server %ds + cleanup %ds = %ds",
|
||||
s.Drain, s.Server, s.Cleanup, s.Total())
|
||||
|
||||
if s.Grace <= 0 {
|
||||
log.Infof("shutdown budget: extend.shutdown.grace is not set, so nothing is compared against it - "+
|
||||
"for reference `docker stop` allows %ds and Kubernetes terminationGracePeriodSeconds defaults to %ds",
|
||||
dockerStopGraceSeconds, kubernetesGraceSeconds)
|
||||
return
|
||||
}
|
||||
if over := s.Overrun(); over > 0 {
|
||||
// A minimum, not a target. This is somebody else's deployment under
|
||||
// constraints this process cannot see, so the honest thing to state is
|
||||
// how much is missing - the repository's own files are where there is
|
||||
// standing to ask for headroom on top, and checksilent does that.
|
||||
log.Warnf("shutdown budget of %ds does not fit inside the %ds of extend.shutdown.grace: "+
|
||||
"SIGKILL arrives while the cleanup callbacks are still running, and the work they "+
|
||||
"were about to finish is lost. It needs at least %ds more, or %ds less budget.",
|
||||
s.Total(), s.Grace, over, over)
|
||||
return
|
||||
}
|
||||
log.Infof("shutdown budget of %ds fits inside the %ds of extend.shutdown.grace", s.Total(), s.Grace)
|
||||
}
|
||||
|
||||
// The budgets a shutdown spends when extend.shutdown configures nothing:
|
||||
// drainTimeout keeps the process serving after the stop signal, then
|
||||
// shutdownTimeout waits for in-flight requests, then cleanupTimeout is what
|
||||
// the BeforeExit callbacks get.
|
||||
//
|
||||
// The seconds come from config, which is where an absent field falls back, so
|
||||
// the default is one number rather than two that can drift apart.
|
||||
//
|
||||
// They are consumed one after the other, so their sum is what has to stay
|
||||
// inside the orchestrator's grace period: `docker stop` allows 10s by default
|
||||
// before it sends SIGKILL, and 0+5+3 leaves room for the process to finish
|
||||
// returning. Raising one without lowering another buys nothing - the budget
|
||||
// that runs out is the orchestrator's, and reportShutdownBudget is what says
|
||||
// so at start-up.
|
||||
var (
|
||||
drainTimeout = time.Duration(ext.DefaultDrainSeconds) * time.Second
|
||||
shutdownTimeout = time.Duration(ext.DefaultServerSeconds) * time.Second
|
||||
cleanupTimeout = time.Duration(ext.DefaultCleanupSeconds) * time.Second
|
||||
)
|
||||
|
||||
// armStopSignals registers for the stop signals and returns the channel they
|
||||
// arrive on together with the function that restores the default disposition.
|
||||
//
|
||||
// SIGTERM is what actually arrives in production: `docker stop`, a Kubernetes
|
||||
// pod deletion and `systemctl stop` all send it, and Go terminates the process
|
||||
// immediately for a signal nobody listens for. Registering only os.Interrupt
|
||||
// meant every graceful shutdown below the wait was dead code outside a
|
||||
// terminal.
|
||||
//
|
||||
// Registering is separate from waiting so a caller can arm before it announces
|
||||
// that it is ready: a signal that arrives between the two is delivered to the
|
||||
// default handler, which for both of these means the process dies without
|
||||
// running any of this.
|
||||
func armStopSignals() (<-chan os.Signal, func()) {
|
||||
quit := make(chan os.Signal, 1)
|
||||
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
|
||||
return quit, func() { signal.Stop(quit) }
|
||||
}
|
||||
|
||||
// startServing binds srv.Addr, hands the listener to srv on its own goroutine,
|
||||
// and announces AfterListen.
|
||||
//
|
||||
// The bind is done here rather than left to ListenAndServe, which binds on the
|
||||
// goroutine that serves. That put the failure every deployment actually hits -
|
||||
// "address already in use" - on a goroutine nobody was reading, so the banner
|
||||
// went on to claim the server was up, and there would be no way to keep
|
||||
// AfterListen from announcing a socket that does not exist. A hook there is
|
||||
// promised a reachable port; the only way to keep that promise is for the bind
|
||||
// to have already happened on this goroutine.
|
||||
//
|
||||
// AfterListen is announced synchronously. Running it in a goroutine to save the
|
||||
// few milliseconds would let it overlap the shutdown: on a fast SIGTERM the
|
||||
// cleanup callbacks could finish before the startup ones had.
|
||||
//
|
||||
// Both ways of failing to start are therefore checked before the announcement:
|
||||
// the bind, and - with ssl enabled - the certificate.
|
||||
func startServing(srv *http.Server, useTLS bool, pem, key string) error {
|
||||
if useTLS {
|
||||
// Read the certificate before anything is announced. ServeTLS reads
|
||||
// these files itself, but on the serving goroutine - so a bad
|
||||
// certificate used to surface after AfterListen had already promised a
|
||||
// reachable port. Loading it here costs one extra read and moves the
|
||||
// failure onto this goroutine, where run() can return it.
|
||||
//
|
||||
// ServeTLS still does the real work below rather than this handing it a
|
||||
// tls.Listener: that is what sets up HTTP/2 negotiation, and taking it
|
||||
// over here would quietly drop h2 for every TLS deployment.
|
||||
if _, err := tls.LoadX509KeyPair(pem, key); err != nil {
|
||||
return errors.Wrap(err, "tls certificate")
|
||||
}
|
||||
}
|
||||
|
||||
ln, err := net.Listen("tcp", srv.Addr)
|
||||
if err != nil {
|
||||
return errors.Wrap(err, "listen")
|
||||
}
|
||||
|
||||
go func() {
|
||||
// 服务连接
|
||||
var err error
|
||||
if useTLS {
|
||||
err = srv.ServeTLS(ln, pem, key)
|
||||
} else {
|
||||
err = srv.Serve(ln)
|
||||
}
|
||||
if err != nil && !errors.Is(err, http.ErrServerClosed) {
|
||||
// Still fatal, as it was. Neither the bind nor the certificate is
|
||||
// among the errors that reach here any more - both are checked
|
||||
// above, on the caller's goroutine. What is left is a serve that
|
||||
// failed after the port was taken, and carrying on would park the
|
||||
// process on <-quit with nothing serving.
|
||||
log.Fatal("serve: ", err)
|
||||
}
|
||||
}()
|
||||
|
||||
sdk.Runtime.RunPhase(runtime.AfterListen)
|
||||
return nil
|
||||
}
|
||||
|
||||
// shutdownServer stops srv, giving in-flight requests up to timeout to finish.
|
||||
//
|
||||
// It returns the error instead of exiting on it. A caller that exits here skips
|
||||
// its own cleanup, and Shutdown fails precisely when there was something left
|
||||
// to clean up after.
|
||||
func shutdownServer(srv *http.Server, timeout time.Duration) error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), timeout)
|
||||
defer cancel()
|
||||
return srv.Shutdown(ctx)
|
||||
}
|
||||
|
||||
// runShutdownHooks runs the BeforeExit callbacks with timeout to share.
|
||||
//
|
||||
// What the budget bounds is the wait, not the work. When it is gone RunShutdown
|
||||
// stops waiting and returns; a callback that never looks at its context carries
|
||||
// on until the process exits, and may leave a partial write behind. Go cannot
|
||||
// cancel a function that does not check for cancellation, which is why the
|
||||
// callbacks are handed a context at all.
|
||||
func runShutdownHooks(timeout time.Duration) error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), timeout)
|
||||
defer cancel()
|
||||
return sdk.Runtime.RunShutdown(ctx)
|
||||
}
|
||||
|
||||
// buildRouter announces BeforeRouter, builds the engine, and then drains the
|
||||
// startup registries.
|
||||
//
|
||||
// The order is the contract. BeforeRouter is the last point at which a module
|
||||
// can still affect how routes are built, so it has to run while there is no
|
||||
// engine yet. The before registry runStartupHooks drains is a different moment
|
||||
// despite the name: those callbacks run after initRouter has built the engine.
|
||||
// Two lines apart, and describing them as equivalent is a mistake this
|
||||
// repository has already made once in writing.
|
||||
func buildRouter() {
|
||||
sdk.Runtime.RunPhase(runtime.BeforeRouter)
|
||||
initRouter()
|
||||
runStartupHooks()
|
||||
}
|
||||
|
||||
// runStartupHooks runs the router registries and then the before callbacks.
|
||||
//
|
||||
// The package-level slice runs first and in its existing order, so a fork that
|
||||
// only ever appended to AppRouters sees no change at all. The core registry
|
||||
// runs second, through RunAppRouters: a module can register through
|
||||
// sdk.Runtime.SetAppRouters and no longer has to import this command package -
|
||||
// which is a main package's plumbing - just to be routed.
|
||||
//
|
||||
// The loop over the core registry now lives in core, which is what brings the
|
||||
// panic guard and the registration seal with it. RunBefore closes a gap rather
|
||||
// than moving one: the open-source edition never executed the before callbacks
|
||||
// at all, so SetBefore was accepted and silently ignored. It has to stay ahead
|
||||
// of ListenAndServe, because a callback registered WithFatal exits the process
|
||||
// and that must not happen to one that is already serving.
|
||||
func runStartupHooks() {
|
||||
for _, f := range AppRouters {
|
||||
f()
|
||||
}
|
||||
sdk.Runtime.RunAppRouters()
|
||||
sdk.Runtime.RunBefore()
|
||||
}
|
||||
|
||||
//var Router runtime.Router
|
||||
|
||||
func tip() {
|
||||
@@ -179,10 +605,38 @@ func initRouter() {
|
||||
r.Use(handler.TlsHandler())
|
||||
}
|
||||
//r.Use(middleware.Metrics())
|
||||
r.Use(common.Sentinel()).
|
||||
r.Use(exemptProbes(common.Sentinel())).
|
||||
Use(common.RequestId(pkg.TrafficKey)).
|
||||
Use(api.SetRequestLogger)
|
||||
|
||||
common.InitMiddleware(r)
|
||||
|
||||
}
|
||||
|
||||
// probePaths are the two routes the rate limiter must not answer for.
|
||||
var probePaths = map[string]bool{
|
||||
otherrouter.APIPrefix + otherrouter.HealthPath: true,
|
||||
otherrouter.APIPrefix + otherrouter.ReadyPath: true,
|
||||
}
|
||||
|
||||
// exemptProbes wraps a middleware so the health and readiness routes skip it.
|
||||
//
|
||||
// The limiter is installed on the engine and the probes are routes like any
|
||||
// other, so above the threshold they are answered with 429 as well. A liveness
|
||||
// probe that collects 429s fails its threshold and the container is restarted,
|
||||
// which takes capacity out of a deployment that is already short of it and
|
||||
// pushes the rest closer to the threshold - the limiter working exactly as
|
||||
// intended is what causes it. It is the argument common/health makes about
|
||||
// restarting a process whose database is unreachable, applied to load.
|
||||
//
|
||||
// Wrapping rather than teaching the limiter about these paths: the limiter
|
||||
// lives under common/, which may not import the package that registers them.
|
||||
func exemptProbes(h gin.HandlerFunc) gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
if probePaths[c.FullPath()] {
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
h(c)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
)
|
||||
|
||||
// freshRuntime hands the test its own Runtime and puts the old one back.
|
||||
//
|
||||
// Both registries close permanently the first time they are run, and
|
||||
// sdk.Runtime is a process-wide singleton, so a test that runs the startup
|
||||
// hooks would otherwise leave every later test in this binary registering into
|
||||
// a closed registry - which is only an ERROR log, not a failure. The symptom
|
||||
// is a test that passes alone and loses its routes when run with the others.
|
||||
func freshRuntime(t *testing.T) {
|
||||
t.Helper()
|
||||
previous := sdk.Runtime
|
||||
t.Cleanup(func() { sdk.Runtime = previous })
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
}
|
||||
|
||||
// Acceptance 1 and 2 together: the package-level slice a fork appends to and
|
||||
// the core registry a module registers through both run, package-level first,
|
||||
// registration order preserved inside each.
|
||||
//
|
||||
// The order matters beyond neatness. A module that appends to AppRouters has to
|
||||
// import go-admin/cmd/api, which is why every module used to need a seven-line
|
||||
// file in the command package; SetAppRouters is the way out of that. Running
|
||||
// the old registry first is what makes the change invisible to anyone who never
|
||||
// takes it.
|
||||
func TestRunStartupHooksRunsBothRegistriesInOrder(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
savedPackage := AppRouters
|
||||
t.Cleanup(func() { AppRouters = savedPackage })
|
||||
|
||||
var order []string
|
||||
AppRouters = []func(){
|
||||
func() { order = append(order, "package-1") },
|
||||
func() { order = append(order, "package-2") },
|
||||
}
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "runtime-1") })
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "runtime-2") })
|
||||
|
||||
runStartupHooks()
|
||||
|
||||
const want = "package-1,package-2,runtime-1,runtime-2"
|
||||
if got := strings.Join(order, ","); got != want {
|
||||
t.Errorf("ran %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 17: a before callback registered through core actually runs.
|
||||
//
|
||||
// It did not, ever: core stored the callbacks and nothing executed them, so
|
||||
// SetBefore was accepted and silently did nothing. The gap survived because
|
||||
// core offered the registry without ever running it, leaving each consumer to
|
||||
// write - or forget - its own loop.
|
||||
func TestBeforeCallbacksRun(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
savedPackage := AppRouters
|
||||
t.Cleanup(func() { AppRouters = savedPackage })
|
||||
AppRouters = nil
|
||||
|
||||
var order []string
|
||||
sdk.Runtime.SetBefore(func() { order = append(order, "before-1") })
|
||||
sdk.Runtime.SetBefore(func() { order = append(order, "before-2") })
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "router") })
|
||||
|
||||
runStartupHooks()
|
||||
|
||||
// Routers first, then before: both happen ahead of ListenAndServe, and a
|
||||
// router callback is what puts the engine in place for anything that comes
|
||||
// after it.
|
||||
const want = "router,before-1,before-2"
|
||||
if got := strings.Join(order, ","); got != want {
|
||||
t.Errorf("ran %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// A panicking module must not take the server down with it. The guard lives in
|
||||
// core; this asserts that go-admin actually goes through it rather than around
|
||||
// it with a loop of its own.
|
||||
func TestAPanickingRouterDoesNotStopStartup(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
savedPackage := AppRouters
|
||||
t.Cleanup(func() { AppRouters = savedPackage })
|
||||
AppRouters = nil
|
||||
|
||||
var order []string
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "first") })
|
||||
sdk.Runtime.SetAppRouters(func() { panic("a third-party module blew up") })
|
||||
sdk.Runtime.SetAppRouters(func() { order = append(order, "third") })
|
||||
sdk.Runtime.SetBefore(func() { order = append(order, "before") })
|
||||
|
||||
runStartupHooks()
|
||||
|
||||
const want = "first,third,before"
|
||||
if got := strings.Join(order, ","); got != want {
|
||||
t.Errorf("ran %q, want %q", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The default AppRouters must keep the admin routes on it. Emptying the slice
|
||||
// would not fail to compile anywhere - it would just serve a server with no
|
||||
// admin API and no error.
|
||||
func TestAdminRouterIsRegisteredOnThePackageSlice(t *testing.T) {
|
||||
if len(AppRouters) == 0 {
|
||||
t.Fatal("AppRouters is empty; the admin router is registered in init()")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
otherrouter "go-admin/app/other/router"
|
||||
"go-admin/common/health"
|
||||
ext "go-admin/config"
|
||||
)
|
||||
|
||||
// The seconds in the configuration and the durations the sequence waits on are
|
||||
// two spellings of one budget, and only one of them is printed at start-up.
|
||||
func TestBudgetFromSeconds(t *testing.T) {
|
||||
got := budgetFrom(ext.ShutdownBudget{Drain: 10, Server: 5, Cleanup: 3})
|
||||
want := budget{
|
||||
drain: 10 * time.Second,
|
||||
server: 5 * time.Second,
|
||||
cleanup: 3 * time.Second,
|
||||
}
|
||||
if got != want {
|
||||
t.Errorf("budgetFrom = %+v, want %+v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The package variables and config.Default*Seconds have to say the same thing.
|
||||
// They are the same default written twice - once as durations for the shutdown
|
||||
// and once as seconds for the fallback - and a deployment that configures
|
||||
// nothing is entitled to one answer, not two.
|
||||
func TestDefaultBudgetIsTheConfiguredFallback(t *testing.T) {
|
||||
unconfigured, err := ext.Shutdown{}.Budget()
|
||||
if err != nil {
|
||||
t.Fatalf("the empty section did not resolve: %v", err)
|
||||
}
|
||||
if got, want := defaultBudget(), budgetFrom(unconfigured); got != want {
|
||||
t.Errorf("defaultBudget = %+v, want the unconfigured budget %+v", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The rate limiter must not answer for the probes.
|
||||
//
|
||||
// It is installed on the engine, so without this the probes are limited like
|
||||
// any other route and answer 429 above the threshold. A liveness probe that
|
||||
// collects 429s fails its threshold and the container is restarted - taking
|
||||
// capacity out of a deployment that is already short of it and pushing the
|
||||
// rest closer to the threshold. The limiter working exactly as designed is
|
||||
// what would cause it.
|
||||
//
|
||||
// The stand-in rejects everything rather than being a real limiter: what is
|
||||
// under test is which requests reach it, and a real one would need the traffic
|
||||
// to cross a threshold before it said anything.
|
||||
func TestTheProbesSkipTheRateLimiter(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
var reached []string
|
||||
r := gin.New()
|
||||
r.Use(exemptProbes(func(c *gin.Context) {
|
||||
reached = append(reached, c.FullPath())
|
||||
c.AbortWithStatus(http.StatusTooManyRequests)
|
||||
}))
|
||||
v1 := r.Group(otherrouter.APIPrefix)
|
||||
otherrouter.RegisterMonitorRouter(v1)
|
||||
v1.GET("/business", func(c *gin.Context) { c.Status(http.StatusOK) })
|
||||
|
||||
for _, tc := range []struct {
|
||||
path string
|
||||
limited bool
|
||||
}{
|
||||
{otherrouter.APIPrefix + otherrouter.HealthPath, false},
|
||||
{otherrouter.APIPrefix + otherrouter.ReadyPath, false},
|
||||
// Not a probe, and deliberately not exempt: the exemption is for the
|
||||
// two routes an orchestrator acts on, not for everything under
|
||||
// /api/v1 that happens to be unauthenticated.
|
||||
{otherrouter.APIPrefix + "/metrics", true},
|
||||
{otherrouter.APIPrefix + "/business", true},
|
||||
} {
|
||||
t.Run(tc.path, func(t *testing.T) {
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, tc.path, nil))
|
||||
|
||||
if tc.limited {
|
||||
if w.Code != http.StatusTooManyRequests {
|
||||
t.Errorf("answered %d, want the middleware's 429 - it was skipped for a route that is not a probe", w.Code)
|
||||
}
|
||||
return
|
||||
}
|
||||
if w.Code == http.StatusTooManyRequests {
|
||||
t.Errorf("answered 429; a probe that can be rate-limited gets the container restarted under load")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Said separately, because a probe could also answer 429 by itself: what
|
||||
// has to be true is that the middleware never saw the request.
|
||||
for _, p := range reached {
|
||||
if probePaths[p] {
|
||||
t.Errorf("the middleware ran for %s", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// /health has to stay 200 while draining, and it is the assertion most easily
|
||||
// lost by accident: making the liveness probe follow the readiness flag reads
|
||||
// like tidying up, and it turns every rolling restart into a kubelet-issued
|
||||
// kill part-way through the drain.
|
||||
func TestHealthStaysUpWhileDraining(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
r := gin.New()
|
||||
v1 := r.Group(otherrouter.APIPrefix)
|
||||
otherrouter.RegisterMonitorRouter(v1)
|
||||
|
||||
ask := func(path string) int {
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, path, nil))
|
||||
return w.Code
|
||||
}
|
||||
|
||||
if got := ask(otherrouter.APIPrefix + otherrouter.HealthPath); got != http.StatusOK {
|
||||
t.Fatalf("/health answered %d before draining, want 200", got)
|
||||
}
|
||||
|
||||
// Process-wide and one-way - nothing clears it - so this is the last thing
|
||||
// in this package that may run in-process and care. Everything else that
|
||||
// exercises draining does so in a child process of its own.
|
||||
health.BeginDraining()
|
||||
|
||||
if got := ask(otherrouter.APIPrefix + otherrouter.HealthPath); got != http.StatusOK {
|
||||
t.Errorf("/health answered %d while draining, want 200 - liveness is "+
|
||||
"\"should I restart you\", and the answer during a drain is no", got)
|
||||
}
|
||||
if got := ask(otherrouter.APIPrefix + otherrouter.ReadyPath); got != http.StatusServiceUnavailable {
|
||||
t.Errorf("/ready answered %d while draining, want 503", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,739 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"syscall"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
|
||||
otherrouter "go-admin/app/other/router"
|
||||
)
|
||||
|
||||
// The signal path cannot be exercised in-process: delivering a signal to the
|
||||
// test binary would race with the test framework, and the disposition changes
|
||||
// are global. So the test re-executes itself as a child, and the child runs
|
||||
// gracefulShutdown - the same function run() runs, not a second copy of the
|
||||
// sequence. A test that reproduces the sequence asserts against its own copy:
|
||||
// move BeginDraining after the drain window and the process regresses while
|
||||
// the test stays green, which is the failure mode this file exists to avoid.
|
||||
//
|
||||
// The child serves the real probe routes on an http.Server of its own rather
|
||||
// than the configured one: this repository's CI has no database
|
||||
// (.github/workflows/go.yml runs neither MySQL nor a sqlite-tagged build), and
|
||||
// none of what is under test needs one. /ready answers 503 either way - with
|
||||
// no database its checks fail - so the assertions below are on the draining
|
||||
// answer specifically, not on the status code alone.
|
||||
const (
|
||||
childEnv = "GO_ADMIN_SIGNAL_CHILD"
|
||||
childStuckEnv = "GO_ADMIN_SIGNAL_CHILD_STUCK"
|
||||
childHangConn = "GO_ADMIN_SIGNAL_CHILD_HANGCONN"
|
||||
childSlowCleanup = "GO_ADMIN_SIGNAL_CHILD_SLOWCLEANUP"
|
||||
childDrainMS = "GO_ADMIN_SIGNAL_CHILD_DRAIN_MS"
|
||||
markerAddr = "CHILD-ADDR"
|
||||
markerReady = "CHILD-READY"
|
||||
markerSignal = "CHILD-SIGNAL"
|
||||
markerShutdown = "CHILD-SHUTDOWN-OK"
|
||||
markerCleanup = "CHILD-CLEANUP-RAN"
|
||||
markerTook = "CHILD-TOOK-NS"
|
||||
markerExiting = "CHILD-EXITING"
|
||||
)
|
||||
|
||||
// childPingRoute is an ordinary route, registered beside the probes so the
|
||||
// window can be checked for what it promises: requests arriving inside it are
|
||||
// served, not refused. Refusing them would move the outage earlier instead of
|
||||
// avoiding it.
|
||||
const childPingRoute = "/signal-test-ping"
|
||||
|
||||
var (
|
||||
readyPath = otherrouter.APIPrefix + otherrouter.ReadyPath
|
||||
healthPath = otherrouter.APIPrefix + otherrouter.HealthPath
|
||||
pingPath = otherrouter.APIPrefix + childPingRoute
|
||||
)
|
||||
|
||||
// TestSignalChild is the child process. It is skipped in a normal run.
|
||||
func TestSignalChild(t *testing.T) {
|
||||
if os.Getenv(childEnv) != "1" {
|
||||
t.Skip("child process entry point")
|
||||
}
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
engine := gin.New()
|
||||
v1 := engine.Group(otherrouter.APIPrefix)
|
||||
otherrouter.RegisterMonitorRouter(v1)
|
||||
v1.GET(childPingRoute, func(c *gin.Context) { c.String(http.StatusOK, "pong") })
|
||||
|
||||
ln, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
fmt.Println("listen:", err)
|
||||
os.Exit(3)
|
||||
}
|
||||
// accepted fires once the server has taken a connection off the listener.
|
||||
// Dialling is not enough: Shutdown only waits for connections the server
|
||||
// has already accepted, so calling it between the dial and the accept
|
||||
// finds nothing to wait for and returns immediately.
|
||||
accepted := make(chan struct{}, 1)
|
||||
srv := &http.Server{
|
||||
Handler: engine,
|
||||
ConnState: func(_ net.Conn, state http.ConnState) {
|
||||
if state == http.StateNew {
|
||||
select {
|
||||
case accepted <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
}
|
||||
},
|
||||
}
|
||||
go func() { _ = srv.Serve(ln) }()
|
||||
|
||||
// The budget the child spends. Nothing here calls bootstrap.SetupConfig, so
|
||||
// with no environment set this is the budget of a deployment that
|
||||
// configures no extend.shutdown section at all.
|
||||
b := defaultBudget()
|
||||
if ms := os.Getenv(childDrainMS); ms != "" {
|
||||
n, err := strconv.Atoi(ms)
|
||||
if err != nil {
|
||||
fmt.Println("drain:", err)
|
||||
os.Exit(4)
|
||||
}
|
||||
b.drain = time.Duration(n) * time.Millisecond
|
||||
}
|
||||
|
||||
// A BeforeExit callback, registered the way a module would. What the tests
|
||||
// below care about is whether it runs at all - after a Shutdown that
|
||||
// failed, and after its own budget has been spent.
|
||||
sdk.Runtime.SetShutdown(func(ctx context.Context) {
|
||||
switch {
|
||||
case os.Getenv(childStuckEnv) == "1":
|
||||
// Stands in for a cleanup hook that never finishes. The point of
|
||||
// restoring the signal disposition after the drain window is that
|
||||
// a second signal still reaches the default handler and kills this.
|
||||
time.Sleep(2 * time.Minute)
|
||||
case os.Getenv(childSlowCleanup) == "1":
|
||||
// Outlasts the budget on purpose, and does not consult ctx - which
|
||||
// is the case the contract is explicit about: what the context
|
||||
// bounds is the wait, not the work.
|
||||
time.Sleep(2 * time.Second)
|
||||
}
|
||||
fmt.Println(markerCleanup)
|
||||
_ = os.Stdout.Sync()
|
||||
})
|
||||
switch {
|
||||
case os.Getenv(childStuckEnv) == "1":
|
||||
b.cleanup = 2 * time.Minute
|
||||
case os.Getenv(childSlowCleanup) == "1":
|
||||
b.cleanup = 300 * time.Millisecond
|
||||
}
|
||||
|
||||
// Arm before announcing readiness. Doing it the other way round leaves a
|
||||
// window in which the parent's signal reaches the default handler and
|
||||
// kills the child before any of this runs - which is exactly the failure
|
||||
// this whole change is about, so the test must not reproduce it by
|
||||
// accident.
|
||||
quit, disarm := armStopSignals()
|
||||
|
||||
fmt.Println(markerAddr, ln.Addr().String())
|
||||
fmt.Println(markerReady)
|
||||
_ = os.Stdout.Sync()
|
||||
|
||||
sig := <-quit
|
||||
fmt.Println(markerSignal, sig)
|
||||
_ = os.Stdout.Sync()
|
||||
|
||||
if os.Getenv(childHangConn) == "1" {
|
||||
// Dialled here, not at start-up. net/http stops counting a StateNew
|
||||
// connection against Shutdown once it is more than five seconds old,
|
||||
// so a connection opened before the wait would age out on a slow CI
|
||||
// run and Shutdown would succeed - leaving the test asserting nothing.
|
||||
c, err := net.Dial("tcp", ln.Addr().String())
|
||||
if err != nil {
|
||||
fmt.Println("dial:", err)
|
||||
os.Exit(5)
|
||||
}
|
||||
defer func() { _ = c.Close() }()
|
||||
|
||||
// And wait for the accept, for the opposite reason: an unaccepted
|
||||
// connection is not one Shutdown waits for either.
|
||||
select {
|
||||
case <-accepted:
|
||||
case <-time.After(10 * time.Second):
|
||||
fmt.Println("the server never accepted the stalling connection")
|
||||
os.Exit(6)
|
||||
}
|
||||
|
||||
// A connection that has sent nothing keeps Shutdown busy: net/http
|
||||
// only treats a StateNew connection as idle once it is more than five
|
||||
// seconds old. A short budget makes the timeout deterministic without
|
||||
// waiting out the real one.
|
||||
b.server = 300 * time.Millisecond
|
||||
}
|
||||
|
||||
started := time.Now()
|
||||
serverErr, cleanupErr := gracefulShutdown(srv, quit, disarm, b)
|
||||
spent := time.Since(started)
|
||||
|
||||
if serverErr != nil {
|
||||
// Deliberately not fatal, and deliberately not a bare return: the
|
||||
// point is that whatever follows still runs.
|
||||
fmt.Println("shutdown error:", serverErr)
|
||||
} else {
|
||||
fmt.Println(markerShutdown)
|
||||
}
|
||||
if cleanupErr != nil {
|
||||
fmt.Println("cleanup error:", cleanupErr)
|
||||
}
|
||||
fmt.Println(markerTook, spent.Nanoseconds())
|
||||
fmt.Println(markerExiting)
|
||||
_ = os.Stdout.Sync()
|
||||
}
|
||||
|
||||
func startChild(t *testing.T, stuck bool, extraEnv ...string) (*exec.Cmd, chan string) {
|
||||
t.Helper()
|
||||
|
||||
r, w, err := os.Pipe()
|
||||
if err != nil {
|
||||
t.Fatalf("pipe: %v", err)
|
||||
}
|
||||
cmd := exec.Command(os.Args[0], "-test.run=TestSignalChild", "-test.v")
|
||||
cmd.Env = append(os.Environ(), childEnv+"=1")
|
||||
if stuck {
|
||||
cmd.Env = append(cmd.Env, childStuckEnv+"=1")
|
||||
}
|
||||
cmd.Env = append(cmd.Env, extraEnv...)
|
||||
cmd.Stdout = w
|
||||
cmd.Stderr = w
|
||||
if err := cmd.Start(); err != nil {
|
||||
t.Fatalf("start child: %v", err)
|
||||
}
|
||||
_ = w.Close()
|
||||
|
||||
lines := make(chan string, 256)
|
||||
go func() {
|
||||
defer close(lines)
|
||||
buf := make([]byte, 4096)
|
||||
var acc strings.Builder
|
||||
for {
|
||||
n, err := r.Read(buf)
|
||||
if n > 0 {
|
||||
acc.Write(buf[:n])
|
||||
for {
|
||||
s := acc.String()
|
||||
i := strings.IndexByte(s, '\n')
|
||||
if i < 0 {
|
||||
break
|
||||
}
|
||||
lines <- s[:i]
|
||||
acc.Reset()
|
||||
acc.WriteString(s[i+1:])
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
if acc.Len() > 0 {
|
||||
lines <- acc.String()
|
||||
}
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
t.Cleanup(func() {
|
||||
_ = cmd.Process.Kill()
|
||||
_, _ = cmd.Process.Wait()
|
||||
_ = r.Close()
|
||||
})
|
||||
return cmd, lines
|
||||
}
|
||||
|
||||
// await drains lines until one contains want, or the deadline passes. It
|
||||
// returns everything it saw, so a failure says what the child actually did,
|
||||
// and the matching line, so a marker can carry a value.
|
||||
func await(t *testing.T, lines chan string, want string, d time.Duration) ([]string, string) {
|
||||
t.Helper()
|
||||
var seen []string
|
||||
deadline := time.After(d)
|
||||
for {
|
||||
select {
|
||||
case l, ok := <-lines:
|
||||
if !ok {
|
||||
t.Fatalf("child output ended before %q; saw:\n%s", want, strings.Join(seen, "\n"))
|
||||
}
|
||||
seen = append(seen, l)
|
||||
if strings.Contains(l, want) {
|
||||
return seen, l
|
||||
}
|
||||
case <-deadline:
|
||||
t.Fatalf("timed out waiting for %q; saw:\n%s", want, strings.Join(seen, "\n"))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// childAddr waits for the address the child is listening on.
|
||||
func childAddr(t *testing.T, lines chan string) string {
|
||||
t.Helper()
|
||||
_, line := await(t, lines, markerAddr, 30*time.Second)
|
||||
fields := strings.Fields(line)
|
||||
return fields[len(fields)-1]
|
||||
}
|
||||
|
||||
// took reads the nanoseconds gracefulShutdown spent, as the child measured
|
||||
// them. Measured inside the child on purpose: the parent's own clock includes
|
||||
// process scheduling, which is the noise the tightest assertion here cannot
|
||||
// afford.
|
||||
func took(t *testing.T, lines chan string, d time.Duration) time.Duration {
|
||||
t.Helper()
|
||||
_, line := await(t, lines, markerTook, d)
|
||||
fields := strings.Fields(line)
|
||||
ns, err := strconv.ParseInt(fields[len(fields)-1], 10, 64)
|
||||
if err != nil {
|
||||
t.Fatalf("unreadable %s line %q: %v", markerTook, line, err)
|
||||
}
|
||||
return time.Duration(ns)
|
||||
}
|
||||
|
||||
// sample is one answer, or the refusal that replaced it.
|
||||
type sample struct {
|
||||
at time.Time
|
||||
path string
|
||||
// status is zero when the connection could not be made at all, which is
|
||||
// what a closed listener looks like from outside.
|
||||
status int
|
||||
draining bool
|
||||
// willClose is what the server answered about the connection: the header
|
||||
// it sends is Connection: close, which the transport consumes and reports
|
||||
// here rather than leaving in Response.Header.
|
||||
willClose bool
|
||||
}
|
||||
|
||||
// probe asks once, on a connection of its own.
|
||||
//
|
||||
// A new transport per request, because a connection opened before the signal
|
||||
// can still be served after the listener is closed: reusing one would let this
|
||||
// test pass against a shutdown that had already broken the listener. Keep-alive
|
||||
// is left enabled so the server's own Connection: close is observable - a
|
||||
// client that asked for close would get that header back either way, and the
|
||||
// assertion would prove nothing.
|
||||
func probe(addr, path string) sample {
|
||||
tr := &http.Transport{}
|
||||
defer tr.CloseIdleConnections()
|
||||
c := &http.Client{Transport: tr, Timeout: 3 * time.Second}
|
||||
|
||||
s := sample{at: time.Now(), path: path}
|
||||
resp, err := c.Get("http://" + addr + path)
|
||||
if err != nil {
|
||||
return s
|
||||
}
|
||||
defer func() { _ = resp.Body.Close() }()
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
s.status = resp.StatusCode
|
||||
s.willClose = resp.Close
|
||||
s.draining = strings.Contains(string(body), `"status":"draining"`)
|
||||
return s
|
||||
}
|
||||
|
||||
// watcher polls the child until it stops accepting connections, keeping every
|
||||
// answer.
|
||||
type watcher struct {
|
||||
mu sync.Mutex
|
||||
samples []sample
|
||||
done chan struct{}
|
||||
}
|
||||
|
||||
func watch(addr string, paths ...string) *watcher {
|
||||
w := &watcher{done: make(chan struct{})}
|
||||
go func() {
|
||||
defer close(w.done)
|
||||
for {
|
||||
refused := false
|
||||
for _, p := range paths {
|
||||
s := probe(addr, p)
|
||||
w.mu.Lock()
|
||||
w.samples = append(w.samples, s)
|
||||
w.mu.Unlock()
|
||||
if s.status == 0 {
|
||||
refused = true
|
||||
}
|
||||
}
|
||||
if refused {
|
||||
return
|
||||
}
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
}()
|
||||
return w
|
||||
}
|
||||
|
||||
// sawDraining reports whether /ready has answered "draining" yet.
|
||||
func (w *watcher) sawDraining() bool {
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
for _, s := range w.samples {
|
||||
if s.path == readyPath && s.draining {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (w *watcher) wait(t *testing.T, d time.Duration) []sample {
|
||||
t.Helper()
|
||||
select {
|
||||
case <-w.done:
|
||||
case <-time.After(d):
|
||||
t.Fatal("the child never stopped accepting connections")
|
||||
}
|
||||
w.mu.Lock()
|
||||
defer w.mu.Unlock()
|
||||
return w.samples
|
||||
}
|
||||
|
||||
func describe(samples []sample) string {
|
||||
var b strings.Builder
|
||||
for _, s := range samples {
|
||||
fmt.Fprintf(&b, " %s %s -> %d draining=%v willClose=%v\n",
|
||||
s.at.Format("15:04:05.000"), s.path, s.status, s.draining, s.willClose)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// Acceptance 19. Registering only os.Interrupt meant SIGTERM - the signal
|
||||
// `docker stop`, Kubernetes and systemd all send - terminated the process
|
||||
// before any of the shutdown path ran. Both must now reach it.
|
||||
func TestBothSignalsRunTheShutdownPath(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
sig syscall.Signal
|
||||
}{
|
||||
{"SIGINT", syscall.SIGINT},
|
||||
{"SIGTERM", syscall.SIGTERM},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
cmd, lines := startChild(t, false)
|
||||
await(t, lines, markerReady, 30*time.Second)
|
||||
|
||||
if err := cmd.Process.Signal(tc.sig); err != nil {
|
||||
t.Fatalf("signal: %v", err)
|
||||
}
|
||||
|
||||
await(t, lines, markerSignal, 10*time.Second)
|
||||
await(t, lines, markerShutdown, 10*time.Second)
|
||||
await(t, lines, markerExiting, 10*time.Second)
|
||||
|
||||
if err := cmd.Wait(); err != nil {
|
||||
t.Fatalf("child exited with %v, want a clean exit", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 20. quit is a buffered channel and signal.Notify stays armed, so
|
||||
// without restoring the disposition a second signal only refills the buffer:
|
||||
// once SIGTERM is registered, a shutdown that hangs could not be interrupted by
|
||||
// anything short of SIGKILL.
|
||||
//
|
||||
// The hang is now a cleanup callback that never returns, which is where a
|
||||
// shutdown actually hangs, and it is reached through gracefulShutdown - so this
|
||||
// also pins where the disposition is restored. Restore it before the drain
|
||||
// window and the window itself becomes the interruptible part; restore it never
|
||||
// and this test hangs.
|
||||
func TestASecondSignalStillKillsAStuckShutdown(t *testing.T) {
|
||||
cmd, lines := startChild(t, true)
|
||||
await(t, lines, markerReady, 30*time.Second)
|
||||
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("first signal: %v", err)
|
||||
}
|
||||
await(t, lines, markerSignal, 10*time.Second)
|
||||
|
||||
// The child is now on its way into a cleanup that will not finish on its
|
||||
// own. Signalled repeatedly rather than once: the marker is printed just
|
||||
// before gracefulShutdown is entered, and the disposition is not restored
|
||||
// until the drain window is over - zero seconds here, but not zero
|
||||
// instructions - so a single signal sent immediately after the marker can
|
||||
// still land in the buffered channel and be dropped. Which of them does
|
||||
// the killing is not the assertion; that one of them can is.
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- cmd.Wait() }()
|
||||
|
||||
retry := time.NewTicker(200 * time.Millisecond)
|
||||
defer retry.Stop()
|
||||
deadline := time.After(15 * time.Second)
|
||||
for {
|
||||
select {
|
||||
case err := <-done:
|
||||
if err == nil {
|
||||
t.Fatal("child exited cleanly; it was supposed to be killed by the second signal")
|
||||
}
|
||||
return
|
||||
case <-retry.C:
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("second signal: %v", err)
|
||||
}
|
||||
case <-deadline:
|
||||
t.Fatal("the second signal did not kill a stuck shutdown - the escape hatch is gone")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 21. srv.Shutdown reports an error exactly when connections were
|
||||
// still in flight, and the old code answered that with log.Fatal - an
|
||||
// unconditional os.Exit(1). Everything after it, which is where the cleanup
|
||||
// hooks will hang, never ran. A failed Shutdown must not end the process.
|
||||
func TestShutdownTimeoutDoesNotStopWhatFollows(t *testing.T) {
|
||||
cmd, lines := startChild(t, false, childHangConn+"=1")
|
||||
await(t, lines, markerReady, 30*time.Second)
|
||||
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("signal: %v", err)
|
||||
}
|
||||
await(t, lines, markerSignal, 10*time.Second)
|
||||
|
||||
seen, _ := await(t, lines, markerExiting, 20*time.Second)
|
||||
|
||||
var timedOut bool
|
||||
for _, l := range seen {
|
||||
if strings.Contains(l, "shutdown error:") {
|
||||
timedOut = true
|
||||
}
|
||||
}
|
||||
if !timedOut {
|
||||
t.Fatalf("Shutdown did not time out, so this test proves nothing; saw:\n%s",
|
||||
strings.Join(seen, "\n"))
|
||||
}
|
||||
var cleaned bool
|
||||
for _, l := range seen {
|
||||
if strings.Contains(l, markerCleanup) {
|
||||
cleaned = true
|
||||
}
|
||||
}
|
||||
if !cleaned {
|
||||
t.Fatalf("the BeforeExit callback did not run after a failed Shutdown; saw:\n%s",
|
||||
strings.Join(seen, "\n"))
|
||||
}
|
||||
if err := cmd.Wait(); err != nil {
|
||||
t.Fatalf("child exited with %v after a failed Shutdown, want a clean exit", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A callback that outlasts its budget must not take the process with it, and
|
||||
// must not be waited for: RunShutdown reports the deadline and returns, the
|
||||
// callback carries on, and the process still exits cleanly. This is the half of
|
||||
// the contract that is easy to get backwards - the context bounds the wait, not
|
||||
// the work, because Go cannot cancel a function that does not check for it.
|
||||
func TestACleanupThatOutlastsItsBudgetIsAbandonedNotAwaited(t *testing.T) {
|
||||
cmd, lines := startChild(t, false, childSlowCleanup+"=1")
|
||||
await(t, lines, markerReady, 30*time.Second)
|
||||
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("signal: %v", err)
|
||||
}
|
||||
await(t, lines, markerSignal, 10*time.Second)
|
||||
|
||||
// The budget is 300ms and the callback sleeps two seconds. If RunShutdown
|
||||
// waited for it, this marker would not arrive for two seconds; the one
|
||||
// second here is what makes "abandoned, not awaited" the thing asserted.
|
||||
seen, _ := await(t, lines, markerExiting, 1*time.Second)
|
||||
|
||||
var reported bool
|
||||
for _, l := range seen {
|
||||
if strings.Contains(l, "cleanup error:") {
|
||||
reported = true
|
||||
}
|
||||
if strings.Contains(l, markerCleanup) {
|
||||
t.Fatalf("the slow callback finished before the process moved on, so nothing was abandoned; saw:\n%s",
|
||||
strings.Join(seen, "\n"))
|
||||
}
|
||||
}
|
||||
if !reported {
|
||||
t.Fatalf("RunShutdown returned no error for a callback that outlasted the budget; saw:\n%s",
|
||||
strings.Join(seen, "\n"))
|
||||
}
|
||||
if err := cmd.Wait(); err != nil {
|
||||
t.Fatalf("child exited with %v, want a clean exit despite the abandoned callback", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The core acceptance: with a drain window configured, something outside the
|
||||
// process can observe that this instance is draining, on a connection it opens
|
||||
// after the signal, and can still be served while it does.
|
||||
//
|
||||
// Two windows rather than one. A single value proves only that something takes
|
||||
// that long, which a hard-coded sleep anywhere in the sequence would satisfy;
|
||||
// two say the wait is the configured one.
|
||||
//
|
||||
// What each answer is for:
|
||||
//
|
||||
// - /ready reporting "draining" is the window being observable at all. The
|
||||
// status code alone would not say it: with no database configured the
|
||||
// probe's own checks fail and 503 is also the answer before the signal.
|
||||
// - The server refusing to keep those connections alive is the window being
|
||||
// useful. It keeps them alive until Shutdown sets shuttingDown(), so
|
||||
// without switching keep-alive off here a balancer's pool would sit
|
||||
// untouched for the whole window and be cut at the end of it anyway. The
|
||||
// header saying so is Connection: close; the transport consumes it and
|
||||
// reports it as Response.Close, which is what a sample records.
|
||||
// - /health staying 200 is the window not asking to be restarted, and the
|
||||
// ordinary route staying 200 is the window not refusing work. Draining is
|
||||
// "stop sending me new work", not "reject what arrives".
|
||||
func TestTheDrainWindowIsObservableWhileStillServing(t *testing.T) {
|
||||
for _, drain := range []time.Duration{300 * time.Millisecond, 1200 * time.Millisecond} {
|
||||
t.Run(drain.String(), func(t *testing.T) {
|
||||
cmd, lines := startChild(t, false,
|
||||
fmt.Sprintf("%s=%d", childDrainMS, drain.Milliseconds()))
|
||||
addr := childAddr(t, lines)
|
||||
await(t, lines, markerReady, 30*time.Second)
|
||||
|
||||
w := watch(addr, readyPath, healthPath, pingPath)
|
||||
// Long enough for a round of answers from a server that is not yet
|
||||
// draining, which is what the keep-alive assertion below compares
|
||||
// against.
|
||||
time.Sleep(150 * time.Millisecond)
|
||||
|
||||
signalAt := time.Now()
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("signal: %v", err)
|
||||
}
|
||||
|
||||
samples := w.wait(t, drain+30*time.Second)
|
||||
spent := took(t, lines, 10*time.Second)
|
||||
await(t, lines, markerExiting, 10*time.Second)
|
||||
|
||||
if spent < drain {
|
||||
t.Errorf("the shutdown took %s, want at least the %s window", spent, drain)
|
||||
}
|
||||
|
||||
var refusedAt = -1
|
||||
for i, s := range samples {
|
||||
if s.status == 0 {
|
||||
refusedAt = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if refusedAt < 0 {
|
||||
t.Fatalf("the child never stopped accepting; saw:\n%s", describe(samples))
|
||||
}
|
||||
|
||||
var keptAliveBefore, drainingInside, closedInside bool
|
||||
for _, s := range samples[:refusedAt] {
|
||||
switch s.path {
|
||||
case readyPath:
|
||||
if s.at.Before(signalAt) && !s.draining && !s.willClose {
|
||||
keptAliveBefore = true
|
||||
}
|
||||
if s.at.After(signalAt) && s.draining {
|
||||
drainingInside = true
|
||||
if s.willClose {
|
||||
closedInside = true
|
||||
}
|
||||
}
|
||||
case healthPath, pingPath:
|
||||
if s.status != http.StatusOK {
|
||||
t.Errorf("%s answered %d before the listener closed, want 200;\n%s",
|
||||
s.path, s.status, describe(samples))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if !keptAliveBefore {
|
||||
t.Fatalf("no answer before the signal kept the connection alive, so the header assertion below proves nothing;\n%s",
|
||||
describe(samples))
|
||||
}
|
||||
if !drainingInside {
|
||||
t.Errorf("no answer inside the window reported draining; the flip and the closed listener were not far enough apart to observe;\n%s",
|
||||
describe(samples))
|
||||
}
|
||||
if !closedInside {
|
||||
t.Errorf("answers inside the window still kept the connection alive, so a pooled connection survives the whole window and is cut at the end of it anyway;\n%s",
|
||||
describe(samples))
|
||||
}
|
||||
|
||||
if err := cmd.Wait(); err != nil {
|
||||
t.Fatalf("child exited with %v, want a clean exit", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The default has to be no window at all: a process that configures no
|
||||
// extend.shutdown section must shut down the way it did before the section
|
||||
// existed.
|
||||
//
|
||||
// Asserted as a sequence rather than as a duration. How long a shutdown takes
|
||||
// is decided by how much the cleanup callbacks have to do, so "as fast as
|
||||
// before" is not falsifiable; "nothing was inserted between the signal and the
|
||||
// listener closing" is.
|
||||
func TestAnUnconfiguredShutdownAddsNoWindow(t *testing.T) {
|
||||
cmd, lines := startChild(t, false)
|
||||
await(t, lines, markerReady, 30*time.Second)
|
||||
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("signal: %v", err)
|
||||
}
|
||||
await(t, lines, markerSignal, 10*time.Second)
|
||||
|
||||
spent := took(t, lines, 10*time.Second)
|
||||
if spent > 100*time.Millisecond {
|
||||
t.Errorf("an unconfigured shutdown spent %s between the signal and exiting; "+
|
||||
"with no drain window and no cleanup callbacks it must be immediate", spent)
|
||||
}
|
||||
|
||||
if err := cmd.Wait(); err != nil {
|
||||
t.Fatalf("child exited with %v, want a clean exit", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A second signal during the window ends it early rather than killing the
|
||||
// process. Somebody sending another kill wants this over with sooner, and the
|
||||
// answer to that is to stop draining - not to skip the cleanup, which is what
|
||||
// the default disposition would do.
|
||||
//
|
||||
// This is the pair to TestASecondSignalStillKillsAStuckShutdown: the escape
|
||||
// hatch has to be closed for the length of the window and open after it.
|
||||
func TestASecondSignalEndsTheDrainWindowEarly(t *testing.T) {
|
||||
// Long enough that the shutdown cannot plausibly have taken this long on
|
||||
// its own, short enough that the test does not sit out the whole window
|
||||
// when the early exit is missing - it fails on the reported duration
|
||||
// instead of on a timeout, which says which of the two broke.
|
||||
const window = 10 * time.Second
|
||||
cmd, lines := startChild(t, false,
|
||||
fmt.Sprintf("%s=%d", childDrainMS, window.Milliseconds()))
|
||||
addr := childAddr(t, lines)
|
||||
await(t, lines, markerReady, 30*time.Second)
|
||||
|
||||
w := watch(addr, readyPath)
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("first signal: %v", err)
|
||||
}
|
||||
|
||||
deadline := time.Now().Add(15 * time.Second)
|
||||
for !w.sawDraining() {
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("the child never reported draining, so the second signal below would not land inside the window")
|
||||
}
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
|
||||
if err := cmd.Process.Signal(syscall.SIGTERM); err != nil {
|
||||
t.Fatalf("second signal: %v", err)
|
||||
}
|
||||
|
||||
spent := took(t, lines, window+20*time.Second)
|
||||
if spent >= window {
|
||||
t.Errorf("the window ran its full %s despite a second signal (%s); the signal was ignored", window, spent)
|
||||
}
|
||||
await(t, lines, markerExiting, 10*time.Second)
|
||||
|
||||
if err := cmd.Wait(); err != nil {
|
||||
t.Fatalf("child exited with %v; a second signal inside the window must end the window, not the process", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
// A mistyped --app used to be indistinguishable from an up-to-date database on
|
||||
// all three paths: `migrate` printed that the app was unknown and exited 0,
|
||||
// while `--dry-run` and `status` printed "nothing to apply" and "none
|
||||
// recorded" - the same words a database with nothing pending produces. An
|
||||
// operator scripting `migrate --app crmm && deploy` therefore deployed against
|
||||
// a database the migrations never touched.
|
||||
func TestAppRegistrationErrorRejectsAnUnknownCode(t *testing.T) {
|
||||
restore := appCode
|
||||
t.Cleanup(func() { appCode = restore })
|
||||
|
||||
appCode = "doesnotexist"
|
||||
err := appRegistrationError()
|
||||
if err == nil {
|
||||
t.Fatal("an unregistered app code must be an error, not an empty run")
|
||||
}
|
||||
if !strings.Contains(err.Error(), `"doesnotexist"`) {
|
||||
t.Errorf("the message must quote what was typed; got %q", err)
|
||||
}
|
||||
// Listing what is registered is what turns the error into a fix: the typo
|
||||
// is usually one letter away from something in this list.
|
||||
if !strings.Contains(err.Error(), migration.FrameworkAppCode) {
|
||||
t.Errorf("the message must list the registered codes; got %q", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppRegistrationErrorAcceptsWhatIsRegistered(t *testing.T) {
|
||||
restore := appCode
|
||||
t.Cleanup(func() { appCode = restore })
|
||||
|
||||
for _, code := range []string{
|
||||
"", // no --app at all: every migration runs
|
||||
migration.FrameworkAppCode, // "core", the framework's own
|
||||
strings.ToUpper(migration.FrameworkAppCode), // codes normalize to lower case
|
||||
} {
|
||||
appCode = code
|
||||
if err := appRegistrationError(); err != nil {
|
||||
t.Errorf("appCode %q must be accepted; got %v", code, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
+312
-15
@@ -1,21 +1,46 @@
|
||||
package migration
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
contractmigration "github.com/go-admin-team/go-admin-core/v2/sdk/contract/migration"
|
||||
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
var Migrate = &Migration{
|
||||
version: make(map[string]func(db *gorm.DB, version string) error),
|
||||
var Migrate = newMigration()
|
||||
|
||||
// contractSnapshot is contractmigration.Snapshot, indirected through a
|
||||
// package-level variable so tests can substitute an isolated
|
||||
// *contractmigration.Registry's Snapshot instead of reaching into
|
||||
// go-admin-core's single process-wide registry, which every *Migration in
|
||||
// this process - test-local or the package-level Migrate - reads through the
|
||||
// same call. See mergedEntries.
|
||||
var contractSnapshot = contractmigration.Snapshot
|
||||
|
||||
func newMigration() *Migration {
|
||||
return &Migration{version: make(map[string]versionEntry)}
|
||||
}
|
||||
|
||||
// versionEntry is one registered migration plus the app it belongs to. The
|
||||
// empty app code means the framework itself, which is also what the
|
||||
// sys_migration.app_code column defaults to, so history written before this
|
||||
// field existed reads back correctly with no backfill.
|
||||
type versionEntry struct {
|
||||
appCode string
|
||||
fn func(db *gorm.DB, version string) error
|
||||
}
|
||||
|
||||
type Migration struct {
|
||||
db *gorm.DB
|
||||
version map[string]func(db *gorm.DB, version string) error
|
||||
version map[string]versionEntry
|
||||
mutex sync.Mutex
|
||||
}
|
||||
|
||||
@@ -27,20 +52,285 @@ func (e *Migration) SetDb(db *gorm.DB) {
|
||||
e.db = db
|
||||
}
|
||||
|
||||
// SetVersion registers a migration owned by the framework. Signature and
|
||||
// behaviour are unchanged: every existing call site in version/*.go keeps
|
||||
// compiling and keeps writing common.Migration{Version: version} with no app
|
||||
// code, which is the correct meaning of "framework".
|
||||
func (e *Migration) SetVersion(k string, f func(db *gorm.DB, version string) error) {
|
||||
e.mutex.Lock()
|
||||
defer e.mutex.Unlock()
|
||||
e.version[k] = f
|
||||
e.setVersion(k, "", f)
|
||||
}
|
||||
|
||||
func (e *Migration) Migrate() {
|
||||
versions := make([]string, 0)
|
||||
for k := range e.version {
|
||||
func (e *Migration) setVersion(k, appCode string, f func(db *gorm.DB, version string) error) {
|
||||
e.mutex.Lock()
|
||||
defer e.mutex.Unlock()
|
||||
e.version[k] = versionEntry{appCode: appCode, fn: f}
|
||||
}
|
||||
|
||||
// AppMigrationFunc is the signature of a migration registered through ForApp.
|
||||
//
|
||||
// It receives appCode explicitly because the migration - not the framework -
|
||||
// writes its own completion row, normally as the last statement inside its own
|
||||
// transaction. That is what makes "the schema change and the record of it
|
||||
// commit together" true, and the framework cannot insert the row on the
|
||||
// migration's behalf without giving that up. Handing the code to the function
|
||||
// is what stops an app's migrations from silently recording themselves as the
|
||||
// framework's.
|
||||
type AppMigrationFunc func(db *gorm.DB, version, appCode string) error
|
||||
|
||||
// AppRegistrar is a per-app view over a registry.
|
||||
type AppRegistrar struct {
|
||||
m *Migration
|
||||
appCode string
|
||||
}
|
||||
|
||||
// FrameworkAppCode is the name migrate status prints for migrations that belong
|
||||
// to the framework rather than to an app, and the name --app accepts to select
|
||||
// them. The stored app code for those is the empty string; this is only the
|
||||
// spelling humans use. It is reserved - ForApp rejects it - so that every group
|
||||
// heading status prints is also a value --app understands.
|
||||
const FrameworkAppCode = "core"
|
||||
|
||||
// ForApp returns a registrar that records migrations under code.
|
||||
//
|
||||
// The code is lower-cased: sys_migration.version sorts as ASCII, so mixed case
|
||||
// would order MyApp before crm for no reason a reader could guess, and the two
|
||||
// spellings would group as two different apps in migrate status.
|
||||
//
|
||||
// An empty or reserved code panics rather than falling back to the framework.
|
||||
// Registration happens in init(), so this fires the first time the binary runs
|
||||
// anywhere, which is the point: an app whose migrations quietly file themselves
|
||||
// under the framework is exactly the class of silent failure this work is meant
|
||||
// to remove. Framework migrations call Migrate.SetVersion directly.
|
||||
func ForApp(code string) *AppRegistrar { return Migrate.ForApp(code) }
|
||||
|
||||
// ForApp is the same on an explicit registry, which is what tests use.
|
||||
func (e *Migration) ForApp(code string) *AppRegistrar {
|
||||
code = NormalizeAppCode(code)
|
||||
switch code {
|
||||
case "":
|
||||
panic("migration.ForApp: empty app code; framework migrations use Migrate.SetVersion")
|
||||
case FrameworkAppCode:
|
||||
panic("migration.ForApp: app code " + FrameworkAppCode + " is reserved for the framework")
|
||||
}
|
||||
return &AppRegistrar{m: e, appCode: code}
|
||||
}
|
||||
|
||||
// AppCode reports the code this registrar files migrations under, after
|
||||
// normalisation.
|
||||
func (r *AppRegistrar) AppCode() string { return r.appCode }
|
||||
|
||||
// SetVersion registers an app-owned migration under k, which is the bare
|
||||
// timestamp taken from the file name exactly as framework migrations do.
|
||||
//
|
||||
// What reaches sys_migration.version is the namespaced form; the version string
|
||||
// handed to f is that same namespaced string, so a migration that writes
|
||||
// common.Migration{Version: version, AppCode: appCode} records the key the
|
||||
// registry will look for next time.
|
||||
func (r *AppRegistrar) SetVersion(k string, f AppMigrationFunc) {
|
||||
key := namespacedKey(r.appCode, k)
|
||||
r.m.setVersion(key, r.appCode, func(db *gorm.DB, version string) error {
|
||||
return f(db, version, r.appCode)
|
||||
})
|
||||
}
|
||||
|
||||
// namespacedKey scopes k to appCode so two apps cannot collide on the
|
||||
// sys_migration.version primary key by minting the same millisecond timestamp.
|
||||
// Framework migrations (appCode == "") stay bare, matching every version string
|
||||
// already in production.
|
||||
func namespacedKey(appCode, k string) string {
|
||||
if appCode == "" {
|
||||
return k
|
||||
}
|
||||
return appCode + "-" + k
|
||||
}
|
||||
|
||||
// mergedEntries returns every migration this process knows about: the
|
||||
// host's own registry (e.version, filled by version/*.go and
|
||||
// version-local/*.go through SetVersion/ForApp) plus whatever a third-party
|
||||
// application registered through go-admin-core's sdk/contract/migration
|
||||
// package (PRD 006, F9's host wiring).
|
||||
//
|
||||
// That package keeps its own process-wide registry, entirely separate from
|
||||
// e.version, because a third-party application cannot reach into this
|
||||
// process to call an unexported method on *Migration - contract/migration's
|
||||
// package-level ForApp/Snapshot are the only door open to it. Without this
|
||||
// merge, migrate/status/--dry-run would only ever see the host's own
|
||||
// migrations: an application's ForApp("crm").SetVersion(...) would compile,
|
||||
// register successfully into contract/migration's registry, and then never
|
||||
// run, with no error anywhere - the exact silent gap this method closes.
|
||||
//
|
||||
// Entry and versionEntry are structurally identical (an app code plus a
|
||||
// func(db, version) error); the conversion below exists only because they
|
||||
// are two distinct named types, one per package, not because the data
|
||||
// differs.
|
||||
func (e *Migration) mergedEntries() map[string]versionEntry {
|
||||
e.mutex.Lock()
|
||||
out := make(map[string]versionEntry, len(e.version))
|
||||
for k, v := range e.version {
|
||||
out[k] = v
|
||||
}
|
||||
e.mutex.Unlock()
|
||||
|
||||
for k, entry := range contractSnapshot() {
|
||||
if _, exists := out[k]; exists {
|
||||
// contract/migration.ForApp namespaces every app-owned key as
|
||||
// appCode + "-" + k, and appCode is reserved from ""/"core", so
|
||||
// this should never collide with a host-registered key. If it
|
||||
// somehow does, the host's own registration wins rather than
|
||||
// silently overwriting it.
|
||||
continue
|
||||
}
|
||||
out[k] = versionEntry{appCode: entry.AppCode, fn: entry.Fn}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// StatusEntry is one row of migrate status.
|
||||
type StatusEntry struct {
|
||||
AppCode string
|
||||
Version string
|
||||
Registered bool
|
||||
Applied bool
|
||||
ApplyTime *time.Time
|
||||
}
|
||||
|
||||
// Status merges the in-process registry with sys_migration, so it reports all
|
||||
// three shapes at once: registered but not applied, registered and applied, and
|
||||
// applied while nothing registers it any more - a row left behind by a
|
||||
// migration file that was deleted, or by an app that was uninstalled.
|
||||
//
|
||||
// It only reads. Nothing here creates or alters a table, which is what lets
|
||||
// both `status` and `--dry-run` run against a database without touching it.
|
||||
func (e *Migration) Status() ([]StatusEntry, error) {
|
||||
if e.db == nil {
|
||||
return nil, fmt.Errorf("migration: no database configured")
|
||||
}
|
||||
|
||||
all := e.mergedEntries()
|
||||
registered := make(map[string]string, len(all))
|
||||
for k, v := range all {
|
||||
registered[k] = v.appCode
|
||||
}
|
||||
|
||||
applied := make(map[string]common.Migration)
|
||||
// A database that has never been migrated has no sys_migration table.
|
||||
// Reporting everything as pending is the honest answer there; erroring out
|
||||
// would make status useless in exactly the case it is most wanted.
|
||||
if e.db.Migrator().HasTable(&common.Migration{}) {
|
||||
var rows []common.Migration
|
||||
if err := e.db.Find(&rows).Error; err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, r := range rows {
|
||||
applied[r.Version] = r
|
||||
}
|
||||
}
|
||||
|
||||
versions := make(map[string]struct{}, len(registered)+len(applied))
|
||||
for k := range registered {
|
||||
versions[k] = struct{}{}
|
||||
}
|
||||
for k := range applied {
|
||||
versions[k] = struct{}{}
|
||||
}
|
||||
list := make([]string, 0, len(versions))
|
||||
for k := range versions {
|
||||
list = append(list, k)
|
||||
}
|
||||
sort.Strings(list)
|
||||
|
||||
out := make([]StatusEntry, 0, len(list))
|
||||
for _, v := range list {
|
||||
entry := StatusEntry{Version: v}
|
||||
if code, ok := registered[v]; ok {
|
||||
entry.Registered = true
|
||||
entry.AppCode = code
|
||||
}
|
||||
if row, ok := applied[v]; ok {
|
||||
entry.Applied = true
|
||||
t := row.ApplyTime
|
||||
entry.ApplyTime = &t
|
||||
if !entry.Registered {
|
||||
// Nothing registers this version any more, so the database is
|
||||
// the only source left for what it belonged to.
|
||||
entry.AppCode = row.AppCode
|
||||
}
|
||||
}
|
||||
out = append(out, entry)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Migrate applies every registered migration that has not been applied yet,
|
||||
// across all apps. Existing callers are unaffected.
|
||||
func (e *Migration) Migrate() { e.run(allApps) }
|
||||
|
||||
// MigrateApp applies only the migrations registered under appCode. Pass
|
||||
// FrameworkAppCode for the framework's own migrations.
|
||||
func (e *Migration) MigrateApp(appCode string) { e.run(AppFilter(appCode)) }
|
||||
|
||||
// NormalizeAppCode applies the same rule ForApp does, so a code typed on the
|
||||
// command line matches one written in an init().
|
||||
func NormalizeAppCode(code string) string {
|
||||
return strings.ToLower(strings.TrimSpace(code))
|
||||
}
|
||||
|
||||
// AppFilter turns a code as typed into the code stored in the registry, so
|
||||
// "core" selects the framework's migrations, whose stored code is empty.
|
||||
func AppFilter(code string) string {
|
||||
code = NormalizeAppCode(code)
|
||||
if code == FrameworkAppCode {
|
||||
return ""
|
||||
}
|
||||
return code
|
||||
}
|
||||
|
||||
// DisplayAppCode is the inverse: what to print for a stored code.
|
||||
func DisplayAppCode(code string) string {
|
||||
if code == "" {
|
||||
return FrameworkAppCode
|
||||
}
|
||||
return code
|
||||
}
|
||||
|
||||
// AppCodes lists the app codes with at least one registered migration, framework
|
||||
// included under its display name, sorted.
|
||||
func (e *Migration) AppCodes() []string {
|
||||
all := e.mergedEntries()
|
||||
seen := map[string]struct{}{}
|
||||
for _, v := range all {
|
||||
seen[DisplayAppCode(v.appCode)] = struct{}{}
|
||||
}
|
||||
|
||||
out := make([]string, 0, len(seen))
|
||||
for code := range seen {
|
||||
out = append(out, code)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func (e *Migration) run(appCode string) {
|
||||
all := e.mergedEntries()
|
||||
versions := make([]string, 0, len(all))
|
||||
entries := make(map[string]versionEntry, len(all))
|
||||
for k, v := range all {
|
||||
if appCode != allApps && v.appCode != appCode {
|
||||
continue
|
||||
}
|
||||
versions = append(versions, k)
|
||||
entries[k] = v
|
||||
}
|
||||
if !sort.StringsAreSorted(versions) {
|
||||
sort.Strings(versions)
|
||||
sort.Strings(versions)
|
||||
|
||||
// A mistyped --app would otherwise select nothing and report "no
|
||||
// migrations to apply", which reads exactly like "already up to date".
|
||||
if appCode != allApps && len(versions) == 0 {
|
||||
log.Printf("no migrations are registered for app %q; registered: %s",
|
||||
DisplayAppCode(appCode), strings.Join(e.AppCodes(), ", "))
|
||||
return
|
||||
}
|
||||
|
||||
var err error
|
||||
var count int64
|
||||
applied := 0
|
||||
@@ -56,7 +346,7 @@ func (e *Migration) Migrate() {
|
||||
continue
|
||||
}
|
||||
log.Printf("applying migration %s", v)
|
||||
if err = (e.version[v])(e.db.Debug(), v); err != nil {
|
||||
if err = entries[v].fn(e.db.Debug(), v); err != nil {
|
||||
log.Fatalf("migration %s failed: %v", v, err)
|
||||
}
|
||||
applied++
|
||||
@@ -68,7 +358,14 @@ func (e *Migration) Migrate() {
|
||||
}
|
||||
}
|
||||
|
||||
// allApps is the sentinel run() takes to mean "do not filter". It is distinct
|
||||
// from the empty app code, which selects the framework's own migrations.
|
||||
const allApps = "\x00all"
|
||||
|
||||
// GetFilename derives a migration's version from its file name. The rule
|
||||
// lives in contract/migration, because an application registering through
|
||||
// that package names its files by the same convention and must land on the
|
||||
// same version string; a second copy here is a second thing to keep in step.
|
||||
func GetFilename(s string) string {
|
||||
s = filepath.Base(s)
|
||||
return s[:13]
|
||||
return contractmigration.GetFilename(s)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,582 @@
|
||||
package migration
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"log"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/logger"
|
||||
|
||||
contractmigration "github.com/go-admin-team/go-admin-core/v2/sdk/contract/migration"
|
||||
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
// withContractRegistry points contractSnapshot at an isolated
|
||||
// *contractmigration.Registry for the duration of one test, instead of
|
||||
// go-admin-core's single process-wide one - see contractSnapshot's doc
|
||||
// comment for why that indirection exists. Restored on cleanup so other
|
||||
// tests in this package keep seeing an empty contract registry regardless of
|
||||
// run order.
|
||||
func withContractRegistry(t *testing.T) *contractmigration.Registry {
|
||||
t.Helper()
|
||||
reg := contractmigration.NewRegistry()
|
||||
orig := contractSnapshot
|
||||
contractSnapshot = reg.Snapshot
|
||||
t.Cleanup(func() { contractSnapshot = orig })
|
||||
return reg
|
||||
}
|
||||
|
||||
func newTestDB(t *testing.T) *gorm.DB {
|
||||
t.Helper()
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("open sqlite: %v", err)
|
||||
}
|
||||
if err = db.AutoMigrate(&common.Migration{}); err != nil {
|
||||
t.Fatalf("automigrate: %v", err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
// recordFor is what an app's migration is expected to do: write its own
|
||||
// completion row, with the version it was handed and the app code it was told
|
||||
// it belongs to.
|
||||
func recordFor(db *gorm.DB, version, appCode string) error {
|
||||
return db.Create(&common.Migration{Version: version, AppCode: appCode}).Error
|
||||
}
|
||||
|
||||
func rowsByVersion(t *testing.T, db *gorm.DB) map[string]common.Migration {
|
||||
t.Helper()
|
||||
var rows []common.Migration
|
||||
if err := db.Find(&rows).Error; err != nil {
|
||||
t.Fatalf("read sys_migration: %v", err)
|
||||
}
|
||||
out := make(map[string]common.Migration, len(rows))
|
||||
for _, r := range rows {
|
||||
out[r.Version] = r
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Acceptance 9: a migration registered through ForApp("x") lands in
|
||||
// sys_migration with app_code "x".
|
||||
//
|
||||
// The registry cannot write that row for the migration, because the row is the
|
||||
// migration's own last statement inside its own transaction. So the only thing
|
||||
// that can make this true is handing the code to the function - which is why
|
||||
// AppMigrationFunc takes three parameters.
|
||||
func TestForAppRecordsItsAppCode(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
m.Migrate()
|
||||
|
||||
rows := rowsByVersion(t, db)
|
||||
row, ok := rows["x-1786800001000"]
|
||||
if !ok {
|
||||
t.Fatalf("no row for x-1786800001000; got %v", rows)
|
||||
}
|
||||
if row.AppCode != "x" {
|
||||
t.Errorf("app_code = %q, want %q", row.AppCode, "x")
|
||||
}
|
||||
}
|
||||
|
||||
// The framework path is untouched: same signature, and an empty app code, which
|
||||
// is what the column defaults to and what every row written before this field
|
||||
// existed reads back as.
|
||||
func TestSetVersionStillRecordsTheFrameworkAsEmpty(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.Migrate()
|
||||
|
||||
rows := rowsByVersion(t, db)
|
||||
row, ok := rows["1786700009000"]
|
||||
if !ok {
|
||||
t.Fatalf("no row for 1786700009000; got %v", rows)
|
||||
}
|
||||
if row.AppCode != "" {
|
||||
t.Errorf("app_code = %q, want empty (framework)", row.AppCode)
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 12: --app x runs x's migrations and touches nothing else.
|
||||
func TestMigrateAppRunsOnlyThatApp(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
ran := map[string]bool{}
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
ran["core"] = true
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran["x"] = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
m.ForApp("y").SetVersion("1786800002000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran["y"] = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.MigrateApp("x")
|
||||
|
||||
if !ran["x"] {
|
||||
t.Error("x did not run")
|
||||
}
|
||||
if ran["y"] || ran["core"] {
|
||||
t.Errorf("MigrateApp(x) also ran %v", ran)
|
||||
}
|
||||
rows := rowsByVersion(t, db)
|
||||
if len(rows) != 1 {
|
||||
t.Fatalf("sys_migration has %d rows, want 1: %v", len(rows), rows)
|
||||
}
|
||||
}
|
||||
|
||||
// "core" is what status prints for the framework, so --app core has to select
|
||||
// it. The stored code is the empty string; AppFilter is the translation.
|
||||
func TestMigrateAppCoreSelectsTheFramework(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
ran := map[string]bool{}
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
ran["core"] = true
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("x").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran["x"] = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.MigrateApp(FrameworkAppCode)
|
||||
|
||||
if !ran["core"] {
|
||||
t.Error("framework migration did not run")
|
||||
}
|
||||
if ran["x"] {
|
||||
t.Error("--app core also ran x")
|
||||
}
|
||||
}
|
||||
|
||||
// Zero-argument Migrate keeps meaning "everything", which is what every
|
||||
// existing caller relies on.
|
||||
func TestMigrateRunsEveryApp(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
var order []string
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
order = append(order, version)
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("bbb").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
order = append(order, version)
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
m.ForApp("aaa").SetVersion("1786800002000", func(db *gorm.DB, version, appCode string) error {
|
||||
order = append(order, version)
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.Migrate()
|
||||
|
||||
// Namespacing puts every framework migration - bare digits - ahead of every
|
||||
// app migration, and orders apps by code rather than by whose timestamp
|
||||
// happened to be smaller. aaa's file is the newer of the two and still runs
|
||||
// first. Cross-app order is not promised, but this is the order, and it is
|
||||
// the one to notice changed.
|
||||
want := []string{"1786700009000", "aaa-1786800002000", "bbb-1786800001000"}
|
||||
if len(order) != len(want) {
|
||||
t.Fatalf("ran %v, want %v", order, want)
|
||||
}
|
||||
for i := range want {
|
||||
if order[i] != want[i] {
|
||||
t.Fatalf("ran %v, want %v", order, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Two apps minting the same millisecond timestamp used to mean one of them was
|
||||
// read as already applied and silently skipped. The namespace prefix is what
|
||||
// makes that impossible without changing the primary key.
|
||||
func TestNamespacingKeepsTwoAppsWithTheSameTimestampApart(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
const sameTimestamp = "1786800001000"
|
||||
ran := 0
|
||||
for _, app := range []string{"crm", "oms"} {
|
||||
m.ForApp(app).SetVersion(sameTimestamp, func(db *gorm.DB, version, appCode string) error {
|
||||
ran++
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
}
|
||||
m.Migrate()
|
||||
|
||||
if ran != 2 {
|
||||
t.Errorf("ran %d migrations, want 2", ran)
|
||||
}
|
||||
rows := rowsByVersion(t, db)
|
||||
for _, want := range []string{"crm-" + sameTimestamp, "oms-" + sameTimestamp} {
|
||||
if _, ok := rows[want]; !ok {
|
||||
t.Errorf("missing %s; got %v", want, rows)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNamespacedKeyLeavesFrameworkVersionsBare(t *testing.T) {
|
||||
if got := namespacedKey("", "1786700009000"); got != "1786700009000" {
|
||||
t.Errorf("framework version was rewritten to %q", got)
|
||||
}
|
||||
if got := namespacedKey("crm", "1786800001000"); got != "crm-1786800001000" {
|
||||
t.Errorf("namespacedKey = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
// An app code differing only in case would group as two apps in status and sort
|
||||
// before every lower-case one, for no reason a reader could guess.
|
||||
func TestForAppNormalisesTheCode(t *testing.T) {
|
||||
m := newMigration()
|
||||
if got := m.ForApp(" CRM ").AppCode(); got != "crm" {
|
||||
t.Errorf("AppCode = %q, want crm", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestForAppRejectsReservedCodes(t *testing.T) {
|
||||
for _, code := range []string{"", " ", FrameworkAppCode, "CORE"} {
|
||||
t.Run("code="+code, func(t *testing.T) {
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Errorf("ForApp(%q) did not panic", code)
|
||||
}
|
||||
}()
|
||||
newMigration().ForApp(code)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatusReportsPendingAppliedAndOrphaned(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
applied := time.Date(2026, 8, 25, 14, 3, 11, 0, time.UTC)
|
||||
if err := db.Create(&common.Migration{Version: "1786700009000", ApplyTime: applied}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Recorded, but nothing registers it any more.
|
||||
if err := db.Create(&common.Migration{Version: "gone-1786800000000", ApplyTime: applied, AppCode: "gone"}).Error; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error { return nil })
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error { return nil })
|
||||
|
||||
entries, err := m.Status()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
byVersion := map[string]StatusEntry{}
|
||||
for _, e := range entries {
|
||||
byVersion[e.Version] = e
|
||||
}
|
||||
|
||||
if e := byVersion["1786700009000"]; !e.Applied || !e.Registered || e.AppCode != "" {
|
||||
t.Errorf("framework entry = %+v", e)
|
||||
} else if e.ApplyTime == nil || !e.ApplyTime.Equal(applied) {
|
||||
t.Errorf("framework apply time = %v, want %v", e.ApplyTime, applied)
|
||||
}
|
||||
if e := byVersion["crm-1786800001000"]; e.Applied || !e.Registered || e.AppCode != "crm" {
|
||||
t.Errorf("crm entry = %+v", e)
|
||||
}
|
||||
if e := byVersion["gone-1786800000000"]; !e.Applied || e.Registered || e.AppCode != "gone" {
|
||||
t.Errorf("orphaned entry = %+v", e)
|
||||
}
|
||||
}
|
||||
|
||||
// Acceptance 11 rests on this: status and --dry-run both go through Status, and
|
||||
// Status must not create the table it reads.
|
||||
func TestStatusDoesNotCreateItsTable(t *testing.T) {
|
||||
db, err := gorm.Open(sqlite.Open("file:"+t.Name()+"?mode=memory&cache=shared"), &gorm.Config{
|
||||
Logger: logger.Default.LogMode(logger.Silent),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error { return nil })
|
||||
|
||||
entries, err := m.Status()
|
||||
if err != nil {
|
||||
t.Fatalf("Status on a database with no sys_migration: %v", err)
|
||||
}
|
||||
if len(entries) != 1 || entries[0].Applied {
|
||||
t.Errorf("entries = %+v, want one pending", entries)
|
||||
}
|
||||
if db.Migrator().HasTable(&common.Migration{}) {
|
||||
t.Error("Status created sys_migration; it must only read")
|
||||
}
|
||||
}
|
||||
|
||||
// The completion row is the migration's own last statement, inside its own
|
||||
// transaction. A migration that fails must leave no record of having run, or
|
||||
// the next run skips it and the schema stays half-changed with nothing to say
|
||||
// so.
|
||||
func TestFailedMigrationLeavesNoRecord(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := recordFor(tx, version, appCode); err != nil {
|
||||
return err
|
||||
}
|
||||
return errTestMigrationFailed
|
||||
})
|
||||
})
|
||||
|
||||
// run() calls log.Fatal on failure, which would take the test binary with
|
||||
// it, so drive the registered function directly - the point here is the
|
||||
// transaction boundary, not the scheduler.
|
||||
entry := m.version["crm-1786800001000"]
|
||||
if err := entry.fn(db, "crm-1786800001000"); err == nil {
|
||||
t.Fatal("migration reported success")
|
||||
}
|
||||
if rows := rowsByVersion(t, db); len(rows) != 0 {
|
||||
t.Errorf("sys_migration has %v after a failed migration", rows)
|
||||
}
|
||||
}
|
||||
|
||||
var errTestMigrationFailed = &testError{"boom"}
|
||||
|
||||
type testError struct{ s string }
|
||||
|
||||
func (e *testError) Error() string { return e.s }
|
||||
|
||||
// A mistyped --app used to select nothing and print "no migrations to apply",
|
||||
// which reads as "already up to date" - the command reports success and does
|
||||
// nothing, which is the failure mode this whole batch exists to remove.
|
||||
func TestMigrateAppOnAnUnknownCodeSaysSo(t *testing.T) {
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
m.ForApp("crm").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
var buf bytes.Buffer
|
||||
log.SetOutput(&buf)
|
||||
t.Cleanup(func() { log.SetOutput(os.Stderr) })
|
||||
|
||||
m.MigrateApp("crmm")
|
||||
|
||||
if !strings.Contains(buf.String(), `no migrations are registered for app "crmm"`) {
|
||||
t.Errorf("output = %q", buf.String())
|
||||
}
|
||||
if !strings.Contains(buf.String(), "registered: core, crm") {
|
||||
t.Errorf("the message must list what is registered; got %q", buf.String())
|
||||
}
|
||||
if rows := rowsByVersion(t, db); len(rows) != 0 {
|
||||
t.Errorf("a typo ran %v", rows)
|
||||
}
|
||||
}
|
||||
|
||||
// This is the acceptance test for PRD 006's host-wiring gap: a migration
|
||||
// registered through contract/migration.ForApp - the only door open to a
|
||||
// third-party application - must actually run, be recorded under its app
|
||||
// code, and show up in AppCodes/Status/--app the same as one registered
|
||||
// through the host's own m.ForApp. Before mergedEntries existed, m.Migrate()
|
||||
// never looked at contract/migration's registry at all, so this compiled,
|
||||
// registered, and silently never ran.
|
||||
func TestMergedEntriesRunsAContractRegisteredAppMigration(t *testing.T) {
|
||||
reg := withContractRegistry(t)
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
ran := false
|
||||
reg.ForApp("order").SetVersion("1793800000000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.Migrate()
|
||||
|
||||
if !ran {
|
||||
t.Fatal("contract-registered migration did not run")
|
||||
}
|
||||
rows := rowsByVersion(t, db)
|
||||
row, ok := rows["order-1793800000000"]
|
||||
if !ok {
|
||||
t.Fatalf("no row for order-1793800000000; got %v", rows)
|
||||
}
|
||||
if row.AppCode != "order" {
|
||||
t.Errorf("app_code = %q, want %q", row.AppCode, "order")
|
||||
}
|
||||
}
|
||||
|
||||
// migrate status and --dry-run both read Status; a contract-registered
|
||||
// migration has to appear there under its app code exactly like a
|
||||
// host-registered one, both before and after it is applied.
|
||||
func TestMergedEntriesStatusIncludesContractRegisteredMigrations(t *testing.T) {
|
||||
reg := withContractRegistry(t)
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
reg.ForApp("order").SetVersion("1793800000000", func(db *gorm.DB, version, appCode string) error {
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
entries, err := m.Status()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
byVersion := map[string]StatusEntry{}
|
||||
for _, e := range entries {
|
||||
byVersion[e.Version] = e
|
||||
}
|
||||
e, ok := byVersion["order-1793800000000"]
|
||||
if !ok || !e.Registered || e.Applied || e.AppCode != "order" {
|
||||
t.Fatalf("pending contract entry = %+v (ok=%v)", e, ok)
|
||||
}
|
||||
|
||||
m.Migrate()
|
||||
|
||||
entries, err = m.Status()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
byVersion = map[string]StatusEntry{}
|
||||
for _, e := range entries {
|
||||
byVersion[e.Version] = e
|
||||
}
|
||||
if e := byVersion["order-1793800000000"]; !e.Applied {
|
||||
t.Errorf("applied contract entry = %+v", e)
|
||||
}
|
||||
}
|
||||
|
||||
// AppCodes feeds both --app's typo detection (appRegistrationError) and the
|
||||
// group headings status prints; a contract-registered app has to appear
|
||||
// there or a real "go-admin migrate --app order" would be told the app does
|
||||
// not exist.
|
||||
func TestMergedEntriesAppCodesIncludesContractRegisteredApps(t *testing.T) {
|
||||
reg := withContractRegistry(t)
|
||||
m := newMigration()
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error { return nil })
|
||||
reg.ForApp("order").SetVersion("1793800000000", func(db *gorm.DB, version, appCode string) error { return nil })
|
||||
|
||||
got := m.AppCodes()
|
||||
want := []string{"core", "order"}
|
||||
if len(got) != len(want) {
|
||||
t.Fatalf("AppCodes = %v, want %v", got, want)
|
||||
}
|
||||
for i := range want {
|
||||
if got[i] != want[i] {
|
||||
t.Fatalf("AppCodes = %v, want %v", got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --app order has to actually run only order's migrations - the same
|
||||
// per-app isolation MigrateApp already gives host-registered apps - even
|
||||
// though order is registered in a different registry entirely.
|
||||
func TestMergedEntriesMigrateAppRunsOnlyThatContractApp(t *testing.T) {
|
||||
reg := withContractRegistry(t)
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
ran := map[string]bool{}
|
||||
m.SetVersion("1786700009000", func(db *gorm.DB, version string) error {
|
||||
ran["core"] = true
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
reg.ForApp("order").SetVersion("1793800000000", func(db *gorm.DB, version, appCode string) error {
|
||||
ran["order"] = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.MigrateApp("order")
|
||||
|
||||
if !ran["order"] {
|
||||
t.Error("order did not run")
|
||||
}
|
||||
if ran["core"] {
|
||||
t.Errorf("MigrateApp(order) also ran %v", ran)
|
||||
}
|
||||
}
|
||||
|
||||
// A host-registered key is not supposed to collide with a namespaced
|
||||
// contract key (see mergedEntries' doc comment), but if it somehow did, the
|
||||
// host's own registration must win rather than a third-party application
|
||||
// silently overwriting a framework migration under the same key.
|
||||
func TestMergedEntriesHostRegistrationWinsOnKeyCollision(t *testing.T) {
|
||||
reg := withContractRegistry(t)
|
||||
db := newTestDB(t)
|
||||
m := newMigration()
|
||||
m.SetDb(db)
|
||||
|
||||
hostRan, contractRan := false, false
|
||||
m.ForApp("dup").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
hostRan = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
reg.ForApp("dup").SetVersion("1786800001000", func(db *gorm.DB, version, appCode string) error {
|
||||
contractRan = true
|
||||
return recordFor(db, version, appCode)
|
||||
})
|
||||
|
||||
m.Migrate()
|
||||
|
||||
if !hostRan {
|
||||
t.Error("host registration did not run")
|
||||
}
|
||||
if contractRan {
|
||||
t.Error("contract registration ran; host registration should have won the collision")
|
||||
}
|
||||
}
|
||||
|
||||
// GetFilename must stay the same rule the contract package applies, since an
|
||||
// application registering through contract/migration names its files by that
|
||||
// convention and has to land on the same version string. Pinning the reject
|
||||
// case is what catches a re-divergence: a local copy that only sliced would
|
||||
// return "add_orders.go" here and register a migration under a key that never
|
||||
// matches anything.
|
||||
func TestGetFilenameDelegatesToTheContractRule(t *testing.T) {
|
||||
if got := GetFilename("version/1786700001000_demo_menu.go"); got != "1786700001000" {
|
||||
t.Fatalf("GetFilename = %q, want %q", got, "1786700001000")
|
||||
}
|
||||
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Fatal("a file name carrying no version did not panic")
|
||||
}
|
||||
}()
|
||||
GetFilename("version/add_orders.go")
|
||||
}
|
||||
@@ -15,6 +15,14 @@ type Model struct {
|
||||
Id int `json:"id" gorm:"primaryKey;autoIncrement;comment:主键编码"`
|
||||
}
|
||||
|
||||
// ModelTime is frozen at the schema shape these tables had before
|
||||
// 1786700003000 converted deleted_at to a NOT NULL millisecond marker. That is
|
||||
// correct for the migrations ordered before the conversion, and wrong for any
|
||||
// added after it: writes put NULL into a NOT NULL column, and reads are scoped
|
||||
// "WHERE deleted_at IS NULL" and match nothing.
|
||||
//
|
||||
// Migrations after that version seed through the runtime models in app/.
|
||||
// TestPostConversionMigrationsAvoidFrozenSeedModels enforces this.
|
||||
type ModelTime struct {
|
||||
CreatedAt time.Time `json:"createdAt" gorm:"comment:创建时间"`
|
||||
UpdatedAt time.Time `json:"updatedAt" gorm:"comment:最后更新时间"`
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
package version
|
||||
|
||||
import (
|
||||
"runtime"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
// Normalize sys_role.data_scope to one of the five values
|
||||
// actions.Permission recognizes, ahead of PRD 006 F14/H2 making its
|
||||
// unrecognized-scope branch fail closed instead of fail open.
|
||||
//
|
||||
// Before that change, an empty or unrecognized data_scope fell into
|
||||
// Permission's default branch, which returned the query untouched - exactly
|
||||
// the same SQL as data_scope "1" (全部数据权限). The seed data shipped
|
||||
// precisely that: config/db.sql's built-in admin role (role_id 1) carries an
|
||||
// empty data_scope rather than "1". Once the default starts matching no
|
||||
// rows instead, that role would silently lose all visibility everywhere
|
||||
// actions.Permission is used, the moment a deployment turns EnableDP on.
|
||||
//
|
||||
// Rewriting every value outside {1,2,3,4,5} to "1" keeps each such role's
|
||||
// effective visibility exactly what it already was - a role that intended a
|
||||
// tighter scope was never getting it under the old fail-open default either,
|
||||
// so this does not tighten anything a deployment was relying on. Whether to
|
||||
// tighten it further is left to whoever owns that role.
|
||||
func init() {
|
||||
_, fileName, _, _ := runtime.Caller(0)
|
||||
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700005000NormalizeRoleDataScope)
|
||||
}
|
||||
|
||||
func _1786700005000NormalizeRoleDataScope(db *gorm.DB, version string) error {
|
||||
return db.Transaction(func(tx *gorm.DB) error {
|
||||
if err := normalizeRoleDataScope(tx); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Create(&common.Migration{Version: version}).Error
|
||||
})
|
||||
}
|
||||
|
||||
// normalizeRoleDataScope is split out so tests can run it against a database
|
||||
// that only has sys_role, without also standing up sys_migration.
|
||||
//
|
||||
// The explicit "IS NULL OR" matters: sys_role.data_scope has no NOT NULL
|
||||
// constraint, and SQL's three-valued logic makes `NULL NOT IN (...)`
|
||||
// evaluate to NULL rather than TRUE, so a bare NOT IN clause silently skips
|
||||
// NULL rows instead of normalizing them.
|
||||
func normalizeRoleDataScope(tx *gorm.DB) error {
|
||||
return tx.Exec(
|
||||
"UPDATE sys_role SET data_scope = '1' WHERE data_scope IS NULL OR data_scope NOT IN ('1', '2', '3', '4', '5')",
|
||||
).Error
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
package version
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/glebarez/sqlite"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type roleDataScopeRow struct {
|
||||
RoleId int `gorm:"column:role_id;primaryKey;autoIncrement"`
|
||||
DataScope string `gorm:"column:data_scope"`
|
||||
}
|
||||
|
||||
func (roleDataScopeRow) TableName() string { return "sys_role" }
|
||||
|
||||
func openRoleTable(t *testing.T) *gorm.DB {
|
||||
t.Helper()
|
||||
|
||||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{})
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(&roleDataScopeRow{}); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
return db
|
||||
}
|
||||
|
||||
// The migration exists because the shipped admin role is exactly this case:
|
||||
// config/db.sql's role_id 1 carries an empty data_scope. Reproduces the seed
|
||||
// data literally rather than a made-up example.
|
||||
func TestNormalizesTheEmptyDataScopeTheSeedDataShips(t *testing.T) {
|
||||
db := openRoleTable(t)
|
||||
if err := db.Create(&roleDataScopeRow{RoleId: 1, DataScope: ""}).Error; err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
|
||||
if err := normalizeRoleDataScope(db); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
|
||||
var row roleDataScopeRow
|
||||
if err := db.First(&row, 1).Error; err != nil {
|
||||
t.Fatalf("read back: %v", err)
|
||||
}
|
||||
if row.DataScope != "1" {
|
||||
t.Fatalf("data_scope = %q, want %q", row.DataScope, "1")
|
||||
}
|
||||
}
|
||||
|
||||
// The five recognized values must survive untouched - this migration
|
||||
// normalizes what Permission cannot make sense of, not what it already can.
|
||||
func TestLeavesRecognizedScopesAlone(t *testing.T) {
|
||||
db := openRoleTable(t)
|
||||
valid := []string{"1", "2", "3", "4", "5"}
|
||||
for i, scope := range valid {
|
||||
if err := db.Create(&roleDataScopeRow{RoleId: i + 1, DataScope: scope}).Error; err != nil {
|
||||
t.Fatalf("seed %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
|
||||
if err := normalizeRoleDataScope(db); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
|
||||
var rows []roleDataScopeRow
|
||||
if err := db.Order("role_id").Find(&rows).Error; err != nil {
|
||||
t.Fatalf("read back: %v", err)
|
||||
}
|
||||
for i, row := range rows {
|
||||
if row.DataScope != valid[i] {
|
||||
t.Errorf("role %d: data_scope = %q, want %q (untouched)", row.RoleId, row.DataScope, valid[i])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A garbage value (not just empty) must be normalized the same way as empty -
|
||||
// both are "not one of the five", and the migration's WHERE clause has to
|
||||
// catch both.
|
||||
func TestNormalizesGarbageScopesToo(t *testing.T) {
|
||||
db := openRoleTable(t)
|
||||
if err := db.Create(&roleDataScopeRow{RoleId: 1, DataScope: "6"}).Error; err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
|
||||
if err := normalizeRoleDataScope(db); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
|
||||
var row roleDataScopeRow
|
||||
if err := db.First(&row, 1).Error; err != nil {
|
||||
t.Fatalf("read back: %v", err)
|
||||
}
|
||||
if row.DataScope != "1" {
|
||||
t.Fatalf("data_scope = %q, want %q", row.DataScope, "1")
|
||||
}
|
||||
}
|
||||
|
||||
// A NULL data_scope must be normalized too. sys_role.data_scope has no NOT
|
||||
// NULL constraint, and `NULL NOT IN (...)` evaluates to NULL rather than
|
||||
// TRUE under SQL's three-valued logic, so a bare NOT IN clause would leave
|
||||
// this row untouched - the exact gap that let a NULL-scoped role go blind
|
||||
// once Permission's default branch starts fail-closing.
|
||||
func TestNormalizesNullDataScope(t *testing.T) {
|
||||
db := openRoleTable(t)
|
||||
if err := db.Exec("INSERT INTO sys_role (role_id, data_scope) VALUES (1, NULL)").Error; err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
|
||||
if err := normalizeRoleDataScope(db); err != nil {
|
||||
t.Fatalf("migrate: %v", err)
|
||||
}
|
||||
|
||||
var row roleDataScopeRow
|
||||
if err := db.First(&row, 1).Error; err != nil {
|
||||
t.Fatalf("read back: %v", err)
|
||||
}
|
||||
if row.DataScope != "1" {
|
||||
t.Fatalf("data_scope = %q, want %q", row.DataScope, "1")
|
||||
}
|
||||
}
|
||||
|
||||
// Running it twice must be safe: it is a plain UPDATE, not DDL, but
|
||||
// sys_migration only records success once, and an operator who reruns
|
||||
// `migrate` on a partially-applied database has to be able to trust that.
|
||||
func TestNormalizeRoleDataScopeIsRepeatable(t *testing.T) {
|
||||
db := openRoleTable(t)
|
||||
if err := db.Create(&roleDataScopeRow{RoleId: 1, DataScope: ""}).Error; err != nil {
|
||||
t.Fatalf("seed: %v", err)
|
||||
}
|
||||
|
||||
for i := 0; i < 3; i++ {
|
||||
if err := normalizeRoleDataScope(db); err != nil {
|
||||
t.Fatalf("migrate %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
|
||||
var row roleDataScopeRow
|
||||
if err := db.First(&row, 1).Error; err != nil {
|
||||
t.Fatalf("read back: %v", err)
|
||||
}
|
||||
if row.DataScope != "1" {
|
||||
t.Fatalf("data_scope = %q, want %q", row.DataScope, "1")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
package version
|
||||
|
||||
import (
|
||||
"runtime"
|
||||
|
||||
"gorm.io/gorm"
|
||||
|
||||
adminmodels "go-admin/app/admin/models"
|
||||
"go-admin/cmd/migrate/migration"
|
||||
common "go-admin/common/models"
|
||||
)
|
||||
|
||||
// Add sys_menu.app_code and sys_api.app_code ahead of PRD 006 F9's Seeder.
|
||||
//
|
||||
// Every row a third-party application's migration writes through
|
||||
// seed.SeedMenus must be attributable to the app that wrote it, so
|
||||
// installing, auditing, or removing one application does not require
|
||||
// guessing which rows belong to it - see go-admin-core's docs/contract.md,
|
||||
// "Application-supplied menu and API entries", for the requirement this
|
||||
// satisfies.
|
||||
//
|
||||
// Ordered after 1786700003000, so importing cmd/migrate/migration/models is
|
||||
// banned here (see schema_coverage_test.go's
|
||||
// TestPostConversionMigrationsAvoidFrozenSeedModels): AddColumn instead
|
||||
// reads the runtime models' own gorm tags directly, which is also what
|
||||
// makes the column this adds match the one the admin Seeder writes through
|
||||
// those same structs.
|
||||
func init() {
|
||||
_, fileName, _, _ := runtime.Caller(0)
|
||||
migration.Migrate.SetVersion(migration.GetFilename(fileName), _1786700006000AppCodeColumns)
|
||||
}
|
||||
|
||||
func _1786700006000AppCodeColumns(db *gorm.DB, version string) error {
|
||||
m := db.Migrator()
|
||||
if !m.HasColumn(&adminmodels.SysMenu{}, "AppCode") {
|
||||
if err := m.AddColumn(&adminmodels.SysMenu{}, "AppCode"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if !m.HasColumn(&adminmodels.SysApi{}, "AppCode") {
|
||||
if err := m.AddColumn(&adminmodels.SysApi{}, "AppCode"); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return db.Create(&common.Migration{Version: version}).Error
|
||||
}
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
// The repository carries two ModelTime types. The one in
|
||||
@@ -79,9 +81,13 @@ func runtimeSoftDeleteTables(t *testing.T) map[string]string {
|
||||
}
|
||||
|
||||
func importsRuntimeModels(f *ast.File) bool {
|
||||
return importsPackage(f, "go-admin/common/models")
|
||||
}
|
||||
|
||||
func importsPackage(f *ast.File, pkg string) bool {
|
||||
for _, imp := range f.Imports {
|
||||
p, err := strconv.Unquote(imp.Path.Value)
|
||||
if err == nil && p == "go-admin/common/models" {
|
||||
if err == nil && p == pkg {
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -171,3 +177,89 @@ func repoRoot(t *testing.T) string {
|
||||
t.Fatal("go.mod not found above the test directory")
|
||||
return ""
|
||||
}
|
||||
|
||||
// softDeleteConversion is the version at which sys_api, sys_menu and the rest
|
||||
// stop storing deleted_at as a nullable timestamp and start storing the NOT
|
||||
// NULL millisecond marker.
|
||||
const softDeleteConversion = 1786700003000
|
||||
|
||||
// versionPrefixLen is the width migration.GetFilename slices off a filename.
|
||||
const versionPrefixLen = 13
|
||||
|
||||
// Migrations ordered after the conversion must not seed rows through
|
||||
// cmd/migrate/migration/models.
|
||||
//
|
||||
// That package's ModelTime still declares a nullable gorm.DeletedAt, which is
|
||||
// correct for the migrations that predate the conversion - it is the shape the
|
||||
// column had when they ran. Reusing it afterwards writes NULL into a NOT NULL
|
||||
// column and the migration fails on its first insert:
|
||||
//
|
||||
// NOT NULL constraint failed: sys_api.deleted_at
|
||||
//
|
||||
// A fresh database never catches this, because every migration using that
|
||||
// package today is ordered before the conversion and so runs while the column
|
||||
// is still nullable. Only a migration added afterwards hits it, which in
|
||||
// practice means the next person adding a business module - the reference
|
||||
// they copy, 1786700001000_demo_menu.go, is itself one of the safe ones.
|
||||
//
|
||||
// Reads through that package are worse than writes, which is why the whole
|
||||
// import is banned rather than just the inserts. gorm scopes a nullable
|
||||
// DeletedAt as "WHERE deleted_at IS NULL", and after the conversion live rows
|
||||
// hold 0, so the row is simply not there:
|
||||
//
|
||||
// frozen SysRole -> record not found
|
||||
// runtime SysRole -> roleId=1
|
||||
//
|
||||
// 1786700001000_demo_menu.go looks the admin role up that way and treats
|
||||
// ErrRecordNotFound as "roles are not seeded yet, skip authorisation". A
|
||||
// post-conversion copy that switched its inserts to the runtime models but
|
||||
// kept this lookup would seed the menu, grant nothing, and still record the
|
||||
// migration as applied - the menu appears, its buttons do nothing, and no
|
||||
// error is reported anywhere.
|
||||
func TestPostConversionMigrationsAvoidFrozenSeedModels(t *testing.T) {
|
||||
const frozenModels = "go-admin/cmd/migrate/migration/models"
|
||||
|
||||
dir, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
checked := 0
|
||||
for _, e := range entries {
|
||||
name := e.Name()
|
||||
if e.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") {
|
||||
continue
|
||||
}
|
||||
// GetFilename is what every migration uses to derive its own version,
|
||||
// so the two stay in step if the filename convention ever changes.
|
||||
if len(name) < versionPrefixLen {
|
||||
continue
|
||||
}
|
||||
version, err := strconv.ParseInt(migration.GetFilename(name), 10, 64)
|
||||
if err != nil || version <= softDeleteConversion {
|
||||
continue // not a versioned migration, or one that predates the change
|
||||
}
|
||||
checked++
|
||||
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, filepath.Join(dir, name), nil, parser.ImportsOnly)
|
||||
if err != nil {
|
||||
t.Fatalf("parse %s: %v", name, err)
|
||||
}
|
||||
if importsPackage(f, frozenModels) {
|
||||
t.Errorf("%s is ordered after the soft-delete conversion but seeds through %s;\n"+
|
||||
" that package writes a nullable deleted_at and will fail with\n"+
|
||||
" \"NOT NULL constraint failed\" on its first insert.\n"+
|
||||
" Use the runtime models under app/ instead - they carry the marker.",
|
||||
name, frozenModels)
|
||||
}
|
||||
}
|
||||
|
||||
if checked == 0 {
|
||||
t.Fatal("no post-conversion migrations found; the scan is broken, not the code")
|
||||
}
|
||||
}
|
||||
|
||||
+141
-8
@@ -3,12 +3,16 @@ package migrate
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"text/template"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"gorm.io/gorm"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/config/source/file"
|
||||
"github.com/spf13/cobra"
|
||||
|
||||
@@ -25,6 +29,8 @@ var (
|
||||
generate bool
|
||||
goAdmin bool
|
||||
host string
|
||||
appCode string
|
||||
dryRun bool
|
||||
StartCmd = &cobra.Command{
|
||||
Use: "migrate",
|
||||
Short: "Initialize the database",
|
||||
@@ -33,14 +39,31 @@ var (
|
||||
run()
|
||||
},
|
||||
}
|
||||
statusCmd = &cobra.Command{
|
||||
Use: "status",
|
||||
Short: "List applied and pending migrations, grouped by app",
|
||||
Example: "go-admin migrate status -c config/settings.yml",
|
||||
Run: func(cmd *cobra.Command, args []string) {
|
||||
runStatus()
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
// fixme 在您看不见代码的时候运行迁移,我觉得是不安全的,所以编译后最好不要去执行迁移
|
||||
func init() {
|
||||
StartCmd.PersistentFlags().StringVarP(&configYml, "config", "c", "config/settings.yml", "Start server with provided configuration file")
|
||||
StartCmd.PersistentFlags().BoolVarP(&generate, "generate", "g", false, "generate migration file")
|
||||
StartCmd.PersistentFlags().BoolVarP(&goAdmin, "goAdmin", "a", false, "generate go-admin migration file")
|
||||
StartCmd.PersistentFlags().BoolVarP(&goAdmin, "goAdmin", "a", false, "with -g, write the generated file to version/ instead of version-local/ (does not affect which migrations run)")
|
||||
StartCmd.PersistentFlags().StringVarP(&host, "domain", "d", "*", "select tenant host")
|
||||
|
||||
// --app is deliberately long-only. -a already means "generate into
|
||||
// version/ rather than version-local/", which is about writing a template
|
||||
// file, not about which migrations run; giving the two the same letter
|
||||
// would be a trap.
|
||||
StartCmd.PersistentFlags().StringVar(&appCode, "app", "", "limit to the migrations of one app (\""+migration.FrameworkAppCode+"\" for the framework's own)")
|
||||
StartCmd.Flags().BoolVar(&dryRun, "dry-run", false, "list what would be applied, in order, and write nothing")
|
||||
|
||||
StartCmd.AddCommand(statusCmd)
|
||||
}
|
||||
|
||||
func run() {
|
||||
@@ -58,7 +81,12 @@ func run() {
|
||||
}
|
||||
}
|
||||
|
||||
func migrateModel() error {
|
||||
// resolveDB picks the tenant database and hands it to the registry.
|
||||
//
|
||||
// It creates and alters nothing, which is what lets status and --dry-run share
|
||||
// it: those two must be able to run against a production database without
|
||||
// leaving a trace.
|
||||
func resolveDB() (*gorm.DB, error) {
|
||||
if host == "" {
|
||||
host = "*"
|
||||
}
|
||||
@@ -73,29 +101,134 @@ func migrateModel() error {
|
||||
}
|
||||
}
|
||||
if db == nil {
|
||||
return fmt.Errorf("未找到数据库配置")
|
||||
return nil, fmt.Errorf("未找到数据库配置")
|
||||
}
|
||||
if config.DatabasesConfig[host].Driver == "mysql" {
|
||||
//初始化数据库时候用
|
||||
db.Set("gorm:table_options", "ENGINE=InnoDB CHARSET=utf8mb4")
|
||||
}
|
||||
err := db.Debug().AutoMigrate(&models.Migration{})
|
||||
return db, nil
|
||||
}
|
||||
|
||||
// exitUnlessAppRegistered ends the command when --app names something no
|
||||
// migration was registered under.
|
||||
//
|
||||
// Every path took a typo as "nothing matched" and reported success: `migrate`
|
||||
// printed that the app was unknown and still exited 0, while `--dry-run` and
|
||||
// `status` said "nothing to apply" and "none recorded" - which is what an
|
||||
// up-to-date database says too, so the output does not even hint at the typo.
|
||||
// An operator running `go-admin migrate --app crmm && deploy` gets the deploy.
|
||||
//
|
||||
// Checked against the registry, which init() has already filled, so this runs
|
||||
// before any database work and costs nothing. It lives in the command layer
|
||||
// because the exit code does: the migration package stays callable from a test
|
||||
// without taking the process down with it.
|
||||
func exitUnlessAppRegistered() {
|
||||
if err := appRegistrationError(); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
// appRegistrationError carries the decision on its own so it can be tested;
|
||||
// exitUnlessAppRegistered is only the os.Exit around it. Nil means --app was
|
||||
// either empty or names a registered app.
|
||||
func appRegistrationError() error {
|
||||
if appCode == "" {
|
||||
return nil
|
||||
}
|
||||
want := migration.DisplayAppCode(migration.AppFilter(appCode))
|
||||
registered := migration.Migrate.AppCodes()
|
||||
for _, c := range registered {
|
||||
if c == want {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("no migrations are registered for app %q; registered: %s",
|
||||
want, strings.Join(registered, ", "))
|
||||
}
|
||||
|
||||
func migrateModel() error {
|
||||
db, err := resolveDB()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// sys_migration is the one table that never goes through a versioned
|
||||
// migration - it is the table that records them. AutoMigrate realigns it
|
||||
// on every run, which is how the app_code column reaches an existing
|
||||
// database without anyone writing a migration for it.
|
||||
if err = db.Debug().AutoMigrate(&models.Migration{}); err != nil {
|
||||
return err
|
||||
}
|
||||
migration.Migrate.SetDb(db.Debug())
|
||||
if appCode != "" {
|
||||
migration.Migrate.MigrateApp(appCode)
|
||||
return nil
|
||||
}
|
||||
migration.Migrate.Migrate()
|
||||
return err
|
||||
return nil
|
||||
}
|
||||
|
||||
func initDB() {
|
||||
// Before the database is touched, so a typo cannot get as far as looking
|
||||
// like a successful no-op on either path below.
|
||||
exitUnlessAppRegistered()
|
||||
|
||||
//3. 初始化数据库链接
|
||||
database.Setup()
|
||||
|
||||
if dryRun {
|
||||
db, err := resolveDB()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
migration.Migrate.SetDb(db)
|
||||
entries, err := migration.Migrate.Status()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
if err = printPending(os.Stdout, entries, appCode); err != nil {
|
||||
fmt.Println(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
//4. 数据库迁移
|
||||
fmt.Println("数据库迁移开始")
|
||||
_ = migrateModel()
|
||||
if err := migrateModel(); err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
fmt.Println(`数据库基础数据初始化成功`)
|
||||
}
|
||||
|
||||
func runStatus() {
|
||||
config.Setup(
|
||||
file.NewSource(file.WithPath(configYml)),
|
||||
func() {
|
||||
exitUnlessAppRegistered()
|
||||
|
||||
database.Setup()
|
||||
db, err := resolveDB()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
migration.Migrate.SetDb(db)
|
||||
entries, err := migration.Migrate.Status()
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
return
|
||||
}
|
||||
if err = printStatus(os.Stdout, entries, appCode); err != nil {
|
||||
fmt.Println(err)
|
||||
}
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
func genFile() error {
|
||||
t1, err := template.ParseFiles("template/migrate.template")
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
const applyTimeLayout = "2006-01-02 15:04:05"
|
||||
|
||||
// printStatus lists every migration this binary knows about together with every
|
||||
// row already in sys_migration, grouped by app.
|
||||
//
|
||||
// filter is an app code as typed on the command line; empty means every app.
|
||||
func printStatus(w io.Writer, entries []migration.StatusEntry, filter string) error {
|
||||
entries = filterByApp(entries, filter)
|
||||
|
||||
groups, order := groupByApp(entries)
|
||||
if len(order) == 0 {
|
||||
_, err := fmt.Fprintln(w, "no migrations registered and none recorded")
|
||||
return err
|
||||
}
|
||||
|
||||
// One width for the whole listing rather than one per group: the versions
|
||||
// of two apps line up, so a long list can be read down the column.
|
||||
width := versionWidth(entries)
|
||||
|
||||
var applied, pending, orphaned int
|
||||
for i, app := range order {
|
||||
if i > 0 {
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
fmt.Fprintf(w, "[%s]\n", app)
|
||||
for _, e := range groups[app] {
|
||||
state := "pending"
|
||||
switch {
|
||||
case e.Applied && !e.Registered:
|
||||
state = "orphaned"
|
||||
orphaned++
|
||||
case e.Applied:
|
||||
state = "applied"
|
||||
applied++
|
||||
default:
|
||||
pending++
|
||||
}
|
||||
fmt.Fprintln(w, strings.TrimRight(
|
||||
fmt.Sprintf(" %-*s%-*s%s", stateWidth, state, width, e.Version, formatApplyTime(e.ApplyTime)), " "))
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Fprintf(w, "\n%d applied, %d pending across %d app(s)\n", applied, pending, len(order))
|
||||
if orphaned > 0 {
|
||||
fmt.Fprintf(w, "%d orphaned: recorded in sys_migration, but nothing in this binary registers them.\n"+
|
||||
"Expected after a migration file is removed or an app is uninstalled; they will not run again.\n", orphaned)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// printPending is --dry-run: the same data as status, narrowed to what an
|
||||
// actual run would do and printed in the order it would do it.
|
||||
//
|
||||
// It reads and prints. Every write path - AutoMigrate on sys_migration
|
||||
// included - is on the other branch in initDB, so a dry run leaves the database
|
||||
// byte for byte as it found it.
|
||||
func printPending(w io.Writer, entries []migration.StatusEntry, filter string) error {
|
||||
entries = filterByApp(entries, filter)
|
||||
|
||||
fmt.Fprintln(w, "dry-run: nothing will be written")
|
||||
|
||||
pending := make([]migration.StatusEntry, 0, len(entries))
|
||||
for _, e := range entries {
|
||||
// An orphaned row is recorded and unregistered; a real run cannot
|
||||
// apply it, so a dry run must not offer to.
|
||||
if !e.Applied && e.Registered {
|
||||
pending = append(pending, e)
|
||||
}
|
||||
}
|
||||
if len(pending) == 0 {
|
||||
_, err := fmt.Fprintln(w, "nothing to apply")
|
||||
return err
|
||||
}
|
||||
|
||||
appWidth := 0
|
||||
for _, e := range pending {
|
||||
if n := len(migration.DisplayAppCode(e.AppCode)) + 2; n > appWidth {
|
||||
appWidth = n
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Fprintln(w, "would apply, in this order:")
|
||||
for _, e := range pending {
|
||||
fmt.Fprintf(w, " %-*s%s\n", appWidth+2, "["+migration.DisplayAppCode(e.AppCode)+"]", e.Version)
|
||||
}
|
||||
fmt.Fprintf(w, "\n%d migration(s) pending\n", len(pending))
|
||||
return nil
|
||||
}
|
||||
|
||||
// stateWidth is the width of the applied/pending/orphaned column, sized to the
|
||||
// longest of the three plus a gap.
|
||||
const stateWidth = len("orphaned") + 2
|
||||
|
||||
func versionWidth(entries []migration.StatusEntry) int {
|
||||
width := 0
|
||||
for _, e := range entries {
|
||||
if n := len(e.Version) + 2; n > width {
|
||||
width = n
|
||||
}
|
||||
}
|
||||
return width
|
||||
}
|
||||
|
||||
// filterByApp keeps the entries of one app. The filter is matched after the
|
||||
// same normalisation ForApp applies, so --app CRM finds crm.
|
||||
func filterByApp(entries []migration.StatusEntry, filter string) []migration.StatusEntry {
|
||||
if filter == "" {
|
||||
return entries
|
||||
}
|
||||
want := migration.AppFilter(filter)
|
||||
out := make([]migration.StatusEntry, 0, len(entries))
|
||||
for _, e := range entries {
|
||||
if e.AppCode == want {
|
||||
out = append(out, e)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// groupByApp buckets entries by display name and returns the buckets plus the
|
||||
// order to print them in: the framework first, then apps alphabetically. That
|
||||
// is also the order a full run executes them in, because version strings sort
|
||||
// as ASCII and the framework's are bare digits.
|
||||
func groupByApp(entries []migration.StatusEntry) (map[string][]migration.StatusEntry, []string) {
|
||||
groups := make(map[string][]migration.StatusEntry)
|
||||
for _, e := range entries {
|
||||
app := migration.DisplayAppCode(e.AppCode)
|
||||
groups[app] = append(groups[app], e)
|
||||
}
|
||||
order := make([]string, 0, len(groups))
|
||||
for app := range groups {
|
||||
order = append(order, app)
|
||||
}
|
||||
sort.Slice(order, func(i, j int) bool {
|
||||
if (order[i] == migration.FrameworkAppCode) != (order[j] == migration.FrameworkAppCode) {
|
||||
return order[i] == migration.FrameworkAppCode
|
||||
}
|
||||
return order[i] < order[j]
|
||||
})
|
||||
return groups, order
|
||||
}
|
||||
|
||||
func formatApplyTime(t *time.Time) string {
|
||||
if t == nil {
|
||||
return ""
|
||||
}
|
||||
return t.Format(applyTimeLayout)
|
||||
}
|
||||
@@ -0,0 +1,170 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"go-admin/cmd/migrate/migration"
|
||||
)
|
||||
|
||||
func at(s string) *time.Time {
|
||||
t, err := time.Parse(applyTimeLayout, s)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return &t
|
||||
}
|
||||
|
||||
// The order Status returns: version strings sorted as ASCII.
|
||||
func sampleEntries() []migration.StatusEntry {
|
||||
return []migration.StatusEntry{
|
||||
{Version: "1786700001000", AppCode: "", Registered: true, Applied: true, ApplyTime: at("2026-08-20 10:00:00")},
|
||||
{Version: "1786700005000", AppCode: "", Registered: true},
|
||||
{Version: "crm-1786800001000", AppCode: "crm", Registered: true, Applied: true, ApplyTime: at("2026-08-25 14:03:11")},
|
||||
{Version: "crm-1786800002000", AppCode: "crm", Registered: true},
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintStatusGroupsByApp(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, sampleEntries(), ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
|
||||
for _, want := range []string{
|
||||
"[core]",
|
||||
"[crm]",
|
||||
"applied 1786700001000 2026-08-20 10:00:00",
|
||||
"pending 1786700005000",
|
||||
"applied crm-1786800001000 2026-08-25 14:03:11",
|
||||
"pending crm-1786800002000",
|
||||
"2 applied, 2 pending across 2 app(s)",
|
||||
} {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("output missing %q:\n%s", want, got)
|
||||
}
|
||||
}
|
||||
// The framework heads the list, because that is the order a full run
|
||||
// executes in.
|
||||
if strings.Index(got, "[core]") > strings.Index(got, "[crm]") {
|
||||
t.Errorf("core is not listed first:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A row nobody registers any more is neither applied-and-current nor pending.
|
||||
// Calling it applied would say the migration is in this binary, which is what
|
||||
// sends someone looking for a file that was deleted.
|
||||
func TestPrintStatusMarksOrphanedRows(t *testing.T) {
|
||||
entries := append(sampleEntries(), migration.StatusEntry{
|
||||
Version: "gone-1786800000000", AppCode: "gone", Applied: true, ApplyTime: at("2026-08-01 09:00:00"),
|
||||
})
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, entries, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if !strings.Contains(got, "orphaned gone-1786800000000") {
|
||||
t.Errorf("orphaned row not marked:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "nothing in this binary registers them") {
|
||||
t.Errorf("orphaned rows need an explanation:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "2 applied, 2 pending") {
|
||||
t.Errorf("orphaned rows must not be counted as applied:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintStatusFiltersByApp(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, sampleEntries(), "crm"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if strings.Contains(got, "[core]") {
|
||||
t.Errorf("--app crm listed the framework:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "across 1 app(s)") {
|
||||
t.Errorf("output = %s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// status prints [core]; --app core has to mean the same thing.
|
||||
func TestPrintStatusAppCoreSelectsTheFramework(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, sampleEntries(), migration.FrameworkAppCode); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if strings.Contains(got, "[crm]") {
|
||||
t.Errorf("--app core listed crm:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "[core]") {
|
||||
t.Errorf("--app core listed nothing:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintStatusOnAnEmptyRegistry(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printStatus(&buf, nil, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(buf.String(), "no migrations registered and none recorded") {
|
||||
t.Errorf("output = %s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintPendingListsOnlyPendingInOrder(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printPending(&buf, sampleEntries(), ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
|
||||
if !strings.Contains(got, "dry-run: nothing will be written") {
|
||||
t.Errorf("dry-run must say it writes nothing:\n%s", got)
|
||||
}
|
||||
if strings.Contains(got, "1786700001000\n") || strings.Contains(got, "crm-1786800001000") {
|
||||
t.Errorf("dry-run listed already applied migrations:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "[core] 1786700005000") || !strings.Contains(got, "[crm] crm-1786800002000") {
|
||||
t.Errorf("dry-run is missing pending migrations:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "2 migration(s) pending") {
|
||||
t.Errorf("output = %s", got)
|
||||
}
|
||||
if strings.Index(got, "1786700005000") > strings.Index(got, "crm-1786800002000") {
|
||||
t.Errorf("dry-run order does not match run order:\n%s", got)
|
||||
}
|
||||
}
|
||||
|
||||
// An orphaned row is applied and unregistered; a dry run must not offer to
|
||||
// apply it, because a real run cannot.
|
||||
func TestPrintPendingSkipsOrphanedRows(t *testing.T) {
|
||||
entries := []migration.StatusEntry{
|
||||
{Version: "gone-1786800000000", AppCode: "gone", Applied: true, ApplyTime: at("2026-08-01 09:00:00")},
|
||||
}
|
||||
var buf bytes.Buffer
|
||||
if err := printPending(&buf, entries, ""); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(buf.String(), "nothing to apply") {
|
||||
t.Errorf("output = %s", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrintPendingFiltersByApp(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
if err := printPending(&buf, sampleEntries(), "CRM"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got := buf.String()
|
||||
if strings.Contains(got, "[core]") {
|
||||
t.Errorf("--app CRM listed the framework:\n%s", got)
|
||||
}
|
||||
if !strings.Contains(got, "1 migration(s) pending") {
|
||||
t.Errorf("output = %s", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
package actions_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/glebarez/sqlite"
|
||||
"gorm.io/gorm"
|
||||
gormlogger "gorm.io/gorm/logger"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
|
||||
"go-admin/common/actions"
|
||||
"go-admin/common/dto"
|
||||
"go-admin/common/models"
|
||||
)
|
||||
|
||||
// capturingLogger records every SQL statement GORM actually executes, so a
|
||||
// test can inspect it the way inspecting a *gorm.DB's own Statement cannot:
|
||||
// IndexAction builds and executes its query in one unbroken chain
|
||||
// (db.Model(...).Scopes(...).Find(...)...Count(...)) and never hands the
|
||||
// built statement back to its caller.
|
||||
type capturingLogger struct {
|
||||
gormlogger.Interface
|
||||
mu sync.Mutex
|
||||
stmts []string
|
||||
}
|
||||
|
||||
func (l *capturingLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
|
||||
sql, _ := fc()
|
||||
l.mu.Lock()
|
||||
l.stmts = append(l.stmts, sql)
|
||||
l.mu.Unlock()
|
||||
}
|
||||
|
||||
func (l *capturingLogger) all() string {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
return strings.Join(l.stmts, "\n")
|
||||
}
|
||||
|
||||
// probeRow is a minimal model satisfying models.ActiveRecord through the
|
||||
// same embeds a real app/admin model uses, so IndexAction sees exactly the
|
||||
// shape it is written against.
|
||||
type probeRow struct {
|
||||
models.Model
|
||||
models.ControlBy
|
||||
Name string
|
||||
}
|
||||
|
||||
func (probeRow) TableName() string { return "action_probe_row" }
|
||||
func (e *probeRow) Generate() models.ActiveRecord { o := *e; return &o }
|
||||
func (e *probeRow) GetId() interface{} { return e.Id }
|
||||
|
||||
// probeIndexReq is a minimal dto.Index: no search tags, page defaults.
|
||||
type probeIndexReq struct {
|
||||
dto.Pagination `search:"-"`
|
||||
}
|
||||
|
||||
// Generate returns a copy, the way every dto.Index in this repository does:
|
||||
// IndexAction closes over one instance and serves every request to the route
|
||||
// from it, so returning the receiver would share one struct across them. The
|
||||
// probe has to model that faithfully or it is not the shape IndexAction is
|
||||
// written against.
|
||||
func (p *probeIndexReq) Generate() dto.Index { o := *p; return &o }
|
||||
func (p *probeIndexReq) Bind(*gin.Context) error { return nil }
|
||||
func (p *probeIndexReq) GetNeedSearch() interface{} { return *p }
|
||||
|
||||
type pageEnvelope struct {
|
||||
Code int32 `json:"code"`
|
||||
}
|
||||
|
||||
// TestIndexActionAppliesDataPermission is an end-to-end guard core's own
|
||||
// test suite cannot provide. The five generic CRUD actions in this package
|
||||
// (create/delete/index/update/view.go) were not lowered to core (PRD 006
|
||||
// F3) - they still call actions.Permission directly, in this repository, on
|
||||
// a code path core knows nothing about. core's tests pin down what
|
||||
// Permission does for a given scope; nothing pinned down whether this
|
||||
// package's own Actions still remember to call it at all. This runs
|
||||
// IndexAction exactly as a real request would, against a real in-memory
|
||||
// database, and inspects the SQL GORM actually executed - not just that
|
||||
// the handler returned success, which it would just as happily do with no
|
||||
// filter applied at all.
|
||||
func TestProbeIndexReqGenerateReturnsAFreshInstance(t *testing.T) {
|
||||
p := &probeIndexReq{}
|
||||
got := p.Generate()
|
||||
if got == dto.Index(p) {
|
||||
t.Fatal("Generate returned the receiver; IndexAction would share one instance across every request to the route")
|
||||
}
|
||||
}
|
||||
|
||||
func TestIndexActionAppliesDataPermission(t *testing.T) {
|
||||
previous := config.ApplicationConfig.EnableDP
|
||||
config.ApplicationConfig.EnableDP = true
|
||||
t.Cleanup(func() { config.ApplicationConfig.EnableDP = previous })
|
||||
|
||||
cl := &capturingLogger{Interface: gormlogger.Default.LogMode(gormlogger.Silent)}
|
||||
db, err := gorm.Open(sqlite.Open(":memory:"), &gorm.Config{Logger: cl})
|
||||
if err != nil {
|
||||
t.Fatalf("open: %v", err)
|
||||
}
|
||||
if err := db.AutoMigrate(&probeRow{}); err != nil {
|
||||
t.Fatalf("AutoMigrate: %v", err)
|
||||
}
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
w := httptest.NewRecorder()
|
||||
c, _ := gin.CreateTestContext(w)
|
||||
c.Request = httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
c.Set("db", db)
|
||||
c.Set(actions.PermissionKey, &actions.DataPermission{DataScope: actions.DataScopeSelf, UserId: 7})
|
||||
|
||||
actions.IndexAction(&probeRow{}, &probeIndexReq{}, func() interface{} { return &[]probeRow{} })(c)
|
||||
|
||||
var body pageEnvelope
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
|
||||
t.Fatalf("decoding response body %q: %v", w.Body.String(), err)
|
||||
}
|
||||
if body.Code != http.StatusOK {
|
||||
t.Fatalf("response code = %d, want %d; body=%s", body.Code, http.StatusOK, w.Body.String())
|
||||
}
|
||||
|
||||
sql := cl.all()
|
||||
const wantFragment = "action_probe_row.create_by = "
|
||||
if !strings.Contains(sql, wantFragment) {
|
||||
t.Fatalf("IndexAction did not apply the data-permission scope to its query; want SQL containing %q, got:\n%s", wantFragment, sql)
|
||||
}
|
||||
}
|
||||
+32
-122
@@ -1,138 +1,48 @@
|
||||
package actions
|
||||
|
||||
import (
|
||||
"errors"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/jwtauth/user"
|
||||
log "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/response"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
"gorm.io/gorm"
|
||||
|
||||
contractactions "github.com/go-admin-team/go-admin-core/v2/sdk/contract/actions"
|
||||
)
|
||||
|
||||
type DataPermission struct {
|
||||
DataScope string
|
||||
UserId int
|
||||
DeptId int
|
||||
RoleId int
|
||||
}
|
||||
// DataPermission is a thin alias of go-admin-core's sdk/contract/actions
|
||||
// (PRD 006 F3/F5).
|
||||
type DataPermission = contractactions.DataPermission
|
||||
|
||||
// The five values sys_role.data_scope can hold, referenced directly from
|
||||
// go-admin-core's sdk/contract/actions rather than restated as literals -
|
||||
// see that package's DataScope* doc comment and PRD 006's hard constraint 4.
|
||||
const (
|
||||
DataScopeAll = contractactions.DataScopeAll
|
||||
DataScopeCustom = contractactions.DataScopeCustom
|
||||
DataScopeDept = contractactions.DataScopeDept
|
||||
DataScopeDeptTree = contractactions.DataScopeDeptTree
|
||||
DataScopeSelf = contractactions.DataScopeSelf
|
||||
)
|
||||
|
||||
// PermissionAction, Permission, GetPermissionFromContext and
|
||||
// IsValidDataScope forward to go-admin-core's sdk/contract/actions (PRD 006
|
||||
// F3/F5). create.go/delete.go/index.go/update.go/view.go in this package
|
||||
// (the generic CRUD actions, which do not move to core) call Permission and
|
||||
// GetPermissionFromContext by these same names and are unchanged by the
|
||||
// move: the names now resolve to forwards instead of local definitions, and
|
||||
// the behaviour is identical either way.
|
||||
func PermissionAction() gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
// Permission() below returns the query untouched when data permission
|
||||
// is off, so the lookup that feeds it has nothing to feed. It used to
|
||||
// run anyway: a sys_user join on every list, detail, update and delete,
|
||||
// with the result discarded.
|
||||
if !config.ApplicationConfig.EnableDP {
|
||||
c.Set(PermissionKey, new(DataPermission))
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
|
||||
userId := user.GetUserIdStr(c)
|
||||
if userId == "" {
|
||||
c.Set(PermissionKey, new(DataPermission))
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
|
||||
// The token already carries what the scope is decided by. Reading it
|
||||
// there costs nothing, and goes no more stale than rolekey does - which
|
||||
// Casbin has always read from the token.
|
||||
if p, ok := permissionFromClaims(c); ok {
|
||||
c.Set(PermissionKey, p)
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
|
||||
db, err := pkg.GetOrm(c)
|
||||
if err != nil {
|
||||
log.Error(err)
|
||||
return
|
||||
}
|
||||
msgID := pkg.GenerateMsgIDFromContext(c)
|
||||
p, err := newDataPermission(db, userId)
|
||||
if err != nil {
|
||||
log.Errorf("MsgID[%s] PermissionAction error: %s", msgID, err)
|
||||
response.Error(c, 500, err, "权限范围鉴定错误")
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
c.Set(PermissionKey, p)
|
||||
c.Next()
|
||||
}
|
||||
}
|
||||
|
||||
// permissionFromClaims builds the scope from the token, reporting false when
|
||||
// the token predates deptid being carried. Such a token still exists until it
|
||||
// expires, and it has to keep working.
|
||||
func permissionFromClaims(c *gin.Context) (*DataPermission, bool) {
|
||||
claims := user.ExtractClaims(c)
|
||||
if claims["deptid"] == nil || claims["datascope"] == nil {
|
||||
return nil, false
|
||||
}
|
||||
scope, ok := claims["datascope"].(string)
|
||||
if !ok {
|
||||
return nil, false
|
||||
}
|
||||
return &DataPermission{
|
||||
DataScope: scope,
|
||||
UserId: user.GetUserId(c),
|
||||
DeptId: user.GetDeptId(c),
|
||||
RoleId: user.GetRoleId(c),
|
||||
}, true
|
||||
}
|
||||
|
||||
func newDataPermission(tx *gorm.DB, userId interface{}) (*DataPermission, error) {
|
||||
var err error
|
||||
p := &DataPermission{}
|
||||
|
||||
err = tx.Table("sys_user").
|
||||
Select("sys_user.user_id", "sys_role.role_id", "sys_user.dept_id", "sys_role.data_scope").
|
||||
Joins("left join sys_role on sys_role.role_id = sys_user.role_id").
|
||||
Where("sys_user.user_id = ?", userId).
|
||||
Scan(p).Error
|
||||
if err != nil {
|
||||
err = errors.New("获取用户数据出错 msg:" + err.Error())
|
||||
return nil, err
|
||||
}
|
||||
return p, nil
|
||||
return contractactions.PermissionAction()
|
||||
}
|
||||
|
||||
func Permission(tableName string, p *DataPermission) func(db *gorm.DB) *gorm.DB {
|
||||
return func(db *gorm.DB) *gorm.DB {
|
||||
if !config.ApplicationConfig.EnableDP {
|
||||
return db
|
||||
}
|
||||
switch p.DataScope {
|
||||
case "2":
|
||||
return db.Where(tableName+".create_by in (select sys_user.user_id from sys_role_dept left join sys_user on sys_user.dept_id=sys_role_dept.dept_id where sys_role_dept.role_id = ?)", p.RoleId)
|
||||
case "3":
|
||||
return db.Where(tableName+".create_by in (SELECT user_id from sys_user where dept_id = ? )", p.DeptId)
|
||||
case "4":
|
||||
return db.Where(tableName+".create_by in (SELECT user_id from sys_user where sys_user.dept_id in(select dept_id from sys_dept where dept_path like ? ))", "%/"+pkg.IntToString(p.DeptId)+"/%")
|
||||
case "5":
|
||||
return db.Where(tableName+".create_by = ?", p.UserId)
|
||||
default:
|
||||
return db
|
||||
}
|
||||
}
|
||||
return contractactions.Permission(tableName, p)
|
||||
}
|
||||
|
||||
func getPermissionFromContext(c *gin.Context) *DataPermission {
|
||||
p := new(DataPermission)
|
||||
if pm, ok := c.Get(PermissionKey); ok {
|
||||
switch pm.(type) {
|
||||
case *DataPermission:
|
||||
p = pm.(*DataPermission)
|
||||
}
|
||||
}
|
||||
return p
|
||||
}
|
||||
|
||||
// GetPermissionFromContext 提供非action写法数据范围约束
|
||||
func GetPermissionFromContext(c *gin.Context) *DataPermission {
|
||||
return getPermissionFromContext(c)
|
||||
return contractactions.GetPermissionFromContext(c)
|
||||
}
|
||||
|
||||
// IsValidDataScope reports whether s is one of the five values Permission
|
||||
// recognizes. See go-admin-core's sdk/contract/actions.IsValidDataScope.
|
||||
func IsValidDataScope(s string) bool {
|
||||
return contractactions.IsValidDataScope(s)
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
package actions
|
||||
package actions_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
@@ -6,76 +6,101 @@ import (
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
|
||||
"go-admin/common/actions"
|
||||
)
|
||||
|
||||
// No database is placed in the context on purpose. The middleware needs one
|
||||
// only to run the sys_user join, so reaching the handler proves it did not.
|
||||
func runPermission(t *testing.T, claims jwt.MapClaims) (*DataPermission, bool) {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
// The detailed data-permission regression suite (claims parsing, the
|
||||
// GetOrm-unavailable abort, the SQL each data scope produces) now lives in
|
||||
// go-admin-core's sdk/contract/actions, alongside the logic itself (PRD 006
|
||||
// F3). What is left to test here is the shim's own wiring: that this
|
||||
// package's exported names still round-trip through the same *gin.Context
|
||||
// key core's PermissionAction and GetPermissionFromContext use.
|
||||
//
|
||||
// This file lives in package actions_test, an external test, deliberately:
|
||||
// it exercises PermissionAction and GetPermissionFromContext exactly as an
|
||||
// app/admin Service does, through this package's public API only, not
|
||||
// through anything internal a forward could paper over.
|
||||
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
if claims != nil {
|
||||
c.Set(jwt.JwtPayloadKey, claims)
|
||||
}
|
||||
|
||||
PermissionAction()(c)
|
||||
|
||||
value, exists := c.Get(PermissionKey)
|
||||
if !exists {
|
||||
return nil, false
|
||||
}
|
||||
p, _ := value.(*DataPermission)
|
||||
return p, true
|
||||
}
|
||||
|
||||
// Permission() returns the query untouched when data permission is off, so the
|
||||
// lookup feeding it has nothing to feed. It used to run regardless: a sys_user
|
||||
// join on every list, detail, update and delete, discarded immediately.
|
||||
func TestNoLookupWhenDataPermissionIsOff(t *testing.T) {
|
||||
previous := config.ApplicationConfig.EnableDP
|
||||
config.ApplicationConfig.EnableDP = false
|
||||
t.Cleanup(func() { config.ApplicationConfig.EnableDP = previous })
|
||||
|
||||
if _, ok := runPermission(t, jwt.MapClaims{"identity": float64(7)}); !ok {
|
||||
t.Fatal("the request needed a database even though data permission is off")
|
||||
}
|
||||
}
|
||||
|
||||
func TestScopeComesFromTheTokenWhenItCarriesOne(t *testing.T) {
|
||||
// TestPermissionKeyMatchesWhatPermissionActionSets guards PRD 006's hard
|
||||
// constraint 4: PermissionKey must be declared as
|
||||
// `const PermissionKey = contractactions.PermissionKey`, a direct
|
||||
// reference, never a restated literal (see type.go). PermissionAction is
|
||||
// core's middleware and always writes under core's own key. This test reads
|
||||
// the value back with actions.PermissionKey exactly as code outside
|
||||
// GetPermissionFromContext would - c.Get(actions.PermissionKey) is a real,
|
||||
// if uncommon, way to read the value go-admin has always allowed, and it is
|
||||
// the one call site where an independently declared PermissionKey would
|
||||
// stop working without GetPermissionFromContext's own forward hiding it.
|
||||
//
|
||||
// If PermissionKey were ever re-declared as an independent literal in this
|
||||
// package, a later edit to core's copy would make this test fail without a
|
||||
// single byte of this package having changed - which is the silent-failure
|
||||
// mode hard constraint 4 exists to rule out (evaluation S2).
|
||||
func TestPermissionKeyMatchesWhatPermissionActionSets(t *testing.T) {
|
||||
previous := config.ApplicationConfig.EnableDP
|
||||
config.ApplicationConfig.EnableDP = true
|
||||
t.Cleanup(func() { config.ApplicationConfig.EnableDP = previous })
|
||||
|
||||
p, ok := runPermission(t, jwt.MapClaims{
|
||||
gin.SetMode(gin.TestMode)
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
c.Set(jwt.JwtPayloadKey, jwt.MapClaims{
|
||||
"identity": float64(7),
|
||||
"roleid": float64(3),
|
||||
"deptid": float64(5),
|
||||
"datascope": "4",
|
||||
"datascope": actions.DataScopeDeptTree,
|
||||
})
|
||||
|
||||
actions.PermissionAction()(c)
|
||||
|
||||
value, ok := c.Get(actions.PermissionKey)
|
||||
if !ok {
|
||||
t.Fatal("the token carried the scope and a database was still needed")
|
||||
t.Fatal("PermissionAction did not set the key actions.PermissionKey names; the two have diverged")
|
||||
}
|
||||
if p.DataScope != "4" || p.UserId != 7 || p.DeptId != 5 || p.RoleId != 3 {
|
||||
t.Fatalf("scope read as %+v", p)
|
||||
p, ok := value.(*actions.DataPermission)
|
||||
if !ok || p.DataScope != actions.DataScopeDeptTree || p.DeptId != 5 {
|
||||
t.Fatalf("value under actions.PermissionKey = %#v, want a DataPermission carrying the token's scope", value)
|
||||
}
|
||||
}
|
||||
|
||||
// A token minted before deptid was carried is still valid until it expires, and
|
||||
// has to keep working - by falling back to the query, which needs a database.
|
||||
func TestATokenWithoutDeptIdFallsBackToTheQuery(t *testing.T) {
|
||||
// TestGetPermissionFromContextRoundTrips is the same guard from the other
|
||||
// exported entry point: GetPermissionFromContext must read back exactly
|
||||
// what PermissionAction wrote, both reached through this package's own
|
||||
// forwards rather than core's directly.
|
||||
func TestGetPermissionFromContextRoundTrips(t *testing.T) {
|
||||
previous := config.ApplicationConfig.EnableDP
|
||||
config.ApplicationConfig.EnableDP = true
|
||||
t.Cleanup(func() { config.ApplicationConfig.EnableDP = previous })
|
||||
|
||||
if _, ok := runPermission(t, jwt.MapClaims{
|
||||
gin.SetMode(gin.TestMode)
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
c.Set(jwt.JwtPayloadKey, jwt.MapClaims{
|
||||
"identity": float64(7),
|
||||
"roleid": float64(3),
|
||||
"datascope": "4",
|
||||
}); ok {
|
||||
t.Fatal("an old token was served from claims it does not have")
|
||||
"deptid": float64(5),
|
||||
"datascope": actions.DataScopeSelf,
|
||||
})
|
||||
|
||||
actions.PermissionAction()(c)
|
||||
|
||||
p := actions.GetPermissionFromContext(c)
|
||||
if p.DataScope != actions.DataScopeSelf || p.UserId != 7 {
|
||||
t.Fatalf("GetPermissionFromContext() = %+v, want DataScope=%q UserId=7", p, actions.DataScopeSelf)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsValidDataScope(t *testing.T) {
|
||||
for _, s := range []string{actions.DataScopeAll, actions.DataScopeCustom, actions.DataScopeDept, actions.DataScopeDeptTree, actions.DataScopeSelf} {
|
||||
if !actions.IsValidDataScope(s) {
|
||||
t.Errorf("IsValidDataScope(%q) = false, want true", s)
|
||||
}
|
||||
}
|
||||
if actions.IsValidDataScope("6") {
|
||||
t.Error(`IsValidDataScope("6") = true, want false`)
|
||||
}
|
||||
}
|
||||
|
||||
+11
-3
@@ -1,5 +1,13 @@
|
||||
package actions
|
||||
|
||||
const (
|
||||
PermissionKey = "dataPermission"
|
||||
)
|
||||
import contractactions "github.com/go-admin-team/go-admin-core/v2/sdk/contract/actions"
|
||||
|
||||
// PermissionKey is a direct reference to go-admin-core's sdk/contract/actions
|
||||
// constant, not a restated literal - see that package's PermissionKey doc
|
||||
// comment. PRD 006's hard constraint 4 requires this form for exactly this
|
||||
// symbol: PermissionAction (below) sets the gin context key it owns, and
|
||||
// GetPermissionFromContext reads it back; an independently declared literal
|
||||
// here would let the two silently drift apart if core's copy ever changed
|
||||
// without this one following. common/actions/shim_test.go carries the
|
||||
// regression test for that failure mode.
|
||||
const PermissionKey = contractactions.PermissionKey
|
||||
|
||||
@@ -63,7 +63,11 @@ func setupSimpleDatabase(host string, c *toolsConfig.Database) {
|
||||
log.Info(pkg.Green(c.Driver + " connect success !"))
|
||||
}
|
||||
|
||||
e := mycasbin.Setup(db, "")
|
||||
// Keyed by host, matching the database this enforcer reads from. Passing
|
||||
// the same key for every host would hand each one the enforcer built from
|
||||
// whichever database was configured first, and the rest would be decided
|
||||
// by a casbin_rule table that is not theirs.
|
||||
e := mycasbin.Setup(db, host)
|
||||
|
||||
sdk.Runtime.SetDbByTenant(host, db)
|
||||
sdk.Runtime.SetCasbinByTenant(host, e)
|
||||
|
||||
+12
-71
@@ -1,74 +1,15 @@
|
||||
package dto
|
||||
|
||||
type AutoForm struct {
|
||||
Fields []Field `json:"fields"`
|
||||
FormRef string `json:"formRef"`
|
||||
FormModel string `json:"formModel"`
|
||||
Size string `json:"size"`
|
||||
LabelPosition string `json:"labelPosition"`
|
||||
LabelWidth int `json:"labelWidth"`
|
||||
FormRules string `json:"formRules"`
|
||||
Gutter int `json:"gutter"`
|
||||
Disabled bool `json:"disabled"`
|
||||
Span int `json:"span"`
|
||||
FormBtns bool `json:"formBtns"`
|
||||
}
|
||||
import contractdto "github.com/go-admin-team/go-admin-core/v2/sdk/contract/dto"
|
||||
|
||||
type Config struct {
|
||||
Label string `json:"label"`
|
||||
LabelWidth interface{} `json:"labelWidth"`
|
||||
ShowLabel bool `json:"showLabel"`
|
||||
ChangeTag bool `json:"changeTag"`
|
||||
Tag string `json:"tag"`
|
||||
TagIcon string `json:"tagIcon"`
|
||||
Required bool `json:"required"`
|
||||
Layout string `json:"layout"`
|
||||
Span int `json:"span"`
|
||||
Document string `json:"document"`
|
||||
RegList []interface{} `json:"regList"`
|
||||
FormId int `json:"formId"`
|
||||
RenderKey int64 `json:"renderKey"`
|
||||
DefaultValue interface{} `json:"defaultValue"`
|
||||
ShowTip bool `json:"showTip,omitempty"`
|
||||
ButtonText string `json:"buttonText,omitempty"`
|
||||
FileSize int `json:"fileSize,omitempty"`
|
||||
SizeUnit string `json:"sizeUnit,omitempty"`
|
||||
}
|
||||
|
||||
type Option struct {
|
||||
Label string `json:"label"`
|
||||
Value string `json:"value"`
|
||||
}
|
||||
|
||||
type Slot struct {
|
||||
Prepend string `json:"prepend,omitempty"`
|
||||
Append string `json:"append,omitempty"`
|
||||
ListType bool `json:"list-type,omitempty"`
|
||||
Options []Option `json:"options,omitempty"`
|
||||
}
|
||||
|
||||
type Field struct {
|
||||
Config Config `json:"__config__"`
|
||||
Slot Slot `json:"__slot__"`
|
||||
Placeholder string `json:"placeholder,omitempty"`
|
||||
Style Style `json:"style,omitempty"`
|
||||
Clearable bool `json:"clearable,omitempty"`
|
||||
PrefixIcon string `json:"prefix-icon,omitempty"`
|
||||
SuffixIcon string `json:"suffix-icon,omitempty"`
|
||||
Maxlength interface{} `json:"maxlength"`
|
||||
ShowWordLimit bool `json:"show-word-limit,omitempty"`
|
||||
Readonly bool `json:"readonly,omitempty"`
|
||||
Disabled bool `json:"disabled"`
|
||||
VModel string `json:"__vModel__"`
|
||||
Action string `json:"action,omitempty"`
|
||||
Accept string `json:"accept,omitempty"`
|
||||
Name string `json:"name,omitempty"`
|
||||
AutoUpload bool `json:"auto-upload,omitempty"`
|
||||
ListType string `json:"list-type,omitempty"`
|
||||
Multiple bool `json:"multiple,omitempty"`
|
||||
Filterable bool `json:"filterable,omitempty"`
|
||||
}
|
||||
|
||||
type Style struct {
|
||||
Width string `json:"width"`
|
||||
}
|
||||
// AutoForm and the types below describe a form built by go-admin-ui's form
|
||||
// designer. They are thin aliases of go-admin-core's sdk/contract/dto (PRD
|
||||
// 006 F2/F5).
|
||||
type (
|
||||
AutoForm = contractdto.AutoForm
|
||||
Config = contractdto.Config
|
||||
Option = contractdto.Option
|
||||
Slot = contractdto.Slot
|
||||
Field = contractdto.Field
|
||||
Style = contractdto.Style
|
||||
)
|
||||
|
||||
+7
-102
@@ -1,106 +1,11 @@
|
||||
package dto
|
||||
|
||||
import (
|
||||
vd "github.com/bytedance/go-tagexpr/v2/validator"
|
||||
"net/http"
|
||||
import contractdto "github.com/go-admin-team/go-admin-core/v2/sdk/contract/dto"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/api"
|
||||
// ObjectById, ObjectGetReq and ObjectDeleteReq are thin aliases of
|
||||
// go-admin-core's sdk/contract/dto (PRD 006 F2/F5).
|
||||
type (
|
||||
ObjectById = contractdto.ObjectById
|
||||
ObjectGetReq = contractdto.ObjectGetReq
|
||||
ObjectDeleteReq = contractdto.ObjectDeleteReq
|
||||
)
|
||||
|
||||
type ObjectById struct {
|
||||
Id int `uri:"id"`
|
||||
Ids []int `json:"ids"`
|
||||
}
|
||||
|
||||
func (s *ObjectById) Bind(ctx *gin.Context) error {
|
||||
var err error
|
||||
log := api.GetRequestLogger(ctx)
|
||||
err = ctx.ShouldBindUri(s)
|
||||
if err != nil {
|
||||
log.Warnf("ShouldBindUri error: %s", err.Error())
|
||||
return err
|
||||
}
|
||||
if ctx.Request.Method == http.MethodDelete {
|
||||
err = ctx.ShouldBind(&s)
|
||||
if err != nil {
|
||||
log.Warnf("ShouldBind error: %s", err.Error())
|
||||
return err
|
||||
}
|
||||
if len(s.Ids) > 0 {
|
||||
return nil
|
||||
}
|
||||
if s.Ids == nil {
|
||||
s.Ids = make([]int, 0)
|
||||
}
|
||||
if s.Id != 0 {
|
||||
s.Ids = append(s.Ids, s.Id)
|
||||
}
|
||||
}
|
||||
if err = vd.Validate(s); err != nil {
|
||||
log.Errorf("Validate error: %s", err.Error())
|
||||
return err
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *ObjectById) GetId() interface{} {
|
||||
if len(s.Ids) > 0 {
|
||||
s.Ids = append(s.Ids, s.Id)
|
||||
return s.Ids
|
||||
}
|
||||
return s.Id
|
||||
}
|
||||
|
||||
type ObjectGetReq struct {
|
||||
Id int `uri:"id"`
|
||||
}
|
||||
|
||||
func (s *ObjectGetReq) Bind(ctx *gin.Context) error {
|
||||
var err error
|
||||
log := api.GetRequestLogger(ctx)
|
||||
err = ctx.ShouldBindUri(s)
|
||||
if err != nil {
|
||||
log.Warnf("ShouldBindUri error: %s", err.Error())
|
||||
return err
|
||||
}
|
||||
if err = vd.Validate(s); err != nil {
|
||||
log.Errorf("Validate error: %s", err.Error())
|
||||
return err
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *ObjectGetReq) GetId() interface{} {
|
||||
return s.Id
|
||||
}
|
||||
|
||||
type ObjectDeleteReq struct {
|
||||
Ids []int `json:"ids"`
|
||||
}
|
||||
|
||||
func (s *ObjectDeleteReq) Bind(ctx *gin.Context) error {
|
||||
var err error
|
||||
log := api.GetRequestLogger(ctx)
|
||||
err = ctx.ShouldBind(&s)
|
||||
if err != nil {
|
||||
log.Warnf("ShouldBind error: %s", err.Error())
|
||||
return err
|
||||
}
|
||||
if len(s.Ids) > 0 {
|
||||
return nil
|
||||
}
|
||||
if s.Ids == nil {
|
||||
s.Ids = make([]int, 0)
|
||||
}
|
||||
|
||||
if err = vd.Validate(s); err != nil {
|
||||
log.Errorf("Validate error: %s", err.Error())
|
||||
return err
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *ObjectDeleteReq) GetId() interface{} {
|
||||
return s.Ids
|
||||
}
|
||||
|
||||
+6
-4
@@ -2,11 +2,13 @@ package dto
|
||||
|
||||
import (
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
|
||||
contractdto "github.com/go-admin-team/go-admin-core/v2/sdk/contract/dto"
|
||||
)
|
||||
|
||||
// OrderDest forwards to go-admin-core's sdk/contract/dto (PRD 006 F2/F5). A
|
||||
// function cannot be aliased the way a type can, so this is a pure
|
||||
// pass-through rather than a `func X = pkg.X` form Go does not have.
|
||||
func OrderDest(sort string, bl bool) func(db *gorm.DB) *gorm.DB {
|
||||
return func(db *gorm.DB) *gorm.DB {
|
||||
return db.Order(clause.OrderByColumn{Column: clause.Column{Name: sort}, Desc: bl})
|
||||
}
|
||||
return contractdto.OrderDest(sort, bl)
|
||||
}
|
||||
|
||||
@@ -1,20 +1,7 @@
|
||||
package dto
|
||||
|
||||
type Pagination struct {
|
||||
PageIndex int `form:"pageIndex"`
|
||||
PageSize int `form:"pageSize"`
|
||||
}
|
||||
import contractdto "github.com/go-admin-team/go-admin-core/v2/sdk/contract/dto"
|
||||
|
||||
func (m *Pagination) GetPageIndex() int {
|
||||
if m.PageIndex <= 0 {
|
||||
m.PageIndex = 1
|
||||
}
|
||||
return m.PageIndex
|
||||
}
|
||||
|
||||
func (m *Pagination) GetPageSize() int {
|
||||
if m.PageSize <= 0 {
|
||||
m.PageSize = 10
|
||||
}
|
||||
return m.PageSize
|
||||
}
|
||||
// Pagination is a thin alias of go-admin-core's sdk/contract/dto (PRD 006
|
||||
// F2/F5).
|
||||
type Pagination = contractdto.Pagination
|
||||
|
||||
+19
-68
@@ -1,80 +1,31 @@
|
||||
package dto
|
||||
|
||||
import (
|
||||
"github.com/go-admin-team/go-admin-core/v2/tools/search"
|
||||
"go-admin/common/global"
|
||||
"gorm.io/gorm"
|
||||
|
||||
contractdto "github.com/go-admin-team/go-admin-core/v2/sdk/contract/dto"
|
||||
)
|
||||
|
||||
type GeneralDelDto struct {
|
||||
Id int `uri:"id" json:"id" validate:"required"`
|
||||
Ids []int `json:"ids"`
|
||||
}
|
||||
|
||||
func (g GeneralDelDto) GetIds() []int {
|
||||
ids := make([]int, 0)
|
||||
// Id 此前在 else 分支里被重复追加:仅传 Id 时会得到 [5 5],
|
||||
// 同一条记录被执行两次删除
|
||||
if g.Id > 0 {
|
||||
ids = append(ids, g.Id)
|
||||
}
|
||||
for _, id := range g.Ids {
|
||||
if id > 0 {
|
||||
ids = append(ids, id)
|
||||
}
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
//方式全部删除
|
||||
ids = append(ids, 0)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
type GeneralGetDto struct {
|
||||
Id int `uri:"id" json:"id" validate:"required"`
|
||||
}
|
||||
// GeneralDelDto and GeneralGetDto are thin aliases of go-admin-core's
|
||||
// sdk/contract/dto (PRD 006 F2/F5).
|
||||
type (
|
||||
GeneralDelDto = contractdto.GeneralDelDto
|
||||
GeneralGetDto = contractdto.GeneralGetDto
|
||||
)
|
||||
|
||||
// MakeCondition and Paginate forward to go-admin-core's sdk/contract/dto
|
||||
// (PRD 006 F2/F5). This file used to read go-admin/common/global.Driver to
|
||||
// pick the SQL dialect MakeCondition resolves search tags against; the
|
||||
// lowered version instead reads db.Dialector.Name() from inside the closure
|
||||
// it returns, which is always the driver the caller's own *gorm.DB is bound
|
||||
// to - correct even when a multi-tenant host has more than one database
|
||||
// open with different drivers, which a single package-level variable could
|
||||
// never be. global.Driver itself is untouched and still readable, but
|
||||
// nothing in this package reads it anymore.
|
||||
func MakeCondition(q interface{}) func(db *gorm.DB) *gorm.DB {
|
||||
return func(db *gorm.DB) *gorm.DB {
|
||||
condition := &search.GormCondition{
|
||||
GormPublic: search.GormPublic{},
|
||||
Join: make([]*search.GormJoin, 0),
|
||||
}
|
||||
search.ResolveSearchQuery(global.Driver, q, condition)
|
||||
for _, join := range condition.Join {
|
||||
if join == nil {
|
||||
continue
|
||||
}
|
||||
db = db.Joins(join.JoinOn)
|
||||
for k, v := range join.Where {
|
||||
db = db.Where(k, v...)
|
||||
}
|
||||
for k, v := range join.Or {
|
||||
db = db.Or(k, v...)
|
||||
}
|
||||
for _, o := range join.Order {
|
||||
db = db.Order(o)
|
||||
}
|
||||
}
|
||||
for k, v := range condition.Where {
|
||||
db = db.Where(k, v...)
|
||||
}
|
||||
for k, v := range condition.Or {
|
||||
db = db.Or(k, v...)
|
||||
}
|
||||
for _, o := range condition.Order {
|
||||
db = db.Order(o)
|
||||
}
|
||||
return db
|
||||
}
|
||||
return contractdto.MakeCondition(q)
|
||||
}
|
||||
|
||||
func Paginate(pageSize, pageIndex int) func(db *gorm.DB) *gorm.DB {
|
||||
return func(db *gorm.DB) *gorm.DB {
|
||||
offset := (pageIndex - 1) * pageSize
|
||||
if offset < 0 {
|
||||
offset = 0
|
||||
}
|
||||
return db.Offset(offset).Limit(pageSize)
|
||||
}
|
||||
return contractdto.Paginate(pageSize, pageIndex)
|
||||
}
|
||||
|
||||
+7
-18
@@ -1,21 +1,10 @@
|
||||
package dto
|
||||
|
||||
import (
|
||||
"github.com/gin-gonic/gin"
|
||||
"go-admin/common/models"
|
||||
import contractdto "github.com/go-admin-team/go-admin-core/v2/sdk/contract/dto"
|
||||
|
||||
// Index and Control are thin aliases of go-admin-core's sdk/contract/dto
|
||||
// (PRD 006 F2/F5).
|
||||
type (
|
||||
Index = contractdto.Index
|
||||
Control = contractdto.Control
|
||||
)
|
||||
|
||||
type Index interface {
|
||||
Generate() Index
|
||||
Bind(ctx *gin.Context) error
|
||||
GetPageIndex() int
|
||||
GetPageSize() int
|
||||
GetNeedSearch() interface{}
|
||||
}
|
||||
|
||||
type Control interface {
|
||||
Generate() Control
|
||||
Bind(ctx *gin.Context) error
|
||||
GenerateM() (models.ActiveRecord, error)
|
||||
GetId() interface{}
|
||||
}
|
||||
|
||||
@@ -7,5 +7,12 @@ const (
|
||||
|
||||
var (
|
||||
// Driver 数据库驱动
|
||||
//
|
||||
// Deprecated: common/dto.MakeCondition stopped reading this after PRD
|
||||
// 006 F2/F5 - it now takes the dialect from the *gorm.DB passed to the
|
||||
// scope it returns instead of this process-wide variable. Driver is
|
||||
// still set (common/database/initialize.go) and still readable for fork
|
||||
// code that reads it directly, but it is no longer this framework's own
|
||||
// path to the current SQL dialect.
|
||||
Driver string
|
||||
)
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
package global
|
||||
|
||||
// Status values written to sys_opera_log.status.
|
||||
//
|
||||
// They live here rather than in app/admin/service/dto because
|
||||
// common/middleware/logger.go writes the operation-log message and needs them.
|
||||
// A package promised as a stable contract must not compile-depend on a
|
||||
// business module: a fork that replaces or drops app/admin would otherwise
|
||||
// stop compiling common/middleware, which is not something a contract package
|
||||
// is allowed to do. See docs/contract.md.
|
||||
const (
|
||||
OperaStatusEnabled = "1"
|
||||
OperaStatusDisabled = "2"
|
||||
)
|
||||
@@ -0,0 +1,177 @@
|
||||
// Package health answers whether this process should be sent traffic.
|
||||
//
|
||||
// The two questions an orchestrator asks are not the same one, and go-admin
|
||||
// answers them at two endpoints:
|
||||
//
|
||||
// - /health is liveness: is the process there at all. It stays a bare 200,
|
||||
// because the honest answer to "should I restart you" is almost always no.
|
||||
// Restarting a process because its database is unreachable turns one
|
||||
// outage into a crash loop that also loses the connection pool, the cache
|
||||
// and every in-flight request.
|
||||
// - /ready is readiness: should this instance receive requests now. It fails
|
||||
// while the dependencies are unreachable, and - the part that only exists
|
||||
// because of the life-cycle phases - it fails as soon as shutdown begins,
|
||||
// before the server stops accepting.
|
||||
//
|
||||
// # What the draining answer is worth
|
||||
//
|
||||
// Order alone does not produce a window. Answering before the server stops
|
||||
// accepting is the right order - the reverse reports the state after the
|
||||
// connections are already cut - but with nothing between the two they are
|
||||
// microseconds apart, and a poller on a multi-second interval never sees the
|
||||
// 503.
|
||||
//
|
||||
// The delay between them is extend.shutdown.drain, which is zero unless it is
|
||||
// configured. On the shipped defaults this is therefore still an answer that
|
||||
// can be read rather than one anything acts on; a deployment that sets a drain
|
||||
// window is the one that gets a window to act in.
|
||||
//
|
||||
// What acts on it depends on who does the removing. A load balancer that polls
|
||||
// /ready takes this instance out when it reads the 503, and the window has to
|
||||
// cover its check interval times its failure threshold, plus however long the
|
||||
// removal takes to apply. On Kubernetes the endpoint is withdrawn when the Pod
|
||||
// receives a deletionTimestamp, concurrently with SIGTERM and regardless of
|
||||
// what the probe returns - there the window covers the delay in that removal
|
||||
// reaching every node, and the 503 is what makes the state observable.
|
||||
package health
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
)
|
||||
|
||||
// draining is set when the process starts shutting down.
|
||||
//
|
||||
// It is kept here rather than read back from core: BeginShutdown sets a flag on
|
||||
// the Application, but nothing exports it, and one host wanting to know is not
|
||||
// yet a reason to widen that interface.
|
||||
var draining atomic.Bool
|
||||
|
||||
// BeginDraining records that shutdown has started, so readiness fails from now
|
||||
// on. It is called with BeginShutdown, before anything is taken apart.
|
||||
func BeginDraining() { draining.Store(true) }
|
||||
|
||||
// Draining reports whether shutdown has begun.
|
||||
func Draining() bool { return draining.Load() }
|
||||
|
||||
// Check is one dependency and what asking it produced.
|
||||
type Check struct {
|
||||
Name string `json:"name"`
|
||||
OK bool `json:"ok"`
|
||||
Err string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// Ready asks every dependency this process cannot serve a request without.
|
||||
//
|
||||
// The queue is deliberately absent. Nothing on AdapterQueue answers "are you
|
||||
// reachable" without publishing something, the memory backend cannot fail, and
|
||||
// a queue that is down degrades logging rather than stopping requests - which
|
||||
// is a reason to alert, not a reason to leave the load balancer pool.
|
||||
func Ready(ctx context.Context) []Check {
|
||||
return []Check{
|
||||
safely("database", func() error { return pingDB(ctx) }),
|
||||
safely("cache", probeCache),
|
||||
}
|
||||
}
|
||||
|
||||
// safely turns a panic into a failed check.
|
||||
//
|
||||
// Not defensive habit: the accessors hand back wrappers, not the resources.
|
||||
// sdk.Runtime.GetCacheAdapter builds a runtime.Cache around whatever is
|
||||
// configured and returns it even when nothing is - so the value is not nil, the
|
||||
// cache inside it is, and the first call dereferences it. A nil check cannot
|
||||
// see that, and the same is true of GetQueueAdapter.
|
||||
//
|
||||
// Whatever the reason, a probe is the last thing that should be able to take
|
||||
// the process down: the caller is asking whether this instance is well, and
|
||||
// killing it to answer is the wrong reply.
|
||||
func safely(name string, fn func() error) (c Check) {
|
||||
c = Check{Name: name}
|
||||
defer func() {
|
||||
if r := recover(); r != nil {
|
||||
c.OK, c.Err = false, fmt.Sprintf("the check panicked: %v", r)
|
||||
}
|
||||
}()
|
||||
if err := fn(); err != nil {
|
||||
c.Err = err.Error()
|
||||
return c
|
||||
}
|
||||
c.OK = true
|
||||
return c
|
||||
}
|
||||
|
||||
// Healthy reports whether every check passed.
|
||||
func Healthy(checks []Check) bool {
|
||||
for _, c := range checks {
|
||||
if !c.OK {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func pingDB(ctx context.Context) error {
|
||||
db := sdk.Runtime.GetDb()
|
||||
if db == nil {
|
||||
return errors.New("no database configured")
|
||||
}
|
||||
sqlDB, err := db.DB()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return sqlDB.PingContext(ctx)
|
||||
}
|
||||
|
||||
// cacheProbePrefix names the probe's keys. The key itself is per probe, not
|
||||
// fixed: two /ready requests arriving together - or two instances sharing one
|
||||
// redis, which is the normal deployment - would otherwise overwrite each
|
||||
// other's value between the write and the read and each conclude the cache was
|
||||
// broken. A readiness probe that reports false negatives under load takes
|
||||
// healthy instances out of the pool, which is worse than not probing.
|
||||
const cacheProbePrefix = "go-admin:health:"
|
||||
|
||||
// cacheProbeTTL is short because these keys are write-once and never read
|
||||
// again by anyone else; it only has to outlive the read that follows.
|
||||
const cacheProbeTTL = 30
|
||||
|
||||
func probeCache() error {
|
||||
adapter := sdk.Runtime.GetCacheAdapter()
|
||||
if adapter == nil {
|
||||
return errors.New("no cache configured")
|
||||
}
|
||||
|
||||
suffix := make([]byte, 8)
|
||||
if _, err := rand.Read(suffix); err != nil {
|
||||
return fmt.Errorf("could not build a probe key: %w", err)
|
||||
}
|
||||
key := cacheProbePrefix + hex.EncodeToString(suffix)
|
||||
|
||||
// Written and read back rather than only read: a cache that answers "miss"
|
||||
// for every key - a client pointed at the wrong server - is
|
||||
// indistinguishable from a healthy one on a read alone.
|
||||
want := time.Now().Format(time.RFC3339Nano)
|
||||
if err := adapter.Set(key, want, cacheProbeTTL); err != nil {
|
||||
return err
|
||||
}
|
||||
// Best effort, and its error is deliberately dropped: the verdict is
|
||||
// already decided by the read below, and a cache that cannot delete a key
|
||||
// it just wrote is not a reason to refuse traffic. The TTL is the real
|
||||
// cleanup.
|
||||
defer func() { _ = adapter.Del(key) }()
|
||||
|
||||
got, err := adapter.Get(key)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if got != want {
|
||||
return errors.New("the cache returned a different value than was written")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,244 @@
|
||||
package health
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
|
||||
)
|
||||
|
||||
func freshRuntime(t *testing.T) {
|
||||
t.Helper()
|
||||
previous := sdk.Runtime
|
||||
t.Cleanup(func() { sdk.Runtime = previous })
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
}
|
||||
|
||||
// fakeCache answers whatever the test needs it to.
|
||||
type fakeCache struct {
|
||||
mu sync.Mutex
|
||||
setErr error
|
||||
getErr error
|
||||
getBack string // returned instead of what was written, when non-empty
|
||||
stored map[string]string
|
||||
|
||||
// oneSlot makes the cache keep a single value however many keys are
|
||||
// written, which is what a shared probe key turns any cache into.
|
||||
oneSlot bool
|
||||
slot string
|
||||
|
||||
// setBarrier, when set, holds every writer until all of them have written.
|
||||
// Without it the probes are short enough that the scheduler usually runs
|
||||
// them one after another, and a shared key survives by luck rather than by
|
||||
// design - which would leave the test below asserting nothing.
|
||||
setBarrier *barrier
|
||||
}
|
||||
|
||||
// barrier releases every waiter once n of them have arrived.
|
||||
type barrier struct {
|
||||
n int
|
||||
mu sync.Mutex
|
||||
got int
|
||||
ch chan struct{}
|
||||
}
|
||||
|
||||
func newBarrier(n int) *barrier { return &barrier{n: n, ch: make(chan struct{})} }
|
||||
|
||||
func (b *barrier) wait() {
|
||||
b.mu.Lock()
|
||||
b.got++
|
||||
if b.got == b.n {
|
||||
close(b.ch)
|
||||
}
|
||||
b.mu.Unlock()
|
||||
<-b.ch
|
||||
}
|
||||
|
||||
func (c *fakeCache) String() string { return "fake" }
|
||||
|
||||
func (c *fakeCache) Set(key string, val interface{}, _ int) error {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.setErr != nil {
|
||||
return c.setErr
|
||||
}
|
||||
v, _ := val.(string)
|
||||
if c.oneSlot {
|
||||
c.slot = v
|
||||
return nil
|
||||
}
|
||||
if c.stored == nil {
|
||||
c.stored = map[string]string{}
|
||||
}
|
||||
c.stored[key] = v
|
||||
c.mu.Unlock()
|
||||
if c.setBarrier != nil {
|
||||
// Outside the lock on purpose: waiting while holding it would deadlock
|
||||
// every other writer before the barrier could fill.
|
||||
c.setBarrier.wait()
|
||||
}
|
||||
c.mu.Lock()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *fakeCache) Get(key string) (string, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.getErr != nil {
|
||||
return "", c.getErr
|
||||
}
|
||||
if c.getBack != "" {
|
||||
return c.getBack, nil
|
||||
}
|
||||
if c.oneSlot {
|
||||
return c.slot, nil
|
||||
}
|
||||
return c.stored[key], nil
|
||||
}
|
||||
|
||||
func (c *fakeCache) Del(key string) error {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
delete(c.stored, key)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *fakeCache) HashGet(_, _ string) (string, error) { return "", nil }
|
||||
func (c *fakeCache) HashDel(_, _ string) error { return nil }
|
||||
func (c *fakeCache) Increase(string) error { return nil }
|
||||
func (c *fakeCache) Decrease(string) error { return nil }
|
||||
func (c *fakeCache) Expire(string, time.Duration) error { return nil }
|
||||
|
||||
var _ corestorage.AdapterCache = (*fakeCache)(nil)
|
||||
|
||||
func named(checks []Check, name string) Check {
|
||||
for _, c := range checks {
|
||||
if c.Name == name {
|
||||
return c
|
||||
}
|
||||
}
|
||||
return Check{Name: name, Err: "check not reported at all"}
|
||||
}
|
||||
|
||||
// A cache that accepts writes and answers every read with a different value is
|
||||
// the failure this probe exists for - a client pointed at the wrong server, or
|
||||
// one that silently drops everything. A read alone cannot tell that apart from
|
||||
// a healthy cache with a cold key, which is why the probe writes first.
|
||||
func TestCacheProbeFailsWhenTheValueDoesNotComeBack(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
sdk.Runtime.SetCacheAdapter(&fakeCache{getBack: "something else"})
|
||||
|
||||
got := named(Ready(context.Background()), "cache")
|
||||
if got.OK {
|
||||
t.Error("the cache check passed although the value written was not the value read back")
|
||||
}
|
||||
if got.Err == "" {
|
||||
t.Error("the failing check reported no reason")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCacheProbePassesWhenTheValueComesBack(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
sdk.Runtime.SetCacheAdapter(&fakeCache{})
|
||||
|
||||
if got := named(Ready(context.Background()), "cache"); !got.OK {
|
||||
t.Errorf("the cache check failed for a cache that works: %s", got.Err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCacheProbeReportsAWriteFailure(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
sdk.Runtime.SetCacheAdapter(&fakeCache{setErr: errors.New("connection refused")})
|
||||
|
||||
got := named(Ready(context.Background()), "cache")
|
||||
if got.OK {
|
||||
t.Error("the cache check passed although the write failed")
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing configured is the case that used to take the process down rather
|
||||
// than answer. GetCacheAdapter builds a wrapper around whatever is configured
|
||||
// and returns it even when nothing is, so the value is not nil, the cache
|
||||
// inside it is, and Set dereferences it - a probe that panics is the worst
|
||||
// possible answer to "are you well".
|
||||
//
|
||||
// Every check has to be reported, passing or not. A probe that omits what it
|
||||
// could not reach reads as a shorter list of healthy things.
|
||||
func TestEveryDependencyIsReportedEvenWithNothingConfigured(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
checks := Ready(context.Background())
|
||||
for _, name := range []string{"database", "cache"} {
|
||||
c := named(checks, name)
|
||||
if c.Err == "check not reported at all" {
|
||||
t.Errorf("%s was not reported", name)
|
||||
}
|
||||
if c.OK {
|
||||
t.Errorf("%s passed with nothing configured", name)
|
||||
}
|
||||
}
|
||||
if Healthy(checks) {
|
||||
t.Error("Healthy said yes for a process with no database and no cache")
|
||||
}
|
||||
}
|
||||
|
||||
// BeginDraining sets the flag and Draining reports it, before anything else is
|
||||
// taken apart. That is the whole of what can be checked from inside the
|
||||
// process: whether anyone outside gets to read it depends on
|
||||
// extend.shutdown.drain, which is zero unless it is configured, and on who is
|
||||
// routing traffic here - the package comment has both. The subprocess tests in
|
||||
// cmd/api are where a reader on the other end of a socket sees the 503.
|
||||
func TestDrainingIsObservableOnceItBegins(t *testing.T) {
|
||||
previous := draining.Load()
|
||||
t.Cleanup(func() { draining.Store(previous) })
|
||||
|
||||
draining.Store(false)
|
||||
if Draining() {
|
||||
t.Fatal("Draining reported true before shutdown began")
|
||||
}
|
||||
BeginDraining()
|
||||
if !Draining() {
|
||||
t.Error("Draining still reported false after BeginDraining")
|
||||
}
|
||||
}
|
||||
|
||||
// Two probes at once must both pass. With one fixed key they overwrite each
|
||||
// other's value between the write and the read, and a readiness probe that
|
||||
// reports false negatives under load takes healthy instances out of the pool -
|
||||
// which is worse than not probing at all.
|
||||
func TestConcurrentProbesDoNotOverwriteEachOther(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
const probes = 16
|
||||
sdk.Runtime.SetCacheAdapter(&fakeCache{setBarrier: newBarrier(probes)})
|
||||
|
||||
var wg sync.WaitGroup
|
||||
failures := make(chan string, probes)
|
||||
for i := 0; i < probes; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if c := named(Ready(context.Background()), "cache"); !c.OK {
|
||||
failures <- c.Err
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(failures)
|
||||
|
||||
var n int
|
||||
var first string
|
||||
for err := range failures {
|
||||
if n == 0 {
|
||||
first = err
|
||||
}
|
||||
n++
|
||||
}
|
||||
if n > 0 {
|
||||
t.Errorf("%d of %d concurrent probes called a healthy cache broken; first: %s", n, probes, first)
|
||||
}
|
||||
}
|
||||
@@ -3,11 +3,19 @@ package middleware
|
||||
import (
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
log "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"go-admin/common/middleware/handler"
|
||||
)
|
||||
|
||||
// authMiddleware is the single JWT middleware instance the whole process
|
||||
// shares. InitMiddleware builds it once, before any module registers its
|
||||
// routes; GetAuthMiddleware is how a module gets it back instead of calling
|
||||
// AuthInit itself and building another, functionally-equivalent-but-distinct
|
||||
// instance.
|
||||
var authMiddleware *jwt.GinJWTMiddleware
|
||||
|
||||
// AuthInit jwt验证new
|
||||
func AuthInit() (*jwt.GinJWTMiddleware, error) {
|
||||
timeout := time.Hour
|
||||
@@ -33,4 +41,23 @@ func AuthInit() (*jwt.GinJWTMiddleware, error) {
|
||||
TimeFunc: time.Now,
|
||||
})
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
// GetAuthMiddleware returns the shared JWT middleware instance InitMiddleware
|
||||
// built at startup. Application modules (app/admin, app/jobs, app/other,
|
||||
// app/demo) call this instead of AuthInit so their router chains - which
|
||||
// still need the instance itself for authMiddleware.MiddlewareFunc() and
|
||||
// authMiddleware.LoginHandler, not just the bound closure registered under
|
||||
// sdk.Runtime's JwtTokenCheck key - end up using the same instance the host
|
||||
// registered, rather than one each.
|
||||
//
|
||||
// It fails loudly instead of returning nil: an InitRouter that runs before
|
||||
// InitMiddleware has a real startup-ordering bug, not a case to paper over
|
||||
// with a nil *jwt.GinJWTMiddleware that would panic much further down the
|
||||
// call chain with a far less useful stack trace.
|
||||
func GetAuthMiddleware() *jwt.GinJWTMiddleware {
|
||||
if authMiddleware == nil {
|
||||
log.Fatal("JWT middleware not initialized; InitMiddleware must run before any module's InitRouter")
|
||||
}
|
||||
return authMiddleware
|
||||
}
|
||||
|
||||
+91
-17
@@ -1,29 +1,103 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"net/http"
|
||||
)
|
||||
|
||||
// defaultDemoMsg is what a refused request is told when nothing is configured.
|
||||
//
|
||||
// It is the string this middleware used to carry hard-coded, kept verbatim so
|
||||
// that a deployment which never set application.demomsg is answered exactly as
|
||||
// it was before.
|
||||
const defaultDemoMsg = "谢谢您的参与,但为了大家更好的体验,所以本次提交就算了吧!\U0001F600\U0001F600\U0001F600"
|
||||
|
||||
// demoWriteRoutes are routes that change something despite being registered as
|
||||
// GET, so the method alone does not say whether they are safe to serve.
|
||||
//
|
||||
// All three belong to the code generator: two write Go source files onto the
|
||||
// server's filesystem and the third inserts menus, APIs and casbin rules into
|
||||
// the database. They are registered under a group whose own name says it does
|
||||
// no role check, and a demo deployment lets anybody log in - so on a demo host
|
||||
// they were reachable by any visitor, and the menus one had in fact been used.
|
||||
//
|
||||
// Spelled as gin route patterns, which is what Context.FullPath returns, so a
|
||||
// path parameter matches whatever value it is given.
|
||||
//
|
||||
// This list cannot be checked from here: common/ may not import app/, so this
|
||||
// package cannot see which routes exist. What keeps it honest is a test beside
|
||||
// the routes themselves - see app/other/router - which registers them and
|
||||
// fails if any entry here has stopped being a real route.
|
||||
//
|
||||
// It also does not close the general hole. Nothing stops the next GET handler
|
||||
// that writes something from being added without an entry here, and no static
|
||||
// check can tell a handler that writes from one that reads. Demo mode refuses
|
||||
// the routes it has been told about; that is the whole of the guarantee.
|
||||
var demoWriteRoutes = map[string]bool{
|
||||
"/api/v1/gen/toproject/:tableId": true,
|
||||
"/api/v1/gen/apitofile/:tableId": true,
|
||||
"/api/v1/gen/todb/:tableId": true,
|
||||
}
|
||||
|
||||
// DemoWriteRoutes returns the routes demo mode refuses despite their method.
|
||||
//
|
||||
// Exported only so the test that lives beside the route registrations can
|
||||
// check every one of them still exists; nothing else should need it.
|
||||
func DemoWriteRoutes() []string {
|
||||
out := make([]string, 0, len(demoWriteRoutes))
|
||||
for route := range demoWriteRoutes {
|
||||
out = append(out, route)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// demoAllows reports whether demo mode lets a request through.
|
||||
//
|
||||
// route is the matched gin route pattern and uri the raw request target; the
|
||||
// two are different things and both are needed. The route is what identifies a
|
||||
// handler regardless of the values in its path parameters, and it is empty for
|
||||
// a request that matched nothing - which is why the login and logout checks
|
||||
// still read the raw target, as they always did.
|
||||
func demoAllows(method, route, uri string) bool {
|
||||
if demoWriteRoutes[route] {
|
||||
return false
|
||||
}
|
||||
return method == http.MethodGet ||
|
||||
method == http.MethodOptions ||
|
||||
uri == "/api/v1/login" ||
|
||||
uri == "/api/v1/logout"
|
||||
}
|
||||
|
||||
// demoMessage is the answer a refused request gets.
|
||||
//
|
||||
// application.demomsg has been in the configuration all along and nothing read
|
||||
// it: the message was hard-coded here, and the demo host's configured string
|
||||
// happened to be identical, so the setting looked like it worked. An empty
|
||||
// value falls back rather than answering with nothing.
|
||||
func demoMessage() string {
|
||||
if msg := config.ApplicationConfig.DemoMsg; msg != "" {
|
||||
return msg
|
||||
}
|
||||
return defaultDemoMsg
|
||||
}
|
||||
|
||||
// DemoEvn refuses anything that would change state while mode is demo.
|
||||
func DemoEvn() gin.HandlerFunc {
|
||||
return func(c *gin.Context) {
|
||||
method := c.Request.Method
|
||||
if config.ApplicationConfig.Mode == "demo" {
|
||||
if method == "GET" ||
|
||||
method == "OPTIONS" ||
|
||||
c.Request.RequestURI == "/api/v1/login" ||
|
||||
c.Request.RequestURI == "/api/v1/logout" {
|
||||
c.Next()
|
||||
} else {
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"code": 500,
|
||||
"msg": "谢谢您的参与,但为了大家更好的体验,所以本次提交就算了吧!\U0001F600\U0001F600\U0001F600",
|
||||
})
|
||||
c.Abort()
|
||||
return
|
||||
}
|
||||
if config.ApplicationConfig.Mode != "demo" {
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
c.Next()
|
||||
if demoAllows(c.Request.Method, c.FullPath(), c.Request.RequestURI) {
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
c.JSON(http.StatusOK, gin.H{
|
||||
"code": 500,
|
||||
"msg": demoMessage(),
|
||||
})
|
||||
c.Abort()
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,144 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
)
|
||||
|
||||
// demoMode puts the process in demo mode for one test and puts it back.
|
||||
func demoMode(t *testing.T, mode, msg string) {
|
||||
t.Helper()
|
||||
previousMode, previousMsg := config.ApplicationConfig.Mode, config.ApplicationConfig.DemoMsg
|
||||
t.Cleanup(func() {
|
||||
config.ApplicationConfig.Mode = previousMode
|
||||
config.ApplicationConfig.DemoMsg = previousMsg
|
||||
})
|
||||
config.ApplicationConfig.Mode, config.ApplicationConfig.DemoMsg = mode, msg
|
||||
}
|
||||
|
||||
// The method is not enough on its own. Three of the generator's routes are
|
||||
// registered as GET and write anyway - two of them onto the server's
|
||||
// filesystem, one into the database - so a guard that reads only the method
|
||||
// serves them to anybody who can log in, which on a demo host is everybody.
|
||||
func TestDemoRefusesTheWritesThatAreServedOverGET(t *testing.T) {
|
||||
const login = "/api/v1/login"
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
method string
|
||||
route, uri string
|
||||
wantThrough bool
|
||||
}{
|
||||
{"a plain read", http.MethodGet, "/api/v1/dept", "/api/v1/dept", true},
|
||||
{"a write, by method", http.MethodPost, "/api/v1/dept", "/api/v1/dept", false},
|
||||
{"login is how a visitor gets in", http.MethodPost, login, login, true},
|
||||
{"logout", http.MethodPost, "/api/v1/logout", "/api/v1/logout", true},
|
||||
{"preflight", http.MethodOptions, "/api/v1/dept", "/api/v1/dept", true},
|
||||
// A request that matched no route has an empty pattern, and the guard
|
||||
// still has to refuse it by method - this is what a POST to a path
|
||||
// that does not exist looks like from in here.
|
||||
{"a write to nothing at all", http.MethodPost, "", "/api/v1/__probe__", false},
|
||||
|
||||
// The three this change is about.
|
||||
{"generator writes the database", http.MethodGet,
|
||||
"/api/v1/gen/todb/:tableId", "/api/v1/gen/todb/3", false},
|
||||
{"generator writes source files", http.MethodGet,
|
||||
"/api/v1/gen/toproject/:tableId", "/api/v1/gen/toproject/3", false},
|
||||
{"generator writes an api file", http.MethodGet,
|
||||
"/api/v1/gen/apitofile/:tableId", "/api/v1/gen/apitofile/3", false},
|
||||
|
||||
// The read-only half of the generator has to keep working, or the demo
|
||||
// host cannot demonstrate the feature at all. Refusing too much is as
|
||||
// much of a defect as refusing too little.
|
||||
{"generator preview stays available", http.MethodGet,
|
||||
"/api/v1/gen/preview/:tableId", "/api/v1/gen/preview/3", true},
|
||||
{"generator table tree stays available", http.MethodGet,
|
||||
"/api/v1/gen/tabletree", "/api/v1/gen/tabletree", true},
|
||||
{"table list stays available", http.MethodGet,
|
||||
"/api/v1/db/tables/page", "/api/v1/db/tables/page", true},
|
||||
{"column list stays available", http.MethodGet,
|
||||
"/api/v1/db/columns/page", "/api/v1/db/columns/page", true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if got := demoAllows(tc.method, tc.route, tc.uri); got != tc.wantThrough {
|
||||
t.Errorf("demoAllows(%s %s) = %v, want %v", tc.method, tc.route, got, tc.wantThrough)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Everything above is about demo mode only. A deployment that is not a demo
|
||||
// runs the generator for real, and a guard that reached it there would have
|
||||
// taken the feature away from every production install.
|
||||
func TestOutsideDemoModeNothingIsRefused(t *testing.T) {
|
||||
demoMode(t, "prod", "")
|
||||
gin.SetMode(gin.TestMode)
|
||||
|
||||
for _, route := range append(DemoWriteRoutes(), "/api/v1/dept") {
|
||||
t.Run(route, func(t *testing.T) {
|
||||
r := gin.New()
|
||||
r.Use(DemoEvn())
|
||||
r.GET(route, func(c *gin.Context) { c.String(http.StatusOK, "served") })
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, requestFor(route), nil))
|
||||
if w.Body.String() != "served" {
|
||||
t.Errorf("answered %q; outside demo mode the handler must run", w.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The refusal has to come back as the demo message rather than a 403 or a 404:
|
||||
// the front end shows it to the visitor, and the point of a demo host is that
|
||||
// being turned away is explained.
|
||||
func TestDemoRefusalCarriesTheConfiguredMessage(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
const route = "/api/v1/gen/todb/:tableId"
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, configured, want string
|
||||
}{
|
||||
{"configured", "come back tomorrow", "come back tomorrow"},
|
||||
// A deployment that never set application.demomsg keeps the answer it
|
||||
// already had; an empty setting must not become an empty message.
|
||||
{"not configured", "", defaultDemoMsg},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
demoMode(t, "demo", tc.configured)
|
||||
|
||||
r := gin.New()
|
||||
r.Use(DemoEvn())
|
||||
r.GET(route, func(c *gin.Context) { c.String(http.StatusOK, "served") })
|
||||
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, requestFor(route), nil))
|
||||
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("answered %d, want 200 so the front end reads the body", w.Code)
|
||||
}
|
||||
if body := w.Body.String(); !strings.Contains(body, tc.want) {
|
||||
t.Errorf("body %q does not carry %q", body, tc.want)
|
||||
}
|
||||
if strings.Contains(w.Body.String(), "served") {
|
||||
t.Error("the handler ran; the request was supposed to be refused")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// requestFor turns a route pattern into a request target by giving every path
|
||||
// parameter a value.
|
||||
func requestFor(route string) string {
|
||||
segments := strings.Split(route, "/")
|
||||
for i, segment := range segments {
|
||||
if strings.HasPrefix(segment, ":") {
|
||||
segments[i] = "1"
|
||||
}
|
||||
}
|
||||
return strings.Join(segments, "/")
|
||||
}
|
||||
@@ -1,7 +1,6 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"go-admin/app/admin/models"
|
||||
"go-admin/common"
|
||||
"net/http"
|
||||
|
||||
@@ -163,19 +162,31 @@ func LogOut(c *gin.Context) {
|
||||
|
||||
}
|
||||
|
||||
// Authorizator decides whether a parsed identity may proceed. It authorizes
|
||||
// every identity IdentityHandler was able to build, which is what it has always
|
||||
// done.
|
||||
//
|
||||
// It used to also assert data["user"] and data["role"] into app/admin/models
|
||||
// types and copy five fields onto the context. Those two keys are not in the
|
||||
// map: IdentityHandler builds it from the token claims and puts in
|
||||
// IdentityKey / UserName / RoleKey / UserId / RoleIds / DataScope. Both
|
||||
// assertions therefore failed on every request, and because the ok result was
|
||||
// discarded, the five c.Set calls stored zero values and the function returned
|
||||
// true regardless.
|
||||
//
|
||||
// Nothing in this repository or in go-admin-core reads role / roleIds /
|
||||
// userId / userName / dataScope off the context - the open-source data
|
||||
// permission path reads the JWT claims through
|
||||
// common/actions.Permission -> user.GetUserIdStr(c). Dropping the block
|
||||
// therefore removes five zero values nobody read, and with them the last
|
||||
// import of app/admin from a contract package.
|
||||
//
|
||||
// Anything maintaining its own copy of this file must check its own consumers
|
||||
// before taking this change: a codebase that does read those keys off the
|
||||
// context needs Authorizator to keep setting them.
|
||||
func Authorizator(data interface{}, c *gin.Context) bool {
|
||||
|
||||
if v, ok := data.(map[string]interface{}); ok {
|
||||
u, _ := v["user"].(models.SysUser)
|
||||
r, _ := v["role"].(models.SysRole)
|
||||
c.Set("role", r.RoleName)
|
||||
c.Set("roleIds", r.RoleId)
|
||||
c.Set("userId", u.UserId)
|
||||
c.Set("userName", u.Username)
|
||||
c.Set("dataScope", r.DataScope)
|
||||
return true
|
||||
}
|
||||
return false
|
||||
_, ok := data.(map[string]interface{})
|
||||
return ok
|
||||
}
|
||||
|
||||
func Unauthorized(c *gin.Context, code int, message string) {
|
||||
|
||||
@@ -2,15 +2,20 @@ package middleware
|
||||
|
||||
import (
|
||||
"github.com/gin-gonic/gin"
|
||||
jwt "github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
log "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
"go-admin/common/actions"
|
||||
)
|
||||
|
||||
// These alias core's own constants (see sdk/runtime.GetHandlerFunc's contract
|
||||
// doc, section 9) rather than redeclaring the same three strings, so a typo
|
||||
// here can no longer split registration and lookup into two different keys
|
||||
// that both happen to compile.
|
||||
const (
|
||||
JwtTokenCheck string = "JwtToken"
|
||||
RoleCheck string = "AuthCheckRole"
|
||||
PermissionCheck string = "PermissionAction"
|
||||
JwtTokenCheck = runtime.JwtTokenCheck
|
||||
RoleCheck = runtime.RoleCheck
|
||||
PermissionCheck = runtime.PermissionCheck
|
||||
)
|
||||
|
||||
func InitMiddleware(r *gin.Engine) {
|
||||
@@ -29,7 +34,26 @@ func InitMiddleware(r *gin.Engine) {
|
||||
r.Use(Secure)
|
||||
// 链路追踪
|
||||
//r.Use(middleware.Trace())
|
||||
sdk.Runtime.SetMiddleware(JwtTokenCheck, (*jwt.GinJWTMiddleware).MiddlewareFunc)
|
||||
|
||||
// Build the shared JWT middleware instance here, before any module
|
||||
// registers routes (initRouter runs ahead of runStartupHooks, which is
|
||||
// what invokes each module's InitRouter - see cmd/api/server.go). Doing
|
||||
// it once here, instead of once per module via AuthInit, is what makes
|
||||
// GetAuthMiddleware and sdk.Runtime.GetHandlerFunc(JwtTokenCheck) both
|
||||
// resolve to a single, meaningful instance instead of "whichever module
|
||||
// happened to initialize last".
|
||||
//
|
||||
// SetMiddleware must be given a bound closure (authMiddleware.MiddlewareFunc()),
|
||||
// not the unbound method expression (*jwt.GinJWTMiddleware).MiddlewareFunc:
|
||||
// the latter has no receiver bound to it, so GetHandlerFunc's type
|
||||
// assertion to gin.HandlerFunc always fails for it.
|
||||
var err error
|
||||
authMiddleware, err = AuthInit()
|
||||
if err != nil {
|
||||
// A process with no JWT middleware must not start serving requests.
|
||||
log.Fatalf("JWT Init Error, %s", err.Error())
|
||||
}
|
||||
sdk.Runtime.SetMiddleware(JwtTokenCheck, authMiddleware.MiddlewareFunc())
|
||||
sdk.Runtime.SetMiddleware(RoleCheck, AuthCheckRole())
|
||||
sdk.Runtime.SetMiddleware(PermissionCheck, actions.PermissionAction())
|
||||
}
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
)
|
||||
|
||||
// freshRuntime hands the test its own Runtime and puts the old one back, the
|
||||
// same pattern cmd/api/server_test.go uses: sdk.Runtime is a process-wide
|
||||
// singleton, and a test that registers into it would otherwise leak state
|
||||
// into every other test in the binary.
|
||||
func freshRuntime(t *testing.T) {
|
||||
t.Helper()
|
||||
previous := sdk.Runtime
|
||||
t.Cleanup(func() { sdk.Runtime = previous })
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
}
|
||||
|
||||
// TestInitMiddlewareRegistersUsableJwtHandlerFunc is the reverse proof for
|
||||
// hoisting the JWT instance's construction into InitMiddleware:
|
||||
// sdk.Runtime.GetHandlerFunc(JwtTokenCheck) must hand back ok=true and a
|
||||
// non-nil gin.HandlerFunc, not just something GetMiddleware can return as an
|
||||
// untyped interface{}.
|
||||
//
|
||||
// Before this change, InitMiddleware registered the unbound method
|
||||
// expression (*jwt.GinJWTMiddleware).MiddlewareFunc under this key - a value
|
||||
// with no receiver bound to it, which is not a gin.HandlerFunc no matter how
|
||||
// a caller asserts its type. Reverting the registration below to that
|
||||
// expression makes GetHandlerFunc report ok=false; it does not fail to
|
||||
// compile, because (*jwt.GinJWTMiddleware).MiddlewareFunc has a well-formed,
|
||||
// unrelated method-expression type that SetMiddleware's interface{} param
|
||||
// happily accepts.
|
||||
func TestInitMiddlewareRegistersUsableJwtHandlerFunc(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
previousSecret := config.JwtConfig.Secret
|
||||
config.JwtConfig.Secret = "test-secret-key"
|
||||
t.Cleanup(func() { config.JwtConfig.Secret = previousSecret })
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
InitMiddleware(gin.New())
|
||||
|
||||
h, ok := sdk.Runtime.GetHandlerFunc(JwtTokenCheck)
|
||||
if !ok {
|
||||
t.Fatal("GetHandlerFunc(JwtTokenCheck) reported ok=false after InitMiddleware ran")
|
||||
}
|
||||
if h == nil {
|
||||
t.Fatal("GetHandlerFunc(JwtTokenCheck) reported ok=true but returned a nil handler")
|
||||
}
|
||||
}
|
||||
|
||||
// TestInitMiddlewareBuildsOneSharedJwtInstance locks down the fix for the
|
||||
// four-instances problem: GetAuthMiddleware must return the very instance
|
||||
// InitMiddleware built and handed to sdk.Runtime, not a lookalike built
|
||||
// separately by whichever caller asks first.
|
||||
func TestInitMiddlewareBuildsOneSharedJwtInstance(t *testing.T) {
|
||||
freshRuntime(t)
|
||||
|
||||
previousSecret := config.JwtConfig.Secret
|
||||
config.JwtConfig.Secret = "test-secret-key"
|
||||
t.Cleanup(func() { config.JwtConfig.Secret = previousSecret })
|
||||
|
||||
gin.SetMode(gin.TestMode)
|
||||
InitMiddleware(gin.New())
|
||||
|
||||
shared := GetAuthMiddleware()
|
||||
if shared == nil {
|
||||
t.Fatal("GetAuthMiddleware returned nil after InitMiddleware ran")
|
||||
}
|
||||
if shared != authMiddleware {
|
||||
t.Error("GetAuthMiddleware did not return the package-level instance InitMiddleware built")
|
||||
}
|
||||
}
|
||||
+67
-20
@@ -1,19 +1,18 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"go-admin/app/admin/service/dto"
|
||||
"errors"
|
||||
"go-admin/common"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/jwtauth/user"
|
||||
"github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/api"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
@@ -28,19 +27,14 @@ func LoggerToFile() gin.HandlerFunc {
|
||||
// 开始时间
|
||||
startTime := time.Now()
|
||||
// 处理请求
|
||||
//
|
||||
// The body is only read when it has a destination. operParam below is
|
||||
// the only consumer, and it is written when logger.enableddb is on -
|
||||
// off in the shipped configuration, where reading the body was a copy
|
||||
// of every request made and discarded.
|
||||
var body string
|
||||
switch c.Request.Method {
|
||||
case http.MethodPost, http.MethodPut, http.MethodGet, http.MethodDelete:
|
||||
bf := bytes.NewBuffer(nil)
|
||||
wt := bufio.NewWriter(bf)
|
||||
_, err := io.Copy(wt, c.Request.Body)
|
||||
if err != nil {
|
||||
log.Warnf("copy body error, %s", err.Error())
|
||||
err = nil
|
||||
}
|
||||
rb, _ := ioutil.ReadAll(bf)
|
||||
c.Request.Body = ioutil.NopCloser(bytes.NewBuffer(rb))
|
||||
body = string(rb)
|
||||
if config.LoggerConfig.EnabledDB {
|
||||
body = readOperParam(c, log)
|
||||
}
|
||||
|
||||
c.Next()
|
||||
@@ -100,10 +94,55 @@ func LoggerToFile() gin.HandlerFunc {
|
||||
}
|
||||
}
|
||||
|
||||
// SetDBOperLog 写入操作日志表 fixme 该方法后续即将弃用
|
||||
func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) {
|
||||
// operParamLimit caps what is copied out of a request body for the operation
|
||||
// log. A file upload is a POST like any other and reaches this middleware
|
||||
// before any handler, so without a limit the whole upload is held in memory to
|
||||
// write a log row - a 16MB upload allocated about 67MB. The limit also keeps
|
||||
// the value inside the column, which is TEXT.
|
||||
const operParamLimit = 32 << 10
|
||||
|
||||
log := api.GetRequestLogger(c)
|
||||
// readOperParam copies the start of the request body for the operation log and
|
||||
// leaves the request readable by the handler.
|
||||
//
|
||||
// The body is not buffered whole: the handler reads the part copied here from
|
||||
// memory and the rest straight from the connection, so what this holds is
|
||||
// bounded by operParamLimit however large the request is.
|
||||
func readOperParam(c *gin.Context, log *logger.Helper) string {
|
||||
switch c.Request.Method {
|
||||
case http.MethodPost, http.MethodPut, http.MethodGet, http.MethodDelete:
|
||||
default:
|
||||
return ""
|
||||
}
|
||||
if c.Request.Body == nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
rest := c.Request.Body
|
||||
head := make([]byte, operParamLimit)
|
||||
n, err := io.ReadFull(rest, head)
|
||||
if err != nil && !errors.Is(err, io.EOF) && !errors.Is(err, io.ErrUnexpectedEOF) {
|
||||
log.Warnf("read body for the operation log: %s", err)
|
||||
}
|
||||
head = head[:n]
|
||||
|
||||
c.Request.Body = readCloser{
|
||||
Reader: io.MultiReader(bytes.NewReader(head), rest),
|
||||
Closer: rest,
|
||||
}
|
||||
return string(head)
|
||||
}
|
||||
|
||||
type readCloser struct {
|
||||
io.Reader
|
||||
io.Closer
|
||||
}
|
||||
|
||||
// operaLogFields builds the message written to the operation log queue.
|
||||
//
|
||||
// Split out of SetDBOperLog so the field set can be asserted in a test: the
|
||||
// consumer on the other end of the queue reads these keys by name, so a
|
||||
// dropped or renamed key costs a column in sys_opera_log and reports nothing.
|
||||
func operaLogFields(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) map[string]interface{} {
|
||||
l := make(map[string]interface{})
|
||||
l["_fullPath"] = c.FullPath()
|
||||
l["operUrl"] = reqUri
|
||||
@@ -120,10 +159,18 @@ func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string
|
||||
l["createBy"] = user.GetUserId(c)
|
||||
l["updateBy"] = user.GetUserId(c)
|
||||
if status == http.StatusOK {
|
||||
l["status"] = dto.OperaStatusEnabel
|
||||
l["status"] = global.OperaStatusEnabled
|
||||
} else {
|
||||
l["status"] = dto.OperaStatusDisable
|
||||
l["status"] = global.OperaStatusDisabled
|
||||
}
|
||||
return l
|
||||
}
|
||||
|
||||
// SetDBOperLog 写入操作日志表 fixme 该方法后续即将弃用
|
||||
func SetDBOperLog(c *gin.Context, clientIP string, statusCode int, reqUri string, reqMethod string, latencyTime time.Duration, body string, result string, status int) {
|
||||
|
||||
log := api.GetRequestLogger(c)
|
||||
l := operaLogFields(c, clientIP, statusCode, reqUri, reqMethod, latencyTime, body, result, status)
|
||||
q := sdk.Runtime.GetQueuePrefix(c.Request.Host)
|
||||
message, err := sdk.Runtime.GetStreamMessage("", global.OperateLog, l)
|
||||
if err != nil {
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
)
|
||||
|
||||
// serveWithLogger runs one request through the logger middleware and returns
|
||||
// what the handler saw, with logger.enableddb set as given.
|
||||
func serveWithLogger(t testing.TB, enabledDB bool, method, body string) string {
|
||||
t.Helper()
|
||||
|
||||
prev := config.LoggerConfig.EnabledDB
|
||||
config.LoggerConfig.EnabledDB = enabledDB
|
||||
t.Cleanup(func() { config.LoggerConfig.EnabledDB = prev })
|
||||
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
r := gin.New()
|
||||
r.Use(LoggerToFile())
|
||||
|
||||
var seen string
|
||||
handler := func(c *gin.Context) {
|
||||
b, err := io.ReadAll(c.Request.Body)
|
||||
if err != nil {
|
||||
t.Errorf("handler could not read the body: %v", err)
|
||||
}
|
||||
seen = string(b)
|
||||
c.Status(http.StatusOK)
|
||||
}
|
||||
r.Handle(method, "/probe", handler)
|
||||
|
||||
req := httptest.NewRequest(method, "/probe", strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
r.ServeHTTP(httptest.NewRecorder(), req)
|
||||
return seen
|
||||
}
|
||||
|
||||
// The middleware rewrites Request.Body so it can log the parameters. Whatever
|
||||
// else it does, the handler has to receive the request the client sent - all
|
||||
// of it, whether or not the operation log is on, and whether or not the body
|
||||
// is longer than what gets logged.
|
||||
func TestHandlerStillSeesTheWholeBody(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
enabledDB bool
|
||||
body string
|
||||
}{
|
||||
{"log off, short body", false, `{"username":"admin"}`},
|
||||
{"log on, short body", true, `{"username":"admin"}`},
|
||||
{"log off, empty body", false, ""},
|
||||
{"log on, empty body", true, ""},
|
||||
// Longer than operParamLimit: the logged copy is truncated, the body is not.
|
||||
{"log on, body past the limit", true, strings.Repeat("x", operParamLimit+4096)},
|
||||
{"log off, body past the limit", false, strings.Repeat("y", operParamLimit+4096)},
|
||||
// Exactly at the boundary, where a fencepost error would show.
|
||||
{"log on, body exactly at the limit", true, strings.Repeat("z", operParamLimit)},
|
||||
}
|
||||
|
||||
for _, c := range cases {
|
||||
t.Run(c.name, func(t *testing.T) {
|
||||
for _, method := range []string{http.MethodPost, http.MethodPut, http.MethodDelete} {
|
||||
if got := serveWithLogger(t, c.enabledDB, method, c.body); got != c.body {
|
||||
t.Errorf("%s: handler saw %d bytes, the client sent %d",
|
||||
method, len(got), len(c.body))
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// The body is read for one reason - operParam on the operation log row - and
|
||||
// that row is only written when logger.enableddb is on. With it off, reading
|
||||
// the body is a copy of every request made and thrown away, and a file upload
|
||||
// is a POST like any other: 16MB of upload allocated about 67MB here.
|
||||
//
|
||||
// Allocation counts are deterministic across machines; wall-clock is not.
|
||||
func TestBodyIsNotCopiedWhenTheOperationLogIsOff(t *testing.T) {
|
||||
const size = 1 << 20
|
||||
body := strings.Repeat("x", size)
|
||||
|
||||
prev := config.LoggerConfig.EnabledDB
|
||||
config.LoggerConfig.EnabledDB = false
|
||||
t.Cleanup(func() { config.LoggerConfig.EnabledDB = prev })
|
||||
|
||||
gin.SetMode(gin.ReleaseMode)
|
||||
r := gin.New()
|
||||
r.Use(LoggerToFile())
|
||||
r.POST("/probe", func(c *gin.Context) { c.Status(http.StatusOK) })
|
||||
|
||||
payload := []byte(body)
|
||||
run := func() {
|
||||
req := httptest.NewRequest(http.MethodPost, "/probe", bytes.NewReader(payload))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
r.ServeHTTP(httptest.NewRecorder(), req)
|
||||
}
|
||||
|
||||
var before, after uint64
|
||||
before = heapAllocs()
|
||||
run()
|
||||
after = heapAllocs()
|
||||
|
||||
// The handler never reads the body, so a request that does not copy it
|
||||
// should allocate far less than the body's size. The old middleware
|
||||
// allocated about four times the body.
|
||||
if grew := after - before; grew > size/2 {
|
||||
t.Errorf("a %d-byte request allocated %d bytes with the operation log off; "+
|
||||
"the body should not be read when nothing consumes it", size, grew)
|
||||
}
|
||||
}
|
||||
|
||||
func heapAllocs() uint64 {
|
||||
var m runtime.MemStats
|
||||
runtime.GC()
|
||||
runtime.ReadMemStats(&m)
|
||||
return m.TotalAlloc
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"go-admin/common/global"
|
||||
)
|
||||
|
||||
// The operation-log consumer reads these keys by name off the queue message.
|
||||
// Losing one costs a column in sys_opera_log and reports nothing - the request
|
||||
// still succeeds, the log row is just wrong.
|
||||
//
|
||||
// This locks the set down across the move of the status constants out of
|
||||
// app/admin/service/dto, which touched every request path.
|
||||
var operaLogKeys = []string{
|
||||
"_fullPath", "operUrl", "operIp", "operLocation", "operName",
|
||||
"requestMethod", "operParam", "operTime", "jsonResult", "latencyTime",
|
||||
"statusCode", "userAgent", "createBy", "updateBy", "status",
|
||||
}
|
||||
|
||||
func TestOperaLogFieldsAreComplete(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodPost, "/api/v1/sys-user", nil)
|
||||
c.Request.Header.Set("User-Agent", "go-test")
|
||||
|
||||
l := operaLogFields(c, "127.0.0.1", http.StatusOK, "/api/v1/sys-user", http.MethodPost,
|
||||
12*time.Millisecond, `{"a":1}`, `{"code":200}`, http.StatusOK)
|
||||
|
||||
for _, k := range operaLogKeys {
|
||||
if _, ok := l[k]; !ok {
|
||||
t.Errorf("operation log is missing %q", k)
|
||||
}
|
||||
}
|
||||
if len(l) != len(operaLogKeys) {
|
||||
t.Errorf("operation log has %d fields, expected %d; update operaLogKeys deliberately, not to make this pass",
|
||||
len(l), len(operaLogKeys))
|
||||
}
|
||||
if got := l["operUrl"]; got != "/api/v1/sys-user" {
|
||||
t.Errorf("operUrl = %v", got)
|
||||
}
|
||||
if got := l["userAgent"]; got != "go-test" {
|
||||
t.Errorf("userAgent = %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
// status is what tells a failed request from a successful one in the log table.
|
||||
// It is a string, and it is the one field whose source package changed.
|
||||
func TestOperaLogStatusMapping(t *testing.T) {
|
||||
gin.SetMode(gin.TestMode)
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
status int
|
||||
want string
|
||||
}{
|
||||
{"ok", http.StatusOK, global.OperaStatusEnabled},
|
||||
{"error", http.StatusInternalServerError, global.OperaStatusDisabled},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
c, _ := gin.CreateTestContext(httptest.NewRecorder())
|
||||
c.Request = httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
l := operaLogFields(c, "127.0.0.1", tc.status, "/", http.MethodGet, 0, "", "", tc.status)
|
||||
if l["status"] != tc.want {
|
||||
t.Fatalf("status = %v, want %v", l["status"], tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,10 +1,11 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"github.com/casbin/casbin/v3/util"
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
mycasbin "github.com/go-admin-team/go-admin-core/v2/casbin"
|
||||
"github.com/go-admin-team/go-admin-core/v2/jwtauth"
|
||||
"github.com/go-admin-team/go-admin-core/v2/response"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
@@ -26,11 +27,9 @@ func AuthCheckRole() gin.HandlerFunc {
|
||||
c.Next()
|
||||
return
|
||||
}
|
||||
for _, i := range CasbinExclude {
|
||||
if util.KeyMatch2(c.Request.URL.Path, i.Url) && c.Request.Method == i.Method {
|
||||
casbinExclude = true
|
||||
break
|
||||
}
|
||||
casbinExclude, err = excludedFromCasbin(c.Request.Method, c.Request.URL.Path)
|
||||
if err != nil {
|
||||
log.Errorf("AuthCheckRole: %s", err)
|
||||
}
|
||||
if casbinExclude {
|
||||
log.Infof("Casbin exclusion, no validation method:%s path:%s", c.Request.Method, c.Request.URL.Path)
|
||||
@@ -59,3 +58,59 @@ func AuthCheckRole() gin.HandlerFunc {
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
// EnforceRoleFor reports whether the caller's role has explicit Casbin
|
||||
// permission to act on path with method.
|
||||
//
|
||||
// AuthCheckRole never calls Enforce for a route CasbinExclude lists - that
|
||||
// is the whole point of the list. A handler on such a route can still need
|
||||
// the real answer for part of what it does: sys_user.go's Update shares its
|
||||
// excluded route between the personal-center screen editing the caller's own
|
||||
// record (which is why the route is excluded at all) and an admin editing
|
||||
// someone else's, and only the second case is meant to require a policy
|
||||
// grant. That handler asks here instead of assuming the middleware already
|
||||
// checked.
|
||||
func EnforceRoleFor(c *gin.Context, path, method string) (bool, error) {
|
||||
data, ok := c.Get(jwtauth.JwtPayloadKey)
|
||||
if !ok {
|
||||
return false, nil
|
||||
}
|
||||
v, ok := data.(jwtauth.MapClaims)
|
||||
if !ok {
|
||||
return false, nil
|
||||
}
|
||||
if v["rolekey"] == "admin" {
|
||||
return true, nil
|
||||
}
|
||||
e := sdk.Runtime.GetCasbinByTenant(c.Request.Host)
|
||||
return e.Enforce(v["rolekey"], path, method)
|
||||
}
|
||||
|
||||
// excludedFromCasbin reports whether the route skips the permission check.
|
||||
//
|
||||
// It runs for every non-admin request, so the order matters: the method rules
|
||||
// out most entries with a string compare, where the path test costs a pattern
|
||||
// match. mycasbin.KeyMatch2 answers what casbin's util.KeyMatch2 answers
|
||||
// without recompiling the pattern every time, which is what made this loop
|
||||
// expensive - about 2,500 allocations per request against a 32-entry list.
|
||||
//
|
||||
// A pattern that will not compile is a bug in CasbinExclude rather than in the
|
||||
// request, so the entry is skipped and the scan continues; the error comes
|
||||
// back for the caller to log.
|
||||
func excludedFromCasbin(method, path string) (bool, error) {
|
||||
var bad error
|
||||
for _, i := range CasbinExclude {
|
||||
if method != i.Method {
|
||||
continue
|
||||
}
|
||||
ok, err := mycasbin.KeyMatch2(path, i.Url)
|
||||
if err != nil {
|
||||
bad = fmt.Errorf("CasbinExclude entry %q is not a valid pattern: %w", i.Url, err)
|
||||
continue
|
||||
}
|
||||
if ok {
|
||||
return true, bad
|
||||
}
|
||||
}
|
||||
return false, bad
|
||||
}
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
package middleware
|
||||
|
||||
import "testing"
|
||||
|
||||
// excluded is excludedFromCasbin with the error dropped: these tests are about
|
||||
// the answer and its cost, and CasbinExclude has no malformed entry to report.
|
||||
func excluded(t testing.TB, path, method string) bool {
|
||||
t.Helper()
|
||||
ok, err := excludedFromCasbin(method, path)
|
||||
if err != nil {
|
||||
t.Fatalf("CasbinExclude holds a pattern that will not compile: %s", err)
|
||||
}
|
||||
return ok
|
||||
}
|
||||
|
||||
// TestCasbinExcludeScanMatches pins the behaviour the scan has to keep: an
|
||||
// excluded route is recognised, a protected one is not, and the method has to
|
||||
// agree.
|
||||
func TestCasbinExcludeScanMatches(t *testing.T) {
|
||||
cases := []struct {
|
||||
path, method string
|
||||
want bool
|
||||
}{
|
||||
{"/api/v1/health", "GET", true},
|
||||
{"/api/v1/login", "POST", true},
|
||||
{"/api/v1/roleMenuTreeselect/12", "GET", true},
|
||||
{"/api/v1/dept", "GET", false},
|
||||
{"/api/v1/sys-user", "GET", false},
|
||||
// Same path, wrong method: sys-user is excluded for PUT only.
|
||||
{"/api/v1/sys-user", "PUT", true},
|
||||
{"/api/v1/health", "POST", false},
|
||||
}
|
||||
for _, c := range cases {
|
||||
if got := excluded(t, c.path, c.method); got != c.want {
|
||||
t.Errorf("excludedFromCasbin(%s %s) = %v, want %v", c.method, c.path, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestCasbinExcludeScanAllocationBudget is what keeps the scan cheap.
|
||||
//
|
||||
// The list is walked per request with a pattern match per entry, and
|
||||
// casbin's util.KeyMatch2 compiles a regexp on every call - the whole scan
|
||||
// cost about 2,566 allocations that way. Going back to it fails this test.
|
||||
//
|
||||
// Allocation counts are deterministic across machines; wall-clock is not.
|
||||
func TestCasbinExcludeScanAllocationBudget(t *testing.T) {
|
||||
// A protected route, so the scan runs to the end without an early match -
|
||||
// the case every authenticated business request hits.
|
||||
const path, method = "/api/v1/dept", "GET"
|
||||
|
||||
if excluded(t, path, method) {
|
||||
t.Fatalf("setup failed: %s is in the exclusion list", path)
|
||||
}
|
||||
|
||||
// The budget covers the GET entries that carry a path parameter, which
|
||||
// still need a match. Measured at 0 for the cached matcher; the headroom
|
||||
// is for entries being added to the list.
|
||||
const budget = 64
|
||||
|
||||
got := testing.AllocsPerRun(100, func() {
|
||||
_, _ = excludedFromCasbin(method, path)
|
||||
})
|
||||
if got > budget {
|
||||
t.Errorf("scanning CasbinExclude allocates %.0f times, budget is %d\n"+
|
||||
"casbin's util.KeyMatch2 costs about 2566 here; use mycasbin.KeyMatch2",
|
||||
got, budget)
|
||||
}
|
||||
}
|
||||
|
||||
// BenchmarkCasbinExcludeScan reports what the scan adds to a request.
|
||||
func BenchmarkCasbinExcludeScan(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
b.RunParallel(func(pb *testing.PB) {
|
||||
for pb.Next() {
|
||||
_, _ = excludedFromCasbin("GET", "/api/v1/dept")
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,29 +1,54 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/alibaba/sentinel-golang/core/system"
|
||||
sentinel "github.com/alibaba/sentinel-golang/pkg/adapters/gin"
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
log "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
|
||||
"go-admin/config"
|
||||
)
|
||||
|
||||
// Sentinel 限流
|
||||
//
|
||||
// The threshold comes from extend.ratelimit.inboundqps; see config.RateLimit
|
||||
// for the values it accepts.
|
||||
func Sentinel() gin.HandlerFunc {
|
||||
qps := config.ExtConfig.RateLimit.Threshold()
|
||||
if qps <= 0 {
|
||||
log.Info("rate limit disabled by extend.ratelimit.inboundqps")
|
||||
return func(c *gin.Context) { c.Next() }
|
||||
}
|
||||
|
||||
if _, err := system.LoadRules([]*system.Rule{
|
||||
{
|
||||
MetricType: system.InboundQPS,
|
||||
TriggerCount: 200,
|
||||
Strategy: system.BBR,
|
||||
TriggerCount: qps,
|
||||
// InboundQPS is compared against TriggerCount directly - the
|
||||
// adaptive strategy is only consulted for Load and CpuUsage. BBR
|
||||
// stood here and read as if the limit adapted to the machine, which
|
||||
// it never did.
|
||||
Strategy: system.NoAdaptive,
|
||||
},
|
||||
}); err != nil {
|
||||
log.Fatalf("Unexpected error: %+v", err)
|
||||
}
|
||||
|
||||
log.Infof("rate limit: %.0f inbound req/s", qps)
|
||||
|
||||
return sentinel.SentinelMiddleware(
|
||||
sentinel.WithBlockFallback(func(ctx *gin.Context) {
|
||||
ctx.AbortWithStatusJSON(200, map[string]interface{}{
|
||||
// 429, not 200. Everything that reads the status line rather than
|
||||
// the body counts a 200 as served: load balancers, metrics,
|
||||
// client-side retry, and load tests - a benchmark against the old
|
||||
// behaviour reported the limiter's own rejections as successful
|
||||
// traffic and overstated throughput by more than tenfold.
|
||||
ctx.AbortWithStatusJSON(http.StatusTooManyRequests, map[string]interface{}{
|
||||
"msg": "too many request; the quota used up!",
|
||||
"code": 500,
|
||||
"code": http.StatusTooManyRequests,
|
||||
})
|
||||
}),
|
||||
)
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"github.com/alibaba/sentinel-golang/core/system"
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"go-admin/config"
|
||||
)
|
||||
|
||||
// serve builds a router with the limiter in front of a handler that always
|
||||
// succeeds, so any non-200 comes from the limiter.
|
||||
func serve(t *testing.T) *gin.Engine {
|
||||
t.Helper()
|
||||
gin.SetMode(gin.TestMode)
|
||||
r := gin.New()
|
||||
r.Use(Sentinel())
|
||||
r.GET("/ping", func(c *gin.Context) { c.Status(http.StatusOK) })
|
||||
return r
|
||||
}
|
||||
|
||||
func get(t *testing.T, r *gin.Engine) *httptest.ResponseRecorder {
|
||||
t.Helper()
|
||||
w := httptest.NewRecorder()
|
||||
r.ServeHTTP(w, httptest.NewRequest(http.MethodGet, "/ping", nil))
|
||||
return w
|
||||
}
|
||||
|
||||
// TestSentinelRejectsWithTooManyRequests pins the status code. A rejected
|
||||
// request used to answer 200 with the failure only in the body, so every layer
|
||||
// that reads the status line - load balancers, metrics, client retry, load
|
||||
// tests - counted it as served.
|
||||
func TestSentinelRejectsWithTooManyRequests(t *testing.T) {
|
||||
one := 1.0
|
||||
config.ExtConfig.RateLimit = config.RateLimit{InboundQPS: &one}
|
||||
t.Cleanup(func() {
|
||||
config.ExtConfig.RateLimit = config.RateLimit{}
|
||||
_ = system.ClearRules()
|
||||
})
|
||||
|
||||
r := serve(t)
|
||||
|
||||
var rejected *httptest.ResponseRecorder
|
||||
for i := 0; i < 20; i++ {
|
||||
if w := get(t, r); w.Code != http.StatusOK {
|
||||
rejected = w
|
||||
break
|
||||
}
|
||||
}
|
||||
if rejected == nil {
|
||||
t.Fatal("a limit of 1 req/s let 20 requests through; the limiter is not engaged")
|
||||
}
|
||||
if rejected.Code != http.StatusTooManyRequests {
|
||||
t.Errorf("rejected with %d, want %d", rejected.Code, http.StatusTooManyRequests)
|
||||
}
|
||||
|
||||
// The body's code must agree with the status line; they disagreed before.
|
||||
var body struct {
|
||||
Code int `json:"code"`
|
||||
Msg string `json:"msg"`
|
||||
}
|
||||
if err := json.Unmarshal(rejected.Body.Bytes(), &body); err != nil {
|
||||
t.Fatalf("rejection body is not json: %v", err)
|
||||
}
|
||||
if body.Code != http.StatusTooManyRequests {
|
||||
t.Errorf("body code = %d, want %d", body.Code, http.StatusTooManyRequests)
|
||||
}
|
||||
if body.Msg == "" {
|
||||
t.Error("rejection carries no message")
|
||||
}
|
||||
}
|
||||
|
||||
// TestSentinelDisabledByZero covers the escape hatch: a deployment behind its
|
||||
// own gateway has no use for a second limiter.
|
||||
//
|
||||
// It asserts on the loaded rules rather than on traffic. Sentinel measures QPS
|
||||
// over a sliding window, so a burst issued inside one bucket is not counted
|
||||
// before the bucket closes - a few hundred requests sail past a threshold of
|
||||
// 200 in a test, and "no request was rejected" would pass whether or not the
|
||||
// limiter is disabled. Whether a rule was installed at all does not depend on
|
||||
// timing.
|
||||
func TestSentinelDisabledByZero(t *testing.T) {
|
||||
if err := system.ClearRules(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
zero := 0.0
|
||||
config.ExtConfig.RateLimit = config.RateLimit{InboundQPS: &zero}
|
||||
t.Cleanup(func() {
|
||||
config.ExtConfig.RateLimit = config.RateLimit{}
|
||||
_ = system.ClearRules()
|
||||
})
|
||||
|
||||
r := serve(t)
|
||||
if rules := system.GetRules(); len(rules) != 0 {
|
||||
t.Errorf("limiter disabled but %d rule(s) were loaded: %+v", len(rules), rules)
|
||||
}
|
||||
|
||||
for i := 0; i < 500; i++ {
|
||||
if w := get(t, r); w.Code != http.StatusOK {
|
||||
t.Fatalf("request %d got %d with the limiter disabled", i, w.Code)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -34,6 +34,7 @@ var CasbinExclude = []UrlInfo{
|
||||
{Url: "/api/v1/user/pwd", Method: "PUT"},
|
||||
{Url: "/api/v1/metrics", Method: "GET"},
|
||||
{Url: "/api/v1/health", Method: "GET"},
|
||||
{Url: "/api/v1/ready", Method: "GET"},
|
||||
{Url: "/", Method: "GET"},
|
||||
{Url: "/api/v1/server-monitor", Method: "GET"},
|
||||
{Url: "/api/v1/public/uploadFile", Method: "POST"},
|
||||
|
||||
+9
-37
@@ -1,41 +1,13 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"time"
|
||||
import contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
|
||||
"gorm.io/plugin/soft_delete"
|
||||
// ControlBy, Model and ModelTime are thin aliases of go-admin-core's
|
||||
// sdk/contract/models (PRD 006 F1/F5). A type alias is the same type, not a
|
||||
// new one, so every model that embeds these keeps its GORM tags, JSON tags
|
||||
// and method set untouched.
|
||||
type (
|
||||
ControlBy = contractmodels.ControlBy
|
||||
Model = contractmodels.Model
|
||||
ModelTime = contractmodels.ModelTime
|
||||
)
|
||||
|
||||
type ControlBy struct {
|
||||
CreateBy int `json:"createBy" gorm:"index;comment:创建者"`
|
||||
UpdateBy int `json:"updateBy" gorm:"index;comment:更新者"`
|
||||
}
|
||||
|
||||
// SetCreateBy 设置创建人id
|
||||
func (e *ControlBy) SetCreateBy(createBy int) {
|
||||
e.CreateBy = createBy
|
||||
}
|
||||
|
||||
// SetUpdateBy 设置修改人id
|
||||
func (e *ControlBy) SetUpdateBy(updateBy int) {
|
||||
e.UpdateBy = updateBy
|
||||
}
|
||||
|
||||
type Model struct {
|
||||
Id int `json:"id" gorm:"primaryKey;autoIncrement;comment:主键编码"`
|
||||
}
|
||||
|
||||
type ModelTime struct {
|
||||
CreatedAt time.Time `json:"createdAt" gorm:"comment:创建时间"`
|
||||
UpdatedAt time.Time `json:"updatedAt" gorm:"comment:最后更新时间"`
|
||||
|
||||
// DeletedAt is milliseconds since the epoch, zero while the row is live,
|
||||
// and never null.
|
||||
//
|
||||
// A nullable marker cannot take part in a unique index. Two live rows are
|
||||
// (name, NULL) and (name, NULL), and NULL is not equal to NULL, so the
|
||||
// index permits both — it looks like a constraint and enforces nothing.
|
||||
// With zero for live rows the pair collides, while two deletions of the
|
||||
// same name differ by their timestamps and both remain.
|
||||
DeletedAt soft_delete.DeletedAt `json:"-" gorm:"softDelete:milli;index;comment:删除时间"`
|
||||
}
|
||||
|
||||
+11
-7
@@ -1,11 +1,15 @@
|
||||
package models
|
||||
|
||||
// Menu 菜单中的类型枚举值
|
||||
import contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
|
||||
// Directory, Menu and Button are the menu type enum values used by
|
||||
// sys_menu.menu_type, referenced directly from go-admin-core's
|
||||
// sdk/contract/models rather than restated as literals: PRD 006's hard
|
||||
// constraint 4 requires `const X = pkg.X` for exactly this reason - two
|
||||
// independently written copies of the same value can be edited out of step,
|
||||
// where a direct reference cannot.
|
||||
const (
|
||||
// Directory 目录
|
||||
Directory string = "M"
|
||||
// Menu 菜单
|
||||
Menu string = "C"
|
||||
// Button 按钮
|
||||
Button string = "F"
|
||||
Directory = contractmodels.Directory
|
||||
Menu = contractmodels.Menu
|
||||
Button = contractmodels.Button
|
||||
)
|
||||
|
||||
@@ -1,12 +1,10 @@
|
||||
package models
|
||||
|
||||
import "time"
|
||||
import contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
|
||||
type Migration struct {
|
||||
Version string `gorm:"primaryKey"`
|
||||
ApplyTime time.Time `gorm:"autoCreateTime"`
|
||||
}
|
||||
|
||||
func (Migration) TableName() string {
|
||||
return "sys_migration"
|
||||
}
|
||||
// Migration is the sys_migration row model (data). It is unrelated to
|
||||
// cmd/migrate/migration.Migration, the in-process registration table this
|
||||
// package's TableName has nothing to do with - see
|
||||
// go-admin-core's sdk/contract/models.Migration doc comment for why the two
|
||||
// share a name.
|
||||
type Migration = contractmodels.Migration
|
||||
|
||||
@@ -1,30 +1,10 @@
|
||||
package models
|
||||
|
||||
type Response struct {
|
||||
// 代码
|
||||
Code int `json:"code" example:"200"`
|
||||
// 数据集
|
||||
Data interface{} `json:"data"`
|
||||
// 消息
|
||||
Msg string `json:"msg"`
|
||||
RequestId string `json:"requestId"`
|
||||
}
|
||||
import contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
|
||||
type Page struct {
|
||||
List interface{} `json:"list"`
|
||||
Count int `json:"count"`
|
||||
PageIndex int `json:"pageIndex"`
|
||||
PageSize int `json:"pageSize"`
|
||||
}
|
||||
|
||||
// ReturnOK 正常返回
|
||||
func (res *Response) ReturnOK() *Response {
|
||||
res.Code = 200
|
||||
return res
|
||||
}
|
||||
|
||||
// ReturnError 错误返回
|
||||
func (res *Response) ReturnError(code int) *Response {
|
||||
res.Code = code
|
||||
return res
|
||||
}
|
||||
// Response and Page are thin aliases of go-admin-core's sdk/contract/models
|
||||
// (PRD 006 F1/F5).
|
||||
type (
|
||||
Response = contractmodels.Response
|
||||
Page = contractmodels.Page
|
||||
)
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
package models
|
||||
|
||||
import "gorm.io/gorm/schema"
|
||||
import contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
|
||||
type ActiveRecord interface {
|
||||
schema.Tabler
|
||||
SetCreateBy(createBy int)
|
||||
SetUpdateBy(updateBy int)
|
||||
Generate() ActiveRecord
|
||||
GetId() interface{}
|
||||
}
|
||||
// ActiveRecord is self-referencing (Generate() ActiveRecord), which is why
|
||||
// it must stay a type alias rather than a defined type: aliasing preserves
|
||||
// identity with go-admin-core's sdk/contract/models.ActiveRecord, so a
|
||||
// model whose Generate() returns that interface still satisfies this one. A
|
||||
// defined type here would break every implementer's method set - see
|
||||
// go-admin-core's sdk/contract/models package tests for the counterproof
|
||||
// (PRD 006 counterproof A).
|
||||
type ActiveRecord = contractmodels.ActiveRecord
|
||||
|
||||
+4
-39
@@ -1,42 +1,7 @@
|
||||
package models
|
||||
|
||||
import (
|
||||
"gorm.io/gorm"
|
||||
import contractmodels "github.com/go-admin-team/go-admin-core/v2/sdk/contract/models"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/pkg"
|
||||
)
|
||||
|
||||
// BaseUser 密码登录基础用户
|
||||
type BaseUser struct {
|
||||
Username string `json:"username" gorm:"type:varchar(100);comment:用户名"`
|
||||
Salt string `json:"-" gorm:"type:varchar(255);comment:加盐;<-"`
|
||||
PasswordHash string `json:"-" gorm:"type:varchar(128);comment:密码hash;<-"`
|
||||
Password string `json:"password" gorm:"-"`
|
||||
}
|
||||
|
||||
// SetPassword 设置密码
|
||||
func (u *BaseUser) SetPassword(value string) {
|
||||
u.Password = value
|
||||
u.generateSalt()
|
||||
u.PasswordHash = u.GetPasswordHash()
|
||||
}
|
||||
|
||||
// GetPasswordHash 获取密码hash
|
||||
func (u *BaseUser) GetPasswordHash() string {
|
||||
passwordHash, err := pkg.SetPassword(u.Password, u.Salt)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
return passwordHash
|
||||
}
|
||||
|
||||
// generateSalt 生成加盐值
|
||||
func (u *BaseUser) generateSalt() {
|
||||
u.Salt = pkg.GenerateRandomKey16()
|
||||
}
|
||||
|
||||
// Verify 验证密码
|
||||
func (u *BaseUser) Verify(db *gorm.DB, tableName string) bool {
|
||||
db.Table(tableName).Where("username = ?", u.Username).First(u)
|
||||
return u.GetPasswordHash() == u.PasswordHash
|
||||
}
|
||||
// BaseUser is a thin alias of go-admin-core's sdk/contract/models (PRD 006
|
||||
// F1/F5).
|
||||
type BaseUser = contractmodels.BaseUser
|
||||
|
||||
@@ -8,11 +8,14 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"sync"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/captcha"
|
||||
corelog "github.com/go-admin-team/go-admin-core/v2/logger"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/captcha"
|
||||
)
|
||||
|
||||
// Setup 配置storage组件
|
||||
@@ -20,6 +23,7 @@ func Setup() {
|
||||
setupCache()
|
||||
setupCaptcha()
|
||||
setupQueue()
|
||||
registerQueueDrain()
|
||||
}
|
||||
|
||||
func setupCache() {
|
||||
@@ -34,17 +38,190 @@ func setupCaptcha() {
|
||||
captcha.SetStore(captcha.NewCacheStore(sdk.Runtime.GetCacheAdapter(), 600))
|
||||
}
|
||||
|
||||
var (
|
||||
queueMu sync.Mutex
|
||||
// installed is the adapter setupQueue built, kept so the next reload can
|
||||
// shut it down, and counted so a consumer can tell one from the next.
|
||||
installed interface{ Shutdown() }
|
||||
installedGen uint64
|
||||
// drainRegistered records that the BeforeExit callback is on the runtime,
|
||||
// so that a reload does not add another one.
|
||||
drainRegistered bool
|
||||
)
|
||||
|
||||
// setShutdown is sdk.Runtime.SetShutdown, indirected so that registering can
|
||||
// be observed.
|
||||
//
|
||||
// It has to be: the runtime does not report how many callbacks a phase holds,
|
||||
// and shutdownQueue takes the adapter on its first run, so every registration
|
||||
// after the first returns immediately and changes nothing anybody can see. A
|
||||
// reload adding one callback per round would therefore be invisible from the
|
||||
// outside - which is exactly how it would survive.
|
||||
var setShutdown = func(f func(context.Context)) { sdk.Runtime.SetShutdown(f) }
|
||||
|
||||
// registerQueueDrain puts shutdownQueue on the BeforeExit phase, once.
|
||||
//
|
||||
// Setup is one of the callbacks bootstrap.SetupConfig re-runs on every
|
||||
// configuration change, so registering from it without a guard would leave one
|
||||
// callback per reload - each shutting down the same adapter, each reported
|
||||
// separately when the budget runs out.
|
||||
//
|
||||
// A flag under the existing mutex rather than a sync.Once: the tests in this
|
||||
// package already save and restore installed and installedGen to keep one test
|
||||
// from deciding what the next one sees, and a sync.Once cannot be put back.
|
||||
func registerQueueDrain() {
|
||||
queueMu.Lock()
|
||||
first := !drainRegistered
|
||||
drainRegistered = true
|
||||
queueMu.Unlock()
|
||||
|
||||
if first {
|
||||
setShutdown(shutdownQueue)
|
||||
}
|
||||
}
|
||||
|
||||
// drainedInTime shuts q down and reports whether it finished before ctx expired.
|
||||
func drainedInTime(ctx context.Context, q interface{ Shutdown() }) bool {
|
||||
done := make(chan struct{})
|
||||
go func() {
|
||||
defer close(done)
|
||||
q.Shutdown()
|
||||
}()
|
||||
return finishedBeforeDeadline(ctx, done)
|
||||
}
|
||||
|
||||
// finishedBeforeDeadline waits for done or for ctx, and resolves a tie in
|
||||
// favour of done.
|
||||
//
|
||||
// The tie is the reason this is a function of its own rather than one select
|
||||
// inline. Both channels can be ready when the select runs, select picks at
|
||||
// random among ready cases, and so a single look reports an overrun for a
|
||||
// drain that completed - about half the times it lands there, which is exactly
|
||||
// often enough to be dismissed as noise. core's own RunShutdown re-checks for
|
||||
// this reason.
|
||||
//
|
||||
// Taking channels rather than a queue is what makes it testable: a closed done
|
||||
// and an expired ctx can be handed in together, which is the state a race
|
||||
// would otherwise have to be caught in.
|
||||
func finishedBeforeDeadline(ctx context.Context, done <-chan struct{}) bool {
|
||||
select {
|
||||
case <-done:
|
||||
return true
|
||||
case <-ctx.Done():
|
||||
}
|
||||
|
||||
select {
|
||||
case <-done:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// shutdownQueue drains the queue this package installed, on the way out.
|
||||
//
|
||||
// Nothing used to. core's Memory.Shutdown closes the queue and waits for every
|
||||
// consumer to finish what it is holding, and the legacy adapter cancels its
|
||||
// context and closes the underlying queue - but neither ran at exit, so the
|
||||
// process left with the login log, the operation log and the API sync still
|
||||
// buffered, and left reporting success.
|
||||
//
|
||||
// The adapter is read here rather than captured at registration because a
|
||||
// reload replaces it. Registration happens once per process; this runs against
|
||||
// whatever is current when the signal arrives.
|
||||
//
|
||||
// Only an adapter this package installed. sdk.Runtime.GetQueueAdapter never
|
||||
// returns nil - with no queue section configured the runtime wraps its own
|
||||
// fallback queue - so going through that accessor would shut down a queue this
|
||||
// package neither built nor started.
|
||||
//
|
||||
// The adapter is taken, not read: after this the package owns nothing, so a
|
||||
// reload arriving mid-shutdown builds a new one instead of being handed a
|
||||
// closed one to shut down again. Both implementations tolerate a second
|
||||
// Shutdown, so this is about who owns it rather than about a crash.
|
||||
func shutdownQueue(ctx context.Context) {
|
||||
queueMu.Lock()
|
||||
q := installed
|
||||
installed = nil
|
||||
queueMu.Unlock()
|
||||
|
||||
if q == nil {
|
||||
return
|
||||
}
|
||||
|
||||
if drainedInTime(ctx, q) {
|
||||
corelog.Info("queue: drained")
|
||||
return
|
||||
}
|
||||
// The wait is what the budget bounds, not the work: Shutdown takes no
|
||||
// context and is still running in that goroutine. Saying so here names what
|
||||
// is being lost, which the generic overrun message cannot.
|
||||
corelog.Warnf("queue: the shutdown budget ran out while the queue was still draining - " +
|
||||
"whatever it had not delivered goes with the process. Raise extend.shutdown.cleanup " +
|
||||
"if this recurs.")
|
||||
}
|
||||
|
||||
// QueueGeneration reports how many times this package has installed a queue
|
||||
// adapter. It changes every time setupQueue builds a new one, which is on
|
||||
// every configuration reload, and stays 0 for as long as the configuration has
|
||||
// no queue section at all - in which case nothing is installed and callers are
|
||||
// working with the runtime's own fallback queue.
|
||||
//
|
||||
// It exists because there is no way to ask for the adapter's identity from the
|
||||
// outside. sdk.Runtime.GetQueueAdapter and GetQueuePrefix build a fresh
|
||||
// runtime.Queue wrapper on every call, so comparing what two calls return
|
||||
// compares two wrappers and never matches, however many times the underlying
|
||||
// adapter has been replaced. This package creates the adapter, so this is the
|
||||
// only place that knows. A counter rather than the adapter itself keeps the
|
||||
// comparison on a uint64: an adapter type that is not comparable would panic
|
||||
// an `==` between two interface values.
|
||||
func QueueGeneration() uint64 {
|
||||
queueMu.Lock()
|
||||
defer queueMu.Unlock()
|
||||
return installedGen
|
||||
}
|
||||
|
||||
func setupQueue() {
|
||||
if config.QueueConfig.Empty() {
|
||||
return
|
||||
}
|
||||
if q := sdk.Runtime.GetQueueAdapter(); q != nil {
|
||||
q.Shutdown()
|
||||
}
|
||||
|
||||
queueMu.Lock()
|
||||
defer queueMu.Unlock()
|
||||
|
||||
queueAdapter, err := config.QueueConfig.Setup()
|
||||
if err != nil {
|
||||
log.Fatalf("queue setup error, %s\n", err.Error())
|
||||
}
|
||||
|
||||
previous := installed
|
||||
sdk.Runtime.SetQueueAdapter(queueAdapter)
|
||||
go queueAdapter.Run()
|
||||
installed = queueAdapter
|
||||
installedGen++
|
||||
|
||||
// The previous adapter goes down after the new one is installed, not
|
||||
// before. Shutdown waits for its consumers to deliver what it still holds,
|
||||
// and for that whole wait the runtime would otherwise be handing producers
|
||||
// a queue that has stopped accepting: every Append in the window comes back
|
||||
// ErrQueueClosed, and both call sites in common/middleware log it. Swapping
|
||||
// first leaves no such window - a producer gets the new queue or the old
|
||||
// one, and both work.
|
||||
//
|
||||
// Only an adapter this package installed. GetQueueAdapter never returns
|
||||
// nil - with nothing configured the runtime falls back to its own memory
|
||||
// queue and wraps that - so the `if q := GetQueueAdapter(); q != nil` this
|
||||
// replaces was always true, and shut down the fallback queue on the very
|
||||
// first start, before anything had used it.
|
||||
if previous != nil {
|
||||
previous.Shutdown()
|
||||
}
|
||||
|
||||
// Deliberately not started here. Run has to come after the consumers have
|
||||
// registered: the contract implementations refuse a registration once the
|
||||
// queue is running (storage.ErrQueueAlreadyStarted), and the legacy
|
||||
// adapter this repository still goes through swallows that error rather
|
||||
// than reporting it - its own comment says the interface gives it no way
|
||||
// to tell the caller. Starting here and registering afterwards is
|
||||
// therefore a race that loses consumers in silence. Whoever registers is
|
||||
// the one that starts it.
|
||||
}
|
||||
|
||||
@@ -0,0 +1,333 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
|
||||
)
|
||||
|
||||
// countingQueue stands in for an installed adapter. Only Shutdown is exercised
|
||||
// - the drain callback never publishes or consumes - so the rest of
|
||||
// AdapterQueue is deliberately absent: `installed` is typed on Shutdown alone,
|
||||
// and widening the fake would only invite it to be used for something else.
|
||||
type countingQueue struct {
|
||||
calls atomic.Int32
|
||||
block chan struct{}
|
||||
|
||||
// started is closed on the way into Shutdown, so a test can wait for the
|
||||
// call rather than assume the goroutine that makes it was scheduled. The
|
||||
// caller returns on its own deadline while Shutdown is still running, so
|
||||
// reading calls straight after that return is a race with the increment.
|
||||
startOnce sync.Once
|
||||
started chan struct{}
|
||||
}
|
||||
|
||||
func newCountingQueue() *countingQueue {
|
||||
return &countingQueue{started: make(chan struct{})}
|
||||
}
|
||||
|
||||
func (q *countingQueue) Shutdown() {
|
||||
q.startOnce.Do(func() { close(q.started) })
|
||||
q.calls.Add(1)
|
||||
if q.block != nil {
|
||||
<-q.block
|
||||
}
|
||||
}
|
||||
|
||||
// waitStarted blocks until Shutdown has been entered, or fails the test.
|
||||
func (q *countingQueue) waitStarted(t *testing.T) {
|
||||
t.Helper()
|
||||
select {
|
||||
case <-q.started:
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("Shutdown was never called")
|
||||
}
|
||||
}
|
||||
|
||||
// isolate gives the test its own runtime and its own view of what this package
|
||||
// has installed, and puts the process-wide state back afterwards.
|
||||
//
|
||||
// Same isolation as TestSetupBumpsTheQueueGenerationOnEveryReload, plus
|
||||
// drainRegistered: it is what stops a reload registering a second callback, so
|
||||
// leaving it set would make every later test in this binary see a package that
|
||||
// has already registered.
|
||||
func isolate(t *testing.T) {
|
||||
t.Helper()
|
||||
|
||||
prevQ, prevC := config.QueueConfig, config.CacheConfig
|
||||
prevRuntime := sdk.Runtime
|
||||
queueMu.Lock()
|
||||
prevInstalled, prevGen, prevRegistered := installed, installedGen, drainRegistered
|
||||
queueMu.Unlock()
|
||||
|
||||
t.Cleanup(func() {
|
||||
config.QueueConfig, config.CacheConfig = prevQ, prevC
|
||||
sdk.Runtime = prevRuntime
|
||||
queueMu.Lock()
|
||||
installed, installedGen, drainRegistered = prevInstalled, prevGen, prevRegistered
|
||||
queueMu.Unlock()
|
||||
})
|
||||
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
queueMu.Lock()
|
||||
installed, drainRegistered = nil, false
|
||||
queueMu.Unlock()
|
||||
}
|
||||
|
||||
// setInstalled puts a fake where setupQueue would have left the real adapter.
|
||||
//
|
||||
// Legitimate because the callback reads `installed` when it runs rather than
|
||||
// capturing it at registration - that is the property that lets a reload
|
||||
// replace the adapter and still have the right one drained.
|
||||
func setInstalled(q interface{ Shutdown() }) {
|
||||
queueMu.Lock()
|
||||
installed = q
|
||||
queueMu.Unlock()
|
||||
}
|
||||
|
||||
func currentInstalled() interface{ Shutdown() } {
|
||||
queueMu.Lock()
|
||||
defer queueMu.Unlock()
|
||||
return installed
|
||||
}
|
||||
|
||||
// Issue #911: nothing shut the queue down at exit, so whatever was buffered
|
||||
// went with the process.
|
||||
//
|
||||
// This is the half that matters most - a callback is on BeforeExit and it
|
||||
// reaches the adapter this package installed. It says nothing about how many
|
||||
// times the callback was registered; see the test below for that.
|
||||
func TestSetupPutsTheQueueDrainOnBeforeExit(t *testing.T) {
|
||||
isolate(t)
|
||||
config.CacheConfig = &config.Cache{Memory: struct{}{}}
|
||||
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
|
||||
|
||||
Setup()
|
||||
Setup()
|
||||
Setup()
|
||||
|
||||
q := newCountingQueue()
|
||||
setInstalled(q)
|
||||
|
||||
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
|
||||
t.Fatalf("RunShutdown: %v", err)
|
||||
}
|
||||
|
||||
if got := q.calls.Load(); got != 1 {
|
||||
t.Errorf("Shutdown called %d times, want 1 - 0 means nothing registered the drain", got)
|
||||
}
|
||||
}
|
||||
|
||||
// Setup is re-run on every configuration change, so registering from it has to
|
||||
// be guarded: a callback per reload would leave the shutdown phase holding a
|
||||
// row of identical entries, each timed and each eligible to be named as the one
|
||||
// that overran the budget.
|
||||
//
|
||||
// Counted at the seam rather than through the effect. The test above cannot see
|
||||
// this - shutdownQueue takes the adapter on its first run, so the second and
|
||||
// third callbacks find nothing and return, and three registrations produce
|
||||
// exactly the same observable result as one. That is a good property of the
|
||||
// callback and a blind spot for any test that goes through it.
|
||||
func TestSetupRegistersTheDrainOncePerProcessHoweverManyReloads(t *testing.T) {
|
||||
isolate(t)
|
||||
|
||||
previous := setShutdown
|
||||
t.Cleanup(func() { setShutdown = previous })
|
||||
registrations := 0
|
||||
setShutdown = func(func(context.Context)) { registrations++ }
|
||||
|
||||
config.CacheConfig = &config.Cache{Memory: struct{}{}}
|
||||
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
|
||||
|
||||
Setup()
|
||||
Setup()
|
||||
Setup()
|
||||
|
||||
if registrations != 1 {
|
||||
t.Errorf("three reloads registered the drain %d times, want 1", registrations)
|
||||
}
|
||||
}
|
||||
|
||||
// The drain reaches the adapter that is current when the signal arrives, not
|
||||
// one captured while wiring up. A reload replaces the adapter, and draining the
|
||||
// one that was installed at start-up would drain something nobody has published
|
||||
// to since.
|
||||
func TestTheDrainRunsAgainstTheAdapterInstalledLast(t *testing.T) {
|
||||
isolate(t)
|
||||
config.CacheConfig = &config.Cache{Memory: struct{}{}}
|
||||
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
|
||||
Setup()
|
||||
|
||||
first, second := newCountingQueue(), newCountingQueue()
|
||||
setInstalled(first)
|
||||
setInstalled(second)
|
||||
|
||||
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
|
||||
t.Fatalf("RunShutdown: %v", err)
|
||||
}
|
||||
|
||||
if first.calls.Load() != 0 {
|
||||
t.Error("the adapter that was replaced was shut down; the callback captured it instead of " +
|
||||
"reading it when it ran")
|
||||
}
|
||||
if second.calls.Load() != 1 {
|
||||
t.Errorf("the current adapter was shut down %d times, want 1", second.calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
// Nothing installed is the shipped default: settings.yml has no queue section,
|
||||
// so setupQueue returns early and the runtime's own fallback queue is what
|
||||
// callers get. Shutting that down would close a queue this package neither
|
||||
// built nor started.
|
||||
func TestTheDrainDoesNothingWhenThisPackageInstalledNothing(t *testing.T) {
|
||||
isolate(t)
|
||||
config.CacheConfig = &config.Cache{Memory: struct{}{}}
|
||||
config.QueueConfig = &config.Queue{}
|
||||
Setup()
|
||||
|
||||
if currentInstalled() != nil {
|
||||
t.Fatal("an empty queue configuration installed an adapter, so this test asserts nothing")
|
||||
}
|
||||
if err := sdk.Runtime.RunShutdown(context.Background()); err != nil {
|
||||
t.Fatalf("RunShutdown: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// The budget bounds the wait, not the work. A consumer that never finishes must
|
||||
// not hold the process past its grace period - SIGKILL would arrive mid-write
|
||||
// instead of at a point of the process's choosing.
|
||||
func TestTheDrainStopsWaitingWhenTheBudgetIsGone(t *testing.T) {
|
||||
isolate(t)
|
||||
|
||||
blocked := newCountingQueue()
|
||||
blocked.block = make(chan struct{})
|
||||
t.Cleanup(func() { close(blocked.block) })
|
||||
setInstalled(blocked)
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
|
||||
defer cancel()
|
||||
|
||||
returned := make(chan struct{})
|
||||
go func() {
|
||||
defer close(returned)
|
||||
shutdownQueue(ctx)
|
||||
}()
|
||||
|
||||
select {
|
||||
case <-returned:
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("shutdownQueue did not return after its context expired - it waits on a Shutdown " +
|
||||
"that takes no context, so the wait has to be bounded here")
|
||||
}
|
||||
|
||||
// Waited for rather than read straight after the return: Shutdown runs on a
|
||||
// goroutine that the caller does not join, so the increment is not ordered
|
||||
// against the caller giving up on its deadline.
|
||||
blocked.waitStarted(t)
|
||||
if blocked.calls.Load() != 1 {
|
||||
t.Errorf("Shutdown called %d times, want 1 - the drain has to be attempted even when it "+
|
||||
"cannot be waited out", blocked.calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
// After the drain this package owns nothing. A reload arriving mid-shutdown
|
||||
// then builds a new adapter rather than being handed a closed one as its
|
||||
// `previous` to shut down again.
|
||||
func TestTheDrainGivesUpOwnershipOfTheAdapter(t *testing.T) {
|
||||
isolate(t)
|
||||
setInstalled(newCountingQueue())
|
||||
|
||||
shutdownQueue(context.Background())
|
||||
|
||||
if got := currentInstalled(); got != nil {
|
||||
t.Errorf("installed is %T after the drain, want nil", got)
|
||||
}
|
||||
}
|
||||
|
||||
// runtimeQueue is a full AdapterQueue, so it can be handed to the runtime
|
||||
// rather than only to this package's own record of what it installed.
|
||||
type runtimeQueue struct {
|
||||
countingQueue
|
||||
}
|
||||
|
||||
func (q *runtimeQueue) String() string { return "runtime-fake" }
|
||||
func (q *runtimeQueue) Append(corestorage.Messager) error { return nil }
|
||||
func (q *runtimeQueue) Register(string, corestorage.ConsumerFunc) {}
|
||||
func (q *runtimeQueue) Run() {}
|
||||
|
||||
// The drain must not reach a queue this package did not install.
|
||||
//
|
||||
// sdk.Runtime.GetQueueAdapter never returns nil: with no queue section
|
||||
// configured it wraps the runtime's own fallback, and the wrapper's Shutdown
|
||||
// forwards. Reaching for the accessor would therefore look like it worked and
|
||||
// would close a queue this package neither built nor started - the same shape
|
||||
// as the `if q := GetQueueAdapter(); q != nil` that setupQueue already had to
|
||||
// drop.
|
||||
//
|
||||
// The previous test's empty-configuration case cannot see this: it only checks
|
||||
// that RunShutdown returns, which it would either way.
|
||||
func TestTheDrainNeverReachesTheRuntimesOwnQueue(t *testing.T) {
|
||||
isolate(t)
|
||||
|
||||
onTheRuntime := &runtimeQueue{countingQueue: *newCountingQueue()}
|
||||
sdk.Runtime.SetQueueAdapter(onTheRuntime)
|
||||
|
||||
if currentInstalled() != nil {
|
||||
t.Fatal("this package installed something, so the distinction under test is not set up")
|
||||
}
|
||||
if sdk.Runtime.GetQueueAdapter() == nil {
|
||||
t.Fatal("the accessor returned nil, so it is no longer the trap this guards")
|
||||
}
|
||||
|
||||
shutdownQueue(context.Background())
|
||||
|
||||
if got := onTheRuntime.calls.Load(); got != 0 {
|
||||
t.Errorf("the runtime's queue was shut down %d times - the drain went through "+
|
||||
"GetQueueAdapter instead of the adapter this package installed", got)
|
||||
}
|
||||
}
|
||||
|
||||
// A drain that finishes in the same instant the budget expires counts as
|
||||
// finished.
|
||||
//
|
||||
// Both channels are ready when the select runs, and select picks at random
|
||||
// among ready cases, so a single look reports an overrun for a drain that
|
||||
// completed - roughly half the times it lands here. The repetition is what
|
||||
// makes that visible: one iteration passes either way.
|
||||
func TestATieBetweenTheDeadlineAndTheDrainGoesToTheDrain(t *testing.T) {
|
||||
expired, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
<-expired.Done()
|
||||
|
||||
done := make(chan struct{})
|
||||
close(done)
|
||||
|
||||
for i := 0; i < 1000; i++ {
|
||||
if !finishedBeforeDeadline(expired, done) {
|
||||
t.Fatalf("iteration %d of 1000: both the deadline and the drain were ready and the "+
|
||||
"deadline won - a drain that completed is being reported as an overrun", i)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The other side of it. A drain that really has not finished has to be
|
||||
// reported, or the warning never fires and the tie-break above has quietly
|
||||
// turned into "always say it drained".
|
||||
func TestADrainThatHasNotFinishedIsReportedAsAnOverrun(t *testing.T) {
|
||||
expired, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
<-expired.Done()
|
||||
|
||||
stillRunning := make(chan struct{}) // never closed
|
||||
|
||||
if finishedBeforeDeadline(expired, stillRunning) {
|
||||
t.Error("an unfinished drain was reported as having finished in time")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
|
||||
"github.com/go-admin-team/go-admin-core/v2/storage/queue"
|
||||
)
|
||||
|
||||
// redisAddrEnv points these tests at a server. They are skipped without it, so
|
||||
// a developer with no redis running still gets a green run - and CI sets it,
|
||||
// which is the point: the ordering rule they cover is invisible on the memory
|
||||
// backend, and memory is the default. A suite that only ever exercised the
|
||||
// default would report success for a queue that silently drops every consumer.
|
||||
const redisAddrEnv = "GO_ADMIN_TEST_REDIS_ADDR"
|
||||
|
||||
func redisAddr(t *testing.T) string {
|
||||
t.Helper()
|
||||
addr := os.Getenv(redisAddrEnv)
|
||||
if addr != "" {
|
||||
return addr
|
||||
}
|
||||
// Skipping locally is the point; skipping in CI is the failure this whole
|
||||
// file exists to prevent. A workflow that renamed the variable, or dropped
|
||||
// the service, would otherwise go green while these two tests quietly did
|
||||
// nothing - which is the same shape as the defect they cover.
|
||||
if os.Getenv("CI") != "" {
|
||||
t.Fatalf("%s is not set while CI is: the redis-backed queue tests must not skip here", redisAddrEnv)
|
||||
}
|
||||
t.Skipf("%s is not set; skipping the redis-backed queue tests", redisAddrEnv)
|
||||
return ""
|
||||
}
|
||||
|
||||
// newRedisQueue builds the queue the same way setupQueue does - through
|
||||
// config.QueueConfig.Setup - so that what is under test is the adapter this
|
||||
// repository actually gets, LegacyQueueAdapter and all, rather than a redis
|
||||
// client wired up by the test.
|
||||
func newRedisQueue(t *testing.T, prefix string) corestorage.AdapterQueue {
|
||||
t.Helper()
|
||||
previous := config.QueueConfig
|
||||
t.Cleanup(func() { config.QueueConfig = previous })
|
||||
|
||||
config.QueueConfig = &config.Queue{
|
||||
Redis: &config.RedisQueue{
|
||||
RedisOptions: config.RedisOptions{Addr: redisAddr(t)},
|
||||
Group: prefix,
|
||||
KeyPrefix: prefix,
|
||||
},
|
||||
}
|
||||
q, err := config.QueueConfig.Setup()
|
||||
if err != nil {
|
||||
t.Fatalf("queue setup: %v", err)
|
||||
}
|
||||
t.Cleanup(q.Shutdown)
|
||||
return q
|
||||
}
|
||||
|
||||
func message(t *testing.T, stream string) corestorage.Messager {
|
||||
t.Helper()
|
||||
m := &queue.Message{}
|
||||
m.SetStream(stream)
|
||||
m.SetValues(map[string]interface{}{"hello": "world"})
|
||||
return m
|
||||
}
|
||||
|
||||
// Registered first, then started: the consumer gets the message. This is the
|
||||
// order setupQueue and attachQueueConsumers now produce between them.
|
||||
func TestRedisQueueDeliversToAConsumerRegisteredBeforeTheStart(t *testing.T) {
|
||||
stream := "t-ordered"
|
||||
q := newRedisQueue(t, "gotest-ordered")
|
||||
|
||||
got := make(chan struct{}, 1)
|
||||
q.Register(stream, func(corestorage.Messager) error {
|
||||
select {
|
||||
case got <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
return nil
|
||||
})
|
||||
go q.Run()
|
||||
|
||||
// Give Start a moment to reach its read loop before publishing.
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
if err := q.Append(message(t, stream)); err != nil {
|
||||
t.Fatalf("append: %v", err)
|
||||
}
|
||||
|
||||
select {
|
||||
case <-got:
|
||||
case <-time.After(15 * time.Second):
|
||||
t.Fatal("the consumer never received the message")
|
||||
}
|
||||
}
|
||||
|
||||
// Started first, then registered: the registration is refused and every
|
||||
// publish afterwards fails.
|
||||
//
|
||||
// Subscribe answers ErrQueueAlreadyStarted, and LegacyQueueAdapter.Register
|
||||
// returns nothing, so the caller cannot know - that part is silent. What is not
|
||||
// silent is the consequence: no consumer group was created, so Publish refuses
|
||||
// the topic with ErrNoHandler on every single request, and go-admin's call
|
||||
// sites log that at error level while the login and operation log rows are
|
||||
// never written.
|
||||
//
|
||||
// This is the test the memory backend cannot provide. queue.Memory's Register
|
||||
// starts another consumer goroutine whatever the state, so the same code passes
|
||||
// there - which is how the defect survived, memory being the default.
|
||||
func TestRedisQueueRefusesAConsumerRegisteredAfterTheStart(t *testing.T) {
|
||||
stream := "t-late"
|
||||
q := newRedisQueue(t, "gotest-late")
|
||||
|
||||
go q.Run()
|
||||
time.Sleep(500 * time.Millisecond)
|
||||
|
||||
q.Register(stream, func(corestorage.Messager) error { return nil })
|
||||
|
||||
err := q.Append(message(t, stream))
|
||||
if err == nil {
|
||||
t.Fatal("a message was accepted for a topic whose registration came after Start; " +
|
||||
"if the backend now accepts late registration, the ordering rule in setupQueue can be revisited")
|
||||
}
|
||||
if !errors.Is(err, corestorage.ErrNoHandler) {
|
||||
t.Fatalf("append failed with %v, want %v - the test is meant to pin the "+
|
||||
"missing-consumer path, not any error at all", err, corestorage.ErrNoHandler)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
|
||||
"github.com/go-admin-team/go-admin-core/v2/storage/queue"
|
||||
)
|
||||
|
||||
// sampleSize is how many publishes have to land inside the reload before the
|
||||
// measurement is taken. Waiting on the count rather than on wall clock keeps
|
||||
// the window the test covers the same on a loaded runner as on an idle one.
|
||||
const sampleSize = 200
|
||||
|
||||
func swapMsg() corestorage.Messager {
|
||||
m := new(queue.Message)
|
||||
m.SetStream("t")
|
||||
m.SetValues(map[string]interface{}{"a": "b"})
|
||||
return m
|
||||
}
|
||||
|
||||
// A reload must never leave producers holding a queue that has stopped
|
||||
// accepting.
|
||||
//
|
||||
// Shutdown waits for its consumers to deliver what the queue still holds. Taking
|
||||
// the old adapter down before installing the new one meant the runtime pointed
|
||||
// at a closed queue for that entire wait: every Append in the window came back
|
||||
// ErrQueueClosed, and both call sites in common/middleware log it at error
|
||||
// level. Installing first leaves no window - a producer gets the new queue or
|
||||
// the old one, and both accept.
|
||||
//
|
||||
// The difference is only visible during that wait, which is why the test holds
|
||||
// a consumer rather than checking the state after Setup has returned: by then
|
||||
// the two orders look identical.
|
||||
//
|
||||
// One refusal survives the fix and is not something this ordering can reach.
|
||||
// GetQueuePrefix hands back a wrapper that captured the adapter, so a producer
|
||||
// that fetched before the swap and appends after Shutdown has begun is still
|
||||
// holding the old one. That window is one call wide and closing it means
|
||||
// resolving the adapter inside Append, which is core's to change. What the
|
||||
// ordering removes is the sustained window: every producer that fetches during
|
||||
// the wait. The test publishes from a single goroutine, so at most one of its
|
||||
// calls can straddle the swap - which is what makes "more than one" the line
|
||||
// between the two orders rather than a tolerance.
|
||||
func TestAReloadNeverPointsProducersAtAClosedQueue(t *testing.T) {
|
||||
prevQ, prevC := config.QueueConfig, config.CacheConfig
|
||||
prevRuntime := sdk.Runtime
|
||||
prevInstalled, prevGen := installed, installedGen
|
||||
t.Cleanup(func() {
|
||||
config.QueueConfig, config.CacheConfig = prevQ, prevC
|
||||
sdk.Runtime = prevRuntime
|
||||
queueMu.Lock()
|
||||
installed, installedGen = prevInstalled, prevGen
|
||||
queueMu.Unlock()
|
||||
})
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
config.CacheConfig = &config.Cache{Memory: struct{}{}}
|
||||
// Sized so the buffer cannot fill while the consumer is held: a full queue
|
||||
// returns an error of its own, and this test needs every error other than
|
||||
// ErrQueueClosed to mean something it does not model has happened.
|
||||
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 4096}}
|
||||
|
||||
Setup()
|
||||
|
||||
// A consumer that will not finish until this test lets it, so the reload's
|
||||
// Shutdown has something to wait for.
|
||||
release := make(chan struct{})
|
||||
consuming := make(chan struct{})
|
||||
var picked sync.Once
|
||||
first := sdk.Runtime.GetQueuePrefix("")
|
||||
first.Register("t", func(corestorage.Messager) error {
|
||||
picked.Do(func() { close(consuming) })
|
||||
<-release
|
||||
return nil
|
||||
})
|
||||
go first.Run()
|
||||
for i := 0; i < 4; i++ {
|
||||
if err := first.Append(swapMsg()); err != nil {
|
||||
t.Fatalf("seed append %d: %v", i, err)
|
||||
}
|
||||
}
|
||||
select {
|
||||
case <-consuming:
|
||||
case <-time.After(10 * time.Second):
|
||||
t.Fatal("the consumer never picked a message up, so the reload has nothing to wait for")
|
||||
}
|
||||
|
||||
reloaded := make(chan struct{})
|
||||
go func() { Setup(); close(reloaded) }()
|
||||
|
||||
// Publish continuously while the reload is in progress.
|
||||
var refused atomic.Int64
|
||||
var attempts atomic.Int64
|
||||
unexpected := make(chan error, 1)
|
||||
stop := make(chan struct{})
|
||||
// publishing is closed by the producer on its way out. The test joins on it
|
||||
// before returning: t.Cleanup restores sdk.Runtime, and a producer still in
|
||||
// flight would be reading the variable that restore writes.
|
||||
publishing := make(chan struct{})
|
||||
go func() {
|
||||
defer close(publishing)
|
||||
for {
|
||||
select {
|
||||
case <-stop:
|
||||
return
|
||||
default:
|
||||
}
|
||||
attempts.Add(1)
|
||||
err := sdk.Runtime.GetQueuePrefix("").Append(swapMsg())
|
||||
switch {
|
||||
case err == nil:
|
||||
case errors.Is(err, corestorage.ErrQueueClosed):
|
||||
refused.Add(1)
|
||||
default:
|
||||
// Kept rather than counted: an Append refused for some other
|
||||
// reason would otherwise leave refused at zero and the test
|
||||
// green while nothing was reaching a queue at all.
|
||||
select {
|
||||
case unexpected <- err:
|
||||
default:
|
||||
}
|
||||
}
|
||||
time.Sleep(time.Millisecond)
|
||||
}
|
||||
}()
|
||||
|
||||
deadline := time.After(30 * time.Second)
|
||||
for attempts.Load() < sampleSize {
|
||||
select {
|
||||
case <-deadline:
|
||||
t.Fatalf("only %d publishes landed inside the reload; the window was never sampled", attempts.Load())
|
||||
case <-time.After(time.Millisecond):
|
||||
}
|
||||
}
|
||||
close(release)
|
||||
|
||||
select {
|
||||
case <-reloaded:
|
||||
case <-time.After(30 * time.Second):
|
||||
t.Fatal("the reload never finished")
|
||||
}
|
||||
close(stop)
|
||||
<-publishing
|
||||
|
||||
select {
|
||||
case err := <-unexpected:
|
||||
t.Fatalf("a publish failed for a reason this test does not model: %v", err)
|
||||
default:
|
||||
}
|
||||
if n := refused.Load(); n > 1 {
|
||||
t.Errorf("%d of %d publishes during the reload were refused: producers were pointed at the closed queue",
|
||||
n, attempts.Load())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/config"
|
||||
"github.com/go-admin-team/go-admin-core/v2/sdk/runtime"
|
||||
)
|
||||
|
||||
// Issue #892: a configuration reload replaces the queue adapter and the
|
||||
// consumers registered against the previous one are attached to a queue nobody
|
||||
// publishes to any more.
|
||||
//
|
||||
// The fix has two halves. attachQueueConsumers gives a new queue its own
|
||||
// consumers and the same queue none, which cmd/api covers against a queue the
|
||||
// test controls. This is the other half: that a reload actually produces a new
|
||||
// queue for it to notice. Setup is what config re-runs on every change, so
|
||||
// calling it twice is what a reload does to this package.
|
||||
func TestSetupBumpsTheQueueGenerationOnEveryReload(t *testing.T) {
|
||||
// Setup writes the process-wide sdk.Runtime - the cache and queue adapters -
|
||||
// and this package's own record of what it installed. Restoring all of it
|
||||
// keeps the test from deciding what a later test in this binary sees,
|
||||
// which is the same isolation cmd/api's freshRuntime provides.
|
||||
prevQ, prevC := config.QueueConfig, config.CacheConfig
|
||||
prevRuntime := sdk.Runtime
|
||||
prevInstalled, prevGen := installed, installedGen
|
||||
t.Cleanup(func() {
|
||||
config.QueueConfig, config.CacheConfig = prevQ, prevC
|
||||
sdk.Runtime = prevRuntime
|
||||
queueMu.Lock()
|
||||
installed, installedGen = prevInstalled, prevGen
|
||||
queueMu.Unlock()
|
||||
})
|
||||
sdk.Runtime = runtime.NewConfig()
|
||||
|
||||
config.CacheConfig = &config.Cache{Memory: struct{}{}}
|
||||
config.QueueConfig = &config.Queue{Memory: &config.QueueMemory{PoolSize: 10}}
|
||||
|
||||
before := QueueGeneration()
|
||||
Setup()
|
||||
first := QueueGeneration()
|
||||
Setup()
|
||||
second := QueueGeneration()
|
||||
|
||||
t.Logf("before=%d first=%d second=%d", before, first, second)
|
||||
if first == before {
|
||||
t.Fatal("the first Setup did not install a queue")
|
||||
}
|
||||
if second == first {
|
||||
t.Fatal("a second Setup - which is what a configuration reload does - did not install a new one")
|
||||
}
|
||||
}
|
||||
@@ -336,6 +336,6 @@ INSERT INTO sys_post (post_id, post_name, post_code, sort, status, remark, creat
|
||||
(2, '首席技术执行官', 'CTO', 2, '2','首席技术执行官', 1, 1,'2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL),
|
||||
(3, '首席运营官', 'COO', 3, '2','测试工程师', 1, 1,'2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
INSERT INTO sys_role (role_id, role_name, status, role_key, role_sort, flag, remark, admin, data_scope, create_by, update_by, created_at, updated_at, deleted_at)VALUES
|
||||
(1, '系统管理员', '2', 'admin', 1, '', '', 1, '', 1, 1, '2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
(1, '系统管理员', '2', 'admin', 1, '', '', 1, '1', 1, 1, '2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
INSERT INTO sys_user VALUES (1, 'admin', '$2a$10$/Glr4g9Svr6O0kvjsRJCXu3f0W8/dsP3XZyVNi1019ratWpSPMyw.', 'zhangwj', '13818888888', 1, '', '', '1', '1@qq.com', 1, 1, '', '2', 1, 1, '2021-05-13 19:56:37.914', '2021-05-13 19:56:40.205', NULL);
|
||||
-- 数据完成 ;
|
||||
+1
-1
@@ -318,6 +318,6 @@ INSERT INTO sys_menu_api_rule VALUES (46, 156);
|
||||
INSERT INTO sys_post VALUES (1, '首席执行官', 'CEO', 0, '2','首席执行官', 1, 1, '2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
INSERT INTO sys_post VALUES (2, '首席技术执行官', 'CTO', 2, '2','首席技术执行官', 1, 1,'2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
INSERT INTO sys_post VALUES (3, '首席运营官', 'COO', 3, '2','测试工程师', 1, 1,'2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
INSERT INTO sys_role VALUES (1, '系统管理员', '2', 'admin', 1, '', '', true, '', 1, 1, '2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
INSERT INTO sys_role VALUES (1, '系统管理员', '2', 'admin', 1, '', '', true, '1', 1, 1, '2021-05-13 19:56:37.913', '2021-05-13 19:56:37.913', NULL);
|
||||
INSERT INTO sys_user VALUES (1, 'admin', '$2a$10$/Glr4g9Svr6O0kvjsRJCXu3f0W8/dsP3XZyVNi1019ratWpSPMyw.', 'zhangwj', '13818888888', 1, '', '', '1', '1@qq.com', 1, 1, '', '2', 1, 1, '2021-05-13 19:56:37.914', '2021-05-13 19:56:40.205', NULL);
|
||||
-- 数据完成 ;
|
||||
@@ -1,5 +1,10 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var ExtConfig Extend
|
||||
|
||||
// Extend 扩展配置
|
||||
@@ -12,6 +17,40 @@ var ExtConfig Extend
|
||||
type Extend struct {
|
||||
AMap AMap // 这里配置对应配置文件的结构即可
|
||||
FileStore FileStore
|
||||
RateLimit RateLimit
|
||||
Shutdown Shutdown
|
||||
}
|
||||
|
||||
// DefaultInboundQPS is the limit applied when nothing is configured. It is the
|
||||
// value that used to be hard-coded in the middleware, so an existing deployment
|
||||
// that adds nothing to settings.yml keeps the behaviour it already had.
|
||||
const DefaultInboundQPS = 200
|
||||
|
||||
// RateLimit 全局入站限流。
|
||||
//
|
||||
// extend:
|
||||
// ratelimit:
|
||||
// inboundqps: 200 # 每秒入站请求上限;填 0 关闭限流
|
||||
//
|
||||
// The threshold used to live in common/middleware/sentinel.go as a constant,
|
||||
// which made 200 QPS the ceiling of every deployment with nothing in the
|
||||
// configuration to reveal it.
|
||||
type RateLimit struct {
|
||||
// InboundQPS caps inbound requests per second across the process.
|
||||
//
|
||||
// Absent means DefaultInboundQPS, zero disables the limiter, and a positive
|
||||
// value is the threshold. The pointer is what separates "not configured"
|
||||
// from "configured to zero" - the two need different answers and a plain
|
||||
// float64 cannot tell them apart.
|
||||
InboundQPS *float64
|
||||
}
|
||||
|
||||
// Threshold reports the limit to apply. Zero means no limiting.
|
||||
func (r RateLimit) Threshold() float64 {
|
||||
if r.InboundQPS == nil {
|
||||
return DefaultInboundQPS
|
||||
}
|
||||
return *r.InboundQPS
|
||||
}
|
||||
|
||||
type AMap struct {
|
||||
@@ -39,3 +78,133 @@ type ObjectStore struct {
|
||||
func (o ObjectStore) Configured() bool {
|
||||
return o.Endpoint != "" && o.AccessKeyID != "" && o.AccessKeySecret != "" && o.BucketName != ""
|
||||
}
|
||||
|
||||
// Default budgets for a graceful shutdown, in seconds. Each applies to the
|
||||
// matching field of extend.shutdown when that field is absent, and together
|
||||
// they are what the process spent before the section existed - so a deployment
|
||||
// that configures nothing keeps the shutdown it already had.
|
||||
//
|
||||
// The drain default is zero deliberately. The three budgets are spent one
|
||||
// after the other, and once their sum reaches the orchestrator's stop grace
|
||||
// period the process is killed part-way through its cleanup callbacks, which
|
||||
// is worse than not draining at all. `docker stop` allows ten seconds by
|
||||
// default and 5+3 already leaves little room, so a non-zero default here would
|
||||
// slow down every existing shutdown to buy something only a load balancer that
|
||||
// polls /ready can collect.
|
||||
const (
|
||||
DefaultDrainSeconds = 0
|
||||
DefaultServerSeconds = 5
|
||||
DefaultCleanupSeconds = 3
|
||||
)
|
||||
|
||||
// Shutdown is how long a graceful shutdown may spend, stage by stage.
|
||||
//
|
||||
// extend:
|
||||
// shutdown:
|
||||
// drain: 0
|
||||
// server: 5
|
||||
// cleanup: 3
|
||||
// grace: 30
|
||||
//
|
||||
// Every field is a pointer for the reason RateLimit.InboundQPS is: nil means
|
||||
// "not configured" and takes the default, while a value that was written down
|
||||
// is taken literally, zero included. Without that separation `server: 0` - do
|
||||
// not wait for in-flight requests, which is a reasonable thing to ask under a
|
||||
// very short grace period - could not be said at all, and `drain: 0` would
|
||||
// have to mean something different from `server: 0` in the same section.
|
||||
type Shutdown struct {
|
||||
// Drain is how long to keep serving normally after a stop signal arrives.
|
||||
// Throughout it /ready answers 503 and keep-alive is switched off, which
|
||||
// is what gives whatever routes traffic here time to stop routing it
|
||||
// before the listener closes. Zero is no window: the readiness flip and
|
||||
// the listener closing are then microseconds apart and nothing observes
|
||||
// the first.
|
||||
//
|
||||
// What the window is worth depends on who does the removing and on what
|
||||
// basis; the package comment in common/health has the two cases, and they
|
||||
// do not want the same value.
|
||||
Drain *int
|
||||
// Server is how long the server waits for in-flight requests once the
|
||||
// listener is closed.
|
||||
Server *int
|
||||
// Cleanup is how long the BeforeExit callbacks get after that.
|
||||
Cleanup *int
|
||||
// Grace is the stop grace period the orchestrator gives this process -
|
||||
// `docker stop --timeout`, or terminationGracePeriodSeconds. Nothing reads
|
||||
// it during a shutdown; it exists so start-up can say whether the budget
|
||||
// fits inside it. Absent means no comparison is made, because the
|
||||
// reference values differ threefold between runtimes and a fixed threshold
|
||||
// would warn about configurations that are correct.
|
||||
Grace *int
|
||||
}
|
||||
|
||||
// ShutdownBudget is what a shutdown will actually spend, in seconds, after the
|
||||
// fallbacks have been applied.
|
||||
type ShutdownBudget struct {
|
||||
Drain int
|
||||
Server int
|
||||
Cleanup int
|
||||
// Grace is zero when extend.shutdown.grace was not configured.
|
||||
Grace int
|
||||
}
|
||||
|
||||
// Budget resolves the configured section into the values that will be spent.
|
||||
//
|
||||
// A negative is refused rather than corrected. A wait cannot be negative, so
|
||||
// there is no reading of one to honour, and quietly turning it into zero would
|
||||
// be the failure this whole section exists to remove: written down, accepted,
|
||||
// and not what happens. It is returned as an error rather than reported here
|
||||
// so that the rule can be checked without ending the process.
|
||||
func (s Shutdown) Budget() (ShutdownBudget, error) {
|
||||
var negative []string
|
||||
for _, f := range []struct {
|
||||
name string
|
||||
value *int
|
||||
}{
|
||||
{"drain", s.Drain},
|
||||
{"server", s.Server},
|
||||
{"cleanup", s.Cleanup},
|
||||
{"grace", s.Grace},
|
||||
} {
|
||||
if f.value != nil && *f.value < 0 {
|
||||
negative = append(negative, fmt.Sprintf("%s: %d", f.name, *f.value))
|
||||
}
|
||||
}
|
||||
if len(negative) > 0 {
|
||||
return ShutdownBudget{}, fmt.Errorf(
|
||||
"extend.shutdown was given a negative number of seconds (%s); "+
|
||||
"a wait cannot be negative, and 0 is how to say \"do not wait\"",
|
||||
strings.Join(negative, ", "))
|
||||
}
|
||||
|
||||
return ShutdownBudget{
|
||||
Drain: budgetSeconds(s.Drain, DefaultDrainSeconds),
|
||||
Server: budgetSeconds(s.Server, DefaultServerSeconds),
|
||||
Cleanup: budgetSeconds(s.Cleanup, DefaultCleanupSeconds),
|
||||
Grace: budgetSeconds(s.Grace, 0),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func budgetSeconds(configured *int, fallback int) int {
|
||||
if configured != nil {
|
||||
return *configured
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
// Total is the whole of the shutdown, since the three stages run one after the
|
||||
// other.
|
||||
func (b ShutdownBudget) Total() int { return b.Drain + b.Server + b.Cleanup }
|
||||
|
||||
// Overrun reports how many seconds have to be found for the budget to fit
|
||||
// inside the configured grace period. It is zero when no grace period was
|
||||
// configured and when the budget already fits.
|
||||
//
|
||||
// Fitting means strictly less: the grace period is when SIGKILL is sent, so a
|
||||
// budget that ends exactly then leaves the last callback no time to return.
|
||||
func (b ShutdownBudget) Overrun() int {
|
||||
if b.Grace <= 0 || b.Total() < b.Grace {
|
||||
return 0
|
||||
}
|
||||
return b.Total() - b.Grace + 1
|
||||
}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user