Compare commits

..
114 Commits
Author SHA1 Message Date
Wu Clan bc484bf414 Merge branch 'master' into pre-tenant
# Conflicts:
#	backend/common/context.py
2026-08-14 17:49:39 +08:00
Wu Clan 7568275126 Add multiple database data source config (#1232)
* Add multiple database data source config

* Fix comments

* Fix deploy env
2026-08-14 17:44:45 +08:00
Wu Clan 67a6a58e57 Bump dependencies and pre-commits (#1231) 2026-08-13 11:09:42 +08:00
Wu Clan 8e5c379853 Bump dependencies and pre-commits (#1228)
* Bump dependencies and pre-commits

* Fix lint
2026-07-21 15:19:20 +08:00
Wu Clan 2dffe8f6a8 Update the changelog for v1.15.0 (#1226) 2026-07-10 18:01:52 +08:00
Wu Clan c802f67a5a Update the version number to 1.15.0 (#1225) 2026-07-10 17:58:49 +08:00
Wu Clan 1916261795 Update plugin config format validation (#1224) 2026-07-10 17:55:24 +08:00
Wu Clan 325016adc8 Update xdb database file to release (#1223) 2026-07-10 09:53:36 +08:00
Wu Clan 09fd380c3a Refactor plugin dynamic config loading (#1222) 2026-07-09 17:20:01 +08:00
IAseven e6292747bd Fix duplicate log reporting in OTEL (#1221) 2026-07-08 15:57:43 +08:00
Wu Clan a96b2cfad1 Update document link addresses in the CLI (#1220) 2026-07-08 13:55:22 +08:00
Wu Clan a0a1bde88c Update capability plugin config validation (#1219) 2026-07-08 11:48:51 +08:00
9251a5d5fd Add support for capability plugins (#1217)
* feat(plugin): support capability plugins

Add capability plugin metadata and parsing so plugins without app/api sections can participate in dependency ordering and hooks without being routed as app plugins.

Also rebuild missing plugin status cache from the normalized plugin config when a route dependency checks plugin state.

* refactor code

---------

Co-authored-by: linwu@onemt.com.cn <linwu@onemt.com.cn>
Co-authored-by: Wu Clan <jianhengwu0407@gmail.com>
2026-07-08 11:19:20 +08:00
Wu Clan e3d2c68592 Bump dependencies and pre-commits (#1218) 2026-07-06 14:46:29 +08:00
Wu Clan 4d6970c6cc Add is superuser protocol to ctx (#1216) 2026-07-03 13:40:58 +08:00
Wu Clan d4f78d6448 Fix JWT auth dependency error propagation (#1215) 2026-07-02 16:20:26 +08:00
Wu Clan efec3308df Add underlying security config for JWT (#1214) 2026-07-02 14:42:02 +08:00
Wu Clan df045ac9f0 Fix merge errors 2026-07-01 00:07:34 +08:00
Wu Clan 70159d4615 Merge branch 'master' into pre-tenant
# Conflicts:
#	backend/app/admin/service/auth_service.py
#	backend/plugin/core.py
2026-06-30 23:37:51 +08:00
Wu Clan 0441907b00 Fix redis rate limiter bucket routing (#1212)
* Fix redis rate limiter bucket routing

* Add rate limiter function docstrings

* Optimize Redis rate limiter time source
2026-06-29 14:06:57 +08:00
Wu Clan 08ee3878c1 Refactor cache key and prefix APIs (#1210) 2026-06-25 14:19:37 +08:00
Wu Clan c7e686588e Add AI dynamic configuration menu (#1209) 2026-06-24 11:42:53 +08:00
Wu Clan 495046c978 Fix redis pubsub socket timeout (#1206) 2026-06-20 23:58:29 +08:00
Wu Clan ffaa1282cc Bump dependencies and pre-commits (#1205) 2026-06-19 22:47:33 +08:00
Wu Clan 15dda9ab6e Refactor AI response regeneration 2026-06-16 15:10:38 +08:00
Wu Clan 814068a9e6 Add plugin deps command to the CLI (#1200)
* Add plugin deps command to the CLI

* Add dependency sync command to the CLI

* Fix plugin dependency index fallback
2026-06-16 11:25:53 +08:00
IAseven 39cdfdfb00 Fix user salt column LargeBinary length (#1197) 2026-06-03 17:11:32 +08:00
Wu Clan 4b730099cd Optimize plugin core and add otel hooks (#1196) 2026-06-01 18:15:59 +08:00
Wu Clan 974972b490 Add celery explicit idempotent protection (#1195) 2026-06-01 18:15:31 +08:00
Wu Clan bffc564dd7 Fix tenant context 2026-06-01 02:08:50 +08:00
Wu Clan 4722dceff0 Fix lint 2026-05-31 16:23:50 +08:00
Wu Clan 4eded4f55a Merge branch 'master' into pre-tenant
# Conflicts:
#	backend/app/admin/service/auth_service.py
#	backend/common/security/jwt.py
#	backend/core/registrar.py
#	backend/middleware/access_middleware.py
#	backend/plugin/oauth2/api/v1/github.py
#	backend/plugin/oauth2/api/v1/google.py
#	backend/plugin/oauth2/crud/crud_user_social.py
#	backend/plugin/oauth2/service/user_social_service.py
#	backend/utils/dynamic_config.py
2026-05-31 16:20:49 +08:00
Wu Clan c9b4678148 Update the changelog for v1.14.0 (#1194) 2026-05-30 16:04:53 +08:00
Wu Clan d20bc91465 Update the version number to 1.14.0 (#1193) 2026-05-30 16:02:03 +08:00
Wu Clan bc8d1ab9e3 Fix task scheduler unpack fields (#1192) 2026-05-30 15:54:12 +08:00
Wu Clan a5e9a0e805 Refactor physical deletion to logical deletion (#1191)
* Refactor physical deletion to logical deletion

* Update the usage of unique indexes

* Update scheduler update and delete

* Fix lint

* Add some missing logic

* Optimize codegen get by id
2026-05-30 15:02:00 +08:00
Wu Clan 1897b3198e Optimize the usage of setex for redis (#1190) 2026-05-29 11:14:47 +08:00
Wu Clan 6b9c7d81de Bump dependencies and pre-commits (#1189) 2026-05-29 11:00:32 +08:00
Wu Clan e9b83d3c93 Optimize Grafana observability metrics and config (#1188)
* Optimize Grafana observability metrics and config

* Update docker images
2026-05-29 09:41:50 +08:00
Wu Clan 0f0e94c9b1 Add opera log task cancel to lifespan (#1187) 2026-05-28 11:44:09 +08:00
Wu Clan ba0f30cde1 Optimize operation log truncation handling (#1186) 2026-05-28 11:13:37 +08:00
Wu Clan 9f3673d812 Optimize user session logging and deletion (#1185) 2026-05-28 10:25:43 +08:00
Wu Clan ef9a6b554c Fix cache invalidation decorator usage (#1182) 2026-05-26 16:14:47 +08:00
Wu Clan dbfdc6762e Optimize SQL script execution in the CLI (#1181) 2026-05-25 11:06:06 +08:00
Wu Clan 2e86d79150 Optimize some global variable definitions (#1178) 2026-05-15 11:55:22 +08:00
Wu Clan 5352f98c13 Optimize database operations within loops (#1177)
* Optimize database operations within loops

* Optimize opera log
2026-05-14 11:38:38 +08:00
Wu Clan 6011b3aea2 Merge branch 'master' into pre-tenant 2026-05-07 13:43:14 +08:00
Wu Clan e32b4232c5 Update xdb dependency and database file (#1171) 2026-05-07 13:42:31 +08:00
Wu Clan 8d03e61995 Fix user tenant binding judgment 2026-05-07 10:45:54 +08:00
Wu Clan 0731f38702 Update the changelog for v1.13.4 (#1170) 2026-04-28 18:07:19 +08:00
Wu Clan f7051d4f9d Merge branch 'master' into pre-tenant 2026-04-28 18:04:53 +08:00
Wu Clan 7ebcdda26f Update the version number to 1.13.4 (#1169) 2026-04-28 18:03:05 +08:00
Wu Clan f32f7e56a2 Bump dependencies and pre-commits (#1168) 2026-04-28 17:57:14 +08:00
Wu Clan 330639d975 Fix typing warnings for return methods (#1167) 2026-04-28 16:42:23 +08:00
Wu Clan 0fa5468483 Fix multi level cache storage logic (#1166) 2026-04-27 17:54:09 +08:00
Wu Clan bfe4aa91ce Add uninstall plugin operation checks (#1165)
* Add uninstall plugin operation checks

* Fix lint
2026-04-27 17:36:00 +08:00
Wu Clan 5d85bc98d2 Optimize the core code ordering of plugins (#1164) 2026-04-27 09:51:18 +08:00
Wu Clan 8f90f764b6 Fix the add plugin CLI command config (#1163) 2026-04-26 23:18:40 +08:00
Wu Clan 49d12f7825 Add the snowflake ID enabled config (#1162) 2026-04-26 21:56:28 +08:00
Wu Clan b60c9ab17c Fix Grafana tempo and observability config (#1161)
* Add database and queue observability

* Update prometheus

* Delete the wrong changes

* Fix Grafana tempo and observability config
2026-04-26 13:09:47 +08:00
Wu Clan 576b2fc5d0 Add database and queue observability (#1160)
* Add database and queue observability

* Update prometheus

* Delete the wrong changes
2026-04-26 13:07:03 +08:00
Wu Clan 35a368ae04 Update Grafana dashboard and datasource config (#1159) 2026-04-21 18:02:37 +08:00
Wu Clan 776482c5d6 Update plugin README content format (#1158)
* Update plugin README content format

* Fix lint
2026-04-19 21:36:57 +08:00
Toby WongandWu Clan 92f182cd09 Add plugin depends on and lifecycle ordering (#1153)
* feat: add priority support to LifespanManager.register

* feat: support lifespan stages and plugin dependencies

* Optimize implementations

* Fix dependency resolution

* Fix the depends on type

---------

Co-authored-by: Wu Clan <jianhengwu0407@gmail.com>
2026-04-17 16:54:48 +08:00
Wu Clan 0428a2b07d Bump dependencies and pre-commits (#1156) 2026-04-14 23:58:51 +08:00
Wu Clan 9f193c7e7c Fix mysql enum data type mismatch (#1155) 2026-04-14 23:03:15 +08:00
Wu Clan 4e0ca6a214 Remove cleanup operations after plugin install (#1154)
* Remove cleanup operations after plugin install

* Fix lint
2026-04-13 15:09:45 +08:00
IAseven 45842d917a Fix app pytest client fixture scope (#1152)
* 🐞 fix:修复测试模块client scope=module时异常

* 🌈 style:

* 🌈 style:
2026-04-12 23:43:37 +08:00
Wu Clan 8719c63424 Update the architecture description in README (#1150) 2026-04-08 21:53:10 +08:00
Wu Clan d71db7d58d Update the changelog for v1.13.3 (#1149) 2026-04-08 21:05:52 +08:00
Wu Clan 1f287bbb0c Merge branch 'master' into pre-tenant
# Conflicts:
#	.pre-commit-config.yaml
#	pyproject.toml
#	requirements.txt
#	uv.lock
2026-04-08 21:04:56 +08:00
Wu Clan 02b5e670cc Update the version number to 1.13.3 (#1148) 2026-04-08 20:45:21 +08:00
Wu Clan 41f53d7225 Bump dependencies and pre-commits (#1147) 2026-04-08 20:44:59 +08:00
YuWorm b71179cea9 Add plugin system hook function mechanism (#1142)
* Enhance plugin system with lifespan registry, setup hooks, and patching utilities

* fix: remove registrar.py unused-import

* Optimize implementations

* Update function docs

* update replace_middleware func docs

* Recover deleted comments
2026-04-08 10:35:02 +08:00
Wu Clan 41a64bf64c Add superpowers to gitignore file (#1146) 2026-04-08 10:34:06 +08:00
Wu Clan afacc84da3 Fix custom email model return type (#1145) 2026-04-08 10:33:46 +08:00
Wu Clan 3bbe460beb Update custom reload filter call rules (#1144) 2026-04-08 09:10:19 +08:00
Wu Clan 131bfcf063 Add the installation frontend plugin CLI (#1141) 2026-04-03 09:51:19 +08:00
Wu Clan 283d762141 Refactor plugin dependency cache and check (#1140) 2026-04-02 17:00:20 +08:00
Wu Clan bd9737cc6e Fix the return of registered task results (#1139) 2026-04-02 13:56:29 +08:00
Wu Clan 6d381dba74 Fix current superuser permission validation (#1138) 2026-04-02 12:38:21 +08:00
Wu Clan 7e6856a5e0 Fix plugin dependencies for redis instance (#1133) 2026-03-30 14:37:31 +08:00
Wu Clan c96181bd08 Update plugin config and dependency cache (#1130)
* Update plugin config and dependency cache

* Fix deps check
2026-03-27 23:50:58 +08:00
Wu Clan 7a934fee32 Update the plugin requirements check (#1129) 2026-03-27 22:42:30 +08:00
Wu Clan 5bec0e083b Add custom cursor pagination support (#1128)
* Add custom cursor pagination support

* Update fastapi pagination dependency
2026-03-25 20:46:22 +08:00
Wu Clan a97dab1716 Update plugins and monitoring permissions (#1126) 2026-03-24 15:59:22 +08:00
Wu Clan 5ff7f4bc69 Fix code generation menu sql scripts (#1127) 2026-03-24 15:59:06 +08:00
Wu Clan 94f643c587 Fix the logic related to the status field (#1125) 2026-03-24 11:43:09 +08:00
Wu Clan c98ea57a94 Update the changelog for v1.13.2 (#1124) 2026-03-23 16:58:54 +08:00
Wu Clan 05e00b287b Fix deps 2026-03-23 16:49:31 +08:00
Wu Clan 4a08938de7 Merge branch 'master' into pre-tenant
# Conflicts:
#	uv.lock
2026-03-23 16:48:15 +08:00
Wu Clan 6c079e41cc Fix lint 2026-03-19 13:08:18 +08:00
Wu Clan 505ae9e59a Merge branch 'master' into pre-tenant 2026-03-19 13:03:55 +08:00
Wu Clan 3a0dcb8e9f Fix deps and pre-commits 2026-03-19 13:03:15 +08:00
Wu Clan a5cc8ac31b Fix log config 2026-03-19 12:13:25 +08:00
Wu Clan 96e0635f84 Update get_by_sid query 2026-03-19 12:05:15 +08:00
Wu Clan 15bea9e149 Merge branch 'master' into pre-tenant
# Conflicts:
#	backend/app/admin/crud/crud_user.py
#	backend/app/admin/service/auth_service.py
#	backend/cli.py
#	backend/common/dataclasses.py
#	backend/common/security/jwt.py
#	backend/common/security/rbac.py
#	backend/middleware/access_middleware.py
#	backend/plugin/core.py
#	backend/utils/dynamic_config.py
#	backend/utils/dynamic_import.py
2026-03-19 11:58:45 +08:00
Wu Clan ee00470ced Fix socket auth 2026-03-15 23:11:02 +08:00
Wu Clan fbbf6480ee Fix lint 2026-03-15 19:14:57 +08:00
Wu Clan 4e7db61ab2 Fix install plugin cli 2026-03-15 19:04:40 +08:00
Wu Clan f6507c86dd Add destroy SQL scripts for OAuth2 plugin 2026-03-15 19:04:23 +08:00
Wu Clan 0dd79ee3be Optimize permission logic 2026-03-15 17:42:04 +08:00
Wu Clan 46ad7a5d7f Add OAuth2 tenant support 2026-03-15 02:48:53 +08:00
Wu Clan e5b8a5242d Fix known issues and improvements 2026-03-15 01:34:05 +08:00
Wu Clan 22f1a243a0 Update tenant id in ctx 2026-03-14 18:35:42 +08:00
Wu Clan feb2bcbe82 Fix token for tenant id 2026-03-13 18:46:45 +08:00
Wu Clan c256f5c4ff Fix errors when the tenant is close 2026-03-12 23:34:44 +08:00
Wu Clan 80146224ab Fix and improve tenant-related impls 2026-03-12 23:12:30 +08:00
Wu Clan 87b8f6dbb1 Merge branch 'master' into pre-tenant 2026-03-09 21:59:31 +08:00
Wu Clan ed1a164f55 Update tenant check 2026-03-08 20:59:58 +08:00
Wu Clan f5285a4947 FIx lint and merge master 2026-03-08 19:14:45 +08:00
Wu Clan 74d159334f Merge branch 'master' into pre-tenant 2026-03-08 19:14:23 +08:00
Wu Clan 83c3a71378 Update some __table_args__ 2026-03-08 19:09:41 +08:00
Wu Clan f0856f0c6f Add tenant mode related adaptations 2026-03-08 18:39:34 +08:00
174 changed files with 9510 additions and 5085 deletions
+5 -1
View File
@@ -7,7 +7,11 @@ venv/
.python-version .python-version
.ruff_cache/ .ruff_cache/
.pytest_cache/ .pytest_cache/
logs/
.claude/ .claude/
.serena/ .serena/
.agents/ .agents/
.logs/ .superpowers/
superpowers/
.codegraph/
.pi/
+3 -3
View File
@@ -11,7 +11,7 @@ repos:
- id: check-toml - id: check-toml
- repo: https://github.com/tombi-toml/tombi-pre-commit - repo: https://github.com/tombi-toml/tombi-pre-commit
rev: v0.9.2 rev: v1.3.0
hooks: hooks:
- id: tombi-lint - id: tombi-lint
args: ["--offline"] args: ["--offline"]
@@ -19,7 +19,7 @@ repos:
args: ["--offline"] args: ["--offline"]
- repo: https://github.com/charliermarsh/ruff-pre-commit - repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.15.5 rev: v0.16.2
hooks: hooks:
- id: ruff-check - id: ruff-check
args: args:
@@ -29,7 +29,7 @@ repos:
- id: ruff-format - id: ruff-format
- repo: https://github.com/astral-sh/uv-pre-commit - repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.10.9 rev: 0.12.3
hooks: hooks:
- id: uv-lock - id: uv-lock
- id: uv-export - id: uv-export
+17 -4
View File
@@ -4,14 +4,14 @@
"description": "JSON Schema for FastAPI Best Architecture plugin.toml files. See: https://fastapi-practices.github.io/fastapi_best_architecture_docs/plugin/dev.html", "description": "JSON Schema for FastAPI Best Architecture plugin.toml files. See: https://fastapi-practices.github.io/fastapi_best_architecture_docs/plugin/dev.html",
"type": "object", "type": "object",
"required": ["plugin", "app"], "required": ["plugin"],
"additionalProperties": false, "additionalProperties": false,
"properties": { "properties": {
"plugin": { "plugin": {
"type": "object", "type": "object",
"description": "Plugin metadata", "description": "Plugin metadata",
"required": ["summary", "version", "description", "author", "tags", "database"], "required": ["summary", "version", "description", "author", "tags"],
"additionalProperties": false, "additionalProperties": false,
"x-tombi-table-keys-order": "schema", "x-tombi-table-keys-order": "schema",
"properties": { "properties": {
@@ -55,12 +55,21 @@
"database": { "database": {
"type": "array", "type": "array",
"minItems": 1, "minItems": 1,
"description": "Supported databases", "description": "Supported databases. Required for app-level and extend-level plugins, optional for capability-level plugins without models.",
"items": { "items": {
"type": "string", "type": "string",
"enum": ["mysql", "postgresql"] "enum": ["mysql", "postgresql"]
}, },
"x-tombi-array-values-order": "ascending" "x-tombi-array-values-order": "ascending"
},
"depends_on": {
"type": "array",
"description": "List of plugin names this plugin depends on (used for startup order)",
"items": {
"type": "string",
"minLength": 1
},
"x-tombi-array-values-order": "ascending"
} }
} }
}, },
@@ -102,7 +111,11 @@
"oneOf": [ "oneOf": [
{ "type": "string" }, { "type": "string" },
{ "type": "number" }, { "type": "number" },
{ "type": "boolean" } { "type": "boolean" },
{
"type": "array",
"items": { "type": "string" }
}
] ]
} }
}, },
+178
View File
@@ -1,3 +1,176 @@
<a id="v1.15.0"></a>
# [v1.15.0](https://github.com/fastapi-practices/fastapi-best-architecture/releases/tag/v1.15.0) - 2026-07-10
## What's Changed
* Update changelog for v1.14.0 by [@wu-clan](https://github.com/wu-clan) in [#1194](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1194)
* Add celery explicit idempotent protection by [@wu-clan](https://github.com/wu-clan) in [#1195](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1195)
* Optimize plugin core and add otel hooks by [@wu-clan](https://github.com/wu-clan) in [#1196](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1196)
* Fix user salt column LargeBinary length by [@IAseven](https://github.com/IAseven) in [#1197](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1197)
* Add plugin deps command to the CLI by [@wu-clan](https://github.com/wu-clan) in [#1200](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1200)
* Bump dependencies and pre-commits by [@wu-clan](https://github.com/wu-clan) in [#1205](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1205)
* Fix redis pubsub socket timeout by [@wu-clan](https://github.com/wu-clan) in [#1206](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1206)
* Add AI dynamic configuration menu by [@wu-clan](https://github.com/wu-clan) in [#1209](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1209)
* Refactor cache key and prefix APIs by [@wu-clan](https://github.com/wu-clan) in [#1210](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1210)
* Fix redis rate limiter bucket routing by [@wu-clan](https://github.com/wu-clan) in [#1212](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1212)
* Add underlying security config for JWT by [@wu-clan](https://github.com/wu-clan) in [#1214](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1214)
* Fix JWT auth dependency error propagation by [@wu-clan](https://github.com/wu-clan) in [#1215](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1215)
* Add is superuser protocol to ctx by [@wu-clan](https://github.com/wu-clan) in [#1216](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1216)
* Bump dependencies and pre-commits by [@wu-clan](https://github.com/wu-clan) in [#1218](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1218)
* Add support for capability plugins by [@yzbf-lin](https://github.com/yzbf-lin) in [#1217](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1217)
* Update capability plugin config validation by [@wu-clan](https://github.com/wu-clan) in [#1219](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1219)
* Update document link addresses in the CLI by [@wu-clan](https://github.com/wu-clan) in [#1220](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1220)
* Fix duplicate log reporting in OTEL by [@IAseven](https://github.com/IAseven) in [#1221](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1221)
* Refactor plugin dynamic config loading by [@wu-clan](https://github.com/wu-clan) in [#1222](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1222)
* Update xdb database file to release by [@wu-clan](https://github.com/wu-clan) in [#1223](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1223)
* Update plugin config format validation by [@wu-clan](https://github.com/wu-clan) in [#1224](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1224)
* Update the version number to 1.15.0 by [@wu-clan](https://github.com/wu-clan) in [#1225](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1225)
**Full Changelog**: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.14.0...v1.15.0
## Contributors
<a href="https://github.com/IAseven"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2FIAseven.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@IAseven"></a>
<a href="https://github.com/wu-clan"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fwu-clan.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@wu-clan"></a>
<a href="https://github.com/yzbf-lin"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fyzbf-lin.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@yzbf-lin"></a>
[Changes][v1.15.0]
<a id="v1.14.0"></a>
# [v1.14.0](https://github.com/fastapi-practices/fastapi-best-architecture/releases/tag/v1.14.0) - 2026-05-30
## What's Changed
* Update changelog for v1.13.4 by [@wu-clan](https://github.com/wu-clan) in [#1170](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1170)
* Update xdb dependency and database file by [@wu-clan](https://github.com/wu-clan) in [#1171](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1171)
* Optimize database operations within loops by [@wu-clan](https://github.com/wu-clan) in [#1177](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1177)
* Optimize some global variable definitions by [@wu-clan](https://github.com/wu-clan) in [#1178](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1178)
* Optimize SQL script execution in the CLI by [@wu-clan](https://github.com/wu-clan) in [#1181](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1181)
* Fix cache invalidation decorator usage by [@wu-clan](https://github.com/wu-clan) in [#1182](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1182)
* Optimize user session logging and deletion by [@wu-clan](https://github.com/wu-clan) in [#1185](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1185)
* Optimize operation log truncation handling by [@wu-clan](https://github.com/wu-clan) in [#1186](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1186)
* Add opera log task cancel to lifespan by [@wu-clan](https://github.com/wu-clan) in [#1187](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1187)
* Optimize Grafana observability metrics and config by [@wu-clan](https://github.com/wu-clan) in [#1188](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1188)
* Bump dependencies and pre-commits by [@wu-clan](https://github.com/wu-clan) in [#1189](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1189)
* Optimize the usage of setex for redis by [@wu-clan](https://github.com/wu-clan) in [#1190](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1190)
* Refactor physical deletion to logical deletion by [@wu-clan](https://github.com/wu-clan) in [#1191](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1191)
* Fix task scheduler unpack fields by [@wu-clan](https://github.com/wu-clan) in [#1192](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1192)
* Update the version number to 1.14.0 by [@wu-clan](https://github.com/wu-clan) in [#1193](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1193)
**Full Changelog**: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.4...v1.14.0
## Contributors
<a href="https://github.com/wu-clan"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fwu-clan.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@wu-clan"></a>
[Changes][v1.14.0]
<a id="v1.13.4"></a>
# [v1.13.4](https://github.com/fastapi-practices/fastapi-best-architecture/releases/tag/v1.13.4) - 2026-04-28
## What's Changed
* Update changelog for v1.13.3 by [@wu-clan](https://github.com/wu-clan) in [#1149](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1149)
* Update the architecture description in README by [@wu-clan](https://github.com/wu-clan) in [#1150](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1150)
* Fix app pytest client fixture scope by [@IAseven](https://github.com/IAseven) in [#1152](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1152)
* Remove cleanup operations after plugin install by [@wu-clan](https://github.com/wu-clan) in [#1154](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1154)
* Fix mysql enum data type mismatch by [@wu-clan](https://github.com/wu-clan) in [#1155](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1155)
* Bump dependencies and pre-commits by [@wu-clan](https://github.com/wu-clan) in [#1156](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1156)
* Add plugin depends on and lifecycle ordering by [@AH-Toby](https://github.com/AH-Toby) in [#1153](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1153)
* Update plugin README content format by [@wu-clan](https://github.com/wu-clan) in [#1158](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1158)
* Update Grafana dashboard and datasource config by [@wu-clan](https://github.com/wu-clan) in [#1159](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1159)
* Add database and queue observability by [@wu-clan](https://github.com/wu-clan) in [#1160](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1160)
* Fix Grafana tempo and observability config by [@wu-clan](https://github.com/wu-clan) in [#1161](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1161)
* Add the snowflake ID enabled config by [@wu-clan](https://github.com/wu-clan) in [#1162](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1162)
* Fix the add plugin CLI command config by [@wu-clan](https://github.com/wu-clan) in [#1163](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1163)
* Optimize the core code ordering of plugins by [@wu-clan](https://github.com/wu-clan) in [#1164](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1164)
* Add uninstall plugin operation checks by [@wu-clan](https://github.com/wu-clan) in [#1165](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1165)
* Fix multi level cache storage logic by [@wu-clan](https://github.com/wu-clan) in [#1166](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1166)
* Fix typing warnings for return methods by [@wu-clan](https://github.com/wu-clan) in [#1167](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1167)
* Bump dependencies and pre-commits by [@wu-clan](https://github.com/wu-clan) in [#1168](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1168)
* Update the version number to 1.13.4 by [@wu-clan](https://github.com/wu-clan) in [#1169](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1169)
## New Contributors
* [@AH-Toby](https://github.com/AH-Toby) made their first contribution in [#1153](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1153)
**Full Changelog**: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.3...v1.13.4
## Contributors
<a href="https://github.com/AH-Toby"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2FAH-Toby.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@AH-Toby"></a>
<a href="https://github.com/IAseven"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2FIAseven.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@IAseven"></a>
<a href="https://github.com/wu-clan"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fwu-clan.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@wu-clan"></a>
[Changes][v1.13.4]
<a id="v1.13.3"></a>
# [v1.13.3](https://github.com/fastapi-practices/fastapi-best-architecture/releases/tag/v1.13.3) - 2026-04-08
## What's Changed
* Update changelog for v1.13.2 by [@wu-clan](https://github.com/wu-clan) in [#1124](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1124)
* Fix the logic related to the status field by [@wu-clan](https://github.com/wu-clan) in [#1125](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1125)
* Fix code generation menu sql scripts by [@wu-clan](https://github.com/wu-clan) in [#1127](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1127)
* Update plugins and monitoring permissions by [@wu-clan](https://github.com/wu-clan) in [#1126](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1126)
* Add custom cursor pagination support by [@wu-clan](https://github.com/wu-clan) in [#1128](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1128)
* Update the plugin requirements check by [@wu-clan](https://github.com/wu-clan) in [#1129](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1129)
* Update plugin config and dependency cache by [@wu-clan](https://github.com/wu-clan) in [#1130](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1130)
* Fix plugin dependencies for redis instance by [@wu-clan](https://github.com/wu-clan) in [#1133](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1133)
* Fix current superuser permission validation by [@wu-clan](https://github.com/wu-clan) in [#1138](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1138)
* Fix the return of registered task results by [@wu-clan](https://github.com/wu-clan) in [#1139](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1139)
* Refactor plugin dependency cache and check by [@wu-clan](https://github.com/wu-clan) in [#1140](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1140)
* Add the installation frontend plugin CLI by [@wu-clan](https://github.com/wu-clan) in [#1141](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1141)
* Update custom reload filter call rules by [@wu-clan](https://github.com/wu-clan) in [#1144](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1144)
* Fix custom email model return type by [@wu-clan](https://github.com/wu-clan) in [#1145](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1145)
* Add superpowers to gitignore file by [@wu-clan](https://github.com/wu-clan) in [#1146](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1146)
* Add plugin system hook function mechanism by [@yuWorm](https://github.com/yuWorm) in [#1142](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1142)
* Bump dependencies and pre-commits by [@wu-clan](https://github.com/wu-clan) in [#1147](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1147)
* Update the version number to 1.13.3 by [@wu-clan](https://github.com/wu-clan) in [#1148](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1148)
## New Contributors
* [@yuWorm](https://github.com/yuWorm) made their first contribution in [#1142](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1142)
**Full Changelog**: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.2...v1.13.3
## Contributors
<a href="https://github.com/wu-clan"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fwu-clan.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@wu-clan"></a>
<a href="https://github.com/yuWorm"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2FyuWorm.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@yuWorm"></a>
[Changes][v1.13.3]
<a id="v1.13.2"></a>
# [v1.13.2](https://github.com/fastapi-practices/fastapi-best-architecture/releases/tag/v1.13.2) - 2026-03-23
## What's Changed
* Update changelog for v1.13.1 by [@wu-clan](https://github.com/wu-clan) in [#1103](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1103)
* Fix log type check and default formatter by [@wu-clan](https://github.com/wu-clan) in [#1104](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1104)
* Replace the uvicorn runner with granian by [@wu-clan](https://github.com/wu-clan) in [#1105](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1105)
* Update user id context setting location by [@wu-clan](https://github.com/wu-clan) in [#1110](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1110)
* Update refresh token geo-checksum response by [@wu-clan](https://github.com/wu-clan) in [#1111](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1111)
* Update plugin dynamic import check by [@wu-clan](https://github.com/wu-clan) in [#1112](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1112)
* Fix rbac permission checking logic by [@wu-clan](https://github.com/wu-clan) in [#1113](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1113)
* Fix user menu permission code update by [@wu-clan](https://github.com/wu-clan) in [#1114](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1114)
* Fix typos in code and comments by [@wu-clan](https://github.com/wu-clan) in [#1115](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1115)
* Fix online api user id acquisition by [@wu-clan](https://github.com/wu-clan) in [#1116](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1116)
* Fix plugin status change cache missing by [@wu-clan](https://github.com/wu-clan) in [#1118](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1118)
* Fix zip plugin installation and unzip by [@wu-clan](https://github.com/wu-clan) in [#1120](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1120)
* Update plugin config and sql validate by [@wu-clan](https://github.com/wu-clan) in [#1122](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1122)
* Add sql scripts for OAuth2 plugin by [@wu-clan](https://github.com/wu-clan) in [#1121](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1121)
* Update the version number to 1.13.2 by [@wu-clan](https://github.com/wu-clan) in [#1123](https://github.com/fastapi-practices/fastapi-best-architecture/pull/1123)
**Full Changelog**: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.1...v1.13.2
## Contributors
<a href="https://github.com/wu-clan"><img src="https://wsrv.nl/?url=https%3A%2F%2Fgithub.com%2Fwu-clan.png&w=128&h=128&fit=cover&mask=circle" width="64" height="64" alt="@wu-clan"></a>
[Changes][v1.13.2]
<a id="v1.13.1"></a> <a id="v1.13.1"></a>
# [v1.13.1](https://github.com/fastapi-practices/fastapi-best-architecture/releases/tag/v1.13.1) - 2026-03-08 # [v1.13.1](https://github.com/fastapi-practices/fastapi-best-architecture/releases/tag/v1.13.1) - 2026-03-08
@@ -1407,6 +1580,11 @@
[Changes][v1.0.0] [Changes][v1.0.0]
[v1.15.0]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.14.0...v1.15.0
[v1.14.0]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.4...v1.14.0
[v1.13.4]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.3...v1.13.4
[v1.13.3]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.2...v1.13.3
[v1.13.2]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.1...v1.13.2
[v1.13.1]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.0...v1.13.1 [v1.13.1]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.13.0...v1.13.1
[v1.13.0]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.12.3...v1.13.0 [v1.13.0]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.12.3...v1.13.0
[v1.12.3]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.12.2...v1.12.3 [v1.12.3]: https://github.com/fastapi-practices/fastapi-best-architecture/compare/v1.12.2...v1.12.3
+7 -9
View File
@@ -2,12 +2,12 @@
ARG SERVER_TYPE=fba_server ARG SERVER_TYPE=fba_server
# === Python environment from uv === # === Python environment from uv ===
FROM ghcr.io/astral-sh/uv:python3.10-bookworm-slim AS builder FROM ghcr.io/astral-sh/uv:python3.10-trixie-slim AS builder
# Used for build Python packages # Used for build Python packages
RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources \ RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources \
&& apt-get update \ && apt-get update \
&& apt-get install -y --no-install-recommends gcc python3-dev \ && apt-get install -y --no-install-recommends gcc make python3-dev \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
COPY . /fba COPY . /fba
@@ -26,20 +26,18 @@ RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=bind,source=pyproject.toml,target=pyproject.toml \ --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
uv sync --locked --no-default-groups --group server --no-install-project uv sync --locked --no-default-groups --group server --no-install-project
# Preinstall plugin dependencies
RUN --mount=type=cache,target=/root/.cache/uv \
python -c "from backend.plugin.requirements import install_requirements; install_requirements(None)"
# === Runtime base server image === # === Runtime base server image ===
FROM python:3.10-slim-bookworm AS base_server FROM ghcr.io/astral-sh/uv:python3.10-trixie-slim AS base_server
RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources \ RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources \
&& apt-get update \ && apt-get update \
&& apt-get install -y --no-install-recommends curl ca-certificates supervisor \ && apt-get install -y --no-install-recommends curl ca-certificates supervisor \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
ADD https://astral.sh/uv/install.sh /uv-installer.sh
RUN sh /uv-installer.sh && rm /uv-installer.sh
ENV PATH="/root/.local/bin/:$PATH"
COPY --from=builder /fba /fba COPY --from=builder /fba /fba
COPY --from=builder /usr/local /usr/local COPY --from=builder /usr/local /usr/local
+5 -7
View File
@@ -23,15 +23,13 @@ English | [简体中文](./README.zh-CN.md)
</div> </div>
## Pseudo 3-tier architecture ## Three-tier architecture
The mvc architecture is a common design pattern in python web, but the 3-tier architecture is even more fascinating The MVC architecture is common in Python Web as a general design pattern, but the three-tier architecture is more
fascinating
In python web development, there is no common standard for the concept of 3-tier architecture, so we'll call it a However, please note that we do not have a traditional multi-app (microservices) directory structure (django,
pseudo 3-tier architecture here springBoot...). ), but a self-righteous directory structure that you can give it any twist if you don't like this model!
But please note that we don't have a traditional multi-app structure (django, springBoot...) If you don't like this
pattern, use templates to transform it to your heart's content!
| workflow | java | fastapi_best_architecture | | workflow | java | fastapi_best_architecture |
|----------------|----------------|---------------------------| |----------------|----------------|---------------------------|
+3 -5
View File
@@ -23,13 +23,11 @@
</div> </div>
## 三层架构 ## 三层架构
mvc 架构作为常规设计模式,在 python web 中很常见,但是三层架构更令人着迷 mvc 架构作为常规设计模式,在 python web 中很常见,但是三层架构更令人着迷
在 python web 开发中,三层架构的概念并没有通用标准,所以这里我们称之为伪三层架构 但请注意,我们并没有传统的多 app (微服务)目录结构(django、springBoot...),而是自以为是的目录结构,如果您不喜欢这种模式,可以对其进行任意改造!
但请注意,我们并没有传统的多应用程序结构(django、springBoot...),如果您不喜欢这种模式,可以使用模板对其进行随意改造!
| 工作流程 | java | fastapi_best_architecture | | 工作流程 | java | fastapi_best_architecture |
|------|----------------|---------------------------| |------|----------------|---------------------------|
+3
View File
@@ -6,6 +6,9 @@ DATABASE_HOST='127.0.0.1'
DATABASE_PORT=5432 DATABASE_PORT=5432
DATABASE_USER='postgres' DATABASE_USER='postgres'
DATABASE_PASSWORD='123456' DATABASE_PASSWORD='123456'
# 额外数据源
# DATABASE_SOURCES='{"read": "postgresql+asyncpg://user:password@host:5432/fba"}'
DATABASE_SOURCES='{}'
# Redis # Redis
REDIS_HOST='127.0.0.1' REDIS_HOST='127.0.0.1'
REDIS_PORT=6379 REDIS_PORT=6379
+11 -11
View File
@@ -2,16 +2,16 @@ import sqlalchemy as sa
from backend.utils.dynamic_import import get_all_models from backend.utils.dynamic_import import get_all_models
# import all models for auto create db tables
for cls in get_all_models(): def _register_model_globals() -> None:
if isinstance(cls, sa.Table): """导入所有模型并注册到 backend 模块命名空间"""
table_name = cls.name for model_obj in get_all_models():
if table_name not in globals(): model_name = model_obj.name if isinstance(model_obj, sa.Table) else model_obj.__name__
globals()[table_name] = cls if model_name not in globals():
else: globals()[model_name] = model_obj
class_name = cls.__name__
if class_name not in globals():
globals()[class_name] = cls
__version__ = '1.13.2' _register_model_globals()
__version__ = '1.15.0'
+3 -3
View File
@@ -11,7 +11,7 @@ from sqlalchemy.ext.asyncio import async_engine_from_config
from backend.common.model import MappedBase from backend.common.model import MappedBase
from backend.core import path_conf from backend.core import path_conf
from backend.core.path_conf import BASE_PATH from backend.core.path_conf import BASE_PATH
from backend.database.db import SQLALCHEMY_DATABASE_URL from backend.database.db import get_database_url
if not os.path.exists(path_conf.ALEMBIC_VERSION_DIR): if not os.path.exists(path_conf.ALEMBIC_VERSION_DIR):
os.makedirs(path_conf.ALEMBIC_VERSION_DIR) os.makedirs(path_conf.ALEMBIC_VERSION_DIR)
@@ -35,7 +35,7 @@ target_metadata = MappedBase.metadata
# ... etc. # ... etc.
config.set_main_option( config.set_main_option(
'sqlalchemy.url', 'sqlalchemy.url',
SQLALCHEMY_DATABASE_URL.render_as_string(hide_password=False).replace('%', '%%'), get_database_url().render_as_string(hide_password=False).replace('%', '%%'),
) )
@@ -67,7 +67,7 @@ def run_migrations_offline() -> None:
def do_run_migrations(connection: Connection) -> None: def do_run_migrations(connection: Connection) -> None:
def process_revision_directives(context, revision, directives) -> None: # noqa: ANN001 def process_revision_directives(context, revision, directives) -> None: # ruff:ignore[missing-type-function-argument]
"""当迁移无变化时,不生成迁移记录""" """当迁移无变化时,不生成迁移记录"""
if config.cmd_opts.autogenerate: if config.cmd_opts.autogenerate:
script = directives[0] script = directives[0]
+16 -6
View File
@@ -7,20 +7,22 @@ from fastapi import APIRouter, Path, Query
from backend.app.admin.schema.token import GetTokenDetail from backend.app.admin.schema.token import GetTokenDetail
from backend.common.enums import StatusType from backend.common.enums import StatusType
from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base
from backend.common.security.jwt import DependsJwtAuth, DependsSuperUser, jwt_decode, revoke_token from backend.common.security.jwt import DependsSuperUser, jwt_decode, revoke_token
from backend.core.conf import settings from backend.core.conf import settings
from backend.database.redis import redis_client from backend.database.redis import redis_client
router = APIRouter() router = APIRouter()
@router.get('', summary='获取在线用户', dependencies=[DependsJwtAuth]) @router.get('', summary='获取在线用户', dependencies=[DependsSuperUser])
async def get_sessions( async def get_sessions(
username: Annotated[str | None, Query(description='用户名')] = None, username: Annotated[str | None, Query(description='用户名')] = None,
) -> ResponseSchemaModel[list[GetTokenDetail]]: ) -> ResponseSchemaModel[list[GetTokenDetail]]:
token_keys = await redis_client.get_prefix(f'{settings.TOKEN_REDIS_PREFIX}:*') token_keys = await redis_client.get_by_prefix(settings.TOKEN_REDIS_PREFIX)
online_clients = await redis_client.smembers(settings.TOKEN_ONLINE_REDIS_PREFIX) online_clients = await redis_client.smembers(settings.TOKEN_ONLINE_REDIS_PREFIX)
data: list[GetTokenDetail] = [] data: list[GetTokenDetail] = []
if not token_keys:
return response_base.success(data=data)
def append_token_detail() -> None: def append_token_detail() -> None:
data.append( data.append(
@@ -37,8 +39,12 @@ async def get_sessions(
), ),
) )
for key in token_keys: token_values = await redis_client.mget(*token_keys)
token = await redis_client.get(key) token_details: list[GetTokenDetail] = []
extra_info_keys: list[str] = []
for token in token_values:
if not token:
continue
token_payload = jwt_decode(token) token_payload = jwt_decode(token)
user_id = token_payload.user_id user_id = token_payload.user_id
session_uuid = token_payload.session_uuid session_uuid = token_payload.session_uuid
@@ -55,7 +61,11 @@ async def get_sessions(
last_login_time='未知', last_login_time='未知',
expire_time=token_payload.expire_time, expire_time=token_payload.expire_time,
) )
extra_info = await redis_client.get(f'{settings.TOKEN_EXTRA_INFO_REDIS_PREFIX}:{user_id}:{session_uuid}') token_details.append(token_detail)
extra_info_keys.append(f'{settings.TOKEN_EXTRA_INFO_REDIS_PREFIX}:{user_id}:{session_uuid}')
extra_infos = await redis_client.mget(*extra_info_keys) if extra_info_keys else []
for token_detail, extra_info in zip(token_details, extra_infos, strict=True):
if extra_info: if extra_info:
extra_info = json.loads(extra_info) extra_info = json.loads(extra_info)
# 排除 swagger 登录生成的 token # 排除 swagger 登录生成的 token
+4 -4
View File
@@ -20,16 +20,16 @@ from backend.app.admin.schema.monitor import (
SysInfo, SysInfo,
) )
from backend.common.response.response_schema import ResponseSchemaModel, response_base from backend.common.response.response_schema import ResponseSchemaModel, response_base
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsSuperUser
from backend.utils.format import fmt_bytes, fmt_seconds from backend.utils.format import fmt_bytes, fmt_seconds
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
router = APIRouter() router = APIRouter()
@router.get('', summary='Server 监控', dependencies=[DependsJwtAuth]) @router.get('', summary='Server 监控', dependencies=[DependsSuperUser])
async def get_server_info() -> ResponseSchemaModel[ServerMonitorInfo]: # noqa: C901 async def get_server_info() -> ResponseSchemaModel[ServerMonitorInfo]: # ruff:ignore[complex-structure]
def get_all_info() -> ServerMonitorInfo: # noqa: C901 def get_all_info() -> ServerMonitorInfo: # ruff:ignore[complex-structure]
# CPU 信息 # CPU 信息
cpu_data = { cpu_data = {
'physical_num': psutil.cpu_count(logical=False) or 0, 'physical_num': psutil.cpu_count(logical=False) or 0,
+8 -23
View File
@@ -1,6 +1,6 @@
from typing import Annotated, Any from typing import Annotated, Any
from fastapi import APIRouter, Depends, File, Path, UploadFile from fastapi import APIRouter, File, Path, UploadFile
from fastapi.params import Query from fastapi.params import Query
from starlette.responses import StreamingResponse from starlette.responses import StreamingResponse
@@ -8,20 +8,18 @@ from backend.app.admin.service.plugin_service import plugin_service
from backend.common.enums import PluginType from backend.common.enums import PluginType
from backend.common.response.response_code import CustomResponse from backend.common.response.response_code import CustomResponse
from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsSuperUser
from backend.common.security.permission import RequestPermission
from backend.common.security.rbac import DependsRBAC
router = APIRouter() router = APIRouter()
@router.get('', summary='获取所有插件', dependencies=[DependsJwtAuth]) @router.get('', summary='获取所有插件', dependencies=[DependsSuperUser])
async def get_all_plugins() -> ResponseSchemaModel[list[dict[str, Any]]]: async def get_all_plugins() -> ResponseSchemaModel[list[dict[str, Any]]]:
plugins = await plugin_service.get_all() plugins = await plugin_service.get_all()
return response_base.success(data=plugins) return response_base.success(data=plugins)
@router.get('/changed', summary='是否存在插件变更', dependencies=[DependsJwtAuth]) @router.get('/changed', summary='是否存在插件变更', dependencies=[DependsSuperUser])
async def plugin_changed() -> ResponseSchemaModel[bool]: async def plugin_changed() -> ResponseSchemaModel[bool]:
plugins = await plugin_service.changed() plugins = await plugin_service.changed()
return response_base.success(data=bool(plugins)) return response_base.success(data=bool(plugins))
@@ -31,10 +29,7 @@ async def plugin_changed() -> ResponseSchemaModel[bool]:
'', '',
summary='安装插件', summary='安装插件',
description='使用插件 zip 压缩包或 git 仓库地址进行安装(仅开发环境)', description='使用插件 zip 压缩包或 git 仓库地址进行安装(仅开发环境)',
dependencies=[ dependencies=[DependsSuperUser],
Depends(RequestPermission('sys:plugin:install')),
DependsRBAC,
],
) )
async def install_plugin( async def install_plugin(
type: Annotated[PluginType, Query(description='插件类型')], type: Annotated[PluginType, Query(description='插件类型')],
@@ -54,10 +49,7 @@ async def install_plugin(
'/{plugin}', '/{plugin}',
summary='卸载插件', summary='卸载插件',
description='此操作会直接删除插件依赖,但不会直接删除插件,而是将插件移动到备份目录(仅开发环境)', description='此操作会直接删除插件依赖,但不会直接删除插件,而是将插件移动到备份目录(仅开发环境)',
dependencies=[ dependencies=[DependsSuperUser],
Depends(RequestPermission('sys:plugin:uninstall')),
DependsRBAC,
],
) )
async def uninstall_plugin(plugin: Annotated[str, Path(description='插件名称')]) -> ResponseModel: async def uninstall_plugin(plugin: Annotated[str, Path(description='插件名称')]) -> ResponseModel:
await plugin_service.uninstall(plugin=plugin) await plugin_service.uninstall(plugin=plugin)
@@ -66,20 +58,13 @@ async def uninstall_plugin(plugin: Annotated[str, Path(description='插件名称
) )
@router.put( @router.put('/{plugin}/status', summary='更新插件状态', dependencies=[DependsSuperUser])
'/{plugin}/status',
summary='更新插件状态',
dependencies=[
Depends(RequestPermission('sys:plugin:edit')),
DependsRBAC,
],
)
async def update_plugin_status(plugin: Annotated[str, Path(description='插件名称')]) -> ResponseModel: async def update_plugin_status(plugin: Annotated[str, Path(description='插件名称')]) -> ResponseModel:
await plugin_service.update_status(plugin=plugin) await plugin_service.update_status(plugin=plugin)
return response_base.success() return response_base.success()
@router.get('/{plugin}', summary='下载插件', dependencies=[DependsJwtAuth]) @router.get('/{plugin}', summary='下载插件', dependencies=[DependsSuperUser])
async def download_plugin(plugin: Annotated[str, Path(description='插件名称')]) -> StreamingResponse: async def download_plugin(plugin: Annotated[str, Path(description='插件名称')]) -> StreamingResponse:
bio = await plugin_service.build(plugin=plugin) bio = await plugin_service.build(plugin=plugin)
return StreamingResponse( return StreamingResponse(
+27 -6
View File
@@ -6,6 +6,7 @@ from sqlalchemy_crud_plus import CRUDPlus
from backend.app.admin.model import DataRule from backend.app.admin.model import DataRule
from backend.app.admin.schema.data_rule import CreateDataRuleParam, UpdateDataRuleParam from backend.app.admin.schema.data_rule import CreateDataRuleParam, UpdateDataRuleParam
from backend.utils.timezone import timezone
class CRUDDataRule(CRUDPlus[DataRule]): class CRUDDataRule(CRUDPlus[DataRule]):
@@ -19,7 +20,7 @@ class CRUDDataRule(CRUDPlus[DataRule]):
:param pk: 规则 ID :param pk: 规则 ID
:return: :return:
""" """
return await self.select_model(db, pk) return await self.select_model(db, pk, deleted=0)
async def get_select(self, name: str | None) -> Select: async def get_select(self, name: str | None) -> Select:
""" """
@@ -28,7 +29,7 @@ class CRUDDataRule(CRUDPlus[DataRule]):
:param name: 规则名称 :param name: 规则名称
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if name is not None: if name is not None:
filters['name__like'] = f'%{name}%' filters['name__like'] = f'%{name}%'
@@ -43,7 +44,7 @@ class CRUDDataRule(CRUDPlus[DataRule]):
:param name: 规则名称 :param name: 规则名称
:return: :return:
""" """
return await self.select_model_by_column(db, name=name) return await self.select_model_by_column(db, name=name, deleted=0)
async def get_all(self, db: AsyncSession) -> Sequence[DataRule]: async def get_all(self, db: AsyncSession) -> Sequence[DataRule]:
""" """
@@ -52,7 +53,17 @@ class CRUDDataRule(CRUDPlus[DataRule]):
:param db: 数据库会话 :param db: 数据库会话
:return: :return:
""" """
return await self.select_models(db) return await self.select_models(db, deleted=0)
async def get_all_by_ids(self, db: AsyncSession, pks: list[int]) -> Sequence[DataRule]:
"""
通过 ID 列表批量获取数据规则
:param db: 数据库会话
:param pks: 规则 ID 列表
:return:
"""
return await self.select_models(db, id__in=pks, deleted=0)
async def create(self, db: AsyncSession, obj: CreateDataRuleParam) -> None: async def create(self, db: AsyncSession, obj: CreateDataRuleParam) -> None:
""" """
@@ -73,7 +84,7 @@ class CRUDDataRule(CRUDPlus[DataRule]):
:param obj: 更新规则参数 :param obj: 更新规则参数
:return: :return:
""" """
return await self.update_model(db, pk, obj) return await self.update_model_by_column(db, obj, id=pk, deleted=0)
async def delete(self, db: AsyncSession, pks: list[int]) -> int: async def delete(self, db: AsyncSession, pks: list[int]) -> int:
""" """
@@ -83,7 +94,17 @@ class CRUDDataRule(CRUDPlus[DataRule]):
:param pks: 规则 ID 列表 :param pks: 规则 ID 列表
:return: :return:
""" """
return await self.delete_model_by_column(db, allow_multiple=True, id__in=pks) return await self.delete_model_by_column(
db,
allow_multiple=True,
logical_deletion=True,
deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id__in=pks,
deleted=0,
)
data_rule_dao: CRUDDataRule = CRUDDataRule(DataRule) data_rule_dao: CRUDDataRule = CRUDDataRule(DataRule)
+34 -8
View File
@@ -1,7 +1,7 @@
from collections.abc import Sequence from collections.abc import Sequence
from typing import Any from typing import Any
from sqlalchemy import Select, delete, insert from sqlalchemy import Select, and_, delete, insert
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus, JoinConfig from sqlalchemy_crud_plus import CRUDPlus, JoinConfig
@@ -13,6 +13,7 @@ from backend.app.admin.schema.data_scope import (
UpdateDataScopeRuleParam, UpdateDataScopeRuleParam,
) )
from backend.utils.serializers import select_join_serialize from backend.utils.serializers import select_join_serialize
from backend.utils.timezone import timezone
class CRUDDataScope(CRUDPlus[DataScope]): class CRUDDataScope(CRUDPlus[DataScope]):
@@ -26,7 +27,7 @@ class CRUDDataScope(CRUDPlus[DataScope]):
:param pk: 范围 ID :param pk: 范围 ID
:return: :return:
""" """
return await self.select_model(db, pk) return await self.select_model(db, pk, deleted=0)
async def get_by_name(self, db: AsyncSession, name: str) -> DataScope | None: async def get_by_name(self, db: AsyncSession, name: str) -> DataScope | None:
""" """
@@ -36,7 +37,7 @@ class CRUDDataScope(CRUDPlus[DataScope]):
:param name: 范围名称 :param name: 范围名称
:return: :return:
""" """
return await self.select_model_by_column(db, name=name) return await self.select_model_by_column(db, name=name, deleted=0)
async def get_join(self, db: AsyncSession, pk: int) -> Any: async def get_join(self, db: AsyncSession, pk: int) -> Any:
""" """
@@ -49,9 +50,14 @@ class CRUDDataScope(CRUDPlus[DataScope]):
result = await self.select_models( result = await self.select_models(
db, db,
id=pk, id=pk,
deleted=0,
join_conditions=[ join_conditions=[
JoinConfig(model=data_scope_rule, join_on=data_scope_rule.c.data_scope_id == self.model.id), JoinConfig(model=data_scope_rule, join_on=data_scope_rule.c.data_scope_id == self.model.id),
JoinConfig(model=DataRule, join_on=DataRule.id == data_scope_rule.c.data_rule_id, fill_result=True), JoinConfig(
model=DataRule,
join_on=and_(DataRule.id == data_scope_rule.c.data_rule_id, DataRule.deleted == 0),
fill_result=True,
),
], ],
) )
@@ -64,7 +70,17 @@ class CRUDDataScope(CRUDPlus[DataScope]):
:param db: 数据库会话 :param db: 数据库会话
:return: :return:
""" """
return await self.select_models(db) return await self.select_models(db, deleted=0)
async def get_all_by_ids(self, db: AsyncSession, pks: list[int]) -> Sequence[DataScope]:
"""
通过 ID 列表批量获取数据范围
:param db: 数据库会话
:param pks: 范围 ID 列表
:return:
"""
return await self.select_models(db, id__in=pks, deleted=0)
async def get_select(self, name: str | None, status: int | None) -> Select: async def get_select(self, name: str | None, status: int | None) -> Select:
""" """
@@ -74,7 +90,7 @@ class CRUDDataScope(CRUDPlus[DataScope]):
:param status: 范围状态 :param status: 范围状态
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if name is not None: if name is not None:
filters['name__like'] = f'%{name}%' filters['name__like'] = f'%{name}%'
@@ -102,7 +118,7 @@ class CRUDDataScope(CRUDPlus[DataScope]):
:param obj: 更新数据范围参数 :param obj: 更新数据范围参数
:return: :return:
""" """
return await self.update_model(db, pk, obj) return await self.update_model_by_column(db, obj, id=pk, deleted=0)
@staticmethod @staticmethod
async def update_rules(db: AsyncSession, pk: int, rule_ids: UpdateDataScopeRuleParam) -> int: async def update_rules(db: AsyncSession, pk: int, rule_ids: UpdateDataScopeRuleParam) -> int:
@@ -135,7 +151,17 @@ class CRUDDataScope(CRUDPlus[DataScope]):
:param pks: 范围 ID 列表 :param pks: 范围 ID 列表
:return: :return:
""" """
return await self.delete_model_by_column(db, allow_multiple=True, id__in=pks) return await self.delete_model_by_column(
db,
allow_multiple=True,
logical_deletion=True,
deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id__in=pks,
deleted=0,
)
data_scope_dao: CRUDDataScope = CRUDDataScope(DataScope) data_scope_dao: CRUDDataScope = CRUDDataScope(DataScope)
+25 -8
View File
@@ -1,13 +1,14 @@
from collections.abc import Sequence from collections.abc import Sequence
from typing import Any from typing import Any
from sqlalchemy import ColumnElement from sqlalchemy import ColumnElement, and_
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus, JoinConfig from sqlalchemy_crud_plus import CRUDPlus, JoinConfig
from backend.app.admin.model import Dept, User from backend.app.admin.model import Dept, User
from backend.app.admin.schema.dept import CreateDeptParam, UpdateDeptParam from backend.app.admin.schema.dept import CreateDeptParam, UpdateDeptParam
from backend.utils.serializers import select_join_serialize from backend.utils.serializers import select_join_serialize
from backend.utils.timezone import timezone
class CRUDDept(CRUDPlus[Dept]): class CRUDDept(CRUDPlus[Dept]):
@@ -21,7 +22,7 @@ class CRUDDept(CRUDPlus[Dept]):
:param dept_id: 部门 ID :param dept_id: 部门 ID
:return: :return:
""" """
return await self.select_model_by_column(db, id=dept_id, del_flag=False) return await self.select_model_by_column(db, id=dept_id, deleted=0)
async def get_by_name(self, db: AsyncSession, name: str) -> Dept | None: async def get_by_name(self, db: AsyncSession, name: str) -> Dept | None:
""" """
@@ -31,7 +32,7 @@ class CRUDDept(CRUDPlus[Dept]):
:param name: 部门名称 :param name: 部门名称
:return: :return:
""" """
return await self.select_model_by_column(db, name=name, del_flag=False) return await self.select_model_by_column(db, name=name, deleted=0)
async def get_all( async def get_all(
self, self,
@@ -53,7 +54,7 @@ class CRUDDept(CRUDPlus[Dept]):
:param status: 部门状态 :param status: 部门状态
:return: :return:
""" """
filters = {'del_flag': False} filters = {'deleted': 0}
if name is not None: if name is not None:
filters['name__like'] = f'%{name}%' filters['name__like'] = f'%{name}%'
@@ -85,7 +86,7 @@ class CRUDDept(CRUDPlus[Dept]):
:param obj: 更新部门参数 :param obj: 更新部门参数
:return: :return:
""" """
return await self.update_model(db, dept_id, obj) return await self.update_model_by_column(db, obj, id=dept_id, deleted=0)
async def delete(self, db: AsyncSession, dept_id: int) -> int: async def delete(self, db: AsyncSession, dept_id: int) -> int:
""" """
@@ -95,7 +96,16 @@ class CRUDDept(CRUDPlus[Dept]):
:param dept_id: 部门 ID :param dept_id: 部门 ID
:return: :return:
""" """
return await self.delete_model_by_column(db, id=dept_id, logical_deletion=True, deleted_flag_column='del_flag') return await self.delete_model_by_column(
db,
logical_deletion=True,
deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id=dept_id,
deleted=0,
)
async def get_join(self, db: AsyncSession, dept_id: int) -> Any | None: async def get_join(self, db: AsyncSession, dept_id: int) -> Any | None:
""" """
@@ -108,7 +118,14 @@ class CRUDDept(CRUDPlus[Dept]):
result = await self.select_model( result = await self.select_model(
db, db,
dept_id, dept_id,
join_conditions=[JoinConfig(model=User, join_on=User.dept_id == self.model.id, fill_result=True)], deleted=0,
join_conditions=[
JoinConfig(
model=User,
join_on=and_(User.dept_id == self.model.id, User.deleted == 0),
fill_result=True,
)
],
) )
return select_join_serialize(result, relationships=['Dept-o2m-User']) return select_join_serialize(result, relationships=['Dept-o2m-User'])
@@ -120,7 +137,7 @@ class CRUDDept(CRUDPlus[Dept]):
:param dept_id: 部门 ID :param dept_id: 部门 ID
:return: :return:
""" """
return await self.select_models(db, parent_id=dept_id, del_flag=False) return await self.select_models(db, parent_id=dept_id, deleted=0)
dept_dao: CRUDDept = CRUDDept(Dept) dept_dao: CRUDDept = CRUDDept(Dept)
+27 -7
View File
@@ -6,6 +6,7 @@ from sqlalchemy_crud_plus import CRUDPlus
from backend.app.admin.model import Menu, role_menu from backend.app.admin.model import Menu, role_menu
from backend.app.admin.schema.menu import CreateMenuParam, UpdateMenuParam from backend.app.admin.schema.menu import CreateMenuParam, UpdateMenuParam
from backend.utils.timezone import timezone
class CRUDMenu(CRUDPlus[Menu]): class CRUDMenu(CRUDPlus[Menu]):
@@ -19,7 +20,7 @@ class CRUDMenu(CRUDPlus[Menu]):
:param menu_id: 菜单 ID :param menu_id: 菜单 ID
:return: :return:
""" """
return await self.select_model(db, menu_id) return await self.select_model(db, menu_id, deleted=0)
async def get_by_title(self, db: AsyncSession, title: str) -> Menu | None: async def get_by_title(self, db: AsyncSession, title: str) -> Menu | None:
""" """
@@ -29,7 +30,7 @@ class CRUDMenu(CRUDPlus[Menu]):
:param title: 菜单标题 :param title: 菜单标题
:return: :return:
""" """
return await self.select_model_by_column(db, title=title, type__ne=2) return await self.select_model_by_column(db, title=title, type__ne=2, deleted=0)
async def get_all(self, db: AsyncSession, title: str | None, status: int | None) -> Sequence[Menu]: async def get_all(self, db: AsyncSession, title: str | None, status: int | None) -> Sequence[Menu]:
""" """
@@ -40,7 +41,7 @@ class CRUDMenu(CRUDPlus[Menu]):
:param status: 菜单状态 :param status: 菜单状态
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if title is not None: if title is not None:
filters['title__like'] = f'%{title}%' filters['title__like'] = f'%{title}%'
@@ -57,13 +58,23 @@ class CRUDMenu(CRUDPlus[Menu]):
:param menu_ids: 菜单 ID 列表 :param menu_ids: 菜单 ID 列表
:return: :return:
""" """
filters = {'type__in': [0, 1, 3, 4]} filters = {'type__in': [0, 1, 3, 4], 'deleted': 0}
if menu_ids: if menu_ids:
filters['id__in'] = menu_ids filters['id__in'] = menu_ids
return await self.select_models_order(db, 'sort', 'asc', **filters) return await self.select_models_order(db, 'sort', 'asc', **filters)
async def get_all_by_ids(self, db: AsyncSession, menu_ids: list[int]) -> Sequence[Menu]:
"""
通过 ID 列表批量获取菜单
:param db: 数据库会话
:param menu_ids: 菜单 ID 列表
:return:
"""
return await self.select_models(db, id__in=menu_ids, deleted=0)
async def create(self, db: AsyncSession, obj: CreateMenuParam) -> None: async def create(self, db: AsyncSession, obj: CreateMenuParam) -> None:
""" """
创建菜单 创建菜单
@@ -83,7 +94,7 @@ class CRUDMenu(CRUDPlus[Menu]):
:param obj: 更新菜单参数 :param obj: 更新菜单参数
:return: :return:
""" """
return await self.update_model(db, menu_id, obj) return await self.update_model_by_column(db, obj, id=menu_id, deleted=0)
async def delete(self, db: AsyncSession, menu_id: int) -> int: async def delete(self, db: AsyncSession, menu_id: int) -> int:
""" """
@@ -96,7 +107,16 @@ class CRUDMenu(CRUDPlus[Menu]):
role_menu_stmt = delete(role_menu).where(role_menu.c.menu_id == menu_id) role_menu_stmt = delete(role_menu).where(role_menu.c.menu_id == menu_id)
await db.execute(role_menu_stmt) await db.execute(role_menu_stmt)
return await self.delete_model(db, menu_id) return await self.delete_model_by_column(
db,
logical_deletion=True,
deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id=menu_id,
deleted=0,
)
async def get_children(self, db: AsyncSession, menu_id: int) -> Sequence[Menu | None]: async def get_children(self, db: AsyncSession, menu_id: int) -> Sequence[Menu | None]:
""" """
@@ -106,7 +126,7 @@ class CRUDMenu(CRUDPlus[Menu]):
:param menu_id: 菜单 ID :param menu_id: 菜单 ID
:return: :return:
""" """
return await self.select_models(db, parent_id=menu_id) return await self.select_models(db, parent_id=menu_id, deleted=0)
menu_dao: CRUDMenu = CRUDMenu(Menu) menu_dao: CRUDMenu = CRUDMenu(Menu)
+65 -17
View File
@@ -1,7 +1,7 @@
from collections.abc import Sequence from collections.abc import Sequence
from typing import Any from typing import Any
from sqlalchemy import Select, delete, insert, select from sqlalchemy import Select, and_, delete, insert, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus, JoinConfig from sqlalchemy_crud_plus import CRUDPlus, JoinConfig
@@ -14,7 +14,19 @@ from backend.app.admin.schema.role import (
UpdateRoleParam, UpdateRoleParam,
UpdateRoleScopeParam, UpdateRoleScopeParam,
) )
from backend.core.conf import settings
from backend.utils.serializers import select_join_serialize from backend.utils.serializers import select_join_serialize
from backend.utils.timezone import timezone
if settings.TENANT_ENABLED:
try:
from backend.plugin.tenant.utils import get_tenant_dict as inject_tenant_dict
except ImportError:
raise ImportError('租户插件方法导入失败,请联系系统管理员')
else:
def inject_tenant_dict(obj: dict[str, Any]) -> dict[str, Any]:
return obj
class CRUDRole(CRUDPlus[Role]): class CRUDRole(CRUDPlus[Role]):
@@ -28,7 +40,7 @@ class CRUDRole(CRUDPlus[Role]):
:param role_id: 角色 ID :param role_id: 角色 ID
:return: :return:
""" """
return await self.select_model(db, role_id) return await self.select_model(db, role_id, deleted=0)
@staticmethod @staticmethod
async def get_menus(db: AsyncSession, role_id: int) -> Sequence[Menu] | None: async def get_menus(db: AsyncSession, role_id: int) -> Sequence[Menu] | None:
@@ -39,7 +51,11 @@ class CRUDRole(CRUDPlus[Role]):
:param role_id: 角色 ID :param role_id: 角色 ID
:return: :return:
""" """
menu_stmt = select(Menu).join(role_menu, Menu.id == role_menu.c.menu_id).where(role_menu.c.role_id == role_id) menu_stmt = (
select(Menu)
.join(role_menu, Menu.id == role_menu.c.menu_id)
.where(role_menu.c.role_id == role_id, Menu.deleted == 0)
)
result = await db.execute(menu_stmt) result = await db.execute(menu_stmt)
return result.scalars().all() return result.scalars().all()
@@ -54,11 +70,20 @@ class CRUDRole(CRUDPlus[Role]):
result = await self.select_models( result = await self.select_models(
db, db,
id=role_id, id=role_id,
deleted=0,
join_conditions=[ join_conditions=[
JoinConfig(model=role_menu, join_on=role_menu.c.role_id == self.model.id), JoinConfig(model=role_menu, join_on=role_menu.c.role_id == self.model.id),
JoinConfig(model=Menu, join_on=Menu.id == role_menu.c.menu_id, fill_result=True), JoinConfig(
model=Menu,
join_on=and_(Menu.id == role_menu.c.menu_id, Menu.deleted == 0),
fill_result=True,
),
JoinConfig(model=role_data_scope, join_on=role_data_scope.c.role_id == self.model.id), JoinConfig(model=role_data_scope, join_on=role_data_scope.c.role_id == self.model.id),
JoinConfig(model=DataScope, join_on=DataScope.id == role_data_scope.c.data_scope_id, fill_result=True), JoinConfig(
model=DataScope,
join_on=and_(DataScope.id == role_data_scope.c.data_scope_id, DataScope.deleted == 0),
fill_result=True,
),
], ],
) )
@@ -71,7 +96,17 @@ class CRUDRole(CRUDPlus[Role]):
:param db: 数据库会话 :param db: 数据库会话
:return: :return:
""" """
return await self.select_models(db) return await self.select_models(db, deleted=0)
async def get_all_by_ids(self, db: AsyncSession, role_ids: list[int]) -> Sequence[Role]:
"""
通过 ID 列表批量获取角色
:param db: 数据库会话
:param role_ids: 角色 ID 列表
:return:
"""
return await self.select_models(db, id__in=role_ids, deleted=0)
async def get_select(self, name: str | None, status: int | None) -> Select: async def get_select(self, name: str | None, status: int | None) -> Select:
""" """
@@ -82,7 +117,7 @@ class CRUDRole(CRUDPlus[Role]):
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if name is not None: if name is not None:
filters['name__like'] = f'%{name}%' filters['name__like'] = f'%{name}%'
@@ -99,7 +134,7 @@ class CRUDRole(CRUDPlus[Role]):
:param name: 角色名称 :param name: 角色名称
:return: :return:
""" """
return await self.select_model_by_column(db, name=name) return await self.select_model_by_column(db, name=name, deleted=0)
async def create(self, db: AsyncSession, obj: CreateRoleParam) -> None: async def create(self, db: AsyncSession, obj: CreateRoleParam) -> None:
""" """
@@ -120,7 +155,7 @@ class CRUDRole(CRUDPlus[Role]):
:param obj: 更新角色参数 :param obj: 更新角色参数
:return: :return:
""" """
return await self.update_model(db, role_id, obj) return await self.update_model_by_column(db, obj, id=role_id, deleted=0)
@staticmethod @staticmethod
async def update_menus(db: AsyncSession, role_id: int, menu_ids: UpdateRoleMenuParam) -> int: async def update_menus(db: AsyncSession, role_id: int, menu_ids: UpdateRoleMenuParam) -> int:
@@ -136,9 +171,11 @@ class CRUDRole(CRUDPlus[Role]):
await db.execute(role_menu_stmt) await db.execute(role_menu_stmt)
if menu_ids.menus: if menu_ids.menus:
role_menu_data = [ role_menu_data = []
CreateRoleMenuParam(role_id=role_id, menu_id=menu_id).model_dump() for menu_id in menu_ids.menus for menu_id in menu_ids.menus:
] menu_dict = CreateRoleMenuParam(role_id=role_id, menu_id=menu_id).model_dump()
role_menu_data.append(inject_tenant_dict(menu_dict))
role_menu_stmt = insert(role_menu) role_menu_stmt = insert(role_menu)
await db.execute(role_menu_stmt, role_menu_data) await db.execute(role_menu_stmt, role_menu_data)
@@ -158,10 +195,11 @@ class CRUDRole(CRUDPlus[Role]):
await db.execute(role_scope_stmt) await db.execute(role_scope_stmt)
if scope_ids.scopes: if scope_ids.scopes:
role_scope_data = [ role_scope_data = []
CreateRoleScopeParam(role_id=role_id, data_scope_id=scope_id).model_dump() for scope_id in scope_ids.scopes:
for scope_id in scope_ids.scopes scope_dict = CreateRoleScopeParam(role_id=role_id, data_scope_id=scope_id).model_dump()
] role_scope_data.append(inject_tenant_dict(scope_dict))
role_scope_stmt = insert(role_data_scope) role_scope_stmt = insert(role_data_scope)
await db.execute(role_scope_stmt, role_scope_data) await db.execute(role_scope_stmt, role_scope_data)
@@ -175,7 +213,17 @@ class CRUDRole(CRUDPlus[Role]):
:param role_ids: 角色 ID 列表 :param role_ids: 角色 ID 列表
:return: :return:
""" """
return await self.delete_model_by_column(db, allow_multiple=True, id__in=role_ids) return await self.delete_model_by_column(
db,
allow_multiple=True,
logical_deletion=True,
deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id__in=role_ids,
deleted=0,
)
role_dao: CRUDRole = CRUDRole(Role) role_dao: CRUDRole = CRUDRole(Role)
+107 -32
View File
@@ -1,8 +1,9 @@
from collections.abc import Sequence
from typing import Any from typing import Any
import bcrypt import bcrypt
from sqlalchemy import Select, delete, insert, select from sqlalchemy import Select, and_, delete, insert, select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus, JoinConfig from sqlalchemy_crud_plus import CRUDPlus, JoinConfig
@@ -25,11 +26,23 @@ from backend.app.admin.schema.user import (
UpdateUserParam, UpdateUserParam,
) )
from backend.app.admin.utils.password_security import get_hash_password from backend.app.admin.utils.password_security import get_hash_password
from backend.common.enums import StatusType
from backend.common.exception import errors from backend.common.exception import errors
from backend.core.conf import settings
from backend.plugin.core import check_plugin_installed from backend.plugin.core import check_plugin_installed
from backend.utils.serializers import select_join_serialize from backend.utils.serializers import select_join_serialize
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
if settings.TENANT_ENABLED:
try:
from backend.plugin.tenant.utils import get_tenant_dict as inject_tenant_dict
except ImportError:
raise ImportError('租户插件方法导入失败,请联系系统管理员')
else:
def inject_tenant_dict(obj: dict[str, Any]) -> dict[str, Any]:
return obj
class CRUDUser(CRUDPlus[User]): class CRUDUser(CRUDPlus[User]):
"""用户数据库操作类""" """用户数据库操作类"""
@@ -42,7 +55,7 @@ class CRUDUser(CRUDPlus[User]):
:param user_id: 用户 ID :param user_id: 用户 ID
:return: :return:
""" """
return await self.select_model(db, user_id) return await self.select_model(db, user_id, deleted=0)
async def get_by_username(self, db: AsyncSession, username: str) -> User | None: async def get_by_username(self, db: AsyncSession, username: str) -> User | None:
""" """
@@ -52,7 +65,17 @@ class CRUDUser(CRUDPlus[User]):
:param username: 用户名 :param username: 用户名
:return: :return:
""" """
return await self.select_model_by_column(db, username=username) return await self.select_model_by_column(db, username=username, deleted=0)
async def get_all_by_usernames(self, db: AsyncSession, usernames: list[str]) -> Sequence[User]:
"""
通过用户名列表批量获取用户
:param db: 数据库会话
:param usernames: 用户名列表
:return:
"""
return await self.select_models(db, username__in=usernames, deleted=0)
async def get_by_nickname(self, db: AsyncSession, nickname: str) -> User | None: async def get_by_nickname(self, db: AsyncSession, nickname: str) -> User | None:
""" """
@@ -62,7 +85,7 @@ class CRUDUser(CRUDPlus[User]):
:param nickname: 用户昵称 :param nickname: 用户昵称
:return: :return:
""" """
return await self.select_model_by_column(db, nickname=nickname) return await self.select_model_by_column(db, nickname=nickname, deleted=0)
async def check_email(self, db: AsyncSession, email: str) -> User | None: async def check_email(self, db: AsyncSession, email: str) -> User | None:
""" """
@@ -72,7 +95,7 @@ class CRUDUser(CRUDPlus[User]):
:param email: 电子邮箱 :param email: 电子邮箱
:return: :return:
""" """
return await self.select_model_by_column(db, email=email) return await self.select_model_by_column(db, email=email, deleted=0)
async def get_select(self, dept: int | None, username: str | None, phone: str | None, status: int | None) -> Select: async def get_select(self, dept: int | None, username: str | None, phone: str | None, status: int | None) -> Select:
""" """
@@ -84,7 +107,7 @@ class CRUDUser(CRUDPlus[User]):
:param status: 用户状态 :param status: 用户状态
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if dept: if dept:
filters['dept_id'] = dept filters['dept_id'] = dept
@@ -99,9 +122,17 @@ class CRUDUser(CRUDPlus[User]):
'id', 'id',
'desc', 'desc',
join_conditions=[ join_conditions=[
JoinConfig(model=Dept, join_on=Dept.id == self.model.dept_id, fill_result=True), JoinConfig(
model=Dept,
join_on=and_(Dept.id == self.model.dept_id, Dept.deleted == 0),
fill_result=True,
),
JoinConfig(model=user_role, join_on=user_role.c.user_id == self.model.id), JoinConfig(model=user_role, join_on=user_role.c.user_id == self.model.id),
JoinConfig(model=Role, join_on=Role.id == user_role.c.role_id, fill_result=True), JoinConfig(
model=Role,
join_on=and_(Role.id == user_role.c.role_id, Role.deleted == 0),
fill_result=True,
),
], ],
**filters, **filters,
) )
@@ -119,16 +150,21 @@ class CRUDUser(CRUDPlus[User]):
dict_obj = obj.model_dump(exclude={'roles'}) dict_obj = obj.model_dump(exclude={'roles'})
dict_obj.update({'salt': salt}) dict_obj.update({'salt': salt})
new_user = self.model(**dict_obj) new_user = self.model(**dict_obj)
db.add(new_user) db.add(new_user)
await db.flush() await db.flush()
if obj.roles: if obj.roles:
role_stmt = select(Role).where(Role.id.in_(obj.roles)) role_stmt = select(Role).where(Role.id.in_(obj.roles), Role.deleted == 0)
result = await db.execute(role_stmt) result = await db.execute(role_stmt)
roles = result.scalars().all() roles = result.scalars().all()
user_role_data = [AddUserRoleParam(user_id=new_user.id, role_id=role.id).model_dump() for role in roles] user_role_data = []
for role in roles:
role_dict = AddUserRoleParam(user_id=new_user.id, role_id=role.id).model_dump()
user_role_data.append(inject_tenant_dict(role_dict))
user_role_stmt = insert(user_role) user_role_stmt = insert(user_role)
await db.execute(user_role_stmt, user_role_data) await db.execute(user_role_stmt, user_role_data)
@@ -142,15 +178,19 @@ class CRUDUser(CRUDPlus[User]):
""" """
dict_obj = obj.model_dump() dict_obj = obj.model_dump()
dict_obj.update({'is_staff': True, 'salt': None}) dict_obj.update({'is_staff': True, 'salt': None})
new_user = self.model(**dict_obj) new_user = self.model(**dict_obj)
db.add(new_user) db.add(new_user)
await db.flush() await db.flush()
role_stmt = select(Role) role_stmt = select(Role).where(Role.status == StatusType.enable, Role.deleted == 0)
result = await db.execute(role_stmt) result = await db.execute(role_stmt)
role = result.scalars().first() # 默认绑定第一个角色 role = result.scalars().first() # 默认绑定第一个角色
if role is None:
raise errors.NotFoundError(msg='未找到可用角色,请联系系统管理员')
user_role_stmt = insert(user_role).values(AddUserRoleParam(user_id=new_user.id, role_id=role.id).model_dump()) user_role_data = inject_tenant_dict(AddUserRoleParam(user_id=new_user.id, role_id=role.id).model_dump())
user_role_stmt = insert(user_role).values(user_role_data)
await db.execute(user_role_stmt) await db.execute(user_role_stmt)
async def update(self, db: AsyncSession, user_id: int, obj: UpdateUserParam) -> int: async def update(self, db: AsyncSession, user_id: int, obj: UpdateUserParam) -> int:
@@ -165,17 +205,21 @@ class CRUDUser(CRUDPlus[User]):
role_ids = obj.roles role_ids = obj.roles
del obj.roles del obj.roles
count = await self.update_model(db, user_id, obj) count = await self.update_model_by_column(db, obj, id=user_id, deleted=0)
user_role_stmt = delete(user_role).where(user_role.c.user_id == user_id) user_role_stmt = delete(user_role).where(user_role.c.user_id == user_id)
await db.execute(user_role_stmt) await db.execute(user_role_stmt)
if role_ids: if role_ids:
role_stmt = select(Role).where(Role.id.in_(role_ids)) role_stmt = select(Role).where(Role.id.in_(role_ids), Role.deleted == 0)
result = await db.execute(role_stmt) result = await db.execute(role_stmt)
roles = result.scalars().all() roles = result.scalars().all()
user_role_data = [AddUserRoleParam(user_id=user_id, role_id=role.id).model_dump() for role in roles] user_role_data = []
for role in roles:
role_dict = AddUserRoleParam(user_id=user_id, role_id=role.id).model_dump()
user_role_data.append(inject_tenant_dict(role_dict))
user_role_stmt = insert(user_role) user_role_stmt = insert(user_role)
await db.execute(user_role_stmt, user_role_data) await db.execute(user_role_stmt, user_role_data)
@@ -189,7 +233,7 @@ class CRUDUser(CRUDPlus[User]):
:param username: 用户名 :param username: 用户名
:return: :return:
""" """
return await self.update_model_by_column(db, {'last_login_time': timezone.now()}, username=username) return await self.update_model_by_column(db, {'last_login_time': timezone.now()}, username=username, deleted=0)
async def update_password_changed_time(self, db: AsyncSession, user_id: int) -> int: async def update_password_changed_time(self, db: AsyncSession, user_id: int) -> int:
""" """
@@ -199,7 +243,9 @@ class CRUDUser(CRUDPlus[User]):
:param user_id: 用户 ID :param user_id: 用户 ID
:return: :return:
""" """
return await self.update_model(db, user_id, {'last_password_changed_time': timezone.now()}) return await self.update_model_by_column(
db, {'last_password_changed_time': timezone.now()}, id=user_id, deleted=0
)
async def update_nickname(self, db: AsyncSession, user_id: int, nickname: str) -> int: async def update_nickname(self, db: AsyncSession, user_id: int, nickname: str) -> int:
""" """
@@ -210,7 +256,7 @@ class CRUDUser(CRUDPlus[User]):
:param nickname: 用户昵称 :param nickname: 用户昵称
:return: :return:
""" """
return await self.update_model(db, user_id, {'nickname': nickname}) return await self.update_model_by_column(db, {'nickname': nickname}, id=user_id, deleted=0)
async def update_avatar(self, db: AsyncSession, user_id: int, avatar: str) -> int: async def update_avatar(self, db: AsyncSession, user_id: int, avatar: str) -> int:
""" """
@@ -221,7 +267,7 @@ class CRUDUser(CRUDPlus[User]):
:param avatar: 头像地址 :param avatar: 头像地址
:return: :return:
""" """
return await self.update_model(db, user_id, {'avatar': avatar}) return await self.update_model_by_column(db, {'avatar': avatar}, id=user_id, deleted=0)
async def update_email(self, db: AsyncSession, user_id: int, email: str) -> int: async def update_email(self, db: AsyncSession, user_id: int, email: str) -> int:
""" """
@@ -232,7 +278,7 @@ class CRUDUser(CRUDPlus[User]):
:param email: 邮箱 :param email: 邮箱
:return: :return:
""" """
return await self.update_model(db, user_id, {'email': email}) return await self.update_model_by_column(db, {'email': email}, id=user_id, deleted=0)
async def reset_password(self, db: AsyncSession, pk: int, password: str) -> int: async def reset_password(self, db: AsyncSession, pk: int, password: str) -> int:
""" """
@@ -245,7 +291,7 @@ class CRUDUser(CRUDPlus[User]):
""" """
salt = bcrypt.gensalt() salt = bcrypt.gensalt()
new_pwd = get_hash_password(password, salt) new_pwd = get_hash_password(password, salt)
return await self.update_model(db, pk, {'password': new_pwd, 'salt': salt}, flush=True) return await self.update_model_by_column(db, {'password': new_pwd, 'salt': salt}, flush=True, id=pk, deleted=0)
async def set_super(self, db: AsyncSession, user_id: int, *, is_super: bool) -> int: async def set_super(self, db: AsyncSession, user_id: int, *, is_super: bool) -> int:
""" """
@@ -256,7 +302,7 @@ class CRUDUser(CRUDPlus[User]):
:param is_super: 是否超级管理员 :param is_super: 是否超级管理员
:return: :return:
""" """
return await self.update_model(db, user_id, {'is_superuser': is_super}) return await self.update_model_by_column(db, {'is_superuser': is_super}, id=user_id, deleted=0)
async def set_staff(self, db: AsyncSession, user_id: int, *, is_staff: bool) -> int: async def set_staff(self, db: AsyncSession, user_id: int, *, is_staff: bool) -> int:
""" """
@@ -267,7 +313,7 @@ class CRUDUser(CRUDPlus[User]):
:param is_staff: 是否可登录后台 :param is_staff: 是否可登录后台
:return: :return:
""" """
return await self.update_model(db, user_id, {'is_staff': is_staff}) return await self.update_model_by_column(db, {'is_staff': is_staff}, id=user_id, deleted=0)
async def set_status(self, db: AsyncSession, user_id: int, status: int) -> int: async def set_status(self, db: AsyncSession, user_id: int, status: int) -> int:
""" """
@@ -278,7 +324,7 @@ class CRUDUser(CRUDPlus[User]):
:param status: 状态 :param status: 状态
:return: :return:
""" """
return await self.update_model(db, user_id, {'status': status}) return await self.update_model_by_column(db, {'status': status}, id=user_id, deleted=0)
async def set_multi_login(self, db: AsyncSession, user_id: int, *, multi_login: bool) -> int: async def set_multi_login(self, db: AsyncSession, user_id: int, *, multi_login: bool) -> int:
""" """
@@ -289,7 +335,7 @@ class CRUDUser(CRUDPlus[User]):
:param multi_login: 是否允许多端登录 :param multi_login: 是否允许多端登录
:return: :return:
""" """
return await self.update_model(db, user_id, {'is_multi_login': multi_login}) return await self.update_model_by_column(db, {'is_multi_login': multi_login}, id=user_id, deleted=0)
async def delete(self, db: AsyncSession, user_id: int) -> int: async def delete(self, db: AsyncSession, user_id: int) -> int:
""" """
@@ -310,7 +356,16 @@ class CRUDUser(CRUDPlus[User]):
user_role_stmt = delete(user_role).where(user_role.c.user_id == user_id) user_role_stmt = delete(user_role).where(user_role.c.user_id == user_id)
await db.execute(user_role_stmt) await db.execute(user_role_stmt)
return await self.delete_model(db, user_id) return await self.delete_model_by_column(
db,
logical_deletion=True,
deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id=user_id,
deleted=0,
)
async def get_join( async def get_join(
self, self,
@@ -327,7 +382,7 @@ class CRUDUser(CRUDPlus[User]):
:param username: 用户名 :param username: 用户名
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if user_id: if user_id:
filters['id'] = user_id filters['id'] = user_id
@@ -337,15 +392,35 @@ class CRUDUser(CRUDPlus[User]):
result = await self.select_models( result = await self.select_models(
db, db,
join_conditions=[ join_conditions=[
JoinConfig(model=Dept, join_on=Dept.id == self.model.dept_id, fill_result=True), JoinConfig(
model=Dept,
join_on=and_(Dept.id == self.model.dept_id, Dept.deleted == 0),
fill_result=True,
),
JoinConfig(model=user_role, join_on=user_role.c.user_id == self.model.id), JoinConfig(model=user_role, join_on=user_role.c.user_id == self.model.id),
JoinConfig(model=Role, join_on=Role.id == user_role.c.role_id, fill_result=True), JoinConfig(
model=Role,
join_on=and_(Role.id == user_role.c.role_id, Role.deleted == 0),
fill_result=True,
),
JoinConfig(model=role_menu, join_on=role_menu.c.role_id == Role.id), JoinConfig(model=role_menu, join_on=role_menu.c.role_id == Role.id),
JoinConfig(model=Menu, join_on=Menu.id == role_menu.c.menu_id, fill_result=True), JoinConfig(
model=Menu,
join_on=and_(Menu.id == role_menu.c.menu_id, Menu.deleted == 0),
fill_result=True,
),
JoinConfig(model=role_data_scope, join_on=role_data_scope.c.role_id == Role.id), JoinConfig(model=role_data_scope, join_on=role_data_scope.c.role_id == Role.id),
JoinConfig(model=DataScope, join_on=DataScope.id == role_data_scope.c.data_scope_id, fill_result=True), JoinConfig(
model=DataScope,
join_on=and_(DataScope.id == role_data_scope.c.data_scope_id, DataScope.deleted == 0),
fill_result=True,
),
JoinConfig(model=data_scope_rule, join_on=data_scope_rule.c.data_scope_id == DataScope.id), JoinConfig(model=data_scope_rule, join_on=data_scope_rule.c.data_scope_id == DataScope.id),
JoinConfig(model=DataRule, join_on=DataRule.id == data_scope_rule.c.data_rule_id, fill_result=True), JoinConfig(
model=DataRule,
join_on=and_(DataRule.id == data_scope_rule.c.data_rule_id, DataRule.deleted == 0),
fill_result=True,
),
], ],
**filters, **filters,
) )
+5 -1
View File
@@ -9,9 +9,13 @@ class DataRule(Base):
"""数据规则表""" """数据规则表"""
__tablename__ = 'sys_data_rule' __tablename__ = 'sys_data_rule'
__table_args__ = (
sa.UniqueConstraint('name', 'deleted', name='uk_sys_data_rule_name_deleted'),
{'comment': '数据规则表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
name: Mapped[str] = mapped_column(sa.String(512), unique=True, comment='名称') name: Mapped[str] = mapped_column(sa.String(512), comment='名称')
model: Mapped[str] = mapped_column(sa.String(64), comment='模型名称') model: Mapped[str] = mapped_column(sa.String(64), comment='模型名称')
column: Mapped[str] = mapped_column(sa.String(32), comment='模型字段名') column: Mapped[str] = mapped_column(sa.String(32), comment='模型字段名')
operator: Mapped[int] = mapped_column(comment='运算符(0and、1or') operator: Mapped[int] = mapped_column(comment='运算符(0and、1or')
+5 -1
View File
@@ -9,7 +9,11 @@ class DataScope(Base):
"""数据范围表""" """数据范围表"""
__tablename__ = 'sys_data_scope' __tablename__ = 'sys_data_scope'
__table_args__ = (
sa.UniqueConstraint('name', 'deleted', name='uk_sys_data_scope_name_deleted'),
{'comment': '数据范围表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
name: Mapped[str] = mapped_column(sa.String(64), unique=True, comment='名称') name: Mapped[str] = mapped_column(sa.String(64), comment='名称')
status: Mapped[int] = mapped_column(default=1, comment='状态(0停用 1正常)') status: Mapped[int] = mapped_column(default=1, comment='状态(0停用 1正常)')
+14 -3
View File
@@ -2,14 +2,26 @@ import sqlalchemy as sa
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import Base, id_key from backend.common.model import Base, TenantMixin, id_key
from backend.core.conf import settings
class Dept(Base): class Dept(Base, TenantMixin):
"""部门表""" """部门表"""
__tablename__ = 'sys_dept' __tablename__ = 'sys_dept'
if settings.TENANT_ENABLED:
__table_args__ = (
sa.UniqueConstraint('name', 'tenant_id', 'deleted', name='uk_sys_dept_name_tenant_deleted'),
{'comment': '部门表'},
)
else:
__table_args__ = (
sa.UniqueConstraint('name', 'deleted', name='uk_sys_dept_name_deleted'),
{'comment': '部门表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
name: Mapped[str] = mapped_column(sa.String(64), comment='部门名称') name: Mapped[str] = mapped_column(sa.String(64), comment='部门名称')
sort: Mapped[int] = mapped_column(default=0, comment='排序') sort: Mapped[int] = mapped_column(default=0, comment='排序')
@@ -17,7 +29,6 @@ class Dept(Base):
phone: Mapped[str | None] = mapped_column(sa.String(11), default=None, comment='手机') phone: Mapped[str | None] = mapped_column(sa.String(11), default=None, comment='手机')
email: Mapped[str | None] = mapped_column(sa.String(64), default=None, comment='邮箱') email: Mapped[str | None] = mapped_column(sa.String(64), default=None, comment='邮箱')
status: Mapped[int] = mapped_column(default=1, comment='部门状态(0停用 1正常)') status: Mapped[int] = mapped_column(default=1, comment='部门状态(0停用 1正常)')
del_flag: Mapped[bool] = mapped_column(default=False, comment='删除标志(0删除 1存在)')
# 父级部门 # 父级部门
parent_id: Mapped[int | None] = mapped_column(sa.BigInteger, default=None, index=True, comment='父部门ID') parent_id: Mapped[int | None] = mapped_column(sa.BigInteger, default=None, index=True, comment='父部门ID')
+2 -2
View File
@@ -4,11 +4,11 @@ import sqlalchemy as sa
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import DataClassBase, TimeZone, UniversalText, id_key from backend.common.model import DataClassBase, TenantMixin, TimeZone, UniversalText, id_key
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
class LoginLog(DataClassBase): class LoginLog(DataClassBase, TenantMixin):
"""登录日志表""" """登录日志表"""
__tablename__ = 'sys_login_log' __tablename__ = 'sys_login_log'
+11
View File
@@ -1,6 +1,14 @@
import sqlalchemy as sa import sqlalchemy as sa
from backend.common.model import MappedBase from backend.common.model import MappedBase
from backend.core.conf import settings
# 租户列定义(根据配置决定是否添加)
_tenant_columns = (
(lambda: [sa.Column('tenant_id', sa.BigInteger, nullable=False, index=True, comment='租户ID')])
if settings.TENANT_ENABLED
else list
)
# 用户角色表 # 用户角色表
user_role = sa.Table( user_role = sa.Table(
@@ -9,6 +17,7 @@ user_role = sa.Table(
sa.Column('id', sa.BigInteger, primary_key=True, unique=True, index=True, autoincrement=True, comment='主键ID'), sa.Column('id', sa.BigInteger, primary_key=True, unique=True, index=True, autoincrement=True, comment='主键ID'),
sa.Column('user_id', sa.BigInteger, primary_key=True, comment='用户ID'), sa.Column('user_id', sa.BigInteger, primary_key=True, comment='用户ID'),
sa.Column('role_id', sa.BigInteger, primary_key=True, comment='角色ID'), sa.Column('role_id', sa.BigInteger, primary_key=True, comment='角色ID'),
*_tenant_columns(),
) )
# 角色菜单表 # 角色菜单表
@@ -18,6 +27,7 @@ role_menu = sa.Table(
sa.Column('id', sa.BigInteger, primary_key=True, unique=True, index=True, autoincrement=True, comment='主键ID'), sa.Column('id', sa.BigInteger, primary_key=True, unique=True, index=True, autoincrement=True, comment='主键ID'),
sa.Column('role_id', sa.BigInteger, primary_key=True, comment='角色ID'), sa.Column('role_id', sa.BigInteger, primary_key=True, comment='角色ID'),
sa.Column('menu_id', sa.BigInteger, primary_key=True, comment='菜单ID'), sa.Column('menu_id', sa.BigInteger, primary_key=True, comment='菜单ID'),
*_tenant_columns(),
) )
# 角色数据范围表 # 角色数据范围表
@@ -27,6 +37,7 @@ role_data_scope = sa.Table(
sa.Column('id', sa.BigInteger, primary_key=True, unique=True, index=True, autoincrement=True, comment='主键 ID'), sa.Column('id', sa.BigInteger, primary_key=True, unique=True, index=True, autoincrement=True, comment='主键 ID'),
sa.Column('role_id', sa.BigInteger, primary_key=True, comment='角色 ID'), sa.Column('role_id', sa.BigInteger, primary_key=True, comment='角色 ID'),
sa.Column('data_scope_id', sa.BigInteger, primary_key=True, comment='数据范围 ID'), sa.Column('data_scope_id', sa.BigInteger, primary_key=True, comment='数据范围 ID'),
*_tenant_columns(),
) )
# 数据范围规则表 # 数据范围规则表
+2 -2
View File
@@ -4,11 +4,11 @@ import sqlalchemy as sa
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import DataClassBase, TimeZone, UniversalText, id_key from backend.common.model import DataClassBase, TenantMixin, TimeZone, UniversalText, id_key
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
class OperaLog(DataClassBase): class OperaLog(DataClassBase, TenantMixin):
"""操作日志表""" """操作日志表"""
__tablename__ = 'sys_opera_log' __tablename__ = 'sys_opera_log'
+15 -3
View File
@@ -2,16 +2,28 @@ import sqlalchemy as sa
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import Base, UniversalText, id_key from backend.common.model import Base, TenantMixin, UniversalText, id_key
from backend.core.conf import settings
class Role(Base): class Role(Base, TenantMixin):
"""角色表""" """角色表"""
__tablename__ = 'sys_role' __tablename__ = 'sys_role'
if settings.TENANT_ENABLED:
__table_args__ = (
sa.UniqueConstraint('name', 'tenant_id', 'deleted', name='uk_sys_role_name_tenant_deleted'),
{'comment': '角色表'},
)
else:
__table_args__ = (
sa.UniqueConstraint('name', 'deleted', name='uk_sys_role_name_deleted'),
{'comment': '角色表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
name: Mapped[str] = mapped_column(sa.String(32), unique=True, comment='角色名称') name: Mapped[str] = mapped_column(sa.String(32), comment='角色名称')
status: Mapped[int] = mapped_column(default=1, comment='角色状态(0停用 1正常)') status: Mapped[int] = mapped_column(default=1, comment='角色状态(0停用 1正常)')
is_filter_scopes: Mapped[bool] = mapped_column(default=True, comment='过滤数据权限(0否 1是)') is_filter_scopes: Mapped[bool] = mapped_column(default=True, comment='过滤数据权限(0否 1是)')
remark: Mapped[str | None] = mapped_column(UniversalText, default=None, comment='备注') remark: Mapped[str | None] = mapped_column(UniversalText, default=None, comment='备注')
+19 -5
View File
@@ -4,23 +4,37 @@ import sqlalchemy as sa
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import Base, TimeZone, id_key from backend.common.model import Base, TenantMixin, TimeZone, id_key
from backend.core.conf import settings
from backend.database.db import uuid4_str from backend.database.db import uuid4_str
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
class User(Base): class User(Base, TenantMixin):
"""用户表""" """用户表"""
__tablename__ = 'sys_user' __tablename__ = 'sys_user'
if settings.TENANT_ENABLED:
__table_args__ = (
sa.UniqueConstraint('username', 'tenant_id', 'deleted', name='uk_sys_user_username_tenant_deleted'),
sa.UniqueConstraint('email', 'tenant_id', 'deleted', name='uk_sys_user_email_tenant_deleted'),
{'comment': '用户表'},
)
else:
__table_args__ = (
sa.UniqueConstraint('username', 'deleted', name='uk_sys_user_username_deleted'),
sa.UniqueConstraint('email', 'deleted', name='uk_sys_user_email_deleted'),
{'comment': '用户表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
uuid: Mapped[str] = mapped_column(sa.String(64), init=False, default_factory=uuid4_str, unique=True) uuid: Mapped[str] = mapped_column(sa.String(64), init=False, default_factory=uuid4_str, unique=True)
username: Mapped[str] = mapped_column(sa.String(64), unique=True, index=True, comment='用户名') username: Mapped[str] = mapped_column(sa.String(64), index=True, comment='用户名')
nickname: Mapped[str] = mapped_column(sa.String(64), comment='昵称') nickname: Mapped[str] = mapped_column(sa.String(64), comment='昵称')
password: Mapped[str | None] = mapped_column(sa.String(256), comment='密码') password: Mapped[str | None] = mapped_column(sa.String(256), comment='密码')
salt: Mapped[bytes | None] = mapped_column(sa.LargeBinary(255), comment='加密盐') salt: Mapped[bytes | None] = mapped_column(sa.LargeBinary(256), comment='加密盐')
email: Mapped[str | None] = mapped_column(sa.String(256), default=None, unique=True, index=True, comment='邮箱') email: Mapped[str | None] = mapped_column(sa.String(256), default=None, index=True, comment='邮箱')
phone: Mapped[str | None] = mapped_column(sa.String(11), default=None, comment='手机号') phone: Mapped[str | None] = mapped_column(sa.String(11), default=None, comment='手机号')
avatar: Mapped[str | None] = mapped_column(sa.String(256), default=None, comment='头像') avatar: Mapped[str | None] = mapped_column(sa.String(256), default=None, comment='头像')
status: Mapped[int] = mapped_column(default=1, index=True, comment='用户账号状态(0停用 1正常)') status: Mapped[int] = mapped_column(default=1, index=True, comment='用户账号状态(0停用 1正常)')
@@ -4,11 +4,11 @@ import sqlalchemy as sa
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import DataClassBase, TimeZone, id_key from backend.common.model import DataClassBase, TenantMixin, TimeZone, id_key
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
class UserPasswordHistory(DataClassBase): class UserPasswordHistory(DataClassBase, TenantMixin):
"""用户密码历史记录表""" """用户密码历史记录表"""
__tablename__ = 'sys_user_password_history' __tablename__ = 'sys_user_password_history'
+2 -1
View File
@@ -32,9 +32,10 @@ class GetDeptDetail(DeptSchemaBase):
model_config = ConfigDict(from_attributes=True) model_config = ConfigDict(from_attributes=True)
id: int = Field(description='部门 ID') id: int = Field(description='部门 ID')
del_flag: bool = Field(description='是否删除') deleted: int = Field(description='是否删除0:否;id:是)')
created_time: datetime = Field(description='创建时间') created_time: datetime = Field(description='创建时间')
updated_time: datetime | None = Field(None, description='更新时间') updated_time: datetime | None = Field(None, description='更新时间')
deleted_time: datetime | None = Field(None, description='删除时间')
class GetDeptTree(GetDeptDetail): class GetDeptTree(GetDeptDetail):
+7
View File
@@ -8,6 +8,7 @@ from backend.app.admin.schema.dept import GetDeptDetail
from backend.app.admin.schema.role import GetRoleWithRelationDetail from backend.app.admin.schema.role import GetRoleWithRelationDetail
from backend.common.enums import StatusType from backend.common.enums import StatusType
from backend.common.schema import CustomEmailStr, CustomPhoneNumber, SchemaBase, ser_string from backend.common.schema import CustomEmailStr, CustomPhoneNumber, SchemaBase, ser_string
from backend.core.conf import settings
class AuthSchemaBase(SchemaBase): class AuthSchemaBase(SchemaBase):
@@ -20,6 +21,9 @@ class AuthSchemaBase(SchemaBase):
class AuthLoginParam(AuthSchemaBase): class AuthLoginParam(AuthSchemaBase):
"""用户登录参数""" """用户登录参数"""
if settings.TENANT_ENABLED:
tenant_id: int = Field(description='租户 ID')
uuid: str | None = Field(None, description='验证码 UUID') uuid: str | None = Field(None, description='验证码 UUID')
captcha: str | None = Field(None, description='验证码') captcha: str | None = Field(None, description='验证码')
@@ -80,6 +84,9 @@ class GetUserInfoDetail(UserInfoSchemaBase):
model_config = ConfigDict(from_attributes=True) model_config = ConfigDict(from_attributes=True)
if settings.TENANT_ENABLED:
tenant_id: int = Field(description='租户 ID')
dept_id: int | None = Field(None, description='部门 ID') dept_id: int | None = Field(None, description='部门 ID')
id: int = Field(description='用户 ID') id: int = Field(description='用户 ID')
uuid: str = Field(description='用户 UUID') uuid: str = Field(description='用户 UUID')
+25 -9
View File
@@ -12,12 +12,13 @@ from backend.app.admin.service.login_log_service import login_log_service
from backend.app.admin.service.user_password_history_service import password_security_service from backend.app.admin.service.user_password_history_service import password_security_service
from backend.app.admin.utils.password_security import password_verify from backend.app.admin.utils.password_security import password_verify
from backend.common.context import ctx from backend.common.context import ctx
from backend.common.enums import LoginLogStatusType from backend.common.enums import LoginLogStatusType, StatusType
from backend.common.exception import errors from backend.common.exception import errors
from backend.common.i18n import t from backend.common.i18n import t
from backend.common.log import log from backend.common.log import log
from backend.common.response.response_code import CustomErrorCode from backend.common.response.response_code import CustomErrorCode
from backend.common.security.jwt import ( from backend.common.security.jwt import (
check_tenant_status,
create_access_token, create_access_token,
create_new_token, create_new_token,
create_refresh_token, create_refresh_token,
@@ -74,13 +75,14 @@ class AuthService:
await user_dao.update_login_time(db, obj.username) await user_dao.update_login_time(db, obj.username)
access_token_data = await create_access_token( access_token_data = await create_access_token(
user.id, user.id,
ctx.tenant_id,
multi_login=user.is_multi_login, multi_login=user.is_multi_login,
# extra info # extra info
swagger=True, swagger=True,
) )
return access_token_data.access_token, user return access_token_data.access_token, user
async def login( async def login( # noqa: C901
self, self,
*, *,
db: AsyncSession, db: AsyncSession,
@@ -110,11 +112,21 @@ class AuthService:
raise errors.CustomError(error=CustomErrorCode.CAPTCHA_ERROR) raise errors.CustomError(error=CustomErrorCode.CAPTCHA_ERROR)
await redis_client.delete(f'{settings.LOGIN_CAPTCHA_REDIS_PREFIX}:{obj.uuid}') await redis_client.delete(f'{settings.LOGIN_CAPTCHA_REDIS_PREFIX}:{obj.uuid}')
if settings.TENANT_ENABLED:
if obj.tenant_id is None:
raise errors.RequestError(msg='租户 ID 不能为空')
ctx.tenant_id = obj.tenant_id
await check_tenant_status(db, ctx.tenant_id)
else:
# 登录前先写入当前租户,供后续登录请求流程使用
ctx.tenant_id = settings.TENANT_DEFAULT_ID
user, days_remaining = await self.user_verify(db, obj.username, obj.password) user, days_remaining = await self.user_verify(db, obj.username, obj.password)
await user_dao.update_login_time(db, obj.username) await user_dao.update_login_time(db, obj.username)
await db.refresh(user) await db.refresh(user)
access_token_data = await create_access_token( access_token_data = await create_access_token(
user.id, user.id,
ctx.tenant_id,
multi_login=user.is_multi_login, multi_login=user.is_multi_login,
# extra info # extra info
username=user.username, username=user.username,
@@ -128,6 +140,7 @@ class AuthService:
refresh_token_data = await create_refresh_token( refresh_token_data = await create_refresh_token(
access_token_data.session_uuid, access_token_data.session_uuid,
user.id, user.id,
ctx.tenant_id,
multi_login=user.is_multi_login, multi_login=user.is_multi_login,
) )
response.set_cookie( response.set_cookie(
@@ -186,14 +199,14 @@ class AuthService:
if request.user.is_superuser: if request.user.is_superuser:
menus = await menu_dao.get_all(db, None, None) menus = await menu_dao.get_all(db, None, None)
for menu in menus: for menu in menus:
if menu.perms: if menu.status == StatusType.enable and menu.perms:
codes.update(menu.perms.split(',')) codes.update(menu.perms.split(','))
else: else:
roles = request.user.roles roles = [role for role in request.user.roles if role.status == StatusType.enable]
if roles: if roles:
for role in roles: for role in roles:
for menu in role.menus: for menu in role.menus:
if menu.perms: if menu.status == StatusType.enable and menu.perms:
codes.update(menu.perms.split(',')) codes.update(menu.perms.split(','))
return list(codes) return list(codes)
@@ -213,21 +226,24 @@ class AuthService:
raise errors.RequestError(msg='Refresh Token 已过期,请重新登录') raise errors.RequestError(msg='Refresh Token 已过期,请重新登录')
token_payload = jwt_decode(refresh_token) token_payload = jwt_decode(refresh_token)
ctx.tenant_id = token_payload.tenant_id
user = await user_dao.get(db, token_payload.user_id) user = await user_dao.get(db, token_payload.user_id)
if not user: if not user:
raise errors.NotFoundError(msg='用户不存在') raise errors.NotFoundError(msg='用户不存在')
if not user.status: if not user.status:
raise errors.AuthorizationError(msg='用户已被锁定, 请联系统管理员') raise errors.AuthorizationError(msg='用户已被锁定, 请联系统管理员')
await check_tenant_status(db, ctx.tenant_id)
token_keys = await redis_client.get_by_prefix(f'{settings.TOKEN_REDIS_PREFIX}:{user.id}')
if not user.is_multi_login and [ if not user.is_multi_login and [
key key for key in token_keys if not key.endswith(f':{token_payload.session_uuid}')
for key in await redis_client.get_prefix(f'{settings.TOKEN_REDIS_PREFIX}:{user.id}:*')
if not key.endswith(f':{token_payload.session_uuid}')
]: ]:
raise errors.ForbiddenError(msg='此用户已在异地登录,请重新登录并及时修改密码') raise errors.ForbiddenError(msg='此用户已在异地登录,请重新登录并及时修改密码')
new_token = await create_new_token( new_token = await create_new_token(
refresh_token, refresh_token,
token_payload.session_uuid, token_payload.session_uuid,
user.id, user.id,
ctx.tenant_id,
multi_login=user.is_multi_login, multi_login=user.is_multi_login,
# extra info # extra info
username=user.username, username=user.username,
@@ -121,9 +121,9 @@ class DataScopeService:
data_scope = await data_scope_dao.get(db, pk) data_scope = await data_scope_dao.get(db, pk)
if not data_scope: if not data_scope:
raise errors.NotFoundError(msg='数据范围不存在') raise errors.NotFoundError(msg='数据范围不存在')
for rule_id in rule_ids.rules: if rule_ids.rules:
rule = await data_rule_dao.get(db, rule_id) rules = await data_rule_dao.get_all_by_ids(db, list(set(rule_ids.rules)))
if not rule: if {rule.id for rule in rules} != set(rule_ids.rules):
raise errors.NotFoundError(msg='数据规则不存在') raise errors.NotFoundError(msg='数据规则不存在')
count = await data_scope_dao.update_rules(db, pk, rule_ids) count = await data_scope_dao.update_rules(db, pk, rule_ids)
await user_cache_manager.clear_by_data_scope_id(db, [pk]) await user_cache_manager.clear_by_data_scope_id(db, [pk])
+16 -16
View File
@@ -48,22 +48,22 @@ class LoginLogService:
:return: :return:
""" """
try: try:
obj = CreateLoginLogParam( data = {
user_uuid=user_uuid, 'user_uuid': user_uuid,
username=username, 'username': username,
status=status, 'status': status,
ip=ctx.ip, 'ip': ctx.ip,
country=ctx.country, 'country': ctx.country,
region=ctx.region, 'region': ctx.region,
city=ctx.city, 'city': ctx.city,
user_agent=ctx.user_agent, 'user_agent': ctx.user_agent,
browser=ctx.browser, 'browser': ctx.browser,
os=ctx.os, 'os': ctx.os,
device=ctx.device, 'device': ctx.device,
msg=msg, 'msg': msg,
login_time=login_time, 'login_time': login_time,
) }
# 为后台任务创建独立数据库会话 obj = CreateLoginLogParam(**data)
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
await login_log_dao.create(db, obj) await login_log_dao.create(db, obj)
except Exception as e: except Exception as e:
+2 -1
View File
@@ -7,6 +7,7 @@ from backend.app.admin.crud.crud_menu import menu_dao
from backend.app.admin.model import Menu from backend.app.admin.model import Menu
from backend.app.admin.schema.menu import CreateMenuParam, UpdateMenuParam from backend.app.admin.schema.menu import CreateMenuParam, UpdateMenuParam
from backend.app.admin.utils.cache import user_cache_manager from backend.app.admin.utils.cache import user_cache_manager
from backend.common.enums import StatusType
from backend.common.exception import errors from backend.common.exception import errors
from backend.utils.build_tree import get_tree_data, get_vben5_tree_data from backend.utils.build_tree import get_tree_data, get_vben5_tree_data
@@ -57,7 +58,7 @@ class MenuService:
if request.user.is_superuser: if request.user.is_superuser:
menu_data = await menu_dao.get_sidebar(db, None) menu_data = await menu_dao.get_sidebar(db, None)
else: else:
roles = request.user.roles roles = [role for role in request.user.roles if role.status == StatusType.enable]
menu_ids = set() menu_ids = set()
if roles: if roles:
for role in roles: for role in roles:
+9 -4
View File
@@ -13,6 +13,7 @@ from backend.common.exception import errors
from backend.core.conf import settings from backend.core.conf import settings
from backend.core.path_conf import PLUGIN_DIR from backend.core.path_conf import PLUGIN_DIR
from backend.database.redis import redis_client from backend.database.redis import redis_client
from backend.plugin.core import get_required_plugins
from backend.plugin.installer import install_git_plugin, install_zip_plugin, remove_plugin, zip_plugin from backend.plugin.installer import install_git_plugin, install_zip_plugin, remove_plugin, zip_plugin
from backend.plugin.requirements import uninstall_requirements_async from backend.plugin.requirements import uninstall_requirements_async
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
@@ -26,12 +27,13 @@ class PluginService:
"""获取所有插件""" """获取所有插件"""
changed_key = f'{settings.PLUGIN_REDIS_PREFIX}:changed' changed_key = f'{settings.PLUGIN_REDIS_PREFIX}:changed'
keys = [key async for key in redis_client.scan_iter(f'{settings.PLUGIN_REDIS_PREFIX}:*') if key != changed_key] keys = [key for key in await redis_client.get_by_prefix(settings.PLUGIN_REDIS_PREFIX) if key != changed_key]
if not keys: if not keys:
return [] return []
result = [] result = []
for info in await redis_client.mget(*keys): plugin_infos = await redis_client.mget(*keys)
for info in plugin_infos:
if info is None: if info is None:
continue continue
@@ -76,6 +78,8 @@ class PluginService:
""" """
if settings.ENVIRONMENT != 'dev': if settings.ENVIRONMENT != 'dev':
raise errors.RequestError(msg='禁止在非开发环境下卸载插件') raise errors.RequestError(msg='禁止在非开发环境下卸载插件')
if plugin in get_required_plugins():
raise errors.RequestError(msg=f'插件 {plugin} 为必需插件,禁止卸载')
plugin_dir = anyio.Path(PLUGIN_DIR / plugin) plugin_dir = anyio.Path(PLUGIN_DIR / plugin)
if not await plugin_dir.exists(): if not await plugin_dir.exists():
raise errors.NotFoundError(msg='插件不存在') raise errors.NotFoundError(msg='插件不存在')
@@ -94,7 +98,8 @@ class PluginService:
:param plugin: 插件名称 :param plugin: 插件名称
:return: :return:
""" """
plugin_info = await redis_client.get(f'{settings.PLUGIN_REDIS_PREFIX}:{plugin}') plugin_key = f'{settings.PLUGIN_REDIS_PREFIX}:{plugin}'
plugin_info = await redis_client.get(plugin_key)
if not plugin_info: if not plugin_info:
raise errors.NotFoundError(msg='插件不存在') raise errors.NotFoundError(msg='插件不存在')
plugin_info = json.loads(plugin_info) plugin_info = json.loads(plugin_info)
@@ -106,7 +111,7 @@ class PluginService:
else str(StatusType.disable.value) else str(StatusType.disable.value)
) )
plugin_info['plugin']['enable'] = new_status plugin_info['plugin']['enable'] = new_status
await redis_client.set(f'{settings.PLUGIN_REDIS_PREFIX}:{plugin}', json.dumps(plugin_info, ensure_ascii=False)) await redis_client.set(plugin_key, json.dumps(plugin_info, ensure_ascii=False))
await redis_client.set(f'{settings.PLUGIN_REDIS_PREFIX}:changed', 'true') await redis_client.set(f'{settings.PLUGIN_REDIS_PREFIX}:changed', 'true')
@staticmethod @staticmethod
+6 -6
View File
@@ -145,9 +145,9 @@ class RoleService:
role = await role_dao.get(db, pk) role = await role_dao.get(db, pk)
if not role: if not role:
raise errors.NotFoundError(msg='角色不存在') raise errors.NotFoundError(msg='角色不存在')
for menu_id in menu_ids.menus: if menu_ids.menus:
menu = await menu_dao.get(db, menu_id) menus = await menu_dao.get_all_by_ids(db, list(set(menu_ids.menus)))
if not menu: if {menu.id for menu in menus} != set(menu_ids.menus):
raise errors.NotFoundError(msg='菜单不存在') raise errors.NotFoundError(msg='菜单不存在')
count = await role_dao.update_menus(db, pk, menu_ids) count = await role_dao.update_menus(db, pk, menu_ids)
await user_cache_manager.clear_by_role_id(db, [pk]) await user_cache_manager.clear_by_role_id(db, [pk])
@@ -167,9 +167,9 @@ class RoleService:
role = await role_dao.get(db, pk) role = await role_dao.get(db, pk)
if not role: if not role:
raise errors.NotFoundError(msg='角色不存在') raise errors.NotFoundError(msg='角色不存在')
for scope_id in scope_ids.scopes: if scope_ids.scopes:
scope = await data_scope_dao.get(db, scope_id) scopes = await data_scope_dao.get_all_by_ids(db, list(set(scope_ids.scopes)))
if not scope: if {scope.id for scope in scopes} != set(scope_ids.scopes):
raise errors.NotFoundError(msg='数据范围不存在') raise errors.NotFoundError(msg='数据范围不存在')
count = await role_dao.update_scopes(db, pk, scope_ids) count = await role_dao.update_scopes(db, pk, scope_ids)
await user_cache_manager.clear_by_role_id(db, [pk]) await user_cache_manager.clear_by_role_id(db, [pk])
@@ -57,18 +57,18 @@ class UserPasswordHistoryService:
failure_count = await redis_client.get(f'{settings.LOGIN_FAILURE_PREFIX}:{user_id}') failure_count = await redis_client.get(f'{settings.LOGIN_FAILURE_PREFIX}:{user_id}')
failure_count = int(failure_count) if failure_count else 0 failure_count = int(failure_count) if failure_count else 0
failure_count += 1 failure_count += 1
await redis_client.setex( await redis_client.set(
f'{settings.LOGIN_FAILURE_PREFIX}:{user_id}', f'{settings.LOGIN_FAILURE_PREFIX}:{user_id}',
settings.USER_LOCK_SECONDS,
str(failure_count), str(failure_count),
ex=settings.USER_LOCK_SECONDS,
) )
if failure_count >= settings.USER_LOCK_THRESHOLD: if failure_count >= settings.USER_LOCK_THRESHOLD:
locked_until = timezone.now() + timedelta(seconds=settings.USER_LOCK_SECONDS) locked_until = timezone.now() + timedelta(seconds=settings.USER_LOCK_SECONDS)
await redis_client.setex( await redis_client.set(
f'{settings.USER_LOCK_REDIS_PREFIX}:{user_id}', f'{settings.USER_LOCK_REDIS_PREFIX}:{user_id}',
settings.USER_LOCK_SECONDS,
timezone.to_str(locked_until), timezone.to_str(locked_until),
ex=settings.USER_LOCK_SECONDS,
) )
raise errors.AuthorizationError(msg='登录失败次数过多,账号已被锁定') raise errors.AuthorizationError(msg='登录失败次数过多,账号已被锁定')
+28 -31
View File
@@ -90,12 +90,15 @@ class UserService:
""" """
if await user_dao.get_by_username(db, obj.username): if await user_dao.get_by_username(db, obj.username):
raise errors.ConflictError(msg='用户名已注册') raise errors.ConflictError(msg='用户名已注册')
if obj.email and await user_dao.check_email(db, obj.email):
raise errors.ConflictError(msg='邮箱已被绑定')
if not obj.password: if not obj.password:
raise errors.RequestError(msg='密码不允许为空') raise errors.RequestError(msg='密码不允许为空')
if not await dept_dao.get(db, obj.dept_id): if not await dept_dao.get(db, obj.dept_id):
raise errors.NotFoundError(msg='部门不存在') raise errors.NotFoundError(msg='部门不存在')
for role_id in obj.roles: if obj.roles:
if not await role_dao.get(db, role_id): roles = await role_dao.get_all_by_ids(db, list(set(obj.roles)))
if {role.id for role in roles} != set(obj.roles):
raise errors.NotFoundError(msg='角色不存在') raise errors.NotFoundError(msg='角色不存在')
obj.nickname = obj.nickname or obj.username obj.nickname = obj.nickname or obj.username
await user_dao.add(db, obj) await user_dao.add(db, obj)
@@ -115,17 +118,22 @@ class UserService:
raise errors.NotFoundError(msg='用户不存在') raise errors.NotFoundError(msg='用户不存在')
if obj.username != user.username and await user_dao.get_by_username(db, obj.username): if obj.username != user.username and await user_dao.get_by_username(db, obj.username):
raise errors.ConflictError(msg='用户名已注册') raise errors.ConflictError(msg='用户名已注册')
if obj.email and obj.email != user.email:
email_user = await user_dao.check_email(db, obj.email)
if email_user:
raise errors.ConflictError(msg='邮箱已被绑定')
if obj.dept_id and obj.dept_id != user.dept_id and not await dept_dao.get(db, dept_id=obj.dept_id): if obj.dept_id and obj.dept_id != user.dept_id and not await dept_dao.get(db, dept_id=obj.dept_id):
raise errors.NotFoundError(msg='部门不存在') raise errors.NotFoundError(msg='部门不存在')
for role_id in obj.roles: if obj.roles:
if not await role_dao.get(db, role_id): roles = await role_dao.get_all_by_ids(db, list(set(obj.roles)))
if {role.id for role in roles} != set(obj.roles):
raise errors.NotFoundError(msg='角色不存在') raise errors.NotFoundError(msg='角色不存在')
count = await user_dao.update(db, user.id, obj) count = await user_dao.update(db, user.id, obj)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}') await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count return count
@staticmethod @staticmethod
async def update_permission(*, db: AsyncSession, request: Request, pk: int, type: UserPermissionType) -> int: # noqa: C901 async def update_permission(*, db: AsyncSession, request: Request, pk: int, type: UserPermissionType) -> int: # ruff:ignore[complex-structure]
""" """
更新用户权限 更新用户权限
@@ -170,15 +178,15 @@ class UserService:
# 系统管理员修改自身时,除当前 token 外,其他 token 失效 # 系统管理员修改自身时,除当前 token 外,其他 token 失效
if not new_multi_login: if not new_multi_login:
key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}' key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}'
await redis_client.delete_prefix( await redis_client.delete_by_prefix(
key_prefix, key_prefix,
exclude=f'{key_prefix}:{token_payload.session_uuid}', exclude_keys=f'{key_prefix}:{token_payload.session_uuid}',
) )
else: else:
# 系统管理员修改他人时,他人 token 全部失效 # 系统管理员修改他人时,他人 token 全部失效
if not new_multi_login: if not new_multi_login:
key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}' key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}'
await redis_client.delete_prefix(key_prefix) await redis_client.delete_by_prefix(key_prefix)
case _: case _:
raise errors.RequestError(msg='权限类型不存在') raise errors.RequestError(msg='权限类型不存在')
@@ -205,14 +213,9 @@ class UserService:
history_obj = CreateUserPasswordHistoryParam(user_id=user.id, password=user.password) history_obj = CreateUserPasswordHistoryParam(user_id=user.id, password=user.password)
await password_security_service.save_password_history(db, history_obj) await password_security_service.save_password_history(db, history_obj)
await user_dao.update_password_changed_time(db, user.id) await user_dao.update_password_changed_time(db, user.id)
await redis_client.delete_by_prefix(f'{settings.TOKEN_REDIS_PREFIX}:{user.id}')
key_prefix = [ await redis_client.delete_by_prefix(f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user.id}')
f'{settings.TOKEN_REDIS_PREFIX}:{user.id}', await redis_client.delete_by_prefix(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user.id}',
f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}',
]
for prefix in key_prefix:
await redis_client.delete_prefix(prefix)
return count return count
@staticmethod @staticmethod
@@ -259,6 +262,9 @@ class UserService:
raise errors.RequestError(msg='验证码已失效,请重新获取') raise errors.RequestError(msg='验证码已失效,请重新获取')
if captcha != captcha_code: if captcha != captcha_code:
raise errors.CustomError(error=CustomErrorCode.CAPTCHA_ERROR) raise errors.CustomError(error=CustomErrorCode.CAPTCHA_ERROR)
email_user = await user_dao.check_email(db, email)
if email_user and email_user.id != user_id:
raise errors.ConflictError(msg='邮箱已被绑定')
await redis_client.delete(f'{settings.EMAIL_CAPTCHA_REDIS_PREFIX}:{ctx.ip}') await redis_client.delete(f'{settings.EMAIL_CAPTCHA_REDIS_PREFIX}:{ctx.ip}')
count = await user_dao.update_email(db, user_id, email) count = await user_dao.update_email(db, user_id, email)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}') await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}')
@@ -288,14 +294,9 @@ class UserService:
history_obj = CreateUserPasswordHistoryParam(user_id=user.id, password=user.password) history_obj = CreateUserPasswordHistoryParam(user_id=user.id, password=user.password)
await password_security_service.save_password_history(db, history_obj) await password_security_service.save_password_history(db, history_obj)
await user_dao.update_password_changed_time(db, user.id) await user_dao.update_password_changed_time(db, user.id)
await redis_client.delete_by_prefix(f'{settings.TOKEN_REDIS_PREFIX}:{user_id}')
key_prefix = [ await redis_client.delete_by_prefix(f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}')
f'{settings.TOKEN_REDIS_PREFIX}:{user_id}', await redis_client.delete_by_prefix(f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}')
f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}',
f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}',
]
for prefix in key_prefix:
await redis_client.delete_prefix(prefix)
return count return count
@staticmethod @staticmethod
@@ -311,13 +312,9 @@ class UserService:
if not user: if not user:
raise errors.NotFoundError(msg='用户不存在') raise errors.NotFoundError(msg='用户不存在')
count = await user_dao.delete(db, user.id) count = await user_dao.delete(db, user.id)
key_prefix = [ await redis_client.delete_by_prefix(f'{settings.TOKEN_REDIS_PREFIX}:{user.id}')
f'{settings.TOKEN_REDIS_PREFIX}:{user.id}', await redis_client.delete_by_prefix(f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user.id}')
f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user.id}', await redis_client.delete_by_prefix(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}',
]
for key in key_prefix:
await redis_client.delete_prefix(key)
return count return count
+1 -1
View File
@@ -4,4 +4,4 @@ from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent.parent)) sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent.parent))
from .actions import * # noqa: F403 from .actions import * # ruff:ignore[undefined-local-with-import-star]
+1 -1
View File
@@ -5,7 +5,7 @@ from backend.common.socketio.server import sio
@sio.event @sio.event
async def task_worker_status(sid, data) -> None: # noqa: ANN001 async def task_worker_status(sid, data) -> None: # ruff:ignore[missing-type-function-argument]
"""任务 Worker 状态事件""" """任务 Worker 状态事件"""
worker = await run_in_threadpool(celery_app.control.ping) worker = await run_in_threadpool(celery_app.control.ping)
await sio.emit('task_worker_status', worker, sid) await sio.emit('task_worker_status', worker, sid)
+2 -3
View File
@@ -26,10 +26,9 @@ async def get_task_registered() -> ResponseSchemaModel[list[TaskRegisteredDetail
for task in tasks: for task in tasks:
task_ins = celery_app_tasks.get(task) task_ins = celery_app_tasks.get(task)
if task_ins: if task_ins:
task_doc = task_ins.__doc__ task_registered.append(TaskRegisteredDetail(name=task_ins.__doc__ or task, task=task))
task_registered.append({'name': task_doc or task_ins, 'task': task_ins})
else: else:
task_registered.append({'name': task, 'task': task}) task_registered.append(TaskRegisteredDetail(name=task, task=task))
return response_base.success(data=task_registered) return response_base.success(data=task_registered)
+14 -7
View File
@@ -7,20 +7,27 @@ import celery_aio_pool
from celery.signals import worker_process_init from celery.signals import worker_process_init
from opentelemetry.instrumentation.celery import CeleryInstrumentor from opentelemetry.instrumentation.celery import CeleryInstrumentor
from backend.app.task.tasks.beat import LOCAL_BEAT_SCHEDULE from backend.app.task.tasks.beat import get_local_beat_schedule
from backend.common.enums import DataBaseType from backend.common.enums import DataBaseType
from backend.common.observability.otel import init_resource, init_tracer from backend.common.observability.otel import init_resource, init_tracer
from backend.core.conf import settings from backend.core.conf import settings
from backend.core.path_conf import BASE_PATH from backend.core.path_conf import BASE_PATH
_celery_otel_initialized = False
@worker_process_init.connect(weak=False) @worker_process_init.connect(weak=False)
def init_celery_worker_tracing(*args, **kwargs) -> None: def init_celery_tracing(*args, **kwargs) -> None:
"""初始化 Celery 追踪""" """初始化 Celery 追踪"""
if settings.GRAFANA_METRICS_ENABLE: global _celery_otel_initialized
resource = init_resource('fba_celery_worker')
init_tracer(resource) if not settings.GRAFANA_METRICS_ENABLE or _celery_otel_initialized:
CeleryInstrumentor().instrument() return
resource = init_resource(settings.GRAFANA_CELERY_OTEL_SERVICE_NAME)
init_tracer(resource)
CeleryInstrumentor().instrument()
_celery_otel_initialized = True
def find_task_packages() -> list[str]: def find_task_packages() -> list[str]:
@@ -60,7 +67,7 @@ def init_celery() -> celery.Celery:
database_engine_options={'echo': settings.DATABASE_ECHO}, database_engine_options={'echo': settings.DATABASE_ECHO},
# result_expires=0, # result_expires=0,
# beat_sync_every=1, # beat_sync_every=1,
beat_schedule=LOCAL_BEAT_SCHEDULE, beat_schedule=get_local_beat_schedule(),
beat_scheduler='backend.app.task.utils.schedulers:DatabaseScheduler', beat_scheduler='backend.app.task.utils.schedulers:DatabaseScheduler',
task_cls='backend.app.task.tasks.base:TaskBase', task_cls='backend.app.task.tasks.base:TaskBase',
task_track_started=True, task_track_started=True,
+18 -8
View File
@@ -6,6 +6,7 @@ from sqlalchemy_crud_plus import CRUDPlus
from backend.app.task.model import TaskScheduler from backend.app.task.model import TaskScheduler
from backend.app.task.schema.scheduler import CreateTaskSchedulerParam, UpdateTaskSchedulerParam from backend.app.task.schema.scheduler import CreateTaskSchedulerParam, UpdateTaskSchedulerParam
from backend.utils.timezone import timezone
class CRUDTaskScheduler(CRUDPlus[TaskScheduler]): class CRUDTaskScheduler(CRUDPlus[TaskScheduler]):
@@ -20,7 +21,7 @@ class CRUDTaskScheduler(CRUDPlus[TaskScheduler]):
:param pk: 任务调度 ID :param pk: 任务调度 ID
:return: :return:
""" """
return await task_scheduler_dao.select_model(db, pk) return await task_scheduler_dao.select_model(db, pk, deleted=0)
async def get_all(self, db: AsyncSession) -> Sequence[TaskScheduler]: async def get_all(self, db: AsyncSession) -> Sequence[TaskScheduler]:
""" """
@@ -29,7 +30,7 @@ class CRUDTaskScheduler(CRUDPlus[TaskScheduler]):
:param db: 数据库会话 :param db: 数据库会话
:return: :return:
""" """
return await self.select_models(db) return await self.select_models(db, deleted=0)
async def get_select(self, name: str | None, type: int | None) -> Select: async def get_select(self, name: str | None, type: int | None) -> Select:
""" """
@@ -39,7 +40,7 @@ class CRUDTaskScheduler(CRUDPlus[TaskScheduler]):
:param type: 任务调度类型 :param type: 任务调度类型
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if name is not None: if name is not None:
filters['name__like'] = f'%{name}%' filters['name__like'] = f'%{name}%'
@@ -56,7 +57,7 @@ class CRUDTaskScheduler(CRUDPlus[TaskScheduler]):
:param name: 任务调度名称 :param name: 任务调度名称
:return: :return:
""" """
return await self.select_model_by_column(db, name=name) return await self.select_model_by_column(db, name=name, deleted=0)
async def create(self, db: AsyncSession, obj: CreateTaskSchedulerParam) -> None: async def create(self, db: AsyncSession, obj: CreateTaskSchedulerParam) -> None:
""" """
@@ -106,10 +107,19 @@ class CRUDTaskScheduler(CRUDPlus[TaskScheduler]):
:param pk: 任务调度 ID :param pk: 任务调度 ID
:return: :return:
""" """
task_scheduler = await self.get(db, pk) count = await self.delete_model_by_column(
await db.delete(task_scheduler) db,
TaskScheduler.no_changes = False logical_deletion=True,
return 1 deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id=pk,
deleted=0,
)
if count:
TaskScheduler.no_changes = False
return count
task_scheduler_dao: CRUDTaskScheduler = CRUDTaskScheduler(TaskScheduler) task_scheduler_dao: CRUDTaskScheduler = CRUDTaskScheduler(TaskScheduler)
+8 -8
View File
@@ -22,7 +22,7 @@ class DatabaseBackend(BaseBackend):
task_cls = Task task_cls = Task
taskset_cls = TaskSet taskset_cls = TaskSet
def __init__(self, dburi=None, engine_options=None, url=None, **kwargs) -> None: # noqa: ANN001 def __init__(self, dburi=None, engine_options=None, url=None, **kwargs) -> None: # ruff:ignore[missing-type-function-argument]
# The `url` argument was added later and is used by # The `url` argument was added later and is used by
# the app to set backend by url (celery.app.backends.by_url) # the app to set backend by url (celery.app.backends.by_url)
super().__init__(expires_type=maybe_timedelta, url=url, **kwargs) super().__init__(expires_type=maybe_timedelta, url=url, **kwargs)
@@ -52,14 +52,14 @@ class DatabaseBackend(BaseBackend):
self._create_tables() self._create_tables()
@property @property
def extended_result(self): # noqa: ANN201 def extended_result(self): # ruff:ignore[missing-return-type-undocumented-public-function]
return self.app.conf.find_value_for_key('extended', 'result') return self.app.conf.find_value_for_key('extended', 'result')
def _create_tables(self) -> None: def _create_tables(self) -> None:
"""Create the task and taskset tables.""" """Create the task and taskset tables."""
self.result_session() self.result_session()
def result_session(self, session_manager=None) -> Session: # noqa: ANN001 def result_session(self, session_manager=None) -> Session: # ruff:ignore[missing-type-function-argument]
if session_manager is None: if session_manager is None:
session_manager = self.session_manager session_manager = self.session_manager
return session_manager.session_factory( return session_manager.session_factory(
@@ -69,7 +69,7 @@ class DatabaseBackend(BaseBackend):
) )
@retry @retry
def _store_result(self, task_id, result, state, traceback=None, request=None, **kwargs) -> None: # noqa: ANN001 def _store_result(self, task_id, result, state, traceback=None, request=None, **kwargs) -> None: # ruff:ignore[missing-type-function-argument]
"""Store return value and state of an executed task.""" """Store return value and state of an executed task."""
session = self.result_session() session = self.result_session()
with session_cleanup(session): with session_cleanup(session):
@@ -84,7 +84,7 @@ class DatabaseBackend(BaseBackend):
self._update_result(task, result, state, traceback=traceback, request=request) self._update_result(task, result, state, traceback=traceback, request=request)
session.commit() session.commit()
def _update_result(self, task, result, state, traceback=None, request=None) -> None: # noqa: ANN001 def _update_result(self, task, result, state, traceback=None, request=None) -> None: # ruff:ignore[missing-type-function-argument]
meta = self._get_result_meta( meta = self._get_result_meta(
result=result, result=result,
state=state, state=state,
@@ -106,7 +106,7 @@ class DatabaseBackend(BaseBackend):
setattr(task, column, value) setattr(task, column, value)
@retry @retry
def _get_task_meta_for(self, task_id: str): # noqa: ANN202 def _get_task_meta_for(self, task_id: str): # ruff:ignore[missing-return-type-private-function]
"""Get task meta-data for a task by id.""" """Get task meta-data for a task by id."""
session = self.result_session() session = self.result_session()
with session_cleanup(session): with session_cleanup(session):
@@ -124,7 +124,7 @@ class DatabaseBackend(BaseBackend):
return self.meta_from_decoded(data) return self.meta_from_decoded(data)
@retry @retry
def _save_group(self, group_id: str, result: PickleType): # noqa: ANN202 def _save_group(self, group_id: str, result: PickleType): # ruff:ignore[missing-return-type-private-function]
"""Store the result of an executed group.""" """Store the result of an executed group."""
session = self.result_session() session = self.result_session()
with session_cleanup(session): with session_cleanup(session):
@@ -170,7 +170,7 @@ class DatabaseBackend(BaseBackend):
session.query(self.taskset_cls).filter(self.taskset_cls.date_done < (now - expires)).delete() session.query(self.taskset_cls).filter(self.taskset_cls.date_done < (now - expires)).delete()
session.commit() session.commit()
def __reduce__(self, args=(), kwargs=None): # noqa: ANN001, ANN204 def __reduce__(self, args=(), kwargs=None): # ruff:ignore[missing-type-function-argument, missing-return-type-special-method]
kwargs = kwargs or {} kwargs = kwargs or {}
kwargs.update({'dburi': self.url, 'expires': self.expires, 'engine_options': self.engine_options}) kwargs.update({'dburi': self.url, 'expires': self.expires, 'engine_options': self.engine_options})
return super().__reduce__(args, kwargs) return super().__reduce__(args, kwargs)
+1 -1
View File
@@ -1,2 +1,2 @@
from backend.app.task.model.result import TaskExtended as TaskResult # noqa: F401 from backend.app.task.model.result import TaskExtended as TaskResult # ruff:ignore[unused-import]
from backend.app.task.model.scheduler import TaskScheduler as TaskScheduler from backend.app.task.model.scheduler import TaskScheduler as TaskScheduler
+3 -3
View File
@@ -45,7 +45,7 @@ class Task(MappedBase):
return f'<Task {self.task_id} state: {self.status}>' return f'<Task {self.task_id} state: {self.status}>'
@classmethod @classmethod
def configure(cls, schema=None, name=None) -> None: # noqa: ANN001 def configure(cls, schema=None, name=None) -> None: # ruff:ignore[missing-type-function-argument]
cls.__table__.schema = schema cls.__table__.schema = schema
cls.id.default.schema = schema cls.id.default.schema = schema
cls.__table__.name = name or cls.__tablename__ cls.__table__.name = name or cls.__tablename__
@@ -88,7 +88,7 @@ class TaskSet(MappedBase):
result = sa.Column(PickleType, nullable=True) result = sa.Column(PickleType, nullable=True)
date_done = sa.Column(TimeZone, default=timezone.now, nullable=True) date_done = sa.Column(TimeZone, default=timezone.now, nullable=True)
def __init__(self, taskset_id, result) -> None: # noqa: ANN001 def __init__(self, taskset_id, result) -> None: # ruff:ignore[missing-type-function-argument]
self.taskset_id = taskset_id self.taskset_id = taskset_id
self.result = result self.result = result
@@ -103,7 +103,7 @@ class TaskSet(MappedBase):
return f'<TaskSet: {self.taskset_id}>' return f'<TaskSet: {self.taskset_id}>'
@classmethod @classmethod
def configure(cls, schema=None, name=None) -> None: # noqa: ANN001 def configure(cls, schema=None, name=None) -> None: # ruff:ignore[missing-type-function-argument]
cls.__table__.schema = schema cls.__table__.schema = schema
cls.id.default.schema = schema cls.id.default.schema = schema
cls.__table__.name = name or cls.__tablename__ cls.__table__.name = name or cls.__tablename__
+8 -4
View File
@@ -18,9 +18,13 @@ class TaskScheduler(Base):
"""任务调度表""" """任务调度表"""
__tablename__ = 'task_scheduler' __tablename__ = 'task_scheduler'
__table_args__ = (
sa.UniqueConstraint('name', 'deleted', name='uk_task_scheduler_name_deleted'),
{'comment': '任务调度表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
name: Mapped[str] = mapped_column(sa.String(64), unique=True, comment='任务名称') name: Mapped[str] = mapped_column(sa.String(64), comment='任务名称')
task: Mapped[str] = mapped_column(sa.String(256), comment='要运行的 Celery 任务') task: Mapped[str] = mapped_column(sa.String(256), comment='要运行的 Celery 任务')
args: Mapped[str | None] = mapped_column(sa.JSON(), comment='任务可接收的位置参数') args: Mapped[str | None] = mapped_column(sa.JSON(), comment='任务可接收的位置参数')
kwargs: Mapped[str | None] = mapped_column(sa.JSON(), comment='任务可接收的关键字参数') kwargs: Mapped[str | None] = mapped_column(sa.JSON(), comment='任务可接收的关键字参数')
@@ -43,12 +47,12 @@ class TaskScheduler(Base):
no_changes: bool = False no_changes: bool = False
@staticmethod @staticmethod
def before_insert_or_update(mapper, connection, target) -> None: # noqa: ANN001 def before_insert_or_update(mapper, connection, target) -> None: # ruff:ignore[missing-type-function-argument]
if target.expire_seconds is not None and target.expire_time: if target.expire_seconds is not None and target.expire_time:
raise errors.ConflictError(msg='expires 和 expire_seconds 只能设置一个') raise errors.ConflictError(msg='expires 和 expire_seconds 只能设置一个')
@classmethod @classmethod
def changed(cls, mapper, connection, target) -> None: # noqa: ANN001 def changed(cls, mapper, connection, target) -> None: # ruff:ignore[missing-type-function-argument]
if not target.no_changes: if not target.no_changes:
cls.update_changed(mapper, connection, target) cls.update_changed(mapper, connection, target)
@@ -58,7 +62,7 @@ class TaskScheduler(Base):
await redis_client.set(f'{settings.CELERY_REDIS_PREFIX}:last_update', timezone.to_str(now)) await redis_client.set(f'{settings.CELERY_REDIS_PREFIX}:last_update', timezone.to_str(now))
@classmethod @classmethod
def update_changed(cls, mapper, connection, target) -> None: # noqa: ANN001 def update_changed(cls, mapper, connection, target) -> None: # ruff:ignore[missing-type-function-argument]
asyncio.create_task(cls.update_changed_async()) asyncio.create_task(cls.update_changed_async())
+3 -3
View File
@@ -15,7 +15,7 @@ class TaskBase(Task):
autoretry_for = (SQLAlchemyError,) autoretry_for = (SQLAlchemyError,)
max_retries = settings.CELERY_TASK_MAX_RETRIES max_retries = settings.CELERY_TASK_MAX_RETRIES
async def before_start(self, task_id: str, args, kwargs) -> None: # noqa: ANN001 async def before_start(self, task_id: str, args, kwargs) -> None: # ruff:ignore[missing-type-function-argument]
""" """
任务开始前执行钩子 任务开始前执行钩子
@@ -24,7 +24,7 @@ class TaskBase(Task):
""" """
await task_notification(msg=f'任务 {task_id} 开始执行') await task_notification(msg=f'任务 {task_id} 开始执行')
async def on_success(self, retval: Any, task_id: str, args, kwargs) -> None: # noqa: ANN001 async def on_success(self, retval: Any, task_id: str, args, kwargs) -> None: # ruff:ignore[missing-type-function-argument]
""" """
任务成功后执行钩子 任务成功后执行钩子
@@ -34,7 +34,7 @@ class TaskBase(Task):
""" """
await task_notification(msg=f'任务 {task_id} 执行成功') await task_notification(msg=f'任务 {task_id} 执行成功')
def on_failure(self, exc: Exception, task_id: str, args, kwargs, einfo) -> None: # noqa: ANN001 def on_failure(self, exc: Exception, task_id: str, args, kwargs, einfo) -> None: # ruff:ignore[missing-type-function-argument]
""" """
任务失败后执行钩子 任务失败后执行钩子
+30 -25
View File
@@ -1,29 +1,34 @@
from typing import Any
from celery.schedules import schedule from celery.schedules import schedule
from backend.app.task.utils.tzcrontab import TzAwareCrontab from backend.app.task.utils.tzcrontab import TzAwareCrontab
# 参考:https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html
LOCAL_BEAT_SCHEDULE = { def get_local_beat_schedule() -> dict[str, dict[str, Any]]:
'测试同步任务': { """获取本地 Celery beat 任务配置"""
'task': 'task_demo', # 参考:https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html
'schedule': schedule(30), return {
}, '测试同步任务': {
'测试异步任务': { 'task': 'task_demo',
'task': 'task_demo_async', 'schedule': schedule(30),
'schedule': TzAwareCrontab('1'), },
}, '测试异步任务': {
'测试传参任务': { 'task': 'task_demo_async',
'task': 'task_demo_params', 'schedule': TzAwareCrontab('1'),
'schedule': TzAwareCrontab('1'), },
'args': ['你好,'], '测试传参任务': {
'kwargs': {'world': '世界'}, 'task': 'task_demo_params',
}, 'schedule': TzAwareCrontab('1'),
'清理操作日志': { 'args': ['你好,'],
'task': 'backend.app.task.tasks.db_log.tasks.delete_db_opera_log', 'kwargs': {'world': '世界'},
'schedule': TzAwareCrontab('0', '0', day_of_week='6'), },
}, '清理操作日志': {
'清理登录日志': { 'task': 'backend.app.task.tasks.db_log.tasks.delete_db_opera_log',
'task': 'backend.app.task.tasks.db_log.tasks.delete_db_login_log', 'schedule': TzAwareCrontab('0', '0', day_of_week='6'),
'schedule': TzAwareCrontab('0', '0', day_of_month='15'), },
}, '清理登录日志': {
} 'task': 'backend.app.task.tasks.db_log.tasks.delete_db_login_log',
'schedule': TzAwareCrontab('0', '0', day_of_month='15'),
},
}
+27 -20
View File
@@ -6,7 +6,7 @@ import math
from datetime import datetime, timedelta from datetime import datetime, timedelta
from multiprocessing.util import Finalize from multiprocessing.util import Finalize
from typing import TYPE_CHECKING from typing import TYPE_CHECKING, Final
from celery import current_app, schedules from celery import current_app, schedules
from celery.beat import ScheduleEntry, Scheduler from celery.beat import ScheduleEntry, Scheduler
@@ -31,10 +31,10 @@ if TYPE_CHECKING:
from redis.asyncio.lock import Lock from redis.asyncio.lock import Lock
# 此计划程序必须比常规的 5 分钟更频繁地唤醒,因为它需要考虑对计划的外部更改 # 此计划程序必须比常规的 5 分钟更频繁地唤醒,因为它需要考虑对计划的外部更改
DEFAULT_MAX_INTERVAL = 5 # seconds _DEFAULT_MAX_INTERVAL: Final = 5 # seconds
# 计划锁时长,避免重复创建 # 计划锁时长,避免重复创建
DEFAULT_MAX_LOCK_TIMEOUT = DEFAULT_MAX_INTERVAL * 5 # seconds _DEFAULT_MAX_LOCK_TIMEOUT: Final = _DEFAULT_MAX_INTERVAL * 5 # seconds
logger = get_logger('fba.schedulers') logger = get_logger('fba.schedulers')
@@ -42,7 +42,7 @@ logger = get_logger('fba.schedulers')
class ModelEntry(ScheduleEntry): class ModelEntry(ScheduleEntry):
"""任务调度实体""" """任务调度实体"""
def __init__(self, model: TaskScheduler, app=None) -> None: # noqa:ANN001,C901 def __init__(self, model: TaskScheduler, app=None) -> None: # ruff:ignore[missing-type-function-argument, complex-structure]
super().__init__( super().__init__(
app=app or current_app._get_current_object(), app=app or current_app._get_current_object(),
name=model.name, name=model.name,
@@ -97,7 +97,7 @@ class ModelEntry(ScheduleEntry):
model.no_changes = True model.no_changes = True
self.model.enabled = self.enabled = model.enabled = False self.model.enabled = self.enabled = model.enabled = False
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
stmt = select(TaskScheduler).where(TaskScheduler.id == model.id) stmt = select(TaskScheduler).where(TaskScheduler.id == model.id, TaskScheduler.deleted == 0)
query = await db.execute(stmt) query = await db.execute(stmt)
task = query.scalars().first() task = query.scalars().first()
if task: if task:
@@ -129,7 +129,7 @@ class ModelEntry(ScheduleEntry):
return self.schedule.is_due(self.last_run_at) return self.schedule.is_due(self.last_run_at)
def __next__(self): # noqa: ANN204 def __next__(self): # ruff:ignore[missing-return-type-special-method]
self.model.last_run_time = timezone.now() self.model.last_run_time = timezone.now()
self.model.total_run_count += 1 self.model.total_run_count += 1
self.model.no_changes = True self.model.no_changes = True
@@ -145,7 +145,11 @@ class ModelEntry(ScheduleEntry):
:return: :return:
""" """
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
stmt = select(TaskScheduler).where(TaskScheduler.id == self.model.id).with_for_update() stmt = (
select(TaskScheduler)
.where(TaskScheduler.id == self.model.id, TaskScheduler.deleted == 0)
.with_for_update()
)
query = await db.execute(stmt) query = await db.execute(stmt)
task = query.scalars().first() task = query.scalars().first()
if task: if task:
@@ -157,10 +161,10 @@ class ModelEntry(ScheduleEntry):
logger.warning(f'任务 {self.model.name} 不存在,跳过更新') logger.warning(f'任务 {self.model.name} 不存在,跳过更新')
@classmethod @classmethod
async def from_entry(cls, name, app=None, **entry) -> ModelEntry: # noqa: ANN001 async def from_entry(cls, name, app=None, **entry) -> ModelEntry: # ruff:ignore[missing-type-function-argument]
"""保存或更新本地任务调度""" """保存或更新本地任务调度"""
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
stmt = select(TaskScheduler).where(TaskScheduler.name == name) stmt = select(TaskScheduler).where(TaskScheduler.name == name, TaskScheduler.deleted == 0)
query = await db.execute(stmt) query = await db.execute(stmt)
task = query.scalars().first() task = query.scalars().first()
temp = await cls._unpack_fields(name, **entry) temp = await cls._unpack_fields(name, **entry)
@@ -186,20 +190,20 @@ class ModelEntry(ScheduleEntry):
'interval_every': every, 'interval_every': every,
'interval_period': PeriodType.SECONDS.value, 'interval_period': PeriodType.SECONDS.value,
} }
stmt = select(TaskScheduler).filter_by(**spec) stmt = select(TaskScheduler).filter_by(**spec, deleted=0)
query = await db.execute(stmt) query = await db.execute(stmt)
obj = query.scalars().first() obj = query.scalars().first()
if not obj: if not obj:
obj = TaskScheduler(**CreateTaskSchedulerParam(task=task, **spec).model_dump()) obj = TaskScheduler(**CreateTaskSchedulerParam(task=task, **spec).model_dump())
elif isinstance(schedule, schedules.crontab): elif isinstance(schedule, schedules.crontab):
crontab = f'{schedule._orig_minute} {schedule._orig_hour} {schedule._orig_day_of_month} {schedule._orig_month_of_year} {schedule._orig_day_of_week}' # noqa: E501 crontab = f'{schedule._orig_minute} {schedule._orig_hour} {schedule._orig_day_of_month} {schedule._orig_month_of_year} {schedule._orig_day_of_week}' # ruff:ignore[line-too-long]
crontab_verify(crontab) crontab_verify(crontab)
spec = { spec = {
'name': name, 'name': name,
'type': TaskSchedulerType.CRONTAB.value, 'type': TaskSchedulerType.CRONTAB.value,
'crontab': crontab, 'crontab': crontab,
} }
stmt = select(TaskScheduler).filter_by(**spec) stmt = select(TaskScheduler).filter_by(**spec, deleted=0)
query = await db.execute(stmt) query = await db.execute(stmt)
obj = query.scalars().first() obj = query.scalars().first()
if not obj: if not obj:
@@ -222,10 +226,10 @@ class ModelEntry(ScheduleEntry):
) -> dict: ) -> dict:
model_schedule = await cls.to_model_schedule(name, task, schedule) model_schedule = await cls.to_model_schedule(name, task, schedule)
model_dict = select_as_dict(model_schedule) model_dict = select_as_dict(model_schedule)
for k in ['id', 'created_time', 'updated_time']: for k in ['id', 'created_time', 'updated_time', 'deleted', 'deleted_time']:
try: try:
del model_dict[k] del model_dict[k]
except KeyError: # noqa:PERF203 except KeyError: # ruff:ignore[try-except-in-loop]
continue continue
model_dict.update( model_dict.update(
args=json.dumps(args, ensure_ascii=False) if args else None, args=json.dumps(args, ensure_ascii=False) if args else None,
@@ -284,7 +288,7 @@ class DatabaseScheduler(Scheduler):
self._dirty = set() self._dirty = set()
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
self._finalize = Finalize(self, self.sync, exitpriority=5) self._finalize = Finalize(self, self.sync, exitpriority=5)
self.max_interval = kwargs.get('max_interval') or self.app.conf.beat_max_loop_interval or DEFAULT_MAX_INTERVAL self.max_interval = kwargs.get('max_interval') or self.app.conf.beat_max_loop_interval or _DEFAULT_MAX_INTERVAL
def schedules_equal(self, *args, **kwargs) -> bool: def schedules_equal(self, *args, **kwargs) -> bool:
"""重写父函数""" """重写父函数"""
@@ -293,7 +297,7 @@ class DatabaseScheduler(Scheduler):
return False return False
return super().schedules_equal(*args, **kwargs) return super().schedules_equal(*args, **kwargs)
def reserve(self, entry): # noqa: ANN001, ANN201 def reserve(self, entry): # ruff:ignore[missing-type-function-argument, missing-return-type-undocumented-public-function]
"""重写父函数""" """重写父函数"""
new_entry = next(entry) new_entry = next(entry)
# 需要按名称存储条目,因为条目可能会发生变化 # 需要按名称存储条目,因为条目可能会发生变化
@@ -334,7 +338,7 @@ class DatabaseScheduler(Scheduler):
"""重写父函数""" """重写父函数"""
if self.lock: if self.lock:
logger.debug('beat: Extending lock...') logger.debug('beat: Extending lock...')
run_await(self.lock.extend)(DEFAULT_MAX_LOCK_TIMEOUT, replace_ttl=True) run_await(self.lock.extend)(_DEFAULT_MAX_LOCK_TIMEOUT, replace_ttl=True)
return super().tick(**kwargs) return super().tick(**kwargs)
@@ -383,7 +387,10 @@ class DatabaseScheduler(Scheduler):
"""获取所有任务调度""" """获取所有任务调度"""
async with async_db_session() as db: async with async_db_session() as db:
logger.debug('DatabaseScheduler: Fetching database schedule') logger.debug('DatabaseScheduler: Fetching database schedule')
stmt = select(TaskScheduler).where(TaskScheduler.enabled == True) # noqa: E712 stmt = select(TaskScheduler).where(
TaskScheduler.enabled.is_(True),
TaskScheduler.deleted == 0,
)
query = await db.execute(stmt) query = await db.execute(stmt)
schedulers = query.scalars().all() schedulers = query.scalars().all()
s = {} s = {}
@@ -421,7 +428,7 @@ class DatabaseScheduler(Scheduler):
@beat_init.connect @beat_init.connect
def acquire_distributed_beat_lock(sender=None, **kwargs) -> None: # noqa: ANN001 def acquire_distributed_beat_lock(sender=None, **kwargs) -> None: # ruff:ignore[missing-type-function-argument]
""" """
尝试在启动时获取锁 尝试在启动时获取锁
@@ -435,7 +442,7 @@ def acquire_distributed_beat_lock(sender=None, **kwargs) -> None: # noqa: ANN00
logger.debug('beat: Acquiring lock...') logger.debug('beat: Acquiring lock...')
lock = redis_client.lock( lock = redis_client.lock(
scheduler.lock_key, scheduler.lock_key,
timeout=DEFAULT_MAX_LOCK_TIMEOUT, timeout=_DEFAULT_MAX_LOCK_TIMEOUT,
sleep=scheduler.max_interval, sleep=scheduler.max_interval,
) )
+1 -1
View File
@@ -8,7 +8,7 @@ from backend.utils.timezone import timezone
class TzAwareCrontab(schedules.crontab): class TzAwareCrontab(schedules.crontab):
"""时区感知 Crontab""" """时区感知 Crontab"""
def __init__(self, minute='*', hour='*', day_of_week='*', day_of_month='*', month_of_year='*', app=None) -> None: # noqa: ANN001 def __init__(self, minute='*', hour='*', day_of_week='*', day_of_month='*', month_of_year='*', app=None) -> None: # ruff:ignore[missing-type-function-argument]
super().__init__( super().__init__(
minute=minute, minute=minute,
hour=hour, hour=hour,
+153 -36
View File
@@ -6,7 +6,7 @@ import sys
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from typing import Annotated, Literal from typing import Annotated, Final, Literal
import anyio import anyio
import cappa import cappa
@@ -23,6 +23,7 @@ from starlette.concurrency import run_in_threadpool
from watchfiles import Change, PythonFilter from watchfiles import Change, PythonFilter
from backend import __version__ from backend import __version__
from backend.common.dataclasses import PluginEntry
from backend.common.enums import DataBaseType, PrimaryKeyType from backend.common.enums import DataBaseType, PrimaryKeyType
from backend.common.exception.errors import BaseExceptionError from backend.common.exception.errors import BaseExceptionError
from backend.common.model import MappedBase from backend.common.model import MappedBase
@@ -31,6 +32,7 @@ from backend.core.path_conf import (
BASE_PATH, BASE_PATH,
ENV_EXAMPLE_FILE_PATH, ENV_EXAMPLE_FILE_PATH,
ENV_FILE_PATH, ENV_FILE_PATH,
LOCALE_DIR,
MYSQL_SCRIPT_DIR, MYSQL_SCRIPT_DIR,
PLUGIN_DIR, PLUGIN_DIR,
POSTGRESQL_SCRIPT_DIR, POSTGRESQL_SCRIPT_DIR,
@@ -40,30 +42,43 @@ from backend.database.db import (
async_db_session, async_db_session,
create_database_async_engine, create_database_async_engine,
create_database_async_session, create_database_async_session,
create_database_url, get_database_url,
) )
from backend.database.redis import RedisCli, redis_client from backend.database.redis import RedisCli, redis_client
from backend.plugin.core import build_sql_filename, get_plugin_destroy_sql, get_plugin_sql, get_plugins from backend.plugin.core import (
from backend.plugin.installer import install_git_plugin, install_zip_plugin, zip_plugin get_plugins,
get_required_plugins,
load_plugin_config,
resolve_plugin_order,
)
from backend.plugin.installer import install_git_frontend_plugin, install_git_plugin, install_zip_plugin, zip_plugin
from backend.plugin.installer import remove_plugin as _remove_plugin from backend.plugin.installer import remove_plugin as _remove_plugin
from backend.plugin.requirements import uninstall_requirements_async from backend.plugin.requirements import install_requirements_async, uninstall_requirements_async
from backend.plugin.sql import build_sql_filename, get_plugin_destroy_sql, get_plugin_sql
from backend.plugin.validator import validate_plugin_config
from backend.utils.console import console from backend.utils.console import console
from backend.utils.dynamic_import import import_module_cached from backend.utils.dynamic_import import import_module_cached
from backend.utils.sql_parser import parse_sql_script from backend.utils.sql_parser import parse_sql_script
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
output_help = "\n更多信息,尝试 '[cyan]--help[/]'" _OUTPUT_HELP: Final = "\n更多信息,尝试 '[cyan]--help[/]'"
class CustomReloadFilter(PythonFilter): class CustomReloadFilter(PythonFilter):
"""自定义重载过滤器""" """自定义重载过滤器"""
def __init__(self) -> None: def __init__(self) -> None:
super().__init__(extra_extensions=['.json', '.yaml', '.yml']) self.extra_extensions = ('.json', '.yaml', '.yml')
super().__init__(extra_extensions=self.extra_extensions)
def __call__(self, change: Change, path: str) -> bool: def __call__(self, change: Change, path: str) -> bool:
if RELOAD_LOCK_FILE.exists(): if RELOAD_LOCK_FILE.exists():
return False return False
file_path = Path(path).resolve()
if file_path.suffix in self.extra_extensions and not file_path.is_relative_to(LOCALE_DIR.resolve()):
return False
return super().__call__(change, path) return super().__call__(change, path)
@@ -193,7 +208,7 @@ async def auto_init() -> None:
ok = Prompt.ask('即将[red]新建/重建数据库[/red],确认继续吗?', choices=['y', 'n'], default='n') ok = Prompt.ask('即将[red]新建/重建数据库[/red],确认继续吗?', choices=['y', 'n'], default='n')
if ok.lower() == 'y': if ok.lower() == 'y':
async_init_engine = create_database_async_engine(create_database_url(with_database=False)) async_init_engine = create_database_async_engine(get_database_url(with_database=False))
async with async_init_engine.connect() as conn: async with async_init_engine.connect() as conn:
await conn.execution_options(isolation_level='AUTOCOMMIT') await conn.execution_options(isolation_level='AUTOCOMMIT')
if not await create_database(conn): if not await create_database(conn):
@@ -202,7 +217,7 @@ async def auto_init() -> None:
console.warning('已取消数据库操作') console.warning('已取消数据库操作')
console.print('\n[bold cyan]步骤 3/3:[/] 初始化数据库表和数据', style='bold') console.print('\n[bold cyan]步骤 3/3:[/] 初始化数据库表和数据', style='bold')
async_init_engine = create_database_async_engine(create_database_url()) async_init_engine = create_database_async_engine(get_database_url())
async_init_db_session = create_database_async_session(async_init_engine) async_init_db_session = create_database_async_session(async_init_engine)
redis_init_client = RedisCli( redis_init_client = RedisCli(
host=settings.REDIS_HOST, host=settings.REDIS_HOST,
@@ -219,7 +234,7 @@ async def init(db: AsyncSession, redis: RedisCli) -> None:
"""交互式初始化数据库表结构和数据""" """交互式初始化数据库表结构和数据"""
panel_content = _build_db_config_panel_content() panel_content = _build_db_config_panel_content()
pk_details = panel_content.from_markup( pk_details = panel_content.from_markup(
'[link=https://fastapi-practices.github.io/fastapi_best_architecture_docs/backend/reference/pk.html](了解详情)[/]' '[link=https://docs.fba.wu-clan.cc/fastapi_best_architecture_docs/backend/reference/pk.html](了解详情)[/]'
) )
panel_content.append(pk_details) panel_content.append(pk_details)
panel_content.append('\n\n【Redis 配置】', style='bold green') panel_content.append('\n\n【Redis 配置】', style='bold green')
@@ -249,7 +264,7 @@ async def init(db: AsyncSession, redis: RedisCli) -> None:
settings.TOKEN_REDIS_PREFIX, settings.TOKEN_REDIS_PREFIX,
settings.TOKEN_REFRESH_REDIS_PREFIX, settings.TOKEN_REFRESH_REDIS_PREFIX,
]: ]:
await redis.delete_prefix(prefix) await redis.delete_by_prefix(prefix)
console.note('重建数据库表') console.note('重建数据库表')
conn = await db.connection() conn = await db.connection()
@@ -270,7 +285,7 @@ async def init(db: AsyncSession, redis: RedisCli) -> None:
console.warning('已取消初始化操作') console.warning('已取消初始化操作')
def run(host: str, port: int, reload: bool, workers: int) -> None: # noqa: FBT001 def run(host: str, port: int, reload: bool, workers: int) -> None: # ruff:ignore[boolean-type-hint-positional-argument]
"""启动 API 服务""" """启动 API 服务"""
url = f'http://{host}:{port}' url = f'http://{host}:{port}'
docs_url = url + settings.FASTAPI_DOCS_URL docs_url = url + settings.FASTAPI_DOCS_URL
@@ -301,7 +316,7 @@ def run(host: str, port: int, reload: bool, workers: int) -> None: # noqa: FBT0
panel_content.append(f'\n📡 OpenAPI JSON: {openapi_url}', style='bold magenta') panel_content.append(f'\n📡 OpenAPI JSON: {openapi_url}', style='bold magenta')
panel_content.append('\n🌐 架构官方文档: ', style='bold magenta') panel_content.append('\n🌐 架构官方文档: ', style='bold magenta')
panel_content.append('https://fastapi-practices.github.io/fastapi_best_architecture_docs/') panel_content.append('https://docs.fba.wu-clan.cc/fastapi_best_architecture_docs/')
console.print(Panel(panel_content, title=f'fba (v{__version__})', border_style='purple', padding=(1, 2))) console.print(Panel(panel_content, title=f'fba (v{__version__})', border_style='purple', padding=(1, 2)))
granian.Granian( granian.Granian(
@@ -346,10 +361,11 @@ def run_celery_flower(port: int, basic_auth: str) -> None:
pass pass
async def install_plugin( # noqa: C901 async def install_plugin( # ruff:ignore[complex-structure]
path: str, path: str | None,
repo_url: str, repo_url: str | None,
no_sql: bool, # noqa: FBT001 frontend: bool, # ruff:ignore[boolean-type-hint-positional-argument]
no_sql: bool, # ruff:ignore[boolean-type-hint-positional-argument]
db_type: DataBaseType, db_type: DataBaseType,
pk_type: PrimaryKeyType, pk_type: PrimaryKeyType,
) -> None: ) -> None:
@@ -357,15 +373,24 @@ async def install_plugin( # noqa: C901
if settings.ENVIRONMENT != 'dev': if settings.ENVIRONMENT != 'dev':
raise cappa.Exit('插件安装仅在开发环境可用', code=1) raise cappa.Exit('插件安装仅在开发环境可用', code=1)
if not path and not repo_url:
raise cappa.Exit('path 或 repo_url 必须指定其中一项', code=1)
if path and repo_url:
raise cappa.Exit('path 和 repo_url 不能同时指定', code=1)
plugin_name = None plugin_name = None
console.note('开始安装插件...') console.note('开始安装插件...')
try: try:
if frontend:
if repo_url is None:
raise cappa.Exit('前端插件仅允许通过 Git 仓库地址安装', code=1)
frontend_project_root = Prompt.ask('请输入前端项目根路径')
plugin_name = await install_git_frontend_plugin(repo_url, frontend_project_root)
console.tip(f'前端插件 {plugin_name} 安装成功')
return
if path is None and repo_url is None:
raise cappa.Exit('path 或 repo_url 必须指定其中一项', code=1)
if path and repo_url:
raise cappa.Exit('path 和 repo_url 不能同时指定', code=1)
if path: if path:
plugin_name = await install_zip_plugin(file=path) plugin_name = await install_zip_plugin(file=path)
if repo_url: if repo_url:
@@ -384,9 +409,9 @@ async def install_plugin( # noqa: C901
await conn.run_sync(MappedBase.metadata.create_all) await conn.run_sync(MappedBase.metadata.create_all)
if not no_sql: if not no_sql:
sql_file = await get_plugin_sql(plugin_name, db_type, pk_type) sql_file = await get_plugin_sql(plugin_name, db_type, pk_type, tenant=settings.TENANT_ENABLED)
if sql_file: if sql_file:
console.info(f'正在执行插件 {plugin_name} 初始化 SQL 脚本...') console.info(f'正在执行插件 {plugin_name} 初始化 SQL 脚本{sql_file}')
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
await execute_sql_scripts(db, sql_file) await execute_sql_scripts(db, sql_file)
else: else:
@@ -396,7 +421,56 @@ async def install_plugin( # noqa: C901
raise cappa.Exit(e.msg if isinstance(e, BaseExceptionError) else str(e), code=1) raise cappa.Exit(e.msg if isinstance(e, BaseExceptionError) else str(e), code=1)
async def remove_plugin(plugin: str | None, *, no_sql: bool = False) -> None: # noqa: C901 def should_sync_plugin_deps(plugin: str | None, *, allow_empty: bool) -> bool:
"""检查是否需要同步插件依赖"""
plugins = get_plugins()
if plugin is not None and plugin not in plugins:
raise cappa.Exit(f'插件 {plugin} 不存在', code=1)
if not plugins:
if allow_empty:
console.warning('当前没有已安装的插件,跳过插件依赖同步')
return False
raise cappa.Exit('当前没有已安装的插件', code=1)
return True
async def sync_project_deps() -> None:
"""同步项目依赖"""
console.note('正在同步项目依赖...')
try:
await run_in_threadpool(subprocess.run, ['uv', 'sync'], cwd=BASE_PATH.parent, check=True)
except FileNotFoundError:
raise cappa.Exit('uv 未安装,请先安装 uv', code=1)
except subprocess.CalledProcessError as e:
raise cappa.Exit('项目依赖同步失败', code=e.returncode)
console.tip('项目依赖同步完成')
async def sync_plugin_deps(plugin: str | None = None) -> None:
"""同步插件依赖"""
console.note(f'正在安装插件 {plugin} 依赖...' if plugin else '正在安装所有插件依赖...')
try:
await install_requirements_async(plugin)
except Exception as e:
raise cappa.Exit(e.msg if isinstance(e, BaseExceptionError) else str(e), code=1)
console.tip(f'插件 {plugin} 依赖安装完成' if plugin else '所有插件依赖安装完成')
async def sync_deps(plugin: str | None, *, no_project: bool = False, no_plugin: bool = False) -> None:
"""同步项目和插件依赖"""
if no_project and no_plugin:
raise cappa.Exit('--no-project 和 --no-plugin 不能同时使用', code=1)
if plugin is not None and no_plugin:
raise cappa.Exit('--plugin 和 --no-plugin 不能同时使用', code=1)
should_sync_plugins = False if no_plugin else should_sync_plugin_deps(plugin, allow_empty=not no_project)
if not no_project:
await sync_project_deps()
if should_sync_plugins:
await sync_plugin_deps(plugin)
async def remove_plugin(plugin: str | None, *, no_sql: bool = False) -> None: # ruff:ignore[complex-structure]
"""卸载插件""" """卸载插件"""
if settings.ENVIRONMENT != 'dev': if settings.ENVIRONMENT != 'dev':
raise cappa.Exit('插件卸载仅在开发环境可用', code=1) raise cappa.Exit('插件卸载仅在开发环境可用', code=1)
@@ -409,7 +483,7 @@ async def remove_plugin(plugin: str | None, *, no_sql: bool = False) -> None: #
if not no_sql: if not no_sql:
destroy_sql_file = await get_plugin_destroy_sql(plugin, settings.DATABASE_TYPE, settings.DATABASE_PK_MODE) destroy_sql_file = await get_plugin_destroy_sql(plugin, settings.DATABASE_TYPE, settings.DATABASE_PK_MODE)
if destroy_sql_file: if destroy_sql_file:
console.note(f'正在执行插件 {plugin} 销毁 SQL 脚本...') console.note(f'正在执行插件 {plugin} 销毁 SQL 脚本{destroy_sql_file}')
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
await execute_destroy_sql_scripts(db, destroy_sql_file) await execute_destroy_sql_scripts(db, destroy_sql_file)
else: else:
@@ -447,6 +521,9 @@ async def remove_plugin(plugin: str | None, *, no_sql: bool = False) -> None: #
if plugin not in plugins: if plugin not in plugins:
raise cappa.Exit(f'插件 {plugin} 不存在', code=1) raise cappa.Exit(f'插件 {plugin} 不存在', code=1)
if plugin in get_required_plugins():
raise cappa.Exit(f'插件 {plugin} 为必需插件,禁止卸载', code=1)
try: try:
await remove() await remove()
except Exception as e: except Exception as e:
@@ -461,13 +538,25 @@ async def get_sql_scripts() -> list[str]:
'init', 'init',
settings.DATABASE_PK_MODE, settings.DATABASE_PK_MODE,
suffix='test_data', suffix='test_data',
tenant=settings.TENANT_ENABLED,
) )
if await anyio.Path(main_sql_file).exists(): if await anyio.Path(main_sql_file).exists():
sql_scripts.append(str(main_sql_file)) sql_scripts.append(str(main_sql_file))
plugins = []
for plugin in get_plugins(): for plugin in get_plugins():
plugin_sql = await get_plugin_sql(plugin, settings.DATABASE_TYPE, settings.DATABASE_PK_MODE) plugin_config = load_plugin_config(plugin)
validate_plugin_config(plugin, plugin_config)
plugins.append(PluginEntry(name=plugin, depends_on=plugin_config['plugin'].get('depends_on')))
for plugin in resolve_plugin_order(plugins):
plugin_sql = await get_plugin_sql(
plugin.name,
settings.DATABASE_TYPE,
settings.DATABASE_PK_MODE,
tenant=settings.TENANT_ENABLED,
)
if plugin_sql: if plugin_sql:
sql_scripts.append(plugin_sql) sql_scripts.append(plugin_sql)
@@ -478,8 +567,9 @@ async def execute_sql_scripts(db: AsyncSession, sql_scripts: str, *, is_init: bo
"""解析并执行 SQL 脚本""" """解析并执行 SQL 脚本"""
try: try:
stmts = await parse_sql_script(sql_scripts) stmts = await parse_sql_script(sql_scripts)
conn = await db.connection()
for stmt in stmts: for stmt in stmts:
await db.execute(text(stmt)) await conn.exec_driver_sql(stmt)
except Exception as e: except Exception as e:
raise cappa.Exit(f'SQL 脚本执行失败:{e}', code=1) raise cappa.Exit(f'SQL 脚本执行失败:{e}', code=1)
@@ -491,8 +581,9 @@ async def execute_destroy_sql_scripts(db: AsyncSession, sql_scripts: str) -> Non
"""执行插件销毁 SQL 脚本""" """执行插件销毁 SQL 脚本"""
try: try:
stmts = await parse_sql_script(sql_scripts, is_destroy=True) stmts = await parse_sql_script(sql_scripts, is_destroy=True)
conn = await db.connection()
for stmt in stmts: for stmt in stmts:
await db.execute(text(stmt)) await conn.exec_driver_sql(stmt)
except Exception as e: except Exception as e:
raise cappa.Exit(f'销毁 SQL 脚本执行失败:{e}', code=1) raise cappa.Exit(f'销毁 SQL 脚本执行失败:{e}', code=1)
@@ -654,11 +745,15 @@ class Run:
class Add: class Add:
path: Annotated[ path: Annotated[
str | None, str | None,
cappa.Arg(help='ZIP 插件的本地完整路径'), cappa.Arg(default=None, help='ZIP 插件的本地完整路径'),
] ]
repo_url: Annotated[ repo_url: Annotated[
str | None, str | None,
cappa.Arg(help='Git 插件的仓库地址'), cappa.Arg(default=None, help='Git 插件的仓库地址'),
]
frontend: Annotated[
bool,
cappa.Arg(short='-f', default=False, help='安装前端插件'),
] ]
no_sql: Annotated[ no_sql: Annotated[
bool, bool,
@@ -666,15 +761,15 @@ class Add:
] ]
db_type: Annotated[ db_type: Annotated[
DataBaseType, DataBaseType,
cappa.Arg(default='postgresql', help='执行插件 SQL 脚本的数据库类型'), cappa.Arg(default=settings.DATABASE_TYPE, help='执行插件 SQL 脚本的数据库类型'),
] ]
pk_type: Annotated[ pk_type: Annotated[
PrimaryKeyType, PrimaryKeyType,
cappa.Arg(default='autoincrement', help='执行插件 SQL 脚本数据库主键类型'), cappa.Arg(default=settings.DATABASE_PK_MODE, help='执行插件 SQL 脚本数据库主键类型'),
] ]
async def __call__(self) -> None: async def __call__(self) -> None:
await install_plugin(self.path, self.repo_url, self.no_sql, self.db_type, self.pk_type) await install_plugin(self.path, self.repo_url, self.frontend, self.no_sql, self.db_type, self.pk_type)
@cappa.command(help='移除插件') @cappa.command(help='移除插件')
@@ -693,6 +788,26 @@ class Remove:
await remove_plugin(self.plugin, no_sql=self.no_sql) await remove_plugin(self.plugin, no_sql=self.no_sql)
@cappa.command(help='同步项目和插件依赖', default_long=True)
@dataclass
class Deps:
plugin: Annotated[
str | None,
cappa.Arg(default=None, help='指定插件名称,不指定则同步所有插件依赖'),
]
no_project: Annotated[
bool,
cappa.Arg(default=False, help='跳过项目依赖同步'),
]
no_plugin: Annotated[
bool,
cappa.Arg(default=False, help='跳过插件依赖同步'),
]
async def __call__(self) -> None:
await sync_deps(self.plugin, no_project=self.no_project, no_plugin=self.no_plugin)
@cappa.command(help='格式化代码') @cappa.command(help='格式化代码')
@dataclass @dataclass
class Format: class Format:
@@ -896,7 +1011,9 @@ class FbaCli:
str, str,
cappa.Arg(value_name='PATH', default='', show_default=False, help='在事务中执行 SQL 脚本'), cappa.Arg(value_name='PATH', default='', show_default=False, help='在事务中执行 SQL 脚本'),
] ]
subcmd: cappa.Subcommands[Init | Run | Add | Remove | Format | Celery | CodeGenerator | Alembic | None] = None subcmd: cappa.Subcommands[Init | Run | Add | Remove | Deps | Format | Celery | CodeGenerator | Alembic | None] = (
None
)
async def __call__(self) -> None: async def __call__(self) -> None:
if self.sql: if self.sql:
@@ -905,5 +1022,5 @@ class FbaCli:
def main() -> None: def main() -> None:
output = cappa.Output(error_format=f'{error_format}\n{output_help}') output = cappa.Output(error_format=f'{error_format}\n{_OUTPUT_HELP}')
asyncio.run(cappa.invoke_async(FbaCli, version=__version__, output=output)) asyncio.run(cappa.invoke_async(FbaCli, version=__version__, output=output))
+41 -34
View File
@@ -1,6 +1,7 @@
import functools import functools
from collections.abc import Callable, Sequence from collections.abc import Awaitable, Callable, Sequence
from inspect import isawaitable
from typing import Any, ParamSpec, TypeVar from typing import Any, ParamSpec, TypeVar
from msgspec import json from msgspec import json
@@ -16,12 +17,13 @@ from backend.utils.serializers import select_columns_serialize, select_list_seri
P = ParamSpec('P') P = ParamSpec('P')
T = TypeVar('T') T = TypeVar('T')
_MISSING = object()
def _build_cache_key( async def _build_cache_key(
name: str, namespace: str,
key: str | None, key: str | None,
key_builder: Callable[..., str] | None, key_builder: Callable[..., str | Awaitable[str]] | None,
*args: Any, *args: Any,
**kwargs: Any, **kwargs: Any,
) -> str: ) -> str:
@@ -29,9 +31,9 @@ def _build_cache_key(
if key: if key:
if '.' in key: if '.' in key:
param, field = key.split('.', 1) param, field = key.split('.', 1)
value = kwargs.get(param) value = kwargs.get(param, _MISSING)
if value is None: if value is _MISSING:
raise errors.ServerError(msg=f'缓存键构建失败,参数 "{param}" 不存在或值为空') raise errors.ServerError(msg=f'缓存键构建失败,参数 "{param}" 不存在')
if isinstance(value, list): if isinstance(value, list):
raise errors.ServerError(msg='缓存键构建失败:不支持从列表中提取字段,请使用 key_builder 处理列表参数') raise errors.ServerError(msg='缓存键构建失败:不支持从列表中提取字段,请使用 key_builder 处理列表参数')
@@ -43,16 +45,19 @@ def _build_cache_key(
else: else:
raise errors.ServerError(msg=f'缓存键构建失败,对象中不存在字段 "{field}"') raise errors.ServerError(msg=f'缓存键构建失败,对象中不存在字段 "{field}"')
else: else:
value = kwargs.get(key) value = kwargs.get(key, _MISSING)
if value is None: if value is _MISSING:
raise errors.ServerError(msg=f'缓存键构建失败,参数 "{key}" 不存在或值为空') raise errors.ServerError(msg=f'缓存键构建失败,参数 "{key}" 不存在')
return f'{name}:{value}' return f'{namespace}:{value if value is not None else "none"}'
if key_builder: if key_builder:
return f'{name}:{key_builder(*args, **kwargs)}' value = key_builder(*args, **kwargs)
if isawaitable(value):
value = await value
return f'{namespace}:{value}'
return name return namespace
def _serialize_result(result: Any) -> bytes: def _serialize_result(result: Any) -> bytes:
@@ -100,16 +105,16 @@ def user_key_builder() -> str:
return str(user_id) return str(user_id)
def cached( # noqa: C901 def cached( # ruff:ignore[complex-structure]
name: str, namespace: str,
*, *,
key: str | None = None, key: str | None = None,
key_builder: Callable[..., str] | None = None, key_builder: Callable[..., str | Awaitable[str]] | None = None,
) -> Callable[[Callable[P, T]], Callable[P, T]]: ) -> Callable[[Callable[P, T]], Callable[P, T]]:
""" """
缓存装饰器 缓存装饰器
:param name: 缓存名称通常为缓存 Key 前缀 :param namespace: 缓存命名空间通常为缓存 Key 前缀
:param key: 从方法参数中获取指定参数名的值作为缓存 Key key_builder 互斥 :param key: 从方法参数中获取指定参数名的值作为缓存 Key key_builder 互斥
:param key_builder: 自定义 Key 生成函数 key 互斥 :param key_builder: 自定义 Key 生成函数 key 互斥
:return: :return:
@@ -117,10 +122,10 @@ def cached( # noqa: C901
if key is not None and key_builder is not None: if key is not None and key_builder is not None:
raise errors.ServerError(msg='缓存 key 和 key_builder 不能同时使用') raise errors.ServerError(msg='缓存 key 和 key_builder 不能同时使用')
def decorator(func: Callable[P, T]) -> Callable[P, T]: # noqa: C901 def decorator(func: Callable[P, T]) -> Callable[P, T]: # ruff:ignore[complex-structure]
@functools.wraps(func) @functools.wraps(func)
async def wrapper(*args: P.args, **kwargs: P.kwargs) -> T: async def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
cache_key = _build_cache_key(name, key, key_builder, *args, **kwargs) cache_key = await _build_cache_key(namespace, key, key_builder, *args, **kwargs)
# L1: 本地缓存 # L1: 本地缓存
if settings.CACHE_LOCAL_ENABLED: if settings.CACHE_LOCAL_ENABLED:
@@ -145,14 +150,16 @@ def cached( # noqa: C901
if result is not None: if result is not None:
try: try:
serialized_result = _serialize_result(result)
deserialized_result = _deserialize_result(serialized_result)
# 回填 L1 # 回填 L1
if settings.CACHE_LOCAL_ENABLED: if settings.CACHE_LOCAL_ENABLED:
local_cache_manager.set(cache_key, result) local_cache_manager.set(cache_key, deserialized_result)
# 回填 L2 # 回填 L2
serialized_result = _serialize_result(result)
if settings.CACHE_REDIS_TTL: if settings.CACHE_REDIS_TTL:
await redis_client.setex(cache_key, settings.CACHE_REDIS_TTL, serialized_result) await redis_client.set(cache_key, serialized_result, ex=settings.CACHE_REDIS_TTL)
else: else:
await redis_client.set(cache_key, serialized_result) await redis_client.set(cache_key, serialized_result)
except Exception as e: except Exception as e:
@@ -165,17 +172,17 @@ def cached( # noqa: C901
return decorator return decorator
def cache_invalidate( # noqa: C901 def cache_invalidate( # ruff:ignore[complex-structure]
name: str, namespace: str,
*, *,
key: str | None = None, key: str | None = None,
key_builder: Callable[..., str] | None = None, key_builder: Callable[..., str | Awaitable[str]] | None = None,
atomic: bool = True, atomic: bool = True,
) -> Callable[[Callable[P, T]], Callable[P, T]]: ) -> Callable[[Callable[P, T]], Callable[P, T]]:
""" """
缓存失效装饰器 缓存失效装饰器
:param name: 缓存名称通常为缓存 Key 前缀 :param namespace: 缓存命名空间通常为缓存 Key 前缀
:param key: 从方法参数中获取指定参数名的值作为缓存 Key key_builder 互斥 :param key: 从方法参数中获取指定参数名的值作为缓存 Key key_builder 互斥
:param key_builder: 自定义 Key 生成函数 key 互斥 :param key_builder: 自定义 Key 生成函数 key 互斥
:param atomic: 是否保证缓存原子性 :param atomic: 是否保证缓存原子性
@@ -194,25 +201,25 @@ def cache_invalidate( # noqa: C901
invalidate_error = None invalidate_error = None
try: try:
invalidate_key = _build_cache_key(name, key, key_builder, *args, **kwargs) invalidate_key = await _build_cache_key(namespace, key, key_builder, *args, **kwargs)
# L1 缓存失效 # L1 缓存失效
if settings.CACHE_LOCAL_ENABLED: if settings.CACHE_LOCAL_ENABLED:
if invalidate_key == name: if invalidate_key == namespace:
local_cache_manager.delete_prefix(invalidate_key) local_cache_manager.delete_by_prefix(invalidate_key)
else: else:
local_cache_manager.delete(invalidate_key) local_cache_manager.delete(invalidate_key)
# 广播失效消息(通知其他节点清除本地缓存) # 广播失效消息(通知其他节点清除本地缓存)
if settings.CACHE_LOCAL_ENABLED: if settings.CACHE_LOCAL_ENABLED:
if invalidate_key == name: if invalidate_key == namespace:
await cache_pubsub_manager.publish_invalidation(invalidate_key, is_delete_prefix=True) await cache_pubsub_manager.publish_invalidation(invalidate_key, delete_by_prefix=True)
else: else:
await cache_pubsub_manager.publish_invalidation(invalidate_key) await cache_pubsub_manager.publish_invalidation(invalidate_key, delete_by_prefix=False)
# L2 缓存失效 # L2 缓存失效
if invalidate_key == name: if invalidate_key == namespace:
await redis_client.delete_prefix(invalidate_key) await redis_client.delete_by_prefix(invalidate_key)
else: else:
await redis_client.delete(invalidate_key) await redis_client.delete(invalidate_key)
+12 -6
View File
@@ -10,7 +10,7 @@ class LocalCacheManager:
def __init__(self) -> None: def __init__(self) -> None:
self.hot_cache: cachebox.TTLCache = cachebox.TTLCache( self.hot_cache: cachebox.TTLCache = cachebox.TTLCache(
settings.CACHE_LOCAL_MAXSIZE, ttl=settings.CACHE_LOCAL_TTL settings.CACHE_LOCAL_MAXSIZE, global_ttl=settings.CACHE_LOCAL_TTL
) )
def get(self, key: str) -> Any: def get(self, key: str) -> Any:
@@ -36,17 +36,23 @@ class LocalCacheManager:
"""清空缓存""" """清空缓存"""
self.hot_cache.clear() self.hot_cache.clear()
def delete_prefix(self, prefix: str, exclude: str | list[str] | None = None) -> None: def delete_by_prefix(self, key_prefix: str, exclude_keys: str | list[str] | None = None) -> None:
""" """
删除指定前缀的缓存 删除指定前缀的缓存
:param prefix: 要删除的键前缀 :param key_prefix: 要删除的键前缀
:param exclude: 要排除的键或键列表 :param exclude_keys: 要排除的键或键列表
:return: :return:
""" """
exclude_set = set(exclude) if isinstance(exclude, list) else {exclude} if isinstance(exclude, str) else set() exclude_set = (
set(exclude_keys)
if isinstance(exclude_keys, list)
else {exclude_keys}
if isinstance(exclude_keys, str)
else set()
)
for key in list(self.hot_cache.keys()): for key in list(self.hot_cache.keys()):
if key.startswith(prefix) and key not in exclude_set: if (key == key_prefix or key.startswith(f'{key_prefix}:')) and key not in exclude_set:
try: try:
del self.hot_cache[key] del self.hot_cache[key]
except KeyError: except KeyError:
+10 -10
View File
@@ -13,22 +13,22 @@ class CachePubSubManager:
_pubsub_task: asyncio.Task | None = None _pubsub_task: asyncio.Task | None = None
@staticmethod @staticmethod
async def publish_invalidation(key: str, *, is_delete_prefix: bool) -> None: async def publish_invalidation(cache_key: str, *, delete_by_prefix: bool) -> None:
""" """
发布缓存失效通知 发布缓存失效通知
:param key: 缓存键 :param cache_key: 缓存键
:param is_delete_prefix: 是否删除符合前缀的所有缓存 :param delete_by_prefix: 是否删除符合前缀的所有缓存
:return: :return:
""" """
try: try:
message = json.dumps({'key': key, 'is_delete_prefix': is_delete_prefix}) message = json.dumps({'cache_key': cache_key, 'delete_by_prefix': delete_by_prefix})
await redis_client.publish(settings.CACHE_PUBSUB_CHANNEL, message) await redis_client.publish(settings.CACHE_PUBSUB_CHANNEL, message)
except Exception as e: except Exception as e:
log.warning(f'[CachePubSub] 发布通知失败: {e}') log.warning(f'[CachePubSub] 发布通知失败: {e}')
@staticmethod @staticmethod
async def subscribe_and_listen() -> None: # noqa: C901 async def subscribe_and_listen() -> None: # ruff:ignore[complex-structure]
"""订阅并监听缓存失效通知""" """订阅并监听缓存失效通知"""
reconnect_attempts = 0 reconnect_attempts = 0
@@ -38,7 +38,7 @@ class CachePubSubManager:
try: try:
# 使用独立连接 # 使用独立连接
pubsub_client = RedisCli() pubsub_client = RedisCli(socket_timeout=None)
pubsub = pubsub_client.pubsub() pubsub = pubsub_client.pubsub()
await pubsub.subscribe(settings.CACHE_PUBSUB_CHANNEL) await pubsub.subscribe(settings.CACHE_PUBSUB_CHANNEL)
@@ -49,11 +49,11 @@ class CachePubSubManager:
if message['type'] == 'message': if message['type'] == 'message':
try: try:
data = json.loads(message['data']) data = json.loads(message['data'])
key = data['key'] cache_key = data['cache_key']
if not data['is_delete_prefix']: if not data['delete_by_prefix']:
local_cache_manager.delete(key) local_cache_manager.delete(cache_key)
else: else:
local_cache_manager.delete_prefix(key) local_cache_manager.delete_by_prefix(cache_key)
except json.JSONDecodeError as e: except json.JSONDecodeError as e:
log.warning(f'[CachePubSub] 消息格式错误 {e}') log.warning(f'[CachePubSub] 消息格式错误 {e}')
except Exception as e: except Exception as e:
+2
View File
@@ -22,6 +22,8 @@ class TypedContextProtocol(Protocol):
language: str language: str
user_id: int | None user_id: int | None
is_superuser: bool
tenant_id: int
class TypedContext(TypedContextProtocol, _Context): class TypedContext(TypedContextProtocol, _Context):
+11
View File
@@ -1,6 +1,7 @@
import dataclasses import dataclasses
from datetime import datetime from datetime import datetime
from typing import Any
from fastapi import Response from fastapi import Response
@@ -57,6 +58,7 @@ class NewToken:
@dataclasses.dataclass @dataclasses.dataclass
class TokenPayload: class TokenPayload:
user_id: int user_id: int
tenant_id: int
session_uuid: str session_uuid: str
expire_time: datetime expire_time: datetime
@@ -73,3 +75,12 @@ class SnowflakeInfo:
datacenter_id: int datacenter_id: int
worker_id: int worker_id: int
sequence: int sequence: int
@dataclasses.dataclass(slots=True)
class PluginEntry:
name: str
depends_on: list[str] | None = None
extend: str | None = None
routers: list[str] | None = None
api: dict[str, Any] | None = None
+9
View File
@@ -114,6 +114,7 @@ class PluginLevelType(StrEnum):
"""插件级别类型""" """插件级别类型"""
app = 'app' app = 'app'
capability = 'capability'
extend = 'extend' extend = 'extend'
@@ -145,3 +146,11 @@ class PrimaryKeyType(StrEnum):
autoincrement = 'autoincrement' autoincrement = 'autoincrement'
snowflake = 'snowflake' snowflake = 'snowflake'
class LifespanStage(IntEnum):
"""lifespan 执行阶段"""
core = 0
plugin = 1
tail = 2
@@ -76,7 +76,7 @@ async def _validation_exception_handler(exc: RequestValidationError | Validation
return MsgSpecJSONResponse(status_code=StandardResponseCode.HTTP_422, content=content) return MsgSpecJSONResponse(status_code=StandardResponseCode.HTTP_422, content=content)
def register_exception(app: FastAPI) -> None: # noqa: C901 def register_exception(app: FastAPI) -> None: # ruff:ignore[complex-structure]
@app.exception_handler(HTTPException) @app.exception_handler(HTTPException)
async def http_exception_handler(request: Request, exc: HTTPException): async def http_exception_handler(request: Request, exc: HTTPException):
""" """
@@ -190,6 +190,7 @@ def register_exception(app: FastAPI) -> None: # noqa: C901
else: else:
res = response_base.fail(res=CustomResponseCode.HTTP_500) res = response_base.fail(res=CustomResponseCode.HTTP_500)
content = res.model_dump() content = res.model_dump()
ctx.__request_unknown_exception__ = content
content.update(trace_id=get_request_trace_id()) content.update(trace_id=get_request_trace_id())
return MsgSpecJSONResponse( return MsgSpecJSONResponse(
status_code=StandardResponseCode.HTTP_500, status_code=StandardResponseCode.HTTP_500,
@@ -223,6 +224,10 @@ def register_exception(app: FastAPI) -> None: # noqa: C901
else: else:
res = response_base.fail(res=CustomResponseCode.HTTP_500) res = response_base.fail(res=CustomResponseCode.HTTP_500)
content = res.model_dump() content = res.model_dump()
if isinstance(exc, BaseExceptionError):
ctx.__request_custom_exception__ = content
else:
ctx.__request_unknown_exception__ = content
content.update(trace_id=get_request_trace_id()) content.update(trace_id=get_request_trace_id())
response = MsgSpecJSONResponse( response = MsgSpecJSONResponse(
status_code=exc.code if isinstance(exc, BaseExceptionError) else StandardResponseCode.HTTP_500, status_code=exc.code if isinstance(exc, BaseExceptionError) else StandardResponseCode.HTTP_500,
+79
View File
@@ -0,0 +1,79 @@
from collections.abc import Callable
from contextlib import AbstractAsyncContextManager, AsyncExitStack, asynccontextmanager
from typing import Any, TypeAlias, overload
from fastapi import FastAPI
from backend.common.enums import LifespanStage
LifespanFunc: TypeAlias = Callable[[FastAPI], AbstractAsyncContextManager[dict[str, Any] | None]]
class LifespanManager:
"""FastAPI lifespan 管理器"""
def __init__(self) -> None:
self._lifespans: dict[LifespanStage, list[LifespanFunc]] = {
LifespanStage.core: [],
LifespanStage.plugin: [],
LifespanStage.tail: [],
}
@overload
def register(self, func: LifespanFunc) -> LifespanFunc: ...
@overload
def register(self, *, stage: LifespanStage) -> Callable[[LifespanFunc], LifespanFunc]: ...
def register(
self, func: LifespanFunc | None = None, *, stage: LifespanStage = LifespanStage.core
) -> LifespanFunc | Callable[[LifespanFunc], LifespanFunc]:
"""
注册 lifespan hook
:param func: lifespan hook直接装饰时使用
:param stage: 执行阶段控制粗粒度顺序默认为 core
:return:
"""
def decorator(f: LifespanFunc) -> LifespanFunc:
for hooks in self._lifespans.values():
for fn in hooks:
if fn is f:
return f
self._lifespans[stage].append(f)
return f
if func is not None:
return decorator(func)
return decorator
def build(self) -> LifespanFunc:
"""
构建组合后的 lifespan hook
:return:
"""
@asynccontextmanager
async def combined_lifespan(app: FastAPI): # ruff:ignore[missing-return-type-private-function]
state: dict[str, Any] = {}
async with AsyncExitStack() as exit_stack:
for stage in LifespanStage:
for lifespan_fn in self._lifespans[stage]:
result = await exit_stack.enter_async_context(lifespan_fn(app))
if isinstance(result, dict):
state.update(result)
for key, value in state.items():
setattr(app.state, key, value)
yield state or None
return combined_lifespan
# 创建 lifespan_manager 单例
lifespan_manager = LifespanManager()
+39 -5
View File
@@ -46,10 +46,10 @@ class UniversalText(TypeDecorator[str]):
impl = LONGTEXT if DataBaseType.mysql == settings.DATABASE_TYPE else Text impl = LONGTEXT if DataBaseType.mysql == settings.DATABASE_TYPE else Text
cache_ok = True cache_ok = True
def process_bind_param(self, value: str | None, dialect) -> str | None: # noqa: ANN001 def process_bind_param(self, value: str | None, dialect) -> str | None: # ruff:ignore[missing-type-function-argument]
return value return value
def process_result_value(self, value: str | None, dialect) -> str | None: # noqa: ANN001 def process_result_value(self, value: str | None, dialect) -> str | None: # ruff:ignore[missing-type-function-argument]
return value return value
@@ -63,13 +63,13 @@ class TimeZone(TypeDecorator[datetime]):
def python_type(self) -> type[datetime]: def python_type(self) -> type[datetime]:
return datetime return datetime
def process_bind_param(self, value: datetime | None, dialect) -> datetime | None: # noqa: ANN001 def process_bind_param(self, value: datetime | None, dialect) -> datetime | None: # ruff:ignore[missing-type-function-argument]
if value is not None and value.utcoffset() != timezone.now().utcoffset(): if value is not None and value.utcoffset() != timezone.now().utcoffset():
# TODO 处理夏令时偏移 # TODO 处理夏令时偏移
value = timezone.from_datetime(value) value = timezone.from_datetime(value)
return value return value
def process_result_value(self, value: datetime | None, dialect) -> datetime | None: # noqa: ANN001 def process_result_value(self, value: datetime | None, dialect) -> datetime | None: # ruff:ignore[missing-type-function-argument]
if value is not None and value.tzinfo is None: if value is not None and value.tzinfo is None:
value = value.replace(tzinfo=timezone.tz_info) value = value.replace(tzinfo=timezone.tz_info)
return value return value
@@ -83,6 +83,20 @@ class UserMixin(MappedAsDataclass):
updated_by: Mapped[int | None] = mapped_column(init=False, default=None, sort_order=998, comment='修改者') updated_by: Mapped[int | None] = mapped_column(init=False, default=None, sort_order=998, comment='修改者')
class TenantMixin(MappedAsDataclass):
"""租户 Mixin 数据类"""
if settings.TENANT_ENABLED:
tenant_id: Mapped[int] = mapped_column(
BigInteger,
init=False,
nullable=False,
index=True,
sort_order=997,
comment='租户ID',
)
class DateTimeMixin(MappedAsDataclass): class DateTimeMixin(MappedAsDataclass):
"""日期时间 Mixin 数据类""" """日期时间 Mixin 数据类"""
@@ -102,6 +116,26 @@ class DateTimeMixin(MappedAsDataclass):
) )
class LogicalDeleteMixin(MappedAsDataclass):
"""逻辑删除 Mixin 数据类"""
deleted: Mapped[int] = mapped_column(
BigInteger,
init=False,
default=0,
server_default='0',
sort_order=999,
comment='是否已删除(0:否;id:是)',
)
deleted_time: Mapped[datetime | None] = mapped_column(
TimeZone,
init=False,
default=None,
sort_order=999,
comment='删除时间',
)
class MappedBase(AsyncAttrs, DeclarativeBase): class MappedBase(AsyncAttrs, DeclarativeBase):
""" """
声明式基类, 作为所有基类或数据模型类的父类而存在 声明式基类, 作为所有基类或数据模型类的父类而存在
@@ -134,7 +168,7 @@ class DataClassBase(MappedAsDataclass, MappedBase):
__abstract__ = True __abstract__ = True
class Base(DataClassBase, DateTimeMixin): class Base(DataClassBase, DateTimeMixin, LogicalDeleteMixin):
""" """
声明性数据类基类, 带有数据类集成, 并包含 MiXin 数据类基础表结构 声明性数据类基类, 带有数据类集成, 并包含 MiXin 数据类基础表结构
""" """
+9 -7
View File
@@ -20,9 +20,8 @@ from redis.observability.config import OTelConfig
from redis.observability.providers import get_observability_instance from redis.observability.providers import get_observability_instance
from backend.common.log import log, request_id_filter from backend.common.log import log, request_id_filter
from backend.common.observability.prometheus import PROMETHEUS_APP_NAME
from backend.core.conf import settings from backend.core.conf import settings
from backend.database.db import async_engine from backend.database.db import get_database_engines
from backend.database.redis import redis_client from backend.database.redis import redis_client
@@ -103,18 +102,21 @@ def init_otel(app: FastAPI) -> None:
:param app: FastAPI 应用实例 :param app: FastAPI 应用实例
:return: :return:
""" """
resource = init_resource(PROMETHEUS_APP_NAME) resource = init_resource(settings.GRAFANA_PROMETHEUS_APP_NAME)
init_tracer(resource) init_tracer(resource)
init_metrics(resource) init_metrics(resource)
init_logging(resource) init_logging(resource)
# Redis 原生指标
redis_otel = get_observability_instance() redis_otel = get_observability_instance()
redis_otel.init(OTelConfig()) redis_otel.init(OTelConfig())
AsyncioInstrumentor().instrument() AsyncioInstrumentor().instrument()
LoggingInstrumentor().instrument(set_logging_format=True)
SQLAlchemyInstrumentor().instrument(engine=async_engine.sync_engine)
RedisInstrumentor.instrument_client(redis_client) # type: ignore
HTTPXClientInstrumentor().instrument() HTTPXClientInstrumentor().instrument()
# 禁止自动将 OTel handler 安装到 stdlib root logger
# 避免与上面注册的 LoggingHandlerloguru sink)重复推送。
LoggingInstrumentor().instrument(set_logging_format=True, enable_log_auto_instrumentation=False)
RedisInstrumentor.instrument_client(client=redis_client) # type: ignore
for engine in get_database_engines().values():
SQLAlchemyInstrumentor().instrument(engine=engine.sync_engine)
FastAPIInstrumentor.instrument_app(app) FastAPIInstrumentor.instrument_app(app)
@@ -1,37 +0,0 @@
from prometheus_client import Counter, Gauge, Histogram
# 警告: 此值与以下位置强关联,修改必须同步更新,否则会导致 Grafana 指标数据查询失败:
# - deploy/backend/grafana/fba_datasource.yml
# - deploy/backend/grafana/dashboards/fba_server.json
PROMETHEUS_APP_NAME = 'fba_server'
PROMETHEUS_REQUEST_IN_PROGRESS_GAUGE = Gauge(
name='fba_request_in_progress',
documentation='按方法和路径统计请求的衡量',
labelnames=['app_name', 'method', 'path'],
)
PROMETHEUS_REQUEST_COUNTER = Counter(
name='fba_request_total',
documentation='按方法和路径统计请求总数',
labelnames=['app_name', 'method', 'path'],
)
PROMETHEUS_REQUEST_COST_TIME_HISTOGRAM = Histogram(
name='fba_request_cost_time',
documentation='按方法和路径划分请求耗时的直方图(以 ms 为单位)',
labelnames=['app_name', 'method', 'path'],
)
PROMETHEUS_EXCEPTION_COUNTER = Counter(
name='fba_exception_total',
documentation='按方法,路径和异常类型统计异常总数',
labelnames=['app_name', 'method', 'path', 'exception_type'],
)
PROMETHEUS_RESPONSE_COUNTER = Counter(
name='fba_response_total',
documentation='按方法、路径和状态码统计响应总数',
labelnames=['app_name', 'method', 'path', 'status_code'],
)
@@ -0,0 +1,75 @@
from prometheus_client import Counter, Gauge, Histogram
from backend.core.conf import settings
_PROMETHEUS_FASTAPI_REQUEST_IN_PROGRESS_GAUGE = Gauge(
name='fba_request_in_progress',
documentation='按方法和路径统计当前正在处理的 FastAPI 请求数',
labelnames=['app_name', 'method', 'path'],
)
_PROMETHEUS_FASTAPI_REQUEST_COUNTER = Counter(
name='fba_request_total',
documentation='按方法和路径统计 FastAPI 请求总数',
labelnames=['app_name', 'method', 'path'],
)
_PROMETHEUS_FASTAPI_REQUEST_COST_TIME_HISTOGRAM = Histogram(
name='fba_request_cost_time',
documentation='按方法和路径统计 FastAPI 请求耗时直方图(ms)',
labelnames=['app_name', 'method', 'path'],
)
_PROMETHEUS_FASTAPI_EXCEPTION_COUNTER = Counter(
name='fba_exception_total',
documentation='按方法、路径和异常类型统计 FastAPI 异常总数',
labelnames=['app_name', 'method', 'path', 'exception_type'],
)
_PROMETHEUS_FASTAPI_RESPONSE_COUNTER = Counter(
name='fba_response_total',
documentation='按方法、路径和状态码统计 FastAPI 响应总数',
labelnames=['app_name', 'method', 'path', 'status_code'],
)
def inc_fastapi_request_in_progress(*, method: str, path: str) -> None:
"""增加当前正在处理的 FastAPI 请求数"""
_PROMETHEUS_FASTAPI_REQUEST_IN_PROGRESS_GAUGE.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, method=method, path=path
).inc()
def dec_fastapi_request_in_progress(*, method: str, path: str) -> None:
"""减少当前正在处理的 FastAPI 请求数"""
_PROMETHEUS_FASTAPI_REQUEST_IN_PROGRESS_GAUGE.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, method=method, path=path
).dec()
def inc_fastapi_request(*, method: str, path: str) -> None:
"""记录 FastAPI 请求总数"""
_PROMETHEUS_FASTAPI_REQUEST_COUNTER.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, method=method, path=path
).inc()
def observe_fastapi_request_cost_time(*, method: str, path: str, elapsed: float, trace_id: str) -> None:
"""记录 FastAPI 请求耗时"""
_PROMETHEUS_FASTAPI_REQUEST_COST_TIME_HISTOGRAM.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, method=method, path=path
).observe(amount=elapsed, exemplar={settings.GRAFANA_PROMETHEUS_EXEMPLAR_TRACE_ID_KEY: trace_id})
def inc_fastapi_exception(*, method: str, path: str, exception_type: str) -> None:
"""记录 FastAPI 异常总数"""
_PROMETHEUS_FASTAPI_EXCEPTION_COUNTER.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, method=method, path=path, exception_type=exception_type
).inc()
def inc_fastapi_response(*, method: str, path: str, status_code: int | str) -> None:
"""记录 FastAPI 响应总数"""
_PROMETHEUS_FASTAPI_RESPONSE_COUNTER.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, method=method, path=path, status_code=status_code
).inc()
@@ -0,0 +1,47 @@
import time
from asyncio import Queue
from prometheus_client import Counter, Gauge, Histogram
from backend.core.conf import settings
_PROMETHEUS_QUEUE_SIZE_GAUGE = Gauge(
name='fba_queue_size',
documentation='项目内部异步队列当前长度',
labelnames=['app_name', 'queue_name'],
)
_PROMETHEUS_QUEUE_BATCH_DEQUEUE_COST_TIME_HISTOGRAM = Histogram(
name='fba_queue_batch_dequeue_cost_time',
documentation='项目内部异步队列批量消费耗时(ms',
labelnames=['app_name', 'queue_name'],
)
_PROMETHEUS_QUEUE_EXCEPTION_COUNTER = Counter(
name='fba_queue_exception_total',
documentation='项目内部异步队列异常总数',
labelnames=['app_name', 'queue_name'],
)
def observe_queue_size(queue: Queue, *, queue_name: str) -> None:
"""记录队列当前长度"""
_PROMETHEUS_QUEUE_SIZE_GAUGE.labels(app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, queue_name=queue_name).set(
queue.qsize()
)
def observe_batch_dequeue_cost(start_time: float, *, queue_name: str) -> None:
"""记录批量消费耗时"""
elapsed = round((time.perf_counter() - start_time) * 1000, 3)
_PROMETHEUS_QUEUE_BATCH_DEQUEUE_COST_TIME_HISTOGRAM.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, queue_name=queue_name
).observe(elapsed)
def inc_queue_exception(*, queue_name: str) -> None:
"""记录队列异常"""
_PROMETHEUS_QUEUE_EXCEPTION_COUNTER.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, queue_name=queue_name
).inc()
@@ -0,0 +1,37 @@
from typing import Any
from prometheus_client import Gauge
from sqlalchemy.pool import QueuePool
from backend.core.conf import settings
_PROMETHEUS_SQLALCHEMY_POOL_CONNECTIONS_GAUGE = Gauge(
name='fba_sqlalchemy_pool_connections',
documentation='SQLAlchemy 连接池状态',
labelnames=['app_name', 'source', 'state'],
)
def observe_sqlalchemy_pool_connections(
*_event_args: Any,
pool: QueuePool,
source: str = 'default',
) -> None:
"""监听 SQLAlchemy 连接池状态"""
total_size = pool.size()
checked_out_size = pool.checkedout()
overflow_size = pool.overflow()
idle_size = max(total_size + overflow_size - checked_out_size, 0)
_PROMETHEUS_SQLALCHEMY_POOL_CONNECTIONS_GAUGE.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, source=source, state='size'
).set(total_size)
_PROMETHEUS_SQLALCHEMY_POOL_CONNECTIONS_GAUGE.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, source=source, state='checked_out'
).set(checked_out_size)
_PROMETHEUS_SQLALCHEMY_POOL_CONNECTIONS_GAUGE.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, source=source, state='idle'
).set(idle_size)
_PROMETHEUS_SQLALCHEMY_POOL_CONNECTIONS_GAUGE.labels(
app_name=settings.GRAFANA_PROMETHEUS_APP_NAME, source=source, state='overflow'
).set(overflow_size)
+54
View File
@@ -7,6 +7,7 @@ from typing import TYPE_CHECKING, Any, Generic, TypeVar
from fastapi import Depends, Query from fastapi import Depends, Query
from fastapi_pagination import pagination_ctx from fastapi_pagination import pagination_ctx
from fastapi_pagination.bases import AbstractPage, AbstractParams, RawParams from fastapi_pagination.bases import AbstractPage, AbstractParams, RawParams
from fastapi_pagination.cursor import CursorParams
from fastapi_pagination.ext.sqlalchemy import apaginate from fastapi_pagination.ext.sqlalchemy import apaginate
from fastapi_pagination.links.bases import create_links from fastapi_pagination.links.bases import create_links
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
@@ -54,6 +55,14 @@ class _PageDetails(BaseModel):
links: _Links = Field(description='分页链接') links: _Links = Field(description='分页链接')
class _CursorPageDetails(BaseModel):
"""游标分页详情"""
items: list = Field([], description='当前页数据列表')
next_cursor: str | None = Field(None, description='下一页游标')
has_more: bool = Field(description='是否还有更多数据')
class _CustomPage(_PageDetails, AbstractPage[T], Generic[T]): class _CustomPage(_PageDetails, AbstractPage[T], Generic[T]):
"""自定义分页类""" """自定义分页类"""
@@ -86,6 +95,30 @@ class _CustomPage(_PageDetails, AbstractPage[T], Generic[T]):
) )
class _CustomCursorPage(_CursorPageDetails, AbstractPage[T], Generic[T]):
"""自定义游标分页类"""
__params_type__ = CursorParams
@classmethod
def create(
cls,
items: list,
params: CursorParams,
*,
next_: Any = None,
**kwargs: Any,
) -> Self:
if not isinstance(params, CursorParams):
raise TypeError('CustomCursorPage should be used with CursorParams')
return cls(
items=items,
next_cursor=params.encode_cursor(next_),
has_more=next_ is not None,
)
class PageData(_PageDetails, Generic[SchemaT]): class PageData(_PageDetails, Generic[SchemaT]):
""" """
包含返回数据 schema 的统一返回模型仅适用于分页接口 包含返回数据 schema 的统一返回模型仅适用于分页接口
@@ -111,6 +144,12 @@ class PageData(_PageDetails, Generic[SchemaT]):
items: Sequence[SchemaT] items: Sequence[SchemaT]
class CursorPageData(_CursorPageDetails, Generic[SchemaT]):
"""包含返回数据 schema 的统一返回模型,仅适用于游标分页接口,用法与 PageData 相同"""
items: Sequence[SchemaT]
async def paging_data(db: AsyncSession, select: Select, **kwargs) -> dict[str, Any]: async def paging_data(db: AsyncSession, select: Select, **kwargs) -> dict[str, Any]:
""" """
基于 SQLAlchemy 创建分页数据 基于 SQLAlchemy 创建分页数据
@@ -125,5 +164,20 @@ async def paging_data(db: AsyncSession, select: Select, **kwargs) -> dict[str, A
return page_data return page_data
async def cursor_paging_data(db: AsyncSession, select: Select, **kwargs) -> dict[str, Any]:
"""
基于 SQLAlchemy 创建游标分页数据
:param db: 数据库会话
:param select: SQL 查询语句
:param kwargs: 更多 fastapi-pagination apaginate 参数
:return:
"""
paginated_data: _CustomCursorPage = await apaginate(db, select, **kwargs)
page_data = paginated_data.model_dump()
return page_data
# 分页依赖注入 # 分页依赖注入
DependsPagination = Depends(pagination_ctx(_CustomPage)) DependsPagination = Depends(pagination_ctx(_CustomPage))
DependsCursorPagination = Depends(pagination_ctx(_CustomCursorPage))
+55 -1
View File
@@ -1,31 +1,85 @@
import asyncio import asyncio
import time
from asyncio import Queue from asyncio import Queue
from collections.abc import Awaitable, Callable
from typing import TypeVar
from backend.common.log import log from backend.common.log import log
from backend.common.observability.prometheus.queue import (
inc_queue_exception,
observe_batch_dequeue_cost,
observe_queue_size,
)
T = TypeVar('T')
async def batch_dequeue(queue: Queue, max_items: int, timeout: float) -> list: async def batch_dequeue(queue: Queue[T], max_items: int, timeout: float, *, queue_name: str = 'default') -> list[T]:
""" """
从异步队列中获取多个项目 从异步队列中获取多个项目
:param queue: 用于获取项目的 `asyncio.Queue` 队列 :param queue: 用于获取项目的 `asyncio.Queue` 队列
:param max_items: 从队列中获取的最大项目数量 :param max_items: 从队列中获取的最大项目数量
:param timeout: 总的等待超时时间 :param timeout: 总的等待超时时间
:param queue_name: 队列名称用于 Prometheus 标签
:return: :return:
""" """
items = [] items = []
start = time.perf_counter()
async def collector() -> None: async def collector() -> None:
while len(items) < max_items: while len(items) < max_items:
item = await queue.get() item = await queue.get()
items.append(item) items.append(item)
observe_queue_size(queue, queue_name=queue_name)
try: try:
await asyncio.wait_for(collector(), timeout=timeout) await asyncio.wait_for(collector(), timeout=timeout)
except asyncio.TimeoutError: except asyncio.TimeoutError:
pass pass
except Exception as e: except Exception as e:
inc_queue_exception(queue_name=queue_name)
log.error(f'队列批量获取失败: {e}') log.error(f'队列批量获取失败: {e}')
finally:
observe_batch_dequeue_cost(start, queue_name=queue_name)
observe_queue_size(queue, queue_name=queue_name)
return items return items
async def batch_consume(
queue: Queue[T],
max_items: int,
timeout: float,
handler: Callable[..., Awaitable[None]],
*,
queue_name: str = 'default',
error_message: str = '队列批量处理失败',
item_name: str = '数据',
) -> None:
"""
持续批量消费队列
:param queue: 用于获取项目的 `asyncio.Queue` 队列
:param max_items: 从队列中获取的最大项目数量
:param timeout: 总的等待超时时间
:param handler: 批量处理函数
:param queue_name: 队列名称用于 Prometheus 标签
:param error_message: 处理失败日志消息
:param item_name: 队列数据名称
:return:
"""
while True:
items = await batch_dequeue(queue, max_items=max_items, timeout=timeout, queue_name=queue_name)
if not items:
continue
try:
await handler(items)
except Exception as e:
log.error(f'{error_message},丢失 {len(items)}{item_name}: {e}')
finally:
for _ in items:
queue.task_done()
observe_queue_size(queue, queue_name=queue_name)
+39 -5
View File
@@ -1,4 +1,4 @@
from typing import Any, Generic, TypeVar from typing import Any, Generic, TypeVar, overload
from fastapi import Response from fastapi import Response
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
@@ -69,7 +69,7 @@ class ResponseBase:
*, *,
res: CustomResponseCode | CustomResponse, res: CustomResponseCode | CustomResponse,
data: Any | None, data: Any | None,
) -> ResponseModel | ResponseSchemaModel: ) -> ResponseModel | ResponseSchemaModel[Any]:
""" """
请求返回通用方法 请求返回通用方法
@@ -77,14 +77,32 @@ class ResponseBase:
:param data: 返回数据 :param data: 返回数据
:return: :return:
""" """
return ResponseModel(code=res.code, msg=res.msg, data=data) if data is None:
return ResponseModel(code=res.code, msg=res.msg, data=data)
return ResponseSchemaModel[Any](code=res.code, msg=res.msg, data=data)
@overload
def success(
self,
*,
res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_200,
data: None = None,
) -> ResponseModel: ...
@overload
def success(
self,
*,
res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_200,
data: SchemaT,
) -> ResponseSchemaModel[SchemaT]: ...
def success( def success(
self, self,
*, *,
res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_200, res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_200,
data: Any | None = None, data: Any | None = None,
) -> ResponseModel | ResponseSchemaModel: ) -> ResponseModel | ResponseSchemaModel[Any]:
""" """
成功响应 成功响应
@@ -94,12 +112,28 @@ class ResponseBase:
""" """
return self.__response(res=res, data=data) return self.__response(res=res, data=data)
@overload
def fail(
self,
*,
res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_400,
data: None = None,
) -> ResponseModel: ...
@overload
def fail(
self,
*,
res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_400,
data: SchemaT,
) -> ResponseSchemaModel[SchemaT]: ...
def fail( def fail(
self, self,
*, *,
res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_400, res: CustomResponseCode | CustomResponse = CustomResponseCode.HTTP_400,
data: Any = None, data: Any = None,
) -> ResponseModel | ResponseSchemaModel: ) -> ResponseModel | ResponseSchemaModel[Any]:
""" """
失败响应 失败响应
+1 -1
View File
@@ -14,7 +14,7 @@ class CustomEmailStr(EmailStr):
"""自定义邮箱类型""" """自定义邮箱类型"""
@classmethod @classmethod
def _validate(cls, input_value: str, /) -> str: def _validate(cls, input_value: str, /) -> str | None:
return None if not input_value else validate_email(input_value)[1] return None if not input_value else validate_email(input_value)[1]
+114 -26
View File
@@ -2,14 +2,15 @@ import json
import uuid import uuid
from datetime import timedelta from datetime import timedelta
from typing import Any from typing import Annotated, Any
from fastapi import Depends, Request from fastapi import Depends, Request
from fastapi.security import HTTPBearer from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from fastapi.security.utils import get_authorization_scheme_param from fastapi.security.utils import get_authorization_scheme_param
from jose import ExpiredSignatureError, JWTError, jwt from jose import ExpiredSignatureError, JWTError, jwt
from pydantic_core import from_json from pydantic_core import from_json
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from starlette.authentication import UnauthenticatedUser
from backend.app.admin.model import User from backend.app.admin.model import User
from backend.app.admin.schema.user import GetUserInfoWithRelationDetail from backend.app.admin.schema.user import GetUserInfoWithRelationDetail
@@ -21,9 +22,6 @@ from backend.database.db import async_db_session
from backend.database.redis import redis_client from backend.database.redis import redis_client
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
# JWT dependency injection
DependsJwtAuth = Depends(HTTPBearer())
def jwt_encode(payload: dict[str, Any]) -> str: def jwt_encode(payload: dict[str, Any]) -> str:
""" """
@@ -52,7 +50,8 @@ def jwt_decode(token: str) -> TokenPayload:
session_uuid = payload.get('session_uuid') session_uuid = payload.get('session_uuid')
user_id = payload.get('sub') user_id = payload.get('sub')
expire = payload.get('exp') expire = payload.get('exp')
if not session_uuid or not user_id or not expire: tenant_id = payload.get('tenant_id')
if not session_uuid or not user_id or not expire or tenant_id is None:
raise errors.TokenError(msg='Token 无效') raise errors.TokenError(msg='Token 无效')
except ExpiredSignatureError: except ExpiredSignatureError:
raise errors.TokenError(msg='Token 已过期') raise errors.TokenError(msg='Token 已过期')
@@ -62,14 +61,22 @@ def jwt_decode(token: str) -> TokenPayload:
user_id=int(user_id), user_id=int(user_id),
session_uuid=session_uuid, session_uuid=session_uuid,
expire_time=timezone.from_datetime(timezone.to_utc(expire)), expire_time=timezone.from_datetime(timezone.to_utc(expire)),
tenant_id=int(tenant_id),
) )
async def create_access_token(user_id: int, *, multi_login: bool, **kwargs) -> AccessToken: async def create_access_token(
user_id: int,
tenant_id: int,
*,
multi_login: bool,
**kwargs,
) -> AccessToken:
""" """
生成加密 token 生成加密 token
:param user_id: 用户 ID :param user_id: 用户 ID
:param tenant_id: 租户 ID
:param multi_login: 是否允许多端登录 :param multi_login: 是否允许多端登录
:param kwargs: token 额外信息 :param kwargs: token 额外信息
:return: :return:
@@ -80,34 +87,36 @@ async def create_access_token(user_id: int, *, multi_login: bool, **kwargs) -> A
'session_uuid': session_uuid, 'session_uuid': session_uuid,
'exp': timezone.to_utc(expire).timestamp(), 'exp': timezone.to_utc(expire).timestamp(),
'sub': str(user_id), 'sub': str(user_id),
'tenant_id': tenant_id,
}) })
if not multi_login: if not multi_login:
await redis_client.delete_prefix(f'{settings.TOKEN_REDIS_PREFIX}:{user_id}') await redis_client.delete_by_prefix(f'{settings.TOKEN_REDIS_PREFIX}:{user_id}')
await redis_client.setex( await redis_client.set(
f'{settings.TOKEN_REDIS_PREFIX}:{user_id}:{session_uuid}', f'{settings.TOKEN_REDIS_PREFIX}:{user_id}:{session_uuid}',
settings.TOKEN_EXPIRE_SECONDS,
access_token, access_token,
ex=settings.TOKEN_EXPIRE_SECONDS,
) )
# Token 附加信息单独存储 # Token 附加信息单独存储
if kwargs: if kwargs:
await redis_client.setex( await redis_client.set(
f'{settings.TOKEN_EXTRA_INFO_REDIS_PREFIX}:{user_id}:{session_uuid}', f'{settings.TOKEN_EXTRA_INFO_REDIS_PREFIX}:{user_id}:{session_uuid}',
settings.TOKEN_EXPIRE_SECONDS,
json.dumps(kwargs, ensure_ascii=False), json.dumps(kwargs, ensure_ascii=False),
ex=settings.TOKEN_EXPIRE_SECONDS,
) )
return AccessToken(access_token=access_token, access_token_expire_time=expire, session_uuid=session_uuid) return AccessToken(access_token=access_token, access_token_expire_time=expire, session_uuid=session_uuid)
async def create_refresh_token(session_uuid: str, user_id: int, *, multi_login: bool) -> RefreshToken: async def create_refresh_token(session_uuid: str, user_id: int, tenant_id: int, *, multi_login: bool) -> RefreshToken:
""" """
生成加密刷新 token仅用于创建新的 token 生成加密刷新 token仅用于创建新的 token
:param session_uuid: 会话 UUID :param session_uuid: 会话 UUID
:param user_id: 用户 ID :param user_id: 用户 ID
:param tenant_id: 租户 ID
:param multi_login: 是否允许多端登录 :param multi_login: 是否允许多端登录
:return: :return:
""" """
@@ -116,15 +125,16 @@ async def create_refresh_token(session_uuid: str, user_id: int, *, multi_login:
'session_uuid': session_uuid, 'session_uuid': session_uuid,
'exp': timezone.to_utc(expire).timestamp(), 'exp': timezone.to_utc(expire).timestamp(),
'sub': str(user_id), 'sub': str(user_id),
'tenant_id': tenant_id,
}) })
if not multi_login: if not multi_login:
await redis_client.delete_prefix(f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}') await redis_client.delete_by_prefix(f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}')
await redis_client.setex( await redis_client.set(
f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}:{session_uuid}', f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}:{session_uuid}',
settings.TOKEN_REFRESH_EXPIRE_SECONDS,
refresh_token, refresh_token,
ex=settings.TOKEN_REFRESH_EXPIRE_SECONDS,
) )
return RefreshToken(refresh_token=refresh_token, refresh_token_expire_time=expire) return RefreshToken(refresh_token=refresh_token, refresh_token_expire_time=expire)
@@ -133,6 +143,7 @@ async def create_new_token(
refresh_token: str, refresh_token: str,
session_uuid: str, session_uuid: str,
user_id: int, user_id: int,
tenant_id: int,
*, *,
multi_login: bool, multi_login: bool,
**kwargs, **kwargs,
@@ -143,6 +154,7 @@ async def create_new_token(
:param refresh_token: 刷新 token :param refresh_token: 刷新 token
:param session_uuid: 会话 UUID :param session_uuid: 会话 UUID
:param user_id: 用户 ID :param user_id: 用户 ID
:param tenant_id: 租户 ID
:param multi_login: 是否允许多端登录 :param multi_login: 是否允许多端登录
:param kwargs: token 附加信息 :param kwargs: token 附加信息
:return: :return:
@@ -154,8 +166,18 @@ async def create_new_token(
await redis_client.delete(f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}:{session_uuid}') await redis_client.delete(f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user_id}:{session_uuid}')
await redis_client.delete(f'{settings.TOKEN_REDIS_PREFIX}:{user_id}:{session_uuid}') await redis_client.delete(f'{settings.TOKEN_REDIS_PREFIX}:{user_id}:{session_uuid}')
new_access_token = await create_access_token(user_id, multi_login=multi_login, **kwargs) new_access_token = await create_access_token(
new_refresh_token = await create_refresh_token(new_access_token.session_uuid, user_id, multi_login=multi_login) user_id,
tenant_id,
multi_login=multi_login,
**kwargs,
)
new_refresh_token = await create_refresh_token(
new_access_token.session_uuid,
user_id,
tenant_id,
multi_login=multi_login,
)
return NewToken( return NewToken(
new_access_token=new_access_token.access_token, new_access_token=new_access_token.access_token,
new_access_token_expire_time=new_access_token.access_token_expire_time, new_access_token_expire_time=new_access_token.access_token_expire_time,
@@ -191,6 +213,41 @@ def get_token(request: Request) -> str:
return token return token
async def check_tenant_status(db: AsyncSession, tenant_id: int) -> None:
"""
校验租户状态
:param db: 数据库会话
:param tenant_id: 租户 ID
:return:
"""
if not settings.TENANT_ENABLED:
return
if tenant_id == settings.TENANT_DEFAULT_ID:
return
try:
from backend.plugin.tenant.crud.crud_package import tenant_package_dao
from backend.plugin.tenant.crud.crud_tenant import tenant_dao
except ImportError:
raise errors.ServerError(msg='租户插件方法导入失败,请联系系统管理员')
tenant = await tenant_dao.get(db, tenant_id)
if not tenant:
raise errors.NotFoundError(msg='租户不存在,请联系系统管理员')
if tenant.status == 0:
raise errors.AuthorizationError(msg='租户已被禁用,请联系系统管理员')
if tenant.expire_time and tenant.expire_time < timezone.now():
raise errors.AuthorizationError(msg='租户已过期,请联系系统管理员')
package = await tenant_package_dao.get(db, tenant.package_id)
if package and package.status == 0:
raise errors.AuthorizationError(msg='租户套餐已被禁用,请联系系统管理员')
async def get_current_user(db: AsyncSession, pk: int) -> User: async def get_current_user(db: AsyncSession, pk: int) -> User:
""" """
获取当前用户 获取当前用户
@@ -206,11 +263,14 @@ async def get_current_user(db: AsyncSession, pk: int) -> User:
raise errors.TokenError(msg='Token 无效') raise errors.TokenError(msg='Token 无效')
if not user.status: if not user.status:
raise errors.AuthorizationError(msg='用户已被锁定,请联系系统管理员') raise errors.AuthorizationError(msg='用户已被锁定,请联系系统管理员')
if user.dept and user.dept_id:
if not user.dept.status: if settings.TENANT_ENABLED:
raise errors.AuthorizationError(msg='用户所属部门已被锁定,请联系系统管理员') await check_tenant_status(db, ctx.tenant_id)
if user.dept.del_flag:
raise errors.AuthorizationError(msg='用户所属部门已被删除,请联系系统管理员') if user.dept_id and not user.dept:
raise errors.AuthorizationError(msg='用户所属部门不存在或已被删除,请联系系统管理员')
if user.dept and not user.dept.status:
raise errors.AuthorizationError(msg='用户所属部门已被锁定,请联系系统管理员')
if user.roles: if user.roles:
role_status = [role.status for role in user.roles] role_status = [role.status for role in user.roles]
if all(status == 0 for status in role_status): if all(status == 0 for status in role_status):
@@ -230,10 +290,10 @@ async def get_jwt_user(user_id: int) -> GetUserInfoWithRelationDetail:
async with async_db_session() as db: async with async_db_session() as db:
current_user = await get_current_user(db, user_id) current_user = await get_current_user(db, user_id)
user = GetUserInfoWithRelationDetail.model_validate(current_user) user = GetUserInfoWithRelationDetail.model_validate(current_user)
await redis_client.setex( await redis_client.set(
f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}', f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}',
settings.TOKEN_EXPIRE_SECONDS,
user.model_dump_json(), user.model_dump_json(),
ex=settings.TOKEN_EXPIRE_SECONDS,
) )
else: else:
# TODO: 在恰当的时机,应替换为使用 model_validate_json # TODO: 在恰当的时机,应替换为使用 model_validate_json
@@ -251,6 +311,7 @@ async def jwt_authentication(token: str) -> GetUserInfoWithRelationDetail:
""" """
token_payload = jwt_decode(token) token_payload = jwt_decode(token)
ctx.user_id = token_payload.user_id ctx.user_id = token_payload.user_id
ctx.tenant_id = token_payload.tenant_id
redis_token = await redis_client.get(f'{settings.TOKEN_REDIS_PREFIX}:{ctx.user_id}:{token_payload.session_uuid}') redis_token = await redis_client.get(f'{settings.TOKEN_REDIS_PREFIX}:{ctx.user_id}:{token_payload.session_uuid}')
if not redis_token: if not redis_token:
raise errors.TokenError(msg='Token 已过期') raise errors.TokenError(msg='Token 已过期')
@@ -258,7 +319,31 @@ async def jwt_authentication(token: str) -> GetUserInfoWithRelationDetail:
if token != redis_token: if token != redis_token:
raise errors.TokenError(msg='Token 已失效') raise errors.TokenError(msg='Token 已失效')
return await get_jwt_user(ctx.user_id) user = await get_jwt_user(ctx.user_id)
ctx.is_superuser = user.is_superuser
return user
def jwt_authentication_verify(
request: Request,
token: Annotated[HTTPAuthorizationCredentials, Depends(HTTPBearer())],
) -> str:
"""
JWT 认证依赖
:param request: FastAPI 请求对象
:param token: HTTP Bearer 认证信息
:return:
"""
if isinstance(request.user, UnauthenticatedUser):
if token_exception := ctx.get('__request_jwt_authentication_exception__'):
raise token_exception
raise errors.TokenError
return token.credentials
# JWT 依赖注入
DependsJwtAuth = Depends(jwt_authentication_verify)
def superuser_verify(request: Request, _token: str = DependsJwtAuth) -> bool: def superuser_verify(request: Request, _token: str = DependsJwtAuth) -> bool:
@@ -269,6 +354,9 @@ def superuser_verify(request: Request, _token: str = DependsJwtAuth) -> bool:
:param _token: JWT 令牌 :param _token: JWT 令牌
:return: :return:
""" """
if isinstance(request.user, UnauthenticatedUser):
raise errors.TokenError
superuser = request.user.is_superuser superuser = request.user.is_superuser
if not superuser or not request.user.is_staff: if not superuser or not request.user.is_staff:
raise errors.AuthorizationError raise errors.AuthorizationError
+3 -2
View File
@@ -54,7 +54,7 @@ def get_data_permission_models() -> dict[str, object]:
return {getattr(model, '__name__', str(model)): model for model in get_all_models()} return {getattr(model, '__name__', str(model)): model for model in get_all_models()}
def filter_data_permission( # noqa: C901 def filter_data_permission( # ruff:ignore[complex-structure]
request: Request, *models: type[Model] | AliasedClass | Alias | Table request: Request, *models: type[Model] | AliasedClass | Alias | Table
) -> ColumnElement[bool]: ) -> ColumnElement[bool]:
""" """
@@ -85,8 +85,9 @@ def filter_data_permission( # noqa: C901
if scope.status: if scope.status:
data_rules.update(rule for rule in scope.rules if rule is not None) data_rules.update(rule for rule in scope.rules if rule is not None)
# 启用数据权限过滤,但没有已启用的数据权限
if not data_rules: if not data_rules:
return or_(1 == 1) return or_(1 != 1)
# 目标模型 # 目标模型
target_model_map = ( target_model_map = (
+3 -1
View File
@@ -7,7 +7,7 @@ from backend.common.security.jwt import DependsJwtAuth
from backend.core.conf import settings from backend.core.conf import settings
async def rbac_verify(request: Request, _token: str = DependsJwtAuth) -> None: # noqa: C901 async def rbac_verify(request: Request, _token: str = DependsJwtAuth) -> None: # ruff:ignore[complex-structure]
""" """
RBAC 权限校验鉴权顺序很重要谨慎修改 RBAC 权限校验鉴权顺序很重要谨慎修改
@@ -34,6 +34,8 @@ async def rbac_verify(request: Request, _token: str = DependsJwtAuth) -> None:
# 检测用户角色 # 检测用户角色
user_roles = request.user.roles user_roles = request.user.roles
if not user_roles:
raise errors.AuthorizationError(msg='用户未分配角色,请联系系统管理员')
enabled_roles = [role for role in user_roles if role.status == StatusType.enable] enabled_roles = [role for role in user_roles if role.status == StatusType.enable]
if not enabled_roles: if not enabled_roles:
raise errors.AuthorizationError(msg='用户所属角色已被锁定,请联系系统管理员') raise errors.AuthorizationError(msg='用户所属角色已被锁定,请联系系统管理员')
+1 -1
View File
@@ -1 +1 @@
from .actions import * # noqa: F403 from .actions import * # ruff:ignore[undefined-local-with-import-star]
+21 -1
View File
@@ -14,6 +14,10 @@ from backend.database.redis import redis_client
sio = socketio.AsyncServer( sio = socketio.AsyncServer(
client_manager=socketio.AsyncRedisManager( client_manager=socketio.AsyncRedisManager(
f'redis://:{urllib.parse.quote(settings.REDIS_PASSWORD)}@{settings.REDIS_HOST}:{settings.REDIS_PORT}/{settings.REDIS_DATABASE}', f'redis://:{urllib.parse.quote(settings.REDIS_PASSWORD)}@{settings.REDIS_HOST}:{settings.REDIS_PORT}/{settings.REDIS_DATABASE}',
redis_options={
'socket_timeout': None,
'socket_connect_timeout': settings.REDIS_TIMEOUT,
},
), ),
async_mode='asgi', async_mode='asgi',
cors_allowed_origins=settings.CORS_ALLOWED_ORIGINS, cors_allowed_origins=settings.CORS_ALLOWED_ORIGINS,
@@ -37,6 +41,11 @@ async def connect(sid, environ, auth) -> bool:
# 免授权直连 # 免授权直连
if token == settings.WS_NO_AUTH_MARKER: if token == settings.WS_NO_AUTH_MARKER:
if settings.ENVIRONMENT == 'prod':
log.error('WebSocket 连接失败:生产环境禁止免授权直连')
return False
await redis_client.set(f'{settings.TOKEN_ONLINE_REDIS_PREFIX}:sid:{sid}', session_uuid)
await redis_client.sadd(f'{settings.TOKEN_ONLINE_REDIS_PREFIX}:session:{session_uuid}', sid)
await redis_client.sadd(settings.TOKEN_ONLINE_REDIS_PREFIX, session_uuid) await redis_client.sadd(settings.TOKEN_ONLINE_REDIS_PREFIX, session_uuid)
return True return True
@@ -47,6 +56,8 @@ async def connect(sid, environ, auth) -> bool:
log.info(f'WebSocket 连接失败:{e!s}') log.info(f'WebSocket 连接失败:{e!s}')
return False return False
await redis_client.set(f'{settings.TOKEN_ONLINE_REDIS_PREFIX}:sid:{sid}', session_uuid)
await redis_client.sadd(f'{settings.TOKEN_ONLINE_REDIS_PREFIX}:session:{session_uuid}', sid)
await redis_client.sadd(settings.TOKEN_ONLINE_REDIS_PREFIX, session_uuid) await redis_client.sadd(settings.TOKEN_ONLINE_REDIS_PREFIX, session_uuid)
return True return True
@@ -54,4 +65,13 @@ async def connect(sid, environ, auth) -> bool:
@sio.event @sio.event
async def disconnect(sid) -> None: async def disconnect(sid) -> None:
"""Socket 断开连接事件""" """Socket 断开连接事件"""
await redis_client.spop(settings.TOKEN_ONLINE_REDIS_PREFIX) session_uuid = await redis_client.get(f'{settings.TOKEN_ONLINE_REDIS_PREFIX}:sid:{sid}')
if not session_uuid:
return
session_key = f'{settings.TOKEN_ONLINE_REDIS_PREFIX}:session:{session_uuid}'
await redis_client.delete(f'{settings.TOKEN_ONLINE_REDIS_PREFIX}:sid:{sid}')
await redis_client.srem(session_key, sid)
if await redis_client.scard(session_key) == 0:
await redis_client.delete(session_key)
await redis_client.srem(settings.TOKEN_ONLINE_REDIS_PREFIX, session_uuid)
@@ -4,13 +4,14 @@ import pytest
from starlette.testclient import TestClient from starlette.testclient import TestClient
from backend.app.admin.tests.utils.db import override_get_db
from backend.core.conf import settings from backend.core.conf import settings
from backend.database.db import get_db from backend.database.db import get_db, get_db_transaction
from backend.main import app from backend.main import app
from backend.tests.utils.db import override_get_db, override_get_db_transaction
# 重载数据库 # 重载数据库
app.dependency_overrides[get_db] = override_get_db app.dependency_overrides[get_db] = override_get_db
app.dependency_overrides[get_db_transaction] = override_get_db_transaction
# Test data # Test data
@@ -19,7 +20,7 @@ PYTEST_PASSWORD = '123456'
PYTEST_BASE_URL = f'http://testserver{settings.FASTAPI_API_V1_PATH}' PYTEST_BASE_URL = f'http://testserver{settings.FASTAPI_API_V1_PATH}'
@pytest.fixture(scope='module') @pytest.fixture(scope='session')
def client() -> Generator: def client() -> Generator:
with TestClient(app, base_url=PYTEST_BASE_URL) as c: with TestClient(app, base_url=PYTEST_BASE_URL) as c:
yield c yield c
+39 -9
View File
@@ -4,7 +4,7 @@ from functools import cache
from re import Pattern from re import Pattern
from typing import Any, Literal from typing import Any, Literal
from pydantic import model_validator from pydantic import Field, model_validator
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource, SettingsConfigDict from pydantic_settings import BaseSettings, PydanticBaseSettingsSource, SettingsConfigDict
from backend.core.path_conf import ENV_EXAMPLE_FILE_PATH, ENV_FILE_PATH from backend.core.path_conf import ENV_EXAMPLE_FILE_PATH, ENV_FILE_PATH
@@ -51,6 +51,7 @@ class Settings(BaseSettings):
DATABASE_PORT: int DATABASE_PORT: int
DATABASE_USER: str DATABASE_USER: str
DATABASE_PASSWORD: str DATABASE_PASSWORD: str
DATABASE_SOURCES: dict[str, str] = Field(default_factory=dict)
# 数据库 # 数据库
DATABASE_ECHO: bool | Literal['debug'] = False DATABASE_ECHO: bool | Literal['debug'] = False
@@ -80,6 +81,7 @@ class Settings(BaseSettings):
CACHE_PUBSUB_MAX_RECONNECT_ATTEMPTS: int = 10 # 最大重连次数 CACHE_PUBSUB_MAX_RECONNECT_ATTEMPTS: int = 10 # 最大重连次数
# .env Snowflake # .env Snowflake
SNOWFLAKE_ENABLED: bool = False
SNOWFLAKE_DATACENTER_ID: int | None = None SNOWFLAKE_DATACENTER_ID: int | None = None
SNOWFLAKE_WORKER_ID: int | None = None SNOWFLAKE_WORKER_ID: int | None = None
@@ -99,12 +101,11 @@ class Settings(BaseSettings):
TOKEN_EXTRA_INFO_REDIS_PREFIX: str = 'fba:token_extra_info' TOKEN_EXTRA_INFO_REDIS_PREFIX: str = 'fba:token_extra_info'
TOKEN_ONLINE_REDIS_PREFIX: str = 'fba:token_online' TOKEN_ONLINE_REDIS_PREFIX: str = 'fba:token_online'
TOKEN_REFRESH_REDIS_PREFIX: str = 'fba:refresh_token' TOKEN_REFRESH_REDIS_PREFIX: str = 'fba:refresh_token'
TOKEN_REQUEST_UNDERLYING_SECURITY: bool = True
TOKEN_REQUEST_PATH_EXCLUDE: list[str] = [ # JWT / RBAC 路由白名单 TOKEN_REQUEST_PATH_EXCLUDE: list[str] = [ # JWT / RBAC 路由白名单
f'{FASTAPI_API_V1_PATH}/auth/login', f'{FASTAPI_API_V1_PATH}/auth/login',
] ]
TOKEN_REQUEST_PATH_EXCLUDE_PATTERN: list[Pattern[str]] = [ # JWT / RBAC 路由白名单(正则) TOKEN_REQUEST_PATH_EXCLUDE_PATTERN: list[Pattern[str]] = [] # JWT / RBAC 路由白名单(正则)
rf'^{FASTAPI_API_V1_PATH}/monitors/(redis|server)$',
]
# 用户安全 # 用户安全
USER_LOCK_REDIS_PREFIX: str = 'fba:user:lock' USER_LOCK_REDIS_PREFIX: str = 'fba:user:lock'
@@ -128,10 +129,7 @@ class Settings(BaseSettings):
# RBAC # RBAC
RBAC_ROLE_MENU_MODE: bool = True RBAC_ROLE_MENU_MODE: bool = True
RBAC_ROLE_MENU_EXCLUDE: list[str] = [ RBAC_ROLE_MENU_EXCLUDE: list[str] = []
'sys:monitor:redis',
'sys:monitor:server',
]
# Cookie # Cookie
COOKIE_REFRESH_TOKEN_KEY: str = 'fba_refresh_token' COOKIE_REFRESH_TOKEN_KEY: str = 'fba_refresh_token'
@@ -147,7 +145,8 @@ class Settings(BaseSettings):
DATA_PERMISSION_COLUMN_EXCLUDE: list[str] = [ # 排除允许进行数据过滤的 SQLA 模型列 DATA_PERMISSION_COLUMN_EXCLUDE: list[str] = [ # 排除允许进行数据过滤的 SQLA 模型列
'id', 'id',
'sort', 'sort',
'del_flag', 'deleted',
'deleted_time',
'created_time', 'created_time',
'updated_time', 'updated_time',
] ]
@@ -245,6 +244,11 @@ class Settings(BaseSettings):
OPERA_LOG_QUEUE_MAXSIZE: int = 100000 OPERA_LOG_QUEUE_MAXSIZE: int = 100000
OPERA_LOG_QUEUE_BATCH_CONSUME_SIZE: int = 100 OPERA_LOG_QUEUE_BATCH_CONSUME_SIZE: int = 100
OPERA_LOG_QUEUE_TIMEOUT: int = 60 # 1 分钟 OPERA_LOG_QUEUE_TIMEOUT: int = 60 # 1 分钟
OPERA_LOG_BODY_MAX_SIZE: int = 10240 # 10 KB
# 租户
TENANT_ENABLED: bool = True
TENANT_DEFAULT_ID: int = 0
# Plugin 配置 # Plugin 配置
PLUGIN_REQUIRED: list[str] = ['dict'] PLUGIN_REQUIRED: list[str] = ['dict']
@@ -259,6 +263,17 @@ class Settings(BaseSettings):
# Grafana # Grafana
GRAFANA_METRICS_ENABLE: bool = False GRAFANA_METRICS_ENABLE: bool = False
GRAFANA_OTLP_GRPC_ENDPOINT: str = 'fba_alloy:4317' GRAFANA_OTLP_GRPC_ENDPOINT: str = 'fba_alloy:4317'
# 以下配置为静态定义,修改后需要手动同步相关 Grafana 配置:
# - GRAFANA_PROMETHEUS_APP_NAMEdeploy/backend/grafana/fba_datasource.yml
# deploy/backend/grafana/dashboards/fba_server.json
# - GRAFANA_CELERY_OTEL_SERVICE_NAMEdeploy/backend/grafana/dashboards/fba_celery.json
# - GRAFANA_METRICS_PATHdeploy/backend/grafana/fba_config.alloy
# deploy/backend/grafana/dashboards/fba_server.json
# - GRAFANA_PROMETHEUS_EXEMPLAR_TRACE_ID_KEYdeploy/backend/grafana/fba_datasource.yml
GRAFANA_PROMETHEUS_APP_NAME: str = 'fba_server'
GRAFANA_CELERY_OTEL_SERVICE_NAME: str = 'fba_celery_worker'
GRAFANA_METRICS_PATH: str = '/metrics'
GRAFANA_PROMETHEUS_EXEMPLAR_TRACE_ID_KEY: str = 'TraceID'
################################################## ##################################################
# [ App ] task # [ App ] task
@@ -315,6 +330,21 @@ class Settings(BaseSettings):
EMAIL_CAPTCHA_REDIS_PREFIX: str EMAIL_CAPTCHA_REDIS_PREFIX: str
EMAIL_CAPTCHA_EXPIRE_SECONDS: int EMAIL_CAPTCHA_EXPIRE_SECONDS: int
##################################################
# [ Plugin ] ai
##################################################
# 动态配置
AI_EXA_API_KEY: str = ''
AI_TAVILY_API_KEY: str = ''
# 基础配置(in plugin.toml
AI_CODE_MODE_DYNAMIC_CATALOG: bool = False
AI_CODE_MODE_MAX_RETRIES: int = 3
AI_CODE_MODE_TOOLS: list[str] = []
AI_CONTEXT_WARNING_THRESHOLD: float = 0.8
AI_HTTP_MAX_RETRIES: int = 5
AI_MCP_MAX_RETRIES: int = 1
@model_validator(mode='before') @model_validator(mode='before')
@classmethod @classmethod
def check_env(cls, values: Any) -> Any: def check_env(cls, values: Any) -> Any:
+44 -14
View File
@@ -1,6 +1,6 @@
import asyncio
import os import os
from asyncio import create_task
from collections.abc import AsyncGenerator from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
@@ -18,19 +18,21 @@ from starlette_context.plugins import RequestIdPlugin
from backend import __version__ from backend import __version__
from backend.common.cache.pubsub import cache_pubsub_manager from backend.common.cache.pubsub import cache_pubsub_manager
from backend.common.exception.exception_handler import register_exception from backend.common.exception.exception_handler import register_exception
from backend.common.lifespan import lifespan_manager
from backend.common.log import set_custom_logfile, setup_logging from backend.common.log import set_custom_logfile, setup_logging
from backend.common.observability.otel import init_otel from backend.common.observability.otel import init_otel
from backend.common.response.response_code import StandardResponseCode from backend.common.response.response_code import StandardResponseCode
from backend.core.conf import settings from backend.core.conf import settings
from backend.core.path_conf import STATIC_DIR, UPLOAD_DIR from backend.core.path_conf import STATIC_DIR, UPLOAD_DIR
from backend.database.db import create_tables from backend.database.db import create_tables, dispose_database
from backend.database.redis import redis_client from backend.database.redis import redis_client
from backend.middleware.access_middleware import AccessMiddleware from backend.middleware.access_middleware import AccessMiddleware
from backend.middleware.i18n_middleware import I18nMiddleware from backend.middleware.i18n_middleware import I18nMiddleware
from backend.middleware.jwt_auth_middleware import JwtAuthMiddleware from backend.middleware.jwt_auth_middleware import JwtAuthMiddleware
from backend.middleware.opera_log_middleware import OperaLogMiddleware from backend.middleware.opera_log_middleware import OperaLogMiddleware
from backend.middleware.state_middleware import StateMiddleware from backend.middleware.state_middleware import StateMiddleware
from backend.plugin.core import build_final_router from backend.plugin.hooks import init_plugin_otel_hooks, register_plugin_hooks
from backend.plugin.router import build_final_router
from backend.utils.demo_mode import demo_site from backend.utils.demo_mode import demo_site
from backend.utils.openapi import ensure_unique_route_names, simplify_operation_ids from backend.utils.openapi import ensure_unique_route_names, simplify_operation_ids
from backend.utils.serializers import MsgSpecJSONResponse from backend.utils.serializers import MsgSpecJSONResponse
@@ -38,6 +40,7 @@ from backend.utils.snowflake import snowflake
from backend.utils.trace_id import OtelTraceIdPlugin from backend.utils.trace_id import OtelTraceIdPlugin
@lifespan_manager.register
@asynccontextmanager @asynccontextmanager
async def register_init(app: FastAPI) -> AsyncGenerator[None, None]: async def register_init(app: FastAPI) -> AsyncGenerator[None, None]:
""" """
@@ -53,24 +56,47 @@ async def register_init(app: FastAPI) -> AsyncGenerator[None, None]:
await redis_client.init() await redis_client.init()
# 初始化 snowflake 节点 # 初始化 snowflake 节点
await snowflake.init() if settings.SNOWFLAKE_ENABLED or settings.DATABASE_PK_MODE == 'snowflake':
await snowflake.init()
# 创建操作日志任务 # 创建操作日志任务
create_task(OperaLogMiddleware.consumer()) opera_log_task = asyncio.create_task(OperaLogMiddleware.consumer())
# 启动缓存 Pub/Sub 监听器 # 启动缓存 Pub/Sub 监听器
cache_pubsub_manager.start_listener() cache_pubsub_manager.start_listener()
yield # 注册租户 SQLAlchemy 监听器
if settings.TENANT_ENABLED:
try:
from backend.plugin.tenant.listener import register_tenant_sqlalchemy_listeners
except ImportError:
raise ImportError('租户插件监听器导入失败,请联系系统管理员')
else:
register_tenant_sqlalchemy_listeners()
# 停止缓存 Pub/Sub 监听器 try:
await cache_pubsub_manager.stop_listener() yield
finally:
# 停止缓存 Pub/Sub 监听器
await cache_pubsub_manager.stop_listener()
# 释放 snowflake 节点 # 取消操作日志任务
await snowflake.shutdown() if not opera_log_task.done():
opera_log_task.cancel()
try:
await opera_log_task
except asyncio.CancelledError:
pass
# 关闭 redis 连接 # 释放 snowflake 节点
await redis_client.aclose() if settings.SNOWFLAKE_ENABLED or settings.DATABASE_PK_MODE == 'snowflake':
await snowflake.shutdown()
# 关闭 redis 连接
await redis_client.aclose()
# 释放数据库连接池
await dispose_database()
def register_app() -> FastAPI: def register_app() -> FastAPI:
@@ -84,7 +110,7 @@ def register_app() -> FastAPI:
redoc_url=settings.FASTAPI_REDOC_URL, redoc_url=settings.FASTAPI_REDOC_URL,
openapi_url=settings.FASTAPI_OPENAPI_URL, openapi_url=settings.FASTAPI_OPENAPI_URL,
default_response_class=MsgSpecJSONResponse, default_response_class=MsgSpecJSONResponse,
lifespan=register_init, lifespan=lifespan_manager.build(),
) )
# 注册组件 # 注册组件
@@ -96,6 +122,9 @@ def register_app() -> FastAPI:
register_page(app) register_page(app)
register_exception(app) register_exception(app)
# 注册插件钩子
register_plugin_hooks(app)
if settings.GRAFANA_METRICS_ENABLE: if settings.GRAFANA_METRICS_ENABLE:
register_metrics(app) register_metrics(app)
@@ -230,6 +259,7 @@ def register_metrics(app: FastAPI) -> None:
:return: :return:
""" """
metrics_app = make_asgi_app() metrics_app = make_asgi_app()
app.mount('/metrics', metrics_app) app.mount(settings.GRAFANA_METRICS_PATH, metrics_app)
init_otel(app) init_otel(app)
init_plugin_otel_hooks(app)
+84 -22
View File
@@ -1,25 +1,28 @@
import sys import sys
from collections.abc import AsyncGenerator from collections.abc import AsyncGenerator, Mapping
from typing import Annotated, Any from functools import partial
from typing import Annotated, Any, TypeAlias
from uuid import uuid4 from uuid import uuid4
from fastapi import Depends from fastapi import Depends
from sqlalchemy import URL from sqlalchemy import URL, Engine, event
from sqlalchemy.ext.asyncio import ( from sqlalchemy.ext.asyncio import (
AsyncEngine, AsyncEngine,
AsyncSession, AsyncSession,
async_sessionmaker, async_sessionmaker,
create_async_engine, create_async_engine,
) )
from sqlalchemy.orm import Session
from backend.common.enums import DataBaseType from backend.common.enums import DataBaseType
from backend.common.log import log from backend.common.log import log
from backend.common.model import MappedBase from backend.common.model import MappedBase
from backend.common.observability.prometheus.sqlalchemy import observe_sqlalchemy_pool_connections
from backend.core.conf import settings from backend.core.conf import settings
def create_database_url(*, unittest: bool = False, with_database: bool = True) -> URL: def get_database_url(*, unittest: bool = False, with_database: bool = True) -> URL:
""" """
创建数据库链接 创建数据库链接
@@ -71,30 +74,56 @@ def create_database_async_engine(url: str | URL) -> AsyncEngine:
sys.exit() sys.exit()
def create_database_async_session(engine: AsyncEngine) -> async_sessionmaker[AsyncSession | Any]: class DatabaseSession(Session):
""" """数据库数据源会话"""
创建数据库异步会话
:param engine: 数据库异步引擎 def __init__(
:return: self,
""" *,
source: str = 'default',
source_binds: Mapping[str, Engine] | None = None,
**kwargs: Any,
) -> None:
source_binds = source_binds or {}
try:
engine = source_binds[source]
except KeyError as e:
raise ValueError(f'未知数据库数据源: {source}') from e
kwargs['bind'] = engine
kwargs['binds'] = {MappedBase: engine}
super().__init__(**kwargs)
def create_database_async_session(
async_engine: AsyncEngine,
*,
source_binds: Mapping[str, AsyncEngine] | None = None,
) -> async_sessionmaker[AsyncSession | Any]:
"""创建支持命名数据源的数据库异步会话"""
async_binds = dict(source_binds or {})
async_binds.setdefault('default', async_engine)
sync_binds = {source: bind.sync_engine for source, bind in async_binds.items()}
return async_sessionmaker( return async_sessionmaker(
bind=engine, bind=async_engine,
class_=AsyncSession, class_=AsyncSession,
sync_session_class=DatabaseSession,
source='default',
source_binds=sync_binds,
autoflush=False, # 禁用自动刷新 autoflush=False, # 禁用自动刷新
expire_on_commit=False, # 禁用提交时过期 expire_on_commit=False, # 禁用提交时过期
) )
async def get_db() -> AsyncGenerator[AsyncSession, None]: async def get_db() -> AsyncGenerator[AsyncSession, None]:
"""获取数据会话""" """获取默认数据会话"""
async with async_db_session() as session: async with async_db_session(source='default') as session:
yield session yield session
async def get_db_transaction() -> AsyncGenerator[AsyncSession, None]: async def get_db_transaction() -> AsyncGenerator[AsyncSession, None]:
"""获取带有事务的数据库会话""" """获取默认数据源事务会话"""
async with async_db_session.begin() as session: async with async_db_session(source='default').begin() as session:
yield session yield session
@@ -115,13 +144,46 @@ def uuid4_str() -> str:
return str(uuid4()) return str(uuid4())
# SQLA 数据库链接 # SQLA 异步引擎和会话
SQLALCHEMY_DATABASE_URL = create_database_url() async_engine = create_database_async_engine(get_database_url())
_database_engines: dict[str, AsyncEngine] = {'default': async_engine}
for source, url in settings.DATABASE_SOURCES.items():
if not source or source == 'default':
raise ValueError('DATABASE_SOURCES 数据源名称不能为空且不能为 default')
_database_engines[source] = create_database_async_engine(url)
# SALA 异步引擎和会话 async_db_session = create_database_async_session(async_engine, source_binds=_database_engines)
async_engine = create_database_async_engine(SQLALCHEMY_DATABASE_URL)
async_db_session = create_database_async_session(async_engine)
def get_database_engines() -> Mapping[str, AsyncEngine]:
"""获取所有数据库引擎"""
return _database_engines
async def dispose_database() -> None:
"""释放所有数据库连接池"""
for engine in _database_engines.values():
await engine.dispose()
# SQLA 连接池指标监听
for source, engine in _database_engines.items():
event.listen(
engine.sync_engine.pool,
'connect',
partial(observe_sqlalchemy_pool_connections, pool=engine.sync_engine.pool, source=source),
)
event.listen(
engine.sync_engine.pool,
'checkout',
partial(observe_sqlalchemy_pool_connections, pool=engine.sync_engine.pool, source=source),
)
event.listen(
engine.sync_engine.pool,
'checkin',
partial(observe_sqlalchemy_pool_connections, pool=engine.sync_engine.pool, source=source),
)
# Session Annotated # Session Annotated
CurrentSession = Annotated[AsyncSession, Depends(get_db)] CurrentSession: TypeAlias = Annotated[AsyncSession, Depends(get_db)]
CurrentSessionTransaction = Annotated[AsyncSession, Depends(get_db_transaction)] CurrentSessionTransaction: TypeAlias = Annotated[AsyncSession, Depends(get_db_transaction)]
+23 -9
View File
@@ -16,7 +16,7 @@ class RedisCli(Redis):
port: int = settings.REDIS_PORT, port: int = settings.REDIS_PORT,
password: str = settings.REDIS_PASSWORD, password: str = settings.REDIS_PASSWORD,
db: int = settings.REDIS_DATABASE, db: int = settings.REDIS_DATABASE,
socket_timeout: int = settings.REDIS_TIMEOUT, socket_timeout: int | None = settings.REDIS_TIMEOUT,
socket_connect_timeout: int = settings.REDIS_TIMEOUT, socket_connect_timeout: int = settings.REDIS_TIMEOUT,
*, *,
socket_keepalive: bool = True, socket_keepalive: bool = True,
@@ -62,19 +62,33 @@ class RedisCli(Redis):
log.error('Redis 服务器连接异常 {}', e) log.error('Redis 服务器连接异常 {}', e)
sys.exit() sys.exit()
async def delete_prefix(self, prefix: str, exclude: str | list[str] | None = None, batch_size: int = 1000) -> None: async def delete_by_prefix(
self,
key_prefix: str,
exclude_keys: str | list[str] | None = None,
batch_size: int = 1000,
) -> None:
""" """
删除指定前缀的所有 key 删除指定前缀的所有 key
:param prefix: 要删除的键前缀 :param key_prefix: 要删除的键前缀
:param exclude: 要排除的键或键列表 :param exclude_keys: 要排除的键或键列表
:param batch_size: 批量删除的大小避免一次性删除过多键导致 Redis 阻塞 :param batch_size: 批量删除的大小避免一次性删除过多键导致 Redis 阻塞
:return: :return:
""" """
exclude_set = set(exclude) if isinstance(exclude, list) else {exclude} if isinstance(exclude, str) else set() exclude_set = (
set(exclude_keys)
if isinstance(exclude_keys, list)
else {exclude_keys}
if isinstance(exclude_keys, str)
else set()
)
batch_keys = [] batch_keys = []
async for key in self.scan_iter(match=f'{prefix}*'): if key_prefix not in exclude_set and await self.exists(key_prefix):
batch_keys.append(key_prefix)
async for key in self.scan_iter(match=f'{key_prefix}:*'):
if key not in exclude_set: if key not in exclude_set:
batch_keys.append(key) batch_keys.append(key)
@@ -85,15 +99,15 @@ class RedisCli(Redis):
if batch_keys: if batch_keys:
await self.delete(*batch_keys) await self.delete(*batch_keys)
async def get_prefix(self, prefix: str, count: int = 100) -> list[str]: async def get_by_prefix(self, key_prefix: str, count: int = 100) -> list[str]:
""" """
获取指定前缀的所有 key 获取指定前缀的所有 key
:param prefix: 要搜索的键前缀 :param key_prefix: 要搜索的键前缀
:param count: 每次扫描批次的数量值越大扫描速度越快但会占用更多服务器资源 :param count: 每次扫描批次的数量值越大扫描速度越快但会占用更多服务器资源
:return: :return:
""" """
return [key async for key in self.scan_iter(match=f'{prefix}*', count=count)] return [key async for key in self.scan_iter(match=f'{key_prefix}:*', count=count)]
# 创建 redis 客户端单例 # 创建 redis 客户端单例
+29 -19
View File
@@ -7,30 +7,40 @@ from backend.plugin.requirements import install_requirements
from backend.utils.console import console from backend.utils.console import console
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
_log_prefix = f'{timezone.to_str(timezone.now(), "%Y-%m-%d %H:%M:%S.%M0")} | {"INFO": <8} | - | '
console.print(Text(f'{_log_prefix}检查必需插件...', style='bold cyan')) def _get_log_prefix() -> str:
"""获取启动日志前缀"""
return f'{timezone.to_str(timezone.now(), "%Y-%m-%d %H:%M:%S.%M0")} | {"INFO": <8} | - | '
check_required_plugins()
console.print(Text(f'{_log_prefix}检测插件依赖...', style='bold cyan')) def _prepare_plugins() -> None:
"""检查必需插件并安装缺失依赖"""
log_prefix = _get_log_prefix()
_plugins = get_plugins() console.print(Text(f'{log_prefix}检查必需插件...', style='bold cyan'))
with Progress( check_required_plugins()
SpinnerColumn(finished_text=f'[bold green]{_log_prefix}插件准备就绪[/]'),
TextColumn('{task.description}'),
TextColumn('{task.completed}/{task.total}', style='bold green'),
TimeElapsedColumn(),
console=console,
) as progress:
task = progress.add_task('安装插件依赖...', total=len(_plugins))
for plugin in _plugins:
progress.update(task, description=f'[bold magenta]安装插件 {plugin} 依赖...[/]')
install_requirements(plugin)
progress.advance(task)
progress.update(task, description='[bold green]-[/]')
console.print(Text(f'{_log_prefix}启动服务...', style='bold magenta')) console.print(Text(f'{log_prefix}检测插件依赖...', style='bold cyan'))
plugins = get_plugins()
with Progress(
SpinnerColumn(finished_text=f'[bold green]{log_prefix}插件准备就绪[/]'),
TextColumn('{task.description}'),
TextColumn('{task.completed}/{task.total}', style='bold green'),
TimeElapsedColumn(),
console=console,
) as progress:
task = progress.add_task('安装插件依赖...', total=len(plugins))
for plugin in plugins:
progress.update(task, description=f'[bold magenta]安装插件 {plugin} 依赖...[/]')
install_requirements(plugin)
progress.advance(task)
progress.update(task, description='[bold green]-[/]')
console.print(Text(f'{log_prefix}启动服务...', style='bold magenta'))
_prepare_plugins()
app = register_app() app = register_app()
+63 -9
View File
@@ -5,19 +5,24 @@ from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoin
from backend.common.context import ctx from backend.common.context import ctx
from backend.common.log import log from backend.common.log import log
from backend.common.observability.prometheus import ( from backend.common.observability.prometheus.fastapi import (
PROMETHEUS_APP_NAME, dec_fastapi_request_in_progress,
PROMETHEUS_REQUEST_COUNTER, inc_fastapi_exception,
PROMETHEUS_REQUEST_IN_PROGRESS_GAUGE, inc_fastapi_request,
inc_fastapi_request_in_progress,
inc_fastapi_response,
observe_fastapi_request_cost_time,
) )
from backend.common.response.response_code import StandardResponseCode
from backend.core.conf import settings from backend.core.conf import settings
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
from backend.utils.trace_id import get_request_trace_id
class AccessMiddleware(BaseHTTPMiddleware): class AccessMiddleware(BaseHTTPMiddleware):
"""访问日志中间件""" """访问日志中间件"""
async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response: async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response: # ruff:ignore[complex-structure]
""" """
处理请求并记录访问日志 处理请求并记录访问日志
@@ -37,10 +42,59 @@ class AccessMiddleware(BaseHTTPMiddleware):
if method != 'OPTIONS': if method != 'OPTIONS':
log.debug(f'--> 请求开始[{path if not request.url.query else request.url.path + "?" + request.url.query}]') log.debug(f'--> 请求开始[{path if not request.url.query else request.url.path + "?" + request.url.query}]')
if path.startswith(settings.FASTAPI_API_V1_PATH): should_record_metrics = settings.GRAFANA_METRICS_ENABLE and path.startswith(settings.FASTAPI_API_V1_PATH)
PROMETHEUS_REQUEST_IN_PROGRESS_GAUGE.labels(app_name=PROMETHEUS_APP_NAME, method=method, path=path).inc() if should_record_metrics:
PROMETHEUS_REQUEST_COUNTER.labels(app_name=PROMETHEUS_APP_NAME, method=method, path=path).inc() inc_fastapi_request_in_progress(method=method, path=path)
inc_fastapi_request(method=method, path=path)
response = await call_next(request) # 为每个请求上下文注入默认租户 ID,授权接口认证成功后会覆盖为真实值
ctx.tenant_id = settings.TENANT_DEFAULT_ID
try:
response = await call_next(request)
except Exception as e:
elapsed = round((time.perf_counter() - perf_time) * 1000, 3)
if should_record_metrics:
inc_fastapi_exception(method=method, path=path, exception_type=type(e).__name__)
observe_fastapi_request_cost_time(
method=method, path=path, elapsed=elapsed, trace_id=get_request_trace_id()
)
inc_fastapi_response(
method=method,
path=path,
status_code=getattr(e, 'code', StandardResponseCode.HTTP_500),
)
raise
else:
elapsed = round((time.perf_counter() - perf_time) * 1000, 3)
if should_record_metrics:
exception_type = None
exception_code = None
for exception_key, current_exception_type in {
'__request_authentication_exception__': 'AuthenticationError',
'__request_http_exception__': 'HTTPException',
'__request_validation_exception__': 'RequestValidationError',
'__request_assertion_error__': 'AssertionError',
'__request_custom_exception__': 'BaseExceptionError',
'__request_unknown_exception__': 'Exception',
}.items():
exception = ctx.get(exception_key)
if exception:
exception_type = current_exception_type
exception_code = exception.get('code')
break
if exception_type is not None:
inc_fastapi_exception(method=method, path=path, exception_type=exception_type)
observe_fastapi_request_cost_time(
method=method, path=path, elapsed=elapsed, trace_id=get_request_trace_id()
)
inc_fastapi_response(
method=method,
path=path,
status_code=exception_code or response.status_code,
)
finally:
if should_record_metrics:
dec_fastapi_request_in_progress(method=method, path=path)
return response return response
+8 -2
View File
@@ -7,6 +7,7 @@ from starlette.authentication import AuthenticationError as StarletteAuthenticat
from starlette.requests import HTTPConnection from starlette.requests import HTTPConnection
from backend.app.admin.schema.user import GetUserInfoWithRelationDetail from backend.app.admin.schema.user import GetUserInfoWithRelationDetail
from backend.common.context import ctx
from backend.common.exception.errors import TokenError from backend.common.exception.errors import TokenError
from backend.common.log import log from backend.common.log import log
from backend.common.security.jwt import jwt_authentication from backend.common.security.jwt import jwt_authentication
@@ -49,7 +50,9 @@ class JwtAuthMiddleware(AuthenticationBackend):
:param exc: 认证错误对象 :param exc: 认证错误对象
:return: :return:
""" """
return MsgSpecJSONResponse(content={'code': exc.code, 'msg': exc.msg, 'data': None}, status_code=exc.code) content = {'code': exc.code, 'msg': exc.msg, 'data': None}
ctx.__request_authentication_exception__ = content
return MsgSpecJSONResponse(content=content, status_code=exc.code)
@staticmethod @staticmethod
def extract_token(request: Request) -> str | None: def extract_token(request: Request) -> str | None:
@@ -90,7 +93,10 @@ class JwtAuthMiddleware(AuthenticationBackend):
try: try:
user = await jwt_authentication(token) user = await jwt_authentication(token)
except TokenError as exc: except TokenError as exc:
raise AuthenticationError(code=exc.code, msg=exc.detail, headers=exc.headers) if settings.TOKEN_REQUEST_UNDERLYING_SECURITY:
raise AuthenticationError(code=exc.code, msg=exc.detail, headers=exc.headers)
ctx.__request_jwt_authentication_exception__ = exc
return None
except Exception as e: except Exception as e:
log.exception(f'JWT 授权异常:{e}') log.exception(f'JWT 授权异常:{e}')
raise AuthenticationError(code=getattr(e, 'code', 500), msg=getattr(e, 'msg', 'Internal Server Error')) raise AuthenticationError(code=getattr(e, 'code', 500), msg=getattr(e, 'msg', 'Internal Server Error'))
+123 -110
View File
@@ -2,26 +2,22 @@ import json
import time import time
from asyncio import Queue from asyncio import Queue
from collections import defaultdict
from typing import Any from typing import Any
from fastapi import Response from fastapi import Response
from starlette.datastructures import UploadFile from starlette.datastructures import UploadFile
from starlette.middleware.base import BaseHTTPMiddleware from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request from starlette.requests import Request
from starlette_context import request_cycle_context
from backend.app.admin.schema.opera_log import CreateOperaLogParam from backend.app.admin.schema.opera_log import CreateOperaLogParam
from backend.app.admin.service.opera_log_service import opera_log_service from backend.app.admin.service.opera_log_service import opera_log_service
from backend.common.context import ctx from backend.common.context import ctx
from backend.common.enums import StatusType from backend.common.enums import StatusType
from backend.common.log import log from backend.common.log import log
from backend.common.observability.prometheus import ( from backend.common.observability.prometheus.queue import observe_queue_size
PROMETHEUS_APP_NAME, from backend.common.queue import batch_consume
PROMETHEUS_EXCEPTION_COUNTER,
PROMETHEUS_REQUEST_COST_TIME_HISTOGRAM,
PROMETHEUS_REQUEST_IN_PROGRESS_GAUGE,
PROMETHEUS_RESPONSE_COUNTER,
)
from backend.common.queue import batch_dequeue
from backend.common.response.response_code import StandardResponseCode from backend.common.response.response_code import StandardResponseCode
from backend.core.conf import settings from backend.core.conf import settings
from backend.database.db import async_db_session from backend.database.db import async_db_session
@@ -31,9 +27,10 @@ from backend.utils.trace_id import get_request_trace_id
class OperaLogMiddleware(BaseHTTPMiddleware): class OperaLogMiddleware(BaseHTTPMiddleware):
"""操作日志中间件""" """操作日志中间件"""
opera_log_queue: Queue = Queue(maxsize=settings.OPERA_LOG_QUEUE_MAXSIZE) opera_log_queue_name = 'opera_log_queue'
opera_log_queue: Queue[tuple[dict[str, Any], CreateOperaLogParam]] = Queue(maxsize=settings.OPERA_LOG_QUEUE_MAXSIZE)
async def dispatch(self, request: Request, call_next: Any) -> Response: # noqa: C901 async def dispatch(self, request: Request, call_next: Any) -> Response: # ruff:ignore[complex-structure]
""" """
处理请求并记录操作日志 处理请求并记录操作日志
@@ -67,14 +64,6 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
msg = getattr(e, 'msg', str(e)) msg = getattr(e, 'msg', str(e))
status = StatusType.disable status = StatusType.disable
if path.startswith(settings.FASTAPI_API_V1_PATH):
PROMETHEUS_EXCEPTION_COUNTER.labels(
app_name=PROMETHEUS_APP_NAME,
method=method,
path=path,
exception_type=type(e).__name__,
).inc()
raise raise
else: else:
elapsed = round((time.perf_counter() - ctx.perf_time) * 1000, 3) elapsed = round((time.perf_counter() - ctx.perf_time) * 1000, 3)
@@ -86,6 +75,7 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
'__request_validation_exception__', '__request_validation_exception__',
'__request_assertion_error__', '__request_assertion_error__',
'__request_custom_exception__', '__request_custom_exception__',
'__request_unknown_exception__',
]: ]:
exception = ctx.get(exception_key) exception = ctx.get(exception_key)
if exception: if exception:
@@ -94,11 +84,6 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
status = StatusType.disable status = StatusType.disable
log.error(f'请求异常: {msg}') log.error(f'请求异常: {msg}')
break break
if path.startswith(settings.FASTAPI_API_V1_PATH):
PROMETHEUS_REQUEST_COST_TIME_HISTOGRAM.labels(
app_name=PROMETHEUS_APP_NAME, method=method, path=path
).observe(amount=elapsed, exemplar={'TraceID': get_request_trace_id()})
finally: finally:
# summary 只能在请求后获取 # summary 只能在请求后获取
route = request.scope.get('route') route = request.scope.get('route')
@@ -115,40 +100,41 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
log.info(f'{ctx.ip: <15} | {method: <8} | {code!s: <6} | {path} | {elapsed:.3f}ms') log.info(f'{ctx.ip: <15} | {method: <8} | {code!s: <6} | {path} | {elapsed:.3f}ms')
if should_log_opera and request.method != 'OPTIONS': if should_log_opera and request.method != 'OPTIONS':
opera_log_in = CreateOperaLogParam( opera_log_data = {
trace_id=get_request_trace_id(), 'trace_id': get_request_trace_id(),
username=username, 'username': username,
method=method, 'method': method,
title=summary, 'title': summary,
path=path, 'path': path,
ip=ctx.ip, 'ip': ctx.ip,
country=ctx.country, 'country': ctx.country,
region=ctx.region, 'region': ctx.region,
city=ctx.city, 'city': ctx.city,
user_agent=ctx.user_agent, 'user_agent': ctx.user_agent,
os=ctx.os, 'os': ctx.os,
browser=ctx.browser, 'browser': ctx.browser,
device=ctx.device, 'device': ctx.device,
args=args, 'args': args,
status=status, 'status': status,
code=str(code), 'code': str(code),
msg=msg, 'msg': msg,
cost_time=elapsed, 'cost_time': elapsed,
opera_time=ctx.start_time, 'opera_time': ctx.start_time,
) }
await self.opera_log_queue.put(opera_log_in) if settings.TENANT_ENABLED:
tenant_id = ctx.get('tenant_id')
if tenant_id is None:
raise RuntimeError('opera log context is missing tenant_id')
opera_log_data['tenant_id'] = tenant_id
if path.startswith(settings.FASTAPI_API_V1_PATH): opera_log_in = CreateOperaLogParam(**opera_log_data)
PROMETHEUS_RESPONSE_COUNTER.labels( await self.opera_log_queue.put((ctx.copy(), opera_log_in))
app_name=PROMETHEUS_APP_NAME, method=method, path=path, status_code=code if settings.GRAFANA_METRICS_ENABLE:
).inc() observe_queue_size(self.opera_log_queue, queue_name=self.opera_log_queue_name)
PROMETHEUS_REQUEST_IN_PROGRESS_GAUGE.labels(
app_name=PROMETHEUS_APP_NAME, method=method, path=path
).dec()
return response return response
async def get_request_args(self, request: Request) -> dict[str, Any] | None: # noqa: C901 async def get_request_args(self, request: Request) -> dict[str, Any] | None: # ruff:ignore[complex-structure]
""" """
获取请求参数 获取请求参数
@@ -169,13 +155,23 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
# Tip: .body() 必须在 .form() 之前获取 # Tip: .body() 必须在 .form() 之前获取
# https://github.com/encode/starlette/discussions/1933 # https://github.com/encode/starlette/discussions/1933
content_type = request.headers.get('Content-Type', '').split(';') content_types = [item.strip().lower() for item in request.headers.get('Content-Type', '').split(';')]
is_multipart = 'multipart/form-data' in content_types
is_form = is_multipart or 'application/x-www-form-urlencoded' in content_types
content_length = self.get_content_length(request)
if content_length is not None and content_length > settings.OPERA_LOG_BODY_MAX_SIZE:
args['body'] = self.build_truncated_body(content_length, settings.OPERA_LOG_BODY_MAX_SIZE)
return args or None
if is_multipart and content_length is None:
args['body'] = self.build_truncated_body(None, settings.OPERA_LOG_BODY_MAX_SIZE)
return args or None
# 请求体 # 请求体
body_data = await request.body() body_data = await request.body()
if body_data: if body_data and not is_form:
# 注意:非 json 数据默认使用 data 作为键 # 注意:非 json 数据默认使用 data 作为键
if 'application/json' not in content_type: if 'application/json' not in content_types:
args['data'] = body_data.decode('utf-8', 'ignore') if isinstance(body_data, bytes) else str(body_data) args['data'] = body_data.decode('utf-8', 'ignore') if isinstance(body_data, bytes) else str(body_data)
else: else:
json_data = await request.json() json_data = await request.json()
@@ -184,56 +180,64 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
else: else:
args['data'] = str(json_data) args['data'] = str(json_data)
# 表单参数 if is_form:
form_data = await request.form() # 表单参数
if len(form_data) > 0: form_data = await request.form()
serialized_form = {} if len(form_data) > 0:
for k, v in form_data.items(): serialized_form = {}
if isinstance(v, UploadFile): for k, v in form_data.items():
serialized_form[k] = { if isinstance(v, UploadFile):
'filename': v.filename, serialized_form[k] = {
'content_type': v.content_type, 'filename': v.filename,
'size': v.size, 'content_type': v.content_type,
} 'size': v.size,
}
else:
serialized_form[k] = v
if not is_multipart:
args['x-www-form-urlencoded'] = self.desensitization(serialized_form)
else: else:
serialized_form[k] = v args['form-data'] = self.desensitization(serialized_form)
if 'multipart/form-data' not in content_type:
args['x-www-form-urlencoded'] = self.desensitization(serialized_form)
else:
args['form-data'] = self.desensitization(serialized_form)
if args: if args:
args = self.truncate(args) try:
args_str = json.dumps(args, ensure_ascii=False)
args_size = len(args_str.encode('utf-8'))
if args_size > settings.OPERA_LOG_BODY_MAX_SIZE:
args = self.build_truncated_body(args_size, settings.OPERA_LOG_BODY_MAX_SIZE)
except Exception as e:
log.error(f'请求参数截断处理失败:{e}')
return args or None return args or None
@staticmethod @staticmethod
def truncate(args: dict[str, Any]) -> dict[str, Any]: def get_content_length(request: Request) -> int | None:
""" """
截断处理 获取请求体大小
:param args: 需要截断的请求参数字典 :param request: FastAPI 请求对象
:return: :return:
""" """
max_size = 10240 # 数据最大大小(字节) content_length = request.headers.get('Content-Length')
if not content_length:
return None
return int(content_length)
try: @staticmethod
args_str = json.dumps(args, ensure_ascii=False) def build_truncated_body(original_size: int | None, max_size: int) -> dict[str, Any]:
args_size = len(args_str.encode('utf-8')) """
构建请求体截断信息
if args_size > max_size: :param original_size: 原始请求体大小
truncated_str = args_str[:max_size] :param max_size: 最大允许记录大小
return { :return:
'_truncated': True, """
'_original_size': args_size, return {
'_max_size': max_size, '_truncated': True,
'_message': f'数据过大已截断:原始大小 {args_size} 字节,限制 {max_size} 字节', '_original_size': original_size,
'data_preview': truncated_str, '_max_size': max_size,
} '_message': '请求体过大或大小未知,已跳过操作日志请求体记录',
except Exception as e: }
log.error(f'请求参数截断处理失败:{e}')
return args
@staticmethod @staticmethod
def desensitization(args: dict[str, Any]) -> dict[str, Any]: def desensitization(args: dict[str, Any]) -> dict[str, Any]:
@@ -251,20 +255,29 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
@classmethod @classmethod
async def consumer(cls) -> None: async def consumer(cls) -> None:
"""操作日志消费者""" """操作日志消费者"""
while True:
logs = await batch_dequeue( async def bulk_create_opera_log(logs: list[tuple[dict[str, Any], CreateOperaLogParam]]) -> None:
cls.opera_log_queue, """批量创建操作日志"""
max_items=settings.OPERA_LOG_QUEUE_BATCH_CONSUME_SIZE, if settings.DATABASE_ECHO:
timeout=settings.OPERA_LOG_QUEUE_TIMEOUT, log.info('自动执行【操作日志批量创建】任务...')
) logs_by_tenant = defaultdict(list)
if logs: for context_data, log_in in logs:
try: tenant_id = context_data.get('tenant_id')
if settings.DATABASE_ECHO: if tenant_id is None:
log.info('自动执行【操作日志批量创建】任务...') raise RuntimeError('opera log context is missing tenant_id')
async with async_db_session.begin() as db: logs_by_tenant[tenant_id].append((context_data, log_in))
await opera_log_service.bulk_create(db=db, objs=logs) async with async_db_session.begin() as db:
except Exception as e: for tenant_logs in logs_by_tenant.values():
log.error(f'操作日志入库失败,丢失 {len(logs)} 条日志: {e}') request_context = dict(tenant_logs[0][0])
finally: with request_cycle_context(request_context):
for _ in range(len(logs)): await opera_log_service.bulk_create(db=db, objs=[log_in for _, log_in in tenant_logs])
cls.opera_log_queue.task_done()
await batch_consume(
cls.opera_log_queue,
max_items=settings.OPERA_LOG_QUEUE_BATCH_CONSUME_SIZE,
timeout=settings.OPERA_LOG_QUEUE_TIMEOUT,
handler=bulk_create_opera_log,
queue_name=cls.opera_log_queue_name,
error_message='操作日志入库失败',
item_name='日志',
)
+24 -31
View File
@@ -2,13 +2,22 @@
代码生成器插件,生成通用业务代码 代码生成器插件,生成通用业务代码
> [!TIP] - 支持维护代码生成业务配置与模型列信息
> 当前版本仅包含后端代码生成 - 支持手动模式和自动导表模式生成通用业务代码
- 支持预览、写入磁盘和下载生成结果
> [!WARNING] ## 插件类型
> 由于 jinja2 在渲染模版时,文本方式输出可能存在格式问题,所以 `preview` 接口可能无法直观预览代码,这是为前端进行的预设
## 全局配置 - 应用级插件
## 配置说明
插件目录下 `plugin.toml``[settings]` 中包含以下内容:
```toml
[settings]
CODE_GENERATOR_DOWNLOAD_ZIP_FILENAME = 'fba_generator'
```
`backend/core/conf.py` 中添加以下内容: `backend/core/conf.py` 中添加以下内容:
@@ -16,38 +25,22 @@
################################################## ##################################################
# [ Plugin ] code_generator # [ Plugin ] code_generator
################################################## ##################################################
# 基础配置(in plugin.toml
CODE_GENERATOR_DOWNLOAD_ZIP_FILENAME: str CODE_GENERATOR_DOWNLOAD_ZIP_FILENAME: str
``` ```
## 简介
代码生成器使用 API 调用实现,包含两个模组,设计可能存在缺陷,相关问题请直接提交 issues
### 代码生成业务
包含代码生成的相关配置,详情查看:`code_generator/model/gen_business.py`
### 代码生成模型列
包含代码生成所需要的模型列信息,就像正常定义模型列一样,目前支持的功能有限
## 使用方式 ## 使用方式
1. 启动后端服务,打开 swagger 文档直接操作 1. 安装并启用插件后,重启后端服务
2. 通过第三方 api 调试工具发送接口请求 2. 维护业务配置和模型列信息
3. 同时启动前后端,从页面进行操作 3. 执行预览、生成和下载流程
4. 生成代码会直接写入磁盘,仅必须在开发环境使用
接口参数基本都有说明,请注意查看 ## 卸载说明
### 手动模式 - 卸载插件后,建议同步移除相关插件基础配置和 `backend/core/conf.py` 中的插件配置
- 如项目中已接入代码生成相关页面或自动化流程,请同步清理对应集成
1. 通过创建业务接口手动添加一项业务数据 ## 联系方式
2. 通过模型创建接口手动添加模型列
3. 访问 `preview`(预览),`generate`(磁盘写入),`download`(下载)接口,执行后端代码生成相应工作
### 自动模式 - 作者:`wu-clan`
- 反馈方式:提交 Issue 或 PR
1. 访问 `tables` 接口,获取数据库表名列表
2. 通过 `import` 接口,导入数据库已有的数据库表数据,将自动创建业务表数据和模型表数据
3. 访问 `preview`(预览),`generate`(磁盘写入),`download`(下载)接口,执行后端代码生成相应工作
+1 -1
View File
@@ -68,7 +68,7 @@ async def generate_code(db: CurrentSession, pk: Annotated[int, Path(description=
@router.get('/{pk}', summary='下载代码', dependencies=[DependsJwtAuth]) @router.get('/{pk}', summary='下载代码', dependencies=[DependsJwtAuth])
async def download_code(db: CurrentSession, pk: Annotated[int, Path(description='业务 ID')]): # noqa: ANN201 async def download_code(db: CurrentSession, pk: Annotated[int, Path(description='业务 ID')]): # ruff:ignore[missing-return-type-undocumented-public-function]
bio = await gen_service.download(db=db, pk=pk) bio = await gen_service.download(db=db, pk=pk)
return StreamingResponse( return StreamingResponse(
bio, bio,
@@ -6,6 +6,7 @@ from sqlalchemy_crud_plus import CRUDPlus
from backend.plugin.code_generator.model import GenBusiness from backend.plugin.code_generator.model import GenBusiness
from backend.plugin.code_generator.schema.business import CreateGenBusinessParam, UpdateGenBusinessParam from backend.plugin.code_generator.schema.business import CreateGenBusinessParam, UpdateGenBusinessParam
from backend.utils.timezone import timezone
class CRUDGenBusiness(CRUDPlus[GenBusiness]): class CRUDGenBusiness(CRUDPlus[GenBusiness]):
@@ -19,7 +20,7 @@ class CRUDGenBusiness(CRUDPlus[GenBusiness]):
:param pk: 代码生成业务 ID :param pk: 代码生成业务 ID
:return: :return:
""" """
return await self.select_model(db, pk) return await self.select_model(db, pk, deleted=0)
async def get_by_name(self, db: AsyncSession, name: str) -> GenBusiness | None: async def get_by_name(self, db: AsyncSession, name: str) -> GenBusiness | None:
""" """
@@ -29,7 +30,7 @@ class CRUDGenBusiness(CRUDPlus[GenBusiness]):
:param name: 表名 :param name: 表名
:return: :return:
""" """
return await self.select_model_by_column(db, table_name=name) return await self.select_model_by_column(db, table_name=name, deleted=0)
async def get_all(self, db: AsyncSession) -> Sequence[GenBusiness]: async def get_all(self, db: AsyncSession) -> Sequence[GenBusiness]:
""" """
@@ -38,7 +39,7 @@ class CRUDGenBusiness(CRUDPlus[GenBusiness]):
:param db: 数据库会话 :param db: 数据库会话
:return: :return:
""" """
return await self.select_models(db) return await self.select_models(db, deleted=0)
async def get_select(self, table_name: str | None) -> Select: async def get_select(self, table_name: str | None) -> Select:
""" """
@@ -47,7 +48,7 @@ class CRUDGenBusiness(CRUDPlus[GenBusiness]):
:param table_name: 业务表名 :param table_name: 业务表名
:return: :return:
""" """
filters = {} filters = {'deleted': 0}
if table_name is not None: if table_name is not None:
filters['table_name__like'] = f'%{table_name}%' filters['table_name__like'] = f'%{table_name}%'
@@ -73,7 +74,7 @@ class CRUDGenBusiness(CRUDPlus[GenBusiness]):
:param obj: 更新代码生成业务参数 :param obj: 更新代码生成业务参数
:return: :return:
""" """
return await self.update_model(db, pk, obj) return await self.update_model_by_column(db, obj, id=pk, deleted=0)
async def delete(self, db: AsyncSession, pk: int) -> int: async def delete(self, db: AsyncSession, pk: int) -> int:
""" """
@@ -83,7 +84,16 @@ class CRUDGenBusiness(CRUDPlus[GenBusiness]):
:param pk: 代码生成业务 ID :param pk: 代码生成业务 ID
:return: :return:
""" """
return await self.delete_model(db, pk) return await self.delete_model_by_column(
db,
logical_deletion=True,
deleted_flag_column='deleted',
deleted_flag_value=self.model.id,
deleted_at_column='deleted_time',
deleted_at_factory=timezone.now(),
id=pk,
deleted=0,
)
gen_business_dao: CRUDGenBusiness = CRUDGenBusiness(GenBusiness) gen_business_dao: CRUDGenBusiness = CRUDGenBusiness(GenBusiness)
@@ -4,7 +4,11 @@ from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus from sqlalchemy_crud_plus import CRUDPlus
from backend.plugin.code_generator.model import GenColumn from backend.plugin.code_generator.model import GenColumn
from backend.plugin.code_generator.schema.column import CreateGenColumnParam, UpdateGenColumnParam from backend.plugin.code_generator.schema.column import (
CreateGenColumnInternalParam,
CreateGenColumnParam,
UpdateGenColumnParam,
)
class CRUDGenColumn(CRUDPlus[GenColumn]): class CRUDGenColumn(CRUDPlus[GenColumn]):
@@ -41,6 +45,16 @@ class CRUDGenColumn(CRUDPlus[GenColumn]):
""" """
await self.create_model(db, obj, pd_type=pd_type) await self.create_model(db, obj, pd_type=pd_type)
async def bulk_create(self, db: AsyncSession, objs: list[CreateGenColumnInternalParam]) -> None:
"""
批量创建代码生成模型列
:param db: 数据库会话
:param objs: 创建代码生成模型列参数列表
:return:
"""
await self.create_models(db, objs)
async def update(self, db: AsyncSession, pk: int, obj: UpdateGenColumnParam, pd_type: str | None) -> int: async def update(self, db: AsyncSession, pk: int, obj: UpdateGenColumnParam, pd_type: str | None) -> int:
""" """
更新代码生成模型列 更新代码生成模型列
@@ -126,6 +126,8 @@ class CRUDGen:
column_name <> 'id' column_name <> 'id'
and column_name <> 'created_time' and column_name <> 'created_time'
and column_name <> 'updated_time' and column_name <> 'updated_time'
and column_name <> 'deleted'
and column_name <> 'deleted_time'
and table_name = :table_name and table_name = :table_name
and table_schema = :table_schema and table_schema = :table_schema
order by order by
@@ -180,6 +182,8 @@ class CRUDGen:
and a.attname <> 'id' and a.attname <> 'id'
and a.attname <> 'created_time' and a.attname <> 'created_time'
and a.attname <> 'updated_time' and a.attname <> 'updated_time'
and a.attname <> 'deleted'
and a.attname <> 'deleted_time'
and t.relname = :table_name and t.relname = :table_name
and n.nspname = :table_schema and n.nspname = :table_schema
order by order by
@@ -9,10 +9,14 @@ class GenBusiness(Base):
"""代码生成业务表""" """代码生成业务表"""
__tablename__ = 'gen_business' __tablename__ = 'gen_business'
__table_args__ = (
sa.UniqueConstraint('table_name', 'deleted', name='uk_gen_business_table_name_deleted'),
{'comment': '代码生成业务表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
app_name: Mapped[str] = mapped_column(sa.String(64), comment='应用名称') app_name: Mapped[str] = mapped_column(sa.String(64), comment='应用名称')
table_name: Mapped[str] = mapped_column(sa.String(256), unique=True, comment='表名称') table_name: Mapped[str] = mapped_column(sa.String(256), comment='表名称')
doc_comment: Mapped[str] = mapped_column(sa.String(256), comment='文档注释') doc_comment: Mapped[str] = mapped_column(sa.String(256), comment='文档注释')
table_comment: Mapped[str | None] = mapped_column(sa.String(256), default=None, comment='表描述') table_comment: Mapped[str | None] = mapped_column(sa.String(256), default=None, comment='表描述')
class_name: Mapped[str | None] = mapped_column(sa.String(64), default=None, comment='基础类名') class_name: Mapped[str | None] = mapped_column(sa.String(64), default=None, comment='基础类名')
@@ -9,6 +9,10 @@ class GenColumn(DataClassBase):
"""代码生成模型列表""" """代码生成模型列表"""
__tablename__ = 'gen_column' __tablename__ = 'gen_column'
__table_args__ = (
sa.UniqueConstraint('gen_business_id', 'name', name='uk_gen_column_business_id_name'),
{'comment': '代码生成模型列表'},
)
id: Mapped[id_key] = mapped_column(init=False) id: Mapped[id_key] = mapped_column(init=False)
name: Mapped[str] = mapped_column(sa.String(64), comment='列名称') name: Mapped[str] = mapped_column(sa.String(64), comment='列名称')
@@ -28,6 +28,12 @@ class CreateGenColumnParam(GenColumnSchemaBase):
"""创建代码生成模型列参数""" """创建代码生成模型列参数"""
class CreateGenColumnInternalParam(CreateGenColumnParam):
"""创建代码生成模型列内部参数"""
pd_type: str | None = Field(None, description='列类型对应的 pydantic 类型')
class UpdateGenColumnParam(GenColumnSchemaBase): class UpdateGenColumnParam(GenColumnSchemaBase):
"""更新代码生成模型列参数""" """更新代码生成模型列参数"""
@@ -77,6 +77,11 @@ class GenBusinessService:
:return: :return:
""" """
business = await gen_business_dao.get(db, pk)
if not business:
raise errors.NotFoundError(msg='代码生成业务不存在')
if business.table_name != obj.table_name and await gen_business_dao.get_by_name(db, obj.table_name):
raise errors.ConflictError(msg='代码生成业务已存在')
return await gen_business_dao.update(db, pk, obj) return await gen_business_dao.update(db, pk, obj)
@staticmethod @staticmethod
@@ -89,6 +94,9 @@ class GenBusinessService:
:return: :return:
""" """
business = await gen_business_dao.get(db, pk)
if not business:
raise errors.NotFoundError(msg='代码生成业务不存在')
return await gen_business_dao.delete(db, pk) return await gen_business_dao.delete(db, pk)
@@ -5,6 +5,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from backend.common.enums import DataBaseType from backend.common.enums import DataBaseType
from backend.common.exception import errors from backend.common.exception import errors
from backend.core.conf import settings from backend.core.conf import settings
from backend.plugin.code_generator.crud.crud_business import gen_business_dao
from backend.plugin.code_generator.crud.crud_column import gen_column_dao from backend.plugin.code_generator.crud.crud_column import gen_column_dao
from backend.plugin.code_generator.enums import GenMySQLColumnType, GenPostgreSQLColumnType from backend.plugin.code_generator.enums import GenMySQLColumnType, GenPostgreSQLColumnType
from backend.plugin.code_generator.model import GenColumn from backend.plugin.code_generator.model import GenColumn
@@ -28,6 +29,8 @@ class GenColumnService:
column = await gen_column_dao.get(db, pk) column = await gen_column_dao.get(db, pk)
if not column: if not column:
raise errors.NotFoundError(msg='代码生成模型列不存在') raise errors.NotFoundError(msg='代码生成模型列不存在')
if not await gen_business_dao.get(db, column.gen_business_id):
raise errors.NotFoundError(msg='代码生成业务不存在')
return column return column
@staticmethod @staticmethod
@@ -50,6 +53,8 @@ class GenColumnService:
:return: :return:
""" """
if not await gen_business_dao.get(db, business_id):
raise errors.NotFoundError(msg='代码生成业务不存在')
return await gen_column_dao.get_all_by_business(db, business_id) return await gen_column_dao.get_all_by_business(db, business_id)
@staticmethod @staticmethod
@@ -62,6 +67,9 @@ class GenColumnService:
:return: :return:
""" """
if not await gen_business_dao.get(db, obj.gen_business_id):
raise errors.NotFoundError(msg='代码生成业务不存在')
gen_columns = await gen_column_dao.get_all_by_business(db, obj.gen_business_id) gen_columns = await gen_column_dao.get_all_by_business(db, obj.gen_business_id)
if obj.name in [gen_column.name for gen_column in gen_columns]: if obj.name in [gen_column.name for gen_column in gen_columns]:
raise errors.ForbiddenError(msg='模型列已存在') raise errors.ForbiddenError(msg='模型列已存在')
@@ -81,6 +89,12 @@ class GenColumnService:
""" """
column = await gen_column_dao.get(db, pk) column = await gen_column_dao.get(db, pk)
if not column:
raise errors.NotFoundError(msg='代码生成模型列不存在')
if not await gen_business_dao.get(db, column.gen_business_id):
raise errors.NotFoundError(msg='代码生成业务不存在')
if not await gen_business_dao.get(db, obj.gen_business_id):
raise errors.NotFoundError(msg='代码生成业务不存在')
if obj.name != column.name: if obj.name != column.name:
gen_columns = await gen_column_dao.get_all_by_business(db, obj.gen_business_id) gen_columns = await gen_column_dao.get_all_by_business(db, obj.gen_business_id)
if obj.name in [gen_column.name for gen_column in gen_columns]: if obj.name in [gen_column.name for gen_column in gen_columns]:
@@ -99,6 +113,11 @@ class GenColumnService:
:return: :return:
""" """
column = await gen_column_dao.get(db, pk)
if not column:
raise errors.NotFoundError(msg='代码生成模型列不存在')
if not await gen_business_dao.get(db, column.gen_business_id):
raise errors.NotFoundError(msg='代码生成业务不存在')
return await gen_column_dao.delete(db, pk) return await gen_column_dao.delete(db, pk)
@@ -22,7 +22,7 @@ from backend.plugin.code_generator.crud.crud_column import gen_column_dao
from backend.plugin.code_generator.crud.crud_gen import gen_dao from backend.plugin.code_generator.crud.crud_gen import gen_dao
from backend.plugin.code_generator.model import GenBusiness from backend.plugin.code_generator.model import GenBusiness
from backend.plugin.code_generator.schema.business import CreateGenBusinessParam from backend.plugin.code_generator.schema.business import CreateGenBusinessParam
from backend.plugin.code_generator.schema.column import CreateGenColumnParam from backend.plugin.code_generator.schema.column import CreateGenColumnInternalParam
from backend.plugin.code_generator.schema.gen import ImportParam from backend.plugin.code_generator.schema.gen import ImportParam
from backend.plugin.code_generator.service.column_service import gen_column_service from backend.plugin.code_generator.service.column_service import gen_column_service
from backend.plugin.code_generator.utils.format_code import format_python_code from backend.plugin.code_generator.utils.format_code import format_python_code
@@ -87,12 +87,12 @@ class GenService:
await db.flush() await db.flush()
column_info = await gen_dao.get_all_columns(db, obj.table_schema, table_name) column_info = await gen_dao.get_all_columns(db, obj.table_schema, table_name)
gen_columns = []
for column in column_info: for column in column_info:
column_type = column['column_type'].split('(')[0].upper() column_type = column['column_type'].split('(')[0].upper()
pd_type = sql_type_to_pydantic(column_type) pd_type = sql_type_to_pydantic(column_type)
await gen_column_dao.create( gen_columns.append(
db, CreateGenColumnInternalParam(
CreateGenColumnParam(
name=column['column_name'], name=column['column_name'],
comment=column['column_comment'], comment=column['column_comment'],
type=column_type, type=column_type,
@@ -103,9 +103,10 @@ class GenService:
is_pk=column['is_pk'], is_pk=column['is_pk'],
is_nullable=column['is_nullable'], is_nullable=column['is_nullable'],
gen_business_id=new_business.id, gen_business_id=new_business.id,
pd_type=pd_type,
), ),
pd_type=pd_type,
) )
await gen_column_dao.bulk_create(db, gen_columns)
@staticmethod @staticmethod
async def _render_tpl_code(*, db: AsyncSession, business: GenBusiness) -> dict[str, str]: async def _render_tpl_code(*, db: AsyncSession, business: GenBusiness) -> dict[str, str]:
@@ -5,7 +5,7 @@ set @codegen_menu_id = LAST_INSERT_ID();
insert into sys_menu (title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time) insert into sys_menu (title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time)
values values
('新增业务', 'AddGenCodeBusiness', '', 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, @codegen_menu_id, now(), null), ('新增业务', 'AddGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, @codegen_menu_id, now(), null),
('修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, @codegen_menu_id, now(), null), ('修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, @codegen_menu_id, now(), null),
('删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, @codegen_menu_id, now(), null), ('删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, @codegen_menu_id, now(), null),
('新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, @codegen_menu_id, now(), null), ('新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, @codegen_menu_id, now(), null),
@@ -3,7 +3,7 @@ values (2049629108257816580, 'code_generator.menu', 'PluginCodeGenerator', '/plu
insert into sys_menu (id, title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time) insert into sys_menu (id, title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time)
values values
(2049629108257816581, '新增业务', 'AddGenCodeBusiness', '', 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816581, '新增业务', 'AddGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, 2049629108257816580, now(), null),
(2049629108257816582, '修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816582, '修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, 2049629108257816580, now(), null),
(2049629108257816583, '删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816583, '删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, 2049629108257816580, now(), null),
(2049629108257816584, '新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816584, '新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, 2049629108257816580, now(), null),
@@ -8,7 +8,7 @@ begin
insert into sys_menu (title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time) insert into sys_menu (title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time)
values values
('新增业务', 'AddGenCodeBusiness', '', 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, codegen_menu_id, now(), null), ('新增业务', 'AddGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, codegen_menu_id, now(), null),
('修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, codegen_menu_id, now(), null), ('修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, codegen_menu_id, now(), null),
('删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, codegen_menu_id, now(), null), ('删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, codegen_menu_id, now(), null),
('新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, codegen_menu_id, now(), null), ('新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, codegen_menu_id, now(), null),
@@ -3,7 +3,7 @@ values (2049629108257816580, 'code_generator.menu', 'PluginCodeGenerator', '/plu
insert into sys_menu (id, title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time) insert into sys_menu (id, title, name, path, sort, icon, type, component, perms, status, display, cache, link, remark, parent_id, created_time, updated_time)
values values
(2049629108257816581, '新增业务', 'AddGenCodeBusiness', '', 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816581, '新增业务', 'AddGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:add', 1, 0, 1, '', null, 2049629108257816580, now(), null),
(2049629108257816582, '修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816582, '修改业务', 'EditGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:edit', 1, 0, 1, '', null, 2049629108257816580, now(), null),
(2049629108257816583, '删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816583, '删除业务', 'DeleteGenCodeBusiness', null, 0, null, 2, null, 'codegen:business:del', 1, 0, 1, '', null, 2049629108257816580, now(), null),
(2049629108257816584, '新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, 2049629108257816580, now(), null), (2049629108257816584, '新增模型', 'AddGenCodeModel', null, 0, null, 2, null, 'codegen:model:add', 1, 0, 1, '', null, 2049629108257816580, now(), null),

Some files were not shown because too many files have changed in this diff Show More