Compare commits

..
60 Commits
Author SHA1 Message Date
Wu Clan 2099c33aad Fix alembic migration file missing import (#803) 2025-09-09 17:21:23 +08:00
yzbf b809497aba Fix the subprocess check call failed in windows (#802)
* fix: 修复Windows下subprocess调用失败的问题

* fix: 修复代码格式问题
2025-09-09 16:21:51 +08:00
Wu Clan a6fe8c0e8d Fix console import in main module (#800) 2025-09-07 19:33:55 +08:00
Wu Clan 629cf7ce45 Optimize the code generation interaction logic (#799) 2025-09-06 19:24:02 +08:00
Wu Clan fad6c05d41 Add the code generator to the CLI (#798)
* Add the code generator to the CLI

* Optimize run_await implementation

* Optimize interactions
2025-09-05 23:43:05 +08:00
Wu Clan 77d59d3453 Add schema for the tree data (#796) 2025-09-04 16:38:53 +08:00
Wu Clan 931d6b281a Fix the psycopg dependency naming (#795) 2025-09-04 11:54:39 +08:00
Wu Clan d929f0dbdc Fix sql script syntax error in config plugin (#793) 2025-09-03 19:57:11 +08:00
Wu Clan 6d1b47903e Add the dict type query all interface (#794) 2025-09-03 19:56:56 +08:00
Wu Clan 8420ccf190 Update the notice model and pagination (#792) 2025-09-02 15:51:07 +08:00
Wu Clan 1e38e1918b Bump dependencies and pre-commits (#791) 2025-09-02 12:45:34 +08:00
IAseven 52e78152a1 Fix the tzinfo comparison in the timezone type (#787) 2025-08-29 10:30:42 +08:00
Wu Clan dcb07c60f2 Add the Google OAuth2 login (#790) 2025-08-28 22:10:22 +08:00
Wu Clan 33c9f78cf0 Update the handling of CORS 500 status code (#789) 2025-08-28 15:36:06 +08:00
Wu Clan 37b1033697 Update the opera log exception message record (#788) 2025-08-28 15:01:01 +08:00
Wu Clan 0aa4ebc635 Update the model datetime column type to custom (#786)
* Update the model datetime column type to custom

* Update the schema datetime filed json encoder
2025-08-27 20:07:35 +08:00
language 30d4107670 Fix CLI openapi url in production environment (#785) 2025-08-26 09:39:30 +08:00
Wu Clan dcf235db78 Fix code generation template params and filename (#784) 2025-08-22 13:20:30 +08:00
Wu Clan 1dfcd7ae3c Update the celery task result table creation logic (#783)
* Update the celery task result table creation logic

* Disable beat_sync_every config

* Update the prepared comment
2025-08-21 23:46:52 +08:00
wuyao4 2b28244099 Fix the plugin model object detection logic (#782) 2025-08-21 18:17:50 +08:00
Wu Clan f71071f905 Fix the celery beat distributed lock timeout (#779)
* Fix the celery beat distributed lock timeout

* Fix redis client not close after plugin parse

* Optimize code layout and lock extend
2025-08-21 13:50:57 +08:00
Wu Clan 34c9c398ca Fix redis client not close after plugin parse (#780) 2025-08-21 10:58:53 +08:00
Wu Clan f099777dd3 Fix the docker compose env configuration file (#778)
* Fix the docker compose env configuration file

* Fix CELERY_RABBITMQ_HOST value
2025-08-19 13:21:11 +08:00
Wu Clan c95be82cb9 Update the env parameter value in env file (#777) 2025-08-19 12:50:08 +08:00
Wu Clan 00ec039095 Add test data SQL script for config plugin (#776) 2025-08-19 12:38:22 +08:00
Wu Clan 4d087355b4 Optimize email sending config update logic (#775) 2025-08-19 12:19:31 +08:00
Wu Clan d57fabcbb4 Add bulk update interface for config plugin (#774)
* Add bulk update interface for config plugin

* Remove email sending update
2025-08-19 12:18:38 +08:00
yanlingsishao 3a32522e2b Optimize the celery crontab expression validation (#773) 2025-08-18 22:19:59 +08:00
Wu Clanandwu-clan 22f8850bf7 Update the changelog for v1.8.0 (#772)
This commit was created by changelog-from-release in 'Release changelog' CI workflow

Co-authored-by: wu-clan <wu-clan@users.noreply.github.com>
2025-08-15 20:09:28 +08:00
Wu Clan eeb709c6aa Update the version number to 1.8.0 (#771) 2025-08-15 20:06:21 +08:00
Wu Clan a6bbf2971d Update the menu title in SQL scripts (#770) 2025-08-15 19:58:46 +08:00
Dylan cd48bb4210 Add i18n support for response message (#753)
* feat: i18n support

* Optimize i18n

* Update the locale in the code

* Update the zh-CN file

* Update the en-US file

* Update the reload filter

* Update locale success plugin value

* Fix lint

* Update pydantic error message translation

* Update to minimal implementation

* Fix minimal missing code
2025-08-15 19:57:10 +08:00
Wu Clan 4500dd0128 Add a standalone email sending plugin (#769) 2025-08-13 18:09:42 +08:00
Wu Clan 2b6d8222ad Update the content layout of the config file (#768) 2025-08-13 18:07:02 +08:00
Wu Clan 1b47ab7e83 Optimize the timezone datetime return encoder (#767)
* Optimize the timezone datetime return encoder

* Update the default datetime conversion
2025-08-13 11:16:58 +08:00
Wu Clan bd804e0a38 Update the description for the run file (#766) 2025-08-12 23:28:43 +08:00
Wu Clan 8e8af2032a Optimize naming and preview in code generation (#764) 2025-08-12 16:42:25 +08:00
IAseven e09062eb39 Optimize the opera log storage logic through queue (#750)
*  feat: 操作日志中间件添加批量插入功能

* Delete GEMINI.md

* 🌈 style: 修复格式化错误

* 🐞 fix: 通过asyncio.wait_for兼容py3.10中asyncio.timeout不存在

* 🦄 refactor: 重新组织操作日志批量插入代码逻辑

* 优化代码实现

* 恢复默认配置

* 恢复默认 .gitignore 文件

* 更新队列批处理逻辑
2025-08-07 17:36:10 +08:00
Wu Clan 8c00492e44 Update the naming of table creation function (#760) 2025-08-07 16:57:57 +08:00
Wu Clan 0237d4c7b1 Update log output config and format (#759) 2025-08-07 16:46:56 +08:00
Wu Clan fe3a3b4e86 Optimize the data sort logic of tree nodes (#758) 2025-08-06 18:23:09 +08:00
Wu Clan 65ec721a1c Add business pagination in the code generator (#757) 2025-08-05 21:04:27 +08:00
Wu Clan 4eb76ad6ea Update the opera log desensitization method (#756)
* Update the opera log desensitization method

* Update request args function return
2025-08-05 18:38:49 +08:00
Wu Clan 1f8687155a Fix message format in validation exception handler (#755) 2025-08-05 18:11:02 +08:00
Wu Clan 8591d4e592 Refactor task routes and add control routes (#749) 2025-08-04 13:18:07 +08:00
Wu Clan dedf4e7bae Refactor code generation files and routes (#748)
* Refactor code generation files and routes

* Fix lint
2025-08-04 13:16:41 +08:00
Wu Clan 1e4aa88487 Fix the kwargs params of schedule task (#747) 2025-08-04 13:15:48 +08:00
Wu Clan 0dd745b7a7 Add schedule task demo that contains params (#746) 2025-07-31 18:29:54 +08:00
Wu Clan 6b2402f212 Add some interfaces for user profiles (#745) 2025-07-31 16:46:02 +08:00
Wu Clan 24a487eeea Simplify the plugin status update logic (#744) 2025-07-30 11:31:37 +08:00
Wu Clan 4f574189c7 Fix the error trigger when model auto import (#743) 2025-07-29 22:52:34 +08:00
Wu Clan b559a74cea Add update support for user email and phone (#742) 2025-07-29 22:52:08 +08:00
Wu Clan 83dcdbe59d Update the OAuth2 login password policy (#741)
* Update the OAuth2 login password policy

* Update the crud pwd

* Update the reset pwd service
2025-07-29 22:51:31 +08:00
Wu Clan d64f7c2911 Fix the opera log field encryption (#739) 2025-07-25 19:23:36 +08:00
Wu Clan 53e64bce37 Add auth whitelist regular expression config (#738) 2025-07-24 21:34:24 +08:00
Wu Clan 00a781357b Fix celery CLI option to required (#737) 2025-07-24 21:32:57 +08:00
Wu Clan d7f87ed0ad Update the default cache period for userinfo (#734) 2025-07-21 21:31:55 +08:00
Wu Clan bda9b1d463 Add distributed lock for scheduled task (#732)
* Add distributed lock for scheduled task

* Add the task to extend lock

* Fix the close
2025-07-21 12:29:12 +08:00
Wu Clan e0a106ec51 Simplify task crontab expression validation (#733) 2025-07-18 21:11:54 +08:00
Wu Clan 016361bd68 Update the changelog for v1.7.0 (#729) 2025-07-16 13:40:55 +08:00
140 changed files with 3252 additions and 1675 deletions
+3 -3
View File
@@ -1,6 +1,6 @@
repos: repos:
- repo: https://github.com/pre-commit/pre-commit-hooks - repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0 rev: v6.0.0
hooks: hooks:
# - id: check-added-large-files # - id: check-added-large-files
# - id: end-of-file-fixer # - id: end-of-file-fixer
@@ -8,7 +8,7 @@ repos:
- id: check-toml - id: check-toml
- repo: https://github.com/charliermarsh/ruff-pre-commit - repo: https://github.com/charliermarsh/ruff-pre-commit
rev: v0.11.9 rev: v0.12.11
hooks: hooks:
- id: ruff - id: ruff
args: args:
@@ -20,7 +20,7 @@ repos:
- id: ruff-format - id: ruff-format
- repo: https://github.com/astral-sh/uv-pre-commit - repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.7.3 rev: 0.8.14
hooks: hooks:
- id: uv-lock - id: uv-lock
- id: uv-export - id: uv-export
+81
View File
@@ -1,3 +1,82 @@
<a id="v1.8.0"></a>
# [v1.8.0](https://github.com/fastapi-practices/fastapi_best_architecture/releases/tag/v1.8.0) - 2025-08-15
## What's Changed
* Update changelog for v1.7.0 by [@wu-clan](https://github.com/wu-clan) in [#729](https://github.com/fastapi-practices/fastapi_best_architecture/pull/729)
* Simplify task crontab expression validation by [@wu-clan](https://github.com/wu-clan) in [#733](https://github.com/fastapi-practices/fastapi_best_architecture/pull/733)
* Add distributed lock for scheduled task by [@wu-clan](https://github.com/wu-clan) in [#732](https://github.com/fastapi-practices/fastapi_best_architecture/pull/732)
* Update the default cache period for userinfo by [@wu-clan](https://github.com/wu-clan) in [#734](https://github.com/fastapi-practices/fastapi_best_architecture/pull/734)
* Fix celery CLI option to required by [@wu-clan](https://github.com/wu-clan) in [#737](https://github.com/fastapi-practices/fastapi_best_architecture/pull/737)
* Add auth whitelist regular expression config by [@wu-clan](https://github.com/wu-clan) in [#738](https://github.com/fastapi-practices/fastapi_best_architecture/pull/738)
* Fix the opera log field encryption by [@wu-clan](https://github.com/wu-clan) in [#739](https://github.com/fastapi-practices/fastapi_best_architecture/pull/739)
* Update the OAuth2 login password policy by [@wu-clan](https://github.com/wu-clan) in [#741](https://github.com/fastapi-practices/fastapi_best_architecture/pull/741)
* Add update support for user email and phone by [@wu-clan](https://github.com/wu-clan) in [#742](https://github.com/fastapi-practices/fastapi_best_architecture/pull/742)
* Fix the error trigger when model auto import by [@wu-clan](https://github.com/wu-clan) in [#743](https://github.com/fastapi-practices/fastapi_best_architecture/pull/743)
* Simplify the plugin status update logic by [@wu-clan](https://github.com/wu-clan) in [#744](https://github.com/fastapi-practices/fastapi_best_architecture/pull/744)
* Add some interfaces for user profiles by [@wu-clan](https://github.com/wu-clan) in [#745](https://github.com/fastapi-practices/fastapi_best_architecture/pull/745)
* Add schedule task demo that contains params by [@wu-clan](https://github.com/wu-clan) in [#746](https://github.com/fastapi-practices/fastapi_best_architecture/pull/746)
* Fix the kwargs params of schedule task by [@wu-clan](https://github.com/wu-clan) in [#747](https://github.com/fastapi-practices/fastapi_best_architecture/pull/747)
* Refactor code generation files and routes by [@wu-clan](https://github.com/wu-clan) in [#748](https://github.com/fastapi-practices/fastapi_best_architecture/pull/748)
* Refactor task routes and add control routes by [@wu-clan](https://github.com/wu-clan) in [#749](https://github.com/fastapi-practices/fastapi_best_architecture/pull/749)
* Fix message format in validation exception handler by [@wu-clan](https://github.com/wu-clan) in [#755](https://github.com/fastapi-practices/fastapi_best_architecture/pull/755)
* Update the opera log desensitization method by [@wu-clan](https://github.com/wu-clan) in [#756](https://github.com/fastapi-practices/fastapi_best_architecture/pull/756)
* Add business pagination in the code generator by [@wu-clan](https://github.com/wu-clan) in [#757](https://github.com/fastapi-practices/fastapi_best_architecture/pull/757)
* Optimize the data sort logic of tree nodes by [@wu-clan](https://github.com/wu-clan) in [#758](https://github.com/fastapi-practices/fastapi_best_architecture/pull/758)
* Update log output config and format by [@wu-clan](https://github.com/wu-clan) in [#759](https://github.com/fastapi-practices/fastapi_best_architecture/pull/759)
* Update the naming of table creation function by [@wu-clan](https://github.com/wu-clan) in [#760](https://github.com/fastapi-practices/fastapi_best_architecture/pull/760)
* Optimize the opera log storage logic through queue by [@IAseven](https://github.com/IAseven) in [#750](https://github.com/fastapi-practices/fastapi_best_architecture/pull/750)
* Optimize naming and preview in code generation by [@wu-clan](https://github.com/wu-clan) in [#764](https://github.com/fastapi-practices/fastapi_best_architecture/pull/764)
* Update the description for the run file by [@wu-clan](https://github.com/wu-clan) in [#766](https://github.com/fastapi-practices/fastapi_best_architecture/pull/766)
* Optimize the timezone datetime return encoder by [@wu-clan](https://github.com/wu-clan) in [#767](https://github.com/fastapi-practices/fastapi_best_architecture/pull/767)
* Update the content layout of the config file by [@wu-clan](https://github.com/wu-clan) in [#768](https://github.com/fastapi-practices/fastapi_best_architecture/pull/768)
* Add a standalone email sending plugin by [@wu-clan](https://github.com/wu-clan) in [#769](https://github.com/fastapi-practices/fastapi_best_architecture/pull/769)
* Add i18n support for response message by [@downdawn](https://github.com/downdawn) in [#753](https://github.com/fastapi-practices/fastapi_best_architecture/pull/753)
* Update the menu title in SQL scripts by [@wu-clan](https://github.com/wu-clan) in [#770](https://github.com/fastapi-practices/fastapi_best_architecture/pull/770)
* Update the version number to 1.8.0 by [@wu-clan](https://github.com/wu-clan) in [#771](https://github.com/fastapi-practices/fastapi_best_architecture/pull/771)
## New Contributors
* [@IAseven](https://github.com/IAseven) made their first contribution in [#750](https://github.com/fastapi-practices/fastapi_best_architecture/pull/750)
**Full Changelog**: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.7.0...v1.8.0
[Changes][v1.8.0]
<a id="v1.7.0"></a>
# [v1.7.0](https://github.com/fastapi-practices/fastapi_best_architecture/releases/tag/v1.7.0) - 2025-07-16
## What's Changed
* Update the changelog for v1.6.0 by [@wu-clan](https://github.com/wu-clan) in [#703](https://github.com/fastapi-practices/fastapi_best_architecture/pull/703)
* Update the CLI to be executed async by [@wu-clan](https://github.com/wu-clan) in [#704](https://github.com/fastapi-practices/fastapi_best_architecture/pull/704)
* Fix the code generation schema template by [@wu-clan](https://github.com/wu-clan) in [#706](https://github.com/fastapi-practices/fastapi_best_architecture/pull/706)
* Replace gunicorn deployment to granian by [@wu-clan](https://github.com/wu-clan) in [#705](https://github.com/fastapi-practices/fastapi_best_architecture/pull/705)
* Fix the code generation delete schema template by [@wu-clan](https://github.com/wu-clan) in [#708](https://github.com/fastapi-practices/fastapi_best_architecture/pull/708)
* Update the refresh token verify mechanism by [@wu-clan](https://github.com/wu-clan) in [#710](https://github.com/fastapi-practices/fastapi_best_architecture/pull/710)
* Update the reload excludes for CLI run by [@wu-clan](https://github.com/wu-clan) in [#709](https://github.com/fastapi-practices/fastapi_best_architecture/pull/709)
* Add CLI support for execute sql scripts by [@wu-clan](https://github.com/wu-clan) in [#711](https://github.com/fastapi-practices/fastapi_best_architecture/pull/711)
* Update the granian env to command params by [@wu-clan](https://github.com/wu-clan) in [#712](https://github.com/fastapi-practices/fastapi_best_architecture/pull/712)
* Update the middleware logging accuracy by [@wu-clan](https://github.com/wu-clan) in [#713](https://github.com/fastapi-practices/fastapi_best_architecture/pull/713)
* Update the log output default style by [@wu-clan](https://github.com/wu-clan) in [#714](https://github.com/fastapi-practices/fastapi_best_architecture/pull/714)
* Optimize the analysis of get plugins by [@wu-clan](https://github.com/wu-clan) in [#716](https://github.com/fastapi-practices/fastapi_best_architecture/pull/716)
* Simplify user permission database queries by [@wu-clan](https://github.com/wu-clan) in [#717](https://github.com/fastapi-practices/fastapi_best_architecture/pull/717)
* Update the CLI startup service mode by [@wu-clan](https://github.com/wu-clan) in [#718](https://github.com/fastapi-practices/fastapi_best_architecture/pull/718)
* Add support for celery dynamic tasks by [@wu-clan](https://github.com/wu-clan) in [#715](https://github.com/fastapi-practices/fastapi_best_architecture/pull/715)
* Fix the celery task scheduler query by [@wu-clan](https://github.com/wu-clan) in [#719](https://github.com/fastapi-practices/fastapi_best_architecture/pull/719)
* Update the celery task comment and name by [@wu-clan](https://github.com/wu-clan) in [#720](https://github.com/fastapi-practices/fastapi_best_architecture/pull/720)
* Optimize celery integrations and events by [@wu-clan](https://github.com/wu-clan) in [#721](https://github.com/fastapi-practices/fastapi_best_architecture/pull/721)
* Simplify celery task crontab config by [@wu-clan](https://github.com/wu-clan) in [#722](https://github.com/fastapi-practices/fastapi_best_architecture/pull/722)
* Delete the default value of schema enum data by [@wu-clan](https://github.com/wu-clan) in [#723](https://github.com/fastapi-practices/fastapi_best_architecture/pull/723)
* Fix the parsing of execution task params by [@wu-clan](https://github.com/wu-clan) in [#725](https://github.com/fastapi-practices/fastapi_best_architecture/pull/725)
* Bump granian from 2.4.0 to 2.4.2 by [@wu-clan](https://github.com/wu-clan) in [#727](https://github.com/fastapi-practices/fastapi_best_architecture/pull/727)
* Add CLI support for startup celery services by [@wu-clan](https://github.com/wu-clan) in [#724](https://github.com/fastapi-practices/fastapi_best_architecture/pull/724)
* Fix login and operation log clearing by [@wu-clan](https://github.com/wu-clan) in [#728](https://github.com/fastapi-practices/fastapi_best_architecture/pull/728)
**Full Changelog**: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.6.0...v1.7.0
[Changes][v1.7.0]
<a id="v1.6.0"></a> <a id="v1.6.0"></a>
# [v1.6.0](https://github.com/fastapi-practices/fastapi_best_architecture/releases/tag/v1.6.0) - 2025-06-30 # [v1.6.0](https://github.com/fastapi-practices/fastapi_best_architecture/releases/tag/v1.6.0) - 2025-06-30
@@ -679,6 +758,8 @@
[Changes][v1.0.0] [Changes][v1.0.0]
[v1.8.0]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.7.0...v1.8.0
[v1.7.0]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.6.0...v1.7.0
[v1.6.0]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.5.2...v1.6.0 [v1.6.0]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.5.2...v1.6.0
[v1.5.2]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.5.1...v1.5.2 [v1.5.2]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.5.1...v1.5.2
[v1.5.1]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.5.0...v1.5.1 [v1.5.1]: https://github.com/fastapi-practices/fastapi_best_architecture/compare/v1.5.0...v1.5.1
+12 -8
View File
@@ -1,4 +1,4 @@
# Env: dev、pro # Env
ENVIRONMENT='dev' ENVIRONMENT='dev'
# Database # Database
DATABASE_TYPE='mysql' DATABASE_TYPE='mysql'
@@ -15,13 +15,7 @@ REDIS_DATABASE=0
TOKEN_SECRET_KEY='1VkVF75nsNABBjK_7-qz7GtzNy3AMvktc9TCPwKczCk' TOKEN_SECRET_KEY='1VkVF75nsNABBjK_7-qz7GtzNy3AMvktc9TCPwKczCk'
# Opera Log # Opera Log
OPERA_LOG_ENCRYPT_SECRET_KEY='d77b25790a804c2b4a339dd0207941e4cefa5751935a33735bc73bb7071a005b' OPERA_LOG_ENCRYPT_SECRET_KEY='d77b25790a804c2b4a339dd0207941e4cefa5751935a33735bc73bb7071a005b'
# App Admin # [ App ] task
# OAuth2
OAUTH2_GITHUB_CLIENT_ID='test'
OAUTH2_GITHUB_CLIENT_SECRET='test'
OAUTH2_LINUX_DO_CLIENT_ID='test'
OAUTH2_LINUX_DO_CLIENT_SECRET='test'
# App Task
# Celery # Celery
CELERY_BROKER_REDIS_DATABASE=1 CELERY_BROKER_REDIS_DATABASE=1
# Rabbitmq # Rabbitmq
@@ -29,3 +23,13 @@ CELERY_RABBITMQ_HOST='127.0.0.1'
CELERY_RABBITMQ_PORT=5672 CELERY_RABBITMQ_PORT=5672
CELERY_RABBITMQ_USERNAME='guest' CELERY_RABBITMQ_USERNAME='guest'
CELERY_RABBITMQ_PASSWORD='guest' CELERY_RABBITMQ_PASSWORD='guest'
# [ Plugin ] oauth2
OAUTH2_GITHUB_CLIENT_ID='test'
OAUTH2_GITHUB_CLIENT_SECRET='test'
OAUTH2_GOOGLE_CLIENT_ID='test'
OAUTH2_GOOGLE_CLIENT_SECRET='test'
OAUTH2_LINUX_DO_CLIENT_ID='test'
OAUTH2_LINUX_DO_CLIENT_SECRET='test'
# [ Plugin ] email
EMAIL_USERNAME=''
EMAIL_PASSWORD=''
+4 -4
View File
@@ -1,9 +1,9 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from backend.utils.console import console from backend.common.i18n import i18n
__version__ = '1.7.0' __version__ = '1.8.0'
def get_version() -> str | None: # 初始化 i18n
console.print(f'[cyan]{__version__}[/]') i18n.load_locales()
-3
View File
@@ -3,7 +3,6 @@
# ruff: noqa: F403, F401, I001, RUF100 # ruff: noqa: F403, F401, I001, RUF100
import asyncio import asyncio
import os import os
import sys
from logging.config import fileConfig from logging.config import fileConfig
from alembic import context from alembic import context
@@ -11,8 +10,6 @@ from sqlalchemy import pool
from sqlalchemy.engine import Connection from sqlalchemy.engine import Connection
from sqlalchemy.ext.asyncio import async_engine_from_config from sqlalchemy.ext.asyncio import async_engine_from_config
sys.path.append('../')
from backend.app import get_app_models from backend.app import get_app_models
from backend.common.model import MappedBase from backend.common.model import MappedBase
from backend.core import path_conf from backend.core import path_conf
+1
View File
@@ -7,6 +7,7 @@ Create Date: ${create_date}
""" """
from alembic import op from alembic import op
import sqlalchemy as sa import sqlalchemy as sa
import backend.common.model
${imports if imports else ""} ${imports if imports else ""}
# revision identifiers, used by Alembic. # revision identifiers, used by Alembic.
+8 -16
View File
@@ -1,14 +1,12 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
import inspect
import os.path import os.path
from backend.common.log import log
from backend.core.path_conf import BASE_PATH from backend.core.path_conf import BASE_PATH
from backend.utils.import_parse import import_module_cached from backend.utils.import_parse import get_model_object
def get_app_models(): def get_app_models() -> list[type]:
"""获取 app 所有模型类""" """获取 app 所有模型类"""
app_path = os.path.join(BASE_PATH, 'app') app_path = os.path.join(BASE_PATH, 'app')
list_dirs = os.listdir(app_path) list_dirs = os.listdir(app_path)
@@ -19,21 +17,15 @@ def get_app_models():
if os.path.isdir(os.path.join(app_path, d)) and d != '__pycache__': if os.path.isdir(os.path.join(app_path, d)) and d != '__pycache__':
apps.append(d) apps.append(d)
classes = [] objs = []
for app in apps: for app in apps:
try: module_path = f'backend.app.{app}.model'
module_path = f'backend.app.{app}.model' obj = get_model_object(module_path)
module = import_module_cached(module_path) if obj:
except Exception as e: objs.append(obj)
log.warning(f'应用 {app} 中不包含 model 相关配置: {e}')
continue
for name, obj in inspect.getmembers(module): return objs
if inspect.isclass(obj):
classes.append(obj)
return classes
# import all app models for auto create db tables # import all app models for auto create db tables
+2 -10
View File
@@ -1,23 +1,15 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from fastapi import APIRouter, Depends from fastapi import APIRouter
from backend.common.response.response_schema import ResponseModel, response_base from backend.common.response.response_schema import ResponseModel, response_base
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsJwtAuth
from backend.common.security.permission import RequestPermission
from backend.utils.redis_info import redis_info from backend.utils.redis_info import redis_info
router = APIRouter() router = APIRouter()
@router.get( @router.get('', summary='redis 监控', dependencies=[DependsJwtAuth])
'',
summary='redis 监控',
dependencies=[
Depends(RequestPermission('sys:monitor:redis')),
DependsJwtAuth,
],
)
async def get_redis_info() -> ResponseModel: async def get_redis_info() -> ResponseModel:
data = { data = {
'info': await redis_info.get_info(), 'info': await redis_info.get_info(),
+2 -10
View File
@@ -1,24 +1,16 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from fastapi import APIRouter, Depends from fastapi import APIRouter
from starlette.concurrency import run_in_threadpool from starlette.concurrency import run_in_threadpool
from backend.common.response.response_schema import ResponseModel, response_base from backend.common.response.response_schema import ResponseModel, response_base
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsJwtAuth
from backend.common.security.permission import RequestPermission
from backend.utils.server_info import server_info from backend.utils.server_info import server_info
router = APIRouter() router = APIRouter()
@router.get( @router.get('', summary='server 监控', dependencies=[DependsJwtAuth])
'',
summary='server 监控',
dependencies=[
Depends(RequestPermission('sys:monitor:server')),
DependsJwtAuth,
],
)
async def get_server_info() -> ResponseModel: async def get_server_info() -> ResponseModel:
data = { data = {
# 扔到线程池,避免阻塞 # 扔到线程池,避免阻塞
+3 -3
View File
@@ -1,10 +1,10 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Annotated, Any from typing import Annotated
from fastapi import APIRouter, Depends, Path, Query, Request from fastapi import APIRouter, Depends, Path, Query, Request
from backend.app.admin.schema.dept import CreateDeptParam, GetDeptDetail, UpdateDeptParam from backend.app.admin.schema.dept import CreateDeptParam, GetDeptDetail, GetDeptTree, UpdateDeptParam
from backend.app.admin.service.dept_service import dept_service from backend.app.admin.service.dept_service import dept_service
from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsJwtAuth
@@ -27,7 +27,7 @@ async def get_dept_tree(
leader: Annotated[str | None, Query(description='部门负责人')] = None, leader: Annotated[str | None, Query(description='部门负责人')] = None,
phone: Annotated[str | None, Query(description='联系电话')] = None, phone: Annotated[str | None, Query(description='联系电话')] = None,
status: Annotated[int | None, Query(description='状态')] = None, status: Annotated[int | None, Query(description='状态')] = None,
) -> ResponseSchemaModel[list[dict[str, Any]]]: ) -> ResponseSchemaModel[list[GetDeptTree]]:
dept = await dept_service.get_tree(request=request, name=name, leader=leader, phone=phone, status=status) dept = await dept_service.get_tree(request=request, name=name, leader=leader, phone=phone, status=status)
return response_base.success(data=dept) return response_base.success(data=dept)
+2 -2
View File
@@ -4,7 +4,7 @@ from typing import Annotated, Any
from fastapi import APIRouter, Depends, Path, Query, Request from fastapi import APIRouter, Depends, Path, Query, Request
from backend.app.admin.schema.menu import CreateMenuParam, GetMenuDetail, UpdateMenuParam from backend.app.admin.schema.menu import CreateMenuParam, GetMenuDetail, GetMenuTree, UpdateMenuParam
from backend.app.admin.service.menu_service import menu_service from backend.app.admin.service.menu_service import menu_service
from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsJwtAuth
@@ -30,7 +30,7 @@ async def get_menu(pk: Annotated[int, Path(description='菜单 ID')]) -> Respons
async def get_menu_tree( async def get_menu_tree(
title: Annotated[str | None, Query(description='菜单标题')] = None, title: Annotated[str | None, Query(description='菜单标题')] = None,
status: Annotated[int | None, Query(description='状体')] = None, status: Annotated[int | None, Query(description='状体')] = None,
) -> ResponseSchemaModel[list[dict[str, Any]]]: ) -> ResponseSchemaModel[list[GetMenuTree]]:
menu = await menu_service.get_tree(title=title, status=status) menu = await menu_service.get_tree(title=title, status=status)
return response_base.success(data=menu) return response_base.success(data=menu)
+3 -2
View File
@@ -1,9 +1,10 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Annotated, Any from typing import Annotated
from fastapi import APIRouter, Depends, Path, Query from fastapi import APIRouter, Depends, Path, Query
from backend.app.admin.schema.menu import GetMenuTree
from backend.app.admin.schema.role import ( from backend.app.admin.schema.role import (
CreateRoleParam, CreateRoleParam,
DeleteRoleParam, DeleteRoleParam,
@@ -33,7 +34,7 @@ async def get_all_roles() -> ResponseSchemaModel[list[GetRoleDetail]]:
@router.get('/{pk}/menus', summary='获取角色菜单树', dependencies=[DependsJwtAuth]) @router.get('/{pk}/menus', summary='获取角色菜单树', dependencies=[DependsJwtAuth])
async def get_role_menu_tree( async def get_role_menu_tree(
pk: Annotated[int, Path(description='角色 ID')], pk: Annotated[int, Path(description='角色 ID')],
) -> ResponseSchemaModel[list[dict[str, Any] | None]]: ) -> ResponseSchemaModel[list[GetMenuTree] | None]:
menu = await role_service.get_menu_tree(pk=pk) menu = await role_service.get_menu_tree(pk=pk)
return response_base.success(data=menu) return response_base.success(data=menu)
+47 -5
View File
@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Annotated from typing import Annotated
from fastapi import APIRouter, Depends, Path, Query, Request from fastapi import APIRouter, Body, Depends, Path, Query, Request
from backend.app.admin.schema.role import GetRoleDetail from backend.app.admin.schema.role import GetRoleDetail
from backend.app.admin.schema.user import ( from backend.app.admin.schema.user import (
@@ -71,7 +71,7 @@ async def create_user(request: Request, obj: AddUserParam) -> ResponseSchemaMode
return response_base.success(data=data) return response_base.success(data=data)
@router.put('/{pk}', summary='更新用户信息', dependencies=[DependsJwtAuth]) @router.put('/{pk}', summary='更新用户信息', dependencies=[DependsRBAC])
async def update_user( async def update_user(
request: Request, pk: Annotated[int, Path(description='用户 ID')], obj: UpdateUserParam request: Request, pk: Annotated[int, Path(description='用户 ID')], obj: UpdateUserParam
) -> ResponseModel: ) -> ResponseModel:
@@ -93,11 +93,53 @@ async def update_user_permission(
return response_base.fail() return response_base.fail()
@router.put('/{pk}/password', summary='重置用户密码', dependencies=[DependsJwtAuth]) @router.put('/me/password', summary='更新当前用户密码', dependencies=[DependsJwtAuth])
async def update_user_password(request: Request, obj: ResetPasswordParam) -> ResponseModel:
count = await user_service.update_password(request=request, obj=obj)
if count > 0:
return response_base.success()
return response_base.fail()
@router.put('/{pk}/password', summary='重置用户密码', dependencies=[DependsRBAC])
async def reset_user_password( async def reset_user_password(
pk: Annotated[int, Path(description='用户 ID')], obj: ResetPasswordParam request: Request,
pk: Annotated[int, Path(description='用户 ID')],
password: Annotated[str, Body(embed=True, description='新密码')],
) -> ResponseModel: ) -> ResponseModel:
count = await user_service.reset_pwd(pk=pk, obj=obj) count = await user_service.reset_password(request=request, pk=pk, password=password)
if count > 0:
return response_base.success()
return response_base.fail()
@router.put('/me/nickname', summary='更新当前用户昵称', dependencies=[DependsJwtAuth])
async def update_user_nickname(
request: Request, nickname: Annotated[str, Body(embed=True, description='用户昵称')]
) -> ResponseModel:
count = await user_service.update_nickname(request=request, nickname=nickname)
if count > 0:
return response_base.success()
return response_base.fail()
@router.put('/me/avatar', summary='更新当前用户头像', dependencies=[DependsJwtAuth])
async def update_user_avatar(
request: Request, avatar: Annotated[str, Body(embed=True, description='用户头像地址')]
) -> ResponseModel:
count = await user_service.update_avatar(request=request, avatar=avatar)
if count > 0:
return response_base.success()
return response_base.fail()
@router.put('/me/email', summary='更新当前用户邮箱', dependencies=[DependsJwtAuth])
async def update_user_email(
request: Request,
captcha: Annotated[str, Body(embed=True, description='邮箱验证码')],
email: Annotated[str, Body(embed=True, description='用户邮箱')],
) -> ResponseModel:
count = await user_service.update_email(request=request, captcha=captcha, email=email)
if count > 0: if count > 0:
return response_base.success() return response_base.success()
return response_base.fail() return response_base.fail()
+11 -1
View File
@@ -37,11 +37,21 @@ class CRUDOperaLogDao(CRUDPlus[OperaLog]):
创建操作日志 创建操作日志
:param db: 数据库会话 :param db: 数据库会话
:param obj: 创建操作日志参数 :param obj: 操作日志创建参数
:return: :return:
""" """
await self.create_model(db, obj) await self.create_model(db, obj)
async def bulk_create(self, db: AsyncSession, objs: list[CreateOperaLogParam]) -> None:
"""
批量创建操作日志
:param db: 数据库会话
:param objs: 操作日志创建参数列表
:return:
"""
await self.create_models(db, objs)
async def delete(self, db: AsyncSession, pks: list[int]) -> int: async def delete(self, db: AsyncSession, pks: list[int]) -> int:
""" """
批量删除操作日志 批量删除操作日志
+28 -6
View File
@@ -89,10 +89,8 @@ class CRUDUser(CRUDPlus[User]):
:param obj: 注册用户参数 :param obj: 注册用户参数
:return: :return:
""" """
salt = bcrypt.gensalt()
obj.password = get_hash_password(obj.password, salt)
dict_obj = obj.model_dump() dict_obj = obj.model_dump()
dict_obj.update({'is_staff': True, 'salt': salt}) dict_obj.update({'is_staff': True, 'salt': None})
new_user = self.model(**dict_obj) new_user = self.model(**dict_obj)
stmt = select(Role) stmt = select(Role)
@@ -119,6 +117,17 @@ class CRUDUser(CRUDPlus[User]):
input_user.roles = roles.scalars().all() input_user.roles = roles.scalars().all()
return count return count
async def update_nickname(self, db: AsyncSession, user_id: int, nickname: str) -> int:
"""
更新用户昵称
:param db: 数据库会话
:param user_id: 用户 ID
:param nickname: 用户昵称
:return:
"""
return await self.update_model(db, user_id, {'nickname': nickname})
async def update_avatar(self, db: AsyncSession, user_id: int, avatar: str) -> int: async def update_avatar(self, db: AsyncSession, user_id: int, avatar: str) -> int:
""" """
更新用户头像 更新用户头像
@@ -130,6 +139,17 @@ class CRUDUser(CRUDPlus[User]):
""" """
return await self.update_model(db, user_id, {'avatar': avatar}) return await self.update_model(db, user_id, {'avatar': avatar})
async def update_email(self, db: AsyncSession, user_id: int, email: str) -> int:
"""
更新用户邮箱
:param db: 数据库会话
:param user_id: 用户 ID
:param email: 邮箱
:return:
"""
return await self.update_model(db, user_id, {'email': email})
async def delete(self, db: AsyncSession, user_id: int) -> int: async def delete(self, db: AsyncSession, user_id: int) -> int:
""" """
删除用户 删除用户
@@ -150,16 +170,18 @@ class CRUDUser(CRUDPlus[User]):
""" """
return await self.select_model_by_column(db, email=email) return await self.select_model_by_column(db, email=email)
async def reset_password(self, db: AsyncSession, pk: int, new_pwd: str) -> int: async def reset_password(self, db: AsyncSession, pk: int, password: str) -> int:
""" """
重置用户密码 重置用户密码
:param db: 数据库会话 :param db: 数据库会话
:param pk: 用户 ID :param pk: 用户 ID
:param new_pwd: 新密码(已加密) :param password: 新密码
:return: :return:
""" """
return await self.update_model(db, pk, {'password': new_pwd}) salt = bcrypt.gensalt()
new_pwd = get_hash_password(password, salt)
return await self.update_model(db, pk, {'password': new_pwd, 'salt': salt})
async def get_list(self, dept: int | None, username: str | None, phone: str | None, status: int | None) -> Select: async def get_list(self, dept: int | None, username: str | None, phone: str | None, status: int | None) -> Select:
""" """
+4 -4
View File
@@ -2,12 +2,12 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from datetime import datetime from datetime import datetime
from sqlalchemy import DateTime, String from sqlalchemy import String
from sqlalchemy.dialects.mysql import LONGTEXT from sqlalchemy.dialects.mysql import LONGTEXT
from sqlalchemy.dialects.postgresql import TEXT from sqlalchemy.dialects.postgresql import TEXT
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import DataClassBase, id_key from backend.common.model import DataClassBase, TimeZone, id_key
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
@@ -29,7 +29,7 @@ class LoginLog(DataClassBase):
browser: Mapped[str | None] = mapped_column(String(50), comment='浏览器') browser: Mapped[str | None] = mapped_column(String(50), comment='浏览器')
device: Mapped[str | None] = mapped_column(String(50), comment='设备') device: Mapped[str | None] = mapped_column(String(50), comment='设备')
msg: Mapped[str] = mapped_column(LONGTEXT().with_variant(TEXT, 'postgresql'), comment='提示消息') msg: Mapped[str] = mapped_column(LONGTEXT().with_variant(TEXT, 'postgresql'), comment='提示消息')
login_time: Mapped[datetime] = mapped_column(DateTime(timezone=True), comment='登录时间') login_time: Mapped[datetime] = mapped_column(TimeZone, comment='登录时间')
created_time: Mapped[datetime] = mapped_column( created_time: Mapped[datetime] = mapped_column(
DateTime(timezone=True), init=False, default_factory=timezone.now, comment='创建时间' TimeZone, init=False, default_factory=timezone.now, comment='创建时间'
) )
+4 -4
View File
@@ -2,12 +2,12 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from datetime import datetime from datetime import datetime
from sqlalchemy import DateTime, String from sqlalchemy import String
from sqlalchemy.dialects.mysql import JSON, LONGTEXT from sqlalchemy.dialects.mysql import JSON, LONGTEXT
from sqlalchemy.dialects.postgresql import TEXT from sqlalchemy.dialects.postgresql import TEXT
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import DataClassBase, id_key from backend.common.model import DataClassBase, TimeZone, id_key
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
@@ -35,7 +35,7 @@ class OperaLog(DataClassBase):
code: Mapped[str] = mapped_column(String(20), insert_default='200', comment='操作状态码') code: Mapped[str] = mapped_column(String(20), insert_default='200', comment='操作状态码')
msg: Mapped[str | None] = mapped_column(LONGTEXT().with_variant(TEXT, 'postgresql'), comment='提示消息') msg: Mapped[str | None] = mapped_column(LONGTEXT().with_variant(TEXT, 'postgresql'), comment='提示消息')
cost_time: Mapped[float] = mapped_column(insert_default=0.0, comment='请求耗时(ms') cost_time: Mapped[float] = mapped_column(insert_default=0.0, comment='请求耗时(ms')
opera_time: Mapped[datetime] = mapped_column(DateTime(timezone=True), comment='操作时间') opera_time: Mapped[datetime] = mapped_column(TimeZone, comment='操作时间')
created_time: Mapped[datetime] = mapped_column( created_time: Mapped[datetime] = mapped_column(
DateTime(timezone=True), init=False, default_factory=timezone.now, comment='创建时间' TimeZone, init=False, default_factory=timezone.now, comment='创建时间'
) )
+6 -8
View File
@@ -5,12 +5,12 @@ from __future__ import annotations
from datetime import datetime from datetime import datetime
from typing import TYPE_CHECKING from typing import TYPE_CHECKING
from sqlalchemy import VARBINARY, Boolean, DateTime, ForeignKey, String from sqlalchemy import VARBINARY, Boolean, ForeignKey, String
from sqlalchemy.dialects.postgresql import BYTEA, INTEGER from sqlalchemy.dialects.postgresql import BYTEA, INTEGER
from sqlalchemy.orm import Mapped, mapped_column, relationship from sqlalchemy.orm import Mapped, mapped_column, relationship
from backend.app.admin.model.m2m import sys_user_role from backend.app.admin.model.m2m import sys_user_role
from backend.common.model import Base, id_key from backend.common.model import Base, TimeZone, id_key
from backend.database.db import uuid4_str from backend.database.db import uuid4_str
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
@@ -27,8 +27,8 @@ class User(Base):
uuid: Mapped[str] = mapped_column(String(50), init=False, default_factory=uuid4_str, unique=True) uuid: Mapped[str] = mapped_column(String(50), init=False, default_factory=uuid4_str, unique=True)
username: Mapped[str] = mapped_column(String(20), unique=True, index=True, comment='用户名') username: Mapped[str] = mapped_column(String(20), unique=True, index=True, comment='用户名')
nickname: Mapped[str] = mapped_column(String(20), comment='昵称') nickname: Mapped[str] = mapped_column(String(20), comment='昵称')
password: Mapped[str] = mapped_column(String(255), comment='密码') password: Mapped[str | None] = mapped_column(String(255), comment='密码')
salt: Mapped[bytes] = mapped_column(VARBINARY(255).with_variant(BYTEA(255), 'postgresql'), comment='加密盐') salt: Mapped[bytes | None] = mapped_column(VARBINARY(255).with_variant(BYTEA(255), 'postgresql'), comment='加密盐')
email: Mapped[str | None] = mapped_column(String(50), default=None, unique=True, index=True, comment='邮箱') email: Mapped[str | None] = mapped_column(String(50), default=None, unique=True, index=True, comment='邮箱')
phone: Mapped[str | None] = mapped_column(String(11), default=None, comment='手机号') phone: Mapped[str | None] = mapped_column(String(11), default=None, comment='手机号')
avatar: Mapped[str | None] = mapped_column(String(255), default=None, comment='头像') avatar: Mapped[str | None] = mapped_column(String(255), default=None, comment='头像')
@@ -42,11 +42,9 @@ class User(Base):
is_multi_login: Mapped[bool] = mapped_column( is_multi_login: Mapped[bool] = mapped_column(
Boolean().with_variant(INTEGER, 'postgresql'), default=False, comment='是否重复登陆(0否 1是)' Boolean().with_variant(INTEGER, 'postgresql'), default=False, comment='是否重复登陆(0否 1是)'
) )
join_time: Mapped[datetime] = mapped_column( join_time: Mapped[datetime] = mapped_column(TimeZone, init=False, default_factory=timezone.now, comment='注册时间')
DateTime(timezone=True), init=False, default_factory=timezone.now, comment='注册时间'
)
last_login_time: Mapped[datetime | None] = mapped_column( last_login_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), init=False, onupdate=timezone.now, comment='上次登录' TimeZone, init=False, onupdate=timezone.now, comment='上次登录'
) )
# 部门用户一对多 # 部门用户一对多
+6
View File
@@ -37,3 +37,9 @@ class GetDeptDetail(DeptSchemaBase):
del_flag: bool = Field(description='是否删除') del_flag: bool = Field(description='是否删除')
created_time: datetime = Field(description='创建时间') created_time: datetime = Field(description='创建时间')
updated_time: datetime | None = Field(None, description='更新时间') updated_time: datetime | None = Field(None, description='更新时间')
class GetDeptTree(GetDeptDetail):
"""获取部门树"""
children: list['GetDeptTree'] | None = Field(None, description='子菜单')
+6
View File
@@ -43,3 +43,9 @@ class GetMenuDetail(MenuSchemaBase):
id: int = Field(description='菜单 ID') id: int = Field(description='菜单 ID')
created_time: datetime = Field(description='创建时间') created_time: datetime = Field(description='创建时间')
updated_time: datetime | None = Field(None, description='更新时间') updated_time: datetime | None = Field(None, description='更新时间')
class GetMenuTree(GetMenuDetail):
"""获取菜单树"""
children: list['GetMenuTree'] | None = Field(None, description='子菜单')
+9 -6
View File
@@ -3,7 +3,7 @@
from datetime import datetime from datetime import datetime
from typing import Any from typing import Any
from pydantic import ConfigDict, EmailStr, Field, HttpUrl, model_validator from pydantic import ConfigDict, Field, HttpUrl, model_validator
from typing_extensions import Self from typing_extensions import Self
from backend.app.admin.schema.dept import GetDeptDetail from backend.app.admin.schema.dept import GetDeptDetail
@@ -16,7 +16,7 @@ class AuthSchemaBase(SchemaBase):
"""用户认证基础模型""" """用户认证基础模型"""
username: str = Field(description='用户名') username: str = Field(description='用户名')
password: str | None = Field(description='密码') password: str = Field(description='密码')
class AuthLoginParam(AuthSchemaBase): class AuthLoginParam(AuthSchemaBase):
@@ -28,16 +28,19 @@ class AuthLoginParam(AuthSchemaBase):
class AddUserParam(AuthSchemaBase): class AddUserParam(AuthSchemaBase):
"""添加用户参数""" """添加用户参数"""
nickname: str | None = Field(None, description='昵称')
email: CustomEmailStr | None = Field(None, description='邮箱')
phone: CustomPhoneNumber | None = Field(None, description='手机号码')
dept_id: int = Field(description='部门 ID') dept_id: int = Field(description='部门 ID')
roles: list[int] = Field(description='角色 ID 列表') roles: list[int] = Field(description='角色 ID 列表')
nickname: str | None = Field(None, description='昵称')
class AddOAuth2UserParam(AuthSchemaBase): class AddOAuth2UserParam(AuthSchemaBase):
"""添加 OAuth2 用户参数""" """添加 OAuth2 用户参数"""
password: str | None = Field(None, description='密码')
nickname: str | None = Field(None, description='昵称') nickname: str | None = Field(None, description='昵称')
email: EmailStr = Field(description='邮箱') email: CustomEmailStr | None = Field(None, description='邮箱')
avatar: HttpUrl | None = Field(None, description='头像地址') avatar: HttpUrl | None = Field(None, description='头像地址')
@@ -56,6 +59,8 @@ class UserInfoSchemaBase(SchemaBase):
username: str = Field(description='用户名') username: str = Field(description='用户名')
nickname: str = Field(description='昵称') nickname: str = Field(description='昵称')
avatar: HttpUrl | None = Field(None, description='头像地址') avatar: HttpUrl | None = Field(None, description='头像地址')
email: CustomEmailStr | None = Field(None, description='邮箱')
phone: CustomPhoneNumber | None = Field(None, description='手机号')
class UpdateUserParam(UserInfoSchemaBase): class UpdateUserParam(UserInfoSchemaBase):
@@ -72,8 +77,6 @@ class GetUserInfoDetail(UserInfoSchemaBase):
dept_id: int | None = Field(None, description='部门 ID') dept_id: int | None = Field(None, description='部门 ID')
id: int = Field(description='用户 ID') id: int = Field(description='用户 ID')
uuid: str = Field(description='用户 UUID') uuid: str = Field(description='用户 UUID')
email: CustomEmailStr | None = Field(None, description='邮箱')
phone: CustomPhoneNumber | None = Field(None, description='手机号')
status: StatusType = Field(description='状态') status: StatusType = Field(description='状态')
is_superuser: bool = Field(description='是否超级管理员') is_superuser: bool = Field(description='是否超级管理员')
is_staff: bool = Field(description='是否管理员') is_staff: bool = Field(description='是否管理员')
+5 -4
View File
@@ -13,6 +13,7 @@ from backend.app.admin.schema.user import AuthLoginParam
from backend.app.admin.service.login_log_service import login_log_service from backend.app.admin.service.login_log_service import login_log_service
from backend.common.enums import LoginLogStatusType from backend.common.enums import LoginLogStatusType
from backend.common.exception import errors from backend.common.exception import errors
from backend.common.i18n import t
from backend.common.log import log from backend.common.log import log
from backend.common.response.response_code import CustomErrorCode from backend.common.response.response_code import CustomErrorCode
from backend.common.security.jwt import ( from backend.common.security.jwt import (
@@ -33,7 +34,7 @@ class AuthService:
"""认证服务类""" """认证服务类"""
@staticmethod @staticmethod
async def user_verify(db: AsyncSession, username: str, password: str | None) -> User: async def user_verify(db: AsyncSession, username: str, password: str) -> User:
""" """
验证用户名和密码 验证用户名和密码
@@ -93,7 +94,7 @@ class AuthService:
user = await self.user_verify(db, obj.username, obj.password) user = await self.user_verify(db, obj.username, obj.password)
captcha_code = await redis_client.get(f'{settings.CAPTCHA_LOGIN_REDIS_PREFIX}:{request.state.ip}') captcha_code = await redis_client.get(f'{settings.CAPTCHA_LOGIN_REDIS_PREFIX}:{request.state.ip}')
if not captcha_code: if not captcha_code:
raise errors.RequestError(msg='验证码失效,请重新获取') raise errors.RequestError(msg=t('error.captcha.expired'))
if captcha_code.lower() != obj.captcha.lower(): if captcha_code.lower() != obj.captcha.lower():
raise errors.CustomError(error=CustomErrorCode.CAPTCHA_ERROR) raise errors.CustomError(error=CustomErrorCode.CAPTCHA_ERROR)
await redis_client.delete(f'{settings.CAPTCHA_LOGIN_REDIS_PREFIX}:{request.state.ip}') await redis_client.delete(f'{settings.CAPTCHA_LOGIN_REDIS_PREFIX}:{request.state.ip}')
@@ -137,7 +138,7 @@ class AuthService:
msg=e.msg, msg=e.msg,
), ),
) )
raise errors.RequestError(msg=e.msg, background=task) raise errors.RequestError(code=e.code, msg=e.msg, background=task)
except Exception as e: except Exception as e:
log.error(f'登陆错误: {e}') log.error(f'登陆错误: {e}')
raise e raise e
@@ -151,7 +152,7 @@ class AuthService:
username=obj.username, username=obj.username,
login_time=timezone.now(), login_time=timezone.now(),
status=LoginLogStatusType.success.value, status=LoginLogStatusType.success.value,
msg='登录成功', msg=t('success.login.success'),
), ),
) )
data = GetLoginToken( data = GetLoginToken(
@@ -33,6 +33,17 @@ class OperaLogService:
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
await opera_log_dao.create(db, obj) await opera_log_dao.create(db, obj)
@staticmethod
async def bulk_create(*, objs: list[CreateOperaLogParam]) -> None:
"""
批量创建操作日志
:param objs: 操作日志创建参数列表
:return:
"""
async with async_db_session.begin() as db:
await opera_log_dao.bulk_create(db, objs)
@staticmethod @staticmethod
async def delete(*, obj: DeleteOperaLogParam) -> int: async def delete(*, obj: DeleteOperaLogParam) -> int:
""" """
+4 -8
View File
@@ -29,7 +29,7 @@ class PluginService:
keys = [] keys = []
result = [] result = []
async for key in redis_client.scan_iter(f'{settings.PLUGIN_REDIS_PREFIX}:info:*'): async for key in redis_client.scan_iter(f'{settings.PLUGIN_REDIS_PREFIX}:*'):
keys.append(key) keys.append(key)
for info in await redis_client.mget(*keys): for info in await redis_client.mget(*keys):
@@ -74,8 +74,7 @@ class PluginService:
await uninstall_requirements_async(plugin) await uninstall_requirements_async(plugin)
bacup_dir = os.path.join(PLUGIN_DIR, f'{plugin}.{timezone.now().strftime("%Y%m%d%H%M%S")}.backup') bacup_dir = os.path.join(PLUGIN_DIR, f'{plugin}.{timezone.now().strftime("%Y%m%d%H%M%S")}.backup')
shutil.move(plugin_dir, bacup_dir) shutil.move(plugin_dir, bacup_dir)
await redis_client.delete(f'{settings.PLUGIN_REDIS_PREFIX}:info:{plugin}') await redis_client.delete(f'{settings.PLUGIN_REDIS_PREFIX}:{plugin}')
await redis_client.hdel(f'{settings.PLUGIN_REDIS_PREFIX}:status', plugin)
await redis_client.set(f'{settings.PLUGIN_REDIS_PREFIX}:changed', 'ture') await redis_client.set(f'{settings.PLUGIN_REDIS_PREFIX}:changed', 'ture')
@staticmethod @staticmethod
@@ -86,7 +85,7 @@ class PluginService:
:param plugin: 插件名称 :param plugin: 插件名称
:return: :return:
""" """
plugin_info = await redis_client.get(f'{settings.PLUGIN_REDIS_PREFIX}:info:{plugin}') plugin_info = await redis_client.get(f'{settings.PLUGIN_REDIS_PREFIX}:{plugin}')
if not plugin_info: if not plugin_info:
raise errors.NotFoundError(msg='插件不存在') raise errors.NotFoundError(msg='插件不存在')
plugin_info = json.loads(plugin_info) plugin_info = json.loads(plugin_info)
@@ -98,10 +97,7 @@ class PluginService:
else str(StatusType.disable.value) else str(StatusType.disable.value)
) )
plugin_info['plugin']['enable'] = new_status plugin_info['plugin']['enable'] = new_status
await redis_client.set( await redis_client.set(f'{settings.PLUGIN_REDIS_PREFIX}:{plugin}', json.dumps(plugin_info, ensure_ascii=False))
f'{settings.PLUGIN_REDIS_PREFIX}:info:{plugin}', json.dumps(plugin_info, ensure_ascii=False)
)
await redis_client.hset(f'{settings.PLUGIN_REDIS_PREFIX}:status', plugin, new_status)
@staticmethod @staticmethod
async def build(*, plugin: str) -> io.BytesIO: async def build(*, plugin: str) -> io.BytesIO:
+147 -110
View File
@@ -18,7 +18,8 @@ from backend.app.admin.schema.user import (
) )
from backend.common.enums import UserPermissionType from backend.common.enums import UserPermissionType
from backend.common.exception import errors from backend.common.exception import errors
from backend.common.security.jwt import get_hash_password, get_token, jwt_decode, password_verify, superuser_verify from backend.common.response.response_code import CustomErrorCode
from backend.common.security.jwt import get_token, jwt_decode, password_verify, superuser_verify
from backend.core.conf import settings from backend.core.conf import settings
from backend.database.db import async_db_session from backend.database.db import async_db_session
from backend.database.redis import redis_client from backend.database.redis import redis_client
@@ -103,11 +104,10 @@ class UserService:
:return: :return:
""" """
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
superuser_verify(request)
user = await user_dao.get_with_relation(db, user_id=pk) user = await user_dao.get_with_relation(db, user_id=pk)
if not user: if not user:
raise errors.NotFoundError(msg='用户不存在') raise errors.NotFoundError(msg='用户不存在')
if request.user.username != user.username:
raise errors.ForbiddenError(msg='只能修改自己的信息')
if obj.username != user.username: if obj.username != user.username:
if await user_dao.get_by_username(db, obj.username): if await user_dao.get_by_username(db, obj.username):
raise errors.ConflictError(msg='用户名已注册') raise errors.ConflictError(msg='用户名已注册')
@@ -119,98 +119,7 @@ class UserService:
return count return count
@staticmethod @staticmethod
async def update_superuser(*, request: Request, pk: int) -> int: async def update_permission(*, request: Request, pk: int, type: UserPermissionType) -> int:
"""
更新用户管理员状态
:param request: FastAPI 请求对象
:param pk: 用户 ID
:return:
"""
async with async_db_session.begin() as db:
superuser_verify(request)
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
if pk == request.user.id:
raise errors.ForbiddenError(msg='禁止修改自身权限')
count = await user_dao.set_super(db, pk, not user.status)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count
@staticmethod
async def update_staff(*, request: Request, pk: int) -> int:
"""
更新用户职员状态
:param request: FastAPI 请求对象
:param pk: 用户 ID
:return:
"""
async with async_db_session.begin() as db:
superuser_verify(request)
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
if pk == request.user.id:
raise errors.ForbiddenError(msg='禁止修改自身权限')
count = await user_dao.set_staff(db, pk, not user.is_staff)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count
@staticmethod
async def update_status(*, request: Request, pk: int) -> int:
"""
更新用户状态
:param request: FastAPI 请求对象
:param pk: 用户 ID
:return:
"""
async with async_db_session.begin() as db:
superuser_verify(request)
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
if pk == request.user.id:
raise errors.ForbiddenError(msg='禁止修改自身权限')
count = await user_dao.set_status(db, pk, 0 if user.status == 1 else 1)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count
@staticmethod
async def update_multi_login(*, request: Request, pk: int) -> int:
"""
更新用户多端登录状态
:param request: FastAPI 请求对象
:param pk: 用户 ID
:return:
"""
async with async_db_session.begin() as db:
superuser_verify(request)
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
multi_login = user.is_multi_login if pk != user.id else request.user.is_multi_login
new_multi_login = not multi_login
count = await user_dao.set_multi_login(db, pk, new_multi_login)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
token = get_token(request)
token_payload = jwt_decode(token)
if pk == user.id:
# 系统管理员修改自身时,除当前 token 外,其他 token 失效
if not new_multi_login:
key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}'
await redis_client.delete_prefix(key_prefix, exclude=f'{key_prefix}:{token_payload.session_uuid}')
else:
# 系统管理员修改他人时,他人 token 全部失效
if not new_multi_login:
key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}'
await redis_client.delete_prefix(key_prefix)
return count
async def update_permission(self, *, request: Request, pk: int, type: UserPermissionType) -> int:
""" """
更新用户权限 更新用户权限
@@ -219,38 +128,166 @@ class UserService:
:param type: 权限类型 :param type: 权限类型
:return: :return:
""" """
match type: async with async_db_session.begin() as db:
case UserPermissionType.superuser: superuser_verify(request)
count = await self.update_superuser(request=request, pk=pk) match type:
case UserPermissionType.staff: case UserPermissionType.superuser:
count = await self.update_staff(request=request, pk=pk) user = await user_dao.get(db, pk)
case UserPermissionType.status: if not user:
count = await self.update_status(request=request, pk=pk) raise errors.NotFoundError(msg='用户不存在')
case UserPermissionType.multi_login: if pk == request.user.id:
count = await self.update_multi_login(request=request, pk=pk) raise errors.ForbiddenError(msg='禁止修改自身权限')
case _: count = await user_dao.set_super(db, pk, not user.status)
raise errors.RequestError(msg='权限类型不存在') case UserPermissionType.staff:
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
if pk == request.user.id:
raise errors.ForbiddenError(msg='禁止修改自身权限')
count = await user_dao.set_staff(db, pk, not user.is_staff)
case UserPermissionType.status:
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
if pk == request.user.id:
raise errors.ForbiddenError(msg='禁止修改自身权限')
count = await user_dao.set_status(db, pk, 0 if user.status == 1 else 1)
case UserPermissionType.multi_login:
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
multi_login = user.is_multi_login if pk != user.id else request.user.is_multi_login
new_multi_login = not multi_login
count = await user_dao.set_multi_login(db, pk, new_multi_login)
token = get_token(request)
token_payload = jwt_decode(token)
if pk == user.id:
# 系统管理员修改自身时,除当前 token 外,其他 token 失效
if not new_multi_login:
key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}'
await redis_client.delete_prefix(
key_prefix, exclude=f'{key_prefix}:{token_payload.session_uuid}'
)
else:
# 系统管理员修改他人时,他人 token 全部失效
if not new_multi_login:
key_prefix = f'{settings.TOKEN_REDIS_PREFIX}:{user.id}'
await redis_client.delete_prefix(key_prefix)
case _:
raise errors.RequestError(msg='权限类型不存在')
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count return count
@staticmethod @staticmethod
async def reset_pwd(*, pk: int, obj: ResetPasswordParam) -> int: async def reset_password(*, request: Request, pk: int, password: str) -> int:
""" """
重置用户密码 重置用户密码
:param request: FastAPI 请求对象
:param pk: 用户 ID :param pk: 用户 ID
:param password: 新密码
:return:
"""
async with async_db_session.begin() as db:
superuser_verify(request)
user = await user_dao.get(db, pk)
if not user:
raise errors.NotFoundError(msg='用户不存在')
count = await user_dao.reset_password(db, user.id, password)
key_prefix = [
f'{settings.TOKEN_REDIS_PREFIX}:{user.id}',
f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user.id}',
f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}',
]
for prefix in key_prefix:
await redis_client.delete(prefix)
return count
@staticmethod
async def update_nickname(*, request: Request, nickname: str) -> int:
"""
更新当前用户昵称
:param request: FastAPI 请求对象
:param nickname: 用户昵称
:return:
"""
async with async_db_session.begin() as db:
token = get_token(request)
token_payload = jwt_decode(token)
user = await user_dao.get(db, token_payload.id)
if not user:
raise errors.NotFoundError(msg='用户不存在')
count = await user_dao.update_nickname(db, token_payload.id, nickname)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count
@staticmethod
async def update_avatar(*, request: Request, avatar: str) -> int:
"""
更新当前用户头像
:param request: FastAPI 请求对象
:param avatar: 头像地址
:return:
"""
async with async_db_session.begin() as db:
token = get_token(request)
token_payload = jwt_decode(token)
user = await user_dao.get(db, token_payload.id)
if not user:
raise errors.NotFoundError(msg='用户不存在')
count = await user_dao.update_avatar(db, token_payload.id, avatar)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count
@staticmethod
async def update_email(*, request: Request, captcha: str, email: str) -> int:
"""
更新当前用户邮箱
:param request: FastAPI 请求对象
:param captcha: 邮箱验证码
:param email: 邮箱
:return:
"""
async with async_db_session.begin() as db:
token = get_token(request)
token_payload = jwt_decode(token)
user = await user_dao.get(db, token_payload.id)
if not user:
raise errors.NotFoundError(msg='用户不存在')
captcha_code = await redis_client.get(f'{settings.EMAIL_CAPTCHA_REDIS_PREFIX}:{request.state.ip}')
if not captcha_code:
raise errors.RequestError(msg='验证码已失效,请重新获取')
if captcha != captcha_code:
raise errors.CustomError(error=CustomErrorCode.CAPTCHA_ERROR)
await redis_client.delete(f'{settings.EMAIL_CAPTCHA_REDIS_PREFIX}:{request.state.ip}')
count = await user_dao.update_email(db, token_payload.id, email)
await redis_client.delete(f'{settings.JWT_USER_REDIS_PREFIX}:{user.id}')
return count
@staticmethod
async def update_password(*, request: Request, obj: ResetPasswordParam) -> int:
"""
更新当前用户密码
:param request: FastAPI 请求对象
:param obj: 密码重置参数 :param obj: 密码重置参数
:return: :return:
""" """
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
user = await user_dao.get(db, pk) token = get_token(request)
token_payload = jwt_decode(token)
user = await user_dao.get(db, token_payload.id)
if not user: if not user:
raise errors.NotFoundError(msg='用户不存在') raise errors.NotFoundError(msg='用户不存在')
if not password_verify(obj.old_password, user.password): if not password_verify(obj.old_password, user.password):
raise errors.RequestError(msg='原密码错误') raise errors.RequestError(msg='原密码错误')
if obj.new_password != obj.confirm_password: if obj.new_password != obj.confirm_password:
raise errors.RequestError(msg='密码输入不一致') raise errors.RequestError(msg='密码输入不一致')
new_pwd = get_hash_password(obj.new_password, user.salt) count = await user_dao.reset_password(db, user.id, obj.new_password)
count = await user_dao.reset_password(db, user.id, new_pwd)
key_prefix = [ key_prefix = [
f'{settings.TOKEN_REDIS_PREFIX}:{user.id}', f'{settings.TOKEN_REDIS_PREFIX}:{user.id}',
f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user.id}', f'{settings.TOKEN_REFRESH_REDIS_PREFIX}:{user.id}',
+3 -1
View File
@@ -2,11 +2,13 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from fastapi import APIRouter from fastapi import APIRouter
from backend.app.task.api.v1.control import router as task_control_router
from backend.app.task.api.v1.result import router as task_result_router from backend.app.task.api.v1.result import router as task_result_router
from backend.app.task.api.v1.scheduler import router as task_scheduler_router from backend.app.task.api.v1.scheduler import router as task_scheduler_router
from backend.core.conf import settings from backend.core.conf import settings
v1 = APIRouter(prefix=f'{settings.FASTAPI_API_V1_PATH}/task', tags=['任务']) v1 = APIRouter(prefix=f'{settings.FASTAPI_API_V1_PATH}/tasks', tags=['任务'])
v1.include_router(task_control_router)
v1.include_router(task_result_router, prefix='/results') v1.include_router(task_result_router, prefix='/results')
v1.include_router(task_scheduler_router, prefix='/schedulers') v1.include_router(task_scheduler_router, prefix='/schedulers')
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from typing import Annotated
from fastapi import APIRouter, Depends, Path
from starlette.concurrency import run_in_threadpool
from backend.app.task import celery_app
from backend.app.task.schema.control import TaskRegisteredDetail
from backend.common.exception import errors
from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base
from backend.common.security.jwt import DependsJwtAuth
from backend.common.security.permission import RequestPermission
from backend.common.security.rbac import DependsRBAC
router = APIRouter()
@router.get('/registered', summary='获取已注册的任务', dependencies=[DependsJwtAuth])
async def get_task_registered() -> ResponseSchemaModel[list[TaskRegisteredDetail]]:
inspector = celery_app.control.inspect(timeout=0.5)
registered = await run_in_threadpool(inspector.registered)
if not registered:
raise errors.ServerError(msg='Celery Worker 暂不可用,请稍后重试')
task_registered = []
celery_app_tasks = celery_app.tasks
for _, tasks in registered.items():
for task in tasks:
task_ins = celery_app_tasks.get(task)
if task_ins:
task_doc = task_ins.__doc__
task_registered.append({'name': task_doc or task_ins, 'task': task_ins})
else:
task_registered.append({'name': task, 'task': task})
return response_base.success(data=task_registered)
@router.delete(
'/{task_id}/cancel',
summary='撤销任务',
dependencies=[
Depends(RequestPermission('sys:task:revoke')),
DependsRBAC,
],
)
async def revoke_task(task_id: Annotated[str, Path(description='任务 UUID')]) -> ResponseModel:
workers = await run_in_threadpool(celery_app.control.ping, timeout=0.5)
if not workers:
raise errors.ServerError(msg='Celery Worker 暂不可用,请稍后重试')
celery_app.control.revoke(task_id)
return response_base.success()
-13
View File
@@ -119,16 +119,3 @@ async def delete_task_scheduler(pk: Annotated[int, Path(description='任务调
async def execute_task(pk: Annotated[int, Path(description='任务调度 ID')]) -> ResponseModel: async def execute_task(pk: Annotated[int, Path(description='任务调度 ID')]) -> ResponseModel:
await task_scheduler_service.execute(pk=pk) await task_scheduler_service.execute(pk=pk)
return response_base.success() return response_base.success()
@router.delete(
'/{task_id}/cancel',
summary='撤销任务',
dependencies=[
Depends(RequestPermission('sys:task:revoke')),
DependsRBAC,
],
)
async def revoke_task(task_id: Annotated[str, Path(description='任务 UUID')]) -> ResponseModel:
await task_scheduler_service.revoke(task_id=task_id)
return response_base.success()
+14 -11
View File
@@ -5,7 +5,6 @@ import os
import celery import celery
import celery_aio_pool import celery_aio_pool
from backend.app.task.model.result import OVERWRITE_CELERY_RESULT_GROUP_TABLE_NAME, OVERWRITE_CELERY_RESULT_TABLE_NAME
from backend.app.task.tasks.beat import LOCAL_BEAT_SCHEDULE from backend.app.task.tasks.beat import LOCAL_BEAT_SCHEDULE
from backend.core.conf import settings from backend.core.conf import settings
from backend.core.path_conf import BASE_PATH from backend.core.path_conf import BASE_PATH
@@ -13,7 +12,8 @@ from backend.core.path_conf import BASE_PATH
def find_task_packages(): def find_task_packages():
packages = [] packages = []
for root, dirs, files in os.walk(os.path.join(BASE_PATH, 'app', 'task', 'tasks')): task_dir = os.path.join(BASE_PATH, 'app', 'task', 'tasks')
for root, dirs, files in os.walk(task_dir):
if 'tasks.py' in files: if 'tasks.py' in files:
package = root.replace(str(BASE_PATH.parent) + os.path.sep, '').replace(os.path.sep, '.') package = root.replace(str(BASE_PATH.parent) + os.path.sep, '').replace(os.path.sep, '.')
packages.append(package) packages.append(package)
@@ -29,21 +29,19 @@ def init_celery() -> celery.Celery:
celery.app.trace.build_tracer = celery_aio_pool.build_async_tracer celery.app.trace.build_tracer = celery_aio_pool.build_async_tracer
celery.app.trace.reset_worker_optimizations() celery.app.trace.reset_worker_optimizations()
# https://docs.celeryq.dev/en/stable/userguide/configuration.html
app = celery.Celery( app = celery.Celery(
'fba_celery', 'fba_celery',
broker=f'redis://:{settings.REDIS_PASSWORD}@{settings.REDIS_HOST}:{settings.REDIS_PORT}/{settings.CELERY_BROKER_REDIS_DATABASE}' broker_url=f'redis://:{settings.REDIS_PASSWORD}@{settings.REDIS_HOST}:{settings.REDIS_PORT}/{settings.CELERY_BROKER_REDIS_DATABASE}'
if settings.CELERY_BROKER == 'redis' if settings.CELERY_BROKER == 'redis'
else f'amqp://{settings.CELERY_RABBITMQ_USERNAME}:{settings.CELERY_RABBITMQ_PASSWORD}@{settings.CELERY_RABBITMQ_HOST}:{settings.CELERY_RABBITMQ_PORT}', else f'amqp://{settings.CELERY_RABBITMQ_USERNAME}:{settings.CELERY_RABBITMQ_PASSWORD}@{settings.CELERY_RABBITMQ_HOST}:{settings.CELERY_RABBITMQ_PORT}',
broker_connection_retry_on_startup=True, broker_connection_retry_on_startup=True,
backend=f'db+{settings.DATABASE_TYPE}+{"pymysql" if settings.DATABASE_TYPE == "mysql" else "psycopg"}' result_backend=f'db+{settings.DATABASE_TYPE}+{"pymysql" if settings.DATABASE_TYPE == "mysql" else "psycopg"}'
f'://{settings.DATABASE_USER}:{settings.DATABASE_PASSWORD}@{settings.DATABASE_HOST}:{settings.DATABASE_PORT}/{settings.DATABASE_SCHEMA}', f'://{settings.DATABASE_USER}:{settings.DATABASE_PASSWORD}@{settings.DATABASE_HOST}:{settings.DATABASE_PORT}/{settings.DATABASE_SCHEMA}',
database_engine_options={'echo': settings.DATABASE_ECHO},
database_table_names={
'task': OVERWRITE_CELERY_RESULT_TABLE_NAME,
'group': OVERWRITE_CELERY_RESULT_GROUP_TABLE_NAME,
},
result_extended=True, result_extended=True,
# result_expires=0, # 任务结果自动清理,0 或 None 表示不清理 database_engine_options={'echo': settings.DATABASE_ECHO},
# result_expires=0,
# beat_sync_every=1,
beat_schedule=LOCAL_BEAT_SCHEDULE, beat_schedule=LOCAL_BEAT_SCHEDULE,
beat_scheduler='backend.app.task.utils.schedulers:DatabaseScheduler', beat_scheduler='backend.app.task.utils.schedulers:DatabaseScheduler',
task_cls='backend.app.task.tasks.base:TaskBase', task_cls='backend.app.task.tasks.base:TaskBase',
@@ -52,8 +50,13 @@ def init_celery() -> celery.Celery:
timezone=settings.DATETIME_TIMEZONE, timezone=settings.DATETIME_TIMEZONE,
) )
# 在 Celery 中设置此参数无效
# 参数:https://github.com/celery/celery/issues/7270
app.loader.override_backends = {'db': 'backend.app.task.database:DatabaseBackend'}
# 自动发现任务 # 自动发现任务
app.autodiscover_tasks(find_task_packages()) packages = find_task_packages()
app.autodiscover_tasks(packages)
return app return app
+2 -2
View File
@@ -4,7 +4,7 @@ from sqlalchemy import Select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus from sqlalchemy_crud_plus import CRUDPlus
from backend.app.task.model.result import TaskResult from backend.app.task.model import TaskResult
class CRUDTaskResult(CRUDPlus[TaskResult]): class CRUDTaskResult(CRUDPlus[TaskResult]):
@@ -35,7 +35,7 @@ class CRUDTaskResult(CRUDPlus[TaskResult]):
if task_id is not None: if task_id is not None:
filters['task_id'] = task_id filters['task_id'] = task_id
return await self.select_order('id', **filters) return await self.select_order('id', 'desc', **filters)
async def delete(self, db: AsyncSession, pks: list[int]) -> int: async def delete(self, db: AsyncSession, pks: list[int]) -> int:
""" """
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from celery import states
from celery.backends.base import BaseBackend
from celery.backends.database import retry, session_cleanup
from celery.exceptions import ImproperlyConfigured
from celery.utils.time import maybe_timedelta
from backend.app.task.model.result import Task, TaskExtended, TaskSet
from backend.app.task.session import SessionManager
"""
重写 from celery.backends.database 内部 DatabaseBackend 此类实现与模型配合不佳导致 fba 创建表和 alembic 迁移困难
"""
class DatabaseBackend(BaseBackend):
"""The database result backend."""
# ResultSet.iterate should sleep this much between each pool,
# to not bombard the database with queries.
subpolling_interval = 0.5
task_cls = Task
taskset_cls = TaskSet
def __init__(self, dburi=None, engine_options=None, url=None, **kwargs):
# The `url` argument was added later and is used by
# the app to set backend by url (celery.app.backends.by_url)
super().__init__(expires_type=maybe_timedelta, url=url, **kwargs)
conf = self.app.conf
if self.extended_result:
self.task_cls = TaskExtended
self.url = url or dburi or conf.database_url
self.engine_options = dict(engine_options or {}, **conf.database_engine_options or {})
self.short_lived_sessions = kwargs.get('short_lived_sessions', conf.database_short_lived_sessions)
schemas = conf.database_table_schemas or {}
tablenames = conf.database_table_names or {}
self.task_cls.configure(schema=schemas.get('task'), name=tablenames.get('task'))
self.taskset_cls.configure(schema=schemas.get('group'), name=tablenames.get('group'))
if not self.url:
raise ImproperlyConfigured(
'Missing connection string! Do you have the database_url setting set to a real value?'
)
self.session_manager = SessionManager()
create_tables_at_setup = conf.database_create_tables_at_setup
if create_tables_at_setup is True:
self._create_tables()
@property
def extended_result(self):
return self.app.conf.find_value_for_key('extended', 'result')
def _create_tables(self):
"""Create the task and taskset tables."""
self.ResultSession()
def ResultSession(self, session_manager=None):
if session_manager is None:
session_manager = self.session_manager
return session_manager.session_factory(
dburi=self.url, short_lived_sessions=self.short_lived_sessions, **self.engine_options
)
@retry
def _store_result(self, task_id, result, state, traceback=None, request=None, **kwargs):
"""Store return value and state of an executed task."""
session = self.ResultSession()
with session_cleanup(session):
task = list(session.query(self.task_cls).filter(self.task_cls.task_id == task_id))
task = task and task[0]
if not task:
task = self.task_cls(task_id)
task.task_id = task_id
session.add(task)
session.flush()
self._update_result(task, result, state, traceback=traceback, request=request)
session.commit()
def _update_result(self, task, result, state, traceback=None, request=None):
meta = self._get_result_meta(
result=result, state=state, traceback=traceback, request=request, format_date=False, encode=True
)
# Exclude the primary key id and task_id columns
# as we should not set it None
columns = [column.name for column in self.task_cls.__table__.columns if column.name not in {'id', 'task_id'}]
# Iterate through the columns name of the table
# to set the value from meta.
# If the value is not present in meta, set None
for column in columns:
value = meta.get(column)
setattr(task, column, value)
@retry
def _get_task_meta_for(self, task_id):
"""Get task meta-data for a task by id."""
session = self.ResultSession()
with session_cleanup(session):
task = list(session.query(self.task_cls).filter(self.task_cls.task_id == task_id))
task = task and task[0]
if not task:
task = self.task_cls(task_id)
task.status = states.PENDING
task.result = None
data = task.to_dict()
if data.get('args', None) is not None:
data['args'] = self.decode(data['args'])
if data.get('kwargs', None) is not None:
data['kwargs'] = self.decode(data['kwargs'])
return self.meta_from_decoded(data)
@retry
def _save_group(self, group_id, result):
"""Store the result of an executed group."""
session = self.ResultSession()
with session_cleanup(session):
group = self.taskset_cls(group_id, result)
session.add(group)
session.flush()
session.commit()
return result
@retry
def _restore_group(self, group_id):
"""Get meta-data for group by id."""
session = self.ResultSession()
with session_cleanup(session):
group = session.query(self.taskset_cls).filter(self.taskset_cls.taskset_id == group_id).first()
if group:
return group.to_dict()
@retry
def _delete_group(self, group_id):
"""Delete meta-data for group by id."""
session = self.ResultSession()
with session_cleanup(session):
session.query(self.taskset_cls).filter(self.taskset_cls.taskset_id == group_id).delete()
session.flush()
session.commit()
@retry
def _forget(self, task_id):
"""Forget about result."""
session = self.ResultSession()
with session_cleanup(session):
session.query(self.task_cls).filter(self.task_cls.task_id == task_id).delete()
session.commit()
def cleanup(self):
"""Delete expired meta-data."""
session = self.ResultSession()
expires = self.expires
now = self.app.now()
with session_cleanup(session):
session.query(self.task_cls).filter(self.task_cls.date_done < (now - expires)).delete()
session.query(self.taskset_cls).filter(self.taskset_cls.date_done < (now - expires)).delete()
session.commit()
def __reduce__(self, args=(), kwargs=None):
kwargs = {} if not kwargs else kwargs
kwargs.update({'dburi': self.url, 'expires': self.expires, 'engine_options': self.engine_options})
return super().__reduce__(args, kwargs)
+1
View File
@@ -1,3 +1,4 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from backend.app.task.model.result import TaskExtended as TaskResult
from backend.app.task.model.scheduler import TaskScheduler from backend.app.task.model.scheduler import TaskScheduler
+105 -5
View File
@@ -1,9 +1,109 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from celery.backends.database.models import TaskExtended as TaskResult from datetime import datetime, timezone
OVERWRITE_CELERY_RESULT_TABLE_NAME = 'task_result' import sqlalchemy as sa
OVERWRITE_CELERY_RESULT_GROUP_TABLE_NAME = 'task_group_result'
# 重写表名配置 from celery import states
TaskResult.configure(name=OVERWRITE_CELERY_RESULT_TABLE_NAME) from sqlalchemy.types import PickleType
from backend.common.model import MappedBase
"""
重写 celery.backends.database.models 内部所有模型适配 fba 创建表和 alembic 迁移
"""
class Task(MappedBase):
"""Task result/status."""
__tablename__ = 'task_result'
__table_args__ = {'comment': '任务结果表'}
id = sa.Column(sa.Integer, sa.Sequence('task_id_sequence'), primary_key=True, autoincrement=True)
task_id = sa.Column(sa.String(155), unique=True)
status = sa.Column(sa.String(50), default=states.PENDING)
result = sa.Column(PickleType, nullable=True)
date_done = sa.Column(
sa.DateTime, default=datetime.now(timezone.utc), onupdate=datetime.now(timezone.utc), nullable=True
)
traceback = sa.Column(sa.Text, nullable=True)
def __init__(self, task_id):
self.task_id = task_id
def to_dict(self):
return {
'task_id': self.task_id,
'status': self.status,
'result': self.result,
'traceback': self.traceback,
'date_done': self.date_done,
}
def __repr__(self):
return '<Task {0.task_id} state: {0.status}>'.format(self)
@classmethod
def configure(cls, schema=None, name=None):
cls.__table__.schema = schema
cls.id.default.schema = schema
cls.__table__.name = name or cls.__tablename__
class TaskExtended(Task):
"""For the extend result."""
__tablename__ = 'task_result'
__table_args__ = {'extend_existing': True, 'comment': '任务结果表'}
name = sa.Column(sa.String(155), nullable=True)
args = sa.Column(sa.LargeBinary, nullable=True)
kwargs = sa.Column(sa.LargeBinary, nullable=True)
worker = sa.Column(sa.String(155), nullable=True)
retries = sa.Column(sa.Integer, nullable=True)
queue = sa.Column(sa.String(155), nullable=True)
def to_dict(self):
task_dict = super().to_dict()
task_dict.update({
'name': self.name,
'args': self.args,
'kwargs': self.kwargs,
'worker': self.worker,
'retries': self.retries,
'queue': self.queue,
})
return task_dict
class TaskSet(MappedBase):
"""TaskSet result."""
__tablename__ = 'task_set_result'
__table_args__ = {'comment': '任务集结果表'}
id = sa.Column(sa.Integer, sa.Sequence('taskset_id_sequence'), autoincrement=True, primary_key=True)
taskset_id = sa.Column(sa.String(155), unique=True)
result = sa.Column(PickleType, nullable=True)
date_done = sa.Column(sa.DateTime, default=datetime.now(timezone.utc), nullable=True)
def __init__(self, taskset_id, result):
self.taskset_id = taskset_id
self.result = result
def to_dict(self):
return {
'taskset_id': self.taskset_id,
'result': self.result,
'date_done': self.date_done,
}
def __repr__(self):
return f'<TaskSet: {self.taskset_id}>'
@classmethod
def configure(cls, schema=None, name=None):
cls.__table__.schema = schema
cls.id.default.schema = schema
cls.__table__.name = name or cls.__tablename__
+4 -7
View File
@@ -7,7 +7,6 @@ from datetime import datetime
from sqlalchemy import ( from sqlalchemy import (
JSON, JSON,
Boolean, Boolean,
DateTime,
String, String,
event, event,
) )
@@ -16,7 +15,7 @@ from sqlalchemy.dialects.postgresql import INTEGER, TEXT
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.exception import errors from backend.common.exception import errors
from backend.common.model import Base, id_key from backend.common.model import Base, TimeZone, id_key
from backend.core.conf import settings from backend.core.conf import settings
from backend.database.redis import redis_client from backend.database.redis import redis_client
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
@@ -35,8 +34,8 @@ class TaskScheduler(Base):
queue: Mapped[str | None] = mapped_column(String(255), comment='CELERY_TASK_QUEUES 中定义的队列') queue: Mapped[str | None] = mapped_column(String(255), comment='CELERY_TASK_QUEUES 中定义的队列')
exchange: Mapped[str | None] = mapped_column(String(255), comment='低级别 AMQP 路由的交换机') exchange: Mapped[str | None] = mapped_column(String(255), comment='低级别 AMQP 路由的交换机')
routing_key: Mapped[str | None] = mapped_column(String(255), comment='低级别 AMQP 路由的路由密钥') routing_key: Mapped[str | None] = mapped_column(String(255), comment='低级别 AMQP 路由的路由密钥')
start_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), comment='任务开始触发的时间') start_time: Mapped[datetime | None] = mapped_column(TimeZone, comment='任务开始触发的时间')
expire_time: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), comment='任务不再触发的截止时间') expire_time: Mapped[datetime | None] = mapped_column(TimeZone, comment='任务不再触发的截止时间')
expire_seconds: Mapped[int | None] = mapped_column(comment='任务不再触发的秒数时间差') expire_seconds: Mapped[int | None] = mapped_column(comment='任务不再触发的秒数时间差')
type: Mapped[int] = mapped_column(comment='调度类型(0间隔 1定时)') type: Mapped[int] = mapped_column(comment='调度类型(0间隔 1定时)')
interval_every: Mapped[int | None] = mapped_column(comment='任务再次运行前的间隔周期数') interval_every: Mapped[int | None] = mapped_column(comment='任务再次运行前的间隔周期数')
@@ -49,9 +48,7 @@ class TaskScheduler(Base):
Boolean().with_variant(INTEGER, 'postgresql'), default=True, comment='是否启用任务' Boolean().with_variant(INTEGER, 'postgresql'), default=True, comment='是否启用任务'
) )
total_run_count: Mapped[int] = mapped_column(default=0, comment='任务触发的总次数') total_run_count: Mapped[int] = mapped_column(default=0, comment='任务触发的总次数')
last_run_time: Mapped[datetime | None] = mapped_column( last_run_time: Mapped[datetime | None] = mapped_column(TimeZone, default=None, comment='任务最后触发的时间')
DateTime(timezone=True), default=None, comment='任务最后触发的时间'
)
remark: Mapped[str | None] = mapped_column( remark: Mapped[str | None] = mapped_column(
LONGTEXT().with_variant(TEXT, 'postgresql'), default=None, comment='备注' LONGTEXT().with_variant(TEXT, 'postgresql'), default=None, comment='备注'
) )
+8
View File
@@ -0,0 +1,8 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from backend.common.schema import SchemaBase
class TaskRegisteredDetail(SchemaBase):
name: str
task: str
+6 -1
View File
@@ -3,8 +3,9 @@
from datetime import datetime from datetime import datetime
from typing import Any from typing import Any
from pydantic import ConfigDict, Field from pydantic import ConfigDict, Field, field_serializer
from backend.app.task import celery_app
from backend.common.schema import SchemaBase from backend.common.schema import SchemaBase
@@ -36,3 +37,7 @@ class GetTaskResultDetail(TaskResultSchemaBase):
model_config = ConfigDict(from_attributes=True) model_config = ConfigDict(from_attributes=True)
id: int = Field(description='任务结果 ID') id: int = Field(description='任务结果 ID')
@field_serializer('args', 'kwargs', when_used='unless-none')
def serialize_params(self, value: bytes | None, _info) -> Any:
return celery_app.backend.decode(value)
+1 -1
View File
@@ -3,7 +3,7 @@
from sqlalchemy import Select from sqlalchemy import Select
from backend.app.task.crud.crud_result import task_result_dao from backend.app.task.crud.crud_result import task_result_dao
from backend.app.task.model.result import TaskResult from backend.app.task.model import TaskResult
from backend.app.task.schema.result import DeleteTaskResultParam from backend.app.task.schema.result import DeleteTaskResultParam
from backend.common.exception import errors from backend.common.exception import errors
from backend.database.db import async_db_session from backend.database.db import async_db_session
+2 -29
View File
@@ -65,14 +65,7 @@ class TaskSchedulerService:
if task_scheduler: if task_scheduler:
raise errors.ConflictError(msg='任务调度已存在') raise errors.ConflictError(msg='任务调度已存在')
if obj.type == TaskSchedulerType.CRONTAB: if obj.type == TaskSchedulerType.CRONTAB:
crontab_split = obj.crontab.split(' ') crontab_verify(obj.crontab)
if len(crontab_split) != 5:
raise errors.RequestError(msg='Crontab 表达式非法')
crontab_verify('m', crontab_split[0])
crontab_verify('h', crontab_split[1])
crontab_verify('dow', crontab_split[2])
crontab_verify('dom', crontab_split[3])
crontab_verify('moy', crontab_split[4])
await task_scheduler_dao.create(db, obj) await task_scheduler_dao.create(db, obj)
@staticmethod @staticmethod
@@ -92,14 +85,7 @@ class TaskSchedulerService:
if await task_scheduler_dao.get_by_name(db, obj.name): if await task_scheduler_dao.get_by_name(db, obj.name):
raise errors.ConflictError(msg='任务调度已存在') raise errors.ConflictError(msg='任务调度已存在')
if task_scheduler.type == TaskSchedulerType.CRONTAB: if task_scheduler.type == TaskSchedulerType.CRONTAB:
crontab_split = obj.crontab.split(' ') crontab_verify(obj.crontab)
if len(crontab_split) != 5:
raise errors.RequestError(msg='Crontab 表达式非法')
crontab_verify('m', crontab_split[0])
crontab_verify('h', crontab_split[1])
crontab_verify('dow', crontab_split[2])
crontab_verify('dom', crontab_split[3])
crontab_verify('moy', crontab_split[4])
count = await task_scheduler_dao.update(db, pk, obj) count = await task_scheduler_dao.update(db, pk, obj)
return count return count
@@ -156,18 +142,5 @@ class TaskSchedulerService:
else: else:
celery_app.send_task(name=task_scheduler.task, args=args, kwargs=kwargs) celery_app.send_task(name=task_scheduler.task, args=args, kwargs=kwargs)
@staticmethod
async def revoke(*, task_id: str) -> None:
"""
撤销指定的任务
:param task_id: 任务 UUID
:return:
"""
workers = await run_in_threadpool(celery_app.control.ping, timeout=0.5)
if not workers:
raise errors.ServerError(msg='Celery Worker 暂不可用,请稍后重试')
celery_app.control.revoke(task_id)
task_scheduler_service: TaskSchedulerService = TaskSchedulerService() task_scheduler_service: TaskSchedulerService = TaskSchedulerService()
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from celery.backends.database.session import SessionManager as CelerySessionManager
class SessionManager(CelerySessionManager):
"""
重写 celery SessionManager
"""
def __init__(self):
super().__init__()
# 禁止自动创建 celery 内部定义的任务结果表
self.prepared = True
+8 -1
View File
@@ -4,15 +4,22 @@ from celery.schedules import schedule
from backend.app.task.utils.tzcrontab import TzAwareCrontab from backend.app.task.utils.tzcrontab import TzAwareCrontab
# 参考:https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html
LOCAL_BEAT_SCHEDULE = { LOCAL_BEAT_SCHEDULE = {
'测试同步任务': { '测试同步任务': {
'task': 'task_demo', 'task': 'task_demo',
'schedule': schedule(5), 'schedule': schedule(30),
}, },
'测试异步任务': { '测试异步任务': {
'task': 'task_demo_async', 'task': 'task_demo_async',
'schedule': TzAwareCrontab('1'), 'schedule': TzAwareCrontab('1'),
}, },
'测试传参任务': {
'task': 'task_demo_params',
'schedule': TzAwareCrontab('1'),
'args': ['你好,'],
'kwargs': {'world': '世界'},
},
'清理操作日志': { '清理操作日志': {
'task': 'backend.app.task.tasks.db_log.tasks.delete_db_opera_log', 'task': 'backend.app.task.tasks.db_log.tasks.delete_db_opera_log',
'schedule': TzAwareCrontab('0', '0', day_of_week='6'), 'schedule': TzAwareCrontab('0', '0', day_of_week='6'),
+4 -3
View File
@@ -1,18 +1,19 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from celery import shared_task
from backend.app.admin.service.login_log_service import login_log_service from backend.app.admin.service.login_log_service import login_log_service
from backend.app.admin.service.opera_log_service import opera_log_service from backend.app.admin.service.opera_log_service import opera_log_service
from backend.app.task.celery import celery_app
@celery_app.task @shared_task
async def delete_db_opera_log() -> str: async def delete_db_opera_log() -> str:
"""自动删除数据库操作日志""" """自动删除数据库操作日志"""
await opera_log_service.delete_all() await opera_log_service.delete_all()
return 'Success' return 'Success'
@celery_app.task @shared_task
async def delete_db_login_log() -> str: async def delete_db_login_log() -> str:
"""自动删除数据库登录日志""" """自动删除数据库登录日志"""
await login_log_service.delete_all() await login_log_service.delete_all()
+8 -2
View File
@@ -10,12 +10,18 @@ from backend.app.task.celery import celery_app
@celery_app.task(name='task_demo') @celery_app.task(name='task_demo')
def task_demo() -> str: def task_demo() -> str:
"""示例任务,模拟耗时操作""" """示例任务,模拟耗时操作"""
sleep(20) sleep(30)
return 'test async' return 'test async'
@celery_app.task(name='task_demo_async') @celery_app.task(name='task_demo_async')
async def task_demo_async() -> str: async def task_demo_async() -> str:
"""异步示例任务,模拟耗时操作""" """异步示例任务,模拟耗时操作"""
await asleep(20) await asleep(30)
return 'test async' return 'test async'
@celery_app.task(name='task_demo_params')
async def task_demo_params(hello: str, world: str | None = None) -> str:
"""参数示例任务,模拟传参操作"""
return hello + world
+107 -63
View File
@@ -9,7 +9,9 @@ from multiprocessing.util import Finalize
from celery import current_app, schedules from celery import current_app, schedules
from celery.beat import ScheduleEntry, Scheduler from celery.beat import ScheduleEntry, Scheduler
from celery.signals import beat_init
from celery.utils.log import get_logger from celery.utils.log import get_logger
from redis.asyncio.lock import Lock
from sqlalchemy import select from sqlalchemy import select
from sqlalchemy.exc import DatabaseError, InterfaceError from sqlalchemy.exc import DatabaseError, InterfaceError
@@ -28,6 +30,9 @@ from backend.utils.timezone import timezone
# 此计划程序必须比常规的 5 分钟更频繁地唤醒,因为它需要考虑对计划的外部更改 # 此计划程序必须比常规的 5 分钟更频繁地唤醒,因为它需要考虑对计划的外部更改
DEFAULT_MAX_INTERVAL = 5 # seconds DEFAULT_MAX_INTERVAL = 5 # seconds
# 计划锁时长,避免重复创建
DEFAULT_MAX_LOCK_TIMEOUT = DEFAULT_MAX_INTERVAL * 5 # seconds
logger = get_logger('fba.schedulers') logger = get_logger('fba.schedulers')
@@ -188,21 +193,12 @@ class ModelEntry(ScheduleEntry):
if not obj: if not obj:
obj = TaskScheduler(**CreateTaskSchedulerParam(task=task, **spec).model_dump()) obj = TaskScheduler(**CreateTaskSchedulerParam(task=task, **spec).model_dump())
elif isinstance(schedule, schedules.crontab): elif isinstance(schedule, schedules.crontab):
crontab_minute = schedule._orig_minute if crontab_verify('m', schedule._orig_minute, False) else '*' crontab = f'{schedule._orig_minute} {schedule._orig_hour} {schedule._orig_day_of_week} {schedule._orig_day_of_month} {schedule._orig_month_of_year}' # noqa: E501
crontab_hour = schedule._orig_hour if crontab_verify('h', schedule._orig_hour, False) else '*' crontab_verify(crontab)
crontab_day_of_week = (
schedule._orig_day_of_week if crontab_verify('dom', schedule._orig_day_of_week, False) else '*'
)
crontab_day_of_month = (
schedule._orig_day_of_month if crontab_verify('dom', schedule._orig_day_of_month, False) else '*'
)
crontab_month_of_year = (
schedule._orig_month_of_year if crontab_verify('moy', schedule._orig_month_of_year, False) else '*'
)
spec = { spec = {
'name': name, 'name': name,
'type': TaskSchedulerType.CRONTAB.value, 'type': TaskSchedulerType.CRONTAB.value,
'crontab': f'{crontab_minute} {crontab_hour} {crontab_day_of_week} {crontab_day_of_month} {crontab_month_of_year}', # noqa: E501 'crontab': crontab,
} }
stmt = select(TaskScheduler).filter_by(**spec) stmt = select(TaskScheduler).filter_by(**spec)
query = await db.execute(stmt) query = await db.execute(stmt)
@@ -233,8 +229,8 @@ class ModelEntry(ScheduleEntry):
except KeyError: except KeyError:
continue continue
model_dict.update( model_dict.update(
args=json.dumps(args) if args else None, args=json.dumps(args, ensure_ascii=False) if args else None,
kwargs=json.dumps(kwargs) if kwargs else None, kwargs=json.dumps(kwargs, ensure_ascii=False) if kwargs else None,
**cls._unpack_options(**options or {}), **cls._unpack_options(**options or {}),
**entry, **entry,
) )
@@ -269,6 +265,8 @@ class ModelEntry(ScheduleEntry):
class DatabaseScheduler(Scheduler): class DatabaseScheduler(Scheduler):
"""数据库调度程序"""
Entry = ModelEntry Entry = ModelEntry
_schedule = None _schedule = None
@@ -276,6 +274,9 @@ class DatabaseScheduler(Scheduler):
_initial_read = True _initial_read = True
_heap_invalidated = False _heap_invalidated = False
lock: Lock | None = None
lock_key = f'{settings.CELERY_REDIS_PREFIX}:beat_lock'
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
self.app = kwargs['app'] self.app = kwargs['app']
self._dirty = set() self._dirty = set()
@@ -283,39 +284,26 @@ class DatabaseScheduler(Scheduler):
self._finalize = Finalize(self, self.sync, exitpriority=5) self._finalize = Finalize(self, self.sync, exitpriority=5)
self.max_interval = kwargs.get('max_interval') or self.app.conf.beat_max_loop_interval or DEFAULT_MAX_INTERVAL self.max_interval = kwargs.get('max_interval') or self.app.conf.beat_max_loop_interval or DEFAULT_MAX_INTERVAL
def setup_schedule(self): def install_default_entries(self, data):
"""重写父函数""" """重写父函数"""
logger.info('setup_schedule') entries = {}
tasks = self.schedule if self.app.conf.result_expires:
self.install_default_entries(tasks) entries.setdefault(
self.update_from_dict(self.app.conf.beat_schedule) 'celery.backend_cleanup',
{
'task': 'celery.backend_cleanup',
'schedule': schedules.crontab('0', '4', '*'),
'options': {'expire_seconds': 12 * 3600},
},
)
self.update_from_dict(entries)
async def get_all_task_schedulers(self): def schedules_equal(self, *args, **kwargs):
"""获取所有任务调度""" """重写父函数"""
async with async_db_session() as db: if self._heap_invalidated:
logger.debug('DatabaseScheduler: Fetching database schedule') self._heap_invalidated = False
stmt = select(TaskScheduler).where(TaskScheduler.enabled == 1)
query = await db.execute(stmt)
tasks = query.scalars().all()
s = {}
for task in tasks:
s[task.name] = self.Entry(task, app=self.app)
return s
def schedule_changed(self) -> bool:
"""任务调度变更状态"""
now = timezone.now()
last_update = run_await(redis_client.get)(f'{settings.CELERY_REDIS_PREFIX}:last_update')
if not last_update:
run_await(redis_client.set)(f'{settings.CELERY_REDIS_PREFIX}:last_update', timezone.to_str(now))
return False return False
return super().schedules_equal(*args, **kwargs)
last, ts = self._last_update, timezone.from_str(last_update)
try:
if ts and ts > (last if last else ts):
return True
finally:
self._last_update = now
def reserve(self, entry): def reserve(self, entry):
"""重写父函数""" """重写父函数"""
@@ -324,6 +312,13 @@ class DatabaseScheduler(Scheduler):
self._dirty.add(new_entry.name) self._dirty.add(new_entry.name)
return new_entry return new_entry
def setup_schedule(self):
"""重写父函数"""
logger.info('setup_schedule')
tasks = self.schedule
self.install_default_entries(tasks)
self.update_from_dict(self.app.conf.beat_schedule)
def sync(self): def sync(self):
"""重写父函数""" """重写父函数"""
_tried = set() _tried = set()
@@ -347,6 +342,25 @@ class DatabaseScheduler(Scheduler):
# 请稍后重试(仅针对失败的) # 请稍后重试(仅针对失败的)
self._dirty |= _failed self._dirty |= _failed
def tick(self, **kwargs):
"""重写父函数"""
if self.lock:
logger.debug('beat: Extending lock...')
run_await(self.lock.extend)(DEFAULT_MAX_LOCK_TIMEOUT, replace_ttl=True)
result = super().tick(**kwargs)
return result
def close(self):
"""重写父函数"""
if self.lock:
logger.info('beat: Releasing lock')
if run_await(self.lock.owned)():
run_await(self.lock.release)()
self.lock = None
super().close()
def update_from_dict(self, beat_dict: dict): def update_from_dict(self, beat_dict: dict):
"""重写父函数""" """重写父函数"""
s = {} s = {}
@@ -362,26 +376,32 @@ class DatabaseScheduler(Scheduler):
tasks = self.schedule tasks = self.schedule
tasks.update(s) tasks.update(s)
def install_default_entries(self, data): def schedule_changed(self) -> bool:
"""重写父函数""" """任务调度变更状态"""
entries = {} now = timezone.now()
if self.app.conf.result_expires: last_update = run_await(redis_client.get)(f'{settings.CELERY_REDIS_PREFIX}:last_update')
entries.setdefault( if not last_update:
'celery.backend_cleanup', run_await(redis_client.set)(f'{settings.CELERY_REDIS_PREFIX}:last_update', timezone.to_str(now))
{
'task': 'celery.backend_cleanup',
'schedule': schedules.crontab('0', '4', '*'),
'options': {'expire_seconds': 12 * 3600},
},
)
self.update_from_dict(entries)
def schedules_equal(self, *args, **kwargs):
"""重写父函数"""
if self._heap_invalidated:
self._heap_invalidated = False
return False return False
return super().schedules_equal(*args, **kwargs)
last, ts = self._last_update, timezone.from_str(last_update)
try:
if ts and ts > (last if last else ts):
return True
finally:
self._last_update = now
async def get_all_task_schedulers(self):
"""获取所有任务调度"""
async with async_db_session() as db:
logger.debug('DatabaseScheduler: Fetching database schedule')
stmt = select(TaskScheduler).where(TaskScheduler.enabled == 1)
query = await db.execute(stmt)
tasks = query.scalars().all()
s = {}
for task in tasks:
s[task.name] = self.Entry(task, app=self.app)
return s
@property @property
def schedule(self) -> dict[str, ModelEntry]: def schedule(self) -> dict[str, ModelEntry]:
@@ -410,3 +430,27 @@ class DatabaseScheduler(Scheduler):
# logger.debug(self._schedule) # logger.debug(self._schedule)
return self._schedule return self._schedule
@beat_init.connect
def acquire_distributed_beat_lock(sender=None, **kwargs):
"""
尝试在启动时获取锁
:param sender: 接收方应响应的发送方
:return:
"""
scheduler = sender.scheduler
if not scheduler.lock_key:
return
logger.debug('beat: Acquiring lock...')
lock = redis_client.lock(
scheduler.lock_key,
timeout=DEFAULT_MAX_LOCK_TIMEOUT,
sleep=scheduler.max_interval,
)
run_await(lock.acquire)()
logger.info('beat: Acquired lock')
scheduler.lock = lock
+8 -26
View File
@@ -1,10 +1,9 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from datetime import datetime from datetime import datetime
from typing import Literal
from celery import schedules from celery import schedules
from celery.schedules import ParseException, crontab_parser from celery.schedules import ParseException, crontab
from backend.common.exception import errors from backend.common.exception import errors
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
@@ -53,34 +52,17 @@ class TzAwareCrontab(schedules.crontab):
) )
def crontab_verify(filed: Literal['m', 'h', 'dow', 'dom', 'moy'], value: str, raise_exc: bool = True) -> bool: def crontab_verify(crontab_str: str) -> None:
""" """
验证 Celery crontab 表达式 验证 Celery crontab 表达式
:param filed: 验证的字段 :param crontab_str: 计划表达式
:param value: 验证的值
:param raise_exc: 是否抛出异常
:return: :return:
""" """
valid = True crontab_split = crontab_str.split(' ')
if len(crontab_split) != 5:
raise errors.RequestError(msg='Crontab 表达式非法')
try: try:
match filed: crontab(*crontab_split)
case 'm':
crontab_parser(60, 0).parse(value)
case 'h':
crontab_parser(24, 0).parse(value)
case 'dow':
crontab_parser(7, 0).parse(value)
case 'dom':
crontab_parser(31, 1).parse(value)
case 'moy':
crontab_parser(12, 1).parse(value)
case _:
raise errors.ServerError(msg=f'无效字段:{filed}')
except ParseException: except ParseException:
valid = False raise errors.RequestError(msg='Crontab 表达式非法')
if raise_exc:
raise errors.RequestError(msg=f'crontab 值 {value} 非法')
return valid
+137 -41
View File
@@ -9,25 +9,42 @@ from typing import Annotated, Literal
import cappa import cappa
import granian import granian
from cappa.output import error_format
from rich.panel import Panel from rich.panel import Panel
from rich.prompt import IntPrompt
from rich.table import Table
from rich.text import Text from rich.text import Text
from sqlalchemy import text from sqlalchemy import text
from watchfiles import PythonFilter from watchfiles import PythonFilter
from backend import console, get_version from backend import __version__
from backend.common.enums import DataBaseType, PrimaryKeyType from backend.common.enums import DataBaseType, PrimaryKeyType
from backend.common.exception.errors import BaseExceptionMixin from backend.common.exception.errors import BaseExceptionMixin
from backend.core.conf import settings from backend.core.conf import settings
from backend.database.db import async_db_session from backend.database.db import async_db_session
from backend.plugin.code_generator.schema.code import ImportParam
from backend.plugin.code_generator.service.business_service import gen_business_service
from backend.plugin.code_generator.service.code_service import gen_service
from backend.plugin.tools import get_plugin_sql from backend.plugin.tools import get_plugin_sql
from backend.utils._await import run_await
from backend.utils.console import console
from backend.utils.file_ops import install_git_plugin, install_zip_plugin, parse_sql_script from backend.utils.file_ops import install_git_plugin, install_zip_plugin, parse_sql_script
output_help = '\n更多信息,尝试 "[cyan]--help[/]"'
def run(host: str, port: int, reload: bool, workers: int | None) -> None:
class CustomReloadFilter(PythonFilter):
"""自定义重载过滤器"""
def __init__(self):
super().__init__(extra_extensions=['.json', '.yaml', '.yml'])
def run(host: str, port: int, reload: bool, workers: int) -> None:
url = f'http://{host}:{port}' url = f'http://{host}:{port}'
docs_url = url + settings.FASTAPI_DOCS_URL docs_url = url + settings.FASTAPI_DOCS_URL
redoc_url = url + settings.FASTAPI_REDOC_URL redoc_url = url + settings.FASTAPI_REDOC_URL
openapi_url = url + settings.FASTAPI_OPENAPI_URL openapi_url = url + (settings.FASTAPI_OPENAPI_URL or '')
panel_content = Text() panel_content = Text()
panel_content.append(f'📝 Swagger 文档: {docs_url}\n', style='blue') panel_content.append(f'📝 Swagger 文档: {docs_url}\n', style='blue')
@@ -45,8 +62,8 @@ def run(host: str, port: int, reload: bool, workers: int | None) -> None:
address=host, address=host,
port=port, port=port,
reload=not reload, reload=not reload,
reload_filter=PythonFilter, reload_filter=CustomReloadFilter,
workers=workers or 1, workers=workers,
).serve() ).serve()
@@ -118,13 +135,62 @@ async def execute_sql_scripts(sql_scripts: str) -> None:
console.print(Text('SQL 脚本已执行完成', style='bold green')) console.print(Text('SQL 脚本已执行完成', style='bold green'))
@cappa.command(help='运行 API 服务') async def import_table(
app: str,
table_schema: str,
table_name: str,
) -> None:
try:
obj = ImportParam(app=app, table_schema=table_schema, table_name=table_name)
await gen_service.import_business_and_model(obj=obj)
except Exception as e:
raise cappa.Exit(e.msg if isinstance(e, BaseExceptionMixin) else str(e), code=1)
def generate(gen: bool) -> None:
if not gen:
console.print(output_help)
return
try:
ids = []
results = run_await(gen_business_service.get_all)()
if not results:
raise cappa.Exit('[red]暂无可用的代码生成业务!请先通过 import 命令导入![/]')
table = Table(show_header=True, header_style='bold magenta')
table.add_column('业务编号', style='cyan', no_wrap=True, justify='center')
table.add_column('应用名称', style='green', no_wrap=True)
table.add_column('生成路径', style='yellow')
table.add_column('备注', style='blue')
for result in results:
ids.append(result.id)
table.add_row(
str(result.id),
result.app_name,
result.gen_path or f'应用 {result.app_name} 根路径',
result.remark or '',
)
console.print(table)
business = IntPrompt.ask('请从中选择一个业务编号', choices=[str(_id) for _id in ids])
gen_path = run_await(gen_service.generate)(pk=business)
except Exception as e:
raise cappa.Exit(e.msg if isinstance(e, BaseExceptionMixin) else str(e), code=1)
console.print(Text('\n代码已生成完毕', style='bold green'))
console.print(Text('\n详情请查看:'), Text(gen_path, style='bold magenta'))
@cappa.command(help='运行 API 服务', default_long=True)
@dataclass @dataclass
class Run: class Run:
host: Annotated[ host: Annotated[
str, str,
cappa.Arg( cappa.Arg(
long=True,
default='127.0.0.1', default='127.0.0.1',
help='提供服务的主机 IP 地址,对于本地开发,请使用 `127.0.0.1`。' help='提供服务的主机 IP 地址,对于本地开发,请使用 `127.0.0.1`。'
'要启用公共访问,例如在局域网中,请使用 `0.0.0.0`', '要启用公共访问,例如在局域网中,请使用 `0.0.0.0`',
@@ -132,50 +198,56 @@ class Run:
] ]
port: Annotated[ port: Annotated[
int, int,
cappa.Arg(long=True, default=8000, help='提供服务的主机端口号'), cappa.Arg(default=8000, help='提供服务的主机端口号'),
] ]
no_reload: Annotated[ no_reload: Annotated[
bool, bool,
cappa.Arg(long=True, default=False, help='禁用在(代码)文件更改时自动重新加载服务器'), cappa.Arg(default=False, help='禁用在(代码)文件更改时自动重新加载服务器'),
] ]
workers: Annotated[ workers: Annotated[
int | None, int,
cappa.Arg(long=True, default=None, help='使用多个工作进程,必须与 `--no-reload` 同时使用'), cappa.Arg(default=1, help='使用多个工作进程,必须与 `--no-reload` 同时使用'),
] ]
def __call__(self): def __call__(self):
run(host=self.host, port=self.port, reload=self.no_reload, workers=self.workers) run(host=self.host, port=self.port, reload=self.no_reload, workers=self.workers)
@cappa.command(help='从当前主机启动 Celery worker 服务') @cappa.command(help='从当前主机启动 Celery worker 服务', default_long=True)
@dataclass @dataclass
class Worker: class Worker:
log_level: Annotated[ log_level: Annotated[
Literal['info', 'debug'], Literal['info', 'debug'],
cappa.Arg(long=True, short='-l', default='info', help='日志输出级别'), cappa.Arg(short='-l', default='info', help='日志输出级别'),
] ]
def __call__(self): def __call__(self):
run_celery_worker(log_level=self.log_level) run_celery_worker(log_level=self.log_level)
@cappa.command(help='从当前主机启动 Celery beat 服务') @cappa.command(help='从当前主机启动 Celery beat 服务', default_long=True)
@dataclass @dataclass
class Beat: class Beat:
log_level: Annotated[ log_level: Annotated[
Literal['info', 'debug'], Literal['info', 'debug'],
cappa.Arg(long=True, short='-l', default='info', help='日志输出级别'), cappa.Arg(short='-l', default='info', help='日志输出级别'),
] ]
def __call__(self): def __call__(self):
run_celery_beat(log_level=self.log_level) run_celery_beat(log_level=self.log_level)
@cappa.command(help='从当前主机启动 Celery flower 服务') @cappa.command(help='从当前主机启动 Celery flower 服务', default_long=True)
@dataclass @dataclass
class Flower: class Flower:
port: Annotated[int, cappa.Arg(long=True, default=8555, help='提供服务的主机端口号')] port: Annotated[
basic_auth: Annotated[str, cappa.Arg(long=True, default='admin:123456', help='页面登录的用户名和密码')] int,
cappa.Arg(default=8555, help='提供服务的主机端口号'),
]
basic_auth: Annotated[
str,
cappa.Arg(default='admin:123456', help='页面登录的用户名和密码'),
]
def __call__(self): def __call__(self):
run_celery_flower(port=self.port, basic_auth=self.basic_auth) run_celery_flower(port=self.port, basic_auth=self.basic_auth)
@@ -184,60 +256,84 @@ class Flower:
@cappa.command(help='运行 Celery 服务') @cappa.command(help='运行 Celery 服务')
@dataclass @dataclass
class Celery: class Celery:
subcmd: cappa.Subcommands[Worker | Beat | Flower | None] = None subcmd: cappa.Subcommands[Worker | Beat | Flower]
def __call__(self):
console.print('\n更多信息,尝试 "[cyan]--help[/]"')
@cappa.command(help='新增插件') @cappa.command(help='新增插件', default_long=True)
@dataclass @dataclass
class Add: class Add:
path: Annotated[ path: Annotated[
str | None, str | None,
cappa.Arg(long=True, help='ZIP 插件的本地完整路径'), cappa.Arg(help='ZIP 插件的本地完整路径'),
] ]
repo_url: Annotated[ repo_url: Annotated[
str | None, str | None,
cappa.Arg(long=True, help='Git 插件的仓库地址'), cappa.Arg(help='Git 插件的仓库地址'),
] ]
no_sql: Annotated[ no_sql: Annotated[
bool, bool,
cappa.Arg(long=True, default=False, help='禁用插件 SQL 脚本自动执行'), cappa.Arg(default=False, help='禁用插件 SQL 脚本自动执行'),
] ]
db_type: Annotated[ db_type: Annotated[
DataBaseType, DataBaseType,
cappa.Arg(long=True, default='mysql', help='执行插件 SQL 脚本的数据库类型'), cappa.Arg(default='mysql', help='执行插件 SQL 脚本的数据库类型'),
] ]
pk_type: Annotated[ pk_type: Annotated[
PrimaryKeyType, PrimaryKeyType,
cappa.Arg(long=True, default='autoincrement', help='执行插件 SQL 脚本数据库主键类型'), cappa.Arg(default='autoincrement', help='执行插件 SQL 脚本数据库主键类型'),
] ]
async def __call__(self): async def __call__(self):
await install_plugin(self.path, self.repo_url, self.no_sql, self.db_type, self.pk_type) await install_plugin(self.path, self.repo_url, self.no_sql, self.db_type, self.pk_type)
@cappa.command(help='一个高效的 fba 命令行界面') @cappa.command(help='导入代码生成业务和模型列', default_long=True)
@dataclass @dataclass
class FbaCli: class Import:
version: Annotated[ app: Annotated[
bool,
cappa.Arg(short='-V', long=True, default=False, show_default=False, help='打印当前版本号'),
]
sql: Annotated[
str, str,
cappa.Arg(value_name='PATH', long=True, default='', show_default=False, help='在事务中执行 SQL 脚本'), cappa.Arg(help='应用名称,用于代码生成到指定 app'),
]
table_schema: Annotated[
str,
cappa.Arg(short='tc', default='fba', help='数据库名'),
]
table_name: Annotated[
str,
cappa.Arg(short='tn', help='数据库表名'),
] ]
subcmd: cappa.Subcommands[Run | Celery | Add | None] = None
async def __call__(self): async def __call__(self):
if self.version: await import_table(self.app, self.table_schema, self.table_name)
get_version()
@cappa.command(name='codegen', help='代码生成(体验完整功能,请自行部署 fba vben 前端工程)', default_long=True)
@dataclass
class CodeGenerate:
gen: Annotated[
bool,
cappa.Arg(default=False, show_default=False, help='执行代码生成'),
]
subcmd: cappa.Subcommands[Import | None] = None
def __call__(self):
generate(self.gen)
@cappa.command(help='一个高效的 fba 命令行界面', default_long=True)
@dataclass
class FbaCli:
sql: Annotated[
str,
cappa.Arg(value_name='PATH', default='', show_default=False, help='在事务中执行 SQL 脚本'),
]
subcmd: cappa.Subcommands[Run | Celery | Add | CodeGenerate | None] = None
async def __call__(self):
if self.sql: if self.sql:
await execute_sql_scripts(self.sql) await execute_sql_scripts(self.sql)
def main() -> None: def main() -> None:
output = cappa.Output(error_format='[red]Error[/]: {message}\n\n更多信息,尝试 "[cyan]--help[/]"') output = cappa.Output(error_format=f'{error_format}\n{output_help}')
asyncio.run(cappa.invoke_async(FbaCli, output=output)) asyncio.run(cappa.invoke_async(FbaCli, version=__version__, output=output))
+1
View File
@@ -113,6 +113,7 @@ class UserSocialType(StrEnum):
"""用户社交类型""" """用户社交类型"""
github = 'GitHub' github = 'GitHub'
google = 'Google'
linux_do = 'LinuxDo' linux_do = 'LinuxDo'
+9 -3
View File
@@ -38,9 +38,15 @@ class CustomError(BaseExceptionMixin):
class RequestError(BaseExceptionMixin): class RequestError(BaseExceptionMixin):
"""请求异常""" """请求异常"""
code = StandardResponseCode.HTTP_400 def __init__(
self,
def __init__(self, *, msg: str = 'Bad Request', data: Any = None, background: BackgroundTask | None = None): *,
code: int = StandardResponseCode.HTTP_400,
msg: str = 'Bad Request',
data: Any = None,
background: BackgroundTask | None = None,
):
self.code = code
super().__init__(msg=msg, data=data, background=background) super().__init__(msg=msg, data=data, background=background)
+15 -74
View File
@@ -4,15 +4,12 @@ from fastapi import FastAPI, Request
from fastapi.exceptions import RequestValidationError from fastapi.exceptions import RequestValidationError
from pydantic import ValidationError from pydantic import ValidationError
from starlette.exceptions import HTTPException from starlette.exceptions import HTTPException
from starlette.middleware.cors import CORSMiddleware
from uvicorn.protocols.http.h11_impl import STATUS_PHRASES from uvicorn.protocols.http.h11_impl import STATUS_PHRASES
from backend.common.exception.errors import BaseExceptionMixin from backend.common.exception.errors import BaseExceptionMixin
from backend.common.i18n import i18n, t
from backend.common.response.response_code import CustomResponseCode, StandardResponseCode from backend.common.response.response_code import CustomResponseCode, StandardResponseCode
from backend.common.response.response_schema import response_base from backend.common.response.response_schema import response_base
from backend.common.schema import (
CUSTOM_VALIDATION_ERROR_MESSAGES,
)
from backend.core.conf import settings from backend.core.conf import settings
from backend.utils.serializers import MsgSpecJSONResponse from backend.utils.serializers import MsgSpecJSONResponse
from backend.utils.trace_id import get_request_trace_id from backend.utils.trace_id import get_request_trace_id
@@ -46,18 +43,20 @@ async def _validation_exception_handler(request: Request, exc: RequestValidation
""" """
errors = [] errors = []
for error in exc.errors(): for error in exc.errors():
custom_message = CUSTOM_VALIDATION_ERROR_MESSAGES.get(error['type']) # 非 en-US 语言下,使用自定义错误信息
if custom_message: if i18n.current_language != 'en-US':
ctx = error.get('ctx') custom_message = t(f'pydantic.{error["type"]}')
if not ctx: if custom_message:
error['msg'] = custom_message ctx = error.get('ctx')
else: if not ctx:
error['msg'] = custom_message.format(**ctx) error['msg'] = custom_message
ctx_error = ctx.get('error') else:
if ctx_error: ctx_error = ctx.get('error')
error['ctx']['error'] = ( if ctx_error:
ctx_error.__str__().replace("'", '"') if isinstance(ctx_error, Exception) else None error['msg'] = custom_message.format(**ctx)
) error['ctx']['error'] = (
ctx_error.__str__().replace("'", '"') if isinstance(ctx_error, Exception) else None
)
errors.append(error) errors.append(error)
error = errors[0] error = errors[0]
if error.get('type') == 'json_invalid': if error.get('type') == 'json_invalid':
@@ -193,66 +192,8 @@ def register_exception(app: FastAPI):
else: else:
res = response_base.fail(res=CustomResponseCode.HTTP_500) res = response_base.fail(res=CustomResponseCode.HTTP_500)
content = res.model_dump() content = res.model_dump()
request.state.__request_all_unknown_exception__ = content
content.update(trace_id=get_request_trace_id(request)) content.update(trace_id=get_request_trace_id(request))
return MsgSpecJSONResponse( return MsgSpecJSONResponse(
status_code=StandardResponseCode.HTTP_500, status_code=StandardResponseCode.HTTP_500,
content=content, content=content,
) )
if settings.MIDDLEWARE_CORS:
@app.exception_handler(StandardResponseCode.HTTP_500)
async def cors_custom_code_500_exception_handler(request, exc):
"""
跨域自定义 500 异常处理
`Related issue <https://github.com/encode/starlette/issues/1175>`_
`Solution <https://github.com/fastapi/fastapi/discussions/7847#discussioncomment-5144709>`_
:param request: FastAPI 请求对象
:param exc: 自定义异常
:return:
"""
if isinstance(exc, BaseExceptionMixin):
content = {
'code': exc.code,
'msg': exc.msg,
'data': exc.data,
}
else:
if settings.ENVIRONMENT == 'dev':
content = {
'code': StandardResponseCode.HTTP_500,
'msg': str(exc),
'data': None,
}
else:
res = response_base.fail(res=CustomResponseCode.HTTP_500)
content = res.model_dump()
request.state.__request_cors_500_exception__ = content
content.update(trace_id=get_request_trace_id(request))
response = MsgSpecJSONResponse(
status_code=exc.code if isinstance(exc, BaseExceptionMixin) else StandardResponseCode.HTTP_500,
content=content,
background=exc.background if isinstance(exc, BaseExceptionMixin) else None,
)
origin = request.headers.get('origin')
if origin:
cors = CORSMiddleware(
app=app,
allow_origins=settings.CORS_ALLOWED_ORIGINS,
allow_credentials=True,
allow_methods=['*'],
allow_headers=['*'],
expose_headers=settings.CORS_EXPOSE_HEADERS,
)
response.headers.update(cors.simple_headers)
has_cookie = 'cookie' in request.headers
if cors.allow_all_origins and has_cookie:
response.headers['Access-Control-Allow-Origin'] = origin
elif not cors.allow_all_origins and cors.is_allowed_origin(origin=origin):
response.headers['Access-Control-Allow-Origin'] = origin
response.headers.add_vary_header('Origin')
return response
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import glob
import json
import os
from pathlib import Path
from typing import Any
import yaml
from backend.core.conf import settings
from backend.core.path_conf import LOCALE_DIR
class I18n:
"""国际化管理器"""
def __init__(self):
self.locales: dict[str, dict[str, Any]] = {}
self.current_language: str = settings.I18N_DEFAULT_LANGUAGE
def load_locales(self):
"""加载语言文本"""
patterns = [
os.path.join(LOCALE_DIR, '*.json'),
os.path.join(LOCALE_DIR, '*.yaml'),
os.path.join(LOCALE_DIR, '*.yml'),
]
lang_files = []
for pattern in patterns:
lang_files.extend(glob.glob(pattern))
for lang_file in lang_files:
with open(lang_file, 'r', encoding='utf-8') as f:
lang = Path(lang_file).stem
file_type = Path(lang_file).suffix[1:]
match file_type:
case 'json':
self.locales[lang] = json.loads(f.read())
case 'yaml' | 'yml':
self.locales[lang] = yaml.full_load(f.read())
def t(self, key: str, default: Any | None = None, **kwargs) -> str:
"""
翻译函数
:param key: 目标文本键支持点分隔例如 'response.success'
:param default: 目标语言文本不存在时的默认文本
:param kwargs: 目标文本中的变量参数
:return:
"""
keys = key.split('.')
try:
translation = self.locales[self.current_language]
except KeyError:
keys = 'error.language_not_found'
translation = self.locales[settings.I18N_DEFAULT_LANGUAGE]
for k in keys:
if isinstance(translation, dict) and k in list(translation.keys()):
translation = translation[k]
else:
# Pydantic 兼容
if keys[0] == 'pydantic':
translation = None
else:
translation = key
if translation and kwargs:
translation = translation.format(**kwargs)
return translation or default
# 创建 i18n 单例
i18n = I18n()
# 创建翻译函数实例
t = i18n.t
+24 -7
View File
@@ -3,6 +3,7 @@
import inspect import inspect
import logging import logging
import os import os
import re
import sys import sys
from asgi_correlation_id import correlation_id from asgi_correlation_id import correlation_id
@@ -36,6 +37,18 @@ class InterceptHandler(logging.Handler):
logger.opt(depth=depth, exception=record.exc_info).log(level, record.getMessage()) logger.opt(depth=depth, exception=record.exc_info).log(level, record.getMessage())
def default_formatter(record):
"""默认日志格式化程序"""
# 重写 sqlalchemy echo 输出
# https://github.com/sqlalchemy/sqlalchemy/discussions/12791
record_name = record['name'] or ''
if record_name.startswith('sqlalchemy'):
record['message'] = re.sub(r'\s+', ' ', record['message']).strip()
return settings.LOG_FORMAT if settings.LOG_FORMAT.endswith('\n') else f'{settings.LOG_FORMAT}\n'
def setup_logging() -> None: def setup_logging() -> None:
""" """
设置日志处理器 设置日志处理器
@@ -48,9 +61,11 @@ def setup_logging() -> None:
logging.root.handlers = [InterceptHandler()] logging.root.handlers = [InterceptHandler()]
logging.root.setLevel(settings.LOG_STD_LEVEL) logging.root.setLevel(settings.LOG_STD_LEVEL)
# 配置日志传播规则
for name in logging.root.manager.loggerDict.keys(): for name in logging.root.manager.loggerDict.keys():
# 清空所有默认日志处理器
logging.getLogger(name).handlers = [] logging.getLogger(name).handlers = []
# 配置日志传播规则
if 'uvicorn.access' in name or 'watchfiles.main' in name: if 'uvicorn.access' in name or 'watchfiles.main' in name:
logging.getLogger(name).propagate = False logging.getLogger(name).propagate = False
else: else:
@@ -59,22 +74,24 @@ def setup_logging() -> None:
# Debug log handlers # Debug log handlers
# logging.debug(f'{logging.getLogger(name)}, {logging.getLogger(name).propagate}') # logging.debug(f'{logging.getLogger(name)}, {logging.getLogger(name).propagate}')
# 移除 loguru 默认处理器
logger.remove()
# correlation_id 过滤器 # correlation_id 过滤器
# https://github.com/snok/asgi-correlation-id/issues/7 # https://github.com/snok/asgi-correlation-id/issues/7
def correlation_id_filter(record): def correlation_id_filter(record):
cid = correlation_id.get(settings.TRACE_ID_LOG_DEFAULT_VALUE) cid = correlation_id.get(settings.TRACE_ID_LOG_DEFAULT_VALUE)
record['correlation_id'] = cid[: settings.TRACE_ID_LOG_UUID_LENGTH] record['correlation_id'] = cid[: settings.TRACE_ID_LOG_LENGTH]
return record return record
# 配置 loguru 处理器 # 配置 loguru 处理器
logger.remove() # 移除默认处理器
logger.configure( logger.configure(
handlers=[ handlers=[
{ {
'sink': sys.stdout, 'sink': sys.stdout,
'level': settings.LOG_STD_LEVEL, 'level': settings.LOG_STD_LEVEL,
'format': default_formatter,
'filter': lambda record: correlation_id_filter(record), 'filter': lambda record: correlation_id_filter(record),
'format': settings.LOG_STD_FORMAT,
} }
] ]
) )
@@ -100,7 +117,7 @@ def set_custom_logfile() -> None:
# 日志文件通用配置 # 日志文件通用配置
# https://loguru.readthedocs.io/en/stable/api/logger.html#loguru._logger.Logger.add # https://loguru.readthedocs.io/en/stable/api/logger.html#loguru._logger.Logger.add
log_config = { log_config = {
'format': settings.LOG_FILE_FORMAT, 'format': default_formatter,
'enqueue': True, 'enqueue': True,
'rotation': '00:00', 'rotation': '00:00',
'retention': '7 days', 'retention': '7 days',
@@ -110,7 +127,7 @@ def set_custom_logfile() -> None:
# 标准输出文件 # 标准输出文件
logger.add( logger.add(
str(log_access_file), str(log_access_file),
level=settings.LOG_ACCESS_FILE_LEVEL, level=settings.LOG_FILE_ACCESS_LEVEL,
filter=lambda record: record['level'].no <= 25, filter=lambda record: record['level'].no <= 25,
backtrace=False, backtrace=False,
diagnose=False, diagnose=False,
@@ -120,7 +137,7 @@ def set_custom_logfile() -> None:
# 标准错误文件 # 标准错误文件
logger.add( logger.add(
str(log_error_file), str(log_error_file),
level=settings.LOG_ERROR_FILE_LEVEL, level=settings.LOG_FILE_ERROR_LEVEL,
filter=lambda record: record['level'].no >= 30, filter=lambda record: record['level'].no >= 30,
backtrace=True, backtrace=True,
diagnose=True, diagnose=True,
+27 -3
View File
@@ -3,7 +3,7 @@
from datetime import datetime from datetime import datetime
from typing import Annotated from typing import Annotated
from sqlalchemy import BigInteger, DateTime from sqlalchemy import BigInteger, DateTime, TypeDecorator
from sqlalchemy.ext.asyncio import AsyncAttrs from sqlalchemy.ext.asyncio import AsyncAttrs
from sqlalchemy.orm import DeclarativeBase, Mapped, MappedAsDataclass, declared_attr, mapped_column from sqlalchemy.orm import DeclarativeBase, Mapped, MappedAsDataclass, declared_attr, mapped_column
@@ -51,14 +51,38 @@ class UserMixin(MappedAsDataclass):
updated_by: Mapped[int | None] = mapped_column(init=False, default=None, sort_order=998, comment='修改者') updated_by: Mapped[int | None] = mapped_column(init=False, default=None, sort_order=998, comment='修改者')
class TimeZone(TypeDecorator[datetime]):
"""时区感知 DateTime"""
impl = DateTime(timezone=True)
cache_ok = True
@property
def python_type(self) -> type[datetime]:
return datetime
def process_bind_param(self, value: datetime | None, dialect) -> datetime | None:
if value is not None:
# TODO 处理夏令时偏移
if value.utcoffset() != timezone.now().utcoffset():
value = timezone.from_datetime(value)
return value
def process_result_value(self, value: datetime | None, dialect) -> datetime | None:
if value is not None:
if value.tzinfo is None:
value = value.replace(tzinfo=timezone.tz_info)
return value
class DateTimeMixin(MappedAsDataclass): class DateTimeMixin(MappedAsDataclass):
"""日期时间 Mixin 数据类""" """日期时间 Mixin 数据类"""
created_time: Mapped[datetime] = mapped_column( created_time: Mapped[datetime] = mapped_column(
DateTime(timezone=True), init=False, default_factory=timezone.now, sort_order=999, comment='创建时间' TimeZone, init=False, default_factory=timezone.now, sort_order=999, comment='创建时间'
) )
updated_time: Mapped[datetime | None] = mapped_column( updated_time: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), init=False, onupdate=timezone.now, sort_order=999, comment='更新时间' TimeZone, init=False, onupdate=timezone.now, sort_order=999, comment='更新时间'
) )
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import asyncio
from asyncio import Queue
async def batch_dequeue(queue: Queue, max_items: int, timeout: float) -> list:
"""
从异步队列中获取多个项目
:param queue: 用于获取项目的 `asyncio.Queue` 队列
:param max_items: 从队列中获取的最大项目数量
:param timeout: 总的等待超时时间
:return:
"""
items = []
async def collector():
while len(items) < max_items:
item = await queue.get()
items.append(item)
try:
await asyncio.wait_for(collector(), timeout=timeout)
except asyncio.TimeoutError:
pass
return items
+7 -4
View File
@@ -4,6 +4,8 @@ import dataclasses
from enum import Enum from enum import Enum
from backend.common.i18n import t
class CustomCodeBase(Enum): class CustomCodeBase(Enum):
"""自定义状态码基类""" """自定义状态码基类"""
@@ -16,21 +18,22 @@ class CustomCodeBase(Enum):
@property @property
def msg(self) -> str: def msg(self) -> str:
"""获取状态码信息""" """获取状态码信息"""
return self.value[1] message = self.value[1]
return t(message)
class CustomResponseCode(CustomCodeBase): class CustomResponseCode(CustomCodeBase):
"""自定义响应状态码""" """自定义响应状态码"""
HTTP_200 = (200, '请求成功') HTTP_200 = (200, 'response.success')
HTTP_400 = (400, '请求错误') HTTP_400 = (400, 'response.error')
HTTP_500 = (500, '服务器内部错误') HTTP_500 = (500, '服务器内部错误')
class CustomErrorCode(CustomCodeBase): class CustomErrorCode(CustomCodeBase):
"""自定义错误状态码""" """自定义错误状态码"""
CAPTCHA_ERROR = (40001, '验证码错误') CAPTCHA_ERROR = (40001, 'error.captcha.error')
@dataclasses.dataclass @dataclasses.dataclass
+6 -103
View File
@@ -5,108 +5,7 @@ from typing import Annotated
from pydantic import BaseModel, ConfigDict, EmailStr, Field, validate_email from pydantic import BaseModel, ConfigDict, EmailStr, Field, validate_email
from backend.core.conf import settings from backend.utils.timezone import timezone
# 自定义验证错误信息,参考:
# https://github.com/pydantic/pydantic-core/blob/a5cb7382643415b716b1a7a5392914e50f726528/tests/test_errors.py#L266
# https://github.com/pydantic/pydantic/blob/caa78016433ec9b16a973f92f187a7b6bfde6cb5/docs/errors/errors.md?plain=1#L232
CUSTOM_VALIDATION_ERROR_MESSAGES = {
'no_such_attribute': "对象没有属性 '{attribute}'",
'json_invalid': '无效的 JSON: {error}',
'json_type': 'JSON 输入应为字符串、字节或字节数组',
'recursion_loop': '递归错误 - 检测到循环引用',
'model_type': '输入应为有效的字典或 {class_name} 的实例',
'model_attributes_type': '输入应为有效的字典或可提取字段的对象',
'dataclass_exact_type': '输入应为 {class_name} 的实例',
'dataclass_type': '输入应为字典或 {class_name} 的实例',
'missing': '字段为必填项',
'frozen_field': '字段已冻结',
'frozen_instance': '实例已冻结',
'extra_forbidden': '不允许额外的输入',
'invalid_key': '键应为字符串',
'get_attribute_error': '提取属性时出错: {error}',
'none_required': '输入应为 None',
'enum': '输入应为 {expected}',
'greater_than': '输入应大于 {gt}',
'greater_than_equal': '输入应大于或等于 {ge}',
'less_than': '输入应小于 {lt}',
'less_than_equal': '输入应小于或等于 {le}',
'finite_number': '输入应为有限数字',
'too_short': '{field_type} 在验证后应至少有 {min_length} 个项目,而不是 {actual_length}',
'too_long': '{field_type} 在验证后最多应有 {max_length} 个项目,而不是 {actual_length}',
'string_type': '输入应为有效的字符串',
'string_sub_type': '输入应为字符串,而不是 str 子类的实例',
'string_unicode': '输入应为有效的字符串,无法将原始数据解析为 Unicode 字符串',
'string_pattern_mismatch': "字符串应匹配模式 '{pattern}'",
'string_too_short': '字符串应至少有 {min_length} 个字符',
'string_too_long': '字符串最多应有 {max_length} 个字符',
'dict_type': '输入应为有效的字典',
'mapping_type': '输入应为有效的映射,错误: {error}',
'iterable_type': '输入应为可迭代对象',
'iteration_error': '迭代对象时出错,错误: {error}',
'list_type': '输入应为有效的列表',
'tuple_type': '输入应为有效的元组',
'set_type': '输入应为有效的集合',
'bool_type': '输入应为有效的布尔值',
'bool_parsing': '输入应为有效的布尔值,无法解释输入',
'int_type': '输入应为有效的整数',
'int_parsing': '输入应为有效的整数,无法将字符串解析为整数',
'int_parsing_size': '无法将输入字符串解析为整数,超出最大大小',
'int_from_float': '输入应为有效的整数,得到一个带有小数部分的数字',
'multiple_of': '输入应为 {multiple_of} 的倍数',
'float_type': '输入应为有效的数字',
'float_parsing': '输入应为有效的数字,无法将字符串解析为数字',
'bytes_type': '输入应为有效的字节',
'bytes_too_short': '数据应至少有 {min_length} 个字节',
'bytes_too_long': '数据最多应有 {max_length} 个字节',
'value_error': '值错误,{error}',
'assertion_error': '断言失败,{error}',
'literal_error': '输入应为 {expected}',
'date_type': '输入应为有效的日期',
'date_parsing': '输入应为 YYYY-MM-DD 格式的有效日期,{error}',
'date_from_datetime_parsing': '输入应为有效的日期或日期时间,{error}',
'date_from_datetime_inexact': '提供给日期的日期时间应具有零时间 - 例如为精确日期',
'date_past': '日期应为过去的时间',
'date_future': '日期应为未来的时间',
'time_type': '输入应为有效的时间',
'time_parsing': '输入应为有效的时间格式,{error}',
'datetime_type': '输入应为有效的日期时间',
'datetime_parsing': '输入应为有效的日期时间,{error}',
'datetime_object_invalid': '无效的日期时间对象,得到 {error}',
'datetime_past': '输入应为过去的时间',
'datetime_future': '输入应为未来的时间',
'timezone_naive': '输入不应包含时区信息',
'timezone_aware': '输入应包含时区信息',
'timezone_offset': '需要时区偏移为 {tz_expected},实际得到 {tz_actual}',
'time_delta_type': '输入应为有效的时间差',
'time_delta_parsing': '输入应为有效的时间差,{error}',
'frozen_set_type': '输入应为有效的冻结集合',
'is_instance_of': '输入应为 {class} 的实例',
'is_subclass_of': '输入应为 {class} 的子类',
'callable_type': '输入应为可调用对象',
'union_tag_invalid': "使用 {discriminator} 找到的输入标签 '{tag}' 与任何预期标签不匹配: {expected_tags}",
'union_tag_not_found': '无法使用区分器 {discriminator} 提取标签',
'arguments_type': '参数必须是元组、列表或字典',
'missing_argument': '缺少必需参数',
'unexpected_keyword_argument': '意外的关键字参数',
'missing_keyword_only_argument': '缺少必需的关键字专用参数',
'unexpected_positional_argument': '意外的位置参数',
'missing_positional_only_argument': '缺少必需的位置专用参数',
'multiple_argument_values': '为参数提供了多个值',
'url_type': 'URL 输入应为字符串或 URL',
'url_parsing': '输入应为有效的 URL{error}',
'url_syntax_violation': '输入违反了严格的 URL 语法规则,{error}',
'url_too_long': 'URL 最多应有 {max_length} 个字符',
'url_scheme': 'URL 方案应为 {expected_schemes}',
'uuid_type': 'UUID 输入应为字符串、字节或 UUID 对象',
'uuid_parsing': '输入应为有效的 UUID{error}',
'uuid_version': '预期 UUID 版本为 {expected_version}',
'decimal_type': '十进制输入应为整数、浮点数、字符串或 Decimal 对象',
'decimal_parsing': '输入应为有效的十进制数',
'decimal_max_digits': '十进制输入总共应不超过 {max_digits} 位数字',
'decimal_max_places': '十进制输入应不超过 {decimal_places} 位小数',
'decimal_whole_digits': '十进制输入在小数点前应不超过 {whole_digits} 位数字',
}
CustomPhoneNumber = Annotated[str, Field(pattern=r'^1[3-9]\d{9}$')] CustomPhoneNumber = Annotated[str, Field(pattern=r'^1[3-9]\d{9}$')]
@@ -124,5 +23,9 @@ class SchemaBase(BaseModel):
model_config = ConfigDict( model_config = ConfigDict(
use_enum_values=True, use_enum_values=True,
json_encoders={datetime: lambda x: x.strftime(settings.DATETIME_FORMAT)}, json_encoders={
datetime: lambda x: timezone.to_str(timezone.from_datetime(x))
if x.tzinfo is not None and x.tzinfo != timezone.tz_info
else timezone.to_str(x)
},
) )
+1 -1
View File
@@ -294,7 +294,7 @@ async def jwt_authentication(token: str) -> GetUserInfoWithRelationDetail:
user = GetUserInfoWithRelationDetail(**select_as_dict(current_user)) user = GetUserInfoWithRelationDetail(**select_as_dict(current_user))
await redis_client.setex( await redis_client.setex(
f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}', f'{settings.JWT_USER_REDIS_PREFIX}:{user_id}',
settings.JWT_USER_REDIS_EXPIRE_SECONDS, settings.TOKEN_EXPIRE_SECONDS,
user.model_dump_json(), user.model_dump_json(),
) )
else: else:
+3
View File
@@ -23,6 +23,9 @@ async def rbac_verify(request: Request, _token: str = DependsJwtAuth) -> None:
# API 鉴权白名单 # API 鉴权白名单
if path in settings.TOKEN_REQUEST_PATH_EXCLUDE: if path in settings.TOKEN_REQUEST_PATH_EXCLUDE:
return return
for pattern in settings.TOKEN_REQUEST_PATH_EXCLUDE_PATTERN:
if pattern.match(path):
return
# JWT 授权状态强制校验 # JWT 授权状态强制校验
if not request.auth.scopes: if not request.auth.scopes:
+81 -50
View File
@@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from functools import lru_cache from functools import lru_cache
from typing import Any, Literal from typing import Any, Literal, Pattern
from pydantic import model_validator from pydantic import model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict from pydantic_settings import BaseSettings, SettingsConfigDict
@@ -19,27 +19,8 @@ class Settings(BaseSettings):
case_sensitive=True, case_sensitive=True,
) )
# .env 环境 # .env 当前环境
ENVIRONMENT: Literal['dev', 'pro'] ENVIRONMENT: Literal['dev', 'prod']
# .env 数据库
DATABASE_TYPE: Literal['mysql', 'postgresql']
DATABASE_HOST: str
DATABASE_PORT: int
DATABASE_USER: str
DATABASE_PASSWORD: str
# .env Redis
REDIS_HOST: str
REDIS_PORT: int
REDIS_PASSWORD: str
REDIS_DATABASE: int
# .env Token
TOKEN_SECRET_KEY: str # 密钥 secrets.token_urlsafe(32)
# .env 操作日志加密密钥
OPERA_LOG_ENCRYPT_SECRET_KEY: str # 密钥 os.urandom(32), 需使用 bytes.hex() 方法转换为 str
# FastAPI # FastAPI
FASTAPI_API_V1_PATH: str = '/api/v1' FASTAPI_API_V1_PATH: str = '/api/v1'
@@ -51,15 +32,31 @@ class Settings(BaseSettings):
FASTAPI_OPENAPI_URL: str | None = '/openapi' FASTAPI_OPENAPI_URL: str | None = '/openapi'
FASTAPI_STATIC_FILES: bool = True FASTAPI_STATIC_FILES: bool = True
# .env 数据库
DATABASE_TYPE: Literal['mysql', 'postgresql']
DATABASE_HOST: str
DATABASE_PORT: int
DATABASE_USER: str
DATABASE_PASSWORD: str
# 数据库 # 数据库
DATABASE_ECHO: bool | Literal['debug'] = False DATABASE_ECHO: bool | Literal['debug'] = False
DATABASE_POOL_ECHO: bool | Literal['debug'] = False DATABASE_POOL_ECHO: bool | Literal['debug'] = False
DATABASE_SCHEMA: str = 'fba' DATABASE_SCHEMA: str = 'fba'
DATABASE_CHARSET: str = 'utf8mb4' DATABASE_CHARSET: str = 'utf8mb4'
# .env Redis
REDIS_HOST: str
REDIS_PORT: int
REDIS_PASSWORD: str
REDIS_DATABASE: int
# Redis # Redis
REDIS_TIMEOUT: int = 5 REDIS_TIMEOUT: int = 5
# .env Token
TOKEN_SECRET_KEY: str # 密钥 secrets.token_urlsafe(32)
# Token # Token
TOKEN_ALGORITHM: str = 'HS256' TOKEN_ALGORITHM: str = 'HS256'
TOKEN_EXPIRE_SECONDS: int = 60 * 60 * 24 # 1 天 TOKEN_EXPIRE_SECONDS: int = 60 * 60 * 24 # 1 天
@@ -70,12 +67,13 @@ class Settings(BaseSettings):
TOKEN_REFRESH_REDIS_PREFIX: str = 'fba:refresh_token' TOKEN_REFRESH_REDIS_PREFIX: str = 'fba:refresh_token'
TOKEN_REQUEST_PATH_EXCLUDE: list[str] = [ # JWT / RBAC 路由白名单 TOKEN_REQUEST_PATH_EXCLUDE: list[str] = [ # JWT / RBAC 路由白名单
f'{FASTAPI_API_V1_PATH}/auth/login', f'{FASTAPI_API_V1_PATH}/auth/login',
f'{FASTAPI_API_V1_PATH}/auth/logout', ]
TOKEN_REQUEST_PATH_EXCLUDE_PATTERN: list[Pattern[str]] = [ # JWT / RBAC 路由白名单(正则)
rf'^{FASTAPI_API_V1_PATH}/monitors/(redis|server)$',
] ]
# JWT # JWT
JWT_USER_REDIS_PREFIX: str = 'fba:user' JWT_USER_REDIS_PREFIX: str = 'fba:user'
JWT_USER_REDIS_EXPIRE_SECONDS: int = 60 * 60 * 24 * 7 # 7 天
# RBAC # RBAC
RBAC_ROLE_MENU_MODE: bool = True RBAC_ROLE_MENU_MODE: bool = True
@@ -88,7 +86,11 @@ class Settings(BaseSettings):
COOKIE_REFRESH_TOKEN_KEY: str = 'fba_refresh_token' COOKIE_REFRESH_TOKEN_KEY: str = 'fba_refresh_token'
COOKIE_REFRESH_TOKEN_EXPIRE_SECONDS: int = 60 * 60 * 24 * 7 # 7 天 COOKIE_REFRESH_TOKEN_EXPIRE_SECONDS: int = 60 * 60 * 24 * 7 # 7 天
# 数据权限配置 # 验证码
CAPTCHA_LOGIN_REDIS_PREFIX: str = 'fba:login:captcha'
CAPTCHA_LOGIN_EXPIRE_SECONDS: int = 60 * 5 # 3 分钟
# 数据权限
DATA_PERMISSION_MODELS: dict[str, str] = { # 允许进行数据过滤的 SQLA 模型,它必须以模块字符串的方式定义 DATA_PERMISSION_MODELS: dict[str, str] = { # 允许进行数据过滤的 SQLA 模型,它必须以模块字符串的方式定义
'部门': 'backend.app.admin.model.Dept', '部门': 'backend.app.admin.model.Dept',
} }
@@ -142,22 +144,27 @@ class Settings(BaseSettings):
IP_LOCATION_REDIS_PREFIX: str = 'fba:ip:location' IP_LOCATION_REDIS_PREFIX: str = 'fba:ip:location'
IP_LOCATION_EXPIRE_SECONDS: int = 60 * 60 * 24 # 1 天 IP_LOCATION_EXPIRE_SECONDS: int = 60 * 60 * 24 # 1 天
# 日志(Trace ID) # Trace ID
TRACE_ID_REQUEST_HEADER_KEY: str = 'X-Request-ID' TRACE_ID_REQUEST_HEADER_KEY: str = 'X-Request-ID'
TRACE_ID_LOG_LENGTH: int = 32 # UUID 长度,必须小于等于 32
TRACE_ID_LOG_DEFAULT_VALUE: str = '-' TRACE_ID_LOG_DEFAULT_VALUE: str = '-'
TRACE_ID_LOG_UUID_LENGTH: int = 32 # UUID 长度,必须小于等于 32
# 日志
LOG_FORMAT: str = (
'<green>{time:YYYY-MM-DD HH:mm:ss.SSS}</> | <lvl>{level: <8}</> | <cyan>{correlation_id}</> | <lvl>{message}</>'
)
# 日志(控制台) # 日志(控制台)
LOG_STD_LEVEL: str = 'INFO' LOG_STD_LEVEL: str = 'INFO'
LOG_STD_FORMAT: str = (
'<green>{time:YYYY-MM-DD HH:mm:ss.SSS}</> | <lvl>{level: <8}</> | <cyan>{correlation_id}</> | <lvl>{message}</>'
)
# 日志(文件) # 日志(文件)
LOG_ACCESS_FILE_LEVEL: str = 'INFO' LOG_FILE_ACCESS_LEVEL: str = 'INFO'
LOG_ERROR_FILE_LEVEL: str = 'ERROR' LOG_FILE_ERROR_LEVEL: str = 'ERROR'
LOG_ACCESS_FILENAME: str = 'fba_access.log' LOG_ACCESS_FILENAME: str = 'fba_access.log'
LOG_ERROR_FILENAME: str = 'fba_error.log' LOG_ERROR_FILENAME: str = 'fba_error.log'
LOG_FILE_FORMAT: str = '{time:YYYY-MM-DD HH:mm:ss.SSS} | <lvl>{level: <8}</> | {correlation_id} | <lvl>{message}</>'
# .env 操作日志
OPERA_LOG_ENCRYPT_SECRET_KEY: str # 密钥 os.urandom(32), 需使用 bytes.hex() 方法转换为 str
# 操作日志 # 操作日志
OPERA_LOG_PATH_EXCLUDE: list[str] = [ OPERA_LOG_PATH_EXCLUDE: list[str] = [
@@ -176,27 +183,20 @@ class Settings(BaseSettings):
'new_password', 'new_password',
'confirm_password', 'confirm_password',
] ]
OPERA_LOG_QUEUE_BATCH_CONSUME_SIZE: int = 100
OPERA_LOG_QUEUE_TIMEOUT: int = 60 # 1 分钟
# Plugin 配置 # Plugin 配置
PLUGIN_PIP_CHINA: bool = True PLUGIN_PIP_CHINA: bool = True
PLUGIN_PIP_INDEX_URL: str = 'https://mirrors.aliyun.com/pypi/simple/' PLUGIN_PIP_INDEX_URL: str = 'https://mirrors.aliyun.com/pypi/simple/'
PLUGIN_REDIS_PREFIX: str = 'fba:plugin' PLUGIN_REDIS_PREFIX: str = 'fba:plugin'
# App Admin # I18n 配置
# .env OAuth2 I18N_DEFAULT_LANGUAGE: str = 'zh-CN'
OAUTH2_GITHUB_CLIENT_ID: str
OAUTH2_GITHUB_CLIENT_SECRET: str
OAUTH2_LINUX_DO_CLIENT_ID: str
OAUTH2_LINUX_DO_CLIENT_SECRET: str
# OAuth2 ##################################################
OAUTH2_FRONTEND_REDIRECT_URI: str = 'http://localhost:5173/oauth2/callback' # [ App ] task
##################################################
# 验证码
CAPTCHA_LOGIN_REDIS_PREFIX: str = 'fba:login:captcha'
CAPTCHA_LOGIN_EXPIRE_SECONDS: int = 60 * 5 # 3 分钟
# App Task
# .env Redis # .env Redis
CELERY_BROKER_REDIS_DATABASE: int CELERY_BROKER_REDIS_DATABASE: int
@@ -212,18 +212,49 @@ class Settings(BaseSettings):
CELERY_REDIS_PREFIX: str = 'fba:celery' CELERY_REDIS_PREFIX: str = 'fba:celery'
CELERY_TASK_MAX_RETRIES: int = 5 CELERY_TASK_MAX_RETRIES: int = 5
# Plugin Code Generator ##################################################
# [ Plugin ] code_generator
##################################################
CODE_GENERATOR_DOWNLOAD_ZIP_FILENAME: str = 'fba_generator' CODE_GENERATOR_DOWNLOAD_ZIP_FILENAME: str = 'fba_generator'
##################################################
# [ Plugin ] oauth2
##################################################
# .env
OAUTH2_GITHUB_CLIENT_ID: str
OAUTH2_GITHUB_CLIENT_SECRET: str
OAUTH2_GOOGLE_CLIENT_ID: str
OAUTH2_GOOGLE_CLIENT_SECRET: str
OAUTH2_LINUX_DO_CLIENT_ID: str
OAUTH2_LINUX_DO_CLIENT_SECRET: str
# 基础配置
OAUTH2_FRONTEND_REDIRECT_URI: str = 'http://localhost:5173/oauth2/callback'
##################################################
# [ Plugin ] email
##################################################
# .env
EMAIL_USERNAME: str
EMAIL_PASSWORD: str
# 基础配置
EMAIL_HOST: str = 'smtp.qq.com'
EMAIL_PORT: int = 465
EMAIL_SSL: bool = True
EMAIL_CAPTCHA_REDIS_PREFIX: str = 'fba:email:captcha'
EMAIL_CAPTCHA_EXPIRE_SECONDS: int = 60 * 3 # 3 分钟
@model_validator(mode='before') @model_validator(mode='before')
@classmethod @classmethod
def check_env(cls, values: Any) -> Any: def check_env(cls, values: Any) -> Any:
"""检查环境变量""" """检查环境变量"""
if values.get('ENVIRONMENT') == 'pro': if values.get('ENVIRONMENT') == 'prod':
# FastAPI # FastAPI
values['FASTAPI_OPENAPI_URL'] = None values['FASTAPI_OPENAPI_URL'] = None
values['FASTAPI_STATIC_FILES'] = False values['FASTAPI_STATIC_FILES'] = False
# Task
# task
values['CELERY_BROKER'] = 'rabbitmq' values['CELERY_BROKER'] = 'rabbitmq'
return values return values
+5 -2
View File
@@ -17,8 +17,11 @@ STATIC_DIR = BASE_PATH / 'static'
# 上传文件目录 # 上传文件目录
UPLOAD_DIR = STATIC_DIR / 'upload' UPLOAD_DIR = STATIC_DIR / 'upload'
# 离线 IP 数据库路径
IP2REGION_XDB = STATIC_DIR / 'ip2region.xdb'
# 插件目录 # 插件目录
PLUGIN_DIR = BASE_PATH / 'plugin' PLUGIN_DIR = BASE_PATH / 'plugin'
# 离线 IP 数据库路径 # 国际化文件目录
IP2REGION_XDB = STATIC_DIR / 'ip2region.xdb' LOCALE_DIR = BASE_PATH / 'locale'
+33 -18
View File
@@ -2,6 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
import os import os
from asyncio import create_task
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from typing import AsyncGenerator from typing import AsyncGenerator
@@ -12,15 +13,18 @@ from fastapi import Depends, FastAPI
from fastapi_limiter import FastAPILimiter from fastapi_limiter import FastAPILimiter
from fastapi_pagination import add_pagination from fastapi_pagination import add_pagination
from starlette.middleware.authentication import AuthenticationMiddleware from starlette.middleware.authentication import AuthenticationMiddleware
from starlette.middleware.cors import CORSMiddleware
from starlette.staticfiles import StaticFiles from starlette.staticfiles import StaticFiles
from starlette.types import ASGIApp
from backend.common.exception.exception_handler import register_exception from backend.common.exception.exception_handler import register_exception
from backend.common.log import set_custom_logfile, setup_logging from backend.common.log import set_custom_logfile, setup_logging
from backend.core.conf import settings from backend.core.conf import settings
from backend.core.path_conf import STATIC_DIR, UPLOAD_DIR from backend.core.path_conf import STATIC_DIR, UPLOAD_DIR
from backend.database.db import create_table from backend.database.db import create_tables
from backend.database.redis import redis_client from backend.database.redis import redis_client
from backend.middleware.access_middleware import AccessMiddleware from backend.middleware.access_middleware import AccessMiddleware
from backend.middleware.i18n_middleware import I18nMiddleware
from backend.middleware.jwt_auth_middleware import JwtAuthMiddleware from backend.middleware.jwt_auth_middleware import JwtAuthMiddleware
from backend.middleware.opera_log_middleware import OperaLogMiddleware from backend.middleware.opera_log_middleware import OperaLogMiddleware
from backend.middleware.state_middleware import StateMiddleware from backend.middleware.state_middleware import StateMiddleware
@@ -40,7 +44,11 @@ async def register_init(app: FastAPI) -> AsyncGenerator[None, None]:
:return: :return:
""" """
# 创建数据库表 # 创建数据库表
await create_table() await create_tables()
# 初始化 redis
await redis_client.open()
# 初始化 limiter # 初始化 limiter
await FastAPILimiter.init( await FastAPILimiter.init(
redis=redis_client, redis=redis_client,
@@ -48,17 +56,34 @@ async def register_init(app: FastAPI) -> AsyncGenerator[None, None]:
http_callback=http_limit_callback, http_callback=http_limit_callback,
) )
# 创建操作日志任务
create_task(OperaLogMiddleware.consumer())
yield yield
# 关闭 redis 连接 # 关闭 redis 连接
await redis_client.close() await redis_client.aclose()
# 关闭 limiter
await FastAPILimiter.close()
def register_app() -> FastAPI: def register_app() -> FastAPI:
"""注册 FastAPI 应用""" """注册 FastAPI 应用"""
app = FastAPI(
class MyFastAPI(FastAPI):
if settings.MIDDLEWARE_CORS:
# Related issues
# https://github.com/fastapi/fastapi/discussions/7847
# https://github.com/fastapi/fastapi/discussions/8027
def build_middleware_stack(self) -> ASGIApp:
return CORSMiddleware(
super().build_middleware_stack(),
allow_origins=settings.CORS_ALLOWED_ORIGINS,
allow_credentials=True,
allow_methods=['*'],
allow_headers=['*'],
expose_headers=settings.CORS_EXPOSE_HEADERS,
)
app = MyFastAPI(
title=settings.FASTAPI_TITLE, title=settings.FASTAPI_TITLE,
version=settings.FASTAPI_VERSION, version=settings.FASTAPI_VERSION,
description=settings.FASTAPI_DESCRIPTION, description=settings.FASTAPI_DESCRIPTION,
@@ -124,18 +149,8 @@ def register_middleware(app: FastAPI) -> None:
on_error=JwtAuthMiddleware.auth_exception_handler, on_error=JwtAuthMiddleware.auth_exception_handler,
) )
# CORS # I18n
if settings.MIDDLEWARE_CORS: app.add_middleware(I18nMiddleware)
from fastapi.middleware.cors import CORSMiddleware
app.add_middleware(
CORSMiddleware,
allow_origins=settings.CORS_ALLOWED_ORIGINS,
allow_credentials=True,
allow_methods=['*'],
allow_headers=['*'],
expose_headers=settings.CORS_EXPOSE_HEADERS,
)
# Access log # Access log
app.add_middleware(AccessMiddleware) app.add_middleware(AccessMiddleware)
+5 -1
View File
@@ -75,7 +75,7 @@ async def get_db() -> AsyncGenerator[AsyncSession, None]:
yield session yield session
async def create_table() -> None: async def create_tables() -> None:
"""创建数据库表""" """创建数据库表"""
async with async_engine.begin() as coon: async with async_engine.begin() as coon:
await coon.run_sync(MappedBase.metadata.create_all) await coon.run_sync(MappedBase.metadata.create_all)
@@ -86,7 +86,11 @@ def uuid4_str() -> str:
return str(uuid4()) return str(uuid4())
# SQLA 数据库链接
SQLALCHEMY_DATABASE_URL = create_database_url() SQLALCHEMY_DATABASE_URL = create_database_url()
# SALA 异步引擎和会话
async_engine, async_db_session = create_async_engine_and_session(SQLALCHEMY_DATABASE_URL) async_engine, async_db_session = create_async_engine_and_session(SQLALCHEMY_DATABASE_URL)
# Session Annotated # Session Annotated
CurrentSession = Annotated[AsyncSession, Depends(get_db)] CurrentSession = Annotated[AsyncSession, Depends(get_db)]
+19
View File
@@ -0,0 +1,19 @@
{
"error": {
"captcha": {
"error": "Captcha error",
"expired": "Captcha has expired, please try again"
},
"language_not_found": "Current language pack is not initialized or does not exist"
},
"response": {
"error": "Request error",
"success": "Request success"
},
"success": {
"login": {
"oauth2_success": "Login success (OAuth2)",
"success": "Login success"
}
}
}
+113
View File
@@ -0,0 +1,113 @@
error:
captcha:
error: 验证码错误
expired: 验证码已过期,请重新获取
language_not_found: 当前语言包未初始化或不存在
pydantic:
# 自定义验证错误信息,参考:
# https://github.com/pydantic/pydantic-core/blob/a5cb7382643415b716b1a7a5392914e50f726528/tests/test_errors.py#L266
# https://github.com/pydantic/pydantic/blob/caa78016433ec9b16a973f92f187a7b6bfde6cb5/docs/errors/errors.md?plain=1#L232
arguments_type: 参数必须是元组、列表或字典
assertion_error: '断言失败,{error}'
bool_parsing: 输入应为有效的布尔值,无法解释输入
bool_type: 输入应为有效的布尔值
bytes_too_long: '数据最多应有 {max_length} 个字节'
bytes_too_short: '数据应至少有 {min_length} 个字节'
bytes_type: 输入应为有效的字节
callable_type: 输入应为可调用对象
dataclass_exact_type: '输入应为 {class_name} 的实例'
dataclass_type: '输入应为字典或 {class_name} 的实例'
date_from_datetime_inexact: 提供给日期的日期时间应具有零时间 - 例如为精确日期
date_from_datetime_parsing: '输入应为有效的日期或日期时间,{error}'
date_future: 日期应为未来的时间
date_past: 日期应为过去的时间
date_parsing: '输入应为 YYYY-MM-DD 格式的有效日期,{error}'
date_type: 输入应为有效的日期
datetime_future: 输入应为未来的时间
datetime_object_invalid: '无效的日期时间对象,得到 {error}'
datetime_past: 输入应为过去的时间
datetime_parsing: '输入应为有效的日期时间,{error}'
datetime_type: 输入应为有效的日期时间
decimal_max_digits: '十进制输入总共应不超过 {max_digits} 位数字'
decimal_max_places: '十进制输入应不超过 {decimal_places} 位小数'
decimal_parsing: 输入应为有效的十进制数
decimal_type: 十进制输入应为整数、浮点数、字符串或 Decimal 对象
decimal_whole_digits: '十进制输入在小数点前应不超过 {whole_digits} 位数字'
dict_type: 输入应为有效的字典
email_parsing: '输入应为有效的邮箱地址,{error}'
email_type: 输入应为有效的邮箱地址
enum: '输入应为 {expected}'
extra_forbidden: 不允许额外的输入
finite_number: 输入应为有限数字
float_parsing: 输入应为有效的数字,无法将字符串解析为数字
float_type: 输入应为有效的数字
frozen_field: 字段已冻结
frozen_instance: 实例已冻结
frozen_set_type: 输入应为有效的冻结集合
get_attribute_error: '提取属性时出错: {error}'
greater_than: '输入应大于 {gt}'
greater_than_equal: '输入应大于或等于 {ge}'
int_from_float: 输入应为有效的整数,得到一个带有小数部分的数字
int_parsing: 输入应为有效的整数,无法将字符串解析为整数
int_parsing_size: 无法将输入字符串解析为整数,超出最大大小
int_type: 输入应为有效的整数
invalid_key: 键应为字符串
is_instance_of: '输入应为 {class} 的实例'
is_subclass_of: '输入应为 {class} 的子类'
iteration_error: '迭代对象时出错,错误: {error}'
iterable_type: 输入应为可迭代对象
json_invalid: '无效的 JSON: {error}'
json_type: JSON 输入应为字符串、字节或字节数组
less_than: '输入应小于 {lt}'
less_than_equal: '输入应小于或等于 {le}'
list_type: 输入应为有效的列表
literal_error: '输入应为 {expected}'
mapping_type: '输入应为有效的映射,错误: {error}'
missing: 字段为必填项
missing_argument: 缺少必需参数
missing_keyword_only_argument: 缺少必需的关键字专用参数
missing_positional_only_argument: 缺少必需的位置专用参数
model_attributes_type: 输入应为有效的字典或可提取字段的对象
model_type: '输入应为有效的字典或 {class_name} 的实例'
multiple_argument_values: 为参数提供了多个值
multiple_of: '输入应为 {multiple_of} 的倍数'
no_such_attribute: '对象没有属性 ''{attribute}'''
none_required: 输入应为 None
recursion_loop: 递归错误 - 检测到循环引用
set_type: 输入应为有效的集合
string_pattern_mismatch: '字符串应匹配模式 ''{pattern}'''
string_sub_type: 输入应为字符串,而不是 str 子类的实例
string_too_long: '字符串最多应有 {max_length} 个字符'
string_too_short: '字符串应至少有 {min_length} 个字符'
string_type: 输入应为有效的字符串
string_unicode: 输入应为有效的字符串,无法将原始数据解析为 Unicode 字符串
time_delta_parsing: '输入应为有效的时间差,{error}'
time_delta_type: 输入应为有效的时间差
time_parsing: '输入应为有效的时间格式,{error}'
time_type: 输入应为有效的时间
timezone_aware: 输入应包含时区信息
timezone_naive: 输入不应包含时区信息
timezone_offset: '需要时区偏移为 {tz_expected},实际得到 {tz_actual}'
too_long: '{field_type} 在验证后最多应有 {max_length} 个项目,而不是 {actual_length}'
too_short: '{field_type} 在验证后应至少有 {min_length} 个项目,而不是 {actual_length}'
tuple_type: 输入应为有效的元组
union_tag_invalid: '使用 {discriminator} 找到的输入标签 ''{tag}'' 与任何预期标签不匹配: {expected_tags}'
union_tag_not_found: '无法使用区分器 {discriminator} 提取标签'
unexpected_keyword_argument: 意外的关键字参数
unexpected_positional_argument: 意外的位置参数
url_parsing: '输入应为有效的 URL{error}'
url_scheme: 'URL 方案应为 {expected_schemes}'
url_syntax_violation: '输入违反了严格的 URL 语法规则,{error}'
url_too_long: 'URL 最多应有 {max_length} 个字符'
url_type: URL 输入应为字符串或 URL
uuid_parsing: '输入应为有效的 UUID{error}'
uuid_type: UUID 输入应为字符串、字节或 UUID 对象
uuid_version: '预期 UUID 版本为 {expected_version}'
value_error: '值错误,{error}'
response:
error: 请求错误
success: 请求成功
success:
login:
oauth2_success: 登录成功(OAuth2
success: 登录成功
+1 -1
View File
@@ -3,9 +3,9 @@
from rich.progress import Progress, SpinnerColumn, TextColumn, TimeElapsedColumn from rich.progress import Progress, SpinnerColumn, TextColumn, TimeElapsedColumn
from rich.text import Text from rich.text import Text
from backend import console
from backend.core.registrar import register_app from backend.core.registrar import register_app
from backend.plugin.tools import get_plugins, install_requirements from backend.plugin.tools import get_plugins, install_requirements
from backend.utils.console import console
from backend.utils.timezone import timezone from backend.utils.timezone import timezone
_log_prefix = f'{timezone.to_str(timezone.now(), "%Y-%m-%d %H:%M:%S.%M0")} | {"INFO": <8} | - | ' _log_prefix = f'{timezone.to_str(timezone.now(), "%Y-%m-%d %H:%M:%S.%M0")} | {"INFO": <8} | - | '
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from functools import lru_cache
from typing import Callable
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware
from backend.common.i18n import i18n
class I18nMiddleware(BaseHTTPMiddleware):
"""国际化中间件"""
async def dispatch(self, request: Request, call_next: Callable) -> Response:
"""
处理请求并设置国际化语言
:param request: FastAPI 请求对象
:param call_next: 下一个中间件或路由处理函数
:return:
"""
language = self.get_current_language(request)
# 设置国际化语言
if language and i18n.current_language != language:
i18n.current_language = language
response = await call_next(request)
return response
@lru_cache(maxsize=128)
def get_current_language(self, request: Request) -> str | None:
"""
获取当前请求的语言偏好
:param request: FastAPI 请求对象
:return:
"""
accept_language = request.headers.get('Accept-Language', '')
if not accept_language:
return None
languages = [lang.split(';')[0] for lang in accept_language.split(',')]
lang = languages[0].lower().strip()
# 语言映射
lang_mapping = {
'zh': 'zh-CN',
'zh-cn': 'zh-CN',
'zh-hans': 'zh-CN',
'en': 'en-US',
'en-us': 'en-US',
}
return lang_mapping.get(lang, lang)
+5 -1
View File
@@ -59,8 +59,12 @@ class JwtAuthMiddleware(AuthenticationBackend):
if not token: if not token:
return None return None
if request.url.path in settings.TOKEN_REQUEST_PATH_EXCLUDE: path = request.url.path
if path in settings.TOKEN_REQUEST_PATH_EXCLUDE:
return None return None
for pattern in settings.TOKEN_REQUEST_PATH_EXCLUDE_PATTERN:
if pattern.match(path):
return None
scheme, token = get_authorization_scheme_param(token) scheme, token = get_authorization_scheme_param(token)
if scheme.lower() != 'bearer': if scheme.lower() != 'bearer':
+51 -30
View File
@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
import time import time
from asyncio import create_task from asyncio import Queue
from typing import Any from typing import Any
from asgiref.sync import sync_to_async from asgiref.sync import sync_to_async
@@ -15,6 +15,8 @@ from backend.app.admin.schema.opera_log import CreateOperaLogParam
from backend.app.admin.service.opera_log_service import opera_log_service from backend.app.admin.service.opera_log_service import opera_log_service
from backend.common.enums import OperaLogCipherType, StatusType from backend.common.enums import OperaLogCipherType, StatusType
from backend.common.log import log from backend.common.log import log
from backend.common.queue import batch_dequeue
from backend.common.response.response_code import StandardResponseCode
from backend.core.conf import settings from backend.core.conf import settings
from backend.utils.encrypt import AESCipher, ItsDCipher, Md5Cipher from backend.utils.encrypt import AESCipher, ItsDCipher, Md5Cipher
from backend.utils.trace_id import get_request_trace_id from backend.utils.trace_id import get_request_trace_id
@@ -23,6 +25,8 @@ from backend.utils.trace_id import get_request_trace_id
class OperaLogMiddleware(BaseHTTPMiddleware): class OperaLogMiddleware(BaseHTTPMiddleware):
"""操作日志中间件""" """操作日志中间件"""
opera_log_queue: Queue = Queue(maxsize=100000)
async def dispatch(self, request: Request, call_next: Any) -> Response: async def dispatch(self, request: Request, call_next: Any) -> Response:
""" """
处理请求并记录操作日志 处理请求并记录操作日志
@@ -39,10 +43,8 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
else: else:
method = request.method method = request.method
args = await self.get_request_args(request) args = await self.get_request_args(request)
args = await self.desensitization(args)
# 执行请求 # 执行请求
elapsed = 0.0
code = 200 code = 200
msg = 'Success' msg = 'Success'
status = StatusType.enable status = StatusType.enable
@@ -55,8 +57,6 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
'__request_validation_exception__', '__request_validation_exception__',
'__request_assertion_error__', '__request_assertion_error__',
'__request_custom_exception__', '__request_custom_exception__',
'__request_all_unknown_exception__',
'__request_cors_500_exception__',
]: ]:
exception = getattr(request.state, state, None) exception = getattr(request.state, state, None)
if exception: if exception:
@@ -65,11 +65,12 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
log.error(f'请求异常: {msg}') log.error(f'请求异常: {msg}')
break break
except Exception as e: except Exception as e:
log.error(f'请求异常: {str(e)}') elapsed = (time.perf_counter() - request.state.perf_time) * 1000
code = getattr(e, 'code', code) # 兼容 SQLAlchemy 异常用法 code = getattr(e, 'code', StandardResponseCode.HTTP_500) # 兼容 SQLAlchemy 异常用法
msg = getattr(e, 'msg', msg) msg = getattr(e, 'msg', str(e)) # 不建议使用 traceback 模块获取错误信息,会暴漏代码信息
status = StatusType.disable status = StatusType.disable
error = e error = e
log.error(f'请求异常: {str(e)}')
# 此信息只能在请求后获取 # 此信息只能在请求后获取
_route = request.scope.get('route') _route = request.scope.get('route')
@@ -108,7 +109,7 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
cost_time=elapsed, # 可能和日志存在微小差异(可忽略) cost_time=elapsed, # 可能和日志存在微小差异(可忽略)
opera_time=request.state.start_time, opera_time=request.state.start_time,
) )
create_task(opera_log_service.create(obj=opera_log_in)) # noqa: ignore await self.opera_log_queue.put(opera_log_in)
# 错误抛出 # 错误抛出
if error: if error:
@@ -116,8 +117,7 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
return response return response
@staticmethod async def get_request_args(self, request: Request) -> dict[str, Any] | None:
async def get_request_args(request: Request) -> dict[str, Any]:
""" """
获取请求参数 获取请求参数
@@ -125,26 +125,35 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
:return: :return:
""" """
args = {} args = {}
# 查询参数
query_params = dict(request.query_params) query_params = dict(request.query_params)
if query_params: if query_params:
args['query_params'] = query_params args['query_params'] = await self.desensitization(query_params)
# 路径参数
path_params = request.path_params path_params = request.path_params
if path_params: if path_params:
args['path_params'] = path_params args['path_params'] = await self.desensitization(path_params)
# Tip: .body() 必须在 .form() 之前获取 # Tip: .body() 必须在 .form() 之前获取
# https://github.com/encode/starlette/discussions/1933 # https://github.com/encode/starlette/discussions/1933
content_type = request.headers.get('Content-Type', '').split(';') content_type = request.headers.get('Content-Type', '').split(';')
# 请求体
body_data = await request.body() body_data = await request.body()
if body_data: if body_data:
# 注意:非 json 数据默认使用 body 作为键 # 注意:非 json 数据默认使用 data 作为键
if 'application/json' not in content_type: if 'application/json' not in content_type:
args['data'] = str(body_data) args['data'] = str(body_data)
else: else:
json_data = await request.json() json_data = await request.json()
if isinstance(json_data, dict): if isinstance(json_data, dict):
args['json'] = json_data args['json'] = await self.desensitization(json_data)
else: else:
args['data'] = str(body_data) args['data'] = str(body_data)
# 表单参数
form_data = await request.form() form_data = await request.form()
if len(form_data) > 0: if len(form_data) > 0:
for k, v in form_data.items(): for k, v in form_data.items():
@@ -153,39 +162,51 @@ class OperaLogMiddleware(BaseHTTPMiddleware):
else: else:
form_data = {k: v} form_data = {k: v}
if 'multipart/form-data' not in content_type: if 'multipart/form-data' not in content_type:
args['x-www-form-urlencoded'] = form_data args['x-www-form-urlencoded'] = await self.desensitization(form_data)
else: else:
args['form-data'] = form_data args['form-data'] = await self.desensitization(form_data)
return args return None if not args else args
@staticmethod @staticmethod
@sync_to_async @sync_to_async
def desensitization(args: dict[str, Any]) -> dict[str, Any] | None: def desensitization(args: dict[str, Any]) -> dict[str, Any]:
""" """
脱敏处理 脱敏处理
:param args: 需要脱敏的参数字典 :param args: 需要脱敏的参数字典
:return: :return:
""" """
if not args:
return None
encrypt_type = settings.OPERA_LOG_ENCRYPT_TYPE
encrypt_key_include = settings.OPERA_LOG_ENCRYPT_KEY_INCLUDE
encrypt_secret_key = settings.OPERA_LOG_ENCRYPT_SECRET_KEY
for key, value in args.items(): for key, value in args.items():
if key in encrypt_key_include: if key in settings.OPERA_LOG_ENCRYPT_KEY_INCLUDE:
match encrypt_type: match settings.OPERA_LOG_ENCRYPT_TYPE:
case OperaLogCipherType.aes: case OperaLogCipherType.aes:
args[key] = (AESCipher(encrypt_secret_key).encrypt(value)).hex() args[key] = (AESCipher(settings.OPERA_LOG_ENCRYPT_SECRET_KEY).encrypt(value)).hex()
case OperaLogCipherType.md5: case OperaLogCipherType.md5:
args[key] = Md5Cipher.encrypt(value) args[key] = Md5Cipher.encrypt(value)
case OperaLogCipherType.itsdangerous: case OperaLogCipherType.itsdangerous:
args[key] = ItsDCipher(encrypt_secret_key).encrypt(value) args[key] = ItsDCipher(settings.OPERA_LOG_ENCRYPT_SECRET_KEY).encrypt(value)
case OperaLogCipherType.plan: case OperaLogCipherType.plan:
pass pass
case _: case _:
args[key] = '******' args[key] = '******'
return args return args
@classmethod
async def consumer(cls) -> None:
"""操作日志消费者"""
while True:
logs = await batch_dequeue(
cls.opera_log_queue,
max_items=settings.OPERA_LOG_QUEUE_BATCH_CONSUME_SIZE,
timeout=settings.OPERA_LOG_QUEUE_TIMEOUT,
)
if logs:
try:
if settings.DATABASE_ECHO:
log.info('自动执行【操作日志批量创建】任务...')
await opera_log_service.bulk_create(objs=logs)
finally:
if not cls.opera_log_queue.empty():
cls.opera_log_queue.task_done()
+1 -1
View File
@@ -12,7 +12,7 @@
包含代码生成的相关配置,详情查看:`generator/model/gen_business.py` 包含代码生成的相关配置,详情查看:`generator/model/gen_business.py`
### 2. 代码生成模型 ### 2. 代码生成模型
包含代码生成所需要的模型列信息,就像正常定义模型列一样,目前支持的功能有限 包含代码生成所需要的模型列信息,就像正常定义模型列一样,目前支持的功能有限
+4 -4
View File
@@ -4,11 +4,11 @@ from fastapi import APIRouter
from backend.core.conf import settings from backend.core.conf import settings
from backend.plugin.code_generator.api.v1.business import router as business_router from backend.plugin.code_generator.api.v1.business import router as business_router
from backend.plugin.code_generator.api.v1.column import router as model_router from backend.plugin.code_generator.api.v1.code import router as gen_router
from backend.plugin.code_generator.api.v1.gen import router as gen_router from backend.plugin.code_generator.api.v1.column import router as column_router
v1 = APIRouter(prefix=f'{settings.FASTAPI_API_V1_PATH}/gen', tags=['代码生成']) v1 = APIRouter(prefix=f'{settings.FASTAPI_API_V1_PATH}/generates', tags=['代码生成'])
v1.include_router(business_router, prefix='/businesses') v1.include_router(business_router, prefix='/businesses')
v1.include_router(model_router, prefix='/models') v1.include_router(column_router, prefix='/columns')
v1.include_router(gen_router, prefix='/codes') v1.include_router(gen_router, prefix='/codes')
@@ -2,24 +2,32 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Annotated from typing import Annotated
from fastapi import APIRouter, Depends, Path from fastapi import APIRouter, Depends, Path, Query
from backend.common.pagination import DependsPagination, PageData, paging_data
from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base from backend.common.response.response_schema import ResponseModel, ResponseSchemaModel, response_base
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsJwtAuth
from backend.common.security.permission import RequestPermission from backend.common.security.permission import RequestPermission
from backend.common.security.rbac import DependsRBAC from backend.common.security.rbac import DependsRBAC
from backend.database.db import CurrentSession
from backend.plugin.code_generator.schema.business import ( from backend.plugin.code_generator.schema.business import (
CreateGenBusinessParam, CreateGenBusinessParam,
GetGenBusinessDetail, GetGenBusinessDetail,
UpdateGenBusinessParam, UpdateGenBusinessParam,
) )
from backend.plugin.code_generator.schema.column import GetGenModelDetail from backend.plugin.code_generator.schema.column import GetGenColumnDetail
from backend.plugin.code_generator.service.business_service import gen_business_service from backend.plugin.code_generator.service.business_service import gen_business_service
from backend.plugin.code_generator.service.column_service import gen_model_service from backend.plugin.code_generator.service.column_service import gen_column_service
router = APIRouter() router = APIRouter()
@router.get('/all', summary='获取所有代码生成业务', dependencies=[DependsJwtAuth])
async def get_all_businesses() -> ResponseSchemaModel[list[GetGenBusinessDetail]]:
data = await gen_business_service.get_all()
return response_base.success(data=data)
@router.get('/{pk}', summary='获取代码生成业务详情', dependencies=[DependsJwtAuth]) @router.get('/{pk}', summary='获取代码生成业务详情', dependencies=[DependsJwtAuth])
async def get_business( async def get_business(
pk: Annotated[int, Path(description='业务 ID')], pk: Annotated[int, Path(description='业务 ID')],
@@ -28,17 +36,28 @@ async def get_business(
return response_base.success(data=data) return response_base.success(data=data)
@router.get('', summary='获取所有代码生成业务', dependencies=[DependsJwtAuth]) @router.get(
async def get_all_businesses() -> ResponseSchemaModel[list[GetGenBusinessDetail]]: '',
data = await gen_business_service.get_all() summary='分页获取所有代码生成业务',
return response_base.success(data=data) dependencies=[
DependsJwtAuth,
DependsPagination,
],
)
async def get_businesses_paged(
db: CurrentSession,
table_name: Annotated[str | None, Query(description='代码生成业务表名称')] = None,
) -> ResponseSchemaModel[PageData[GetGenBusinessDetail]]:
business_select = await gen_business_service.get_select(table_name=table_name)
page_data = await paging_data(db, business_select)
return response_base.success(data=page_data)
@router.get('/{pk}/models', summary='获取代码生成业务所有模型', dependencies=[DependsJwtAuth]) @router.get('/{pk}/columns', summary='获取代码生成业务所有模型', dependencies=[DependsJwtAuth])
async def get_business_all_models( async def get_business_all_columns(
pk: Annotated[int, Path(description='业务 ID')], pk: Annotated[int, Path(description='业务 ID')],
) -> ResponseSchemaModel[list[GetGenModelDetail]]: ) -> ResponseSchemaModel[list[GetGenColumnDetail]]:
data = await gen_model_service.get_models(business_id=pk) data = await gen_column_service.get_columns(business_id=pk)
return response_base.success(data=data) return response_base.success(data=data)
@@ -10,8 +10,8 @@ from backend.common.security.jwt import DependsJwtAuth
from backend.common.security.permission import RequestPermission from backend.common.security.permission import RequestPermission
from backend.common.security.rbac import DependsRBAC from backend.common.security.rbac import DependsRBAC
from backend.core.conf import settings from backend.core.conf import settings
from backend.plugin.code_generator.schema.gen import ImportParam from backend.plugin.code_generator.schema.code import ImportParam
from backend.plugin.code_generator.service.gen_service import gen_service from backend.plugin.code_generator.service.code_service import gen_service
router = APIRouter() router = APIRouter()
@@ -19,7 +19,7 @@ router = APIRouter()
@router.get('/tables', summary='获取数据库表') @router.get('/tables', summary='获取数据库表')
async def get_all_tables( async def get_all_tables(
table_schema: Annotated[str, Query(description='数据库名')] = 'fba', table_schema: Annotated[str, Query(description='数据库名')] = 'fba',
) -> ResponseSchemaModel[list[str]]: ) -> ResponseSchemaModel[list[dict[str, str]]]:
data = await gen_service.get_tables(table_schema=table_schema) data = await gen_service.get_tables(table_schema=table_schema)
return response_base.success(data=data) return response_base.success(data=data)
+20 -20
View File
@@ -8,47 +8,47 @@ from backend.common.response.response_schema import ResponseModel, ResponseSchem
from backend.common.security.jwt import DependsJwtAuth from backend.common.security.jwt import DependsJwtAuth
from backend.common.security.permission import RequestPermission from backend.common.security.permission import RequestPermission
from backend.common.security.rbac import DependsRBAC from backend.common.security.rbac import DependsRBAC
from backend.plugin.code_generator.schema.column import CreateGenModelParam, GetGenModelDetail, UpdateGenModelParam from backend.plugin.code_generator.schema.column import CreateGenColumnParam, GetGenColumnDetail, UpdateGenColumnParam
from backend.plugin.code_generator.service.column_service import gen_model_service from backend.plugin.code_generator.service.column_service import gen_column_service
router = APIRouter() router = APIRouter()
@router.get('/types', summary='获取代码生成模型列类型', dependencies=[DependsJwtAuth]) @router.get('/types', summary='获取代码生成模型列类型', dependencies=[DependsJwtAuth])
async def get_model_types() -> ResponseSchemaModel[list[str]]: async def get_column_types() -> ResponseSchemaModel[list[str]]:
model_types = await gen_model_service.get_types() column_types = await gen_column_service.get_types()
return response_base.success(data=model_types) return response_base.success(data=column_types)
@router.get('/{pk}', summary='获取代码生成模型详情', dependencies=[DependsJwtAuth]) @router.get('/{pk}', summary='获取代码生成模型详情', dependencies=[DependsJwtAuth])
async def get_model(pk: Annotated[int, Path(description='模型 ID')]) -> ResponseSchemaModel[GetGenModelDetail]: async def get_column(pk: Annotated[int, Path(description='模型 ID')]) -> ResponseSchemaModel[GetGenColumnDetail]:
data = await gen_model_service.get(pk=pk) data = await gen_column_service.get(pk=pk)
return response_base.success(data=data) return response_base.success(data=data)
@router.post( @router.post(
'', '',
summary='创建代码生成模型', summary='创建代码生成模型',
dependencies=[ dependencies=[
Depends(RequestPermission('codegen:model:add')), Depends(RequestPermission('codegen:column:add')),
DependsRBAC, DependsRBAC,
], ],
) )
async def create_model(obj: CreateGenModelParam) -> ResponseModel: async def create_column(obj: CreateGenColumnParam) -> ResponseModel:
await gen_model_service.create(obj=obj) await gen_column_service.create(obj=obj)
return response_base.success() return response_base.success()
@router.put( @router.put(
'/{pk}', '/{pk}',
summary='更新代码生成模型', summary='更新代码生成模型',
dependencies=[ dependencies=[
Depends(RequestPermission('codegen:model:edit')), Depends(RequestPermission('codegen:column:edit')),
DependsRBAC, DependsRBAC,
], ],
) )
async def update_model(pk: Annotated[int, Path(description='模型 ID')], obj: UpdateGenModelParam) -> ResponseModel: async def update_column(pk: Annotated[int, Path(description='模型 ID')], obj: UpdateGenColumnParam) -> ResponseModel:
count = await gen_model_service.update(pk=pk, obj=obj) count = await gen_column_service.update(pk=pk, obj=obj)
if count > 0: if count > 0:
return response_base.success() return response_base.success()
return response_base.fail() return response_base.fail()
@@ -56,14 +56,14 @@ async def update_model(pk: Annotated[int, Path(description='模型 ID')], obj: U
@router.delete( @router.delete(
'/{pk}', '/{pk}',
summary='删除代码生成模型', summary='删除代码生成模型',
dependencies=[ dependencies=[
Depends(RequestPermission('codegen:model:del')), Depends(RequestPermission('codegen:column:del')),
DependsRBAC, DependsRBAC,
], ],
) )
async def delete_model(pk: Annotated[int, Path(description='模型 ID')]) -> ResponseModel: async def delete_column(pk: Annotated[int, Path(description='模型 ID')]) -> ResponseModel:
count = await gen_model_service.delete(pk=pk) count = await gen_column_service.delete(pk=pk)
if count > 0: if count > 0:
return response_base.success() return response_base.success()
return response_base.fail() return response_base.fail()
@@ -2,6 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Sequence from typing import Sequence
from sqlalchemy import Select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus from sqlalchemy_crud_plus import CRUDPlus
@@ -41,6 +42,20 @@ class CRUDGenBusiness(CRUDPlus[GenBusiness]):
""" """
return await self.select_models(db) return await self.select_models(db)
async def get_list(self, table_name: str | None) -> Select:
"""
获取所有代码生成业务
:param table_name: 业务表名
:return:
"""
filters = {}
if table_name is not None:
filters['table_name__like'] = f'%{table_name}%'
return await self.select_order('id', 'desc', load_strategies={'gen_column': 'noload'}, **filters)
async def create(self, db: AsyncSession, obj: CreateGenBusinessParam) -> None: async def create(self, db: AsyncSession, obj: CreateGenBusinessParam) -> None:
""" """
创建代码生成业务 创建代码生成业务
@@ -2,7 +2,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Sequence from typing import Sequence
from sqlalchemy import Row, text from sqlalchemy import Row, RowMapping, text
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from backend.core.conf import settings from backend.core.conf import settings
@@ -12,7 +12,7 @@ class CRUDGen:
"""代码生成 CRUD 类""" """代码生成 CRUD 类"""
@staticmethod @staticmethod
async def get_all_tables(db: AsyncSession, table_schema: str) -> Sequence[str]: async def get_all_tables(db: AsyncSession, table_schema: str) -> Sequence[RowMapping]:
""" """
获取所有表名 获取所有表名
@@ -22,20 +22,23 @@ class CRUDGen:
""" """
if settings.DATABASE_TYPE == 'mysql': if settings.DATABASE_TYPE == 'mysql':
sql = """ sql = """
SELECT table_name AS table_name FROM information_schema.tables SELECT table_name AS table_name, table_comment AS table_comment
WHERE table_name NOT LIKE 'sys_gen_%' FROM information_schema.tables
WHERE table_name NOT LIKE 'sys_gen_%'
AND table_schema = :table_schema; AND table_schema = :table_schema;
""" """
else: else:
sql = """ sql = """
SELECT table_name AS table_name FROM information_schema.tables SELECT c.relname AS table_name, obj_description(c.oid) AS table_comment
WHERE table_name NOT LIKE 'sys_gen_%' FROM pg_class c
AND table_catalog = :table_schema LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
AND table_schema = 'public'; -- schema 通常是 'public' WHERE c.relkind = 'r'
AND n.nspname = 'public' -- schema 通常是 'public'
AND c.relname NOT LIKE 'sys_gen_%';
""" """
stmt = text(sql).bindparams(table_schema=table_schema) stmt = text(sql).bindparams(table_schema=table_schema)
result = await db.execute(stmt) result = await db.execute(stmt)
return result.scalars().all() return result.mappings().all()
@staticmethod @staticmethod
async def get_table(db: AsyncSession, table_name: str) -> Row[tuple]: async def get_table(db: AsyncSession, table_name: str) -> Row[tuple]:
@@ -48,18 +51,19 @@ class CRUDGen:
""" """
if settings.DATABASE_TYPE == 'mysql': if settings.DATABASE_TYPE == 'mysql':
sql = """ sql = """
SELECT table_name AS table_name, table_comment AS table_comment FROM information_schema.tables SELECT table_name AS table_name, table_comment AS table_comment
FROM information_schema.tables
WHERE table_name NOT LIKE 'sys_gen_%' WHERE table_name NOT LIKE 'sys_gen_%'
AND table_name = :table_name; AND table_name = :table_name;
""" """
else: else:
sql = """ sql = """
SELECT t.tablename AS table_name, SELECT c.relname AS table_name, obj_description(c.oid) AS table_comment
pg_catalog.obj_description(t.tablename::regclass, 'pg_class') AS table_comment FROM pg_class c
FROM pg_tables t LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE t.tablename NOT LIKE 'sys_gen_%' WHERE c.relkind = 'r'
AND t.tablename = :table_name AND n.nspname = 'public' -- schema 通常是 'public'
AND t.schemaname = 'public'; -- schema 通常是 'public' AND c.relname NOT LIKE 'sys_gen_%';
""" """
stmt = text(sql).bindparams(table_name=table_name) stmt = text(sql).bindparams(table_name=table_name)
result = await db.execute(stmt) result = await db.execute(stmt)
@@ -6,11 +6,11 @@ from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy_crud_plus import CRUDPlus from sqlalchemy_crud_plus import CRUDPlus
from backend.plugin.code_generator.model import GenColumn from backend.plugin.code_generator.model import GenColumn
from backend.plugin.code_generator.schema.column import CreateGenModelParam, UpdateGenModelParam from backend.plugin.code_generator.schema.column import CreateGenColumnParam, UpdateGenColumnParam
class CRUDGenModel(CRUDPlus[GenColumn]): class CRUDGenColumn(CRUDPlus[GenColumn]):
"""代码生成模型 CRUD 类""" """代码生成模型 CRUD 类"""
async def get(self, db: AsyncSession, pk: int) -> GenColumn | None: async def get(self, db: AsyncSession, pk: int) -> GenColumn | None:
""" """
@@ -32,24 +32,24 @@ class CRUDGenModel(CRUDPlus[GenColumn]):
""" """
return await self.select_models_order(db, sort_columns='sort', gen_business_id=business_id) return await self.select_models_order(db, sort_columns='sort', gen_business_id=business_id)
async def create(self, db: AsyncSession, obj: CreateGenModelParam, pd_type: str | None) -> None: async def create(self, db: AsyncSession, obj: CreateGenColumnParam, pd_type: str | None) -> None:
""" """
创建代码生成模型 创建代码生成模型
:param db: 数据库会话 :param db: 数据库会话
:param obj: 创建代码生成模型参数 :param obj: 创建代码生成模型参数
:param pd_type: Pydantic 类型 :param pd_type: Pydantic 类型
:return: :return:
""" """
await self.create_model(db, obj, pd_type=pd_type) await self.create_model(db, obj, pd_type=pd_type)
async def update(self, db: AsyncSession, pk: int, obj: UpdateGenModelParam, pd_type: str | None) -> int: async def update(self, db: AsyncSession, pk: int, obj: UpdateGenColumnParam, pd_type: str | None) -> int:
""" """
更新代码生成模型 更新代码生成模型
:param db: 数据库会话 :param db: 数据库会话
:param pk: 代码生成模型 ID :param pk: 代码生成模型 ID
:param obj: 更新代码生成模型参数 :param obj: 更新代码生成模型参数
:param pd_type: Pydantic 类型 :param pd_type: Pydantic 类型
:return: :return:
""" """
@@ -57,13 +57,13 @@ class CRUDGenModel(CRUDPlus[GenColumn]):
async def delete(self, db: AsyncSession, pk: int) -> int: async def delete(self, db: AsyncSession, pk: int) -> int:
""" """
删除代码生成模型 删除代码生成模型
:param db: 数据库会话 :param db: 数据库会话
:param pk: 代码生成模型 ID :param pk: 代码生成模型 ID
:return: :return:
""" """
return await self.delete_model(db, pk) return await self.delete_model(db, pk)
gen_model_dao: CRUDGenModel = CRUDGenModel(GenColumn) gen_column_dao: CRUDGenColumn = CRUDGenColumn(GenColumn)
+2 -2
View File
@@ -3,7 +3,7 @@
from backend.common.enums import StrEnum from backend.common.enums import StrEnum
class GenModelMySQLColumnType(StrEnum): class GenMySQLColumnType(StrEnum):
"""代码生成模型列类型(MySQL""" """代码生成模型列类型(MySQL"""
# Python 类型映射 # Python 类型映射
@@ -68,7 +68,7 @@ class GenModelMySQLColumnType(StrEnum):
YEAR = 'int' YEAR = 'int'
class GenModelPostgreSQLColumnType(StrEnum): class GenPostgreSQLColumnType(StrEnum):
"""代码生成模型列类型(PostgreSQL)""" """代码生成模型列类型(PostgreSQL)"""
# Python 类型映射 # Python 类型映射
@@ -33,5 +33,5 @@ class GenBusiness(Base):
remark: Mapped[str | None] = mapped_column( remark: Mapped[str | None] = mapped_column(
LONGTEXT().with_variant(TEXT, 'postgresql'), default=None, comment='备注' LONGTEXT().with_variant(TEXT, 'postgresql'), default=None, comment='备注'
) )
# 代码生成业务模型一对多 # 代码生成业务模型一对多
gen_column: Mapped[list['GenColumn']] = relationship(init=False, back_populates='gen_business') gen_column: Mapped[list['GenColumn']] = relationship(init=False, back_populates='gen_business')
@@ -31,7 +31,7 @@ class GenColumn(DataClassBase):
is_pk: Mapped[bool] = mapped_column(default=False, comment='是否主键') is_pk: Mapped[bool] = mapped_column(default=False, comment='是否主键')
is_nullable: Mapped[bool] = mapped_column(default=False, comment='是否可为空') is_nullable: Mapped[bool] = mapped_column(default=False, comment='是否可为空')
# 代码生成业务模型一对多 # 代码生成业务模型一对多
gen_business_id: Mapped[int] = mapped_column( gen_business_id: Mapped[int] = mapped_column(
BigInteger, ForeignKey('gen_business.id', ondelete='CASCADE'), default=0, comment='代码生成业务ID' BigInteger, ForeignKey('gen_business.id', ondelete='CASCADE'), default=0, comment='代码生成业务ID'
) )
+1 -1
View File
@@ -1,6 +1,6 @@
[plugin] [plugin]
summary = '代码生成' summary = '代码生成'
version = '0.0.1' version = '0.0.4'
description = '生成通用业务代码' description = '生成通用业务代码'
author = 'wu-clan' author = 'wu-clan'
@@ -14,12 +14,12 @@ class GenBusinessSchemaBase(SchemaBase):
table_name: str = Field(description='表名称(英文)') table_name: str = Field(description='表名称(英文)')
doc_comment: str = Field(description='文档注释(用于函数/参数文档)') doc_comment: str = Field(description='文档注释(用于函数/参数文档)')
table_comment: str | None = Field(None, description='表描述') table_comment: str | None = Field(None, description='表描述')
class_name: str | None = Field(None, description='基础类名 (默认为英文表名称)') class_name: str | None = Field(None, description='用于 python 代码基础类名')
schema_name: str | None = Field(None, description='Schema 名称 (默认为英文表名称)') schema_name: str | None = Field(None, description='用于 python Schema 代码基础类名')
filename: str | None = Field(None, description='基础文件名(默认为英文表名称)') filename: str | None = Field(None, description='用于 python 代码基础文件名')
default_datetime_column: bool = Field(True, description='是否存在默认时间列') default_datetime_column: bool = Field(True, description='是否存在默认时间列')
api_version: str = Field('v1', description='代码生成 api 版本') api_version: str = Field('v1', description='代码生成 api 版本')
gen_path: str | None = Field(None, description='代码生成路径(默认为 app 根路径)') gen_path: str | None = Field(None, description='代码生成路径')
remark: str | None = Field(None, description='备注') remark: str | None = Field(None, description='备注')
@@ -6,7 +6,7 @@ from backend.common.schema import SchemaBase
from backend.plugin.code_generator.utils.type_conversion import sql_type_to_sqlalchemy from backend.plugin.code_generator.utils.type_conversion import sql_type_to_sqlalchemy
class GenModelSchemaBase(SchemaBase): class GenColumnSchemaBase(SchemaBase):
"""代码生成模型基础模型""" """代码生成模型基础模型"""
name: str = Field(description='列名称') name: str = Field(description='列名称')
@@ -26,16 +26,16 @@ class GenModelSchemaBase(SchemaBase):
return sql_type_to_sqlalchemy(v) return sql_type_to_sqlalchemy(v)
class CreateGenModelParam(GenModelSchemaBase): class CreateGenColumnParam(GenColumnSchemaBase):
"""创建代码生成模型参数""" """创建代码生成模型参数"""
class UpdateGenModelParam(GenModelSchemaBase): class UpdateGenColumnParam(GenColumnSchemaBase):
"""更新代码生成模型参数""" """更新代码生成模型参数"""
class GetGenModelDetail(GenModelSchemaBase): class GetGenColumnDetail(GenColumnSchemaBase):
"""获取代码生成模型详情""" """获取代码生成模型详情"""
model_config = ConfigDict(from_attributes=True) model_config = ConfigDict(from_attributes=True)
@@ -2,6 +2,8 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Sequence from typing import Sequence
from sqlalchemy import Select
from backend.common.exception import errors from backend.common.exception import errors
from backend.database.db import async_db_session from backend.database.db import async_db_session
from backend.plugin.code_generator.crud.crud_business import gen_business_dao from backend.plugin.code_generator.crud.crud_business import gen_business_dao
@@ -32,6 +34,16 @@ class GenBusinessService:
async with async_db_session() as db: async with async_db_session() as db:
return await gen_business_dao.get_all(db) return await gen_business_dao.get_all(db)
@staticmethod
async def get_select(*, table_name: str) -> Select:
"""
获取代码生成业务列表查询条件
:param table_name: 业务表名
:return:
"""
return await gen_business_dao.get_list(table_name=table_name)
@staticmethod @staticmethod
async def create(*, obj: CreateGenBusinessParam) -> None: async def create(*, obj: CreateGenBusinessParam) -> None:
""" """
@@ -10,19 +10,20 @@ from typing import Sequence
import aiofiles import aiofiles
from pydantic.alias_generators import to_pascal from pydantic.alias_generators import to_pascal
from sqlalchemy import RowMapping
from backend.common.exception import errors from backend.common.exception import errors
from backend.core.path_conf import BASE_PATH from backend.core.path_conf import BASE_PATH
from backend.database.db import async_db_session from backend.database.db import async_db_session
from backend.plugin.code_generator.crud.crud_business import gen_business_dao from backend.plugin.code_generator.crud.crud_business import gen_business_dao
from backend.plugin.code_generator.crud.crud_column import gen_model_dao from backend.plugin.code_generator.crud.crud_code import gen_dao
from backend.plugin.code_generator.crud.crud_gen import gen_dao from backend.plugin.code_generator.crud.crud_column import gen_column_dao
from backend.plugin.code_generator.model import GenBusiness from backend.plugin.code_generator.model import GenBusiness
from backend.plugin.code_generator.schema.business import CreateGenBusinessParam from backend.plugin.code_generator.schema.business import CreateGenBusinessParam
from backend.plugin.code_generator.schema.column import CreateGenModelParam from backend.plugin.code_generator.schema.code import ImportParam
from backend.plugin.code_generator.schema.gen import ImportParam from backend.plugin.code_generator.schema.column import CreateGenColumnParam
from backend.plugin.code_generator.service.column_service import gen_model_service from backend.plugin.code_generator.service.column_service import gen_column_service
from backend.plugin.code_generator.utils.gen_template import gen_template from backend.plugin.code_generator.utils.code_template import gen_template
from backend.plugin.code_generator.utils.type_conversion import sql_type_to_pydantic from backend.plugin.code_generator.utils.type_conversion import sql_type_to_pydantic
@@ -30,7 +31,7 @@ class GenService:
"""代码生成服务类""" """代码生成服务类"""
@staticmethod @staticmethod
async def get_tables(*, table_schema: str) -> Sequence[str]: async def get_tables(*, table_schema: str) -> Sequence[RowMapping]:
""" """
获取指定 schema 下的所有表名 获取指定 schema 下的所有表名
@@ -43,7 +44,7 @@ class GenService:
@staticmethod @staticmethod
async def import_business_and_model(*, obj: ImportParam) -> None: async def import_business_and_model(*, obj: ImportParam) -> None:
""" """
导入业务和模型数据 导入业务和模型数据
:param obj: 导入参数对象 :param obj: 导入参数对象
:return: :return:
@@ -64,8 +65,8 @@ class GenService:
table_name=table_name, table_name=table_name,
doc_comment=table_info[1] or table_name.split('_')[-1], doc_comment=table_info[1] or table_name.split('_')[-1],
table_comment=table_info[1], table_comment=table_info[1],
class_name=table_name, class_name=to_pascal(table_name),
schema_name=table_name, schema_name=to_pascal(table_name),
filename=table_name, filename=table_name,
).model_dump() ).model_dump()
) )
@@ -76,9 +77,9 @@ class GenService:
for column in column_info: for column in column_info:
column_type = column[-1].split('(')[0].upper() column_type = column[-1].split('(')[0].upper()
pd_type = sql_type_to_pydantic(column_type) pd_type = sql_type_to_pydantic(column_type)
await gen_model_dao.create( await gen_column_dao.create(
db, db,
CreateGenModelParam( CreateGenColumnParam(
name=column[0], name=column[0],
comment=column[-2], comment=column[-2],
type=column_type, type=column_type,
@@ -99,7 +100,7 @@ class GenService:
:param business: 业务对象 :param business: 业务对象
:return: :return:
""" """
gen_models = await gen_model_service.get_models(business_id=business.id) gen_models = await gen_column_service.get_columns(business_id=business.id)
if not gen_models: if not gen_models:
raise errors.NotFoundError(msg='代码生成模型表为空') raise errors.NotFoundError(msg='代码生成模型表为空')
@@ -124,9 +125,22 @@ class GenService:
tpl_code_map = await self.render_tpl_code(business=business) tpl_code_map = await self.render_tpl_code(business=business)
codes = {} codes = {}
for tpl, code in tpl_code_map.items(): for tpl_path, code in tpl_code_map.items():
if tpl.startswith('python'): if tpl_path.startswith('python'):
codes[tpl.replace('.jinja', '.py').split('/')[-1]] = code.encode('utf-8') rootpath = f'fastapi_best_architecture/backend/app/{business.app_name}'
template_name = tpl_path.split('/')[-1]
match template_name:
case 'api.jinja':
filepath = f'{rootpath}/api/{business.api_version}/{business.filename}.py'
case 'crud.jinja':
filepath = f'{rootpath}/crud/crud_{business.filename}.py'
case 'model.jinja':
filepath = f'{rootpath}/model/{business.filename}.py'
case 'schema.jinja':
filepath = f'{rootpath}/schema/{business.filename}.py'
case 'service.jinja':
filepath = f'{rootpath}/service/{business.filename}_service.py'
codes[filepath] = code.encode('utf-8')
return codes return codes
@@ -143,12 +157,12 @@ class GenService:
if not business: if not business:
raise errors.NotFoundError(msg='业务不存在') raise errors.NotFoundError(msg='业务不存在')
gen_path = business.gen_path or 'fba-backend-app-dir' gen_path = business.gen_path or '.../backend/app/'
target_files = gen_template.get_code_gen_paths(business) target_files = gen_template.get_code_gen_paths(business)
return [os.path.join(gen_path, *target_file.split('/')) for target_file in target_files] return [os.path.join(gen_path, *target_file.split('/')) for target_file in target_files]
async def generate(self, *, pk: int) -> None: async def generate(self, *, pk: int) -> str:
""" """
生成代码文件 生成代码文件
@@ -203,6 +217,8 @@ class GenService:
async with aiofiles.open(code_filepath, 'w', encoding='utf-8') as f: async with aiofiles.open(code_filepath, 'w', encoding='utf-8') as f:
await f.write(code) await f.write(code)
return gen_path
async def download(self, *, pk: int) -> io.BytesIO: async def download(self, *, pk: int) -> io.BytesIO:
""" """
下载生成的代码 下载生成的代码
@@ -4,93 +4,93 @@ from typing import Sequence
from backend.common.exception import errors from backend.common.exception import errors
from backend.database.db import async_db_session from backend.database.db import async_db_session
from backend.plugin.code_generator.crud.crud_column import gen_model_dao from backend.plugin.code_generator.crud.crud_column import gen_column_dao
from backend.plugin.code_generator.enums import GenModelMySQLColumnType from backend.plugin.code_generator.enums import GenMySQLColumnType
from backend.plugin.code_generator.model import GenColumn from backend.plugin.code_generator.model import GenColumn
from backend.plugin.code_generator.schema.column import CreateGenModelParam, UpdateGenModelParam from backend.plugin.code_generator.schema.column import CreateGenColumnParam, UpdateGenColumnParam
from backend.plugin.code_generator.utils.type_conversion import sql_type_to_pydantic from backend.plugin.code_generator.utils.type_conversion import sql_type_to_pydantic
class GenModelService: class GenColumnService:
"""代码生成模型服务类""" """代码生成模型服务类"""
@staticmethod @staticmethod
async def get(*, pk: int) -> GenColumn: async def get(*, pk: int) -> GenColumn:
""" """
获取指定 ID 的模型 获取指定 ID 的模型
:param pk: 模型 ID :param pk: 模型 ID
:return: :return:
""" """
async with async_db_session() as db: async with async_db_session() as db:
model = await gen_model_dao.get(db, pk) column = await gen_column_dao.get(db, pk)
if not model: if not column:
raise errors.NotFoundError(msg='代码生成模型不存在') raise errors.NotFoundError(msg='代码生成模型不存在')
return model return column
@staticmethod @staticmethod
async def get_types() -> list[str]: async def get_types() -> list[str]:
"""获取所有 MySQL 列类型""" """获取所有 MySQL 列类型"""
types = GenModelMySQLColumnType.get_member_keys() types = GenMySQLColumnType.get_member_keys()
types.sort() types.sort()
return types return types
@staticmethod @staticmethod
async def get_models(*, business_id: int) -> Sequence[GenColumn]: async def get_columns(*, business_id: int) -> Sequence[GenColumn]:
""" """
获取指定业务的所有模型 获取指定业务的所有模型
:param business_id: 业务 ID :param business_id: 业务 ID
:return: :return:
""" """
async with async_db_session() as db: async with async_db_session() as db:
return await gen_model_dao.get_all_by_business(db, business_id) return await gen_column_dao.get_all_by_business(db, business_id)
@staticmethod @staticmethod
async def create(*, obj: CreateGenModelParam) -> None: async def create(*, obj: CreateGenColumnParam) -> None:
""" """
创建模型 创建模型
:param obj: 创建模型参数 :param obj: 创建模型参数
:return: :return:
""" """
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
gen_models = await gen_model_dao.get_all_by_business(db, obj.gen_business_id) gen_columns = await gen_column_dao.get_all_by_business(db, obj.gen_business_id)
if obj.name in [gen_model.name for gen_model in gen_models]: if obj.name in [gen_column.name for gen_column in gen_columns]:
raise errors.ForbiddenError(msg='禁止添加相同列到同一模型表') raise errors.ForbiddenError(msg='模型列已存在')
pd_type = sql_type_to_pydantic(obj.type) pd_type = sql_type_to_pydantic(obj.type)
await gen_model_dao.create(db, obj, pd_type=pd_type) await gen_column_dao.create(db, obj, pd_type=pd_type)
@staticmethod @staticmethod
async def update(*, pk: int, obj: UpdateGenModelParam) -> int: async def update(*, pk: int, obj: UpdateGenColumnParam) -> int:
""" """
更新模型 更新模型
:param pk: 模型 ID :param pk: 模型 ID
:param obj: 更新模型参数 :param obj: 更新模型参数
:return: :return:
""" """
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
model = await gen_model_dao.get(db, pk) column = await gen_column_dao.get(db, pk)
if obj.name != model.name: if obj.name != column.name:
gen_models = await gen_model_dao.get_all_by_business(db, obj.gen_business_id) gen_columns = await gen_column_dao.get_all_by_business(db, obj.gen_business_id)
if obj.name in [gen_model.name for gen_model in gen_models]: if obj.name in [gen_column.name for gen_column in gen_columns]:
raise errors.ConflictError(msg='模型列名已存在') raise errors.ConflictError(msg='模型列名已存在')
pd_type = sql_type_to_pydantic(obj.type) pd_type = sql_type_to_pydantic(obj.type)
return await gen_model_dao.update(db, pk, obj, pd_type=pd_type) return await gen_column_dao.update(db, pk, obj, pd_type=pd_type)
@staticmethod @staticmethod
async def delete(*, pk: int) -> int: async def delete(*, pk: int) -> int:
""" """
删除模型 删除模型
:param pk: 模型 ID :param pk: 模型 ID
:return: :return:
""" """
async with async_db_session.begin() as db: async with async_db_session.begin() as db:
return await gen_model_dao.delete(db, pk) return await gen_column_dao.delete(db, pk)
gen_model_service: GenModelService = GenModelService() gen_column_service: GenColumnService = GenColumnService()
@@ -23,7 +23,7 @@ class CRUD{{ class_name }}(CRUDPlus[{{ schema_name }}]):
async def get_list(self) -> Select: async def get_list(self) -> Select:
"""获取{{ doc_comment }}列表""" """获取{{ doc_comment }}列表"""
return await self.select_order('created_time', 'desc') return await self.select_order('id', 'desc')
async def get_all(self, db: AsyncSession) -> Sequence[{{ class_name }}]: async def get_all(self, db: AsyncSession) -> Sequence[{{ class_name }}]:
""" """
@@ -66,4 +66,4 @@ class CRUD{{ class_name }}(CRUDPlus[{{ schema_name }}]):
return await self.delete_model_by_column(db, allow_multiple=True, id__in=pks) return await self.delete_model_by_column(db, allow_multiple=True, id__in=pks)
{{ instance_name }}_dao: CRUD{{ class_name }} = CRUD{{ class_name }}({{ class_name }}) {{ table_name }}_dao: CRUD{{ class_name }} = CRUD{{ class_name }}({{ class_name }})
@@ -1,6 +1,9 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
{% if default_datetime_column %}
from datetime import datetime from datetime import datetime
{% endif %}
from uuid import UUID from uuid import UUID
import sqlalchemy as sa import sqlalchemy as sa
@@ -12,10 +15,10 @@ from sqlalchemy.dialects import postgresql
{% endif -%} {% endif -%}
from sqlalchemy.orm import Mapped, mapped_column from sqlalchemy.orm import Mapped, mapped_column
from backend.common.model import {% if default_datetime_column %}Base{% else %}MappedBase{% endif %}, id_key from backend.common.model import {% if default_datetime_column %}Base{% else %}DataClassBase{% endif %}, id_key
class {{ class_name }}({% if default_datetime_column %}Base{% else %}MappedBase{% endif %}): class {{ class_name }}({% if default_datetime_column %}Base{% else %}DataClassBase{% endif %}):
"""{{ table_comment }}""" """{{ table_comment }}"""
__tablename__ = '{{ table_name }}' __tablename__ = '{{ table_name }}'
@@ -1,7 +1,9 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
{% if default_datetime_column %}
from datetime import datetime from datetime import datetime
{% endif %}
from pydantic import ConfigDict, Field from pydantic import ConfigDict, Field
from backend.common.schema import SchemaBase from backend.common.schema import SchemaBase
@@ -75,4 +75,4 @@ class {{ class_name }}Service:
return count return count
{{ instance_name }}_service: {{ class_name }}Service = {{ class_name }}Service() {{ table_name }}_service: {{ class_name }}Service = {{ class_name }}Service()
@@ -3,7 +3,6 @@
from typing import Sequence from typing import Sequence
from jinja2 import Environment, FileSystemLoader, Template, select_autoescape from jinja2 import Environment, FileSystemLoader, Template, select_autoescape
from pydantic.alias_generators import to_pascal, to_snake
from backend.core.conf import settings from backend.core.conf import settings
from backend.plugin.code_generator.model import GenBusiness, GenColumn from backend.plugin.code_generator.model import GenBusiness, GenColumn
@@ -39,13 +38,13 @@ class GenTemplate:
:return: :return:
""" """
files = [] return [
'python/api.jinja',
# python 'python/crud.jinja',
python_template_path = JINJA2_TEMPLATE_DIR / 'python' 'python/model.jinja',
files.extend([f'python/{file.name}' for file in python_template_path.iterdir() if file.is_file()]) 'python/schema.jinja',
'python/service.jinja',
return files ]
@staticmethod @staticmethod
def get_code_gen_paths(business: GenBusiness) -> list[str]: def get_code_gen_paths(business: GenBusiness) -> list[str]:
@@ -73,8 +72,7 @@ class GenTemplate:
:param business: 代码生成业务对象 :param business: 代码生成业务对象
:return: :return:
""" """
target_files = self.get_code_gen_paths(business) code_gen_path_mapping = dict(zip(self.get_template_files(), self.get_code_gen_paths(business)))
code_gen_path_mapping = dict(zip(self.get_template_files(), target_files))
return code_gen_path_mapping[tpl_path] return code_gen_path_mapping[tpl_path]
@staticmethod @staticmethod
@@ -88,12 +86,11 @@ class GenTemplate:
""" """
return { return {
'app_name': business.app_name, 'app_name': business.app_name,
'table_name': to_snake(business.table_name), 'table_name': business.table_name,
'doc_comment': business.doc_comment, 'doc_comment': business.doc_comment,
'table_comment': business.table_comment, 'table_comment': business.table_comment,
'class_name': to_pascal(business.class_name), 'class_name': business.class_name,
'instance_name': to_snake(business.class_name), 'schema_name': business.schema_name,
'schema_name': to_pascal(business.schema_name),
'default_datetime_column': business.default_datetime_column, 'default_datetime_column': business.default_datetime_column,
'permission': str(business.table_name.replace('_', ':')), 'permission': str(business.table_name.replace('_', ':')),
'database_type': settings.DATABASE_TYPE, 'database_type': settings.DATABASE_TYPE,
@@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from backend.core.conf import settings from backend.core.conf import settings
from backend.plugin.code_generator.enums import GenModelMySQLColumnType, GenModelPostgreSQLColumnType from backend.plugin.code_generator.enums import GenMySQLColumnType, GenPostgreSQLColumnType
def sql_type_to_sqlalchemy(typing: str) -> str: def sql_type_to_sqlalchemy(typing: str) -> str:
@@ -12,10 +12,10 @@ def sql_type_to_sqlalchemy(typing: str) -> str:
:return: :return:
""" """
if settings.DATABASE_TYPE == 'mysql': if settings.DATABASE_TYPE == 'mysql':
if typing in GenModelMySQLColumnType.get_member_keys(): if typing in GenMySQLColumnType.get_member_keys():
return typing return typing
else: else:
if typing in GenModelPostgreSQLColumnType.get_member_keys(): if typing in GenPostgreSQLColumnType.get_member_keys():
return typing return typing
return 'String' return 'String'
@@ -29,9 +29,9 @@ def sql_type_to_pydantic(typing: str) -> str:
""" """
try: try:
if settings.DATABASE_TYPE == 'mysql': if settings.DATABASE_TYPE == 'mysql':
return GenModelMySQLColumnType[typing].value return GenMySQLColumnType[typing].value
if typing == 'CHARACTER VARYING': # postgresql 中 DDL VARCHAR 的别名 if typing == 'CHARACTER VARYING': # postgresql 中 DDL VARCHAR 的别名
return 'str' return 'str'
return GenModelPostgreSQLColumnType[typing].value return GenPostgreSQLColumnType[typing].value
except KeyError: except KeyError:
return 'str' return 'str'
@@ -14,12 +14,21 @@ from backend.plugin.config.schema.config import (
CreateConfigParam, CreateConfigParam,
GetConfigDetail, GetConfigDetail,
UpdateConfigParam, UpdateConfigParam,
UpdateConfigsParam,
) )
from backend.plugin.config.service.config_service import config_service from backend.plugin.config.service.config_service import config_service
router = APIRouter() router = APIRouter()
@router.get('/all', summary='获取所有参数配置', dependencies=[DependsJwtAuth])
async def get_all_configs(
type: Annotated[str | None, Query(description='参数配置类型')] = None,
) -> ResponseSchemaModel[list[GetConfigDetail]]:
configs = await config_service.get_all(type=type)
return response_base.success(data=configs)
@router.get('/{pk}', summary='获取参数配置详情', dependencies=[DependsJwtAuth]) @router.get('/{pk}', summary='获取参数配置详情', dependencies=[DependsJwtAuth])
async def get_config(pk: Annotated[int, Path(description='参数配置 ID')]) -> ResponseSchemaModel[GetConfigDetail]: async def get_config(pk: Annotated[int, Path(description='参数配置 ID')]) -> ResponseSchemaModel[GetConfigDetail]:
config = await config_service.get(pk=pk) config = await config_service.get(pk=pk)
@@ -57,6 +66,14 @@ async def create_config(obj: CreateConfigParam) -> ResponseModel:
return response_base.success() return response_base.success()
@router.put('', summary='批量更新参数配置', dependencies=[Depends(RequestPermission('sys.config.edits')), DependsRBAC])
async def bulk_update_config(objs: list[UpdateConfigsParam]) -> ResponseModel:
count = await config_service.bulk_update(objs=objs)
if count > 0:
return response_base.success()
return response_base.fail()
@router.put( @router.put(
'/{pk}', '/{pk}',
summary='更新参数配置', summary='更新参数配置',
+21
View File
@@ -1,5 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
from typing import Sequence
from sqlalchemy import Select from sqlalchemy import Select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
@@ -22,6 +23,16 @@ class CRUDConfig(CRUDPlus[Config]):
""" """
return await self.select_model_by_column(db, id=pk) return await self.select_model_by_column(db, id=pk)
async def get_all(self, db: AsyncSession, type: str) -> Sequence[Config | None]:
"""
通过键名获取参数配置
:param db: 数据库会话
:param type: 参数配置类型
:return:
"""
return await self.select_models(db, type=type)
async def get_by_key(self, db: AsyncSession, key: str) -> Config | None: async def get_by_key(self, db: AsyncSession, key: str) -> Config | None:
""" """
通过键名获取参数配置 通过键名获取参数配置
@@ -70,6 +81,16 @@ class CRUDConfig(CRUDPlus[Config]):
""" """
return await self.update_model(db, pk, obj) return await self.update_model(db, pk, obj)
async def bulk_update(self, db: AsyncSession, objs: list[UpdateConfigParam]) -> int:
"""
批量更新参数配置
:param db: 数据库会话
:param objs: 批量更新参数配置参数
:return:
"""
return await self.bulk_update_models(db, objs)
async def delete(self, db: AsyncSession, pks: list[int]) -> int: async def delete(self, db: AsyncSession, pks: list[int]) -> int:
""" """
批量删除参数配置 批量删除参数配置
+2 -2
View File
@@ -1,7 +1,7 @@
[plugin] [plugin]
summary = '参数配置' summary = '参数配置'
version = '0.0.1' version = '0.0.2'
description = '通常用于前端工程数据展示' description = '通常用于动态配置系统参数/前端工程数据展示'
author = 'wu-clan' author = 'wu-clan'
[app] [app]
+6
View File
@@ -26,6 +26,12 @@ class UpdateConfigParam(ConfigSchemaBase):
"""更新参数配置参数""" """更新参数配置参数"""
class UpdateConfigsParam(UpdateConfigParam):
"""批量更新参数配置参数"""
id: int = Field(description='参数配置 ID')
class GetConfigDetail(ConfigSchemaBase): class GetConfigDetail(ConfigSchemaBase):
"""参数配置详情""" """参数配置详情"""
@@ -10,6 +10,7 @@ from backend.plugin.config.model import Config
from backend.plugin.config.schema.config import ( from backend.plugin.config.schema.config import (
CreateConfigParam, CreateConfigParam,
UpdateConfigParam, UpdateConfigParam,
UpdateConfigsParam,
) )
@@ -30,6 +31,17 @@ class ConfigService:
raise errors.NotFoundError(msg='参数配置不存在') raise errors.NotFoundError(msg='参数配置不存在')
return config return config
@staticmethod
async def get_all(*, type: str | None):
"""
获取所有参数配置
:param type: 参数配置类型
:return:
"""
async with async_db_session() as db:
return await config_dao.get_all(db, type)
@staticmethod @staticmethod
async def get_select(*, name: str | None, type: str | None) -> Select: async def get_select(*, name: str | None, type: str | None) -> Select:
""" """
@@ -75,6 +87,27 @@ class ConfigService:
count = await config_dao.update(db, pk, obj) count = await config_dao.update(db, pk, obj)
return count return count
@staticmethod
async def bulk_update(*, objs: list[UpdateConfigsParam]) -> int:
"""
批量更新参数配置
:param objs: 参数配置批量更新参数
:return:
"""
async with async_db_session.begin() as db:
for batch in range(0, len(objs), 1000):
for obj in objs:
config = await config_dao.get(db, obj.id)
if not config:
raise errors.NotFoundError(msg='参数配置不存在')
if config.key != obj.key:
config = await config_dao.get_by_key(db, obj.key)
if config:
raise errors.ConflictError(msg=f'参数配置 {obj.key} 已存在')
count = await config_dao.bulk_update(db, objs)
return count
@staticmethod @staticmethod
async def delete(*, pks: list[int]) -> int: async def delete(*, pks: list[int]) -> int:
""" """
+8
View File
@@ -0,0 +1,8 @@
insert into sys_config (id, name, type, `key`, value, is_frontend, remark, created_time, updated_time)
values
(1, '状态', 'EMAIL', 'EMAIL_STATUS', '1', 0, null, now(), null),
(2, '服务器地址', 'EMAIL', 'EMAIL_HOST', 'smtp.qq.com', 0, null, now(), null),
(3, '服务器端口', 'EMAIL', 'EMAIL_PORT', '465', 0, null, now(), null),
(4, '邮箱账号', 'EMAIL', 'EMAIL_USERNAME', 'fba@qq.com', 0, null, now(), null),
(5, '邮箱密码', 'EMAIL', 'EMAIL_PASSWORD', '', 0, null, now(), null),
(6, 'SSL 加密', 'EMAIL', 'EMAIL_SSL', '1', 0, null, now(), null),
@@ -0,0 +1,8 @@
insert into sys_config (id, name, type, `key`, value, is_frontend, remark, created_time, updated_time)
values
(2069061886627938304, '状态', 'EMAIL', 'EMAIL_STATUS', '1', 0, null, now(), null),
(2069061886627938305, '服务器地址', 'EMAIL', 'EMAIL_HOST', 'smtp.qq.com', 0, null, now(), null),
(2069061886627938306, '服务器端口', 'EMAIL', 'EMAIL_PORT', '465', 0, null, now(), null),
(2069061886627938307, '邮箱账号', 'EMAIL', 'EMAIL_USERNAME', 'fba@qq.com', 0, null, now(), null),
(2069061886627938308, '邮箱密码', 'EMAIL', 'EMAIL_PASSWORD', '', 0, null, now(), null),
(2069061886627938309, 'SSL 加密', 'EMAIL', 'EMAIL_SSL', '1', 0, null, now(), null),
@@ -0,0 +1,11 @@
insert into sys_config (id, name, type, "key", value, is_frontend, remark, created_time, updated_time)
values
(1, '状态', 'EMAIL', 'EMAIL_STATUS', '1', 0, null, now(), null),
(2, '服务器地址', 'EMAIL', 'EMAIL_HOST', 'smtp.qq.com', 0, null, now(), null),
(3, '服务器端口', 'EMAIL', 'EMAIL_PORT', '465', 0, null, now(), null),
(4, '邮箱账号', 'EMAIL', 'EMAIL_USERNAME', 'fba@qq.com', 0, null, now(), null),
(5, '邮箱密码', 'EMAIL', 'EMAIL_PASSWORD', '', 0, null, now(), null),
(6, 'SSL 加密', 'EMAIL', 'EMAIL_SSL', '1', 0, null, now(), null),
-- reset auto-increment values for each table based on max id
select setval(pg_get_serial_sequence('sys_config', 'id'),coalesce(max(id), 0) + 1, true) from sys_config;
@@ -0,0 +1,8 @@
insert into sys_config (id, name, type, "key", value, is_frontend, remark, created_time, updated_time)
values
(2069061886627938304, '状态', 'EMAIL', 'EMAIL_STATUS', '1', 0, null, now(), null),
(2069061886627938305, '服务器地址', 'EMAIL', 'EMAIL_HOST', 'smtp.qq.com', 0, null, now(), null),
(2069061886627938306, '服务器端口', 'EMAIL', 'EMAIL_PORT', '465', 0, null, now(), null),
(2069061886627938307, '邮箱账号', 'EMAIL', 'EMAIL_USERNAME', 'fba@qq.com', 0, null, now(), null),
(2069061886627938308, '邮箱密码', 'EMAIL', 'EMAIL_PASSWORD', '', 0, null, now(), null),
(2069061886627938309, 'SSL 加密', 'EMAIL', 'EMAIL_SSL', '1', 0, null, now(), null),
@@ -21,6 +21,12 @@ from backend.plugin.dict.service.dict_type_service import dict_type_service
router = APIRouter() router = APIRouter()
@router.get('/all', summary='获取所有字典数据', dependencies=[DependsJwtAuth])
async def get_all_dict_types() -> ResponseSchemaModel[list[GetDictTypeDetail]]:
data = await dict_type_service.get_all()
return response_base.success(data=data)
@router.get('/{pk}', summary='获取字典类型详情', dependencies=[DependsJwtAuth]) @router.get('/{pk}', summary='获取字典类型详情', dependencies=[DependsJwtAuth])
async def get_dict_type( async def get_dict_type(
pk: Annotated[int, Path(description='字典类型 ID')], pk: Annotated[int, Path(description='字典类型 ID')],

Some files were not shown because too many files have changed in this diff Show More