A second instance pointed at the same database did not divide the work, it
overwrote it. Every instance registered the whole enabled list into its own
cron.Cron, and startup ran `UPDATE sys_job SET entry_id = 0 WHERE entry_id
> 0` across the entire table, so the newest process erased the ids the
previous one wrote and put its own over the top. Neither symptom logged
anything: an enabled job fired once per instance, and stopping one from the
UI removed an entry from the wrong process and answered 200 either way.
A supervisor per tenant now keeps the scheduler in step with the lease.
Holding it is not decided once at startup, because both of the other
answers are wrong for longer than a moment:
- an instance that never got the lease keeps asking, so the death of the
holder does not stop the jobs until somebody restarts a process by hand;
- an instance that holds it stops scheduling as soon as its lease has
lapsed, because a holder still scheduling after the lease has gone
elsewhere is the two-schedulers defect reached from the other side.
A failed renewal is not a lost lease. The scheduler keeps running until the
lease could actually have expired: a database that is briefly unreachable
must not stop the jobs, and cannot have handed them to anyone else, because
nobody else can reach it to take the lease either.
Stopping is no longer arranged by startCrontab. A scheduler now stops for
two different reasons and only the supervisor knows which - and since a
start happens every time the lease is taken, registering a shutdown
callback there would add one per leadership change for the life of the
process, because SetShutdown appends.
sys_job.entry_id keeps its meaning. This does not distribute the jobs and is
not meant to: the HTTP side scales, the scheduler stays single-writer.
Fixes#915.
One row per database, taken and renewed by single UPDATE statements whose
RowsAffected the database decides. Nothing uses it yet; the scheduler is
wired to it next.
The two timestamps are epoch milliseconds in a BIGINT rather than timestamp
columns, which is the one decision here worth explaining. A timestamp does
not survive the trip through a driver unchanged: read over go-admin's own
`parseTime=True&loc=Local` DSN, MySQL's UTC_TIMESTAMP arrives relabelled as
local time, and on a UTC+8 host every lease is eight hours out. It is
invisible to any test that compares the lease against itself, because each
instance's own arithmetic stays self-consistent - only the comparison
between two instances is wrong, which is the only comparison that matters.
An integer has no timezone for a driver to apply.
The migration seeds the row free. There is no insert path at runtime, so two
instances starting together cannot race to create the row they are both
trying to claim, and neither has to tell a duplicate-key error apart from a
real one in whichever driver it is running against. A missing row is
therefore reported rather than recovered from: silently never scheduling
anywhere is the worse failure.
The current-time expression differs per dialect and all four are covered:
MySQL, PostgreSQL and SQL Server against real servers, SQLite by default.
#924 buffered the channel Remove returns, so the goroutine it starts can
finish when the caller has already timed out and walked away. That half of
the patch was not covered: reverting the buffer on its own left the whole
suite green, while the timeout assertion it shipped with stayed red only for
the error-return half.
This test abandons twenty channels and then requires every goroutine to be
gone. It waits for the scheduler to empty first, so a goroutine that has not
yet reached its send cannot make the test pass while proving nothing.
Without the buffer: 20 of 20 parked on `chan send` at jobbase.go:216.
The per-tenant setup ended with `defer crontab.Stop()` on the line above
`select {}`. The select never returned, so the deferred call was unreachable
for the life of the process: the scheduler had never once been stopped. And
because setup never returned, the `for k, db := range dbs` loop in Setup never
reached its second iteration - with several tenant databases configured, only
whichever one came first out of the map ever got a scheduler at all.
Both fall out of deleting the select, which was blocking for nothing: cron.Start
is `go c.run()` and has never needed anything to hold the caller.
The stop becomes a BeforeExit callback. cron.Stop returns a context that closes
once the jobs already running have finished, so the shutdown budget has
something real to bound - and giving up on that wait leaves those jobs running
until the process exits, which is better than holding the whole shutdown open
for one job that will not end.
Startup moves from a bare goroutine in run() onto AfterListen. Two reasons: the
phase runs behind core's panic guard, which does not reach across a goroutine
boundary, so a panic while loading jobs used to take the process down; and the
jobs it starts can call the API, which is only true once the socket is
accepting. It can be synchronous now precisely because setup returns.
Tested where it can be: startCrontab is split out so a scheduler can be started
with no database in sight. The job runs every second; after RunShutdown, two
and a half seconds of silence is the assertion. The counter-proof - registering
no callback, which is what this commit replaces - compiles and reports "the job
fired 2 more times after shutdown".
There is one test, not several, because BeforeExit closes to further
registration once it has run; a second RunShutdown in the same binary would
find an empty registry and pass while proving nothing.
**The multi-tenant half has no test.** setup needs a *gorm.DB per tenant before
it reaches the line that was blocking, and this repository's CI has no database
- `make build` is CGO_ENABLED=0 with no sqlite tag. It is the same defect
though: the loop could not advance past a call that never returned.
Four modules each called AuthInit and built their own instance, so which
one Runtime handed back was decided by whichever module initialised last.
The JwtToken key was also registered as an unbound method expression,
which GetHandlerFunc's type assertion can never match - the key was
registered and unusable at the same time.
The instance is now built once here and registered as a bound closure.
Modules read it back through GetAuthMiddleware, which is fatal rather
than nil when called before InitMiddleware has run: a process without a
JWT middleware should not reach the point of serving a request.
Only one call site needs the instance itself rather than the handler
(admin's /login, for LoginHandler); the thirty-odd MiddlewareFunc() call
sites are unchanged.
Claude-Session: https://claude.ai/code/session_01HPTAw8b8tAdFNFn8rKdPYx
Every import of the module changes, not only the seven packages that
moved out of sdk/pkg: Go requires the major version in the path from v2
on. Both happen in one pass —
go run github.com/go-admin-team/go-admin-core/tools/coreupgrade@v2.0.0 -w -v2 .
go mod tidy
— which is the command the release notes give, run here as a consumer
would run it. 210 imports across 95 files.
The compatibility shims this used are gone in v2, so the paths that
moved had to move: sdk/pkg/captcha, sdk/pkg/jwtauth and its user
package, sdk/pkg/response and sdk/pkg/casbin.
The count of unformatted files is unchanged at 34, none of them touched
by this: the tool reformats a file only if it was already gofmt clean,
so a migration cannot disappear into whitespace.
The pinned core dated from April, before sdk stopped being a separate module,
so the build resolved sdk packages from the old module and core packages from
the new one. Dropping the separate requirement is what makes the two agree
again.
Most of the diff is renames that came with that: the tenant accessors gained a
ByTenant suffix, GetDb now returns one database and GetAllDb the map, and
casbin moved to v3.
The change that matters is four call sites moving from GetMemoryQueue to
GetQueuePrefix. GetMemoryQueue returns a queue fixed at construction, so the
login log, the operate log and the api check ran in process no matter what the
settings file selected — a second instance saw none of it. GetQueuePrefix
returns whatever the configuration built, which is the point of being able to
configure a queue at all.
Verified against core at main: build and vet clean. The two file_store failures
are unchanged from before this branch; they need cloud credentials.
wenjianzhang