Files
go-admin/cmd/api/queue_test.go
zhangwenjian 4510b06959 fix🐛: register the queue consumers before the queue is started
setupQueue ended with `go queueAdapter.Run()`, and the three log consumers were
registered afterwards, from setup(). The contract implementations refuse a
registration once the queue is running - memqueue and the redis queue both
answer storage.ErrQueueAlreadyStarted - and Register cannot report it: it
returns nothing, which its own comment in core records as the reason the
interface is deprecated. Start first and register second, across two
goroutines, and the registration is dropped without the caller being able to
tell.

What follows is not quiet. No consumer group was created, so redis refuses
every later publish with storage.ErrNoHandler, and go-admin logs that at error
level from both call sites while the login and operation log rows are simply
never written. The silence is in the registration; the cost shows up on every
request after it.

Which implementation is behind the interface depends on the configuration.
config.QueueConfig.Setup returns queue.NewMemory directly when there is no
redis section - and that one does not care about the order, because its
Register just starts another consumer goroutine. Only a redis section reaches
storage.LegacyQueueAdapter, which wraps the contract implementation and
therefore refuses. So the defect is invisible in the default deployment and
shows up only where redis is configured, dropping the login log, the operation
log and the api check - the three things #892 was about.

The start therefore moves to the code that registers, and nothing starts the
queue but that.

The registration also moves onto AfterResource. It has to: a reload rebuilds
the adapter, and consumers attached to the one that existed at start-up are
attached to a queue nobody publishes to any more. Being on that phase means
running again on every reload, so the callback is idempotent with respect to a
given queue rather than "does nothing the second time" - registering twice on
the same queue would give every message two consumers and write every log row
twice.

Identity for that comes from common/storage, where the adapter is built, as a
generation counter. It cannot come from the accessors: GetQueueAdapter and
GetQueuePrefix build a fresh runtime.Queue wrapper on every call, so comparing
two of them compares two wrappers and never matches however many times the
adapter underneath has been replaced. A counter also keeps the comparison on a
uint64 rather than an `==` between two interface values, which would panic on
an adapter type that is not comparable.

Generation 0 means the configuration has no queue section, so nothing was
installed and the runtime hands back its own memory queue. That case still gets
consumers, because the registration this replaces was unconditional and
dropping it would stop the logs for anyone who commented the section out.

Two things fixed on the way past:

  - `if q := sdk.Runtime.GetQueueAdapter(); q != nil { q.Shutdown() }` was
    always true. GetQueueAdapter never returns nil - with nothing configured
    the runtime falls back to its own memory queue and wraps that - so the
    first start shut down the fallback queue before anything had used it. Only
    an adapter this package installed is shut down now.
  - config.Setup becomes bootstrap.SetupConfig, which is what announces
    AfterResource, and announces it after the callbacks that build the
    resources rather than before.

attachConsumersOnce is split out so the order and the once-ness can be checked
against a queue the test controls; neither can be read back out of a real
adapter. Four tests cover the ordering, both directions of the idempotency
rule, and the unconfigured case. Both counter-proofs compile and fail: calling
Run before the registrations reports each of the three as "came after Run", and
dropping the generation guard reports eight calls where four are wanted.

One honest limit: the counter-proof for the ordering makes Run synchronous.
The original arrangement started the queue on another goroutine, and a race
cannot be made to fail every time - which is the reason the order is enforced
by structure here instead of being left to be noticed in use.
2026-09-05 22:31:12 +08:00

168 lines
4.9 KiB
Go

package api
import (
"strings"
"sync"
"testing"
"time"
corestorage "github.com/go-admin-team/go-admin-core/v2/storage"
)
// recordingQueue records what was done to it, in order. Register and Run are
// the two calls whose order is the point of this file; Append and Shutdown are
// here to satisfy the interface.
type recordingQueue struct {
mu sync.Mutex
events []string
ran chan struct{}
}
func newRecordingQueue() *recordingQueue {
return &recordingQueue{ran: make(chan struct{}, 4)}
}
func (q *recordingQueue) record(e string) {
q.mu.Lock()
q.events = append(q.events, e)
q.mu.Unlock()
}
func (q *recordingQueue) seen() []string {
q.mu.Lock()
defer q.mu.Unlock()
return append([]string(nil), q.events...)
}
func (q *recordingQueue) String() string { return "recording" }
func (q *recordingQueue) Append(corestorage.Messager) error { return nil }
func (q *recordingQueue) Register(name string, _ corestorage.ConsumerFunc) {
q.record("register:" + name)
}
func (q *recordingQueue) Shutdown() {}
func (q *recordingQueue) Run() {
q.record("run")
select {
case q.ran <- struct{}{}:
default:
}
}
// waitForRun waits for Run, which is started on a goroutine.
func (q *recordingQueue) waitForRun(t *testing.T) {
t.Helper()
select {
case <-q.ran:
case <-time.After(5 * time.Second):
t.Fatalf("Run was never called; saw: %v", q.seen())
}
}
// The consumers must be registered before the queue is started. A queue that
// is already running refuses further registration - the contract
// implementations answer storage.ErrQueueAlreadyStarted - and the legacy
// adapter this path goes through drops that error, so the wrong order loses
// consumers with nothing said about it. The memory backend does not care,
// which is exactly why this cannot be left to be noticed in use.
func TestConsumersAreRegisteredBeforeTheQueueIsStarted(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
q := newRecordingQueue()
attachConsumersOnce(1, q)
q.waitForRun(t)
seen := q.seen()
runAt := -1
registers := 0
for i, e := range seen {
switch {
case e == "run":
if runAt < 0 {
runAt = i
}
case strings.HasPrefix(e, "register:"):
registers++
if runAt >= 0 {
t.Errorf("%q came after Run; a running queue refuses registration", e)
}
}
}
if registers != 3 {
t.Errorf("registered %d consumers, want 3; saw %v", registers, seen)
}
if runAt < 0 {
t.Errorf("the queue was never started; saw %v", seen)
}
}
// AfterResource runs again on every configuration reload, so the hook has to
// be idempotent with respect to a given queue - not "does nothing the second
// time". Registering twice on the same queue would give every message two
// consumers and write every log row twice.
func TestTheSameQueueIsNotGivenConsumersTwice(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
q := newRecordingQueue()
attachConsumersOnce(1, q)
q.waitForRun(t)
attachConsumersOnce(1, q)
// Nothing to wait for on the second call, so give a wrong implementation
// the time it would need to show up.
time.Sleep(200 * time.Millisecond)
if n := len(q.seen()); n != 4 {
t.Errorf("%d calls after attaching twice to the same queue, want 4 (3 registers + 1 run); saw %v", n, q.seen())
}
}
// The other half of the same rule: a reload builds a new adapter, and the
// consumers on the old one are attached to a queue nobody publishes to any
// more. A new generation must get its own set.
func TestANewQueueGetsItsOwnConsumers(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
first := newRecordingQueue()
attachConsumersOnce(1, first)
first.waitForRun(t)
second := newRecordingQueue()
attachConsumersOnce(2, second)
second.waitForRun(t)
if n := len(second.seen()); n != 4 {
t.Errorf("the queue from the second generation saw %d calls, want 4; saw %v", n, second.seen())
}
if n := len(first.seen()); n != 4 {
t.Errorf("the queue from the first generation saw %d calls, want 4 - it should not have been touched again; saw %v", n, first.seen())
}
}
// Generation 0 means the configuration has no queue section at all, so nothing
// was installed and the runtime hands back its own memory queue. That case
// still has to get consumers - the registration it replaces was unconditional,
// and dropping it would stop the login and operation logs for anyone who
// commented the section out.
func TestAnUnconfiguredQueueStillGetsConsumers(t *testing.T) {
attachedQueue.Store(0)
t.Cleanup(func() { attachedQueue.Store(0) })
q := newRecordingQueue()
attachConsumersOnce(0, q)
q.waitForRun(t)
if n := len(q.seen()); n != 4 {
t.Errorf("an unconfigured queue saw %d calls, want 4; saw %v", n, q.seen())
}
// And still only once.
attachConsumersOnce(0, q)
time.Sleep(200 * time.Millisecond)
if n := len(q.seen()); n != 4 {
t.Errorf("generation 0 was attached to twice: %d calls, want 4; saw %v", n, q.seen())
}
}