Cannot use a session that has ended on createIndexes spans from check-alerts task #2734

The API service emits MongoExpiredSessionError: Cannot use a session that has ended on mongodb.createIndexes spans. All of them originate from the check-alerts task worker (process.command_args = […, …/packages/api/build/tasks/index.js, check-alerts]).

No functional impact but lots of tracing noise

Example failing trace

A single check-alerts trace (22 ms total); mongo command spans in time order (dns/tcp/sasl and mongodb.create spans omitted):

.017  mongodb.find            ok
.018  mongodb.createIndexes   ok
.019  mongodb.createIndexes   ok
.019  mongodb.createIndexes   ok
.019  mongodb.createIndexes   ok
.020  mongodb.endSessions
.020  mongodb.createIndexes   Error: Cannot use a session that has ended  (0.07 ms)

span status & logs

Failing span:

t:             2026-07-25 15:23:00.021
SpanName:      mongodb.createIndexes
StatusCode:    Error
StatusMessage: Cannot use a session that has ended
stmt:          {"createIndexes":"?","indexes":[{"expireAfterSeconds":"?","background":"?","name":"?","key":{}}]}

All API log lines for the same trace (all severity info):

15:23:00.002  info  check-alerts started at Sat Jul 25 2026 15:23:00 GMT+0000
15:23:00.010  info  Connection established to MongoDB
15:23:00.021  info  finished processing all tasks on task_queue
15:23:00.023  info  Lost connection to MongoDB server
15:23:00.023  info  check-alerts finished in 21.23 ms

Environment

• @hyperdx/api 2.30.1
• ClickStack helm chart 3.0.2; MongoDB 8.0.4, 3-member replica set
  (mongodb-kubernetes-operator)
• mongoose ^6.13.9 (installed 6.13.9), mongodb ^6.15.0 (installed 6.16.0)
• @opentelemetry/instrumentation-mongodb 0.52.0 — transitive via
  @hyperdx/node-opentelemetry ^0.9.0 →
  @opentelemetry/auto-instrumentations-node 0.56.1

Relevant code

Checked at @hyperdx/api@2.31.0 (same structure as deployed 2.30.1):

• packages/api/src/tasks/checkAlerts/index.ts: CheckAlertTask.execute() calls loadProvider(...) → provider.init() → connectDB() (mongoose.connect). The task runner (packages/api/src/tasks/index.ts) calls task.asyncDispose() in a finally block, which calls provider.asyncDispose() → mongooseConnection.close().
• No mongoose.set('autoIndex', …) anywhere in the repo, so mongoose 6's default autoIndex: true applies. Every model compiled at import time kicks off background createIndex calls as soon as the connection opens.
• On a short run, connection.close() (which triggers the driver's endSessions) races those in-flight index builds — the createIndexes commands still queued on the ended session fail with MongoExpiredSessionError. This matches the observed span ordering (endSessions immediately before the failing createIndexes) and the connect→disconnect log gap of ~14 ms.
• The Connection established to MongoDB / Lost connection to MongoDB server log lines come from the connected/disconnected handlers in packages/api/src/models/index.ts.