Repository navigation
Memory leak for Express requests with large attached objects #15528
Description
Is there an existing issue for this?
- I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
- I have reviewed the documentation https://docs.sentry.io/
- I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases
How do you use Sentry?
Sentry Saas (sentry.io)
Which SDK are you using?
@sentry/node
SDK Version
9.2.0
Framework Version
Node 20.16.3
Link to Sentry event
No response
Reproduction Example/SDK Setup
We're seeing a Sentry-related memory leak in an Express web server. I suspect the leak has been present for quite some time, but it became more apparent recently when we started attaching large objects (a couple hundred KB to a few MB) to some of our Express requests.
The issue feels similar to #14965, but the retention stack is slightly different, and the issue was not resolved by upgrading to the latest version of the SDK. Feel free to mark as duplicate and reopen there.
Versions:
- Node: 20.16.3
- Sentry SDK: 9.2.0
- Express: 4.21.2
Sentry usage -- fairly vanilla initialization:
Sentry.init({
dsn: process.env.SENTRY_DSN,
release: process.env.VERSION,
environment: process.env.ENV_NAME,
integrations: [
nodeProfilingIntegration(),
],
});
const app = express();
Sentry.setupExpressErrorHandler(app);
/* Other express app setup */Adding a touch of context in some other error handling middleware, leak is still present if I remove this:
if (user) {
Sentry.setUser({
id: user.id ? user.id : user.toString(),
username: user.name && `${user.name.first} ${user.name.last}`,
email: user.email,
});
}Sample retention stack:
Unfortunately I can't provide a full heapdump or source code from our app to reproduce.
Steps to Reproduce
Sorry, I know how frustrating it is to not get perfect reproduction steps, but I can't share source code. I can take a stab at a minimal reproducing setup if I have some spare time. I'd start by:
- Creating an Express app that attaches some large objects to requests
- Occasionally throwing errors to be caught by Sentry
- Running some non-trivial request load
Expected Result
Flat memory usage
Actual Result
Rising memory usage. Sentry 7.12x.x version is 7.120.1:
Metadata
Metadata
Assignees
Projects
- Status
Activity
AbhiPrasad commentedon Feb 27, 2025
Thanks for writing in @john-matroid
What would help is to get an idea of what these large objects are. Are they holding references to buffers or similar?
The timeouts in the stack are pointing me to our request session logic, but considering all of this is new logic in
v9, it feels strange thatv7also has this problem:sentry-javascript/packages/node/src/integrations/http/SentryHttpInstrumentation.ts
Line 470 in 4a7aa21
Can you confirm that
9.2.0actually leaks? Perhaps the memory usage just goes up but then becomes flat. The increased memory usage is not ideal, but feels better than an OOM.john-matroid commentedon Feb 27, 2025
Appreciate the quick reply here. The objects are ABAC-related permissions objects created by CASL. A lot of the memory use boils down to arrays of objects with a handful of short strings as values.
Happy to run 9.2.0 longer to see if the leak is persistent. In terms of the retention pointers, here's what it looked like for 7.120:
AbhiPrasad commentedon Feb 28, 2025
We might end up having to delete the
requestmanually off the sdk processing metadata once the request is over.@timfish does that seem reasonable to you?
timfish commentedon Feb 28, 2025
Yep that seems reasonable!
lforst commentedon Mar 4, 2025
I would instead just rewrite our express integration not to use the
requestfield on the SDK processing metadata but to use thenormalizedRequestlike we do everywhere else. That seems a little more sane tbh. It looks like an oversight from v9.john-matroid commentedon Mar 4, 2025
Thanks team!
github-actions commentedon Mar 5, 2025
A PR closing this issue has just been released 🚀
This issue was referenced by PR #15570, which was included in the 9.4.0 release.