Reqest entity too large

Do you want to request a feature or report a bug?
Bug

What is the current behavior?
If the user flow involves some big HTTP requests and operations, the error logging fails with "Reqest entity too large" because Sentry tries to send full stack-trace and HTTP request logs to the backend server.

What is the expected behavior?
The SDK should truncate the body if it's too large.

Which versions of Raven.js, and which browser and OS are affected by this issue?
I'm using the version 3.6 with your SaaS offering.

because Sentry tries to send full stack-trace and HTTP request logs to the backend server.

Raven.js does not send HTTP request logs, unless you're doing something custom.

The SDK should truncate the body if it's too large.

It truncates frames. It doesn't send HTTP request bodies so it wouldn't know to do this.

Are you appending your own data manually or using a plugin like redux-raven-middleware? Please attach a copy of a sample outbound report and I can act on this.

I'm using the version 3.6 with your SaaS offering.

This version is pretty out of date now.

I use Angular plugin with default configurations but you're right, the version is pretty out of date. I upgraded to the last version and let you know if it occurs again. Thanks.

Here is a sample:
curl 'https://app.getsentry.com/api/app_id/store/?sentry_version=7&sentry_client=raven-js%2F3.9.2&sentry_key=KEY' -H 'Pragma: no-cache' -H 'Origin: https://app.rakam.io' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36' -H 'Content-Type: text/plain;charset=UTF-8' -H 'Accept: */*' -H 'Cache-Control: no-cache' -H 'Connection: keep-alive' -H 'DNT: 1' --data-binary $'{"project":"nope","logger":"javascript","platform":"javascript","request":{"headers":{"User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"},"url":"https://app.rakam.io/report-playground?query=select%201,2,3&options=%7B%22chartType%22:%22geo-cluster%22,%22limit%22:1000,%22enableStatistics%22:true,%22tableOptions%22:%7B%22enabled%22:true%7D,%22chartOptions%22:%7B%22latitude%22:0,%22longitude%22:1,%22title%22:2%7D,%22variables%22:%5B%5D,%22segments%22:%5B%5D,%22measures%22:%5B0,1,2%5D,%22dimensions%22:%5B%5D%7D&queryOptions=%7B%7D"},"exception":{"values":[{"type":"TypeError","value":"Failed to execute \'appendChild\' on \'Node\': parameter 1 is not of type \'Node\'.","stacktrace":{"frames":[{"filename":"https://app.rakam.io/static/components/raven-js/dist/raven.js","lineno":372,"colno":29,"function":"HTMLImageElement.wrapped","in_app":false},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":3,"colno":20753,"function":"HTMLImageElement.o [as _leaflet_click27]","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":17248,"function":"e._onMouseClick","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":1,"colno":5028,"function":"e.fireEvent","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":23910,"function":"e.togglePopup","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":23724,"function":"e.openPopup","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":23431,"function":"e.openPopup","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":1,"colno":21249,"function":"e.addLayer","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":1,"colno":29748,"function":"e._layerAdd","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":19069,"function":"e.onAdd","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":20008,"function":"e.update","in_app":true},{"filename":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","lineno":2,"colno":21514,"function":"e._updateContent","in_app":true}]}}]},"culprit":"https://app.rakam.io/static/dist/scripts/1.1-353348d0887eb92338a9.js","extra":{"cluster":"4c8e91d0-471e-4041-a11f-f818b2b757a1","name":"rakam-prod","type":"ui","session:duration":3993424},"breadcrumbs":{"values":[{"timestamp":1488365533.259,"category":"ui.click","message":"td > i.margin-horizontal-5.pointer.fa.fa-play-circle.green"},{"timestamp":1488365533.335,"type":"http","category":"xhr","data":{"method":"GET","url":"https://api.mixpanel.com/track/?data=%3D&ip=1&_=1488365533262","status_code":200}},{"timestamp":1488365533.497,"type":"http","category":"xhr","data":{"method":"POST","url":"https://webapp-1290649538.us-west- .......... TONS OF HTTP REQUESTS

TONS OF HTTP REQUESTS

So ... those look like breadcrumbs, and we limit those to 50 per page. If your URLs are explosively large, they could present a problem. So you could set maxBreadcrumbs: 20 or something.

They're quite large because these Mixpanel calls have a big base64 encoded query parameter that includes the event body. I believe that the best option here is to discard some third party analytics service calls such as Mixpanel or truncate the url if it's too large but it seems that both of them requires some work do be done in this repo, right?

Ah, that's very helpful. Yes, we should truncate this client-side.

It looks like I also removed Mixpanel HTTP calls in the snippet since they may include some user sensitive data but that part is particularly large.

Note, in the meantime you can use breadcrumbCallback to truncate (or disable) HTTP breadcrumbs. See config.

We are seeing this today with many large breadcrumbs, and it crashes the browser. I would argue that Sentry.io should NEVER crash the browser regardless of what the user does. Could you please add some sort of automatic client-side checking and truncation of the packet size before throwing an error in the client, like Sentry's Python integration did.

I think this issue needs to be reopened.

What do you mean by "crash the browser"? Like... an actual crash? How did that happen? Can you provide some more details?

It threw an error in the browser, effectively crashing the JS application running there.

As for details on how to reproduce, log a lot of large breadcrumbs, then hit another error. Raven throws its own exception and does not log the original error, either to the console or to the server.

Ok, looking closer at this on my end, I may have been a bit alarmist. Sentry's thrown error is not crashing the application since it starts with our own JS error, but it IS true that the large number of breadcrumbs are preventing sentry from logging the error to the server. I think truncating the breadcrumbs in the client to ensure that the stack trace gets logged to the server should be standard. I would rather see the stack trace with a note about truncated breadcrumbs than nothing at all when my users are encountering real errors.

We'll rethink the approach to this issue in the next major release. Right now, however, it'd be too much overhead to do this operating for every single event and it may impact the performance of our SDK.

We do provide all the required tools so that developers can do that themselves if they are willing to - https://github.com/getsentry/raven-js/blob/deff2f06b40d10389705167c25efc2e737240e3e/packages/raven-js/src/utils.js#L486-L553

If you want to calculate the size of the breadcrumbs every single time and truncate them if they are too heavy, feel free to do that. Here's how you'd approach something like this:

const utils = require('../node_modules/raven-js/src/utils'); // or just extract it to your own file, its up to you

const MAX_SIZE = 50 * 1024;

function truncateBreadcrumbs (data) {
    // Handle edge-case to prevent an infinite loop
    if (utils.jsonSize(utils.stringify(data)) > MAX_SIZE) {
      data.breadcrumbs = data.breadcrumbs.split(0, -1);
      return truncateBreadcrumbs(data)
    } else {
      return data;
    }
}

Raven.config('__YOUR_DSN__', {
  dataCallback: function (data) {
    return truncateBreadcrumbs(data);
  }
}).install();

@kamilogorek

Edit
I had a read through the docs, and I found that the Raven Vue integrations deprecated. I switched to the new centralised integrations package. I noticed that the option attachProps: true is there by default, setting that to false solved the 413 issue.

There isn't much visibility that I could find searching the docs about that option, and that it may be set to false to reduce payload size. It feels like it should be more prominent in your docs. A lot is left to the deduction of the developer.

Now to look into how to dynamically send or not send props depending on if the payload size exceeds the maximum. It would be great if there were a third value for attachProps such as 'auto' that would check the payload size, and if there is space enough, add the prop data.

@kamilogorek - we're getting this error - the app is intrinsically data heavy.

Your code refers to functions that no longer exist.

Would I be right in assuming the only thing those non-existent functions do is allow you to encode JSON with circular references?

If so, if I were to choose another library and use their stringification function, in place of your old utilities, would your code sample work still?

Also, is there a better, built in solution to this? We could probably get away without sending any app state at all, just console log, the error in question and other minutiae.

Otherwise errors gobbled up by Vue don't make it up and aren't reported without the Raven/Vue plugin - unless I've setup the Sentry browser client incorrectly. As only an error allow to propagate uncaught appeared in Sentry.

Cheers

@CaelanStewart

Your code refers to functions that no longer exist.

@CaelanStewart It does, but here (@sentry/utils package) -

sentry-javascript/packages/utils/src/object.ts

Lines 109 to 123 in 5d670a7

    
           export function normalizeToSize<T>( 
        
             object: { [key: string]: any }, 
        
             // Default Node.js REPL depth 
        
             depth: number = 3, 
        
             // 100kB, as 200kB is max payload size, so half sounds reasonable 
        
             maxSize: number = 100 * 1024, 
        
           ): T { 
        
             const serialized = normalize(object, depth); 
        
             if (jsonSize(serialized) > maxSize) { 
        
               return normalizeToSize(object, depth - 1, maxSize); 
        
             } 
        
             return serialized as T; 
        
           }

Would I be right in assuming the only thing those non-existent functions do is allow you to encode JSON with circular references?

Kinda, but also some more:

/**
 * normalize()
 *
 * - Creates a copy to prevent original input mutation
 * - Skip non-enumerablers
 * - Calls `toJSON` if implemented
 * - Removes circular references
 * - Translates non-serializeable values (undefined/NaN/Functions) to serializable format
 * - Translates known global objects/Classes to a string representations
 * - Takes care of Error objects serialization
 * - Optionally limit depth of final output
 */

Also, is there a better, built in solution to this? We could probably get away without sending any app state at all, just console log, the error in question and other minutiae.

You can investigate vue's context (and optional alter/remove it) with beforeSend callback.

Sentry.init({
  beforeSend(event) {
    // event.contexts.vue has all the metadata
    return event;
  }
})

fix(protocol-designer): stop recording Sentry breadcrumbs for Mixpanel (

Navigation Menu

Repository navigation

Reqest entity too large #874

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	export function normalizeToSize<T>(
	object: { [key: string]: any },
	// Default Node.js REPL depth
	depth: number = 3,
	// 100kB, as 200kB is max payload size, so half sounds reasonable
	maxSize: number = 100 * 1024,
	): T {
	const serialized = normalize(object, depth);

	if (jsonSize(serialized) > maxSize) {
	return normalizeToSize(object, depth - 1, maxSize);
	}

	return serialized as T;
	}

Reqest entity too large #874

Description

Activity

benvinegar commented on Mar 1, 2017

buremba commented on Mar 2, 2017

benvinegar commented on Mar 2, 2017

buremba commented on Mar 2, 2017

benvinegar commented on Mar 2, 2017

buremba commented on Mar 2, 2017

benvinegar commented on Mar 20, 2017

derekrprice commented on Apr 6, 2018

derekrprice commented on Apr 6, 2018

kamilogorek commented on Apr 6, 2018

derekrprice commented on Apr 6, 2018

derekrprice commented on Apr 6, 2018

derekrprice commented on Apr 6, 2018

kamilogorek commented on Apr 9, 2018

CaelanStewart commented on Jul 12, 2019

kamilogorek commented on Jul 23, 2019

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions