CSS/img pulled from browser cache won't be reported in matrix #93

Activity

1. gorhill commented on Jan 17, 2014

   

   gorhill

   on Jan 17, 2014

   OwnerAuthor

   More actions

   I started a fix for this, and now one of the side effect of the solution is that requests might be reported twice in the log, because one request might have been recorded through onBeforeRequest() and another through DOM analysis.

   Need to find a way to mitigate this.
2. gorhill commented on Jan 18, 2014

   

   gorhill

   on Jan 18, 2014

   OwnerAuthor

   More actions

   I should give a try to webRequest.onResponseStarted or webRequest.onCompleted and check the details.fromCache flag, looks promising.
3. gorhill commented on Jan 18, 2014

   

   gorhill

   on Jan 18, 2014

   OwnerAuthor

   More actions

   By the way, a good way to verify the bug is:

   • Go to http://arstechnica.com/
   • Open the Statistics page and narrow the log to http://arstechnica.com/
   • Put the text cursor in the address bar of http://arstechnica.com/ and hit Enter
     • Do not force refresh, as this soft clear the cache I believe
   • The Statistics page should show that very few items were logged following the main_frame event.
     • Since currently items are logged only for when items are not fetched from the cache.

   To confirm any fix works, check that all items on the page are logged, not just those items outside the cache.
4. gorhill commented on Jan 18, 2014

   

   gorhill

   on Jan 18, 2014

   OwnerAuthor

   More actions

   Alright, I am losing track of, and forgetting earlier findings and now I am back to square one.

   webRequest.onResponseStarted/details.fromCache is no good, it works for objects pulled from the on-disk cache, not the in-memory cache. This bug here causes the following scenario to happen, which I should have documented first time I entered this bug:

   • Go to http://arstechnica.com/
   • Drill-in to one article on http://arstechnica.com/ (any will do)
   • Wait for HTTPSB's "garbage collector" to kick in -- around 10 minutes (all recorded requests for http://arstechnica.com/ are flushed since the page hasn't been used for a while)
   • Go back to http://arstechnica.com/ using the back button of the browser
   • Matrix for http://arstechnica.com/ is now almost all empty, no image and css reported
     • This is because most of the data is pulled from the in-memory cache.

   This is the bug. Still no solution.
5. my-password-is-password commented on Jan 19, 2014

   

   my-password-is-password

   on Jan 19, 2014

   Contributor

   More actions

   What about using DOM Mutation Observers?

   Found this https://code.google.com/p/mutation-summary/
6. ghost assigned

   tailHey

   on Jan 29, 2014