Request efficient encoding for plotly offline #1174

Activity

1. jonmmease commented on Sep 12, 2018

   

   jonmmease

   on Sep 12, 2018

   Contributor

   More actions

   Thanks for creating the issue @cboulay ,

   In the meantime, let's do an experiment. I'd like to see how much the base64 encoding would help in your situation.

   Take one of the large multi-dimensional numpy arrays that you're dealing with, lets call it a. First convert it to JSON as a list and measure the time and size (in ipython or jupyter):

   import json
   from plotly.utils import PlotlyJSONEncoder
   encode_timing1 = %timeit -o json.dumps(a, cls=PlotlyJSONEncoder)
   res1 = json.dumps(a, cls=PlotlyJSONEncoder)
   decode_timing1 = %timeit -o json.loads(res1)

   Then convert to a base64 encoded string:

   import base64
   encode_timing2 = %timeit -o base64.b64encode(memoryview(a)).decode()
   res2 = base64.b64encode(memoryview(a)).decode()
   decode_timing2 = %timeit -o np.frombuffer(base64.b64decode(res2.encode('utf-8')), dtype=a.dtype)
   len(res2)

   Then compare

   print('Size list: %d chars' % len(res1))
   print('Size base64: %d chars' % len(res2))
   print('Size ratio: %.4f' % (len(res2)/len(res1)))
   
   print('Encode time list: %.6f seconds' % encode_timing1.average)
   print('Encode time base64: %.6f seconds' % encode_timing2.average)
   print('Encode time ratio: %.6f' % (encode_timing2.average / encode_timing1.average))
   
   print('Decode time list: %.6f seconds' % decode_timing1.average)
   print('Decode time base64: %.6f seconds' % decode_timing2.average)
   print('Decode time ratio: %.6f' % (decode_timing2.average / decode_timing1.average))
2. jonmmease

   added

   enhancement

   on Sep 12, 2018
3. gvwilson commented on Jun 13, 2024

   

   gvwilson

   on Jun 13, 2024

   Contributor

   More actions

   Hi - we are currently trying to tidy up Plotly's public repositories to help us focus our efforts on things that will help users most. Since this issue has been sitting for several years, so I'm going to close it. If it's still a concern, we'd be grateful if you could open a new issue (with a short reproducible example if appropriate) so that we can add it to our backlog. Thanks for your help - @gvwilson
4. gvwilson

   closed this as completedon Jun 13, 2024