Skip to content

Upload script failing with UnicodeDecodeError #9

Open

Description

@silvermike

I ran into this problem with the api while trying to upload videos and can reproduce it with vimeo-uploadv2.py

Every video I try to upload fails with a UnicodeDecodeError like:

File "c:\foobar\tools\vimeo\convenience.py", line 111, in upload self._post_to_endpoint(open(file_path)) File "c:\foobar\tools\vimeo\convenience.py", line 94, in _post_to_endpoint headers=headers) File "c:\foobar\tools\vimeo\httplib2wrap\__init__.py", line 34, in request_with_files headers, *args, **kwargs) File "c:\foobar\tools\httplib2\__init__.py", line 1584, in request (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey) File "c:\foobar\tools\httplib2\__init__.py", line 1332, in _request (response, content) = self._conn_request(conn, request_uri, method, body, headers) File "c:\foobar\tools\httplib2\__init__.py", line 1269, in _conn_request conn.request(method, request_uri, body, headers) File "C:\Python27\lib\httplib.py", line 958, in request self._send_request(method, url, body, headers) File "C:\Python27\lib\httplib.py", line 992, in _send_request self.endheaders(body) File "C:\Python27\lib\httplib.py", line 954, in endheaders self._send_output(message_body) File "C:\Python27\lib\httplib.py", line 812, in _send_output msg += message_body UnicodeDecodeError: 'ascii' codec can't decode byte 0xb2 in position 1280: ordinal not in range(128)

Environment:
*python 2.7.3 64 bit
*Windows 7 64 bit
*httplib2 v0.7.5

I dug up this http://bugs.python.org/issue11898 but the conclusion on there is that it is supposed to work.

Any ideas? I tried a couple hacks in the httplib2 wrapper, but couldn't come up with anything that worked.

Activity

  1. dkm commented on Sep 7, 2012

    @dkm
    Owner

    I dug up this http://bugs.python.org/issue11898 but the conclusion on
    there is that it is supposed to work.

    Any ideas? I tried a couple hacks in the httplib2 wrapper, but couldn't
    come up with anything that worked.

    Hi!

    what kind of string are you passing as the title of the video ? It's
    possible that some part of the code does not like something not ascii.
    Can you try that ? If this does not work, can you please give more info
    on how you start the script ?

    Thanks

  2. silvermike commented on Sep 7, 2012

    @silvermike
    Author

    I’m not trying to set a title. I run the script like:

    vimeo-uploadv2.py -k e***_6 -s a**_7 -t 7a -y 7*3 -f z:\edits\videos\000.wmv

    As a sanity check I did upload the same video through the website successfully.

    From: Marc Poulhiès [mailto:notifications@github.com]
    Sent: Friday, September 07, 2012 1:51 AM
    To: dkm/python-vimeo
    Cc: Michael Cirello
    Subject: Re: [python-vimeo] Upload script failing with UnicodeDecodeError (#9)

    I dug up this http://bugs.python.org/issue11898 but the conclusion on
    there is that it is supposed to work.

    Any ideas? I tried a couple hacks in the httplib2 wrapper, but couldn't
    come up with anything that worked.

    Hi!

    what kind of string are you passing as the title of the video ? It's
    possible that some part of the code does not like something not ascii.
    Can you try that ? If this does not work, can you please give more info
    on how you start the script ?

    Thanks

    —
    Reply to this email directly or view it on GitHubhttps://github.com//issues/9#issuecomment-8356658.

  3. silvermike commented on Sep 12, 2012

    @silvermike
    Author

    So here's my analysis and (hackish) solution:

    The error is coming from this part of httplib.py:

        # If msg and message_body are sent in a single send() call,
        # it will avoid performance problems caused by the interaction
        # between delayed ack and the Nagle algorithm.
        if isinstance(message_body, str):
            msg += message_body
    

    At this point msg is unicode, and our message_body is a str, so it tries to convert message_body to unicode and raises the exception.

    I tried to fix it by converting the message_body to a bytearray before sending, thereby skipping the offending line in httplib, but I would just end up with a 708 error from Vimeo and gave up.

    My solution - switch to streaming uploads. Two changes -

    1. When getting the ticket, set the streaming method:
    t = client.vimeo_videos_upload_getTicket(upload_method='streaming')
    
    1. Replace the call to self._post_to_endpoint in the upload function of convenience.py with a call to my streaming upload function:
            import mimetypes
            import os
            import urllib2
    
            class FileWithLength(file):
                def __init__(self, path):
                    self._total = os.path.getsize(path)
                    file.__init__(self, path, 'rb')
    
                def __len__(self):
                    return self._total
    
            file_stream = FileWithLength(file_path)
            type = mimetypes.guess_type(file_path)[0]
            if not type:
                raise VimeoError("Could not guess content type")
    
            request = urllib2.Request(self.endpoint, file_stream)
            request.add_header('Content-Type', type)
            request.get_method = lambda: 'PUT'
            urllib2.urlopen(request)            
    

    This seems a lot simpler to me, but maybe I'm missing a good reason this is a bad idea..

  4. dkm commented on Sep 13, 2012

    @dkm
    Owner

    On 2012-09-13 00:15, silvermike wrote:

    So here's my analysis and (hackish) solution:

    Thanks for looking into this issue even if I did not answer (I'm a bit
    busy now...)

    This seems a lot simpler to me, but maybe I'm missing a good reason
    this is a bad idea..

    Have you tried on something else than windows if you fix/workaround
    does not break ?
    If it's working nicely, I could apply your patch :)

    Marc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions