Skip to content

Special Chars getting butchered #2

Open

Description

@erskingardner

Is there way to specify encoding? Or does it already default to something during the parsing. I'm using node-readability to parse lots and lots of blog pages (from a list of permalinks) but I'm getting lots of special chars (a manner of quotes, hyphens, etc.) that are transforming into messy jumbles (eg. ’)

Activity

  1. arrix commented on Dec 15, 2010

    @arrix
    Owner

    The html parser accepts html source as string, which means it doesn't need to worry about text encoding because a javascript string is expected to contain valid utf-8 data.
    Could you please provide a test case to reproduce the issue?

    If you need to deal with data with encodings other than utf-8, check out node-iconv https://github.com/bnoordhuis/node-iconv

  2. erskingardner commented on Dec 15, 2010

    @erskingardner
    Author

    I actually stepped back a level and handled all the encodings after I do the readability parsing since I need to pick out certain elements anyways. Thanks for the answer and the explanation though.

  3. darkhelmet commented on Feb 12, 2011

    @darkhelmet
    Contributor

    Can you post an example? I'm seeing this too.

  4. adeelraza commented on Feb 16, 2011

    @adeelraza

    I'm seeing an issue with htmlentities. Using the clean-proxy example, try to open this article: http://www.openforum.com/idea-hub/topics/marketing/article/what-we-can-learn-from-justin-bieber-guy-kawasaki

    ” converts to ” .. and similarly other entities too.

  5. added a commit that references this issue on Apr 28, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions