Skip to content

Hang when extracting messages #401

Closed

Description

@rlam3

PYBABEL HANGS ...? HOW TO DEBUG?

The following commpand is:

pybabel extract -F babel.cfg -k lazy_gettext -o messages.pot my_app_folder

The processes has been hanging for a while and I don't know how to debug... Is there an option for this?

Even when i remove everything from the file or comment out the whole .py file that it is hanging on... the process is still hanging...

Activity

  1. rlam3 commented on May 3, 2016

    @rlam3
    Author

    So I looked into pybabel's code and it looks like it is hanging on this particiular loop in frontend.py

    After the catalog.add method it looks back and it hangs... Please let me know what we can do the relieve of this problem? Does there need to be a break some where here?

                    for filename, lineno, message, comments, context in extracted:
    
                        if os.path.isfile(path):
                            filepath = filename  # already normalized
                        else:
                            filepath = os.path.normpath(os.path.join(path, filename))
    
                        catalog.add(message, None, [(filepath, lineno)],
                                    auto_comments=comments, context=context)
  2. akx commented on May 3, 2016

    @akx
    Member

    I don't think it's actually hanging -- maybe add print(filename, lineno, message, comments, context) within that loop's body to see if it somehow manages to find an infinite amount of messages or something?

  3. changed the title [-]PYBABEL HANGS ...? HOW TO DEBUG?[/-] [+]Hang when extracting messages[/+] on May 3, 2016
  4. rlam3 commented on May 3, 2016

    @rlam3
    Author

    @akx How do I find if there is infinite amount of messages?

    From what I've been trying around... it extracts fine as an individual module and file.. Its just doesn't like the fact that it is grouped with other modules together upon extraction...?

    When I did the print statement you suggested... there was only one unique message there. I'm not sure where I could get infinite messages... once the for loop is done with catalog.add it comes back up to the top and it just hangs there at the for loop ...

    pybabel -v extract -F babel.cfg -k gettext -o messages.pot my_app/folder/module 
    
    extracting messages from my_app/my_module/__init__.py
    extracting messages from my_app/my_module/forms.py
    extracting messages from my_app/my_module/views.py
    writing PO template file to messages.pot
    

    I'm not sure if this is of any use, but this is what I get from ctrl+c when it hangs... does it have something to do with the extract_from_dir callback?

    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/bin/pybabel", line 9, in <module>
      load_entry_point('Babel==2.3.4', 'console_scripts', 'pybabel')()
    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/lib/python2.7/site-packages/babel/messages/frontend.py", line 915, in main
      return CommandLineInterface().run(sys.argv)
    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/lib/python2.7/site-packages/babel/messages/frontend.py", line 840, in run
      return cmdinst.run()
    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/lib/python2.7/site-packages/babel/messages/frontend.py", line 454, in run
      for filename, lineno, message, comments, context in extracted:
    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/lib/python2.7/site-packages/babel/messages/extract.py", line 156, in extract_from_dir
      dirpath=absname,
    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/lib/python2.7/site-packages/babel/messages/extract.py", line 193, in check_and_call_extract_file
      if not pathmatch(pattern, filename):
    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/lib/python2.7/site-packages/babel/util.py", line 140, in pathmatch
      match = re.match(''.join(buf) + '$', filename.replace(os.sep, '/'))
    File "/Users/XXXXXX/.virtualenvs/ZZZZZZ/lib/python2.7/re.py", line 141, in match
      return _compile(pattern, flags).match(string)
    KeyboardInterrupt
    

    The craziest thing is that even with the views.py and forms.py and init.py is empty.. the hanging is still present

  5. rlam3 commented on May 3, 2016

    @rlam3
    Author

    I think it is hanging/running in the background for such a long time because it is looking at all the subdirectories of node_modules... is there a way to prevent babel from subdirectories? I've tried using the following but it seems like it is still going through all of them.

    [ignore: **/static/**.*]
    
  6. rlam3 commented on May 3, 2016

    @rlam3
    Author

    How should one be able to strictly ignore folders? OR How should one be able to specify which folders to look into?

    [ignore: static/*.]

    When ignore happens.. Why does another method able to look into the ignored pattern? This doesn't make sense...

    This is not an ideal workaround...:

    #babel.cfg
    [python: mod1/**.py]
    [python: mod2/**.py]
    [python: mod3/**.py]

    Reference:
    #124

  7. akx commented on May 4, 2016

    @akx
    Member

    Adding ignore stanzas won't unfortunately make Babel not even recurse into those directories; checking file patterns against the extractor configuration is done separately.

    For a quick patch in your local installation of Babel, you can add or subdir == "node_modules" to the if statement in this segment here. (That particular segment is actually what causes #53, just pointing that out here.)

  8. akx commented on May 4, 2016

    @akx
    Member

    I removed the bug label because this isn't an actual bug in Babel.

  9. rlam3 commented on May 4, 2016

    @rlam3
    Author

    @akx Thanks for the reply akx but could you please go more in depth on what you mean b subdir? Could you please show me an example of a babel.cfg using it? Thanks.

  10. akx commented on May 5, 2016

    @akx
    Member

    @rlam3

    If you patch the linked segment,

        for root, dirnames, filenames in os.walk(absname):
            for subdir in dirnames:
                if subdir.startswith('.') or subdir.startswith('_'):
                    dirnames.remove(subdir)
            dirnames.sort()
            filenames.sort()
            for filename in filenames:

    to read

        for root, dirnames, filenames in os.walk(absname):
            for subdir in dirnames:
                if subdir.startswith('.') or subdir.startswith('_') or subdir == 'node_modules':
                    dirnames.remove(subdir)
            dirnames.sort()
            filenames.sort()
            for filename in filenames:

    Babel will not even recurse into node_modules directories.

    There is currently no way of doing this via babel.cfg or command line parameters, but it's definitely something to put on the roadmap.

  11. akx commented on Jul 15, 2016

    @akx
    Member

    Closing -- #402 will serve as a proper fix for this behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions