Add support for Implicit Namespace Packages #288

Activity

1. Alexander Sher (AlexanderSher)

   added

   Epic

   on Oct 19, 2018
2. AlexanderSher commented on Oct 24, 2018

   

   Alexander Sher (AlexanderSher)

   on Oct 24, 2018

   Last edited by AlexanderSher

   ContributorAuthor

   More actions

   Namespace packages can be presented in several directories. Consider the following folder structure:

   ├─ package1
   │  └─ parent
   │     └─ child
   │        └─ one.py
   ├─ package2
   │  └─ parent
   │     └─ child
   │        └─ two.py
   └─ root
      └─ app.py
   

   If app.py contains the following code:

   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   import parent.child.one
   import parent.child.two
   
   print(parent.child.__path__)
   try: 
       print(parent.child.__file__)
   except AttributeError:
       print("parent.child have no __file__ attribute")
   print(parent.child.one.__file__)
   print(parent.child.two.__file__)

   executing python root\app.py will show

   _NamespacePath(['path_to_project/package1/parent/child', 'path_to_project/package2/parent/child'])
   parent.child have no __file__ attribute
   path_to_project/package1/parent/child/one.py
   path_to_project/package2/parent/child/two.py
   
3. AlexanderSher commented on Oct 24, 2018

   

   Alexander Sher (AlexanderSher)

   on Oct 24, 2018

   Last edited by AlexanderSher

   ContributorAuthor

   More actions

   from package import * doesn't import anything unless other imports have provided information for the namespace. If app.py contains the following code:

   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   from parent import *
   
   print(child.__path__)

   executing python root/app.py will produce an error:

     File "root/app.py", line 8, in <module>
       print(child.__path__)
   NameError: name 'child' is not defined
   

   Adding import to app.py that can provide information about the module

   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   from parent.child.one import *
   from parent import *
   
   print(child.__path__)

   makes python root/app.py work normally:

   _NamespacePath(['path_to_project/package1/parent/child', 'path_to_project/package2/parent/child'])
   

   However, this won't add other modules inside package. For app.py like this:

   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   
   from parent.child.one import *
   from parent.child import *
   print(one.__file__)
   print(two.__file__)

   executing python root/app.py will produce an error:

   Traceback (most recent call last):
     File "root/app.py", line 14, in <module>
       print(two.__file__)
   NameError: name 'two' is not defined
   
4. AlexanderSher commented on Oct 24, 2018

   

   Alexander Sher (AlexanderSher)

   on Oct 24, 2018

   Last edited by AlexanderSher

   ContributorAuthor

   More actions

   Conflict resolution: if both regular and namespace packages are presented in search paths, namespace packages will be ignored:

   ├─ package1
   │  └─ parent
   │     └─ child
   │        └─ one.py
   ├─ package2
   │  └─ parent
   │     └─ child
   │        └─ two.py
   ├─ package3
   │  ├─ parent
   │  │  └─ child
   │  │    └─ three.py
   │  └─ __init__.py
   └─ root
      └─ app.py
   

   For app.py

   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package3/', 'path_to_project/package2/']
   import parent.child
   
   print(parent.child.__path__)

   executing python root/app.py will produce

   _NamespacePath(['path_to_project/package3/parent/child'])
   
5. AlexanderSher commented on Oct 24, 2018

   

   Alexander Sher (AlexanderSher)

   on Oct 24, 2018

   ContributorAuthor

   More actions

   Functions can affect the import. A bit of edge case. Consider the following code in app.py:

   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   from parent.child import one
   from parent.child import two
   print(one.__file__)
   print(two.__file__)

   Execution of python root/app.py will produce:

   path_to_project/package1/parent/child/one.py
   path_to_project/package2/parent/child/two.py
   

   If imports are moved into scope (i.e. function), there will be an error:

   def add_imports():
       from parent.child import one
       from parent.child import two
       pass
   
   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   add_imports()
   print(one.__file__)
   print(two.__file__)

   Traceback (most recent call last):
     File "root/app.py", line 12, in <module>
       print(one.__file__)
   NameError: name 'one' is not defined
   

   However, from parent.child import * will now import modules that were found while add_imports was executed:

   def add_imports():
       from parent.child import one
       from parent.child import two
       pass
   
   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   add_imports()
   from parent.child import *
   print(one.__file__)
   print(two.__file__)

   path_to_project/package1/parent/child/one.py
   path_to_project/package2/parent/child/two.py
   
6. AlexanderSher commented on Oct 24, 2018

   

   Alexander Sher (AlexanderSher)

   on Oct 24, 2018

   ContributorAuthor

   More actions

   Namespace packages can resolve dependencies through relative imports. For example, in a project structure:

   ├─ package1
   │  └─ parent
   │     └─ child1
   │        └─ one.py
   ├─ package2
   │  └─ parent
   │     └─ child2
   │        └─ two.py
   └─ root
      └─ app.py
   

   and file contents:

   • one.py:

   from ..child2.two import f2
   def f1(): f2(); pass

   • two.py:

   def f1(): print("F1"); pass
   def f2(): print("F2"); pass

   • app.py

   import sys
   sys.path += ['path_to_project/package1/', 'path_to_project/package2/']
   
   from parent.child1.one import *
   f1()

   executing python root/app.py prints F2

   👍React with 👍1Reacted by Don Jayamanne
7. jakebailey commented on Oct 31, 2018

   

   Jake Bailey (jakebailey)

   on Oct 31, 2018

   Last edited by jakebailey

   Member

   More actions

   Do you know if anything you're updating will handle __all__? I've been going through the Django stuff, and found that they make use of * imports and __all__ modification quite a bit. See:

   • https://github.com/django/django/blob/master/django/db/models/__init__.py
   • https://docs.python.org/3/tutorial/modules.html#importing-from-a-package

   EDIT: Testing on Django packages does seem to find stuff alright, but I can come up with an example like:

   • whatever/__init__.py

   from sys import version
   
   __all__ = []

   • app.py

   from whatever import *
   
   print(version)

   Where we (currently) show version as being in scope, but running it fails saying version is not defined, which seems to hint that we might want to see how __all__ is going to play into this.

   Forgive me if this isn't directly namespace package related, but I know you're updating the import resolution code, so I figured I'd ask. I'll make a new issue for this if needed.
8. AlexanderSher commented on Nov 1, 2018

   

   Alexander Sher (AlexanderSher)

   on Nov 1, 2018

   Last edited by AlexanderSher

   ContributorAuthor

   More actions

   This is actually the opposite task - support of the __init__.py, which can be only in ordinal packages. __all__ isn't the only option, you can simply define imports there. So yes, we need another epic for this.

   👍React with 👍1Reacted by Jake Bailey
9. Mikhail Arkhipov (MikhailArkhipov)

   assigned

   Alexander Sher (AlexanderSher)

   on Nov 6, 2018
10. Mikhail Arkhipov (MikhailArkhipov)

    added

    GA

    on Nov 21, 2018
11. addisonElliott commented on Dec 5, 2018

    

    Addison Elliott (addisonElliott)

    on Dec 5, 2018

    Last edited by addisonElliott

    More actions

    👍 Interested in this PR

    In many Python projects, it's typical to alter the sys.path in unit tests to include the Python package.

    This is one of the few things keeping me from fully switching to VS Code for Python dev...

    👍React with 👍2Reacted by wtigotw and Jason Stitt
12. wtigotw commented on Jan 9, 2019

    

    wtigotw

    on Jan 9, 2019

    More actions

    👍 Interested in this PR

    In many Python projects, it's typical to alter the sys.path in unit tests to include the Python package.

    This is one of the few things keeping me from fully switching to VS Code for Python dev...

    Same for me. This is the last major hurdle between me and VS Code as well.

    👍React with 👍1Reacted by Jason Stitt
13. Mikhail Arkhipov (MikhailArkhipov)

    added this to the Feb 2019.1 milestone on Jan 22, 2019
14. Dan Taylor (qubitron)

    removed

    GA

    on Feb 6, 2019
15. MikhailArkhipov commented on Feb 13, 2019

    

    Mikhail Arkhipov (MikhailArkhipov)

    on Feb 13, 2019

    More actions

    Issues applicable to GA in this Epic are closed. So I am going to disconnect to remaining issues since they are not prioritized to GA and close the Epic.
16. Mikhail Arkhipov (MikhailArkhipov)

    closed this as completedon Feb 13, 2019