Repository navigation
Uncertainty between submodule and local definition in new semantic analyzer #6828
Description
Consider this test case:
[case testNewAnalyzerImportFromTopLevelAlias]
import a.b
[file a/__init__.py]
from .b import B
from . import b as c
b = int
x: Forward
class Forward:
...
[file a/b.py]
class B: ...
[builtins fixtures/module.pyi]
It currently fails with Name 'c' is already defined, because on the first iteration mypy sees a.b as module, while on the second one it is a type alias. There is also a similar but different test case:
[case testNewAnalyzerImportFromTopLevelFunction]
import a.b
[file a/__init__.py]
from .b import B
from . import b as c
def b(): ... # can be also b: int
x: Forward
class Forward:
...
[file a/b.py]
class B: ...
[builtins fixtures/module.pyi]
that fails with Name 'b' already defined for a similar reason. Note that we already added some special casing for a similar scenario, see #6630, to match old semantic analyzer (and also Python runtime). Setting priority to high since this causes problems with internal deployment of new analyzer. @JukkaL I assign this to you since this may be related to tracking the order of definitions.
Activity
JukkaL commentedon Jun 12, 2019
It looks like the approach taken in #6630 isn't workable and needs to be replaced with something more general. The new semantic analyzer relies on only creating a non-placeholder symbol node once per name (#6300) and submodules shouldn't deviate from this, since any deviations increase complexity and likely cause bugs.
My current idea is that implicit submodule references shouldn't be added to symbol tables. Instead, we could look them up from a separate "module scope". If a name is defined in a module, we use that. Otherwise, we fall back to the module dictionary to check if the name refers to a module.
To implement this logic, we'd need to keep track of where a definition is valid using the line number of the definition.
There are various edge cases that are still unclear to me and need more thought.
ilevkivskyi commentedon Jun 12, 2019
Generally I like this idea. I was thinking about something similar, but even more global. There is another case where we want to have essentially two nodes in the same symbol table under the same name: the
-redefinitionthing. (Yet another will be conditional imports currently unsupported.)We can make a symbol table essentially a mapping from names to a deque of nodes, but
__getitem__will always return the leading one. There will be a method to access all of them. Plus two methods to push on either end, depending on whether we want to shadow existing name (like with named tuples or plugins), or not (like for redefinitions or for implicit imports).ilevkivskyi commentedon Jun 12, 2019
Yet another related thing to consider is
--allow-redefinition. If we are going to move towards something more SSA-like (e.g. allowing conditional redefinitions), then the current scheme with variable renaming with primes,x,x',x'', etc., might not work well. We will need to be able to record that inthe last
xmay refer to eitherVarnode. With the deque logic this seems possible to address.New semantic analyzer: don't add submodules to symbol tables (#7005)
ilevkivskyi commentedon Jun 17, 2019
I think this was closed by mistake.
New semantic analyzer: fix ambiguity between submodule and local defi…
New semantic analyzer: fix ambiguity between submodule and local defi…
New semantic analyzer: fix ambiguity between submodule and local defi…
New semantic analyzer: don't add submodules to symbol tables (python#…
New semantic analyzer: fix ambiguity between submodule and local defi…