Skip to content

Fix incremental speed regression - #4331

Merged
gvanrossum merged 5 commits into
python:masterfrom
gvanrossum:fix-4135-incremental-speed
Dec 8, 2017
Merged

gvanrossum merged 5 commits into
python:masterfrom
gvanrossum:fix-4135-incremental-speed

Conversation

@gvanrossum

Copy link
Copy Markdown
Member

Fixes #4135.

@gvanrossum
gvanrossum force-pushed the fix-4135-incremental-speed branch from 2421d80 to 47b29de Compare December 6, 2017 19:39
@gvanrossum
gvanrossum requested a review from JukkaL December 6, 2017 21:07
@gvanrossum

Copy link
Copy Markdown
Member Author

Don't merge yet -- this has an O(N**2) component. We can do better by having an "errors" bit per file. (Though it's really per SCC that counts.)

@gvanrossum
gvanrossum force-pushed the fix-4135-incremental-speed branch from 3ecc740 to 257a9d1 Compare December 8, 2017 05:30
@gvanrossum

gvanrossum commented Dec 8, 2017 •

Copy link
Copy Markdown
Member Author

What do you think of this version? It sets a flag transitive_error when there's an error in either the current module or any of its dependencies. (Except it does it it per SCC, so if any dependency of any module in an SCC has this flag set, all modules in the SCC get it set.)

There's one bit of strangeness (see the last local commit) -- one test decided to have a different [stale] list.

(If you'd rather see the previous O(N**2) version, I can probably get it back using git reflog. :-)

@JukkaL

JukkaL commented Dec 8, 2017

Copy link
Copy Markdown
Collaborator

The approach looks good to me. I'm a little worried about the difference in the test case though -- might be worth figuring out why it has changed. Feel free to merge this once you are happy with this (and have fixed Travis).

@gvanrossum

Copy link
Copy Markdown
Member Author

Ah, I understand why that test (testIncrementalWithSilentImportsAndIgnore) had to be modified. There are no errors in c/__init__.py, and it does not depend on modules with errors. However it is processed after a module with errors (c/submodule.py), and with the old approach that meant no cache file would be written for c/__init__.py (in fact that was the bug we're fixing here). But with the new approach the file is not considered to have a (transitive) error, and the logic in write_cache() decides it can write the cache file. So it's a good thing.

@gvanrossum
gvanrossum merged commit 01de89f into python:master Dec 8, 2017
@gvanrossum
gvanrossum deleted the fix-4135-incremental-speed branch December 8, 2017 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants