IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows

@terryjreedy

BPO	14576
Nosy	@terryjreedy, @serwy, @bitdancer, @asvetlov, @roseman

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = None
created_at = <Date 2012-04-14.10:41:05.356>
labels = ['expert-IDLE', 'type-bug', '3.10']
title = 'IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows'
updated_at = <Date 2020-06-08.00:35:50.375>
user = 'https://bugs.python.org/clikkeb'

bugs.python.org fields:

activity = <Date 2020-06-08.00:35:50.375>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = False
closed_date = None
closer = None
components = ['IDLE']
creation = <Date 2012-04-14.10:41:05.356>
creator = 'clikkeb'
dependencies = []
files = []
hgrepos = []
issue_num = 14576
keywords = []
message_count = 16.0
messages = ['158253', '158288', '158367', '158387', '158404', '158432', '158534', '180121', '185892', '212257', '212296', '215588', '228267', '252040', '252041', '252062']
nosy_count = 8.0
nosy_names = ['terry.reedy', 'roger.serwy', 'r.david.murray', 'asvetlov', 'markroseman', 'clikkeb', 'Divyanshu', 'John Gray']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue14576'
versions = ['Python 3.10']

Linked PRs

gh-58781: Do not exit IDLE if the user config directory cannot be created #157636

It's a common issue that IDLE cannot start on Windows because

"IDLE's subprocess didn't make connection.Either IDLE can't start a subprocess or personal firewall software is blocking the connection."

Everyone claim that the user should set the firewal so that IDLE can connect to subprocess via loopback. I found, instead, that this issue is often caused by an incorrect or missing setting of one the following variables: HOME, USERPROFILE or the combination of HOMEPATH and HOMEDRIVE; if these variables don't represent an existent and writable directory, the error occurs.
Try setting HOMEPATH to an unexistent or unwritable directory just before launching idle.bat script.
Check IdleConf.GetUserCfgDir() in module configHandler.py, where the function os.path.expanduser is called to get the user home directory. You might also temporarly patch GetUserCfgDir, setting the userDir variable to an unexistent path just after it has been initialized via os.path.expanduser (line 202).
Should be considered a check on expanduser behaviour?

OS: Windows XP
Python version: 3.2.2

clikkeb

I can confirm that setting HOMEPATH to a non-existent directory will prevent IDLE from starting when using idle.bat. If you modify idle.bat such that python.exe is called instead of pythonw.exe, then IDLE starts normally, but with this console message:

Warning: os.path.expanduser("~") points to
C:\Users\doesnotexist
but the path does not exist.

Normally, the message is written to sys.stderr, which is None when using pythonw. See bpo-13582 for more information.

Thanks for your answer.

Trying to understand how IDLE uses HOMEPATH and USERPROFILE Windows
variables, I have found the following information:

it seems that when executed via Windows command prompt (cmd.exe),
os.path.expanduser refers to USERPROFILE to determine the user's
home directory;
analizing the stack traces of the first calls to
IdleConf.GetUserCfgDir, you can see that GetUserCfgDir (which
calls expanduser) is called three times during IDLE startup:
the first two times it refers to USERPROFILE, the third (called
via run.py, probably after starting a subprocess) it refers to
the combination of HOMEDRIVE and HOMEPATH. (???)
when you start IDLE using pythonw, sys.stderr.write(warn) seems
to raise an AttributeError exception, which is unhandled. This
causes IDLE to stop running when you either start IDLE that way
and the user's home directory is unreachable.

Due to the Python's tricky behaviour in determining the Windows
user's home directory, my opinion would be to consider if it is
worth to go further with this discussion or if it could produce
a benefit to IDLE. For sure, it gave me a little bit of headache.

clikkeb.

It certainly is worthwhile pursing this in some fashion, since at the very least the existing error message needs to be improved. But perhaps there is something more that can be done to gracefully handle this case, instead. I think the next interesting question is to find out why an invalid home directory causes idle to not start up. There's no obvious reason why that should be the case in principle, so I suspect there's some error handling missing somewhere.

This probably also applies to 2.7; someone should test that.

I think that lines 207-210 of GetUserCfgDir should be modified like this:

try:
sys.stderr.write(warn)
except (IOError, AttributeError): # <----
pass #^^^^^^^^^^^^^^

because when you start IDLE via pythonw.exe (that sets sys.stderr to "None"),
the function call sys.stderr.write(warn) raises the following exception:

AttributeError: 'NoneType' object has no attribute 'write'

and IDLE stops running without displaying any exception error, because that
exception is unhandled.

There is a funcion call to sys.stderr.write also at line 222, just before a
"raise SystemExit" statement, which makes ininfluent the missing
AttributeError exception handling.

I guess IdleConf should to have flag like 'writable'.
If user environment points to invalid location (or there are no write access) this flag should be set.
It flag can affect IDLE configuration dialog: user should be notified what him changes will not be permanently saved.
Subprocess can check that flag as well if need.

BTW, there are related bpo-8231.

It is one of the possible solutions.

In combination with the "writable flag" solution, you might create a class
variable in IdleConf (e.g. "user_cfg") that contains the user's home directory;
such variable will be initialized to an empty string by IdleConf.__init__.
On each call, GetUserCfgDir tries to initialize "user_cfg" only if it is an
empty string, otherwise it returns it's content.

Anyway, keep in mind that in GetUserConfig there's an unhandled exception (AttributeError) that might be the cause of the "IDLE doesn't start..." issue reported by users.

I think the next interesting question is to find out why an invalid home directory causes idle to not start up. There's no obvious reason why that should be the case in principle,

As Roger showed, IDLE will start up with an invalid home directory.

so I suspect there's some error handling missing somewhere

Yes, the uncaught exception raised when attempting to display a non-essential but user-useful messages (this is the third issue I know of about this). There are 5 places in configHandler.py where a warning is directly printed to sys.stderr. There are two similar writes in PyShell.py. The first warning is the problem here, but any of them could be a problem and all should be handled better.

Lib/idlelib\PyShell.py: 1357: sys.stderr.write("Error: %s\n" % str(msg))
Lib/idlelib\PyShell.py: 1358: sys.stderr.write(usage_msg)
Lib/idlelib\configHandler.py: 209: sys.stderr.write(warn)
Lib/idlelib\configHandler.py: 223: sys.stderr.write(warn)
Lib/idlelib\configHandler.py: 255: sys.stderr.write(warning)
Lib/idlelib\configHandler.py: 367: sys.stderr.write(warning)
Lib/idlelib\configHandler.py: 624: sys.stderr.write(warning)

Why are warning used instead of message boxes? Perhaps easier, perhaps the latter are not available because the tk loop is not running. Possible solutions:

Discard the message by just catching AttributeError (clikkeb) -- at each place indicated above. This would be better than nothing, but only as a temporary measure, and there is a better temporary measure below.
Rearrange the start up process to make message boxes available, with default configuraton, before the configuration process.
Make a list of messages and display them when it must (on exit, if that is possible with atexit) or in message boxes when it can.
Solve the more general problem of writing to None by making sys.stdout and sys.stderr not be None, bpo-13582. At minimum, even as a temporary measure, use something that just swallows output. Something like

import io
class DummyOut(io.IOBase):
    def write(self, s):
        return len(s)

dum = DummyOut()
print(dum.write('sixsix'))
# 6

This issue triggers the problem described in bpo-13582. It points out problems with IDLE's configuration manager w.r.t. determining the home directory. I changed the title so that it reflects that point.

I found a weird solution for the problem. Exchange the names of python and pythonw in the python33 folder. This makes the IDLE to call both and as a result python opens in both modes simultaneosly, and the subprocess connection error don't shows up.

I do not understand what you mean by "Exchange the names of python and pythonw in the python33 folder.". In any case, idle.bat cannot run both simultaneously.

Perhaps idle.bat should have an option to start with python instead of pythonw.

I hit this issue with an "H:" homedrive that is on a network share, and then in Windows is using "offline files" to keep a local copy. .idlerc was not cached so IDLE worked when online/connected to my work network but not when I was offline.

The temporary workaround was to mark .idlerc as "available offline" when connected. I wanted to document this in case others hit this scenario.

I also noticed that there is a .idlerc directory created in my local user directory c:\users\[username\.idlerc. This one has the recent-files.lst in it. So IDLE is creating two copies of .idlerc - one in the environment-variable-defined (and roaming) home directory, and one in the default local user directory. I would suggest that as this bug is investigated, you keep track of both instances of .idlerc.

For the h: drive issue mentioned above: This was on Python 2.7.10.

John, thank you for the additional information. 2 copies of .idlerc is definitely bad.

The inconsistent home lookup and the crash on writing to a None stderr are long fixed: since 3.8 os.path.expanduser() on Windows uses USERPROFILE, then HOMEDRIVE+HOMEPATH, and ignores HOME, and IDLE computes the user directory once. What remained is that IDLE (and its subprocess) exited at import if .idlerc could not be created, silently with pythonw. #157636 makes IDLE start with default settings and show a warning instead.

Close? If anything is left to do, a fresh issue might be better.

Navigation Menu

Repository navigation

IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows #58781

Linked PRs

3 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

IDLE: inconsistent use of HOMEDRIVE, HOMEPATH, and USERPROFILE on Windows #58781

Description

Linked PRs

Activity

clikkeb commented on Apr 14, 2012

serwy commented on Apr 14, 2012

clikkeb commented on Apr 15, 2012

bitdancer commented on Apr 16, 2012

clikkeb commented on Apr 16, 2012

asvetlov commented on Apr 16, 2012

clikkeb commented on Apr 17, 2012

terryjreedy commented on Jan 17, 2013

serwy commented on Apr 3, 2013

Divyanshu commented on Feb 26, 2014

terryjreedy commented on Feb 26, 2014

3 remaining items

JohnGray commented on Oct 1, 2015

JohnGray commented on Oct 1, 2015

terryjreedy commented on Oct 1, 2015

serhiy-storchaka commented on Sep 16, 2026

terryjreedy commented on Sep 17, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions