[venv] Adding a .gitignore file to virtual environments

@brettcannon

BPO	39236
Nosy	@brettcannon, @vsajip, @asottile, @tirkarthi

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-01-06.19:45:05.714>
labels = ['type-feature', 'library', '3.9']
title = '[venv] Adding a .gitignore file to virtual environments'
updated_at = <Date 2020-01-07.16:32:50.101>
user = 'https://github.com/brettcannon'

bugs.python.org fields:

activity = <Date 2020-01-07.16:32:50.101>
actor = 'vinay.sajip'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-01-06.19:45:05.714>
creator = 'brett.cannon'
dependencies = []
files = []
hgrepos = []
issue_num = 39236
keywords = []
message_count = 2.0
messages = ['359459', '359517']
nosy_count = 4.0
nosy_names = ['brett.cannon', 'vinay.sajip', 'Anthony Sottile', 'xtreak']
pr_nums = []
priority = 'low'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue39236'
versions = ['Python 3.9']

Linked PRs

GH-83417: Allow venv to add a .gitignore file to environments via a new scm_ignore_file parameter #108125

In a discussion on Twitter, the idea of having venv lay down a .gitignore file in a newly created virtual environment that consisted of nothing but * came up (https://twitter.com/codewithanthony/status/1213680829530099713). The purpose would be to help prevent people from inadvertently committing their venv to git. It seems pytest does something similar for .pytest_cache (got one complaint but have chosen to keep it otherwise).

To me this seems like a good enhancement. Since this would mostly benefit beginners then it should probably be an opt-out if we do it at all. Maybe make --no-ignore-file to opt out?

FYI Mercurial does not support subdirectory hgignore files like git does, so this may be git-specific (for now): https://www.selenic.com/mercurial/hgignore.5.html.

I'm -0.5 on this. Beginners will probably shoot themselves in the foot multiple ways using git (or any DVCS), in terms of checking things in they didn't mean to (e.g. build artifacts which are not in the venv). As they can easily undo this, I don't think we need to do this.

Add a `.gitignore` to the created virtual environment

Add a `.gitignore` to the created virtual environment (#32)

I think this is a very good idea and not only would assist beginners but has come to be expected for tools that create directories that should not be version controlled.

Resurrecting this to add some thoughts.
To introduce myself, I run a third-party anti-malware scanning service against the Python Package Index.
We've observed a fairly long-standing trend of venv inclusion in source distributions.
This is problematic for a few reasons:

Many of the commands in the virtual environment creation itself cause flags on our signature analysis schema. We've taken to 'whitelisting' virtual environment hashes on a per-file basis as we encounter them, but this is by no means evergreen. Whitespacing issues and minute differences in versioning have lead to a constant whack-a-mole of trying to maintain these hashes.
This often dramatically increases the package size of smaller files. As I suspect is a 'granted', the individuals most predisposed to unintentional inclusion of their virtual environments in their workflows are also the most likely to have heavily polluted virtual environments in the first place.
This creates a 'recursive checking' problem, where by we're also now checking every dependency of a particular package by virtue of enumerating the virtual environment. This might sound ideal, but as we scan all new and uploaded packages on the Python Package Index, these packages are "secure" in our eyes by virtue of simply existing in the first place.
Because we aggregate the signatures that a package might display, including the virtual environment substantially increases the propensity that a package may be verdicted as potentially malicious. To me, it stands to reason that if you include enough code, anything is going to flag as malware without human analysis.

We would like to scan the package uploaded and make a determination on the code therein, less so the dependencies it uses. When we begin adding additional packages/libraries into the codebase, our whitelisting and other false-positive reducing techniques begin to degrade.

We've seen malicious modifications to virtual environments in the past, where the included libraries are used to contain malicious code or libraries that otherwise would be prohibited from the Python Package Index (or, in some cases, were already removed from the Python Package Index for being malicious.)

As PyPI shifts towards a third-party anti-malware solution, I think this would be extremely beneficial to revisit this conversation with that in mind.

Hatch also does this

Apparently, .bzrignore supports the same syntax (if we want to bother with support Bazaar since it seems to no longer be supported).

I'd say stick with .gitignore for now - I doubt there's enough demand for Bazaar support.

Agreed, let's start with only Git.

Of the open source projects tracked by Open Hub, Bazaar accounts for ~0% (13,042 repos) compared to Git at 74% (1,079,642), and is the least popular of the five SCMs represented:

Better data for what is used now - Stack Overflow survey 2022: https://survey.stackoverflow.co/2022/#section-version-control-version-control-systems . 82% Git, 1% SVN, <1% Mercurial.

pythonGH-83417: Allow `venv` add a `.gitignore` file to environments

please loop me in on the PR.

#108125

GH-83417: Allow `venv` to add a `.gitignore` file to environments via…

pythonGH-83417: Allow `venv` to add a `.gitignore` file to environmen…

For anyone wondering why this seems necessary, because they have a .gitignore file in their .venv already: vscode-python has been doing this on creation of an environment from the beginning (late 2022).

vscode-python has been doing this on creation of an environment from the beginning (late 2022).

To be specific, we have been adding a .gitignore file since we introduced our opinionated "Python: Create Environment" command. Our experience informed my decision to push for this here as we have had zero issues since introducing it in the VS Code extension.

Navigation Menu

Repository navigation

[venv] Adding a .gitignore file to virtual environments #83417

Linked PRs

6 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[venv] Adding a .gitignore file to virtual environments #83417

Description

Linked PRs

Activity

brettcannon commented on Jan 6, 2020

vsajip commented on Jan 7, 2020

ofek commented on Jul 13, 2023

import-pandas-as-numpy commented on Jul 13, 2023

6 remaining items

ofek commented on Aug 10, 2023

brettcannon commented on Aug 10, 2023

vsajip commented on Aug 11, 2023

hugovk commented on Aug 11, 2023

adamchainz commented on Aug 13, 2023

brettcannon commented on Aug 18, 2023

bersbersbers commented on Oct 6, 2023

brettcannon commented on Oct 6, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions