Skip to content
This repository was archived by the owner on May 21, 2025. It is now read-only.
This repository was archived by the owner on May 21, 2025. It is now read-only.

Codecov broken tests #100

Closed
Closed
Codecov broken tests#100

Description

@cpu82
cpu82
opened on May 28, 2017

Hi,

It seems that there are some test cases that fail to pass.

Tested on FreeBSD by invoking the following command:

/usr/local/bin/python2.7 -m pytest -v tests/test.py

http://sprunge.us/hIEC

Is it possible to skip the tests explicitly when invoking the test suite?

Activity

  1. cpu82 commented on May 29, 2017

    @cpu82
    cpu82
    on May 29, 2017
    Author

    I noticed that if the .git directory is present all test cases run fine, except 3 tests skipped.

    % python2.7 -m pytest -v tests/test.py
============================= test session starts ==============================
platform freebsd12 -- Python 2.7.13, pytest-3.1.0, py-1.4.33, pluggy-0.4.0 -- /usr/local/bin/python2.7
cachedir: .cache
rootdir: /usr/home/cpm/work/gh/codecov-python, inifile:
collected 68 items 

tests/test.py::TestUploader::test_bowerrc PASSED
tests/test.py::TestUploader::test_bowerrc_none PASSED
tests/test.py::TestUploader::test_ci_appveyor PASSED
tests/test.py::TestUploader::test_ci_buildkite PASSED
tests/test.py::TestUploader::test_ci_circleci PASSED
tests/test.py::TestUploader::test_ci_codeship PASSED
tests/test.py::TestUploader::test_ci_drone PASSED
tests/test.py::TestUploader::test_ci_gitlab PASSED
tests/test.py::TestUploader::test_ci_jenkins PASSED
tests/test.py::TestUploader::test_ci_jenkins_blue_ocean PASSED
tests/test.py::TestUploader::test_ci_jenkins_env PASSED
tests/test.py::TestUploader::test_ci_magnum PASSED
tests/test.py::TestUploader::test_ci_none PASSED
tests/test.py::TestUploader::test_ci_semaphore PASSED
tests/test.py::TestUploader::test_ci_shippable PASSED
tests/test.py::TestUploader::test_ci_snap PASSED
tests/test.py::TestUploader::test_ci_travis PASSED
tests/test.py::TestUploader::test_ci_wercker PASSED
tests/test.py::TestUploader::test_command PASSED
tests/test.py::TestUploader::test_disable_detect PASSED
tests/test.py::TestUploader::test_disable_gcov SKIPPED
tests/test.py::TestUploader::test_disable_search PASSED
tests/test.py::TestUploader::test_discovers PASSED
tests/test.py::TestUploader::test_exits_0 PASSED
tests/test.py::TestUploader::test_exits_1 PASSED
tests/test.py::TestUploader::test_gcov SKIPPED
tests/test.py::TestUploader::test_ignore_report_1__coverage_worker10 PASSED
tests/test.py::TestUploader::test_ignore_report_2_coverage_jade PASSED
tests/test.py::TestUploader::test_ignore_report_3_include_lst PASSED
tests/test.py::TestUploader::test_ignore_report_4_inputFiles_lst PASSED
tests/test.py::TestUploader::test_ignore_report_5_createdFiles_lst PASSED
tests/test.py::TestUploader::test_ignore_report_6_scoverage_measurements_blackandwhite_xml PASSED
tests/test.py::TestUploader::test_ignore_report_7_test_hello_coverage_txt PASSED
tests/test.py::TestUploader::test_ignore_report_8_conftest_blackwhite_c_gcov PASSED
tests/test.py::TestUploader::test_ignored_path_10__tox PASSED
tests/test.py::TestUploader::test_ignored_path_11_venv PASSED
tests/test.py::TestUploader::test_ignored_path_12__venv_python_2_7 PASSED
tests/test.py::TestUploader::test_ignored_path_1_vendor PASSED
tests/test.py::TestUploader::test_ignored_path_2_node_modules PASSED
tests/test.py::TestUploader::test_ignored_path_3_js_generated_coverage PASSED
tests/test.py::TestUploader::test_ignored_path_4___pycache__ PASSED
tests/test.py::TestUploader::test_ignored_path_5_coverage_instrumented PASSED
tests/test.py::TestUploader::test_ignored_path_6_build_lib PASSED
tests/test.py::TestUploader::test_ignored_path_7_htmlcov PASSED
tests/test.py::TestUploader::test_ignored_path_8__egg_info PASSED
tests/test.py::TestUploader::test_ignored_path_9__git PASSED
tests/test.py::TestUploader::test_include_env PASSED
tests/test.py::TestUploader::test_is_report_10_luacov_report_out PASSED
tests/test.py::TestUploader::test_is_report_11_gcov_info PASSED
tests/test.py::TestUploader::test_is_report_12_nosetests_xml PASSED
tests/test.py::TestUploader::test_is_report_1_coverage_xml PASSED
tests/test.py::TestUploader::test_is_report_2_jacoco_xml PASSED
tests/test.py::TestUploader::test_is_report_3_jacocoTestResults_xml PASSED
tests/test.py::TestUploader::test_is_report_4_coverage_txt PASSED
tests/test.py::TestUploader::test_is_report_5_gcov_lst PASSED
tests/test.py::TestUploader::test_is_report_6_cov_gcov PASSED
tests/test.py::TestUploader::test_is_report_7_info_lcov PASSED
tests/test.py::TestUploader::test_is_report_8_clover_xml PASSED
tests/test.py::TestUploader::test_is_report_9_cobertura_xml PASSED
tests/test.py::TestUploader::test_none_found PASSED
tests/test.py::TestUploader::test_not_jacoco PASSED
tests/test.py::TestUploader::test_read_token_file PASSED
tests/test.py::TestUploader::test_require_branch_1 PASSED
tests/test.py::TestUploader::test_returns_none PASSED
tests/test.py::TestUploader::test_run_coverage SKIPPED
tests/test.py::TestUploader::test_run_coverage_fails PASSED
tests/test.py::TestUploader::test_send PASSED
tests/test.py::TestUploader::test_send_error PASSED

===================== 65 passed, 3 skipped in 2.20 seconds =====================
  2. cpu82
    changed the title [-]codecov broken tests[/-] [+]Codecov broken tests[/+] on May 29, 2017

  3. stevepeak commented on May 30, 2017

    @stevepeak
    stevepeak
    on May 30, 2017
    Contributor

    We welcome a pull request if you feel inclined to adjust the code base for this use case.
    Speaking of which: why is the use case for your teste suite?

  4. cpu82 commented on May 30, 2017

    @cpu82
    cpu82
    on May 30, 2017
    Author

    According to the FreeBSD python port policy [1]: python ports must include a do-test target if the package includes tests.

    For now, I have removed the failed tests using a simple patch [2].

    Therefore, it would be great that all test cases work properly.

    [1] https://wiki.freebsd.org/Python/PortsPolicy#Tests
    [2] https://reviews.freebsd.org/D10873#1609c58f

  5. stevepeak commented on Jun 2, 2017

    @stevepeak
    stevepeak
    on Jun 2, 2017
    Contributor

    @naglalakk can you help out here. Thanks

  6. stevepeak commented on Dec 6, 2017

    @stevepeak
    stevepeak
    on Dec 6, 2017
    Contributor

    I'm cleaning up issues. What is the status of this one?

  7. cpu82 commented on Jan 9, 2018

    @cpu82
    cpu82
    on Jan 9, 2018
    Author

    Hi,

    Today I updated codecov to 2.0.13 but it still failing with a couple of AppVeyor CI tests.

    As a workaround I skipped both tests at the moment.

    --- tests/test.py.orig  2018-01-08 10:57:15 UTC
+++ tests/test.py
@@ -134,7 +134,7 @@ class TestUploader(unittest.TestCase):
                 self.assertEqual(codecov.main(), None)
                 assert post.called and put.called
 
-    @unittest.skipIf(os.getenv('CI') == "True" and os.getenv('APPVEYOR') == 'True', 'Skip AppVeyor CI test')
+    @unittest.skipIf(sys.platform.startswith('freebsd'),'Skip test')
     def test_send(self):
         with patch('requests.post') as post:
             with patch('requests.put') as put:
@@ -249,7 +249,7 @@ class TestUploader(unittest.TestCase):
         else:
             raise Exception("Did not raise AssertionError")
 
-    @unittest.skipIf(os.getenv('CI') == "True" and os.getenv('APPVEYOR') == 'True', 'Skip AppVeyor CI test')
+    @unittest.skipIf(sys.platform.startswith('freebsd'),'Skip test')
     def test_bowerrc_none(self):
         with open(self.bowerrc, 'w+') as f:
             f.write('{"other_key": "tests"}')

    Here is the make test output

    $ make test
===>  License APACHE20 accepted by the user
===>   py27-codecov-2.0.13 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by py27-codecov-2.0.13 for building
===>  Extracting for py27-codecov-2.0.13
=> SHA256 Checksum OK for codecov-codecov-python-v2.0.13_GH0.tar.gz.
===>  Patching for py27-codecov-2.0.13
===>  Applying FreeBSD patches for py27-codecov-2.0.13
===>   py27-codecov-2.0.13 depends on package: py27-setuptools>0 - found
===>   py27-codecov-2.0.13 depends on file: /usr/local/bin/python2.7 - found
===>  Configuring for py27-codecov-2.0.13
running config
===>  Building for py27-codecov-2.0.13
running build
running build_py
creating build
creating build/lib
creating build/lib/codecov
copying codecov/__init__.py -> build/lib/codecov
copying codecov/__main__.py -> build/lib/codecov
running egg_info
creating codecov.egg-info
writing requirements to codecov.egg-info/requires.txt
writing codecov.egg-info/PKG-INFO
writing top-level names to codecov.egg-info/top_level.txt
writing dependency_links to codecov.egg-info/dependency_links.txt
writing entry points to codecov.egg-info/entry_points.txt
writing manifest file 'codecov.egg-info/SOURCES.txt'
reading manifest file 'codecov.egg-info/SOURCES.txt'
writing manifest file 'codecov.egg-info/SOURCES.txt'
===>  Staging for py27-codecov-2.0.13
===>   py27-codecov-2.0.13 depends on package: py27-coverage>0 - found
===>   py27-codecov-2.0.13 depends on package: py27-requests>=2.7.9 - found
===>   py27-codecov-2.0.13 depends on package: py27-setuptools>0 - found
===>   py27-codecov-2.0.13 depends on file: /usr/local/bin/python2.7 - found
===>   Generating temporary packing list
running install
running build
running build_py
running egg_info
writing requirements to codecov.egg-info/requires.txt
writing codecov.egg-info/PKG-INFO
writing top-level names to codecov.egg-info/top_level.txt
writing dependency_links to codecov.egg-info/dependency_links.txt
writing entry points to codecov.egg-info/entry_points.txt
reading manifest file 'codecov.egg-info/SOURCES.txt'
writing manifest file 'codecov.egg-info/SOURCES.txt'
running install_lib
creating /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7
creating /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7/site-packages
creating /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7/site-packages/codecov
copying build/lib/codecov/__init__.py -> /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7/site-packages/codecov
copying build/lib/codecov/__main__.py -> /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7/site-packages/codecov
byte-compiling /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7/site-packages/codecov/__init__.py to __init__.pyc
byte-compiling /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7/site-packages/codecov/__main__.py to __main__.pyc
writing byte-compilation script '/tmp/tmptHoyRL.py'
/usr/local/bin/python2.7 -O /tmp/tmptHoyRL.py
removing /tmp/tmptHoyRL.py
running install_egg_info
Copying codecov.egg-info to /usr/ports/devel/py-codecov/work-py27/stage/usr/local/lib/python2.7/site-packages/codecov-2.0.13-py2.7.egg-info
running install_scripts
Installing codecov script to /usr/ports/devel/py-codecov/work-py27/stage/usr/local/bin
writing list of installed files to '/usr/ports/devel/py-codecov/work-py27/.PLIST.pymodtmp'
===> Creating unique files: Move MAN files needing SUFFIX
===> Creating unique files: Move files needing SUFFIX
Move: bin/codecov --> bin/codecov-2.7
Link: @bin/codecov --> bin/codecov-2.7
====> Compressing man pages (compress-man)
====> Running Q/A tests (stage-qa)
===>  Testing for py27-codecov-2.0.13
===>   py27-codecov-2.0.13 depends on package: py27-ddt>0 - found
===>   py27-codecov-2.0.13 depends on package: py27-mock>0 - found
===>   py27-codecov-2.0.13 depends on package: py27-pytest>0 - found
===>   py27-codecov-2.0.13 depends on file: /usr/local/bin/python2.7 - found
============================= test session starts ==============================
platform freebsd10 -- Python 2.7.14, pytest-3.2.5, py-1.5.2, pluggy-0.4.0 -- /usr/local/bin/python2.7
cachedir: .cache
rootdir: /usr/ports/devel/py-codecov/work-py27/codecov-python-2.0.13, inifile:
plugins: xdist-1.13.1
collected 67 items                                                              

tests/test.py::TestUploader::test_bowerrc PASSED
tests/test.py::TestUploader::test_bowerrc_none <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_appveyor <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_buildkite <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_circleci <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_codeship <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_drone <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_gitlab <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_jenkins <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_jenkins_blue_ocean <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_jenkins_env <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_magnum <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_none <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_semaphore <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_shippable <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_travis <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_ci_wercker <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_command PASSED
tests/test.py::TestUploader::test_disable_detect PASSED
tests/test.py::TestUploader::test_disable_gcov SKIPPED
tests/test.py::TestUploader::test_disable_search PASSED
tests/test.py::TestUploader::test_discovers PASSED
tests/test.py::TestUploader::test_exits_0 PASSED
tests/test.py::TestUploader::test_exits_1 PASSED
tests/test.py::TestUploader::test_gcov SKIPPED
tests/test.py::TestUploader::test_ignore_report_1__coverage_worker10 PASSED
tests/test.py::TestUploader::test_ignore_report_2_coverage_jade PASSED
tests/test.py::TestUploader::test_ignore_report_3_include_lst PASSED
tests/test.py::TestUploader::test_ignore_report_4_inputFiles_lst PASSED
tests/test.py::TestUploader::test_ignore_report_5_createdFiles_lst PASSED
tests/test.py::TestUploader::test_ignore_report_6_scoverage_measurements_blackandwhite_xml PASSED
tests/test.py::TestUploader::test_ignore_report_7_test_hello_coverage_txt PASSED
tests/test.py::TestUploader::test_ignore_report_8_conftest_blackwhite_c_gcov PASSED
tests/test.py::TestUploader::test_ignored_path_10__tox PASSED
tests/test.py::TestUploader::test_ignored_path_11_venv PASSED
tests/test.py::TestUploader::test_ignored_path_12__venv_python_2_7 PASSED
tests/test.py::TestUploader::test_ignored_path_1_vendor PASSED
tests/test.py::TestUploader::test_ignored_path_2_node_modules PASSED
tests/test.py::TestUploader::test_ignored_path_3_js_generated_coverage PASSED
tests/test.py::TestUploader::test_ignored_path_4___pycache__ PASSED
tests/test.py::TestUploader::test_ignored_path_5_coverage_instrumented PASSED
tests/test.py::TestUploader::test_ignored_path_6_build_lib PASSED
tests/test.py::TestUploader::test_ignored_path_7_htmlcov PASSED
tests/test.py::TestUploader::test_ignored_path_8__egg_info PASSED
tests/test.py::TestUploader::test_ignored_path_9__git PASSED
tests/test.py::TestUploader::test_include_env PASSED
tests/test.py::TestUploader::test_is_report_10_luacov_report_out PASSED
tests/test.py::TestUploader::test_is_report_11_gcov_info PASSED
tests/test.py::TestUploader::test_is_report_12_nosetests_xml PASSED
tests/test.py::TestUploader::test_is_report_1_coverage_xml PASSED
tests/test.py::TestUploader::test_is_report_2_jacoco_xml PASSED
tests/test.py::TestUploader::test_is_report_3_jacocoTestResults_xml PASSED
tests/test.py::TestUploader::test_is_report_4_coverage_txt PASSED
tests/test.py::TestUploader::test_is_report_5_gcov_lst PASSED
tests/test.py::TestUploader::test_is_report_6_cov_gcov PASSED
tests/test.py::TestUploader::test_is_report_7_info_lcov PASSED
tests/test.py::TestUploader::test_is_report_8_clover_xml PASSED
tests/test.py::TestUploader::test_is_report_9_cobertura_xml PASSED
tests/test.py::TestUploader::test_none_found PASSED
tests/test.py::TestUploader::test_not_jacoco PASSED
tests/test.py::TestUploader::test_read_token_file PASSED
tests/test.py::TestUploader::test_require_branch_1 PASSED
tests/test.py::TestUploader::test_returns_none PASSED
tests/test.py::TestUploader::test_run_coverage SKIPPED
tests/test.py::TestUploader::test_run_coverage_fails PASSED
tests/test.py::TestUploader::test_send <- ../../../../../local/lib/python2.7/site-packages/unittest2/case.py SKIPPED
tests/test.py::TestUploader::test_send_error PASSED

==================== 47 passed, 20 skipped in 1.72 seconds =====================

  8. thomasrockhu commented on Nov 9, 2020

    @thomasrockhu
    thomasrockhu
    on Nov 9, 2020
    Contributor

    Closing this out as tests should be passing now.

  9. thomasrockhu
    closed this as completedon Nov 9, 2020

  10. yzgyyang commented on Nov 24, 2020

    @yzgyyang
    yzgyyang
    on Nov 24, 2020

    Closing this out as tests should be passing now.

    This is not true - OP was talking about that, by simply removing the .git directory, tests will begin to fail (I got "12 failed, 38 passed, 20 skipped in 3.19s"). This happens if one downloads a release (which doesn't include .git) and try to run the test suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions