Skip to content

Please allow to use external packages for all dependencies (ex. zstd, flatbuffers, etc) to facilitate packaging #3668

Closed

Description

@yurivict

Activity

  1. wesm commented on Feb 16, 2019

    @wesm
    Member

    hi @yurivict what are you asking for exactly? If the user does not have zstd installed, our build system builds the package from source. We should ensure that different kinds of system-level installed packages are properly detected in https://github.com/apache/arrow/blob/master/cpp/cmake_modules/FindZSTD.cmake, though

  2. yurivict commented on Feb 17, 2019

    @yurivict
    ContributorAuthor

    There should be a switch to use only external packages, ex. USE_EXTERNAL_PACKAGES cmake variable.

    My use case: I would like to create a FreeBSD port for arrow. On FreeBSD (as in most systems) package builder doesn't allow ad-hoc file downloads started from the project build process. This is for security reasons. So there should a central switch that uses only external packages.

    If the user does not have zstd installed, our build system builds the package from source.

    Users work on some OSes, which nowadays all have packaging systems (maybe except Windows). Packages should be used. And users shouldn't be building individual softwares themselves, they should use pre-built packages provided by their systems. You are trying to cover up the lack of dependency packages when the right way is to have them created.

  3. changed the title [-]Please use zstd package instead of ExternalProject_Add[/-] [+]Please allow to use external packages for all dependencies (ex. zstd, flatbuffers, etc) to facilitate packaging[/+] on Feb 17, 2019
  4. wesm commented on Feb 17, 2019

    @wesm
    Member

    Can you please open a JIRA issue? There is also a mailing list thread about improvements to the build system to make packaging for Linux distributions easier (the same concerns apply to FreeBSD)

    https://lists.apache.org/thread.html/56e3e5e44901a4ff4f62adc93bb0d34d4c4eac48c993146172e7684e@%3Cdev.arrow.apache.org%3E

    You are trying to cover up the lack of dependency packages when the right way is to have them created.

    I think this statement is a little bit judgmental -- we are absolutely not trying to "cover up" anything. Since Arrow is still a relatively new project, we have been concerned with providing a uniform development environment to all developers of the C++ project to help promote forward library development progress. In that we have been objectively successful

    We don't have enough maintainers to handle support for all of the different third party versions of system libraries on different OS package managers. We have the ExternalProjects set up so that developers are not held back by old packages in their OS PM. This is where we need your help

  5. yurivict commented on Feb 17, 2019

    @yurivict
    ContributorAuthor
  6. yurivict commented on Feb 17, 2019

    @yurivict
    ContributorAuthor

    This is where we need your help

    I am only a volunteer FreeBSD port maintainer. I identify software that needs to be ported, create ports, and later maintain them. If a particular package is not ready to be ported and requires significant efforts, I can't invest too much time into it, because then I'll become a developer on a large number of software packages. I simply don't have enough resources for this, sorry!

    Basically, cmake provides all the tools to depend on the external packages. These are cmake scripts and pkg-config instructions. You should just use these tools, as documented in the cmake documentation.

  7. wesm commented on Feb 17, 2019

    @wesm
    Member

    You should just use these tools, as documented in the cmake documentation.

    I agree, though I take issue with the "should" and "just" words here (see http://www.humansystemsinaction.com/lullaby-language/). There is development to do here, and testing. We will also probably need to document which distros have adequate system-level dependencies to build the project, since not all of them do.

  8. kou commented on Feb 17, 2019

    @kou
    Member

    Thanks for opening the JIRA issue.
    I'm working of this issue to improve MinGW package (and .deb packages) such as #3675. 0.13.0 or 0.14.0 will support more system libraries. You may be able to work on FreeBSD port with 0.13.0 or 0.14.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions