Skip to content

Unable to create database for Maven project #89

Open
Open
Unable to create database for Maven project#89
Labels
CLI

Description

@fade03
fade03
opened on Nov 3, 2021

An error occurred when I used the CLI program to create a database for maven project:

❯ codeql database create java-database --language=java --command='mvn clean install'
Initializing database at /Users/jin/Desktop/micro_service_seclab/java-database.
Running build command: [mvn, clean, install]
[2021-11-03 22:03:17] [ERROR] Spawned process exited abnormally (code 137; tried to run: [/Users/jin/codeql/tools/osx64/preload_tracer, mvn, clean, install])
A fatal error occurred: Exit status 137 from command: [mvn, clean, install]

The following is the log information generated:

...
[2021-11-03 22:05:13] [DETAILS] database init> Found candidate extractor root for java: /Users/jin/codeql/java.
[2021-11-03 22:05:13] [PROGRESS] database init> Successfully loaded extractor Java (java) from /Users/jin/codeql/java.
[2021-11-03 22:05:13] [PROGRESS] database init> Created skeleton CodeQL database at /Users/jin/Desktop/micro_service_seclab/java-database. This in-progress database is ready to be populated by an extractor.
[2021-11-03 22:05:13] Plumbing command codeql database init completed.
[2021-11-03 22:05:13] [PROGRESS] database create> Running build command: [mvn, clean, install]
[2021-11-03 22:05:13] Running plumbing command: codeql database trace-command --working-dir=/Users/jin/Desktop/micro_service_seclab --index-traceless-dbs --no-db-cluster -- /Users/jin/Desktop/micro_service_seclab/java-database mvn clean install
[2021-11-03 22:05:13] Setting up tracing from specification at /Users/jin/codeql/java/tools/compiler-tracing.spec.
[2021-11-03 22:05:13] Picked up extra JVM args: '-javaagent:/Users/jin/codeql/java/tools/codeql-java-agent.jar=ignore-project,java' '-Xbootclasspath/a:/Users/jin/codeql/java/tools/codeql-java-agent.jar'
[2021-11-03 22:05:13] [PROGRESS] database trace-command> Running command in /Users/jin/Desktop/micro_service_seclab: [mvn, clean, install]
[2021-11-03 22:05:16] [ERROR] Spawned process exited abnormally (code 137; tried to run: [/Users/jin/codeql/tools/osx64/preload_tracer, mvn, clean, install])
[2021-11-03 22:05:16] Exception caught at top level: Exit status 137 from command: [mvn, clean, install]
                      com.semmle.cli2.database.DatabaseProcessCommandCommon.executeSubcommand(DatabaseProcessCommandCommon.java:188)
                      com.semmle.cli2.database.TraceCommandCommand.executeSubcommand(TraceCommandCommand.java:86)
                      com.semmle.cli2.picocli.PlumbingRunner.run(PlumbingRunner.java:110)
                      com.semmle.cli2.picocli.SubcommandCommon.runPlumbingInProcess(SubcommandCommon.java:160)
                      com.semmle.cli2.database.CreateCommand.executeSubcommand(CreateCommand.java:151)
                      com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:456)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:205)
                      com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:214)
                      com.semmle.cli2.CodeQL.main(CodeQL.java:98)

The environment variable 'mvn' has been added and is working normally.

platform: macos 12 Monterey / m1 arm64

Activity

  1. github-actions
    added
    CLI
    on Nov 3, 2021

  2. hmakholm commented on Nov 3, 2021

    @hmakholm
    hmakholm
    on Nov 3, 2021
    Contributor

    Unfortunately, database creation on M1-based Macintoshes does not currently work reliably.

    We're actively working on the matter, but it's going to take some time yet to resolve some fascinating interactions between how CodeQL instruments the build environment to observe what is being compiled, and how MacOS supports mix-and-matching between binaries compiled for Intel and ones compiled natively for M1.

  3. fade03 commented on Nov 3, 2021

    @fade03
    fade03
    on Nov 3, 2021
    Author

    Unfortunately, database creation on M1-based Macintoshes does not currently work reliably.

    We're actively working on the matter, but it's going to take some time yet to resolve some fascinating interactions between how CodeQL instruments the build environment to observe what is being compiled, and how MacOS supports mix-and-matching between binaries compiled for Intel and ones compiled natively for M1.

    Thank you for showing me what the problem is, creation of databases in other languages (such as Python) is working :-)

  4. hmakholm commented on Nov 3, 2021

    @hmakholm
    hmakholm
    on Nov 3, 2021
    Contributor

    Yes -- the difference is that for Python there's no "build process" we can attempt to observe, so CodeQL will just gobble up all the .py files it finds in the source tree and assume they're part of the program. With Java we can (usually) do better than that, but a side effect of that is that we have no "take all of the .java files" workflow ready to fall back on when the better solution fails, such as on M1 macs for the time being.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    CLI

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions