Skip to content

Doesn't compile on OSX #39

Open
Open
Doesn't compile on OSX#39

Description

@wesrer
wesrer
opened on Dec 17, 2019

Hi, I am trying to compile from source, but I get a bunch of errors for seemingly missing files. Here's my output for rake compile --trace

** Invoke compile (first_time)
** Invoke compile:universal.x86_64-darwin18 (first_time)
** Invoke compile:nmatrix:universal.x86_64-darwin18 (first_time)
** Invoke copy:nmatrix:universal.x86_64-darwin18:2.3.7 (first_time)
** Invoke lib/ (first_time, not_needed)
** Invoke tmp/universal.x86_64-darwin18/nmatrix/2.3.7/nmatrix.bundle (first_time)
** Invoke tmp/universal.x86_64-darwin18/nmatrix/2.3.7/Makefile (first_time, not_needed)
** Invoke tmp/universal.x86_64-darwin18/nmatrix/2.3.7 (first_time, not_needed)
** Invoke ext//extconf.rb (first_time, not_needed)
** Invoke ext//blas.c (first_time, not_needed)
** Invoke ext//lapack.c (first_time, not_needed)
** Invoke ext//ruby_nmatrix.c (first_time, not_needed)
** Invoke ext//sparse.c (first_time, not_needed)
** Invoke ext//statistics.c (first_time, not_needed)
** Execute tmp/universal.x86_64-darwin18/nmatrix/2.3.7/nmatrix.bundle
cd tmp/universal.x86_64-darwin18/nmatrix/2.3.7
/usr/bin/make
compiling ../../../../ext/ruby_nmatrix.c
../../../../ext/ruby_nmatrix.c:3:10: fatal error: 'cblas.h' file not found
#include "cblas.h"
         ^~~~~~~~~
1 error generated.
make: *** [ruby_nmatrix.o] Error 1
rake aborted!
Command failed with status (2): [/usr/bin/make...]
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/file_utils.rb:67:in `block in create_shell_runner'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/file_utils.rb:57:in `sh'
/Library/Ruby/Gems/2.3.0/gems/rake-compiler-0.9.9/lib/rake/extensiontask.rb:157:in `block (2 levels) in define_compile_tasks'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/fileutils.rb:128:in `chdir'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/fileutils.rb:128:in `cd'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/file_utils_ext.rb:38:in `chdir'
/Library/Ruby/Gems/2.3.0/gems/rake-compiler-0.9.9/lib/rake/extensiontask.rb:156:in `block in define_compile_tasks'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `block in execute'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `execute'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:213:in `block in invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:237:in `block in invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:212:in `block in invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:237:in `block in invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:212:in `block in invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:237:in `block in invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:212:in `block in invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:237:in `block in invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:235:in `invoke_prerequisites'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:212:in `block in invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:182:in `invoke'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:160:in `invoke_task'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `block in top_level'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:125:in `run_with_threads'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:110:in `top_level'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:83:in `block in run'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:186:in `standard_exception_handling'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:80:in `run'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/usr/local/bin/rake:22:in `load'
/usr/local/bin/rake:22:in `<main>'
Tasks: TOP => compile => compile:universal.x86_64-darwin18 => compile:nmatrix:universal.x86_64-darwin18 => copy:nmatrix:universal.x86_64-darwin18:2.3.7 => tmp/universal.x86_64-darwin18/nmatrix/2.3.7/nmatrix.bundle

and here's my output for rake test --trace

** Invoke test (first_time)
** Execute test
/Users/elyah/Sayak/Github/numruby/lib/nmatrix/nmatrix.rb:39: warning: shadowing outer local variable - j
/Users/elyah/Sayak/Github/numruby/lib/nmatrix/nmatrix.rb:71: warning: shadowing outer local variable - j
/Users/elyah/Sayak/Github/numruby/lib/nmatrix/lapack.rb:12: warning: assigned but unused variable - m
/Users/elyah/Sayak/Github/numruby/lib/nmatrix/lapack.rb:12: warning: assigned but unused variable - n
/Users/elyah/Sayak/Github/numruby/lib/nmatrix/lapack.rb:81: warning: assigned but unused variable - lu
/Users/elyah/Sayak/Github/numruby/lib/nmatrix/lapack.rb:81: warning: assigned but unused variable - ipiv

File does not exist: /Users/elyah/Sayak/Github/numruby/lib/nmatrix.so

rake aborted!
Command failed with status (1): [ruby -w -I"lib:test:lib" -I"/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib" "/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/rake_test_loader.rb" "test/blas_test.rb" "test/elementwise_test.rb" "test/lapack_test.rb" "test/nmatrix_test.rb" "test/numruby_test.rb" "test/statistics_test.rb" ]
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/testtask.rb:130:in `block (3 levels) in define'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/file_utils.rb:57:in `sh'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/file_utils.rb:105:in `ruby'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/testtask.rb:117:in `block (2 levels) in define'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/file_utils_ext.rb:59:in `verbose'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/testtask.rb:111:in `block in define'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `block in execute'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:271:in `execute'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:213:in `block in invoke_with_call_chain'
/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:193:in `invoke_with_call_chain'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/task.rb:182:in `invoke'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:160:in `invoke_task'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `each'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:116:in `block in top_level'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:125:in `run_with_threads'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:110:in `top_level'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:83:in `block in run'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:186:in `standard_exception_handling'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/lib/rake/application.rb:80:in `run'
/Library/Ruby/Gems/2.3.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/usr/local/bin/rake:22:in `load'
/usr/local/bin/rake:22:in `<main>'
Tasks: TOP => test

Activity

  1. Uditgulati commented on Dec 19, 2019

    @Uditgulati
    Uditgulati
    on Dec 19, 2019
    Member

    You need to install BLAS and LAPACK libraries first. You need to install corresponding libraries to libopenblas and liblapack for OSX. Install steps for debian based systems is at:

    numruby/travis.sh

    Line 26 in e52b493

    sudo apt-get install -y libopenblas-dev

    Try the answers in this link https://stackoverflow.com/questions/38114201/installing-lapack-and-blas-libraries-for-c-on-mac-os and post output logs here please.

  2. developerfab commented on Dec 30, 2019

    @developerfab
    developerfab
    on Dec 30, 2019

    For OSx Catalina, I have installed libopenblasp with
    brew install openblas
    and for libblapack with
    brew install lapack

    but this doesn't works for me, I think I should add some symbolic link but I'm not sure 🤔 , @wesrer have you solved this problem?

    Sources:

    1. https://gist.github.com/zachmayer/e591cf868b3a381a01d6

  3. Uditgulati commented on Jan 5, 2020

    @Uditgulati
    Uditgulati
    on Jan 5, 2020
    Member

    @developerfab is correct. Both openblas and lapack and not properly symlinked to /usr/local. If you do brew info openblas, you get something like this:

    openblas: stable 0.3.0 (bottled), HEAD [keg-only]
Optimized BLAS library
https://www.openblas.net/
/usr/local/Cellar/openblas/0.3.0 (22 files, 139MB)
  Poured from bottle on 2018-05-31 at 20:42:55
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/openblas.rb
==> Dependencies
Required: gcc ✔
==> Options
--with-openmp
        Enable parallel computations with OpenMP
--HEAD
        Install HEAD version
==> Caveats
This formula is keg-only, which means it was not symlinked into /usr/local,
because macOS provides BLAS and LAPACK in the Accelerate framework.

    So, one way is to either forcefully link openblas and lapack to /usr/local and other is to just add /usr/opt/ paths of openblas and lapack to compilation step, so that they can be linked through that.

    I've made a PR which does this and can be found here at #40.

    @wesrer @developerfab please try this fix and let me know if it fixes the issue on your machine. I got it to work on a friend's mac.

  4. developerfab commented on Jan 6, 2020

    @developerfab
    developerfab
    on Jan 6, 2020

    I added this in my ~/.bash_profile I'm not sure if it is necessary 🤔 :

    export LDFLAGS="-L/usr/local/opt/lapack/lib"
export LDFLAGS="$LDFLAGS:-L/usr/local/opt/openblas/lib"
export CPPFLAGS="-I/usr/local/opt/lapack/include"
export CPPFLAGS="$CPPFLAGS:-I/usr/local/opt/openblas/include"
export PKG_CONFIG_PATH="/usr/local/opt/lapack/lib/pkgconfig"
export PKG_CONFIG_PATH="$PKG_CONFIG_PATH:/usr/local/opt/openblas/lib/pkgconfig"

    The output when I ran the rake compile was:

    64 warnings generated.
linking shared-object nmatrix.bundle
ld: warning: directory not found for option '-L/opt/local/lib'
cd -
mkdir -p tmp/x86_64-darwin19/stage/lib/
install -c tmp/x86_64-darwin19/nmatrix/2.6.5/nmatrix.bundle lib//nmatrix.bundle
cp tmp/x86_64-darwin19/nmatrix/2.6.5/nmatrix.bundle tmp/x86_64-darwin19/stage/lib//nmatrix.bundle

    I think that this ran perfectly.

  5. Uditgulati commented on Jan 6, 2020

    @Uditgulati
    Uditgulati
    on Jan 6, 2020
    Member

    @developerfab this pretty much does the same thing as the PR I submitted. Both provide location for openBLAS and LAPACK so that the compiler can link them. But this approach requires user to manually add the path and is also on system level and not project level. So, I believe the PR solution should be a better one.

    Also, let me know if it works only with the change done in PR and without changing the ‘ ~/.bash_profile’.

  6. iamrajiv commented on Jan 31, 2021

    @iamrajiv
    iamrajiv
    on Jan 31, 2021
    Contributor

    @wesrer Have you solved this issue? I am getting this error when I run rake test

    ➜  numruby git:(master) rake test
/Users/rajivranjansingh/Desktop/numruby/lib/numruby/lapack.rb:12: warning: assigned but unused variable - m
/Users/rajivranjansingh/Desktop/numruby/lib/numruby/lapack.rb:12: warning: assigned but unused variable - n
/Users/rajivranjansingh/Desktop/numruby/lib/numruby/lapack.rb:81: warning: assigned but unused variable - lu
/Users/rajivranjansingh/Desktop/numruby/lib/numruby/lapack.rb:81: warning: assigned but unused variable - ipiv

File does not exist: /Users/rajivranjansingh/Desktop/numruby/lib/numruby.so

rake aborted!
Command failed with status (1)
/Library/Ruby/Gems/2.6.0/gems/rake-13.0.3/exe/rake:27:in `<top (required)>'
Tasks: TOP => test
(See full trace by running task with --trace)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions