You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Running into build problems trying to build for an IBM Power9 system #22
Before I dig too deep into this, I just wanted to know whether vmd-python was ever intended to run on Power systems...? If so, I will append the particular problem I'm seeing.
Oh boy... yes regular VMD will run on anything, including Power8. When making vmd-python I removed a lot of support for architectures I didn't feel comfortable supporting. Post your error and we can totally get it working!
First problem I ran into is that in /usr/include/gnu, the stubs.h file was trying to include stubs-32.h, but that file was not present. I installed everything I could think of, and couldn't get that fixed. So I just hacked stubs.h to point to the file which WAS present (stubs-64-v2.h).
Then, when I tried to do "python setup.py build" again, I got this:
[root@234bc51125a5 vmd-python]# python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /lib64
INC: tcl.h -> /usr/include
WARNING: Could not find library file 'libsqlite3.so' in standard library directories.
Defaulting to: '/usr/lib/libsqlite3.so'
LIB: libsqlite3.so -> /usr/lib
WARNING: Could not find include file 'sqlite3.h' in standard include directories.
Defaulting to: '/usr/include/sqlite3.h'
INC: sqlite3.h -> /usr/include/sqlite3.h
LIB: libexpat.so -> /lib64
INC: expat.h -> /usr/include
WARNING: Could not find library file 'libnetcdf.so' in standard library directories.
Defaulting to: '/usr/lib/libnetcdf.so'
LIB: libnetcdf.so -> /usr/lib
WARNING: Could not find include file 'netcdf.h' in standard include directories.
Defaulting to: '/usr/include/netcdf.h'
INC: netcdf.h -> /usr/include/netcdf.h
Building with:
CC: gcc
CXX: g++
CFLAGS: -I/usr/include
CXXFLAGS: -I/usr/include
LDFLAGS: -L/usr/lib64 -Wl,--no-as-needed,-rpath-link,/usr/lib64
Setting environment variables
Deleting previous plugin directory /vmd-1.9.3/vmd-python/build/lib/vmd/plugins
Compiling plugins to /vmd-1.9.3/vmd-python/build/lib/vmd/plugins
Building Molecule File Reader plugins
/usr/bin/ld: cannot find crti.o: No such file or directory
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libstdc++.so when searching for -lstdc++
/usr/bin/ld: cannot find -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible /usr/lib64/libc.a when searching for -lc
/usr/bin/ld: skipping incompatible //usr/lib64/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible //usr/lib64/libc.a when searching for -lc
/usr/bin/ld: cannot find -lc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [abinitplugin.so] Error 1
make[1]: *** [molfilelibs] Error 1
make: *** [LINUX] Error 2
Traceback (most recent call last):
File "setup.py", line 377, in
'test': VMDTest,
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 39, in run
self.execute(self.compile_vmd, [], msg="Compiling VMD")
File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute
func(*args)
File "setup.py", line 59, in compile_vmd
check_call(cmd, cwd=srcdir)
File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/vmd-1.9.3/vmd-python/vmd/install.sh', 'LINUX', '/vmd-1.9.3/vmd-python/build/lib/vmd', '/usr']' returned non-zero exit status 2
[root@234bc51125a5 vmd-python]#
changed the title [-]Running into build problems trying to build for an IBM Power8 system[/-][+]Running into build problems trying to build for an IBM Power9 system[/+]on Feb 22, 2019
I'll add a check for Power9 systems in setup.py and put the right logic back in the configure script which hopefully will fix your problems with plugins not building.
What's the output of: python -c "import platform; print(platfom.system(), platform.machine()"?
Just FYI, I would be very happy to act as an alpha test site, as you work to get this running on Power architectures. I will not be at all disappointed if this becomes an iterative process over several days.
Also, a question: I see that your implementation supports offscreen buffers (what I have come to know as "openglpbuffer"). My belief is that a prerequisite for painting to such pbuffers is the use of EGL in VMD, but I'm not sure that is true. In any case, does your VMD-as-a-Python-library have the option to use EGL?
Alright, I've made a new branch, ppc64le with an initial attempt. The build logic for this architecture seems to be pretty much identical except for no linkage to libstdc++ (hence the errors you were seeing earlier). Let me know if it works!
I am in the middle of polishing a new release and that does include support for offscreen rendering with EGL if you build with python setup.py build --egl but this is pretty experimental for now. You'll need the EGL development headers and (EGL/egl.h) and the OpenGL entry point library (libOpenGL.so) for the build to succeed.
Hmmmm, that's weird. It's not the init.py, that's a benign error as that file is moved from elsewhere during installation.
Although the output is missing it probably is failing to find libsqlite3. I've committed some better error messages. What distro are you running?
On Debian, you'll need the following packages for the dependencies:
So, I tried on a simpler P9 system, with fewer bells and whistles, and it did give me more debug info:
[fossum@fstgbfen1 ~]$ git clone --branch ppc64le https://github.com/Eigenstate/vmd-python
Cloning into 'vmd-python'...
remote: Enumerating objects: 46, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 9038 (delta 25), reused 30 (delta 14), pack-reused 8992
Receiving objects: 100% (9038/9038), 39.11 MiB | 14.21 MiB/s, done.
Resolving deltas: 100% (3733/3733), done.
Checking out files: 100% (2498/2498), done.
[fossum@fstgbfen1 ~]$ cd vmd-python
[fossum@fstgbfen1 vmd-python]$ python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
Traceback (most recent call last):
File "setup.py", line 407, in
'test': VMDTest,
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 66, in run
self.execute(self.compile_vmd, [], msg="Compiling VMD")
File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute
func(*args)
File "setup.py", line 77, in compile_vmd
self.set_environment_variables(pydir)
File "setup.py", line 284, in set_environment_variables
os.environ["SQLITELIB"] = "-L%s" % self._find_library_dir("libsqlite3")
File "setup.py", line 201, in _find_library_dir
libfile);
RuntimeError: Could not find library 'libsqlite3.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system
[fossum@fstgbfen1 vmd-python]$
On this system, when I type "cat /etc/os-release" I get this:
[fossum@fstgbfen1 vmd-python]$ cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.5 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.5"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.5 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.5:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.5
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.5"
[fossum@fstgbfen1 vmd-python]$
I searched for libsqlite3.so in my /usr directory, and it is not there. Should I be doing a 'yum install' of one or more of the following:
sqlite-devel.ppc64le : Development tools for the sqlite3 embeddable SQL database
sqlite-doc.noarch : Documentation for sqlite
sqlite-tcl.ppc64le : Tcl module for the sqlite3 embeddable SQL database engine
sqlite2-devel.ppc64le : Development files for SQLite
sqlite2-tcl.ppc64le : Tcl bindings for sqlite
sqlite3-dbf.ppc64le : Converter of XBase / FoxPro tables to SQLite
sqlite.ppc64le : Library that implements an embeddable SQL database engine
sqlite2.ppc64le : Embeddable SQL engine in a C library
So, it eventually died, complaining that it couldn't find GL and Xext.
I did the debug version, and here's that output:
[root@f48872560257 vmd-python]# python setup.py --egl --debug install
running install
running build
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
LIB: libsqlite3.so -> /usr/lib64/
INC: sqlite3.h -> /usr/include/
LIB: libexpat.so -> /usr/lib64/
INC: expat.h -> /usr/include/
LIB: libnetcdf.so -> /usr/lib64/
INC: netcdf.h -> /usr/include/
Building with debug symbols
Traceback (most recent call last):
File "setup.py", line 451, in
'test': VMDTest,
File "/usr/lib/python2.7/site-packages/setuptools/init.py", line 143, in setup
return distutils.core.setup(**attrs)
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 417, in run
self.run_command("build")
File "/usr/lib64/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 71, in run
self.compile_vmd()
File "setup.py", line 85, in compile_vmd
self.set_environment_variables(pydir)
File "setup.py", line 332, in set_environment_variables
asandir = self._find_library_dir("libasan")
File "setup.py", line 216, in _find_library_dir
libfile);
RuntimeError: Could not find library 'libasan.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system
[root@f48872560257 vmd-python]#
Notably, when I cleared out the directory, re-cloned it, and tried to build with the debug, it died instantly the same way, rather than building a bunch of stuff for ten minutes. I wonder if there's a bug in the usage of the '--debug' flag...
@austinpagan have you verified that you have this library somewhere on the system?
RuntimeError: Could not find library 'libasan.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system
I looked into building libasan (which apparently is a package called "Address Sanitizer") from source.
The web page https://github.com/google/sanitizers/wiki/AddressSanitizerHowToBuild told me to do:
svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
But "svn" is not found on my system.
I did a "yum search" on "svn" and found a package called "git-svm".
I did a "yum install" on "git-svm".
Now, svn is on my system, and "svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm" checked out a bunch of code!
The next step involved "cmake" but it failed saying I needed cmake version 3.4.3, but yum tells me that my version (2.8.12) is the latest available.
Don't worry about building ASAN - it's a library used to help me debug things but won't help with your specific problem. I know what's going on, just hold off on doing anything until I can fix it. If you could paste the output of what "complaining it can't find GL and Xext" that would be helpful.
@damora I was hoping for a verb... (shy smile)
So, in /usr/lib64, I did "mkdir nvidia".
Then I copied the two libraries into that new directory.
Then I re-created the links to those libraries to mirror what was happening in the original directory.
Then I cleared out the vmd-python directory, and cloned a fresh one.
When I rebuilt, it gave me the exact same error as I documented above.
I shall now wait for wisdom from the package author, unless I misinterpreted your message, and you can clarify for me...
I woke with an inspiration. Maybe I also needed to create links that ended in .so (or perhaps .a)?
So I created those links (both in /usr/lib64 and /usr/lib64/nvidia), and now the command
python setup.py --egl install
ends like this:
byte-compiling /usr/lib/python2.7/site-packages/vmd/init.py to init.pyc
byte-compiling /usr/lib/python2.7/site-packages/vmd/Molecule.py to Molecule.pyc
byte-compiling /usr/lib/python2.7/site-packages/vmd/Label.py to Label.pyc
byte-compiling /usr/lib/python2.7/site-packages/vmd/Material.py to Material.pyc
running install_egg_info
running egg_info
creating vmd_python.egg-info
writing vmd_python.egg-info/PKG-INFO
writing top-level names to vmd_python.egg-info/top_level.txt
writing dependency_links to vmd_python.egg-info/dependency_links.txt
writing manifest file 'vmd_python.egg-info/SOURCES.txt'
reading manifest file 'vmd_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'bld.bat'
warning: no files found matching 'build.sh'
no previously-included directories found matching 'vmd/vmd_src/LINUXAMD64'
no previously-included directories found matching 'vmd/plugins/build'
writing manifest file 'vmd_python.egg-info/SOURCES.txt'
Copying vmd_python.egg-info to /usr/lib/python2.7/site-packages/vmd_python-3.0.2-py2.7.egg-info
running install_scripts
[root@f48872560257 vmd-python]#
Does that look like a proper end to the process?
I will now go rooting around for test programs to run, to see if this is all properly built.
I started a "python session" and typed in the first couple of lines from your README:
from vmd import molecule, vmdnumpy
import numpy as np
mask = vmdnumpy.atomselect(molid_avg, 0, "resname TYR")
The first command resulted in this:
[root@f48872560257 vmd-python]# python
Python 2.7.5 (default, Oct 30 2018, 23:44:29)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
from vmd import molecule, vmdnumpy
/usr/lib/python2.7/site-packages/vmd/init.py:17: UserWarning: Module vmd was already imported from /usr/lib/python2.7/site-packages/vmd/init.pyc, but /vmd-1.9.3/vmd-python is being added to sys.path
from pkg_resources import resource_filename
Info) EGL: node[0] bound to display[0], 4 displays total
Then followed a pile of warnings that looked like this:
Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/xsfplugin.so':
Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/xsfplugin.so
Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pbeqplugin.so':
Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pbeqplugin.so
Then, the next two commands looked like this:
import numpy as np
mask = vmdnumpy.atomselect(molid_avg, 0, "resname TYR")
Traceback (most recent call last):
File "", line 1, in
NameError: name 'molid_avg' is not defined
Are those warnings OK?
Does your distribution have any sample molecules I can use?
Also, I removed the "nvidia" directory from /usr/lib64, as the comment from @damora was intended for a different experiment I was trying, not documented in this stream, and I misunderstood him. My apologies.
On the same day that I created this problem, you commented as follows:
"Remove PBuffer support for now as getting a universal build is tricky
I need libOpenGL.so to link against for headless systems, but it's
not readily available on a lot of distros (including Travis' Ubuntu 14).
Also, there's no conda package for it, so I'm tabling this for now"
Is this also true in our "egl" branch? If not, could you show me a version
of the "render" test that I could use to try rendering something to a PBuffer?
Activity
Eigenstate commentedon Feb 22, 2019
Oh boy... yes regular VMD will run on anything, including Power8. When making vmd-python I removed a lot of support for architectures I didn't feel comfortable supporting. Post your error and we can totally get it working!
austinpagan commentedon Feb 22, 2019
First problem I ran into is that in /usr/include/gnu, the stubs.h file was trying to include stubs-32.h, but that file was not present. I installed everything I could think of, and couldn't get that fixed. So I just hacked stubs.h to point to the file which WAS present (stubs-64-v2.h).
Then, when I tried to do "python setup.py build" again, I got this:
[root@234bc51125a5 vmd-python]# python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /lib64
INC: tcl.h -> /usr/include
WARNING: Could not find library file 'libsqlite3.so' in standard library directories.
Defaulting to: '/usr/lib/libsqlite3.so'
LIB: libsqlite3.so -> /usr/lib
WARNING: Could not find include file 'sqlite3.h' in standard include directories.
Defaulting to: '/usr/include/sqlite3.h'
INC: sqlite3.h -> /usr/include/sqlite3.h
LIB: libexpat.so -> /lib64
INC: expat.h -> /usr/include
WARNING: Could not find library file 'libnetcdf.so' in standard library directories.
Defaulting to: '/usr/lib/libnetcdf.so'
LIB: libnetcdf.so -> /usr/lib
WARNING: Could not find include file 'netcdf.h' in standard include directories.
Defaulting to: '/usr/include/netcdf.h'
INC: netcdf.h -> /usr/include/netcdf.h
Building with:
CC: gcc
CXX: g++
CFLAGS: -I/usr/include
CXXFLAGS: -I/usr/include
LDFLAGS: -L/usr/lib64 -Wl,--no-as-needed,-rpath-link,/usr/lib64
Setting environment variables
Deleting previous plugin directory /vmd-1.9.3/vmd-python/build/lib/vmd/plugins
Compiling plugins to /vmd-1.9.3/vmd-python/build/lib/vmd/plugins
Building Molecule File Reader plugins
/usr/bin/ld: cannot find crti.o: No such file or directory
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libstdc++.so when searching for -lstdc++
/usr/bin/ld: cannot find -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible /usr/lib64/libc.a when searching for -lc
/usr/bin/ld: skipping incompatible //usr/lib64/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible //usr/lib64/libc.a when searching for -lc
/usr/bin/ld: cannot find -lc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [abinitplugin.so] Error 1
make[1]: *** [molfilelibs] Error 1
make: *** [LINUX] Error 2
Traceback (most recent call last):
File "setup.py", line 377, in
'test': VMDTest,
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 39, in run
self.execute(self.compile_vmd, [], msg="Compiling VMD")
File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute
func(*args)
File "setup.py", line 59, in compile_vmd
check_call(cmd, cwd=srcdir)
File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/vmd-1.9.3/vmd-python/vmd/install.sh', 'LINUX', '/vmd-1.9.3/vmd-python/build/lib/vmd', '/usr']' returned non-zero exit status 2
[root@234bc51125a5 vmd-python]#
[-]Running into build problems trying to build for an IBM Power8 system[/-][+]Running into build problems trying to build for an IBM Power9 system[/+]Eigenstate commentedon Feb 22, 2019
I'll add a check for Power9 systems in setup.py and put the right logic back in the configure script which hopefully will fix your problems with plugins not building.
What's the output of:
python -c "import platform; print(platfom.system(), platform.machine()"?austinpagan commentedon Feb 22, 2019
You had some typos, but here's what I think you want:
$ python -c "import platform; print(platform.system(), platform.machine())"
('Linux', 'ppc64le')
$
austinpagan commentedon Feb 23, 2019
Just FYI, I would be very happy to act as an alpha test site, as you work to get this running on Power architectures. I will not be at all disappointed if this becomes an iterative process over several days.
austinpagan commentedon Feb 23, 2019
Also, a question: I see that your implementation supports offscreen buffers (what I have come to know as "openglpbuffer"). My belief is that a prerequisite for painting to such pbuffers is the use of EGL in VMD, but I'm not sure that is true. In any case, does your VMD-as-a-Python-library have the option to use EGL?
Eigenstate commentedon Feb 25, 2019
Alright, I've made a new branch,
ppc64lewith an initial attempt. The build logic for this architecture seems to be pretty much identical except for no linkage to libstdc++ (hence the errors you were seeing earlier). Let me know if it works!I am in the middle of polishing a new release and that does include support for offscreen rendering with EGL if you build with
python setup.py build --eglbut this is pretty experimental for now. You'll need the EGL development headers and (EGL/egl.h) and the OpenGL entry point library (libOpenGL.so) for the build to succeed.austinpagan commentedon Feb 26, 2019
So, I did these things:
git clone --branch ppc64le https://github.com/Eigenstate/vmd-python
pip install --upgrade pip
pip install --trusted-host pypi.python.orig numpy
cd vmd-python
python setup.py build
The last command generated this output:
[vmd-python]# python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
error: None
[vmd-python]#
I will now attempt the "python setup.py install" and the next comment will contain those results.
austinpagan commentedon Feb 26, 2019
heh.
"python setup.py install" gave me almost the exact same output as "python setup.py build":
[vmd-python]# python setup.py install
running install
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
error: None
[vmd-python]#
Maybe that missing init.py file is actually something important?
austinpagan commentedon Feb 26, 2019
For future reference, can you point me to the EGL/egl.h and the libOpenGL.so?
Eigenstate commentedon Feb 26, 2019
Hmmmm, that's weird. It's not the init.py, that's a benign error as that file is moved from elsewhere during installation.
Although the output is missing it probably is failing to find libsqlite3. I've committed some better error messages. What distro are you running?
On Debian, you'll need the following packages for the dependencies:
austinpagan commentedon Feb 26, 2019
Here's what I see when I type "cat /etc/os-release":
NAME="CentOS Linux"
VERSION="7 (AltArch)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (AltArch)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7:server"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
austinpagan commentedon Feb 27, 2019
So, I tried on a simpler P9 system, with fewer bells and whistles, and it did give me more debug info:
[fossum@fstgbfen1 ~]$ git clone --branch ppc64le https://github.com/Eigenstate/vmd-python
Cloning into 'vmd-python'...
remote: Enumerating objects: 46, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 9038 (delta 25), reused 30 (delta 14), pack-reused 8992
Receiving objects: 100% (9038/9038), 39.11 MiB | 14.21 MiB/s, done.
Resolving deltas: 100% (3733/3733), done.
Checking out files: 100% (2498/2498), done.
[fossum@fstgbfen1 ~]$ cd vmd-python
[fossum@fstgbfen1 vmd-python]$ python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
Traceback (most recent call last):
File "setup.py", line 407, in
'test': VMDTest,
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 66, in run
self.execute(self.compile_vmd, [], msg="Compiling VMD")
File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute
func(*args)
File "setup.py", line 77, in compile_vmd
self.set_environment_variables(pydir)
File "setup.py", line 284, in set_environment_variables
os.environ["SQLITELIB"] = "-L%s" % self._find_library_dir("libsqlite3")
File "setup.py", line 201, in _find_library_dir
libfile);
RuntimeError: Could not find library 'libsqlite3.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system
[fossum@fstgbfen1 vmd-python]$
On this system, when I type "cat /etc/os-release" I get this:
[fossum@fstgbfen1 vmd-python]$ cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.5 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.5"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.5 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.5:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.5
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.5"
[fossum@fstgbfen1 vmd-python]$
austinpagan commentedon Feb 27, 2019
I searched for libsqlite3.so in my /usr directory, and it is not there. Should I be doing a 'yum install' of one or more of the following:
sqlite-devel.ppc64le : Development tools for the sqlite3 embeddable SQL database
sqlite-doc.noarch : Documentation for sqlite
sqlite-tcl.ppc64le : Tcl module for the sqlite3 embeddable SQL database engine
sqlite2-devel.ppc64le : Development files for SQLite
sqlite2-tcl.ppc64le : Tcl bindings for sqlite
sqlite3-dbf.ppc64le : Converter of XBase / FoxPro tables to SQLite
sqlite.ppc64le : Library that implements an embeddable SQL database engine
sqlite2.ppc64le : Embeddable SQL engine in a C library
?
42 remaining items
austinpagan commentedon Mar 13, 2019
So, it eventually died, complaining that it couldn't find GL and Xext.
I did the debug version, and here's that output:
[root@f48872560257 vmd-python]# python setup.py --egl --debug install
running install
running build
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
LIB: libsqlite3.so -> /usr/lib64/
INC: sqlite3.h -> /usr/include/
LIB: libexpat.so -> /usr/lib64/
INC: expat.h -> /usr/include/
LIB: libnetcdf.so -> /usr/lib64/
INC: netcdf.h -> /usr/include/
Building with debug symbols
Traceback (most recent call last):
File "setup.py", line 451, in
'test': VMDTest,
File "/usr/lib/python2.7/site-packages/setuptools/init.py", line 143, in setup
return distutils.core.setup(**attrs)
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 417, in run
self.run_command("build")
File "/usr/lib64/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 71, in run
self.compile_vmd()
File "setup.py", line 85, in compile_vmd
self.set_environment_variables(pydir)
File "setup.py", line 332, in set_environment_variables
asandir = self._find_library_dir("libasan")
File "setup.py", line 216, in _find_library_dir
libfile);
RuntimeError: Could not find library 'libasan.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system
[root@f48872560257 vmd-python]#
austinpagan commentedon Mar 13, 2019
Notably, when I cleared out the directory, re-cloned it, and tried to build with the debug, it died instantly the same way, rather than building a bunch of stuff for ten minutes. I wonder if there's a bug in the usage of the '--debug' flag...
damora commentedon Mar 14, 2019
@austinpagan have you verified that you have this library somewhere on the system?
RuntimeError: Could not find library 'libasan.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system
austinpagan commentedon Mar 14, 2019
"ls -ltR | grep libasan" from the root directory returns nothing.
"yum install libasan" says there's no such package.
Jeff found an rpm: libasan-9.0.1-0.8.fc30.ppc64le.rpm
We tried "rpm -i libasan-9.0.1-0.8.fc30.ppc64le.rpm" and it said:
warning: libasan-9.0.1-0.8.fc30.ppc64le.rpm: Header V3 RSA/SHA256 Signature, key ID 3c3359c4: NOKEY
error: Failed dependencies:
libm.so.6(GLIBC_2.23)(64bit) is needed by libasan-9.0.1-0.8.fc30.ppc64le
I then tried "yum localinstall libasan-9.0.1-0.8.fc30.ppc64le.rpm", and it got further, but eventually also said:
Error: Package: libasan-9.0.1-0.8.fc30.ppc64le (/libasan-9.0.1-0.8.fc30.ppc64le)
Requires: libm.so.6(GLIBC_2.23)(64bit)
I tried to search for anything that contained the string "libm.":
[root@f48872560257 vmd-1.9.3]# yum search 'libm.'
Loaded plugins: fastestmirror, ovl
Loading mirror speeds from cached hostfile
Warning: No matches found for: libm.
No matches found
[root@f48872560257 vmd-1.9.3]#
So I'm stumped...
austinpagan commentedon Mar 14, 2019
I looked into building libasan (which apparently is a package called "Address Sanitizer") from source.
The web page https://github.com/google/sanitizers/wiki/AddressSanitizerHowToBuild told me to do:
svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
But "svn" is not found on my system.
I did a "yum search" on "svn" and found a package called "git-svm".
I did a "yum install" on "git-svm".
Now, svn is on my system, and "svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm" checked out a bunch of code!
The next step involved "cmake" but it failed saying I needed cmake version 3.4.3, but yum tells me that my version (2.8.12) is the latest available.
So, I'm stumped AGAIN.
Eigenstate commentedon Mar 14, 2019
Don't worry about building ASAN - it's a library used to help me debug things but won't help with your specific problem. I know what's going on, just hold off on doing anything until I can fix it. If you could paste the output of what "complaining it can't find GL and Xext" that would be helpful.
austinpagan commentedon Mar 17, 2019
I am so sorry that I did not notice you posted this reply. I trust my work email to notify me of these things, and it let me down... sigh
The last lines before the error look like this:
Compiling hash.c --> hash.o ...
Compiling inthash.c --> inthash.o ...
Compiling intstack.c --> intstack.o ...
Compiling ptrstack.c --> ptrstack.o ...
Compiling msmpot.c --> msmpot.o ...
Compiling msmpot_compute.c --> msmpot_compute.o ...
Compiling msmpot_cubic.c --> msmpot_cubic.o ...
Compiling msmpot_setup.c --> msmpot_setup.o ...
Compiling vmdsock.c --> vmdsock.o ...
vmdsock.c: In function ‘vmdsock_accept’:
vmdsock.c:133:30: warning: pointer targets in passing argument 3 of ‘accept’ differ in signedness [-Wpointer-sign]
rc = accept(s->sd, (struct sockaddr *) &s->addr, &len);
^
In file included from /usr/include/netinet/in.h:24:0,
from /usr/include/arpa/inet.h:22,
from vmdsock.c:35:
/usr/include/sys/socket.h:244:12: note: expected ‘socklen_t * restrict’ but argument is of type ‘int *’
extern int accept (int __fd, __SOCKADDR_ARG __addr,
^
Compiling vmddlopen.c --> vmddlopen.o ...
Compiling pcre.c --> pcre.o ...
Compiling fitrms.c --> fitrms.o ...
Linking libvmd.so ...
/usr/bin/ld: cannot find -lGL
/usr/bin/ld: cannot find -lXext
collect2: error: ld returned 1 exit status
austinpagan commentedon Mar 17, 2019
On this system, I find these libraries here:
[root@f48872560257 lib64]# pwd
/usr/lib64
[root@f48872560257 lib64]# ls libGL.*
libGL.so.1 libGL.so.1.7.0
[root@f48872560257 lib64]# ls libXext.*
libXext.so.6 libXext.so.6.4.0
[root@f48872560257 lib64]#
damora commentedon Mar 18, 2019
/usr/lib64/nvidia/
austinpagan commentedon Mar 18, 2019
@damora I was hoping for a verb... (shy smile)
So, in /usr/lib64, I did "mkdir nvidia".
Then I copied the two libraries into that new directory.
Then I re-created the links to those libraries to mirror what was happening in the original directory.
Then I cleared out the vmd-python directory, and cloned a fresh one.
When I rebuilt, it gave me the exact same error as I documented above.
I shall now wait for wisdom from the package author, unless I misinterpreted your message, and you can clarify for me...
austinpagan commentedon Mar 18, 2019
I woke with an inspiration. Maybe I also needed to create links that ended in .so (or perhaps .a)?
So I created those links (both in /usr/lib64 and /usr/lib64/nvidia), and now the command
python setup.py --egl install
ends like this:
byte-compiling /usr/lib/python2.7/site-packages/vmd/init.py to init.pyc
byte-compiling /usr/lib/python2.7/site-packages/vmd/Molecule.py to Molecule.pyc
byte-compiling /usr/lib/python2.7/site-packages/vmd/Label.py to Label.pyc
byte-compiling /usr/lib/python2.7/site-packages/vmd/Material.py to Material.pyc
running install_egg_info
running egg_info
creating vmd_python.egg-info
writing vmd_python.egg-info/PKG-INFO
writing top-level names to vmd_python.egg-info/top_level.txt
writing dependency_links to vmd_python.egg-info/dependency_links.txt
writing manifest file 'vmd_python.egg-info/SOURCES.txt'
reading manifest file 'vmd_python.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'bld.bat'
warning: no files found matching 'build.sh'
no previously-included directories found matching 'vmd/vmd_src/LINUXAMD64'
no previously-included directories found matching 'vmd/plugins/build'
writing manifest file 'vmd_python.egg-info/SOURCES.txt'
Copying vmd_python.egg-info to /usr/lib/python2.7/site-packages/vmd_python-3.0.2-py2.7.egg-info
running install_scripts
[root@f48872560257 vmd-python]#
Does that look like a proper end to the process?
I will now go rooting around for test programs to run, to see if this is all properly built.
austinpagan commentedon Mar 18, 2019
I started a "python session" and typed in the first couple of lines from your README:
from vmd import molecule, vmdnumpy
import numpy as np
mask = vmdnumpy.atomselect(molid_avg, 0, "resname TYR")
The first command resulted in this:
[root@f48872560257 vmd-python]# python
Python 2.7.5 (default, Oct 30 2018, 23:44:29)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Then followed a pile of warnings that looked like this:
Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/xsfplugin.so':
Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/xsfplugin.so
Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pbeqplugin.so':
Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pbeqplugin.so
Then, the next two commands looked like this:
Are those warnings OK?
Does your distribution have any sample molecules I can use?
austinpagan commentedon Mar 18, 2019
Also, I removed the "nvidia" directory from /usr/lib64, as the comment from @damora was intended for a different experiment I was trying, not documented in this stream, and I misunderstood him. My apologies.
austinpagan commentedon Mar 18, 2019
Here's the list of links that got me past the problem:
[root@f48872560257 lib64]# ls -lt libGL.*
lrwxrwxrwx 1 root root 14 Mar 18 11:32 libGL.a -> libGL.so.1.7.0
lrwxrwxrwx 1 root root 14 Mar 18 11:31 libGL.so -> libGL.so.1.7.0
lrwxrwxrwx 1 root root 14 Jan 23 16:15 libGL.so.1 -> libGL.so.1.7.0
-rwxr-xr-x 1 root root 803280 Oct 30 21:26 libGL.so.1.7.0
[root@f48872560257 lib64]# ls -lt libXext.*
lrwxrwxrwx 1 root root 16 Mar 18 11:32 libXext.a -> libXext.so.6.4.0
lrwxrwxrwx 1 root root 16 Mar 18 11:31 libXext.so -> libXext.so.6.4.0
lrwxrwxrwx 1 root root 16 Jan 23 16:15 libXext.so.6 -> libXext.so.6.4.0
-rwxr-xr-x 1 root root 137232 Nov 22 2015 libXext.so.6.4.0
[root@f48872560257 lib64]#
austinpagan commentedon Mar 20, 2019
On the same day that I created this problem, you commented as follows:
"Remove PBuffer support for now as getting a universal build is tricky
I need libOpenGL.so to link against for headless systems, but it's
not readily available on a lot of distros (including Travis' Ubuntu 14).
Also, there's no conda package for it, so I'm tabling this for now"
Is this also true in our "egl" branch? If not, could you show me a version
of the "render" test that I could use to try rendering something to a PBuffer?