Launch metview in Jupyter when installed in different environments #48
Description
I have Jupyter Notebooks installed in my base environment. When creating/using separate environments, I use ipykernel to make them accessible as Jupyter kernels. I recently installed metview into an existing conda environment (entitled "monsoon-pr") following:
conda install 'metview-batch>=5.16.0' -c conda-forge
When launching a new notebook, activating the environment where metview is installed (via the Kernel dropdown), and attempting to import metview following import metview as mv, I get the following error:
Could not run the Metview executable ('metview'); check that the binaries for Metview (version 5 at least) are installed and are in the PATH.
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
Cell In[1], line 1
----> 1 import metview as mv
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/__init__.py:44
42 except Exception as exp:
43 if "METVIEW_PYTHON_ONLY" not in os.environ:
---> 44 raise exp
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/__init__.py:28
25 if len(sys.argv) != 2 or sys.argv[0] != "-m" or sys.argv[1] != "selfcheck":
27 try:
---> 28 from . import bindings as _bindings
30 _bindings.bind_functions(globals(), module_name=__name__)
32 # Remove "_bindings" from the public API.
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/bindings.py:196
192 signal.signal(signal.SIGTERM, self.sigterm)
193 signal.signal(signal.SIGALRM, self.sigalarm)
--> 196 mi = MetviewInvoker()
198 try:
199 ffi = cffi.FFI()
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/bindings.py:114, in MetviewInvoker.__init__(self)
106 if "METVIEW_PYTHON_ONLY" not in os.environ:
107 print(
108 "Could not run the Metview executable ('"
109 + metview_startup_cmd
(...)
112 "and are in the PATH."
113 )
--> 114 raise exp
116 # wait for Metview to respond...
117 wait_start = time.time()
File ~/Miniconda/envs/metview/lib/python3.9/site-packages/metview/bindings.py:104, in MetviewInvoker.__init__(self)
101 print(metview_flags)
103 try:
--> 104 subprocess.Popen(metview_flags)
105 except Exception as exp: # pragma: no cover
106 if "METVIEW_PYTHON_ONLY" not in os.environ:
File ~/Miniconda/envs/metview/lib/python3.9/subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
947 if self.text_mode:
948 self.stderr = io.TextIOWrapper(self.stderr,
949 encoding=encoding, errors=errors)
--> 951 self._execute_child(args, executable, preexec_fn, close_fds,
952 pass_fds, cwd, env,
953 startupinfo, creationflags, shell,
954 p2cread, p2cwrite,
955 c2pread, c2pwrite,
956 errread, errwrite,
957 restore_signals,
958 gid, gids, uid, umask,
959 start_new_session)
960 except:
961 # Cleanup if the child failed starting.
962 for f in filter(None, (self.stdin, self.stdout, self.stderr)):
File ~/Miniconda/envs/metview/lib/python3.9/subprocess.py:1821, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session)
1819 if errno_num != 0:
1820 err_msg = os.strerror(errno_num)
-> 1821 raise child_exception_type(errno_num, err_msg, err_filename)
1822 raise child_exception_type(err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'metview'
I'm assuming this error is because metview is operating out of a different environment than Jupyter. In my Jupyter notebook, os.environ['PATH'] gives my Jupyter path as:
'/jet/home/sferrett/.local/bin:/jet/home/sferrett/bin:/opt/packages/psc.allocations.user/bin:/opt/packages/allocations/bin:/jet/home/sferrett/Miniconda/bin:/jet/home/sferrett/Miniconda/condabin:/jet/home/sferrett/.local/bin:/jet/home/sferrett/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/packages/interact/bin:/opt/puppetlabs/bin'
While in my conda environment with metview (monsoon-pr) in Terminal, my path is:
/jet/home/sferrett/Miniconda/envs/monsoon-pr/bin:/jet/home/sferrett/Miniconda/condabin:/jet/home/sferrett/.local/bin:/jet/home/sferrett/bin:/opt/packages/psc.allocations.user/bin:/opt/packages/allocations/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/packages/interact/bin:/opt/puppetlabs/bin
Is there a way to set the notebook PATH so that it can retrieve metview from the conda environment? I don't want to have to install Jupyter within my conda environment.
Activity
iainrussell commentedon Mar 10, 2023
Hi @savannahferretti,
I'm surprised it can't find Metview, but one thing you can do is, from a terminal where Metview can be found, type
which metview. This will give the full path to the Metview startup script. You can then set environment variable METVIEW_PYTHON_START_CMD to this path (including the 'metview' at the end), either on the command line before starting Jupyter, or you can set this env var from the Python code in your Jupyter notebook before youimport metview.Good luck!
Cheers,
Iain
savannahferretti commentedon Mar 10, 2023
Thanks for the quick response! I am stilling having an issue loading the package. I am unsure if it has to do with the fact that I'm SSH tunneling into Jupyter Notebooks from an HPC. Here is what I did:
In my first Terminal window, I SSHed into the HPC, was allocated a compute node, and launched Jupyter Notebooks. I opened a second Terminal window, mapped the port to my local, then launched Jupyter in my browser. Also from this second window, I activated the environment where metview is installed and executed
which metview, which returned:I deactivated the environment, and from my base environment (where Jupyter Notebooks is installed), I executed:
Executing
printenv METVIEW_PYTHON_START_CMDreturned the same path.After launching my notebook and running
import metview as mv, I still get an error:I tried relaunching Jupyter, but still the same error.
iainrussell commentedon Mar 11, 2023
Hi,
OK, this should help: first, to make sure the environment inside your notebook has the right path set, you can set the env var before importing metview, and to also get more debug output:
savannahferretti commentedon Mar 14, 2023
Hi,
Thanks for the response. Trying the suggestion above still gives the same error (i.e., it cannot find metview):
iainrussell commentedon Jun 18, 2023
Hi Savannah,
Looks like this one dropped off the radar...! Did you manage to fix this issue? It looks like you were in an environment where Metview's Python interface was not installed.
Cheers,
Iain