FileNotFoundError: [Errno 2] No such file or directory: '**/Wdir/php_results/php_db_Prediction_Allhost.csv'
Issue #41
closed
### Welcome to iPHoP ###
Looks like everything is now set up, we will first clean up the input file, and then we will start the host prediction steps themselves
[1/1/Skip] Skipping computation of blastn against microbial genomes...
[1/3/Skip] Skipping blast parsing...
[2/1/Skip] Skipping computation of blastn against CRISPR...
[2/2/Run] Get relevant crispr matches...
[3/1/Run] Running (recoded)WIsH...
[3/2/Run] Get relevant WIsH hits...
[4/1/Run] Running VHM s2 similarities...
[4/2/Run] Get relevant VHM hits...
[5/1/Run] Running PHP...
[5/2/Run] Get relevant PHP hits...
Traceback (most recent call last):
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/bin/iphop", line 10, in <module>
sys.exit(cli())
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/iphop.py", line 128, in cli
args["func"](args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/master_predict.py", line 91, in main
php.run_and_parse_php(args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/php.py", line 28, in run_and_parse_php
get_php_results(args["fasta_file"],args["phprawresult"],args["phpparsed"],logger,args['messages'])
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/php.py", line 42, in get_php_results
df_pred = pd.read_csv(pred_file,delimiter=',',quotechar='"', index_col=0)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/util/_decorators.py", line 311, in wrapper
return func(*args, **kwargs)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 586, in read_csv
return _read(filepath_or_buffer, kwds)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 482, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 811, in __init__
self._engine = self._make_engine(self.engine)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/io/parsers/readers.py", line 1040, in _make_engine
return mapping[engine](self.f, **self.options) # type: ignore[call-arg]
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 51, in __init__
self._open_handles(src, kwds)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/io/parsers/base_parser.py", line 222, in _open_handles
self.handles = get_handle(
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/pandas/io/common.py", line 702, in get_handle
handle = open(
FileNotFoundError: [Errno 2] No such file or directory: 'GN_iphop_results/Wdir/php_results/php_db_Prediction_Allhost.csv'
Comments (35)
-
repo owner
-
reporter
Hi Simon,
Here is the php.log:
Traceback (most recent call last):
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/utils/PHP.py", line 138, in <module>
main()
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/utils/PHP.py", line 68, in main
import pandas as pd
ModuleNotFoundError: No module named 'pandas'.
Should I re-install the package?
Thanks,
Yujie
-
repo owner
Thanks for the follow up. That error is quite strange, as pandas is used throughout, so I’m not sure why this is only an issue with PHP. Can you try to load the iPHoP conda environment, and then try something like:
$ python >>> import pandasAnd check what it says ?
-
reporter
here is the output
/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-43 Final_viral_contigs]$ python
Python 3.10.8 (main, Nov 24 2022, 14:13:03) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pandas'
-
repo owner
This is very strange, considering pandas is a required dependency for the environment. Could you double-check what is the result of
$ conda list -n iphop_envas well as
$ conda activate iphop_env $ which python?
Thanks
-
reporter
Hello,
Here is the output
(base) [yujie.zhang@ceres19-compute-92 Final_viral_contigs]$ conda list /project/yujie-metagenomics/YJ-Personal/iphop_env
# packages in environment at /software/7/apps/miniconda/4.10.1:
#
# Name Version Build Channel
(base) [yujie.zhang@ceres19-compute-92 Final_viral_contigs]$ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env
(/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-92 Final_viral_contigs]$ which python
/project/yujie-metagenomics/YJ-Personal/iphop_env/bin/python
Thanks,
Yujie
-
reporter
I also checked the /project/yujie-metagenomics/YJ-Personal/iphop_env/bin
-
repo owner
Ok so the right python is being used in the conda environment. For the first command, can you add “-n” after list, i.e.: “conda list -n iphop_env” ?
-
reporter
conda list -n /project/yujie-metagenomics/YJ-Personal/iphop_env
CondaValueError: Invalid environment name: '/project/yujie-metagenomics/YJ-Personal/iphop_env'
Characters not allowed: ('/', ' ', ':', '#')
-
repo owner
Yes, conda list -n only wants the name of your environment, not the full path, but I guess it all depends on your conda install. Two things that may be worth trying:
$ conda env list
This should get you a list of the environment names (which you could use in conda list -n)
alternatively:$ conda list -n “/project/yujie-metagenomics/YJ-Personal/iphop_env'“
Maybe the quote will help ?
-
reporter
I see. I tied put the name directly. But it went to my home directory which has no iphop_env.
EnvironmentLocationNotFound: Not a conda environment: /home/yujie.zhang/.conda/envs/iphop_env
conda env list showed this path: /project/yujie-metagenomics/YJ-Personal/iphop_env
/project/yujie-metagenomics/YJ-Personal/genomad
/project/yujie-metagenomics/YJ-Personal/iphop_env
I also tried to add quote, it showed the same output
Characters not allowed: ('/', ' ', ':', '#')
-
reporter
Hi Simon,
I fixed with -p. Here is the info
(base) [yujie.zhang@ceres19-compute-92 Final_viral_contigs]$ conda list -p /project/yujie-metagenomics/YJ-Personal/iphop_env
# packages in environment at /project/yujie-metagenomics/YJ-Personal/iphop_env:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge
_openmp_mutex 4.5 2_gnu
_r-mutex 1.0.1 anacondar_1
abseil-cpp 20210324.2 h9c3ff4c_0
absl-py 1.4.0 pyhd8ed1ab_0
aiohttp 3.8.4 py38h01eb140_1
aiosignal 1.3.1 pyhd8ed1ab_0
alsa-lib 1.2.8 h166bdaf_0
astunparse 1.6.3 pyhd8ed1ab_0
async-timeout 4.0.2 pyhd8ed1ab_0
attrs 23.1.0 pyh71513ae_1
binutils_impl_linux-64 2.40 hf600244_0
biopython 1.79 py38h0a891b7_3
blast 2.12.0 hf3cf87c_4 bioconda
blinker 1.6.2 pyhd8ed1ab_0
boost-cpp 1.74.0 h75c5d50_8
brotli-python 1.0.9 py38hfa26641_9
bwidget 1.9.14 ha770c72_1
bzip2 1.0.8 h7f98852_4
c-ares 1.19.1 hd590300_0
ca-certificates 2023.5.7 hbcca054_0
cached-property 1.5.2 hd8ed1ab_1
cached_property 1.5.2 pyha770c72_1
cachetools 4.2.4 pyhd8ed1ab_0
cairo 1.16.0 ha61ee94_1014
certifi 2023.5.7 pyhd8ed1ab_0
cffi 1.15.1 py38h4a40e3a_3
charset-normalizer 3.1.0 pyhd8ed1ab_0
click 8.0.4 py38h578d9bd_0
crisper_recognition_tool 1.2 hdfd78af_2 bioconda
cryptography 39.0.0 py38h1724139_0
curl 7.86.0 h7bff187_1
diamond 2.0.15 hb97b32f_1 bioconda
entrez-direct 16.2 he881be0_1 bioconda
expat 2.5.0 hcb278e6_1
font-ttf-dejavu-sans-mono 2.37 hab24e00_0
font-ttf-inconsolata 3.000 h77eed37_0
font-ttf-source-code-pro 2.038 h77eed37_0
font-ttf-ubuntu 0.83 hab24e00_0
fontconfig 2.14.2 h14ed4e7_0
fonts-conda-ecosystem 1 0
fonts-conda-forge 1 0
freetype 2.12.1 hca18f0e_1
fribidi 1.0.10 h36c2ea0_0
frozenlist 1.3.3 py38h0a891b7_0
gast 0.4.0 pyh9f0ad1d_0
gcc_impl_linux-64 13.1.0 hc4be1a9_0
gettext 0.21.1 h27087fc_0
gfortran_impl_linux-64 13.1.0 hd511a9b_0
giflib 5.2.1 h0b41bf4_3
google-auth 1.35.0 pyh6c4a22f_0
google-auth-oauthlib 0.4.6 pyhd8ed1ab_0
google-pasta 0.2.0 pyh8c360ce_0
graphite2 1.3.13 h58526e2_1001
grpc-cpp 1.43.2 h9e046d8_3
grpcio 1.43.0 py38hdd6454d_0
gsl 2.7 he838d99_0
gxx_impl_linux-64 13.1.0 hc4be1a9_0
h5py 3.8.0 nompi_py38hd5fa8ee_100
harfbuzz 6.0.0 h8e241bc_0
hdf5 1.12.2 nompi_h2386368_100
hmmer 3.3.2 hdbdd923_4 bioconda
icu 70.1 h27087fc_0
idna 3.4 pyhd8ed1ab_0
importlib-metadata 6.7.0 pyha770c72_0
iphop 1.3.2 pyhdfd78af_0 bioconda
joblib 1.0.1 pyhd8ed1ab_0
jpeg 9e h0b41bf4_3
keras 2.7.0 pyhd8ed1ab_0
keras-preprocessing 1.1.2 pyhd8ed1ab_0
kernel-headers_linux-64 2.6.32 he073ed8_15
keyutils 1.6.1 h166bdaf_0
krb5 1.19.3 h3790be6_0
lcms2 2.14 h6ed2654_0
ld_impl_linux-64 2.40 h41732ed_0
lerc 4.0.0 h27087fc_0
libblas 3.9.0 17_linux64_openblas
libcblas 3.9.0 17_linux64_openblas
libcups 2.3.3 h3e49a29_2
libcurl 7.86.0 h7bff187_1
libdeflate 1.14 h166bdaf_0
libedit 3.1.20191231 he28a2e2_2
libev 4.33 h516909a_1
libexpat 2.5.0 hcb278e6_1
libffi 3.4.2 h7f98852_5
libgcc 7.2.0 h69d50b8_2
libgcc-devel_linux-64 13.1.0 he3cc6c4_0
libgcc-ng 13.1.0 he5830b7_0
libgfortran-ng 13.1.0 h69a702a_0
libgfortran5 13.1.0 h15d22d2_0
libglib 2.76.3 hebfc3b9_0
libgomp 13.1.0 he5830b7_0
libiconv 1.17 h166bdaf_0
libidn2 2.3.4 h166bdaf_0
liblapack 3.9.0 17_linux64_openblas
libnghttp2 1.51.0 hdcd2b5c_0
libnsl 2.0.0 h7f98852_0
libopenblas 0.3.23 pthreads_h80387f5_0
libpng 1.6.39 h753d276_0
libprotobuf 3.19.6 h3eb15da_0
libsanitizer 13.1.0 hfd8a6a1_0
libsqlite 3.42.0 h2797004_0
libssh2 1.10.0 haa6b8db_3
libstdcxx-devel_linux-64 13.1.0 he3cc6c4_0
libstdcxx-ng 13.1.0 hfd8a6a1_0
libtiff 4.4.0 h82bc61c_5
libunistring 0.9.10 h7f98852_0
libuuid 2.38.1 h0b41bf4_0
libwebp-base 1.3.1 hd590300_0
libxcb 1.13 h7f98852_1004
libxml2 2.10.3 hca2bb57_4
libzlib 1.2.13 hd590300_5
make 4.3 hd18ef5c_1
markdown 3.4.3 pyhd8ed1ab_0
markupsafe 2.1.3 py38h01eb140_0
multidict 6.0.4 py38h1de0b5d_0
ncurses 6.4 hcb278e6_0
numpy 1.23.5 py38h7042d01_0
oauthlib 3.2.2 pyhd8ed1ab_0
openjdk 17.0.3 h58dac75_5
openssl 1.1.1u hd590300_0
opt_einsum 3.3.0 pyhd8ed1ab_1
packaging 23.1 pyhd8ed1ab_0
pandas 1.3.5 py38h43a58ef_0
pango 1.50.14 hd33c08f_0
pcre 8.45 h9c3ff4c_0
pcre2 10.40 hc3806b6_0
perl 5.22.0.1 0
perl-app-cpanminus 1.7043 pl5.22.0_0 bioconda
perl-archive-tar 2.18 pl5.22.0_2 bioconda
perl-bioperl 1.6.924 4 bioconda
perl-carp 1.38 pl5.22.0_0 bioconda
perl-common-sense 3.74 0 bioconda
perl-compress-raw-bzip2 2.069 1 bioconda
perl-compress-raw-zlib 2.069 3 bioconda
perl-data-dumper 2.161 pl5.22.0_0 bioconda
perl-exporter 5.72 pl5.22.0_0 bioconda
perl-exporter-tiny 0.042 1 bioconda
perl-extutils-makemaker 7.24 pl5.22.0_1 bioconda
perl-io-compress 2.069 pl5.22.0_2 bioconda
perl-io-zlib 1.10 1 bioconda
perl-json 2.90 1 bioconda
perl-json-xs 2.34 0 bioconda
perl-list-moreutils 0.428 pl5.22.0_0 bioconda
perl-pathtools 3.73 h470a237_2 bioconda
perl-scalar-list-utils 1.45 2 bioconda
perl-test-more 1.001002 pl5.22.0_0 bioconda
perl-threaded 5.32.1 hdfd78af_1 bioconda
perl-yaml 1.24 pl522_1 bioconda
piler-cr 1.06 h4ac6f70_4 bioconda
pip 23.1.2 pyhd8ed1ab_0
pixman 0.40.0 h36c2ea0_0
platformdirs 3.8.0 pyhd8ed1ab_0
pooch 1.7.0 pyha770c72_3
prodigal 2.6.3 h031d066_6 bioconda
protobuf 3.19.6 py38h8dc9893_0
pthread-stubs 0.4 h36c2ea0_1001
pyasn1 0.4.8 py_0
pyasn1-modules 0.2.7 py_0
pycparser 2.21 pyhd8ed1ab_0
pyjwt 2.7.0 pyhd8ed1ab_0
pyopenssl 23.2.0 pyhd8ed1ab_1
pysocks 1.7.1 pyha2e5f31_6
python 3.8.15 h257c98d_0_cpython
python-dateutil 2.8.2 pyhd8ed1ab_0
python-flatbuffers 2.0 pyhd8ed1ab_0
python_abi 3.8 3_cp38
pytz 2023.3 pyhd8ed1ab_0
pyu2f 0.1.5 pyhd8ed1ab_0
r-base 4.0.5 hb87df5d_8
r-lattice 0.20_45 r40hcfec24a_0
r-matrix 1.4_1 r40h0154571_0
r-ranger 0.13.1 r40h03ef668_0
r-rcpp 1.0.9 r40h7525677_1
r-rcppeigen 0.3.3.9.2 r40h43535f1_0
re2 2022.02.01 h9c3ff4c_0
readline 8.2 h8228510_1
requests 2.31.0 pyhd8ed1ab_0
requests-oauthlib 1.3.1 pyhd8ed1ab_0
rsa 4.9 pyhd8ed1ab_0
scikit-learn 0.22.2.post1 py38hcdab131_0
scipy 1.10.1 py38h59b608b_3
sed 4.8 he412f7d_0
setuptools 59.8.0 py38h578d9bd_1
six 1.16.0 pyh6c4a22f_0
snappy 1.1.10 h9fff704_0
sqlite 3.42.0 h2c6b66d_0
sysroot_linux-64 2.12 he073ed8_15
tensorboard 2.6.0 pyhd8ed1ab_1
tensorboard-data-server 0.6.1 py38h2b5fc30_4
tensorboard-plugin-wit 1.8.1 pyhd8ed1ab_0
tensorflow 2.7.0 pypi_0 pypi
tensorflow-base 2.7.1 cpu_py38ha28dbe6_0
tensorflow-decision-forests 0.2.2 pypi_0 pypi
tensorflow-estimator 2.7.1 cpu_py38h4e23bc6_0
termcolor 2.3.0 pyhd8ed1ab_0
tk 8.6.12 h27826a3_0
tktable 2.10 hb7b940f_3
typing-extensions 4.7.1 hd8ed1ab_0
typing_extensions 4.7.1 pyha770c72_0
urllib3 2.0.3 pyhd8ed1ab_1
werkzeug 2.3.6 pyhd8ed1ab_0
wget 1.20.3 ha56f1ee_1
wheel 0.40.0 pyhd8ed1ab_0
wrapt 1.15.0 py38h1de0b5d_0
xorg-fixesproto 5.0 h7f98852_1002
xorg-inputproto 2.3.2 h7f98852_1002
xorg-kbproto 1.0.7 h7f98852_1002
xorg-libice 1.0.10 h7f98852_0
xorg-libsm 1.2.3 hd9c2040_1000
xorg-libx11 1.8.4 h0b41bf4_0
xorg-libxau 1.0.11 hd590300_0
xorg-libxdmcp 1.1.3 h7f98852_0
xorg-libxext 1.3.4 h0b41bf4_2
xorg-libxfixes 5.0.3 h7f98852_1004
xorg-libxi 1.7.10 h7f98852_0
xorg-libxrender 0.9.10 h7f98852_1003
xorg-libxt 1.3.0 hd590300_0
xorg-libxtst 1.2.3 h7f98852_1002
xorg-recordproto 1.14.2 h7f98852_1002
xorg-renderproto 0.11.1 h7f98852_1002
xorg-xextproto 7.3.0 h0b41bf4_1003
xorg-xproto 7.0.31 h7f98852_1007
xz 5.2.6 h166bdaf_0
yarl 1.9.2 py38h01eb140_0
zipp 3.15.0 pyhd8ed1ab_0
zlib 1.2.13 hd590300_5
zstd 1.5.2 h3eb15da_6
Thanks,
Yujie
-
repo owner
Ok, I’m officially very confused as we see pandas in this list, so I don’t understand why PHP can’t see this module..
Can you try the following:
$ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env $ python /project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/utils/PHP.py
-
reporter
(/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-92 Final_viral_contigs]$ python /project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/utils/PHP.py
/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/utils/
please input a correct virusFastaFileDir.
done.
-
repo owner
Ok, so no specific issue here.. I think there is something wrong with your python install though, because conda mentions that python 3.8 is installed, yet when calling python from the command line it looks like you get python 3.10, so the discrepancy may be here. Can you try the following
$ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env $ which python3 $ python3 $ >>> import pandas $ >>> exit() $ cat GN_iphop_results/Wdir/php.cmd $ bash GN_iphop_results/Wdir/php.cmdThis will likely fail, but if so, let’s maybe try to re-install pandas for the 3.10 version as well, i.e.
$ conda activate $ pip install pandasAnd then retry
$ bash GN_iphop_results/Wdir/php.cmd
-
reporter
Thanks! After I did this, no error. Should I just run the iphop again?
$ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env $ which python3 $ python3 $ >>> import pandas $ >>> exit() $ cat GN_iphop_results/Wdir/php.cmd $ bash GN_iphop_results/Wdir/php.cmd
-
repo owner
Weird, but yes, it seems like you could try running again ? But you need to either run in a completely new folder, or make sure to remove the previous PHP output folder (i.e. remove the folder named “php_results” in Wdir/ as well as the file named “phpparsed.csv” if it exists).
-
reporter
Thanks! I checked the php_result in wdir and didn’t see the file named “phpparsed.csv” . Will try to remove the previous PHP output folder and do it again
-
reporter
[6.5/2/Run] Get AAI distance to RaFAH refs...
Traceback (most recent call last):
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/bin/iphop", line 10, in <module>
sys.exit(cli())
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/iphop.py", line 128, in cli
args["func"](args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/master_predict.py", line 95, in main
aai_to_ref.run_and_parse_aai(args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/aai_to_ref.py", line 30, in run_and_parse_aai
get_aai_results(faa_file,db_info,args["aai_out"],args["aai_parsed"])
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/aai_to_ref.py", line 56, in get_aai_results
for index, r in enumerate(SeqIO.parse(faa, 'fasta')):
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/__init__.py", line 605, in parse
return iterator_generator(handle)
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/FastaIO.py", line 223, in __init__
super().__init__(source, mode="t", fmt="Fasta")
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/Interfaces.py", line 45, in __init__
self.stream = open(source, "r" + mode)
FileNotFoundError: [Errno 2] No such file or directory: 'GN_iphop_results-new/Wdir/rafah_out/Full_CDS_Prediction.faa'
Hi Simon,
I followed the guideline to edit ./etc/conda/activate.d/update_bioperl.sh and ./etc/conda/deactivate.d/update_bioperl.sh. Still shown the above error. Did I do something wrong?
./etc/conda/activate.d/update_bioperl.sh
#!/bin/sh
export OLD_PERL5LIB=$PERL5LIB
export PERL5LIB=/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/perl5/site_perl/5.22.0
./etc/conda/deactivate.d/update_bioperl.sh
#!/bin/sh
export PERL5LIB=${OLD_PERL5LIB}
unset OLD_PERL5LIB
I also tired to skip the RaFAH by running the same command line again. It showed the same error above.
Thanks,
Yujie
-
repo owner
Yes, this command will unfortunately fail if RaFAH is skipped. Can you check the content of the folder /project/yujie-metagenomics/YJ-Personal/iphop_env/lib/perl5/site_perl/5.22.0 ?
And to make sure BioPerl is correctly installed, can you test:
$ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env $ perl -e “use Bio::SeqIO;”Finally, we should look into the file “rafah.log” in Wdir, see why there was no cds predicted with RaFAH.
-
reporter
\$ cd /project/yujie-metagenomics/YJ-Personal/iphop_env/lib/perl5/site_perl/5.22.0
\$ ls
Algorithm Carp Devel File Sub x86_64-linux-thread-multi
App Carp.pm Exception IO Test YAML
Archive Class Exporter JSON Text YAML.pm
Bio Compress Exporter.pm JSON.pm URI YAML.pod
Capture Data ExtUtils List URI.pm
(/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-98 5.22.0]$ perl -e “use Bio::SeqIO;”
perl: error while loading shared libraries: libnsl.so.1: cannot open shared object file: No such file or directory
bash: ”: command not found
-
repo owner
Ooh, so this is deeper than a BioPerl issue, unfortunately. This kind of error can be a pain to fix, but let’s try. We will need to install the library into the conda environment, then try again, i.e.
$ conda install -p /project/yujie-metagenomics/YJ-Personal/iphop_env -c conda-forge libnsl $ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env $ perl -e "use Bio::SeqIO;"
-
reporter
conda install -p /project/yujie-metagenomics/YJ-Personal/iphop_env -c conda-forge libnsl
Collecting package metadata (current_repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.10.1
latest version: 23.5.2
Please update conda by running
$ conda update -n base -c defaults conda
# All requested packages already installed.
(base) [yujie.zhang@ceres19-compute-98 Final_viral_contigs]$ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env
(/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-98 Final_viral_contigs]$ perl -e "use Bio::SeqIO;"
perl: error while loading shared libraries: libnsl.so.1: cannot open shared object file: No such file or directory
-
repo owner
Well, at this point this is beyond iPHoP unfortunately, it looks like your perl installation is broken. Do you also get this error when trying to use perl outside of the conda environment ? If so, then this looks like it will require the installation of additional packages (https://www.sultandba.com/2022/03/how-to-solve-error-while-loading-shared.html).
-
reporter
I see. Thanks for your help!
Yujie
-
reporter
Hi Simon,
It’s so weird. I tied to use perl outside of the conda environment and it failed. Then I tired to yum install libnsl outside of the conda environment and showed the errors. But after it, I tried to use perl outside of the conda environment and in iphop env. it all works :
(base) [yujie.zhang@ceres19-compute-98 Final_viral_contigs]$ perl -e "use Bio::SeqIO
> ^C
(base) [yujie.zhang@ceres19-compute-98 Final_viral_contigs]$ conda activate /project/yujie-metagenomics/YJ-Personal/iphop_env
(/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-98 Final_viral_contigs]$ perl -e "use Bio::SeqIO
>
So I just need to remove the old RaFAH output and run the same command line again, right?
-
repo owner
Oh weird, maybe this was a temporary problem ? But yes you are right, you need to remove everything named “RaFAH” in Wdir, and rerun. Fingers crossed !
-
reporter
[6.5/2/Run] Get AAI distance to RaFAH refs...
Traceback (most recent call last):
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/bin/iphop", line 10, in <module>
sys.exit(cli())
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/iphop.py", line 128, in cli
args["func"](args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/master_predict.py", line 95, in main
aai_to_ref.run_and_parse_aai(args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/aai_to_ref.py", line 30, in run_and_parse_aai
get_aai_results(faa_file,db_info,args["aai_out"],args["aai_parsed"])
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/aai_to_ref.py", line 56, in get_aai_results
for index, r in enumerate(SeqIO.parse(faa, 'fasta')):
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/__init__.py", line 605, in parse
return iterator_generator(handle)
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/FastaIO.py", line 223, in __init__
super().__init__(source, mode="t", fmt="Fasta")
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/Interfaces.py", line 45, in __init__
self.stream = open(source, "r" + mode)
FileNotFoundError: [Errno 2] No such file or directory: 'GN_iphop_results-new/Wdir/rafah_out/Full_CDS_Prediction.faa'
(/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-98 Final_viral_contigs]$ perl -e "use Bio::SeqIO
Failed again… I probably need to run it in local computer. Thanks!
Yujie
-
repo owner
What does the log / stdout say prior to “[6.5/2/Run]” ? This does not look like the same issue
-
reporter
### Welcome to iPHoP ###
Looks like everything is now set up, we will first clean up the input file, and then we will start the host prediction steps themselves
[1/1/Skip] Skipping computation of blastn against microbial genomes...
[1/3/Skip] Skipping blast parsing...
[2/1/Skip] Skipping computation of blastn against CRISPR...
[2/2/Skip] Skipping crispr parsing...
[3/1/Skip] Skipping computation of WIsH scores...
[3/2/Skip] Skipping WIsH parsing...
[4/1/Skip] Skipping computation of VHM s2 similarities...
[4/2/Skip] Skipping VHM parsing...
[5/1/Skip] Skipping computation of PHP scores...
[5/2/Skip] Skipping PHP parsing...
[6/1/Run] Running RaFAH...
[6/2/Run] Get relevant RaFAH scores...
RaFAH results were empty this may be ok, but is still unusual, so you may want to check the rafah log (Wdir/rafah.log)
[6.5/1/Run] Running Diamond comparison to RaFAH references...
[6.5/2/Run] Get AAI distance to RaFAH refs...
Traceback (most recent call last):
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/bin/iphop", line 10, in <module>
sys.exit(cli())
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/iphop.py", line 128, in cli
args["func"](args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/master_predict.py", line 95, in main
aai_to_ref.run_and_parse_aai(args)
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/aai_to_ref.py", line 30, in run_and_parse_aai
get_aai_results(faa_file,db_info,args["aai_out"],args["aai_parsed"])
File "/project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/modules/aai_to_ref.py", line 56, in get_aai_results
for index, r in enumerate(SeqIO.parse(faa, 'fasta')):
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/__init__.py", line 605, in parse
return iterator_generator(handle)
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/FastaIO.py", line 223, in __init__
super().__init__(source, mode="t", fmt="Fasta")
File "/home/yujie.zhang/.local/lib/python3.8/site-packages/Bio/SeqIO/Interfaces.py", line 45, in __init__
self.stream = open(source, "r" + mode)
FileNotFoundError: [Errno 2] No such file or directory: 'GN_iphop_results-new/Wdir/rafah_out/Full_CDS_Prediction.faa'
(/project/yujie-metagenomics/YJ-Personal/iphop_env) [yujie.zhang@ceres19-compute-98 Final_viral_contigs]$ perl -e "use Bio::SeqIO
^C
-
repo owner
Ok so it tried to run RaFAH, but apparently was stopped early on. Can you check in the “RaFAH.log” and “RaFAH.cmd” files ?
-
reporter
(base) [yujie.zhang@ceres19-compute-98 Wdir]$ cat rafah.log
perl: error while loading shared libraries: libnsl.so.1: cannot open shared object file: No such file or directory
(base) [yujie.zhang@ceres19-compute-98 Wdir]$ cat rafah.cmd
perl /project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/utils/RaFAH_v0.3.pl /project/yujie-metagenomics/YJ-Personal/iphop_env/lib/python3.8/site-packages/iphop/utils iphop_db/Sept_2021_pub_rw/db --genomes_dir GN_iphop_results-new/Wdir/split_input/ --extension fasta --output_dir GN_iphop_results-new/Wdir/rafah_out --file_prefix Full --threads 1 --predict > GN_iphop_results-new/Wdir/rafah.log 2>&1
-
repo owner
Oh so we are back with the perl error, that is weird .. I don’t understand what is happening, especially since you can actually run perl without an error in this same conda environment, sorry.
-
reporter
Yeah, thank for your help! I will try it in another environment.
Best,
Yujie
-
repo owner
- changed status to closed
- Log in to comment
Hi,
This looks like an issue with PHP, can you check the content of “php.log” in the folder “Wdir” ?
Thanks
Best,
Simon