Fatal error with getting CRISPR arrays from new MAGs on the Docker version
Issue #64
closed
Hi
I noticed the error “*** FATAL ERROR *** Cannot open ./_pf446.f.tmp, Permission denied [13]\nAborted (core dumped)\n'
Warning: skipping sequences due to error: b'\npilercr v1.06” when I tried to add two bacterial MAGs from my study and 1 archaeal MAG from your Wetland MAGs collection to a new iphop database. The new database is indeed created at the end although I ran into yet another issue while trying to run it (see my issue #63). I wonder why the CRISPR arrays were not obtained by the tools here? CRISPRCasTyper tool informed me that there were no CRISPRs in two of my bacterial MAGs but it did find them in your archaeal Wetland MAG (bin.308.fa) and so, I find it rather strange that iphop didn’t get these. I ran docker with sudo too just to see if the “Permission denied” part of the error is related to user privileges but these errors didn’t go away. Below are the command-line that used and the run/error log. Any insights here would be appreciated.
sudo docker run --rm -v /data/V4_Wetland_MAGs_GTDB-tk_results/:/data/V4_Wetland_MAGs_GTDB-tk_results/:rw -v /data/iphop_db/:/data/iphop_db/:rw -v /data/V4_Wetland_bins4iphop/:/data/V4_Wetland_bins4iphop/:rw --user $(id -u):$(id -g) -t simroux/iphop:latest add_to_db --fna_dir /data/V4_Wetland_bins4iphop/ --gtdb_dir /data/V4_Wetland_MAGs_GTDB-tk_results/ --out_dir /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/ --db_dir /data/iphop_db/Sept_2021_pub_rw/
Building a new DB, current time: 08/31/2023 06:05:02
New DB name: /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Host_Genomes/New_host_genomes
New DB title: /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Host_Genomes/New_host_genomes.fna
Sequence type: Nucleotide
Keep MBits: T
Maximum file size: 1000000000B
Adding sequences from FASTA; added 1303 sequences in 0.120366 seconds.
Processing /data/V4_Wetland_bins4iphop/bin.1.fa - bin.1 - results will be in /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.1
Processing /data/V4_Wetland_bins4iphop/bin.2.fa - bin.2 - results will be in /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.2
Processing /data/V4_Wetland_bins4iphop/bin.308.fa - bin.308 - results will be in /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.308
processing files for /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.308
Reading /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.308/crt.spacers
Reading /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.308/pilercr.spacers
writing results in /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.308/merged.nr.spacers
processing files for /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.1
Reading /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.1/crt.spacers
Reading /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.1/pilercr.spacers
writing results in /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.1/merged.nr.spacers
processing files for /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.2
Reading /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.2/crt.spacers
Reading /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.2/pilercr.spacers
writing results in /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.2/merged.nr.spacers
counting kmer ...
Preparing output file
done.
Starting
[1] Get a list of genomes to import...
[2] Import information from GTDBtk trees...
Reading /data/V4_Wetland_MAGs_GTDB-tk_results/gtdbtk.ar53.decorated.tree
Reading /data/V4_Wetland_MAGs_GTDB-tk_results/gtdbtk.bac120.decorated.tree
[3] Load new host genomes in blast database...
Created nucleotide BLAST (alias) database /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Host_Genomes/Host_Genomes with 14529776 sequences
[4] Get CRISPR arrays from new MAGs and add to database...
python /opt/conda/envs/iphop/lib/python3.8/site-packages/iphop/utils/CRISPR/identify_crispr.folder.py -i /data/V4_Wetland_bins4iphop/ -o /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs
Warning: skipping sequences due to error: b'\npilercr v1.06\nhttp://www.drive5.com/piler\\nWritten by Robert C. Edgar\nThis software is donated to the public domain.\nPlease visit web site for requested citation.\n\n\n 0 s 6 Mb ( 0%) Reading sequences from /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.1/temp/seqs/56d57bde-55d7-4577-8301-95d9798b5353\n\r 0 s 43 Mb ( 4%) 100.00% \n 0 s 43 Mb ( 4%) 689 sequences, total length 2581410 bases (3 Mb)\n\n*** FATAL ERROR *** Cannot open ./_pf446.f.tmp, Permission denied [13]\nAborted (core dumped)\n'
Warning: skipping sequences due to error: b'\npilercr v1.06\nhttp://www.drive5.com/piler\\nWritten by Robert C. Edgar\nThis software is donated to the public domain.\nPlease visit web site for requested citation.\n\n\n 0 s 6 Mb ( 0%) Reading sequences from /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.2/temp/seqs/4fa0ce75-9991-44fc-af86-aa47a2a3acac\n\r 0 s 40 Mb ( 4%) 100.00% \n 0 s 40 Mb ( 4%) 235 sequences, total length 507686 bases (1 Mb)\n\n*** FATAL ERROR *** Cannot open ./_pf475.f.tmp, Permission denied [13]\nAborted (core dumped)\n'
Warning: skipping sequences due to error: b'\npilercr v1.06\nhttp://www.drive5.com/piler\\nWritten by Robert C. Edgar\nThis software is donated to the public domain.\nPlease visit web site for requested citation.\n\n\n 0 s 6 Mb ( 0%) Reading sequences from /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs/bin.308/temp/seqs/b9809a80-c4fb-40d3-aaa0-ba02717517b9\n\r 0 s 43 Mb ( 4%) 100.00% \n 0 s 43 Mb ( 4%) 379 sequences, total length 3216935 bases (3 Mb)\n\n*** FATAL ERROR *** Cannot open ./_pf549.f.tmp, Permission denied [13]\nAborted (core dumped)\n'
python /opt/conda/envs/iphop/lib/python3.8/site-packages/iphop/utils/CRISPR/get_crispr_database.py -d /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs
[5] Add new genomes to WIsH database...
[6] Add new genomes to VHM database...
[7] Add new genomes to PHP database...
[8] Now build the new host genome metadata file...
We added 2 additional bacteria genomes and 1 additional archaea genomes
[9] All done
Comments (3)
-
repo owner
-
reporter
Thank you for your response. Looks like I won’t be able to fix this for now.
-
repo owner
- changed status to closed
- Log in to comment
Hi,
So unfortunately, this looks like an error with piler_cr, over which we have little control (seems like piler_cr tries to use tmp file in the current directory, which may not be possible in your OS ?).
You can try to run “python /opt/conda/envs/iphop/lib/python3.8/site-packages/iphop/utils/CRISPR/identify_crispr.folder.py -i /data/V4_Wetland_bins4iphop/ -o /data/iphop_db/Sept_2021_w_V4_Wetland_MAGs_pub_rw/db/Tmp_CRISPRs” separately to confirm, but I’m not sure there will be a quick fix for this since it’s not part of the iPHoP code, sorry.
Best,
Simon