Error in step 3: AttributeError: 'DataFrame' object has no attribute 'append'

[James Kosmopoulos]

Hello, thank you in advance for helping solve this issue. I am running iPHoP 1.3.2 on a collection of viral genomes with a custom MAG database added to the iPHoP database. This is the command I ran:

iphop predict --fa_file [shortened]/VMAGs_at_least_10kb_N-linked.fasta --out_dir [shortened]/iphop_output -t 30 --db_dir [shortened]/Sept_2021_pub_rw  --min_score 95 --no_qc

Running each prediction tool seems to have been successful, but when iPHoP is gathering the results from each tool, this error comes up:

Looks like everything is now set up, we will first clean up the input file, and then we will start the host prediction steps themselves
[1/1/Run] Running blastn against genomes...
[1/3/Run] Get relevant blast matches...
[2/1/Run] Running blastn against CRISPR...
[2/2/Run] Get relevant crispr matches...
[3/1/Run] Running (recoded)WIsH...
### Welcome to iPHoP ###
[3/2/Run] Get relevant WIsH hits...
[4/1/Run] Running VHM s2 similarities...
[4/2/Run] Get relevant VHM hits...
[5/1/Run] Running PHP...
[5/2/Run] Get relevant PHP hits...
[6/1/Run] Running RaFAH...
[6/2/Run] Get relevant RaFAH scores...
[6.5/1/Run] Running Diamond comparison to RaFAH references...
[6.5/2/Run] Get AAI distance to RaFAH refs...
[7] Aggregating all results and formatting for TensorFlow...
[7/1] Loading all parsed data...
[7/2] Loading corresponding host taxonomy...
[7/3] Link matching genomes to representatives and filter out redundant / useless matches...
    Filtering blast data
Traceback (most recent call last):
  File "/storage2/scratch/kosmopoulos/miniconda3/envs/iphop/bin/iphop", line 10, in <module>
    sys.exit(cli())
  File "/storage2/scratch/kosmopoulos/miniconda3/envs/iphop/lib/python3.8/site-packages/iphop/iphop.py", line 128, in cli
    args["func"](args)
  File "/storage2/scratch/kosmopoulos/miniconda3/envs/iphop/lib/python3.8/site-packages/iphop/modules/master_predict.py", line 102, in main
    dataprep.aggregate(args)
  File "/storage2/scratch/kosmopoulos/miniconda3/envs/iphop/lib/python3.8/site-packages/iphop/modules/dataprep.py", line 40, in aggregate
    store_filtered = filter_hits(args,store,store_filtered,host_info)
  File "/storage2/scratch/kosmopoulos/miniconda3/envs/iphop/lib/python3.8/site-packages/iphop/modules/dataprep.py", line 215, in filter_hits
    store_filtered = store_filtered.append(df) ## Append should work directly now that we have matched all column and names nicely
  File "/home/kosmopoulos/.local/lib/python3.8/site-packages/pandas/core/generic.py", line 5989, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'append'

‌

I tried looking into other issues and can’t seem to find this error in other reports. Is this an issue with how the host MAGs are named, perhaps? I appreciate the help, and I’d be happy to provide the input files that I used as needed.

Best,

James

[Simon Roux]

Hi James,

Sorry you’re running into issues :-( this looks like a problem with the version of pandas: iPHoP expects (and should have installed) pandas version 1.3, and this function (“append” for DataFrame) was deprecated in version 2.0. We have seen similar issues before (https://bitbucket.org/srouxjgi/iphop/issues/49/iphop-v132-not-compatible-with-pandas-20) but unfortunately this gets into the sometimes challenging issues of python packages installation, which often varies between systems..

Are you working from conda, and if so, can you check that your pandas version is the expected one (1.3) ?

[James Kosmopoulos]

Hi Simon, thank you for the quick response. My pandas version was in fact 2.0.3. I am working from a conda environment, but perhaps I messed something up while creating the environment.

With the iphop environment activated, I ran

pip3 install pandas==1.3

Which was successful, but I also received the warning:

iphop 1.3.2 requires pandas>=1.3.2, but you have pandas 1.3.0 which is incompatible.

‌

Not sure what that was about, because re-running iphop predict with pandas 1.3 seemed to work and iphop is now proceeding with the rest of the aggregation/formatting step … I’m glad iphop has checkpointing :-)

Issue solved. Thank you! It was probably just an installation mess-up on my end.

[Simon Roux]

Sounds good, thanks for the update ! For the warning, you can probably remove it with “pip install pandas==1.3.2”, but I don’t remember that there is anything specific to 1.3.2 vs 1.3.0 that would cause issues down the line, so I’m cautiously optimistic that the whole thing should work now :-)

[Simon Roux]

Solved