datetime64 and POSIXct in dataframes

Issue #454 resolved
Atreju Tauschinsky
created an issue

Hi,

I seem to be having some issues with timestamp conversion in data.frames:

starting point:

import pandas as pd
import numpy as np
import rpy2.robjects.pandas2ri as rpandas
rpandas.activate()
data = pd.DataFrame({
        'Timestamp': pd.date_range('2017-01-01 00:00:00', periods=20, freq='s')
})

now rdata = rpandas.py2ri(data) works fine and gives me a data.frame: R object with classes: ('POSIXct', 'POSIXt'). I have some doubts about the timezones I get, but since I didn't specify one in the first place I can't complain, really...

However, if I specify UTC:

data = pd.DataFrame({
        'Timestamp': pd.date_range('2017-01-01 00:00:00', periods=20, freq='s', tz='UTC')
})

I get:

rdata = rpandas.py2ri(data)
[...]pandas2ri.py:61: UserWarning: Error while trying to convert the column "Timestamp". Fall back to string conversion. The error is: Cannot pass numpy arrays with non-native byte orders at the moment.
  (name, str(e)))

Funnily enough, if I specify Europe/London I don't get a warning, but I still get characters in R rather than POSIXct.

Last point: If I convert back and forth I expect to get the same thing I started with:

data = pd.DataFrame({
        'Timestamp': pd.date_range('2017-01-01 00:00:00', periods=20, freq='s')
})
rdata = rpandas.py2ri(data)
new_data = rpandas.ri2py(rdata)
data.dtypes [=>Timestamp    datetime64[ns]]
new_data.dtypes [=>Timestamp    float64]

Funnily enough, if I just try to convert the column, rather than the entire dataframe, it at least tries to do the right thing (and then fails, because I'm on mac and it can't find the /etc/timezone file...):

rpandas.ri2py(rdata[0])
[...]/lib/python3.5/site-packages/rpy2/robjects/pandas2ri.py:161: UserWarning: No file /etc/timezone
  warnings.warn('No file %s' % etc_timezone)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "[...]/lib/python3.5/functools.py", line 745, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "[...]/lib/python3.5/site-packages/rpy2/robjects/pandas2ri.py", line 177, in ri2py_floatvector
    res = pandas.to_datetime(tuple(foo))
  File "[...]/lib/python3.5/site-packages/rpy2/robjects/pandas2ri.py", line 176, in <genexpr>
    foo = (tzone.localize(datetime.fromtimestamp(x)) for x in obj)
AttributeError: 'NoneType' object has no attribute 'localize'

any help would be much appreciated!

Comments (5)

  1. Log in to comment