datetime64 and POSIXct in dataframes

Issue #454 resolved
Atreju Tauschinsky created an issue


I seem to be having some issues with timestamp conversion in data.frames:

starting point:

import pandas as pd
import numpy as np
import rpy2.robjects.pandas2ri as rpandas
data = pd.DataFrame({
        'Timestamp': pd.date_range('2017-01-01 00:00:00', periods=20, freq='s')

now rdata = rpandas.py2ri(data) works fine and gives me a data.frame: R object with classes: ('POSIXct', 'POSIXt'). I have some doubts about the timezones I get, but since I didn't specify one in the first place I can't complain, really...

However, if I specify UTC:

data = pd.DataFrame({
        'Timestamp': pd.date_range('2017-01-01 00:00:00', periods=20, freq='s', tz='UTC')

I get:

rdata = rpandas.py2ri(data)
[...] UserWarning: Error while trying to convert the column "Timestamp". Fall back to string conversion. The error is: Cannot pass numpy arrays with non-native byte orders at the moment.
  (name, str(e)))

Funnily enough, if I specify Europe/London I don't get a warning, but I still get characters in R rather than POSIXct.

Last point: If I convert back and forth I expect to get the same thing I started with:

data = pd.DataFrame({
        'Timestamp': pd.date_range('2017-01-01 00:00:00', periods=20, freq='s')
rdata = rpandas.py2ri(data)
new_data = rpandas.ri2py(rdata)
data.dtypes [=>Timestamp    datetime64[ns]]
new_data.dtypes [=>Timestamp    float64]

Funnily enough, if I just try to convert the column, rather than the entire dataframe, it at least tries to do the right thing (and then fails, because I'm on mac and it can't find the /etc/timezone file...):

[...]/lib/python3.5/site-packages/rpy2/robjects/ UserWarning: No file /etc/timezone
  warnings.warn('No file %s' % etc_timezone)
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "[...]/lib/python3.5/", line 745, in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
  File "[...]/lib/python3.5/site-packages/rpy2/robjects/", line 177, in ri2py_floatvector
    res = pandas.to_datetime(tuple(foo))
  File "[...]/lib/python3.5/site-packages/rpy2/robjects/", line 176, in <genexpr>
    foo = (tzone.localize(datetime.fromtimestamp(x)) for x in obj)
AttributeError: 'NoneType' object has no attribute 'localize'

any help would be much appreciated!

Comments (5)

  1. Laurent Gautier
    • changed status to open

    Still an issue with rpy2 in branch default when the pandas/numpy as a time zone specified. Reopening.

  2. Log in to comment