get_iqfeed_data - on WINDOWS - 6011 cannot be opened - Error in socketConnection

Issue #1 closed
Andre Mikulec created an issue

On windows when I execute, I get the following error

> get_iqfeed_data( 'MSFT', '2015-01-01', '2016-01-01', 'hour' )

Error in socketConnection(Sys.info()["nodename"], port = 6011) :
  cannot open the connection
In addition: Warning message:
In socketConnection(Sys.info()["nodename"], port = 6011) :
  TARGETSERVER:6011 cannot be opened

I try some simple debugging

> debug(QuantTools:::.get_iqfeed)

> get_iqfeed_data( 'MSFT', '2015-01-01', '2016-01-01', 'hour' )

Browse[2]>  con2 <- socketConnection(Sys.info()["nodename"], port = 6011)

Error in socketConnection(Sys.info()["nodename"], port = 6011) :
  cannot open the connection
In addition: Warning message:
In socketConnection(Sys.info()["nodename"], port = 6011) :
  TARGETSERVER:6011 cannot be opened

Browse[2]>

Seems, like 'server = TRUE' but does still the socket waits until timeout

> con2 <- socketConnection("localhost", port = 6011, timeout = 5)

In another window at the same time, ...

C:\Users\TargetUser>netstat -a -n | grep 6011

C:\Users\TargetUser>

server = TRUE

> con2 <- socketConnection("localhost", port = 6011, timeout = 5, server = TRUE)

Error in socketConnection("localhost", port = 6011, timeout = 5, server = TRUE) :
  cannot open the connection
In addition: Warning message:
In socketConnection("localhost", port = 6011, timeout = 5, server = TRUE) :
  problem in listening on this socket

In another window at the same time, ...

C:\Users\TargetUser>netstat -a -n | grep 6011
  TCP    0.0.0.0:6011           0.0.0.0:0              LISTENING

So it was listening but DID REQUIRE 'server = TRUE'

# SAME: wait - DOES not RETURN right AWAY
con2 <- socketConnection("localhost", port = 6011, timeout = 5, server = TRUE, blocking = FALSE)
con2 <- socketConnection("localhost", port = 6011, timeout = 5, server = TRUE, blocking = TRUE)
# WAIT 5 SECONDS, THEN
Error in socketConnection("localhost", port = 6011, timeout = 5, server = TRUE) :
  cannot open the connection
In addition: Warning message:
In socketConnection("localhost", port = 6011, timeout = 5, server = TRUE) :
  problem in listening on this socket

How does parallel do it?

library(parallel)

# Calculate the number of cores
no_cores <- detectCores() - 1

# Initiate cluster
cl <- makeCluster(no_cores)
# RETURNS RIGHT AWAY

> str(cl)
List of 1
 $ :List of 3
  ..$ con :Classes 'sockconn', 'connection'  atomic [1:1]
  .. .. ..- attr(*, "conn_id")=<externalptr>
  ..$ host: chr "localhost"
  ..$ rank: int 1
  ..- attr(*, "class")= chr "SOCKnode"
 - attr(*, "class")= chr [1:2] "SOCKcluster" "cluster"


# Finish
stopCluster(cl)

I look more close.

> debug(parallel:::newPSOCKnode)

debug: con <- socketConnection("localhost", port = port, server = TRUE,
    blocking = TRUE, open = "a+b", timeout = timeout)
Browse[2]> ls()
 [1] "arg"          "cmd"          "env"          "machine"      "manual"
 [6] "master"       "methods"      "options"      "outfile"      "port"
[11] "rank"         "renice"       "rscript"      "rscript_args" "timeout"
[16] "useXDR"
Browse[2]> port
[1] 11664
Browse[2]> timeout
[1] 2592000
Browse[2]> n
# WAITS, SO  server = TRUE is correct

I put in 'server = TRUE'

> debug(QuantTools:::.get_iqfeed)
> get_iqfeed_data( 'MSFT', '2015-01-01', '2016-01-01', 'hour' )

> fixInNamespace(".get_iqfeed", pos = "package:QuantTools")

function (cmd, verbose = .settings$iqfeed_verbose) 
{
    con = socketConnection(host = .settings$iqfeed_host, port = .settings$iqfeed_port, 
        open = "a+b", blocking = TRUE, timeout = .settings$iqfeed_timeout, server = TRUE)

I try again, but still it does not return right away

> get_iqfeed_data( 'MSFT', '2015-01-01', '2016-01-01', 'hour' )
Error in socketConnection(host = .settings$iqfeed_host, port = .settings$iqfeed_port,  :
  cannot open the connection
In addition: Warning message:
In socketConnection(host = .settings$iqfeed_host, port = .settings$iqfeed_port,  :
  problem in listening on this socket

I re-examine parallel to try to see what it is doing

makeCluster
makePSOCKcluster
parallel:::newPSOCKnode


    if (machine != "localhost") {
        rshcmd <- getClusterOption("rshcmd", options)
        user <- getClusterOption("user", options)
        cmd <- shQuote(cmd)
        cmd <- paste(rshcmd, "-l", user, machine, cmd)
    }
    if (.Platform$OS.type == "windows") {
        system(cmd, wait = FALSE, input = "")
    }
    else system(cmd, wait = FALSE)

[1] "\"C:/Users/TargetUser/Desktop/R-3.3.1/App/R-Portable/bin/x64/Rscript\" --default-packages=datasets,utils,grDevices,graphics,stats,methods -e \"parallel:::.slaveRSOCK()\" MASTER=localhost PORT=11664 OUT=/dev/null TIMEOUT=2592000 XDR=TRUE"

So it executed an Rscript with

parallel:::.slaveRSOCK()\" MASTER=localhost PORT=11664

before calling socketConnection.

Hmm, I do not have any more ideas, I am not a sockets expert on Windows

The httpuv package may be something to consider

> httpuv:::makeTcpServer
function (host, port, onHeaders, onBodyData, onRequest, onWSOpen,
    onWSMessage, onWSClose)
{
    .Call("httpuv_makeTcpServer", PACKAGE = "httpuv", host, port,
        onHeaders, onBodyData, onRequest, onWSOpen, onWSMessage,
        onWSClose)
}
> library(QuantTools)
Loading required package: data.table
data.table 1.9.6  For help type ?data.table or https://github.com/Rdatatable/data.table/wiki
The fastest way to learn (by data.table authors): https://www.datacamp.com/courses/data-analysis-the-data-table-way
> get_iqfeed_data( 'MSFT', '2015-01-01', '2016-01-01', 'hour' )
Error in socketConnection(host = .settings$iqfeed_host, port = .settings$iqfeed_port,  :
  cannot open the connection
In addition: Warning message:
In socketConnection(host = .settings$iqfeed_host, port = .settings$iqfeed_port,  :
  localhost:9100 cannot be opened
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] QuantTools_0.5.0 data.table_1.9.6

loaded via a namespace (and not attached):
[1] Rcpp_0.12.7  chron_2.3-47
>

Comments (6)

  1. Stanislav Kovalevsky repo owner

    Do you have valid IQFeed subscription? If not this function will not work for you.

  2. Stanislav Kovalevsky repo owner

    Not reproduced

    library( QuantTools )
    QuantTools_settings( list(
      iqfeed_port = 9100, # in your case 6011 
      iqfeed_host = 'localhost'  
    ) )
    get_iqfeed_data( 'MSFT', '2015-01-01', '2016-01-01', 'hour' )
    
    #                      time  open  high   low  close  volume
    #    1: 2015-01-02 05:00:00 46.50 46.50 46.50 46.500     100
    #    2: 2015-01-02 08:00:00 46.72 46.72 46.70 46.700     300
    #    3: 2015-01-02 09:00:00 46.45 46.70 46.45 46.620   43521
    #    4: 2015-01-02 10:00:00 46.65 47.42 46.63 47.272 4626926
    #    5: 2015-01-02 11:00:00 47.28 47.30 46.92 46.955 4476440
    #   ---                                                     
    # 3563: 2015-12-31 16:00:00 55.67 55.84 55.45 55.450 4625142
    # 3564: 2015-12-31 17:00:00 55.48 55.65 55.40 55.530 3604497
    # 3565: 2015-12-31 18:00:00 55.53 55.60 55.48 55.530   35942
    # 3566: 2015-12-31 19:00:00 55.58 55.58 55.53 55.530    1850
    # 3567: 2015-12-31 20:00:00 55.58 55.58 55.50 55.550     400
    
  3. Lin Gao

    I'm having the same issue. This is what I initially tried: QuantTools_settings( list( + iqfeed_port = 60004, + iqfeed_host = 'localhost' + ) )

    I'm using Mac IQFeed and this is my error message:

    'Error in socketConnection(host = .settings$iqfeed_host, port = .settings$iqfeed_port, : cannot open the connection In addition: Warning message: In socketConnection(host = .settings$iqfeed_host, port = .settings$iqfeed_port, : localhost:60004 cannot be opened'

    Then I tried to replace 'localhost' with my IP, it just pause there without feedback any messages. No error message neither.

  4. Stanislav Kovalevsky repo owner

    Hi Lin,

    Seems like your IQFeed app not installed/configured properly. I'm giving you what I do on Windows or Linux/Wine with IQFeed to make it work in QuantTools. I'm sure it is similar on Mac but I couldn't even start their software successfully on OSX. So.. Run IQFeed Diagnostics.exe to make sure you get data. Then if it's okay I keep IQFeed News.exe opened so iqconnect.exe not shut down automatically when I don't retrieve data ( after ~20 seconds iqconnect shuts itself down if no application connected to it) If diagnostics fails you should contact IQFeed support to resolve the issue. Note that QuantTools has no idea if IQFeed software installed on your system, just throws the error it can't connect to socket if it can't connect to socket specified in QuantTools settings (host/port). Hope this helps.

  5. Log in to comment