pyReScene in combination with CurlFtpFS is slow

[Gfy]

pyReScene often reads only a couple of bytes at a time. It seems CurlFtpFS just grabs the needed data. (no prefetching possible?) A lot of requests will occur for very little data.

"I noticed... netstat is going nuts :D"

Should pyReScene be improved for this use case or can CurlFtpFS be tweaked?

[Gfy]

"sysctl settings like tcp reuse"

1m30 for an SRR with 57 RAR files and around 40 archived files. 30m for an SRR with 93 RAR files and around 3k archived files.

[Gfy]

Some of the fuse cache parameters might result in a slightly better performance: -o noatime,ro,allow_other,ssl,no_verify_hostname,no_verify_peer,kernel_cache,auto_cache

[Gfy]

This might be another interesting approach to try out for native ftp support: https://code.google.com/p/pyfilesystem/source/browse/trunk/fs/ftpfs.py#714

A caching layer similar to segments for NZBs can be added to prevent too many ftp calls.

New interesting option: Option to skip .srs creation with pyReScene Auto