Source

ftputil / ftputil.txt

Full commit
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
NAME

    ftputil - abstract Python interface for FTP sessions

----------------------------------------------------------------------
DESCRIPTION

The ftputil module is a high-level interface to the ftplib module. The
FTPHost objects generated from it allow many operations similar to
those of os and os.path.

Examples:

# download some files from the login directory
host = ftputil.FTPHost('ftp.domain.com', 'user', 'password')
names = host.listdir(host.curdir)
for name in names:
    if host.path.isfile(name):
        host.download(name, name, 'b')  # remote, local, binary mode

# make a new directory and copy a remote file into it
host.mkdir('newdir')
source = host.file('index.html', 'r')  # file-like object
target = host.file('newdir/index.html', 'w')  # file-like object
host.copyfileobj(source, target)  # similar to shutil.copyfileobj
source.close()
target.close()

Also, there are FTPHost.lstat and FTPHost.stat to request size and
modification time of a file. The latter can also follow links, similar
to os.stat. Even FTPHost.path.walk works.

The distribution contains a regression test module and a custom
UserTuple class to provide stat results with Python 2.1.

----------------------------------------------------------------------
MODULE CONTENTS

Exception hierarchy
-------------------

The exceptions are organized as follows:

    FTPError
        FTPOSError(FTPError, OSError)
            TemporaryError(FTPOSError)
            PermanentError(FTPOSError)
            ParserError(FTPOSError)
        FTPIOError(FTPError)
        RootDirError(FTPError)

FTPError is the root of the exception hierarchy of the module.

FTPOSError is derived from OSError. This is for similarity between the
os module and FTPHost objects. Compare

    try:
        os.chdir('nonexisting_directory')
    except OSError:
        ...

with

    host = ftputil.FTPHost('host', 'user', 'password')
    try:
        host.chdir('nonexisting_directory')
    except OSError:
        ...

Imagine a function

    def func(path, file):
        ...

which works on the local file system and catches OSErrors. If you
change the parameter list to

    def func(path, file, os=os):
        ...

where os denotes the os module, you can call the function also as

    host = ftputil.FTPHost('host', 'user', 'password')
    func(path, file, os=host)

to use the same code for a local and remote file system. Another
similarity between OSError and FTPOSError is that the latter holds the
FTP server return code in the errno attribute of the exception object
and the error text in strerror.

TemporaryError is raised for FTP return codes from the 4xx category.
This corresponds to ftplib.error_temp (though TemporaryError and
ftplib.error_temp are _not_ identical). PermanentError is raised for
5xx return codes from the FTP server (again, that's similar but _not_
identical to ftplib.error_perm).

ParserError is used for errors during the parsing of directory
listings from the server. This exception is used by the FTPHost
methods stat, lstat, and listdir.

FTPIOError denotes an I/O error on the remote host. This appears
mainly with file-like objects which are retrieved by invoking
FTPHost.file (FTPHost.open is an alias). Compare

    >>> try:
    ...     f = open('notthere')
    ... except IOError, obj:
    ...     print obj.errno
    ...     print obj.strerror
    ...
    2
    No such file or directory

with

    >>> host = ftputil.FTPHost('host', 'user', 'password')
    >>> try:
    ...     f = host.open('notthere')
    ... except IOError, obj:
    ...     print obj.errno
    ...     print obj.strerror
    ...
    550
    550 notthere: No such file or directory.

As you can see, both code snippets are similar. (However, the error
codes aren't the same.)

RootDirError is a special case. Due to the implementation of the
lstat method it is not possible to do a stat on the root directory /.
If you know _any_ way to do it, please let me know. :-)

----------------------------------------------------------------------
FTPHost OBJECTS

Construction
------------

FTPHost instances may be generated with the following call:

    host = ftputil.FTPHost(host, user, password, account,
                           session_factory=ftplib.FTP)

The first four parameters are strings with the same meaning as for the
FTP class in the ftplib module. The keyword argument session_factory
may be used to generate FTP connections with other factories than the
default ftplib.FTP. For example, the M2Crypto distribution uses a
secure FTP class which is derived from ftplib.FTP.

In fact, all positional and keyword arguments other than
session_factory are passed to the factory to generate a new background
session (which happens for every remote file that is opened; see
below).

FTPHost attributes and methods
------------------------------

curdir, pardir, sep
    are strings which denote the current and the parent directory on
    the remote server. sep identifies the path separator. Though RFC
    959 (File Transfer Protocol) notes that these values may be server
    dependent, the Unix counterparts seem to work well in practice,
    even for non-Unix servers.

file(path, mode='r')
    returns a file-like object that is connected to the path on the
    remote host. This path may be absolute or relative to current
    directory on the remote host (this directory can be determined
    with the getcwd method). As with local file objects the default
    mode is 'r', i. e. reading text files. Valid modes are 'r', 'rb',
    'w', and 'wb'.

open(path, mode='r')
    is an alias for file (see above).

copyfileobj(source, target, length=64*1024)
    copies the contents from the file-like object source to the
    file-like object target. The only difference to shutil.copyfileobj
    is the default buffer size.

upload(source, target, mode='')
    copies a local source file (given by a filename, i. e. a string)
    to the remote host under the name target. Both source and target
    may be absolute paths or relative to their corresponding current
    directory (on the local or the remote host, respectively). The
    mode may be '' or 'a' for ASCII uploads or 'b' for binary uploads.
    ASCII mode is the default (again, similar to regular local file
    objects).

download(source, target, mode='')
    performs a download from the remote source to a target file. Both
    source and target are strings. Additionally, the description of
    the upload method applies here, too.

upload_if_newer(source, target, mode='')
    is similar to the upload method. The only difference is that the
    upload is only invoked if the time of the last modification for
    the source file is more recent than that of the target file, or
    the target doesn't exist at all.

download_if_newer(source, target, mode='')
    corresponds to upload_if_newer but performs a download from the
    server to the local host. Read the descriptions of download and
    upload_if_newer for more.

close()
    closes the connection to the remote host. After this, no more
    interaction with the FTP server is possible without using a new
    FTPHost object.

getcwd()
    returns the absolute current directory on the remote host. This
    method acts similar to os.getcwd.

chdir(directory)
    sets the current directory on the FTP server. This resembles
    os.chdir, as you may have expected. :-)

mkdir(path, [mode])
    makes the given directory on the remote host. In the current
    implementation, this doesn't construct "intermediate" directories
    which don't already exist. The mode parameter is ignored. This is
    for compatibilty with os.mkdir if an FTPHost object is passed into
    a function instead of the os module (see the subsection on Python
    exceptions above for an explanation).

rmdir(path)
    removes the given remote directory. Currently, "intermediate"
    directories can't be deleted (as with os.rmdir).

remove(path)
    removes a file on the remote host (similar to os.remove).

unlink(path)
    is an alias for remove.

rename(source, target)
    renames the source file (or directory) on the FTP server.

listdir(path)
    returns a list containing the names of the files and directories
    in the given path; similar to os.listdir.

lstat(path)
    returns an object similar that from os.lstat (a "tuple" with
    additional attributes; see the documentation of the os module for
    details). However, due to the nature of the application, there are
    some important aspects to keep in mind:

    - The result is derived by parsing the output of a DIR command on
      the server. Therefore, the result from FTPHost.lstat can not
      contain nore information than the received text. In particular:

    - User and group ids can only be determined as strings, not as
      numbers, and that only, if the server supplies them. This is
      usually the case with Unix servers but may not be for other FTP
      server programs.

    - Values for the time of the last modification may be rough,
      depending on the information from the server. For timestamps
      older than a year, this usually means that the precision of the
      modification timestamp value is not better than days. For newer
      files, the information may be accurate to a minute.

    - Links can only be recognized on servers that provide this
      information in the DIR output.

    - Items that can't be determined at all, are set to None.

    - There's a special problem with stat'ing the root directory. In
      this case, a RootDirError is raised. This has to do with the
      algorithm used by (l)stat and a know of no one which solves
      this problem.

    Currently, ftputil recognizes the MS Robin FTP server. Otherwise,
    a format commonly used by Unix servers is assumed. If you need to
    parse output from another server type, please contact me under the
    email address at the end of this text.

stat(path)
    return stat information also for files which are pointed to by a
    link. This method follows multiple links until a regular file or
    directory is found. If an infinite link chain is encountered, a
    PermanentError is raised.

FTPHost.path
------------

FTPHost objects contain an attribute named path, similar to os.path.
(See there for documentation.) The following methods can be applied
to the remote host with the same semantics as for os.path:

abspath(path)
basename(path)
commonprefix(path_list)
dirname(path)
exists(path)
getmtime(path)
getsize(path)
isabs(path)
isdir(path)
isfile(path)
islink(path)
join(path1, path2, ...)
normcase(path)
normpath(path)
split(path)
splitdrive(path)
splitext(path)
walk(path, func, arg)

See the section on the os.path module in the Library Reference,
http://www.python.org/doc/current/lib/module-os.path.html

----------------------------------------------------------------------
FTPFile OBJECTS

FTPFile objects as returned by a call to FTPHost.file (or
FTPHost.open) have the following methods - with the same arguments and
semantics as for local files:

close()
read([count])
readline([count])
readlines()
write(data)
writelines(string_sequence)
xreadlines()

and the attribute closed. For details, see the section "File objects"
in the Library Reference,
http://www.python.org/doc/current/lib/bltin-file-objects.html

Note that ftputil supports both binary and text mode with the
appropriate line ending conversions.

----------------------------------------------------------------------
BUGS AND LIMITATIONS

- ftputil needs at least Python 2.1 to work. The only place where
  Python 2.0 is not enough is in FTPHost.listdir. If you need Python
  2.0, and, yet better, also have a patch, contact me under the
  below-mentioned email address.

- Due to the implementation of lstat it can not return a sensible
  value for the root directory /. If you know an implementation that
  can do this, please let me know. The root directory is handled
  appropriately in FTPHost.path.exists/isfile/isdir/islink, though.

- Depending on local time and/or daylight saving time settings
  different local hosts may interpret the modification times in the
  FTP server output differently. If there's a need for it, I would
  provide a method to (manually) set a difference value for the
  local/remote host pair.

- Timeouts of individual child sessions currently are not handled yet.
  This is only a problem if your FTPHost object or the generated
  FTPFile objects are inactive for about ten minutes or longer.

- Until now, I haven't paid attention to thread safety. In principle,
  at least, different FTPFile objects should be usable in different
  threads.

- FTPFile objects in text mode _may not_ support charsets with more
  than one byte per character. Please email me your experiences
  (address below), if you have to work with multibyte text streams in
  FTP sessions.

- The UserTuple class, provided in UserTuple.py, is not thoroughly
  tested. If you encouter problems, please notify me.

----------------------------------------------------------------------
FILES

ftputil.py
    is the main module file.

UserTuple.py
    may be used as the base class for _Stat. If you have at least
    Python version 2.2 you can safely delete the UserTuple module.

ftputil.txt
    is this file.

ftputil_pydoc.txt
    is the development documentation for ftputil, generated with
    Pythons pydoc utility. If you only want to _use_ ftputil, you
    probably don't need this file.

_test_ftputil.py
_mock_ftplib.py
    are needed for the unit tests of ftputil. If you don't want to
    debug or modify ftputil, you can delete these files. _mock_ftplib
    is a "mock" variant of the ftplib module. See below for details.

----------------------------------------------------------------------
SEE ALSO

Mackinnon T, Freeman S, Craig P. 2000.
    Endo-Testing: Unit Testing with Mock Objects
    http://www.mockobjects.com/misc/mockobjects.pdf

Postel J, Reynolds J. 1985.
    RFC 959 - File Transfer Protocol (FTP).
    http://www.ietf.org/rfc/rfc959.txt

Van Rossum G, Drake Jr FL. 2001.
    Python Library Reference.
    http://www.python.org/doc/current/lib/lib.html

----------------------------------------------------------------------
AUTHOR

ftputil is written by Stefan Schwarzer <s.schwarzer@ndh.net>.
Feedback is appreciated.