python-peps / pep-3151.txt

The default branch has multiple heads

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
PEP: 3151
Title: Reworking the OS and IO exception hierarchy
Version: $Revision$
Last-Modified: $Date$
Author: Antoine Pitrou <solipsis@pitrou.net>
BDFL-Delegate: Barry Warsaw
Status: Final
Type: Standards Track
Content-Type: text/x-rst
Created: 2010-07-21
Python-Version: 3.3
Post-History:
Resolution: http://mail.python.org/pipermail/python-dev/2011-October/114033.html

Abstract
========

The standard exception hierarchy is an important part of the Python
language.  It has two defining qualities: it is both generic and
selective.  Generic in that the same exception type can be raised
- and handled - regardless of the context (for example, whether you are
trying to add something to an integer, to call a string method, or to write
an object on a socket, a TypeError will be raised for bad argument types).
Selective in that it allows the user to easily handle (silence, examine,
process, store or encapsulate...) specific kinds of error conditions
while letting other errors bubble up to higher calling contexts.  For
example, you can choose to catch ZeroDivisionErrors without affecting
the default handling of other ArithmeticErrors (such as OverflowErrors).

This PEP proposes changes to a part of the exception hierarchy in
order to better embody the qualities mentioned above: the errors
related to operating system calls (OSError, IOError, mmap.error,
select.error, and all their subclasses).


Rationale
=========

Confusing set of OS-related exceptions
--------------------------------------

OS-related (or system call-related) exceptions are currently a diversity
of classes, arranged in the following sub-hierarchies::

    +-- EnvironmentError
        +-- IOError
            +-- io.BlockingIOError
            +-- io.UnsupportedOperation (also inherits from ValueError)
            +-- socket.error
                +-- socket.gaierror
                +-- socket.herror
                +-- socket.timeout
        +-- OSError
            +-- VMSError
            +-- WindowsError
        +-- mmap.error
    +-- select.error

While some of these distinctions can be explained by implementation
considerations, they are often not very logical at a higher level.  The
line separating OSError and IOError, for example, is often blurry.  Consider
the following::

    >>> os.remove("fff")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    OSError: [Errno 2] No such file or directory: 'fff'
    >>> open("fff")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 2] No such file or directory: 'fff'

The same error condition (a non-existing file) gets cast as two different
exceptions depending on which library function was called.  The reason
for this is that the ``os`` module exclusively raises OSError (or its
subclass WindowsError) while the ``io`` module mostly raises IOError.
However, the user is interested in the nature of the error, not in which
part of the interpreter it comes from (since the latter is obvious from
reading the traceback message or application source code).

In fact, it is hard to think of any situation where OSError should be
caught but not IOError, or the reverse.

A further proof of the ambiguity of this segmentation is that the standard
library itself sometimes has problems deciding.  For example, in the
``select`` module, similar failures will raise ``select.error``, ``OSError``
or ``IOError`` depending on whether you are using select(), a poll object,
a kqueue object, or an epoll object.  This makes user code uselessly
complicated since it has to be prepared to catch various exception types,
depending on which exact implementation of a single primitive it chooses
to use at runtime.

As for WindowsError, it seems to be a pointless distinction.  First, it
only exists on Windows systems, which requires tedious compatibility code
in cross-platform applications (such code can be found in ``Lib/shutil.py``).
Second, it inherits from OSError and is raised for similar errors as OSError
is raised for on other systems. Third, the user wanting access to low-level
exception specifics has to examine the ``errno`` or ``winerror`` attribute
anyway.

.. note::
    `Appendix B`_ surveys the use of the various exception types across
    the interpreter and the standard library.


Lack of fine-grained exceptions
-------------------------------

The current variety of OS-related exceptions doesn't allow the user to filter
easily for the desired kinds of failures.  As an example, consider the task
of deleting a file if it exists.  The Look Before You Leap (LBYL) idiom
suffers from an obvious race condition::

    if os.path.exists(filename):
        os.remove(filename)

If a file named as ``filename`` is created by another thread or process
between the calls to ``os.path.exists`` and ``os.remove``, it won't be
deleted.  This can produce bugs in the application, or even security issues.

Therefore, the solution is to try to remove the file, and ignore the error
if the file doesn't exist (an idiom known as Easier to Ask Forgiveness
than to get Permission, or EAFP).  Careful code will read like the following
(which works under both POSIX and Windows systems)::

    try:
        os.remove(filename)
    except OSError as e:
        if e.errno != errno.ENOENT:
            raise

or even::

    try:
        os.remove(filename)
    except EnvironmentError as e:
        if e.errno != errno.ENOENT:
            raise

This is a lot more to type, and also forces the user to remember the various
cryptic mnemonics from the ``errno`` module.  It imposes an additional
cognitive burden and gets tiresome rather quickly.  Consequently, many
programmers will instead write the following code, which silences exceptions
too broadly::

    try:
        os.remove(filename)
    except OSError:
        pass

``os.remove`` can raise an OSError not only when the file doesn't exist,
but in other possible situations (for example, the filename points to a
directory, or the current process doesn't have permission to remove
the file), which all indicate bugs in the application logic and therefore
shouldn't be silenced.  What the programmer would like to write instead is
something such as::

    try:
        os.remove(filename)
    except FileNotFoundError:
        pass


Compatibility strategy
======================

Reworking the exception hierarchy will obviously change the exact semantics
of at least some existing code.  While it is not possible to improve on the
current situation without changing exact semantics, it is possible to define
a narrower type of compatibility, which we will call *useful compatibility*.

For this we first must explain what we will call *careful* and *careless*
exception handling.  *Careless* (or "naïve") code is defined as code which
blindly catches any of ``OSError``, ``IOError``, ``socket.error``,
``mmap.error``, ``WindowsError``, ``select.error`` without checking the ``errno``
attribute.  This is because such exception types are much too broad to signify
anything.  Any of them can be raised for error conditions as diverse as: a
bad file descriptor (which will usually indicate a programming error), an
unconnected socket (ditto), a socket timeout, a file type mismatch, an invalid
argument, a transmission failure, insufficient permissions, a non-existent
directory, a full filesystem, etc.

(moreover, the use of certain of these exceptions is irregular; `Appendix B`_
exposes the case of the `select`_ module, which raises different exceptions
depending on the implementation)

*Careful* code is defined as code which, when catching any of the above
exceptions, examines the ``errno`` attribute to determine the actual error
condition and takes action depending on it.

Then we can define *useful compatibility* as follows:

* useful compatibility doesn't make exception catching any narrower, but
  it can be broader for *careless* exception-catching code.  Given the following
  kind of snippet, all exceptions caught before this PEP will also be
  caught after this PEP, but the reverse may be false (because the coalescing
  of ``OSError``, ``IOError`` and others means the ``except`` clause throws
  a slightly broader net)::
  
      try:
          ...
          os.remove(filename)
          ...
      except OSError:
          pass

* useful compatibility doesn't alter the behaviour of *careful*
  exception-catching code.  Given the following kind of snippet, the same
  errors should be silenced or re-raised, regardless of whether this PEP
  has been implemented or not::

      try:
          os.remove(filename)
      except OSError as e:
          if e.errno != errno.ENOENT:
              raise

The rationale for this compromise is that careless code can't really be
helped, but at least code which "works" won't suddenly raise errors and
crash.  This is important since such code is likely to be present in
scripts used as cron tasks or automated system administration programs.

Careful code, on the other hand, should not be penalized.  Actually, one
purpose of this PEP is to ease writing careful code.


.. _Step 1:

Step 1: coalesce exception types
================================

The first step of the resolution is to coalesce existing exception types.
The following changes are proposed:

* alias both socket.error and select.error to OSError
* alias mmap.error to OSError
* alias both WindowsError and VMSError to OSError
* alias IOError to OSError
* coalesce EnvironmentError into OSError

Each of these changes doesn't preserve exact compatibility, but it does
preserve *useful compatibility* (see "compatibility" section above).

Each of these changes can be accepted or refused individually, but of course
it is considered that the greatest impact can be achieved if this first step
is accepted in full.  In this case, the IO exception sub-hierarchy would
become::

    +-- OSError   (replacing IOError, WindowsError, EnvironmentError, etc.)
        +-- io.BlockingIOError
        +-- io.UnsupportedOperation (also inherits from ValueError)
        +-- socket.gaierror
        +-- socket.herror
        +-- socket.timeout

Justification
-------------

Not only does this first step present the user a simpler landscape as
explained in the rationale_ section, but it also allows for a better
and more complete resolution of `Step 2`_ (see Prerequisite_).

The rationale for keeping ``OSError`` as the official name for generic
OS-related exceptions is that it, precisely, is more generic than ``IOError``.
``EnvironmentError`` is more tedious to type and also much lesser-known.

The survey in `Appendix B`_ shows that IOError is the dominant
error today in the standard library.  As for third-party Python code,
Google Code Search shows IOError being ten times more popular than
EnvironmentError in user code, and three times more popular than OSError
[3]_.  However, with no intention to deprecate IOError in the middle
term, the lesser popularity of OSError is not a problem.

Exception attributes
--------------------

Since WindowsError is coalesced into OSError, the latter gains a ``winerror``
attribute under Windows.  It is set to None under situations where it is not
meaningful, as is already the case with the ``errno``, ``filename`` and
``strerror`` attributes (for example when OSError is raised directly by
Python code).

Deprecation of names
--------------------

The following paragraphs outline a possible deprecation strategy for
old exception names.  However, it has been decided to keep them as aliases
for the time being.  This decision could be revised in time for Python 4.0.

built-in exceptions
'''''''''''''''''''

Deprecating the old built-in exceptions cannot be done in a straightforward
fashion by intercepting all lookups in the builtins namespace, since these
are performance-critical.  We also cannot work at the object level, since
the deprecated names will be aliased to non-deprecated objects.

A solution is to recognize these names at compilation time, and
then emit a separate ``LOAD_OLD_GLOBAL`` opcode instead of the regular
``LOAD_GLOBAL``.  This specialized opcode will handle the output of a
DeprecationWarning (or PendingDeprecationWarning, depending on the policy
decided upon) when the name doesn't exist in the globals namespace, but
only in the builtins one.  This will be enough to avoid false positives
(for example if someone defines their own ``OSError`` in a module), and
false negatives will be rare (for example when someone accesses ``OSError``
through the ``builtins`` module rather than directly).

module-level exceptions
'''''''''''''''''''''''

The above approach cannot be used easily, since it would require
special-casing some modules when compiling code objects.  However, these
names are by construction much less visible (they don't appear in the
builtins namespace), and lesser-known too, so we might decide to let them
live in their own namespaces.


.. _Step 2:

Step 2: define additional subclasses
====================================

The second step of the resolution is to extend the hierarchy by defining
subclasses which will be raised, rather than their parent, for specific
errno values.  Which errno values is subject to discussion, but a survey
of existing exception matching practices (see `Appendix A`_) helps us
propose a reasonable subset of all values.  Trying to map all errno
mnemonics, indeed, seems foolish, pointless, and would pollute the root
namespace.

Furthermore, in a couple of cases, different errno values could raise
the same exception subclass.  For example, EAGAIN, EALREADY, EWOULDBLOCK
and EINPROGRESS are all used to signal that an operation on a non-blocking
socket would block (and therefore needs trying again later).  They could
therefore all raise an identical subclass and let the user examine the
``errno`` attribute if (s)he so desires (see below "exception
attributes").

Prerequisite
------------

`Step 1`_ is a loose prerequisite for this.

Prerequisite, because some errnos can currently be attached to different
exception classes: for example, ENOENT can be attached to both OSError and
IOError, depending on the context.  If we don't want to break *useful
compatibility*, we can't make an ``except OSError`` (or IOError) fail to
match an exception where it would succeed today.

Loose, because we could decide for a partial resolution of step 2
if existing exception classes are not coalesced: for example, ENOENT could
raise a hypothetical FileNotFoundError where an IOError was previously
raised, but continue to raise OSError otherwise.

The dependency on step 1 could be totally removed if the new subclasses
used multiple inheritance to match with all of the existing superclasses
(or, at least, OSError and IOError, which are arguable the most prevalent
ones).  It would, however, make the hierarchy more complicated and
therefore harder to grasp for the user.

New exception classes
---------------------

The following tentative list of subclasses, along with a description and
the list of errnos mapped to them, is submitted to discussion:

* ``FileExistsError``: trying to create a file or directory which already
  exists (EEXIST)

* ``FileNotFoundError``: for all circumstances where a file and directory is
  requested but doesn't exist (ENOENT)

* ``IsADirectoryError``: file-level operation (open(), os.remove()...)
  requested on a directory (EISDIR)

* ``NotADirectoryError``: directory-level operation requested on something
  else (ENOTDIR)

* ``PermissionError``: trying to run an operation without the adequate access
  rights - for example filesystem permissions (EACCES, EPERM)

* ``BlockingIOError``: an operation would block on an object (e.g. socket) set
  for non-blocking operation (EAGAIN, EALREADY, EWOULDBLOCK, EINPROGRESS);
  this is the existing ``io.BlockingIOError`` with an extended role

* ``BrokenPipeError``: trying to write on a pipe while the other end has been
  closed, or trying to write on a socket which has been shutdown for writing
  (EPIPE, ESHUTDOWN)

* ``InterruptedError``: a system call was interrupted by an incoming signal
  (EINTR)

* ``ConnectionAbortedError``: connection attempt aborted by peer (ECONNABORTED)

* ``ConnectionRefusedError``: connection reset by peer (ECONNREFUSED)

* ``ConnectionResetError``: connection reset by peer (ECONNRESET)

* ``TimeoutError``: connection timed out (ETIMEDOUT); this can be re-cast
  as a generic timeout exception, replacing ``socket.timeout`` and also useful
  for other types of timeout (for example in Lock.acquire())

* ``ChildProcessError``: operation on a child process failed (ECHILD);
  this is raised mainly by the wait() family of functions.

* ``ProcessLookupError``: the given process (as identified by, e.g., its
  process id) doesn't exist (ESRCH).

In addition, the following exception class is proposed for inclusion:

* ``ConnectionError``: a base class for ``ConnectionAbortedError``,
  ``ConnectionRefusedError`` and ``ConnectionResetError``

The following drawing tries to sum up the proposed additions, along with
the corresponding errno values (where applicable).  The root of the
sub-hierarchy (OSError, assuming `Step 1`_ is accepted in full) is not
shown::

    +-- BlockingIOError        EAGAIN, EALREADY, EWOULDBLOCK, EINPROGRESS
    +-- ChildProcessError                                          ECHILD
    +-- ConnectionError
        +-- BrokenPipeError                              EPIPE, ESHUTDOWN
        +-- ConnectionAbortedError                           ECONNABORTED
        +-- ConnectionRefusedError                           ECONNREFUSED
        +-- ConnectionResetError                               ECONNRESET
    +-- FileExistsError                                            EEXIST
    +-- FileNotFoundError                                          ENOENT
    +-- InterruptedError                                            EINTR
    +-- IsADirectoryError                                          EISDIR
    +-- NotADirectoryError                                        ENOTDIR
    +-- PermissionError                                     EACCES, EPERM
    +-- ProcessLookupError                                          ESRCH
    +-- TimeoutError                                            ETIMEDOUT

Naming
------

Various naming controversies can arise.  One of them is whether all
exception class names should end in "``Error``".  In favour is consistency
with the rest of the exception hiearchy, against is concision (especially
with long names such as ``ConnectionAbortedError``).

Exception attributes
--------------------

In order to preserve *useful compatibility*, these subclasses should still
set adequate values for the various exception attributes defined on the
superclass (for example ``errno``, ``filename``, and optionally
``winerror``).

Implementation
--------------

Since it is proposed that the subclasses are raised based purely on the
value of ``errno``, little or no changes should be required in extension
modules (either standard or third-party).

The first possibility is to adapt the ``PyErr_SetFromErrno()`` family
of functions (``PyErr_SetFromWindowsErr()`` under Windows) to raise the
appropriate OSError subclass.  This wouldn't cover, however, Python
code raising OSError directly, using the following idiom (seen in
``Lib/tempfile.py``)::

    raise IOError(_errno.EEXIST, "No usable temporary file name found")

A second possibility, suggested by Marc-Andre Lemburg, is to adapt
``OSError.__new__`` to instantiate the appropriate subclass.  This has
the benefit of also covering Python code such as the above.


Possible objections
===================

Namespace pollution
-------------------

Making the exception hierarchy finer-grained makes the root (or builtins)
namespace larger.  This is to be moderated, however, as:

* only a handful of additional classes are proposed; 

* while standard exception types live in the root namespace, they are
  visually distinguished by the fact that they use the CamelCase convention,
  while almost all other builtins use lowercase naming (except True, False,
  None, Ellipsis and NotImplemented)

An alternative would be to provide a separate module containing the
finer-grained exceptions, but that would defeat the purpose of
encouraging careful code over careless code, since the user would first
have to import the new module instead of using names already accessible.


Earlier discussion
==================

While this is the first time such as formal proposal is made, the idea
has received informal support in the past [1]_; both the introduction
of finer-grained exception classes and the coalescing of OSError and
IOError.

The removal of WindowsError alone has been discussed and rejected
as part of another PEP [2]_, but there seemed to be a consensus that the
distinction with OSError wasn't meaningful.  This supports at least its
aliasing with OSError.


Implementation
==============

The reference implementation has been integrated into Python 3.3.
It was formerly developed in http://hg.python.org/features/pep-3151/ in
branch ``pep-3151``, and also tracked on the bug tracker at
http://bugs.python.org/issue12555.
It has been successfully tested on a variety of systems: Linux, Windows,
OpenIndiana and FreeBSD buildbots.

One source of trouble has been with the respective constructors of ``OSError``
and ``WindowsError``, which were incompatible.  The way it is solved is by
keeping the ``OSError`` signature and adding a fourth optional argument
to allow passing the Windows error code (which is different from the POSIX
errno).  The fourth argument is stored as ``winerror`` and its POSIX
translation as ``errno``.  The ``PyErr_SetFromWindowsErr*`` functions have
been adapted to use the right constructor call.

A slight complication is when the ``PyErr_SetExcFromWindowsErr*`` functions
are called with ``OSError`` rather than ``WindowsError``: the ``errno``
attribute of the exception object would store the Windows error code (such
as 109 for ERROR_BROKEN_PIPE) rather than its POSIX translation (such as 32
for EPIPE), which it does now.  For non-socket error codes, this only occurs
in the private ``_multiprocessing`` module for which there is no compatibility
concern.

.. note::
   For socket errors, the "POSIX errno" as reflected by the ``errno`` module
   is numerically equal to the `Windows Socket error code
   <http://msdn.microsoft.com/en-us/library/ms740668%28v=vs.85%29.aspx>`_
   returned by the ``WSAGetLastError`` system call::

    >>> errno.EWOULDBLOCK
    10035
    >>> errno.WSAEWOULDBLOCK
    10035


Possible alternative
====================

Pattern matching
----------------

Another possibility would be to introduce an advanced pattern matching
syntax when catching exceptions.  For example::

    try:
        os.remove(filename)
    except OSError as e if e.errno == errno.ENOENT:
        pass

Several problems with this proposal:

* it introduces new syntax, which is perceived by the author to be a heavier
  change compared to reworking the exception hierarchy
* it doesn't decrease typing effort significantly
* it doesn't relieve the programmer from the burden of having to remember
  errno mnemonics


Exceptions ignored by this PEP
==============================

This PEP ignores ``EOFError``, which signals a truncated input stream in
various protocol and file format implementations (for example ``GzipFile``).
``EOFError`` is not OS- or IO-related, it is a logical error raised at
a higher level.

This PEP also ignores ``SSLError``, which is raised by the ``ssl`` module
in order to propagate errors signalled by the ``OpenSSL`` library.  Ideally,
``SSLError`` would benefit from a similar but separate treatment since it
defines its own constants for error types (``ssl.SSL_ERROR_WANT_READ``,
etc.).  In Python 3.2, ``SSLError`` is already replaced with ``socket.timeout``
when it signals a socket timeout (see `issue 10272 <http://bugs.python.org/issue10272>`_).

Endly, the fate of ``socket.gaierror`` and ``socket.herror`` is not settled.
While they would deserve less cryptic names, this can be handled separately
from the exception hierarchy reorganization effort.


.. _Appendix A:

Appendix A: Survey of common errnos
===================================

This is a quick inventory of the various errno mnemonics checked for in
the standard library and its tests, as part of ``except`` clauses.

Common errnos with OSError
--------------------------

* ``EBADF``: bad file descriptor (usually means the file descriptor was
  closed)

* ``EEXIST``: file or directory exists

* ``EINTR``: interrupted function call

* ``EISDIR``: is a directory

* ``ENOTDIR``: not a directory

* ``ENOENT``: no such file or directory

* ``EOPNOTSUPP``: operation not supported on socket
  (possible confusion with the existing io.UnsupportedOperation)

* ``EPERM``: operation not permitted (when using e.g. os.setuid())

Common errnos with IOError
--------------------------

* ``EACCES``: permission denied (for filesystem operations)

* ``EBADF``: bad file descriptor (with select.epoll); read operation on a
  write-only GzipFile, or vice-versa

* ``EBUSY``: device or resource busy

* ``EISDIR``: is a directory (when trying to open())

* ``ENODEV``: no such device

* ``ENOENT``: no such file or directory (when trying to open())

* ``ETIMEDOUT``: connection timed out

Common errnos with socket.error
-------------------------------

All these errors may also be associated with a plain IOError, for example
when calling read() on a socket's file descriptor.

* ``EAGAIN``: resource temporarily unavailable (during a non-blocking socket
  call except connect())

* ``EALREADY``: connection already in progress (during a non-blocking
  connect())

* ``EINPROGRESS``: operation in progress (during a non-blocking connect())

* ``EINTR``: interrupted function call

* ``EISCONN``: the socket is connected

* ``ECONNABORTED``: connection aborted by peer (during an accept() call)

* ``ECONNREFUSED``: connection refused by peer

* ``ECONNRESET``: connection reset by peer

* ``ENOTCONN``: socket not connected

* ``ESHUTDOWN``: cannot send after transport endpoint shutdown

* ``EWOULDBLOCK``: same reasons as ``EAGAIN``

Common errnos with select.error
-------------------------------

* ``EINTR``: interrupted function call


.. _Appendix B:

Appendix B: Survey of raised OS and IO errors
=============================================

About VMSError
--------------

VMSError is completely unused by the interpreter core and the standard
library.  It was added as part of the OpenVMS patches submitted in 2002
by Jean-François Piéronne [4]_; the motivation for including VMSError was that
it could be raised by third-party packages.

Interpreter core
----------------

Handling of PYTHONSTARTUP raises IOError (but the error gets discarded)::

    $ PYTHONSTARTUP=foox ./python
    Python 3.2a0 (py3k:82920M, Jul 16 2010, 22:53:23) 
    [GCC 4.4.3] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    Could not open PYTHONSTARTUP
    IOError: [Errno 2] No such file or directory: 'foox'

``PyObject_Print()`` raises IOError when ferror() signals an error on the
`FILE *` parameter (which, in the source tree, is always either stdout or
stderr).

Unicode encoding and decoding using the ``mbcs`` encoding can raise
WindowsError for some error conditions.

Standard library
----------------

bz2
'''

Raises IOError throughout (OSError is unused)::

    >>> bz2.BZ2File("foox", "rb")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 2] No such file or directory
    >>> bz2.BZ2File("LICENSE", "rb").read()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: invalid data stream
    >>> bz2.BZ2File("/tmp/zzz.bz2", "wb").read()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: file is not ready for reading

curses
''''''

Not examined.

dbm.gnu, dbm.ndbm
'''''''''''''''''

_dbm.error and _gdbm.error inherit from IOError::

    >>> dbm.gnu.open("foox")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    _gdbm.error: [Errno 2] No such file or directory

fcntl
'''''

Raises IOError throughout (OSError is unused).

imp module
''''''''''

Raises IOError for bad file descriptors::

    >>> imp.load_source("foo", "foo", 123)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 9] Bad file descriptor

io module
'''''''''

Raises IOError when trying to open a directory under Unix::

    >>> open("Python/", "r")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 21] Is a directory: 'Python/'

Raises IOError or io.UnsupportedOperation (which inherits from the former)
for unsupported operations::

    >>> open("LICENSE").write("bar")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: not writable
    >>> io.StringIO().fileno()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    io.UnsupportedOperation: fileno
    >>> open("LICENSE").seek(1, 1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: can't do nonzero cur-relative seeks

Raises either IOError or TypeError when the inferior I/O layer misbehaves
(i.e. violates the API it is expected to implement).

Raises IOError when the underlying OS resource becomes invalid::

    >>> f = open("LICENSE")
    >>> os.close(f.fileno())
    >>> f.read()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 9] Bad file descriptor

...or for implementation-specific optimizations::

    >>> f = open("LICENSE")
    >>> next(f)
    'A. HISTORY OF THE SOFTWARE\n'
    >>> f.tell()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: telling position disabled by next() call

Raises BlockingIOError (inheriting from IOError) when a call on a non-blocking
object would block.

mmap
''''

Under Unix, raises its own ``mmap.error`` (inheriting from EnvironmentError)
throughout::

    >>> mmap.mmap(123, 10)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    mmap.error: [Errno 9] Bad file descriptor
    >>> mmap.mmap(os.open("/tmp", os.O_RDONLY), 10)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    mmap.error: [Errno 13] Permission denied

Under Windows, however, it mostly raises WindowsError (the source code
also shows a few occurrences of ``mmap.error``)::

    >>> fd = os.open("LICENSE", os.O_RDONLY)
    >>> m = mmap.mmap(fd, 16384)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    WindowsError: [Error 5] Accès refusé
    >>> sys.last_value.errno
    13
    >>> errno.errorcode[13]
    'EACCES'

    >>> m = mmap.mmap(-1, 4096)
    >>> m.resize(16384)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    WindowsError: [Error 87] Paramètre incorrect
    >>> sys.last_value.errno
    22
    >>> errno.errorcode[22]
    'EINVAL'

multiprocessing
'''''''''''''''

Not examined.

os / posix
''''''''''

The ``os`` (or ``posix``) module raises OSError throughout, except under
Windows where WindowsError can be raised instead.

ossaudiodev
'''''''''''

Raises IOError throughout (OSError is unused)::

    >>> ossaudiodev.open("foo", "r")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 2] No such file or directory: 'foo'

readline
''''''''

Raises IOError in various file-handling functions::

    >>> readline.read_history_file("foo")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 2] No such file or directory
    >>> readline.read_init_file("foo")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 2] No such file or directory
    >>> readline.write_history_file("/dev/nonexistent")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    IOError: [Errno 13] Permission denied

select
''''''

* select() and poll objects raise ``select.error``, which doesn't inherit from
  anything (but poll.modify() raises IOError);
* epoll objects raise IOError;
* kqueue objects raise both OSError and IOError.

As a side-note, not deriving from ``EnvironmentError`` means ``select.error``
does not get the useful ``errno`` attribute.  User code must check ``args[0]``
instead::

    >>> signal.alarm(1); select.select([], [], [])
    0
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    select.error: (4, 'Interrupted system call')
    >>> e = sys.last_value
    >>> e
    error(4, 'Interrupted system call')
    >>> e.errno == errno.EINTR
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'error' object has no attribute 'errno'
    >>> e.args[0] == errno.EINTR
    True

signal
''''''

``signal.ItimerError`` inherits from IOError.

socket
''''''

``socket.error`` inherits from IOError.

sys
'''

``sys.getwindowsversion()`` raises WindowsError with a bogus error number
if the ``GetVersionEx()`` call fails.

time
''''

Raises IOError for internal errors in time.time() and time.sleep().

zipimport
'''''''''

zipimporter.get_data() can raise IOError.


Acknowledgments
===============

Significant input has been received from Nick Coghlan.

References
==========

.. [1] "IO module precisions and exception hierarchy":
   http://mail.python.org/pipermail/python-dev/2009-September/092130.html

.. [2] Discussion of "Removing WindowsError" in PEP 348:
   http://www.python.org/dev/peps/pep-0348/#removing-windowserror

.. [3] Google Code Search of ``IOError`` in Python code: `around 40000 results
   <http://www.google.com/codesearch?q=lang%3Apython%20IOError>`_;
   ``OSError``: `around 15200 results
   <http://www.google.com/codesearch?q=lang%3Apython%20OSError>`_;
   ``EnvironmentError``: `around 3000 results
   <http://www.google.com/codesearch?q=lang%3Apython%20EnvironmentError>`_

.. [4] http://bugs.python.org/issue614055

Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.