Exceptions in Python aren’t new, and pretty easy to wrap your head around. Which means you should have no trouble following what this example will output.

import warnings

def bad_func():
    try:
        assert False and True or False
    except AssertionError as outer_err:
        try:
            assert None is False
        except AssertionError as inner_err:
            warnings.warn("None is not False", UserWarning)
            raise inner_err from None
        else:
            return None
    else:
        return True
    finally:
        return False


print(bad_func())

Simple, right?  It raises both except clauses, triggering the warning and so your screen will obviously print False and the  warning.

False
C:/python_file.py:15: UserWarning: None is not False
  warnings.warn("None is not False", UserWarning)

So if you followed that without issue and understand why its happening, go ahead and check out another post, because today we are diving into the depths of Python 3 exception handling.

The basic try except block

If you have coded any semi-resilient Python code, you have used at least the basic try except.

try:
    something()
except:
    # Do not use a `except` without an exception type, more on that later
    pass

If something breaks in the try block, the code in the except block is execute instead of allowing the exception to be raised. However the example code above does something unforgivable, it captures everything. “But that’s what I want it to do!” No, you do not.

Catching everything includes both KeyboardInterrupt and SystemExit. Those Exceptions are outside the normal Exception scope. They are special, because they are caused from external sources. Everything else is caused by something in your code.

That means if you have the following code, it would only exit if it had a SIGKILL sent to it, instead of the regular, and more polite SIGTERM or a user pressing Ctrl+C .

while True:
    try:
        something()
    except:
        pass

It is recommend to catch KeyboardInterrupt and SystemExit expictily and only as needed. A common use case is to do a “soft” exit, where your program tries to finish up any working tasks before exiting. However, in any case you use it, it’s better to be explicit. There should almost always be a separate path for internal vs external errors.

try:
    something()
except Exception as err:
    print(err)
    raise  # A blank except rasies the last exception
except (SystemError, KeyboardInterrupt) as err:
    print('Shutting down safely, please wait until process exits')
    soft_exit()

This way it is clear to the user when either an internal error happened or if it was a user generated event. This is usually only used on the very external event loop or function of the program. For most of your code, the bare minimum you should use in a try except block is:

try:
    something()
except Exception as err:
    print(err)

That way you are capturing all internal exceptions, but none of the system level ones. Also very important, adding a message (in this simple example a print line) will help show what the error was. However, str(err) will only output the error message, not the error type or traceback. We will cover how to get the most of the exception next.

Error messages, custom exceptions and scope

So why does Exception catch everything but the external errors?

Scope

The Exception class is inherited by all the other internal errors. Whereas something like FileNotFoundError is at the lowest point on the totem pole of built-in exception hierarchy. That means it’s as specific as possible, and if you catch that error, anything else will still be raised. 

| Exception
+--| OSError
   +-- FileNotFoundError
   +-- PermissionError

So the further –> you go, the less encapsulation and more specific of error you get.

try:
    raise FileNotFoundError("These aren't the driods you're looking for")
except FileNotFoundError:
    print("It's locked, let's try the next one")
except OSError:
    print("We've got em")

# Output: It's locked, let's try the next one

It works as expected, the exception is caught, its message is not printed, rather the code in the except FileNotFoundError is executed. Now keep in mind that other exceptions at the same level will not catch errors on the same level.

try:
    raise FileNotFoundError("These aren't the driods you're looking for")
except PermissionError:
    print("It's locked, let's try the next one")
except OSError:
    print("We've got em")

# Output: We've got em

Only that specific error or its parent (or parent’s parent, etc..) will work to capture it. Also keep in mind, with try except blocks, order of operations matter, it won’t sort through all the possible exceptions, it will execute only the first matching block.

try:
    raise FileNotFoundError("These aren't the driods you're looking for")
except OSError:
    print("We've got em")
except FileNotFoundError as err:
    print(err)

# Output: We've got em

That means even though FileNotFoundError was a more precise match, it won’t be executed if an except block above it also matches.

In practice, it’s best to capture the known possible lowest level (highest precision) exceptions. That is because if you have a known common failure, there is generally a different path you want to take instead of complete failure. We will use that in this example, where we want to create a file, but don’t want to overwrite existing ones.

index = 0 
while True:
    index += 1
    try:
        create_a_file(f'file_{index}')
        break
    except FileExistsError:
        continue
    except Exception as err:
        print(f"Unexpected error occurred while creating a file: {err}")

Here you can see, if that file already exists, and create_a_file raises a FileExistsError the loop will increase the index, and try again for file_2 and that will continue until a file doesn’t exist and it break out of the loop.

Custom Exceptions

What if you have code that needs to raise an exception under certain conditions? Well, it’s good to use the built-ins only when exactly applicable,  otherwise it is better to create your own. And you should always start with creating that error off of Exception.

class CustomException(Exception):
    """Something went wrong in my program"""

raise CustomException("It's dead Jim")

All user-defined exceptions should also be derived from [Exception] -Python Standard Library

Don’t be fooled into using the more appropriate sounding BaseException, that is the parent of all exceptions, including SystemExit.

You can of course create your own hierarchical system of exceptions, which will behave the same way as the built-in.

class CustomException(Exception):
    """Something went wrong in my program"""

class FileRelatedError(CustomException):
    """Couldn't access or update a file"""

class IORelatedError(CustomException):
    """Couldn't read from it"""


try:
    raise FileRelatedError("That file should NOT be there")
except CustomException as err:
    print(err)

# Output: That file should NOT be there

Traceback and Messages

So far we have been printing messages in the except blocks. Which is fine for small scripts, and I highly recommended including instead of just pass (Unless that is a planned safe route). However, if there is an exception, you probably want to know a bit more. That’s where the traceback comes in, it lets you know some of the most recent calls leading to the exception.

import traceback
import reusables

try:
    reusables.extract('not_a_real_file')
except Exception as err:
    traceback.print_exc()

Using the built-in traceback module, we can use traceback.print_exc() to directly print the traceback to the screen. (Use my_var = traceback.format_exc() if you plan to capture it to a variable instead of printing it.)

Traceback (most recent call last):
  File "C:\python_file.py", line 11, in <module>
    reusables.extract('not_a_real_file')
  File "C:\reusables.py", line 594, in extract
    raise OSError("File does not exist or has zero size")
OSError: File does not exist or has zero size

This becomes a lifesaver when trying to figure out what went wrong where. Here we can see what line of our code executed to cause the exception reusables.extract and where in that library the exception was raised. That way, if we need to dig deeper, or it’s a more complex issue, everything is nicely laid out for us.

“So what, it prints to my screen when it errors out and exits my program.” True, but what if you don’t want your entire program to fail over a trivial section of code, and want to be able to go back and figure out what erred. An even better example of this is with logging.

The standard library logging module logger has a beautiful feature built-in, .exception. Where it will log the full traceback of the last exception.

import logging
log = logging.getLogger(__name__)

try:
    assert False
except Exception as err:
    log.exception("Could not assert to True")

This will log the usual message you provide at the ERROR level, but will then include the full traceback of the last exception. Extremely valuable for services or background programs.

2017-01-20 22:28:34,784 - __main__      ERROR    Could not assert to True
Traceback (most recent call last):
  File "C:/python_file.py", line 12, in <module>
    assert False
AssertionError

Traceback and logging both have a lot of wonderful abilities that I won’t go into detail with here, but make sure to take the time to look into them.

Finally

try except blocks actually have two additional optional clauses, else and finally sections. Finally is very straightforward, no matter what happens in the rest of try except, that code WILL execute. It supersedes all other returns, it will suppress other block’s raises if it has its own raise or return.  (The only case when it won’t run is because the program is terminated during its execution.)

That’s why in the very top example code block, even though there is a raise being called, the False is still returned.

    finally:
        return False

The Finally section is invaluable when you are dealing with stuff that is critical to handle for clean up.  For example, safely closing database connections.

try:
    something()
except CustomExpectedError:
    relaunch_program()
except Exception:
    log.exception("Something broke")
except (SystemError, KeyboardInterrupt):
    wait_for_db_transactions()
finally:
    stop_db_transactions()
    db_commit()
    db_close()

Now that is a rather beautiful try except, it catches lowest case first, will log unexpected errors, has a soft_exit if the user presses Ctrl+C, and will safely close the database connection not matter what, even if Ctrl+C is pressed again while wait_for_db_transactions() is being run. (Notice this isn’t a perfect example, as it will still wait to close the database connection after relauch_program is ended, which means that function should do something like fork off so the finally block isn’t wanting  forever to run.)

What Else?

The final possible section of the try except is else, which is the optional success path. Meaning that code will only run if no exceptions are raised. (Remember that code in the finally section will still run as well if present.)

try:
    a, b, *c = "123,456,789,10,11,12".split(",")
except ValueError:
    a, b, c = '1', '2', '3'
else:
    print(",".join(num for num in a))

# Output: 1,2,3

This helps mainly with flow control in the program. For example, if you have a function that tries multiple ways to interpret a string, you could add an else clause to each of the try except blocks that returns on success.  (Dear goodness don’t actually use this code it’s just an example for flow.)

def what_is_it(incoming):
    try:
        float(incoming)
    except ValueError:
        pass  # This is fine as it's a precise, expected exception
    else:
        return 'number'

    try:
        len(str(incoming).split(",")) > 1
    except Exception:
        traceback.print_exc() # Unsure of possible errors, find out
    else:
        return 'list'

From What?

As you might have noticed in the initial example, there  is a line:

raise inner_err from None

This is new in Python 3 and from my experience, hardly ever used, even though very useful. It sets the context of the exception. Ever seen a page log exception with plenty of During handling of the above exception, another exception occurred: 

This can set it to an exact exception above it, so it only displays the relevant ones.

try:
    [] + {}
except TypeError as type_err:
    try:
        assert False
    except AssertionError as assert_err:
        try:
            "" + []
        except TypeError as type_two_err:
            raise type_two_err from type_err

Without the from the above code would show all three tracebacks, but by setting the last exception’s context to the first one, we get only those two, with a new message between them.

Traceback (most recent call last):
  File "C:/python_file.py", line 10, in <module>
    [] + {}
TypeError: can only concatenate list (not "dict") to list

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:/python_file.py", line 18, in <module>
    raise type_two_err from type_err
  File "C:/python_file.py", line 16, in <module>
    "" + []
TypeError: Can't convert 'list' object to str implicitly

Pretty handy stuff, and setting it to from None will remove all other exceptions from the context, so it only displays that exception’s traceback.

try:
    [] + {}
except TypeError as type_err:
    try:
        assert False
    except AssertionError as assert_err:
        try:
            "" + []
        except TypeError as type_two_err:
            raise type_two_err from None

Traceback (most recent call last):
  File "C:/Users/Chris/PycharmProjects/Reusables/ww.py", line 18, in <module>
    raise type_two_err from None
  File "C:/Users/Chris/PycharmProjects/Reusables/ww.py", line 16, in <module>
    "" + []
TypeError: Can't convert 'list' object to str implicitly

Now you and the users of the program know where those exceptions are raised from. 

Warnings

Another often neglected part of the python standard library is warnings. I will also not delve too deeply into it, but at least I’m trying to raise awareness.

I find the most useful feature of warnings as run once reminders. For example, if you have some code that you think might change in the next release, you can put:

import warnings

def my_func():
    warnings.warn("my_func is changing its name to master_func", FutureWarning)
    master_func()

This will print the warning to stderr, but only the first time by default. So if someone’s code is using this function a thousand times, it doesn’t blast their terminal with warning messages.

To learn more about them, check out Python Module of the Week‘s post on it.

Best Practices

1. Never ever have a blank except, always catch Exception or more precise exceptions for internal errors, and have a separate except block for external errors.
2. Only have the code that may cause a precise error in the try block. It’s better to have multiple try except blocks with limited scope rather than a long section of code in the try block with a generic Exception.
3. Have a meaningful message to users (or yourself) in except blocks.
4. Create your own exceptions instead of just raising Exception.