Have a look at this piece of code:

for message in messages:
    try:
        process(message)
    except:
        logger.error('An exception occurred')

Can you spot the anti-pattern?

The first thing that hits the eye is the overly broad except clause. You want to be as specific as possible when catching exceptions, but at the very least, you should limit the except clause to the class Exception. The class hierarchy of the Python’s built-in exception explains why:

BaseException
 +-- SystemExit
 +-- KeyboardInterrupt
 +-- GeneratorExit
 +-- Exception
      +-- ...

If you do not specify an exception class in an except clause, it will also catch SystemExit and KeyboardInterrupt, preventing you from stopping the program.

In the rare case where you actually want to catch BaseException, I would suggest explicitly spelling out except BaseException instead of just using except without a class to make your intention obvious.

Furthermore, the log message is not really helpful. It only tells you that an exception occurred, but what kind of exception that was, that remains a mystery.

Here is a slightly improved version of the code above:

for message in messages:
    try:
        process(message)
    except Exception as e:
        logger.error('An exception occurred: %s', e)

The except clause is still very broad but if you want to make sure that all messaged get processed, it might be warranted. The log output now also contains the exception and is much more helpful. Or is it?

The log message might just say:

An exception occurred: 'NoneType' object is not subscriptable

Or even worse:

An exception occurred: 'body'

The reason for this terse output is that the string representation of an exception only contains the message of the exception. If you have stared at code for long enough, you might be able to make sense of this message, just like the code of the Matrix reveals its mysteries to an experienced operator.

The canonical string representation that you get with repr() or the format parameter ‘%r’ is slightly better:

for message in message:
    try:
        process(message)
    except Exception as e:
        logger.error('An exception occurred: %r', e)

The log messages would now be:

An exception occurred: TypeError("'NoneType' object is not subscriptable",)

And, respectively:

An exception occurred: KeyError('body',)

This is better and might be enough to spot the error if process() is really simple. But if it is moderately complex, it will just leave you scratching your head.

What you really want to know is not only what kind of exception occurred, but also where. That means you need a stack trace.

If you are using the built-in logging module, you can simply use logger.exception() instead of logger.error() and it will print the stack trace for you:

for message in message:
    try:
        process(message)
    except Exception as e:
        logger.exception('An exception occurred.')

Now we are getting somewhere:

An exception occurred:
Traceback (most recent call last):
  File "processing.py", line 16, in 
    process(message)
  File "processing.py", line 6, in process
    body = message['body']
TypeError: 'NoneType' object is not subscriptable

An exception occurred:
Traceback (most recent call last):
  File "processing.py", line 16, in 
    process(message)
  File "processing.py", line 6, in process
    body = message['body']
KeyError: 'body'

Now we know exactly where the exception is occurring and what kind of exception it is. There is only one valuable piece of information missing: the relevant state of your program that triggers this exception.

What state is relevant depends on your application. For our example, we can keep it simple and just log the argument message:

for message in message:
    try:
        process(message)
    except Exception as e:
        logger.exception('An exception occurred while processing message: %s', message)

Now our log messages paint a complete picture:

An exception occurred while processing message: None
Traceback (most recent call last):
  File "processing.py", line 16, in 
    process(message)
  File "processing.py", line 6, in process
    body = message['body']
TypeError: 'NoneType' object is not subscriptable
An exception occurred while processing message: {}
Traceback (most recent call last):
  File "processing.py", line 16, in 
    process(message)
  File "processing.py", line 6, in process
    body = message['body']
KeyError: 'body'

The first error happens because message is simply None. The second error happens because process() tries to access the key body, but message is an empty dictionary.

If you can’t use Logger.exception(), you might want to look at the traceback module. It contains various functions to print and format exceptions and tracebacks.

Summary

• Use the most specific exception classes in your except statements.
• An except statement without a class is most likely wrong. At the very least, use Exception.
• The broader your except statement, the more information you have to log to figure out the source of the problem. For a generic except Exception, you should log the exception type and message, the stack trace and some relevant state.