Python: Error & Exception Handling - Debugging

One of the best pieces of advice I was ever given. The best way to learn something is to write about it.

As for Python, there is no better way to learn the proper use of the language, then that of reading the knowledge base article related to the packages and features used daily.

They say experience makes and expert. Experience half the knowledge makes no expert, as the saying goes. Too little knowledge is a dangerous thing.

Error handling in Python is no exception. Above all else, it increases the robustness of the code, prevents potential failures from breaking or effecting critical processes and terminating unexpectedly, affecting other processes, corrupting data stores and causing data integrity and quality issues. In a commercial environment, such a scenario can have catastrophic effects on the operation of the business, not to mention financial implications.

To begin with, one must understand the differences between error and exception handling.

Syntax Errors

While it is possible to handle exceptions at runtime, it is not possible to handle all errors. Such as syntax or parsing errors. Another example: OutOfMemoryError is an irrecoverable error.

The above is a simple syntax example. c = a + b, however + is missing. The parser has outputted the offending line and highlighted the earliest point at which the line where the syntax parsing error occurred with an arrow under b. Filename together with a link to the file and line number is provided making it easier to the go the file and line where the syntax error occurred.

The most common cause for the occurrence of OutofMemory errors results from a process or processes exceeding the allocated memory usage. As an example, loading a dataset to memory that is larger than the allocation, data cleaning process, subset of loaded data caused memory overrun or the resulting analytical / data modelling or learning process attempt to write a dataset to memory that is larger than the allocation.

Python may in rare instances raise OutofMemorryErrors. Allowing the process to catch the error, break and recover through exception handling. One thing to keep in mind. The Python memory management adopts the architecture of the C language - malloc() function. There is no guarantee that all processes will recover. In some cases, an unrecoverable crash will occur. As a result, it is not considered good practice to capture MemoryErrors using exception handling.

Exceptions

Even though the code may be syntactically correct, at times may cause an error during execution.

Errors detected during execution are referred to as exceptions. Such errors are not fatal to a process and can be handled/bypassed even, while a process is running.

Typical examples range from, most common ZeroDivionError - simply put - divided by zero. NameError occurs when a local or global variable name being referenced is not found. A good example of TypeError would be performing an arithmetic operation with mixed variables i.e. int + string.

ZeroDivionError

NameError

TypeError

The above examples are referred to as built-in exceptions. Built-in exceptions are identifiers and not reserved keywords.

The preceding part of the error messages shows the context of where the exception occurred in trackback form. The stack trackback lists source lines. The rest of the lines provide details based on the type of exception and what caused it.

A full list of built-in exception is available for reference. It is good to have a quick read through, gain a basic understanding of what one could expect.

Handling Exceptions

Typically all it takes to handle exceptions within code is a try and except block.

In the above example, we are replicating the ZeroDivionError. The template is simply the message layout we want to print plus a place holder for the error name. The message itself contains a custom message plus the reference name for the built-in exception that occurred. More advanced formats can include a place holder for additional arguments.

In the above example, we are replicating a TypeError scenario and printing an extended format to include the reference name to the built-in exception and additional arguments to show the cause of the error.

The try statement functions as follows:

First, the try clause, the portion of the code between the try and except keywords is executed. If no exception occurs the except clause is skipped. The except portion of the try clause statement comes in to play as soon as an exception occurs. Any code after the except clause is executed.

It is very important to make sure that the correct exception handling is put in place. In the event that the exception which occurred is not the designated exception, an unhandled exception will occur.

Up until Python2, one could handle multiple distinct exceptions as a parenthesized tuple-except(error1, error2, error3), this is no longer the case.

In later versions, multiple exceptions are handled through multiple excepts.

In the example above we are handling two exceptions. The first handle's OS related errors, followed by value related exceptions. We are also handling other unexpected errors that may occur through the final except clause.

It is best practice to use the else clause with the except because it avoids accidentally catching exceptions which were not raised by the code with the except clause.

Raising Exceptions

Raising exceptions allows the engineer to force a specified exception.

In the above example, a custom exception with arguments is raised—the except handler prints instance details, arguments and exception. The example goes a step further, maps out the individual arguments and prints out a custom message. It is possible to raise all built-in exceptions individually. This may be useful in cases such as user input, where the user is expected to input a number but instead inputs a string. The built-in exception - TypeError is best suited for this scenario.

User-Defined Exceptions

It is possible to customise exceptions by creating an exception class. Exception classes should be derived from the exception class itself, either directly or indirectly. Exception classes are defined the same way you define any other class. However, these should be kept as simple as possible and should only consist of attributes that provide information about the error to be extracted by the handler.

As a best practice for custom exception classes-is to have a base class followed by subclasses containing the definition for each exception.

Defined / Predefined Clean-up Actions

The 'finally' clause is the clean-up section of the statement. The code written in this section is executed each time no matter the outcome. Example, closing terminating connections to files, databases, servers, etc. The 'finally' executes just before a break, continue or return statement in the event of successful execution.

In the event of two return statements, one in the try statement and another in the 'finally'. It is the return clause in the 'finally' statement takes precedence. M

Many objects define standard clean-up actions to be undertaken when the object is no longer needed, regardless of whether or not the operation using the object succeeded. Such as closing a file stream or database or server connection.