Variable Scope in Python

When you declare a variable, that variable is visible in parts of your program, depending on where you declare it.

If you declare it outside of any function, the variable is visible to any code running after the declaration, including functions:

age = 8

def test():
    print(age)

print(age) # 8
test() # 8
        

We call it a global variable.

If you define a variable inside a function, that variable is a local variable, and it is only visible inside that function. Outside the function, it is not reachable:

def test():
    age = 8
    print(age)

test() # 8

print(age)
# NameError: name 'age' is not defined
        

How to Accept Arguments from the Command Line in Python

Python offers several ways to handle arguments passed when we invoke the program from the command line.

So far you've run programs either from a REPL, or using

python <filename>.py
        

You can pass additional arguments and options when you do so, like this:

python <filename>.py <argument1>
python <filename>.py <argument1> <argument2>
        

A basic way to handle those arguments is to use the sys module from the standard library.

You can get the arguments passed in the sys.argv list:

import sys
print(len(sys.argv))
print(sys.argv)
        

The sys.argv list contains as the first item the name of the file that was run, for example ['main.py'].

This is a simple way, but you have to do a lot of work. You need to validate arguments, make sure their type is correct, and you need to print feedback to the user if they are not using the program correctly.

Python provides another package in the standard library to help you: argparse.

First you import argparse and you call argparse.ArgumentParser(), passing the description of your program:

import argparse

parser = argparse.ArgumentParser(
    description='This program prints the name of my dogs'
)
        

Then you proceed to add arguments you want to accept. For example in this program we accept a -c option to pass a color, like this: python program.py -c red

import argparse

parser = argparse.ArgumentParser(
    description='This program prints a color HEX value'
)

parser.add_argument('-c', '--color', metavar='color', required=True, help='the color to search for')

args = parser.parse_args()

print(args.color) # 'red'
        

If the argument is not specified, the program raises an error:

?  python python program.py
usage: program.py [-h] -c color
program.py: error: the following arguments are required: -c
        

You can set an option to have a specific set of values, using choices:

parser.add_argument('-c', '--color', metavar='color', required=True, choices={'red','yellow'}, help='the color to search for')
        

?  python python program.py -c blue
usage: program.py [-h] -c color
program.py: error: argument -c/--color: invalid choice: 'blue' (choose from 'yellow', 'red')
        

There are more options, but those are the basics.

And there are community packages that provide this functionality, too, like Click and Python Prompt Toolkit.

Lambda Functions in Python

Lambda functions (also called anonymous functions) are tiny functions that have no name and only have one expression as their body.

In Python they are defined using the lambda keyword:

lambda <arguments> : <expression>
        

The body must be a single expression - an expression, not a statement.

This difference is important. An expression returns a value, a statement does not.

The simplest example of a lambda function is a function that doubles the value of a number:

lambda num : num * 2
        

Lambda functions can accept more arguments:

lambda a, b : a * b
        

Lambda functions cannot be invoked directly, but you can assign them to variables:

multiply = lambda a, b : a * b

print(multiply(2, 2)) # 4
        

The utility of lambda functions comes when combined with other Python functionality, for example in combination with map(), filter() and reduce().

Recursion in Python

A function in Python can call itself. That's what recursion is. And it can be pretty useful in many scenarios.

The common way to explain recursion is by using the factorial calculation.

The factorial of a number is the number n mutiplied by n-1, multiplied by n-2... and so on, until reaching the number 1:

3! = 3 * 2 * 1 = 6
4! = 4 * 3 * 2 * 1 = 24
5! = 5 * 4 * 3 * 2 * 1 = 120
        

Using recursion we can write a function that calculates the factorial of any number:

def factorial(n):
    if n == 1: return 1
    return n * factorial(n-1)

print(factorial(3)) #   6
print(factorial(4)) #  24
print(factorial(5)) # 120
        

If inside the factorial() function you call factorial(n) instead of factorial(n-1), you are going to cause an infinite recursion. Python by default will halt recursions at 1000 calls, and when this limit is reached, you will get a RecursionError error.

Recursion is helpful in many places, and it helps us simplify our code when there's no other optimal way to do it, so it's good to know this technique.

Nested Functions in Python

Functions in Python can be nested inside other functions.

A function defined inside a function is visible only inside that function.

This is useful to create utilities that are useful to a function, but not useful outside of it.

You might ask: why should I be "hiding" this function, if it does no harm?

One, because it's always best to hide functionality that's local to a function, and is not useful elsewhere.

Also, because we can make use of closures (more on this later).

Here is an example:

def talk(phrase):
    def say(word):
        print(word)

    words = phrase.split(' ')
    for word in words:
        say(word)

talk('I am going to buy the milk')
        

If you want to access a variable defined in the outer function from the inner function, you first need to declare it as nonlocal:

def count():
    count = 0

    def increment():
        nonlocal count
        count = count + 1
        print(count)

    increment()

count()
        

This is useful especially with closures, as we'll see next.

Closures in Python

If you return a nested function from a function, that nested function has access to the variables defined in that function, even if that function is not active any more.

Here is a simple counter example.

def counter():
    count = 0

    def increment():
        nonlocal count
        count = count + 1
        return count

    return increment

increment = counter()

print(increment()) # 1
print(increment()) # 2
print(increment()) # 3
        

We return the increment() inner function, and that still has access to the state of the count variable even though the counter() function has ended.

Decorators in Python

Decorators are a way to change, enhance, or alter in any way how a function works.

Decorators are defined with the @ symbol followed by the decorator name, just before the function definition.

Example:

@logtime
def hello():
    print('hello!')
        

This hello function has the logtime decorator assigned.

Whenever we call hello(), the decorator is going to be called.

A decorator is a function that takes a function as a parameter, wraps the function in an inner function that performs the job it has to do, and returns that inner function. In other words:

def logtime(func):
    def wrapper():
        # do something before
        val = func()
        # do something after
        return val
    return wrapper
        

Docstrings in Python

Documentation is hugely important, not just to communicate to other people what the goal of a function/class/method/module is, but it also communicates it to yourself.

When you come back to your code 6 or 12 months from now, you might not remember all the knowledge you are holding in your head. At that point, reading your code and understanding what it is supposed to do will be much more difficult.

Comments are one way to help yourself (and others) out:

# this is a comment

num = 1 #this is another comment
        

Another way is to use docstrings.

The utility of docstrings is that they follow conventions. As such they can be processed automatically.

This is how you define a docstring for a function:

def increment(n):
    """Increment a number"""
    return n + 1
        

This is how you define a docstring for a class and a method:

class Dog:
    """A class representing a dog"""
    def __init__(self, name, age):
        """Initialize a new dog"""
        self.name = name
        self.age = age

    def bark(self):
        """Let the dog bark"""
        print('WOF!')
        

Document a module by placing a docstring at the top of the file, for example supposing this is dog.py:

"""Dog module

This module does ... bla bla bla and provides the following classes:

- Dog
...
"""

class Dog:
    """A class representing a dog"""
    def __init__(self, name, age):
        """Initialize a new dog"""
        self.name = name
        self.age = age

    def bark(self):
        """Let the dog bark"""
        print('WOF!')
        

Docstrings can span multiple lines:

def increment(n):
    """Increment
    a number
    """
    return n + 1
        

Python will process those and you can use the help() global function to get the documentation for a class/method/function/module.

For example calling help(increment) will give you this:

Help on function increment in module
__main__:

increment(n)
    Increment
    a number
        

There are many different standards to format docstrings, and you can choose to adhere to your favorite one.

I like Google's standard: https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings

Standards allow to have tools to extract docstrings and automatically generate documentation for your code.

Introspection in Python

Functions, variables, and objects can be analyzed using introspection.

First, using the help() global function we can get the documentation if provided in form of docstrings.

Then, you can use print() to get information about a function:

def increment(n):
    return n + 1

print(increment)

# <function increment at 0x7f420e2973a0>
        

or an object:

class Dog():
    def bark(self):
        print('WOF!')

roger = Dog()

print(roger)

# <__main__.Dog object at 0x7f42099d3340>
        

The type() function gives us the type of an object:

print(type(increment))
# <class 'function'>

print(type(roger))
# <class '__main__.Dog'>

print(type(1))
# <class 'int'>

print(type('test'))
# <class 'str'>
        

The dir() global function lets us find out all the methods and attributes of an object:

print(dir(roger))

# ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'bark']
        

The id() global function shows us the location in memory of any object:

print(id(roger)) # 140227518093024
print(id(1))     # 140227521172384
        

It can be useful to check if two variables point to the same object.

The inspect standard library module gives us more tools to get information about objects, and you can check it out here: https://docs.python.org/3/library/inspect.html

Annotations in Python

Python is dynamically typed. We do not have to specify the type of a variable or function parameter, or a function return value.

Annotations allow us to (optionally) do that.

This is a function without annotations:

def increment(n):
    return n + 1
        

This is the same function with annotations:

def increment(n: int) -> int:
    return n + 1
        

You can also annotate variables:

count: int = 0
        

Python will ignore those annotations. A separate tool called mypy can be run standalone, or integrated by IDE like VS Code or PyCharm to automatically check for type errors statically, while you are coding. It will also help you catch type mismatch bugs before even running the code.

A great help especially when your software becomes large and you need to refactor your code.

Exceptions in Python

It's important to have a way to handle errors, and Python gives us exception handling to do so.

If you wrap lines of code into a try: block:

try:
    # some lines of code
        

If an error occurs, Python will alert you and you can determine which kind of error occurred using a except blocks:

try:
    # some lines of code
except <ERROR1>:
    # handler <ERROR1>
except <ERROR2>:
    # handler <ERROR2>
        

To catch all exceptions you can use except without any error type:

try:
    # some lines of code
except <ERROR1>:
    # handler <ERROR1>
except:
    # catch all other exceptions
        

The else block is run if no exceptions were found:

try:
    # some lines of code
except <ERROR1>:
    # handler <ERROR1>
except <ERROR2>:
    # handler <ERROR2>
else:
    # no exceptions were raised, the code ran successfully
        

A finally block lets you perform some operation in any case, regardless of whether an error occurred or not:

try:
    # some lines of code
except <ERROR1>:
    # handler <ERROR1>
except <ERROR2>:
    # handler <ERROR2>
else:
    # no exceptions were raised, the code ran successfully
finally:
    # do something in any case
        

The specific error that's going to occur depends on the operation you're performing.

For example if you are reading a file, you might get an EOFError. If you divide a number by zero you will get a ZeroDivisionError. If you have a type conversion issue you might get a TypeError.

Try this code:

result = 2 / 0
print(result)
        

The program will terminate with an error:

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    result = 2 / 0
ZeroDivisionError: division by zero
        

and the lines of code after the error will not be executed.

Adding that operation in a try: block lets us recover gracefully and move on with the program:

try:
    result = 2 / 0
except ZeroDivisionError:
    print('Cannot divide by zero!')
finally:
    result = 1

print(result) # 1
        

You can raise exceptions in your own code, too, using the raise statement:

raise Exception('An error occurred!')
        

This raises a general exception, and you can intercept it using:

try:
    raise Exception('An error occurred!')
except Exception as error:
    print(error)
        

You can also define your own exception class, extending from Exception:

class DogNotFoundException(Exception):
    pass
        

pass here means "nothing" and we must use it when we define a class without methods, or a function without code, too.

try:
    raise DogNotFoundException()
except DogNotFoundException:
    print('Dog not found!')