Python: Everything is an object

Hello everyone. And welcome to today's blog post where I'll be sharing my findings of Python with you. In case you're wondering. No. I'm not lying. As I stated in my last blog post, for Python everything is an object. Literally. If you haven't read it, check it out.

Yes. Everything is an object for Python

This situation is by design. Guido van Rossum, Python's creator, stated in a blog post the nature of Python:

One of my goals for Python was to make it so that all objects were "first class." By this, I meant that I wanted all objects that could be named in the language (e.g., integers, strings, functions, classes, modules, methods, etc.) to have equal status. That is, they can be assigned to variables, placed in lists, stored in dictionaries, passed as arguments, and so forth.

This means exactly what you're thinking: in Python, even primitives treated like such (e.g. integers, floats, characters, booleans, etc.) in other programming languages like C/C++, here are also objects. All of them are based (at least on CPython) on a unique structure called a PyObject. Yes, functions are also objects... and objects are objects. No, I'm not joking.

If you don't believe me, then quick tip: use the dir() function in the Python REPL (Python 3 only) in any function really, and you'll see the methods and attributes of each that you can use. Or import a module of yours and you'll also see the methods and attributes of your functions. You'll also see some methods surrounded by two underscores (or "dunders" as they're known) before and after their names. Those are called magic methods. I'll post in the bibliography section a link to an article that talks about them in more depth.

id() and type()

Now that we know that everything is an object. Let's explore this more with the Python REPL and introduce two functions that will help you strengthen your knowledge about Python's inner workings.

Python is a dynamically-typed programming language, meaning that the type of the variables is known at run-time (while your program is running) in contrast with languages like C, which is a statically-typed programming language in which its types are known at compilation time. This is why in C or similar programming languages you have to declare the type of your variable, while in Python you don't.

Anyways, onto the example:

>>> a = 1
>>> type(a)
<class 'int'>
>>> a = "Hello world"
>>> type(a)
<class 'str'>
>>> a = ["I", "am", "a", "list"]
>>> type(a)
<class 'list'>        

See what I mean with dynamically-typed? You can change the type of your variable at any point of your code and it will be reflected back using the type() built-in function. Which will output effectively which type the current variable is an instance of.

"But wait!", I hear you say, dear reader, "what happens to the "1" declared earlier when I change from "1" to "Hello World" in the above example?" Aaah, that's an excellent question that leads me to the id() function which (at least in standard CPython) outputs a decimal representation of the object's memory allocation. Watch:

>>> a = 1
>>> id(a)
4305201568
>>> a = "Hello world"
>>> id(a)
4355025392
>>> a = ["I", "am", "a", "list"]
>>> id(a)
4354985352        

As you can see, each object created has its own ID (its own memory allocation). This helps me to explain this concept: variable names in Python are more like tags than boxes, and it works a little bit backward than other programming languages. While in other programming languages you first create the variable (in general) by storing the value in memory assigned to that name, in Python is the other way around: first, the value is created and then its value is assigned to the name. This explains why you can change the name (a label for that matter) to any data type.

is and ==

A quick detour to explain two important operators in Python that help to strengthen the understanding of the above explanation: is and ==.

The is operator checks if a given object is the same in its ID as another object. In other words, if an object is the same as another object. The == operator instead checks if the contents of an object are the same as the contents of another. This means you can have two objects with exactly the same content but are two completely separate entities. An example:

>>> a = "Hello World"
>>> b = "Hello World"
>>> a == b
True
>>> a is b
False
>>> id(a)
4363413936
>>> id(b)
4363414000        

See? Although we have two variables with two different names with exactly the same content, because they are two different entities, then that's why the is operator throws False as a result of that query.

Mutable vs immutable objects, and how they differ

This leads perfectly to the next point in question. Objects in Python are classified into two behaviors: mutable and immutable.

Mutable, as its name implies, means that objects can mutate (change) so that their ID is the same regardless if you decide to add or delete items to it. Immutable objects instead are fixed: you can't change their contents or their content amount even if you want to. If you change their contents (or their amount) you'll be essentially creating a new object with a new ID... or Python most likely will throw an exception, whichever applies to each case.

Examples of mutable objects include lists, dictionaries, sets, byte arrays, and user-defined objects while immutable objects tend to be your everyday primitives like integers, floats, booleans, strings, and more "Pythonic" data types such as bytes, complex number representations (a + bj, j being the square root of -1, called i in mathematics. Why Python decided to call it j? Because it uses the conventions from electrical engineering), frozen sets, ... and tuples, in a way.

Before explaining the tuples case, an example of an immutable object with integers.

>>> a = 540
>>> id(a)
4327755600
>>> a = a + 20
>>> a
560
>>> id(a)
4363031984        

As you can see, their ID is different, meaning that a different object is created for the sum of "a" and 20. Now let's do the same with strings:

>>> a = "Hello "
>>> id(a)
4363410096
>>> a = a + "World"
>>> a
'Hello World'
>>> id(a)
4363414320        

Same thing. Because strings are immutable, you can't create a new string with the same ID as the old one by appending a new string to it. Look what happens when I try to change a character to the string "Hello World":

>>> a
'Hello World'
>>> a[0] = "M"
Traceback (most recent call last):
? File "<stdin>", line 1, in <module>
TypeError: 'str' object does not support item assignment        

Now you believe me? You can't do that. But you can slice a string to create a new string, though, albeit with a different ID. Remember: strings are also immutable.

But what happens when we want to assign a new name to our "Hello World" string, and then decide to change one of the two strings to another string?

>>> b = a
>>> id(b)
4363414320
>>> id(a)
4363414320
>>> b = "This is another string"
>>> id(b)
4363417688
>>> a
'Hello World'
>>> id(a)
4363414320        

Both had the same ID until I decided to change the string of "b", but the original one "a" wasn't affected by this. Interesting. Let's try to do this with a list and see what happens. Check the next example with Python Tutor and see it for yourself:

>>> a = ["Hello", "World"]
>>> id(a)
4363373960
>>> b = a
>>> id(b)
4363373960
>>> b.append("!")
>>> a
['Hello', 'World', '!']
>>> id(a)
4363373960
>>> id(b)
4363373960
>>> a == b
True
>>> a is b
True        

What?!? How come when I decided to add an item to our list "b", "a" also changed? Remember that lists are mutable and that the object first is created and then assigned to a name? This is why. A mutable object can have any number of names and if you change something it will be reflected to the rest of the names assigned to that value, this explains also why when we use the is operator and the == operator both print True. Because "a" and "b" in this case are the same object. Never forget that. When you create a new name for the same variable, Python never will copy the data to the new name unless you explicitly do so with a slice or a copy method, although that depends on the object. And this can cause a major confusion with people coming from a C (or similar) background.

By the way, the act of accessing and modifying a given value (ID) from a name, and all changes made to the value are reflected in all the different names associated with the value when you call it, it's called aliasing in Python. You know, like hard links? If you don't know about hard links here's a blog entry I made about this topic.

And this brings me to tuples (and frozen sets also). Although they are immutable in content and amount of objects in it, if you decide to store a mutable object in it (like a list or a dictionary inside a tuple), you can modify the content of the list without getting an error. This time, let's do a tuple with two elements, a fruit "grape" and a list of grape colors ("violet" and "green"). Again, if you want to check it in Python Tutor, you will understand this better.

>>> fruit = ("grape", ["green", "violet"])
>>> colors = fruit[1]
>>> id(fruit)
4363370952
>>> id(colors)
4363414344
>>> colors.append("yellow")
>>> id(colors)
4363414344
>>> id(fruit)
4363370952        

Better? I hope so. Because we'll be talking about how mutable and immutable objects are treated when passed as arguments to functions.

Functions and mutable/immutable arguments

What happens when we want to pass an immutable object as an argument to a function and we want to modify its value? What do you think will happen? Will it stay the same or will it change? Let's see. As always, check the following example and see the process by yourself in PythonTutor

def multiply(num):
? ? print(num)
    num *= 2
    print(num)


number = 2
multiply(number)
print(number)



Output:
2
4
2        

Wait. Why I couldn't change the value inside the function even though inside the function it did change? Because integers are immutable, remember? In Python, values passed as arguments in functions change depending on the mutability of the object. If the object is immutable, it doesn't matter if you change it inside an argument, the original value won't change. Think of this as "passing by value" in C programming.

Ok, so what happens on a mutable object? Let's try to copy one list to another list. Here's the PythonTutor link to our example:

def copy_list(list1, list2):
? ? list2 = list1


list1 = [1, 2, 3]
list2 = []
copy_list(list1, list2)
print(list2)



Output:
[]        

Remember that a variable can have multiple labels, and this object is mutable? Well, this is a side effect of that situation. Because inside the function we tried to "copy" one list to the other, instead, it assigned the label from list2 to list1, and that's that. No copy whatsoever. These local assignments are destroyed when the function exits, and if they weren't claimed by any other value outside of the function, nothing will be copied at the end.

Let's do this right this time with slicing and a return statement. PythonTutor link:

def copy_list(list1, list2):
? ? list2 = list1[:]
? ? return list2


list1 = [1, 2, 3]
list2 = [4, 5, 6]
list2 = copy_list(list1, list2)
print(list2)



Output:
[1, 2, 3]        

Now we have two distinct objects, each one with the same contents. And what about modifying only an element of the mutable object? You can do that, and it will work just as you would do by passing an argument as a reference in C. Check it out in PythonTutor:

def append_to_list(list1, value):
? ? list1.append(value)


list1 = [1, 2, 3]
append_to_list(list1, 10)
print(list1)



Output:
[1, 2, 3, 10]        

Some "buts" to the immutability thing on integers...

In Python there's a giant BUT talking about the case of integers. Let's put it in these terms: why 6 is 6 is True while -6 is -6 is False? If each integer created is a distinct and unique object assigned to a label, then why 6 is 6 is True knowing that I created two different objects with a value of 6 each?

The answer is small integer caching. Python loads to memory (RAM) numbers from -5 up to 256 (262 numbers total) when first running any program. The reason for this is because programmers use these numbers so much, for optimization reasons (at least for standard CPython) this number range defined by the macros NSMALLNEGINTS (-5 to 0) and NSMALLPOSINTS (0 to 256) is preloaded to memory. When a programmer decides to use some number from this range, Python searches the number in this array of numbers and uses it directly instead of creating a different object for that number.

Now you know why 6 is 6 and -6 is not -6.

This "but" applies also to empty immutable objects, like tuples, and strings without whitespaces and less than 20 characters.

Functions are also objects, and __slots__

Remember that everything is an object in Python? Well, functions are also objects. Let's do some magic. What do we do when we don't have any arguments passed to a function but we do need to keep track of something inside of it, like a counter? For this, we will use the getattr() function. Watch (in PythonTutor):

def print_counter():
? ? print_counter.counter = getattr(print_counter, "counter", 0) + 1
? ? return print_counter.counter
? ??
for i in range(10):
? ? print(print_counter())



Output:
1
2
3
4
5
6
7
8
9
10
        

Why this happen? Well, I created an attribute to the object print_counter and set it to any value that I wanted. This time with the getattr() function which creates an attribute (if it doesn't exist) and sets its value to the default given as a third argument. Effectively creating an instance attribute of that object. Neat, isn't it?

And if I want to restrict the number of attributes created freely just as in this case? Well, that's why __slots__ come into play.

Because attributes of an object are stored in a dictionary, and dictionaries don't have a fixed number of elements, this explains why you could create additional attributes in the example of the counter above. __slots__ on the other hand replaces __dict__ and it helps reduce the usual overhead dictionaries make. Attribute access times then, improve significantly, saving also memory space. As a side effect, it works like a static structure preventing any more attributes to be created after the creation of an instance.

Well, that's it, everyone. I hope you have a really good day and you have learned something. All my sources will be listed below for you to check out. Thank you for reading, and please share this blog post if you found it useful! See you around!

Bibliography: