Just like other programming languages it has its fair share of gotchas that can trip up a Python newbie. In this blog we will look at 15 of the most common mistakes and how to solve them.
Python is now a very popular and extensively used language. It is used in a huge range of application from web services to data analytics, from animation to games systems and from machine learning systems to integration testing frameworks and DevOps.
Just like other programming languages it has its fair share of gotchas that can trip up a Python newbie.
In this blog we will look at 15 of the most common mistakes and how to solve them.
The very first thing most people notice about Python is that indentation
is key to structuring your program. Unlike languages such as Java and C#,
the way in which you indent your program is meaningful and part of the program itself. For example, you indicate the statement or statements that are part of an if statement via indentation. One of the key things to note about indentation is that statements should be indented to the same level for the same language construct.
For example, the following code intends to indicate that the four print statements are all part of the for loop.
...However, the first print statement is indented to a different level than the following print statements (3 spaces versus 4). This will be an error in Python and if we attempt to run this program it will result in a runtime error being generated:
% python main.py
File "main.py", line 4
print(i * i)
^
IndentationError: unexpected indent
This indicates that there is an IndentationError at line 4 – in other words, the indentations do not match.
One issue that can catch out the unwary is that even if the layout looks correct in an editor, you may still get a runtime error. For example, the following code looks as if it has been laid out correctly in Emacs:
...but when we run this program, we get:
python main.py
File "main.py", line 3
print(i * i)
^
TabError: inconsistent use of tabs and spaces in indentation
This error has been generated because a tab has been used for line 3 while spaces have been used for the other lines. This means that as far as Python is concerned the indentations are not the same.
By convention Python programmers should always use spaces and not tabs (although in practice within a single block of code all that is required is that you are consistent in your use of spaces or tabs). However, programmers should not mix the two.
To resolve this sort of issue you can replace the tab with the equivalent spaces. Many IDEs such as PyCharm, do this for you either when you hit the Tab key or when you ask the IDE to reformat the code. The above code is being written in EMACS which does not do any conversion but also does not display any visual difference and thus this can be a difficult problem to resolve if it is the first time you encounter it.
Python is a very case sensitive
programming language. In Python a variable with the name My_Var is a completely different variable to one called my_var. As Python dynamically declares variables this can cause confusion and lead to unexpected bugs in programs.
For example, in the following program two similarly named, but completely different variables, are declared:
my_var = None
print(my_var)
My_Var = input('Please input your name: ')
print(my_var)
When the program is run and the user enters the number 5, the output generated is:
None Please input your name: John None
This is probably not what the programmer expected. The problem is that the program declares two variables. One is declared on line 1 and initialised with the value None. The second is declared on line 3 and is initialised with the string entered by the user. Whereas the programmer probably planned to reuse the first variable from line 1 in line 3!
To solve this problem developers should take great care with how they name variables. In this case if the developer had followed the standard Python conventions as described by PEP 8 – Style Guide for Python Code (note PEP actually stands for Python Enhancement Proposal), then they would have used lower case for all variable names and thus would have always referred to the variable as my_var.
Python allows a parameter to a function or a method to have a default value. This makes the parameter optional and provides a great deal of flexibility in how a function may be invoked.
For example, the following function takes up to four parameters; the first is a required parameter, but the remaining parameters are all optional:
def greeter(name,
title = 'Dr',
prompt = 'Welcome',
message = 'Live Long and Prosper'):
print(prompt, title, name, '-', message)
When the greeter() function is invoked we can provide one or more arguments as shown below: greeter('Eloise')
greeter('Eloise')
greeter('Lloyd', 'Mr')
greeter('Phoebe', 'Miss', 'Hi')
greeter('Adam', 'Mr', 'Hi', 'May the Force be with you')
As such default parameters are a valuable part of the language.
However, many programmers new to Python misunderstand when the initial value for a parameter is evaluated. In the above we used strings to illustrate the use of default parameters which is ideal as they are immutable types (that is once created they cannot be changed).
However, many other types in Python are mutable such as Lists. It is perfectly possible to modify a List after you have created it.
Such types, if used as default values for parameters, can lead to unexpected results. For example, consider the following function and class with a method:
def do_something(msg, data=[]):
data.append(msg)
print(data)
class Bag:
def do_something(self, msg, data=[]):
data.append(msg)
print(data)
What happens if we run the following program that uses the above function and class:
do_something('John')
do_something('Paul')
do_something('Denise')
bag = Bag()
bag.do_something('Adam')
bag.do_something('Jasmine')
bag.do_something('Theeban')
The output from this program is:
['John'] ['John', 'Paul'] ['John', 'Paul', 'Denise'] ['Adam'] ['Adam', 'Jasmine'] ['Adam', 'Jasmine', 'Theeban']
This may not be what you expected! The issue is that the default value is evaluated only once, at the point at which the function or method is defined. As the value specified for the default value of the data parameter is a List, then the same list object is reused each time the function is invoked with only the msg parameter being supplied.
A common work around for this problem is to use an immutable value for the default. Then within the function or method, the mutable object is assigned to the parameter if the default immutable value is being used. For example:
def do_something(msg, data=None):
if data is None:
data = []
data.append(msg)
print(data)
class Bag:
def do_something(self, msg, data=None):
if data is None:
data = []
data.append(msg)
print(data)
If we now rerun the program that uses the above function and class we now get:
['John'] ['Paul'] ['Bill'] ['Denise'] ['Adam'] ['Jasmine'] ['Theeban']
This mistake is similar to the last one in that it relates to when the default value is evaluated. It occurs because if a function is used to initialise a default parameter it is also executed / evaluated at the point that the function is declared.
For example, the following code allows a programmer to log a message with a time stamp indicating when the message was printed. However, the programmer has made the time point parameter optional by providing a default value for it as supplied by the call to the datetime.now() function.
from datetime import datetime
def log_it(msg, time_point = datetime.now()):
print(msg, time_point)
log_it('In here')
log_it('Went In here', datetime.now())
log_it('Now In here')
The output generated by this code is:
In here 2021-06-29 14:28:25.100567 Went In here 2021-06-29 14:28:25.100793 Now In here 2021-06-29 14:28:25.100567
What you can see from this is that the first and third logged messages have the same time stamp and the 2nd message as a different time stamp. This is because the time_point parameter’s default value is evaluated when the function is defined, that is when it is loaded by the Python runtime and this the time stamp generated for the time_point variable is the point in time when the function came into existence and is thus not related to when the function is called.
The fix for this is like the last mistake in that an immutable value should be used as the default value. If the default value is present then a new timestamp should be generated, for example:
def log_it(msg, time_point = None):
if time_point is None:
time_point = datetime.now()
print(msg, time_point)
In Python ranges are used whenever a sequence of values is required. One typical use of a range is with a for loop. For example, to loop through a set of integers starting at Zero we might write:
for i in range(0, 5):
print(i, end=', ')
print()
However, ranges are what are known as open-closed. That is the initial value specified for the range is included in the range, but the end value is outside or closed from the range, thus when we run this program the output is:
0, 1, 2, 3, 4,
If you want to include the end value in the range a common idiom or pattern that Python programmers use is to explicitly specify the step value to use with the range and add that to the final value. For example:
step = 1
for i in range(0, 5 + step, step):
print(i, end=', ')
print()
The range function can take several different optional values. In this case we are including the optional step value which is the third value passed in. We are now explicitly specifying the step value and adding the step value to the end value for the range. When we run this version of the program the output is:
0, 1, 2, 3, 4, 5,
Python has two operators which may look very similar to a non-programmer; these are the = and the == operators. However, they have very different meanings; one means assignment and the other means equality. That is the = operator is used to assign a value to a variable; while == is used to test for equality. However, it is possible in some situations to use a == when you meant to use a =. For example:
x = False x == True print(x)
The output from this program is:
False
This is because the second statement above is a test to see if x has the value True. The result of this equality test is however, thrown away as it is not assigned to anything and of course the value of x is not changed.
Most IDEs will warn you about this as shown below for the PyCharm tool:
This can help to rectify the error.
Python is quite flexible when it comes to what actually is used to represent True and False, in fact the following rules apply:
This flexibility can however lead to mistakes being made. For example, in the following code tests are performed against the temperature value input by the user:
temp = int(input('The temperature: '))
if temp and temp < 10:
print(f'Its cold {temp}')
else:
print(f'Its warm {temp}')
When this program is run with different inputs for example, 1, 0, and -1 we get different outputs:
|
Run 1 |
Run 2 |
Run 3 |
|
The temperature: 1 Its cold 1 |
The temperature: 0 Its warm 0 |
The temperature: -1 Its cold -1 |
According to this when the temperature is 0 its warm!
The problem is that the conditional part of the if statement will always evaluate to false when the temperature that is input is 0. This is because 0 is treated as being equivalent to False.
This is probably not what the programmer intended. They were probably intending to guard against having a rouge value in temp. It would probably have been better to test for this directly.
In general, the best way to avoid this error is always to work with Boolean values directly using True or False. For example:
if temp is not None and temp < 10:
print(f'Its cold {temp}')
else:
print(f'Its warm {temp}')
Python uses a specific search order to find the definitions of variables, functions, or classes. This search order is known by the acronym LEGB, which stands for:
This means that it is possible to hide a built-in function or class by giving something locally the same name.
For example, in the following code a programmer has decided to store some data in a local variable called list:
list = [1, 2, 3] print(list)
If we run this code, then the output generated is:
[1, 2, 3]
All seems fine until later in the code we try to convert a Set to a List using the list() constructor function:
data = list((1, 2, 3)) print(data)
When we run this code, we get the following error generated by Python:
Traceback (most recent call last):
File main.py", line 61, in <module>
data = list((1, 2, 3))
TypeError: 'list' object is not callable
This error indicates that list() is not a callable, that is it is not something that can be called (or executed). If we look up the Python documentation for the list() constructor function (see https://docs.python.org/3/library/stdtypes.html#typesseq-list) we can see that list() can be used to create a list from an iterable thing and Sets are certainly iterable containers. So what is going on?
The problem is that the name of the variable list has hidden (or shadows) the name of the built-in function list. Thus, when we attempt to call list() Python find the variable list containing the List data structure which it then tries to execute at which point the error is generated.
Most IDEs will warn you that you are shadowing / hiding a built-in name; you can use this to change the =name of your variable, function, or class. For example:
The above uses PyCharm and the IDE is indicating that we are shadowing the built-in name list and that we can ‘Rename the element’.
The way in which Python finds modules to load is by searching a predefined series of locations. By default, this set of locations looks in the current directory before searching within the Python installation for built-in modules.
As Python modules are files, for example the module sys is define din a file called sys.py, it is possible to create a file in the current directory with the same name as a system provided module. For example, sys is a built-in module however if we do have a file called sys.py in the current directory, if our cade references the sys module:
import sys
Python will load the local file instead of the system module. When the programmer then tries to access the functions and classes defined within sys they would find that they were not available.
This can be very confusing the first time this happens. However, if the programmer hangs over the import IDEs will often indicate where the import is coming from:
As we had expected sys to be a built-in module, the fact that it is being loaded from a local directory suggests this is not the correct module. If we renamed the sys.py file to something else and re-examine the import we can see that we are now presented with information on the module:
Python supports a lot of functional programming concepts. Key to the languages ability to do this is the fact that functions are callable objects. That is, they are objects in memory and can be referenced or executed (called).
However, this can lead to some confusion for new developers. For example, the following code is legal Python:
import random
messages = ['Hi', 'Hello', 'Welcome']
def get_mod():
return messages[random.randint(0, len(messages) - 1)]
x = get_mod()
print(x)
y = get_mod
print(y)
The output when this code is run is:
Welcome <function get_mod at 0x1079c60d0>
The first line of the output may be the sort of thing you expected; that is one of the messages is printed out, in this case the 3rd one ‘Welcome’. However, the second line of output may look very strange indeed.
What has happened is that for the declaration of the variable x we have assigned the result returned from calling (or executing) the function get_mod(). However, we have assigned the reference to the function get_mod to the variable y. In effect the variable y is now an alias for the function get_mod. Thus when we print out the value of y we get something indicating a reference to a function called get_mod at a particular location in memory.
From the point of view of the language this is not an error, that is obtaining a reference to a function is a valid thing to do. However, it can be confusing for new programmers and leads to errors such as that seen above.
To correct that it is important to remember to include the () when invoking / calling a function, for example:
y = get_mod() print(y)
There are several ways in which Python allows programmers to process each element in a list in turn. One of these is to use list index access within a for loop, such as:
numbers = [1, 2, 3, 4]
for index in range(len(numbers)):
print(index)
if numbers[index] % 2 == 1:
del numbers[index]
print(numbers)
This loop is being used to remove all odd numbers from the list held in the numbers variable.
When we run this program however, we get an error generated:
0
1
2
Traceback (most recent call last):
File "main.py", line 80, in <module>
if numbers[index] % 2 == 1:
IndexError: list index out of range
The problem here is that we are changing the list that we are processing and thus the data in the list and their positions changes.
There are in fact several Pythonic ways to solve this problem including using a filter function, a list comprehension, iterating over a copy of the numbers list etc. Some of these are shown below:
numbers = [1, 2, 3, 4]
evens = list(filter(lambda i: i % 2 == 0, numbers))
print(evens)
evens2 = [i for i in numbers if i % 2 == 0]
print(evens2)
for item in numbers.copy():
if item % 2 == 1:
numbers.remove(item)
print(numbers)
The output from these examples is:
[2, 4] [2, 4] [2, 4]
It is possible to hold any immutable object within a set. This means that a Set can contain a reference to a Tuple (as that is immutable). We can declare a Set literal using the {} brackets. We can thus write:
s1 = { (1, 2, 3)}
print(s1)
This prints out:
{(1, 2, 3)}
However, we cannot nest Lists or other Sets within a Set as these are not immutable types. The following would both generate a runtime error in Python:
# Can't have the following
s2 = { {1, 2, 3} }
print(s2)
s3 = { [1, 2, 3] }
print(s3)
To solve this problem, we can use Frozensets and Tuples. That is we can use Fozensets to nest one Set inside another. A Frozenset is exactly like a Set except that it is immutable and thus it can be nested within a Set. A Set can be converted into a Fozenset using the frozenset() constructor function. For example:
# Need to convert sets into frozensets
s2 = { frozenset({1, 2, 3}) }
print(s2)
print(s2) For Lists we can convert a list into a tuple to add them to a set, for example:
s3 = { tuple([1, 2, 3]) }
print(s3)
Global variables can be referenced from within a function. For example:
max = 100
def print_max():
print(max)
print_max()
This prints out the value 100. However, if Python thinks you are declaring a new local variable that just happens to have the same name as a global variable it will create a local version which will shadow the global one. For example:
max = 100
def print_max():
max = 95
print(max)
print_max()
print(max)
The output from this is:
95 100
However, if you wish to modify the global variable inside the function, for example as is done below:
max = 100
def print_max():
max = max + 1
print(max)
print_max()
Then when we try to run this code, we will get a runtime error:
Traceback (most recent call last):
File "main.py", line 101, in <module>
print_max()
File "main.py", line 98, in print_max
max = max + 1
UnboundLocalError: local variable 'max' referenced before assignment
This is because the variable max is local to the function and there does not have a value assigned to when we try to access it to add 1 to the current max.
Why does it do this? To protect us from ourselves – Python is really saying 'Do you really want to modify a global variable here?'. Instead, it is treating max as a local variable and as such it is being referenced before a value has been assigned to it.
To tell Python that we know what we are doing and that we want to reference the global variable at this point we need to use the keyword global with the name of the variable. For example:
max = 100
def print_max():
global max
max = max + 1
print(max)
print_max()
print(max)
Now when we try to update the variable max inside the function print_max(), Python knows we mean the global version of the variable and uses that one. The result is that we now print out the value 101 and max is updated to 101 for everyone everywhere!
It is possible to define functions inside other functions, and this can be very useful when we are working with collections of data and operations such as map() (which maps a function to all the elements of a collection in turn).
However, local variables are local to a specific function; even functions defined within another function cannot modify the outer functions local variable (as the inner function is a separate function). They can reference it, just as we could reference the global variable earlier; the issue is again modification.
The global keyword is no help here as the outer function's variables are not global, they are local to a function.
For example, if we define a nested function (inner) inside the parent outer function (outer) and want the inner function to modify the local field we have a problem:
def outer():
title = 'original title'
def inner():
title = 'another title'
print('inner:', title)
inner()
print('outer:', title)
outer()
In this example both outer() and inner() functions modify the title variable. However, they are not the same title variable and as long as this is what we need then that is fine; both functions have their own version of a title local variable.
This can be seen in the output where the outer function maintains its own value for title:
inner: another title outer: original title
However, if what we want is for the inner() function to modify the outer() function's title variable then we have a problem.
This problem can be solved using the nonlocal keyword. This indicates that a variable is not global but is also not local to the current function and Python should look within the scope in which the function is defined to fund a local variable with the same name:
If we now declare title as nonlocal in the inner() function, then it will use the outer() functions version of title (it will be shared between them) and thus when the inner() function changes the title it will change it for both functions:
def outer():
title = 'original title'
def inner():
nonlocal title
title = 'another title'
print('inner:', title)
inner()
print('outer:', title)
outer()
The result of running this is:
inner: another title outer: another title
Care needs to be taken when assigning a mutable type from one variable to another. For example, a List is a mutable type. When you assign a List from one variable to another you actually assign a reference to the List from one variable to another. This means that both variables will reference (point to) the same list. Thus, if the contents of the list is modified via one variable, then it will also be modified for the other variable. For example:
data1 = [1, 2, 3, 4] data2 = data1 data2.append(5) print(data1)
This produces the following output:
[1, 2, 3, 4, 5]
To avoid this, you can create a copy of the original list. This will create a new list and copy the values the original list contains over to the new list. For example:
data1 = [1, 2, 3, 4] data2 = data1.copy() data2.append(5) print(data1)
Now the output from this is:
[1, 2, 3, 4]
If the list contains elements that can themselves hold values you may want to use a deepcopy() for example data1.deepcopy().
If you found this article useful and interesting check out our range of instructor-led Python Courses:
Or take a look at some of our other Python blogs: