Python 3.11 was released on the 24th of October 2022. In this blog we will look at some of the new and noteworthy features of this release and consider why you might want to upgrade to 3.11 ASAP.
Python 3.11 was released on the 24th of October 2022. It is the latest in the Python 3 series which began back in December 2008.
In this blog we will look at some of the new and noteworthy features of this release and consider why you might want to upgrade to 3.11 ASAP.
Let’s start with some points about performance, possibly one of the most important aspects of the release for many developers. The Python documentation claims a significant improvement in performance overall between Python 3.10 and 3.11 and in some areas improvements of 60% are mentioned. Of course, no one ever claims that their new release is slower than their last release, but some experimentation bears out the improved performance (if not quite to the extent that the documentation hints at).
To explore this claim you can use the pyperformance program available for Python. This package will run the benchmark tests for around 60 different Python functions from the standard libraires. To apply this to a particular version of Python (say 3.10.6 and 3.11) all you need to do is to create an environment with the appropriate version of Python and install the pyperformance library.
This can be easily done using Conda (supplied independently or as part of Anaconda). Using Conda it is possible to create a new conda environment configured with the appropriate version of Python. We have done this below. In the following code we have created a new conda environment called python310-testr using Python 3.10.6 and then activated this environment. Once the python310-test environment is active we then conda installed the pyperformance library. Following this we could run the pyperformance benchmark tests.
(base) % conda create --name python310-test python=3.10.6 (base) % conda activate python310-test (python310-test) % conda install pyperformance (python310-test) % pyperformance run -r -o py310res.json
In the above the -r option is used with the pyperformance benchmark to ensure that a rigorous set of tests are performed. The -o option then tells the benchmark program to write the output to a JSON file called pyres310.json.
Next, we want to repeat the process by using a Python 3.11 environment.
(base) % conda create --name python311-test python=3.11 (base) % conda activate python311-test (python311-test) % conda install pyperformance (python311-test) % pyperformance run -r -o py311res.json
Now that we have generated the performance results files for both versions of Python, we can now perform a comparison between them.
To do this using one of the python installations use the compare command available on pyperformance to compare the two files and generate a CSV file that can be viewed using Excel.
% pyperformance compare py310res.json py311res.json --csv perf_results_comparison.csv
The resulting CSV file can then be opened and analysed in a tool such as Excel. The results obtained on a desktop Mac are given below, they should be treated as relative but illustrate that Python 3.11 (the orange line) is in general a bit quicker than Python 3.10.6 (the blue line) across almost all benchmarks.
To run Python programs several things need to happen. The Python runtime /interpreter needs to start up. It needs to read your program, validate it, internally compile it (into bytecodes) and then run it. This means that in the past even very simple programs have had a significant startUp time overhead.
However, the way in which Python 3.11 handles byte codes internally has changed and this can improve startUp times. For a long running program this may not be significant, however for a small script or program then this may be advantageous.
TOML (Tom’s Obvious Minimal Language) is a file format often used for configuration information. Internally Python has used this file format for the pyproject.toml config file since PEP (Python Enhancement proposal)518 was introduced. However, at the time TOML was a relatively new file format and no developer oriented, public support was provided out of the box for this it. However, TOML is now far more established and stable as such Python 3.11 now provides read-only support for TOML format file via the tomllib module.
There are several notable improvements to error and exception handling that are present in Python 3.11 These are considered below.
Exception Groups
An exception group is essentially a group of exceptions wrapped up as a single entity. The exception group can then be treated as a regular exception.
This means that it is possible to raise several exceptions at once. You can then either catch the exception group as a whole, or using the new except* keyword, you can catch an individual exception from with the group.
To create an ExceptionGroup you give it a description of the group and a list of the exceptions it wraps. For example:
ExceptionGroup(“bad data”, [ValueError('Invalid age'), TypeError('Incorrect type')])
You can then raise this ExceptionGroup. To handle the group you can either handle the whole ExceptionGroup, for example:
except ExceptionGroup(“bad data”, [ValueError('Invalid age'), TypeError('Incorrect type')])
Although as this is a little unwieldy and in most cases, developers will be interested in a specific exception, you can use the new except* syntax to catch an embedded exception, for example:
except* ValueError as err:
Zero Cost Error Handling
In Python 3.11 the way in which exceptions are handled internally has changed. Compared to the previous versions of Python, exceptions are now much lighter weight. As part of this change Python now has Zero Cost exceptions.
In previous versions of Python there was always a cost to pay for using a try…exception block. However, in 3.11 there is virtually no overhead for using this construct as long as an Error or Exception is not raised. This brings Python into line with other languages such as Java and C++ and means an overall improvement in the performance of many applications.
Improved Error Reporting
Another useful change associated with error handling is the way in which errors are reported by the Python runtime. In 3.11 this has been improved to reduce the amount of noise included in the generated error call back and to provide improve accuracy in identifying a specific error.
Exception Notes
Exception notes are another new feature in Python 3.11. Exception notes are described in PEP678. This feature allows a programmer to add a note to an exception when it is being handled before rethrowing it. This avoids the need to create application specific exceptions to allow additional information to be provided.
The note can then be viewed in an y error trace back or examined in client code further up the stack. This can be very useful to anyone trying to debug a problem.
To add a note to an exception the add_note() method Is used for example:
try:
// code riaisng a ValueError
except ValueError as err:
err.add_note("Invalid age”)
raise err
Self Type
Python has included the ability to provide type hints for some time now and they were the subject of one of our recent blogs. The facilities provided, however, did not make it possible to indicate that a method on a class returned something of that classes type. This has been rectified in Python 3.11 with the inclusion of the Self type. Note the capitalization here it is Self with a capital ‘S’ and not self with a lower case ‘s’ which is usually used to indicate an instance of a class within a method.
Using the Self type, it is possible to indicate that a method returns an instance of the encapsulating type (or class), for example:
class Person:
def
__init__(self, name: str, age: int):
self.name = name
self.age = age
def
change_name(self, new_name: str) -> Self:
return Person(new_name, self.age)
There are several significant benefits to upgrading to Python 3.11, not least of which is the potential performance improvements that might be gained. However, whether you should upgrade or not of course depends on your own situation. For example, some libraries on which your code depends may not yet be available for 3.11, although it should be noted that most widely used and well-maintained libraries such as NumPy and SciPy already are.
If you found this article interesting you might be interested in our the following Courses: