Python is a very powerful language, there are so many libraries available for it.
However, many developers will complain about its speed by comparison to certain other languages, for example, C or C++.
This is because Python is an interpreted language by design, as opposed to being compiled. Meaning that each time it executes, the instructions need to be converted right there on the spot, making for slower overall execution times.
There are ways to make it faster, for example, the Pypy project which uses a Just-in-Time (JIT) compiler which runs standard python applications much faster than simply using Python on its own. For the most part, Pypy is somewhat of a miracle drop-in replacement, but there are times when it’s not actually faster. In this writeup, I aim to introduce Pypy and show some areas where it excels, but also where it has very little benefit.
An introduction to Pypy
“PyPy is a fast, compliant alternative implementation of the Python language.”
As per the Pypy website:
It is sold as having several advantages and distinct features:
- Speed: thanks to its Just-in-Time compiler, Python programs often run faster on PyPy.
- Memory usage: memory-hungry Python programs (several hundreds of MBs or more) might end up taking less space than they do in CPython.
- Compatibility: PyPy is highly compatible with existing python code. It supports cffi and can run popular python libraries like twisted and django.
- Stackless: PyPy comes by default with support for stackless mode, providing micro-threads for massive concurrency.
- As well as other features.
Over the years, I have heard many great things about this project, and have used it here and there. Even the creator of Python seems to praise it:
“If you want your code to run faster, you should probably just use PyPy.?
Guido van Rossum (creator of Python)
A sample python benchmark script
In order to run some tests, let’s first get a standard python script we can use to test with. To save ourselves a couple of minutes, I grabbed one from StackOverflow.
What this does, is time how long it takes to append a hundred integers to a list. Simple enough.
In order to not mess with our wider Python environment, we will run all our tests in a newly created python virtual environment.
Opening a terminal, we can run the following bash which will create a place for us to run our experiments from, and go in there:
Now we can create a python virtual environment and activate it.
At this stage, we place the python benchmarking code from above into a file called
test1.py. We can see it is in there if we
Now run it with standard Python3 to see how it performs.
On my machine, I got the following output:
Let’s automatically do this 3 times to make sure we are getting a fair assessment:
Once again, on my machine this yielded the following output:
So now we know what to beat!
As I’m on a Mac, let’s install
Homebrew. We install
pypy3 as opposed to
pypy because we are running
If we used
pypy it would only be compatible for Python2 and we don’t want that.
You can also install Pypy on Windows, Linux and other systems, for more on this, read more on the Pypy downloads site.
Running the benchmark on Python
Now that we are all setup, let’s run our Python benchmark again:
Now run it 3 times for consistency:
Running the benchmark on Pypy
Now that we know how Python performs, let’s give Pypy3 a try with the exact same tests:
That’s incredibly fast! Let’s run it 3 times as we did with Python.
Pretty amazing if you ask me!
Complicating matters a bit
So we have discovered that Pypy is pretty fast for simple test comparisons, but what about comparing something else, like some regular looping and global counts?
Use the below code and place it in a file called
This time around we will time it using the CLI’s
time command. Let’s try with Pypy first this time!
Let’s change things around a little and try again; put the following code in a file called
Let’s try a best of 10 on both cases to see how that runs:
We can clearly see that Pypy3 knocked the socks off of Python3 once again, consistently.
Bonus tests with Multiprocessing
Let’s have a go with the following Multiprocessing code; place it in a file called
Running regular good old Python:
Now the same test with Pypy:
It’s almost 3 times slower! Let’s comment out the
First we run Python:
I’m not sure whether to congratulate Python, or complain about Pypy in this instance!?
There were a few discrepancies, initially, I thought it was down to rendering issues using the
print() function, until I tested with the Multiprocessing tests.
Pypy3 is a lot faster than each of our test cases using regular
Python3, barring a few exceptions.
I really wanted to run some tests using
Asyncio but couldn’t as Pypy supports Python 3.6 and Asyncio was only introduced in Python 3.7, so hopefully in the next Pypy release, I will be able to update this post with the findings.
For now, I will continue to use Python3, but always test the execution of my application in Pypy to see if there are speed improvements that I can get for free.
Unfortunately, I’m left a bit dumbfounded as to exactly where the rule and the exception lie with all of this. Anyone care to educate me further?
Featured image: SUPERFAST Trailer (Fast and Furious Spoof Movie)