Testing performance of Python programs can be done in many different ways, environments and modules. In previous article we saw how to do basics tests and measure the execution time: Python test performance and measure time elapsed in seconds with modules time, datetime, timeit, cProfile. In this post I want to present new way of measuring python performance in a Linux top command way. It is supported text and graphical information representing information about the running Python program from a process or a single Python file. The tool has fancy name as py-spy and you can use it in PyCharm or as a console command.
Here you can find more information about the program: Py-Spy: A sampling profiler for Python programs
The advantages of using this sampling profiler are:
- you can test the program while it's running based on the process
- you have top like output
- good visual output save to a image
- working on Linux, OSX and Windows and all python versions: 2.7, 3.3, 3.7
This is the link for the video tutorial: Powerful Python Performance Profiler
Installation of the tool can be done by:
pip install py-spy
If you want to measure performance in PyCharm then you can:
- open PyCharm
- Show terminal - ALT + F12
- Be sure that your virtual environment is used by the name in the brackets before the command (if you use one):
(myenv) user@user-machine:~/PycharmProjects/python/tests$
- install it by:
pip install py-spy
You have two options of usage(it's the same if you test performance in PyCharm or in the console):
py-spy --pid 12345
# OR
py-spy -- python myprogram.py
the result of this is:
Collecting samples from 'python ProfilingIsolation.py' (python v3.6.6)
Total Samples 300
GIL: 0.00%, Active: 100.00%, Threads: 1
%Own %Total OwnTime TotalTime Function (filename:line)
63.00% 63.00% 0.630s 0.630s test (ProfilingIsolation.py:21)
15.50% 15.50% 0.220s 0.220s after (ProfilingIsolation.py:15)
6.00% 6.00% 0.115s 0.115s after (ProfilingIsolation.py:13)
5.50% 5.50% 0.070s 0.070s after (ProfilingIsolation.py:14)
3.50% 3.50% 0.035s 0.035s test (ProfilingIsolation.py:20)
3.00% 3.00% 0.030s 0.030s test (ProfilingIsolation.py:19)
2.50% 2.50% 0.025s 0.025s test (ProfilingIsolation.py:18)
1.00% 1.00% 0.020s 0.020s after (ProfilingIsolation.py:12)
0.00% 100.00% 0.000s 1.50s run (cProfile.py:16)
0.00% 100.00% 0.000s 1.50s runctx (cProfile.py:100)
0.00% 0.00% 0.010s 0.010s before (ProfilingIsolation.py:5)
0.00% 0.00% 0.100s 0.100s before (ProfilingIsolation.py:7)
0.00% 0.00% 0.090s 0.090s before (ProfilingIsolation.py:6)
0.00% 100.00% 0.000s 1.50s run (cProfile.py:95)
0.00% 0.00% 0.155s 0.155s before (ProfilingIsolation.py:8)
0.00% 100.00% 0.000s 1.50s <module> (<string>:1)
0.00% 72.00% 0.000s 0.720s <module> (ProfilingIsolation.py:25)
0.00% 0.00% 0.000s 0.355s <module> (ProfilingIsolation.py:23)
Press Control-C to quit, or ? for help.
The other way of visualization is by using:
py-spy --flame profile.svg --pid 12345
# OR
py-spy --flame profile.svg -- python myprogram.py
The result of this execution is visible below:
The tested code of myprogram.py is below:
import itertools
import cProfile
def before():
for i in range(1, 1000000):
dictA = {'Java': 1, 'Python': 2, 'C++': 3}
dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
dictC = {**dictA, **dictB}Linux, OSX and Windows
def after():
for i in range (1, 1000000):
dictA = {'Java': 1, 'Python': 2, 'C++': 3}
dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
dictA.update(dictB)
def test():
for i in range (1, 1000000):
dictA = {'Java': 1, 'Python': 2, 'C++': 3}
dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
dictC = dict(itertools.chain(dictA.items(), dictB.items()))
cProfile.run('before()')
cProfile.run('after()')
cProfile.run('test()')
which is another way of performance tests in Python. You can create this method and test different executions inside it.