Python performance profiling in Pycharm

Testing performance of Python programs can be done in many different ways, environments and modules. In previous article we saw how to do basics tests and measure the execution time: Python test performance and measure time elapsed in seconds with modules time, datetime, timeit, cProfile. In this post I want to present new way of measuring python performance in a Linux top command way. It is supported text and graphical information representing information about the running Python program from a process or a single Python file. The tool has fancy name as py-spy and you can use it in PyCharm or as a console command.

Here you can find more information about the program: Py-Spy: A sampling profiler for Python programs

The advantages of using this sampling profiler are:

  • you can test the program while it's running based on the process
  • you have top like output
  • good visual output save to a image
  • working on Linux, OSX and Windows and all python versions: 2.7, 3.3, 3.7

This is the link for the video tutorial: Powerful Python Performance Profiler

Installation of the tool can be done by:

pip install py-spy

If you want to measure performance in PyCharm then you can:

  • open PyCharm
  • Show terminal - ALT + F12
  • Be sure that your virtual environment is used by the name in the brackets before the command (if you use one):
(myenv) user@user-machine:~/PycharmProjects/python/tests$
  • install it by:
pip install py-spy

You have two options of usage(it's the same if you test performance in PyCharm or in the console):

py-spy --pid 12345
# OR
py-spy -- python myprogram.py

the result of this is:

Collecting samples from 'python ProfilingIsolation.py' (python v3.6.6)
Total Samples 300
GIL: 0.00%, Active: 100.00%, Threads: 1

  %Own   %Total  OwnTime  TotalTime  Function (filename:line)                                                                                                                                                                                                        
 63.00%  63.00%   0.630s    0.630s   test (ProfilingIsolation.py:21)
 15.50%  15.50%   0.220s    0.220s   after (ProfilingIsolation.py:15)
  6.00%   6.00%   0.115s    0.115s   after (ProfilingIsolation.py:13)
  5.50%   5.50%   0.070s    0.070s   after (ProfilingIsolation.py:14)
  3.50%   3.50%   0.035s    0.035s   test (ProfilingIsolation.py:20)
  3.00%   3.00%   0.030s    0.030s   test (ProfilingIsolation.py:19)
  2.50%   2.50%   0.025s    0.025s   test (ProfilingIsolation.py:18)
  1.00%   1.00%   0.020s    0.020s   after (ProfilingIsolation.py:12)
  0.00% 100.00%   0.000s     1.50s   run (cProfile.py:16)
  0.00% 100.00%   0.000s     1.50s   runctx (cProfile.py:100)
  0.00%   0.00%   0.010s    0.010s   before (ProfilingIsolation.py:5)
  0.00%   0.00%   0.100s    0.100s   before (ProfilingIsolation.py:7)
  0.00%   0.00%   0.090s    0.090s   before (ProfilingIsolation.py:6)
  0.00% 100.00%   0.000s     1.50s   run (cProfile.py:95)
  0.00%   0.00%   0.155s    0.155s   before (ProfilingIsolation.py:8)
  0.00% 100.00%   0.000s     1.50s   <module> (<string>:1)
  0.00%  72.00%   0.000s    0.720s   <module> (ProfilingIsolation.py:25)
  0.00%   0.00%   0.000s    0.355s   <module> (ProfilingIsolation.py:23)

Press Control-C to quit, or ? for help.

The other way of visualization is by using:

py-spy --flame profile.svg --pid 12345
# OR
py-spy --flame profile.svg -- python myprogram.py

The result of this execution is visible below:

profile

The tested code of myprogram.py is below:

import itertools
import cProfile

def before():
    for i in range(1, 1000000):
        dictA = {'Java': 1, 'Python': 2, 'C++': 3}
        dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
        dictC = {**dictA, **dictB}Linux, OSX and Windows


def after():
    for i in range (1, 1000000):
        dictA = {'Java': 1, 'Python': 2, 'C++': 3}
        dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
        dictA.update(dictB)

def test():
    for i in range (1, 1000000):
        dictA = {'Java': 1, 'Python': 2, 'C++': 3}
        dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
        dictC = dict(itertools.chain(dictA.items(), dictB.items()))

cProfile.run('before()')
cProfile.run('after()')
cProfile.run('test()')

which is another way of performance tests in Python. You can create this method and test different executions inside it.

Related Article