Python performance profiling in Pycharm
Testing performance of Python programs can be done in many different ways, environments and modules. In previous article we saw how to do basics tests and measure the execution time: Python test performance and measure time elapsed in seconds with modules time, datetime, timeit, cProfile. In this post I want to present new way of measuring python performance in a Linux top command way. It is supported text and graphical information representing information about the running Python program from a process or a single Python file. The tool has fancy name as py-spy and you can use it in PyCharm or as a console command.
Here you can find more information about the program: Py-Spy: A sampling profiler for Python programs
The advantages of using this sampling profiler are:
- you can test the program while it's running based on the process
- you have top like output
- good visual output save to a image
- working on Linux, OSX and Windows and all python versions: 2.7, 3.3, 3.7
This is the link for the video tutorial: Powerful Python Performance Profiler
Installation of the tool can be done by:
pip install py-spy
If you want to measure performance in PyCharm then you can:
- open PyCharm
- Show terminal - ALT + F12
- Be sure that your virtual environment is used by the name in the brackets before the command (if you use one):
(myenv) user@user-machine:~/PycharmProjects/python/tests$
- install it by:
pip install py-spy
You have two options of usage(it's the same if you test performance in PyCharm or in the console):
py-spy --pid 12345
# OR
py-spy -- python myprogram.py
the result of this is:
Collecting samples from 'python ProfilingIsolation.py' (python v3.6.6)
Total Samples 300
GIL: 0.00%, Active: 100.00%, Threads: 1
%Own %Total OwnTime TotalTime Function (filename:line)
63.00% 63.00% 0.630s 0.630s test (ProfilingIsolation.py:21)
15.50% 15.50% 0.220s 0.220s after (ProfilingIsolation.py:15)
6.00% 6.00% 0.115s 0.115s after (ProfilingIsolation.py:13)
5.50% 5.50% 0.070s 0.070s after (ProfilingIsolation.py:14)
3.50% 3.50% 0.035s 0.035s test (ProfilingIsolation.py:20)
3.00% 3.00% 0.030s 0.030s test (ProfilingIsolation.py:19)
2.50% 2.50% 0.025s 0.025s test (ProfilingIsolation.py:18)
1.00% 1.00% 0.020s 0.020s after (ProfilingIsolation.py:12)
0.00% 100.00% 0.000s 1.50s run (cProfile.py:16)
0.00% 100.00% 0.000s 1.50s runctx (cProfile.py:100)
0.00% 0.00% 0.010s 0.010s before (ProfilingIsolation.py:5)
0.00% 0.00% 0.100s 0.100s before (ProfilingIsolation.py:7)
0.00% 0.00% 0.090s 0.090s before (ProfilingIsolation.py:6)
0.00% 100.00% 0.000s 1.50s run (cProfile.py:95)
0.00% 0.00% 0.155s 0.155s before (ProfilingIsolation.py:8)
0.00% 100.00% 0.000s 1.50s <module> (<string>:1)
0.00% 72.00% 0.000s 0.720s <module> (ProfilingIsolation.py:25)
0.00% 0.00% 0.000s 0.355s <module> (ProfilingIsolation.py:23)
Press Control-C to quit, or ? for help.
The other way of visualization is by using:
py-spy --flame profile.svg --pid 12345
# OR
py-spy --flame profile.svg -- python myprogram.py
The result of this execution is visible below:
The tested code of myprogram.py is below:
import itertools
import cProfile
def before():
for i in range(1, 1000000):
dictA = {'Java': 1, 'Python': 2, 'C++': 3}
dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
dictC = {**dictA, **dictB}Linux, OSX and Windows
def after():
for i in range (1, 1000000):
dictA = {'Java': 1, 'Python': 2, 'C++': 3}
dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
dictA.update(dictB)
def test():
for i in range (1, 1000000):
dictA = {'Java': 1, 'Python': 2, 'C++': 3}
dictB = {'Python': 3, 'C++': 4, 'Fortran': 5}
dictC = dict(itertools.chain(dictA.items(), dictB.items()))
cProfile.run('before()')
cProfile.run('after()')
cProfile.run('test()')
which is another way of performance tests in Python. You can create this method and test different executions inside it.