Python 3 profile method in isolation

Very often there are performance problems or software needs tunning. In Python you can do profiling of a method isolated from the rest of the code. This can be done by using modules for profiling and pickles:

• cProfile
• pickle
  The example below which I'm using when I want to microbenchmark my code (even in 2018) and find what is causing the performance issues:

first you can get all variable values for your method by this code:

# Saving the objects:
var1, var2, var3, var4 =0
with open('/home/user/Soft/objs.pkl', 'w') as f:  # Python 3: open(..., 'wb')
    pickle.dump([var1, var2, var3, var4], f)

Then use this code and paste your method code in the three method: before, after and test:

import pickle
import cProfile

# Getting back the objects:
with open('/home/user/Soft/objs.pkl', 'rb') as f:  # Python 3: open(..., 'rb')
        var1, var2, var3, var4 = pickle.load(f)

# before
def before():
    print('before')

# after
def after():
    print('after')


def test():
    print('test')

cProfile.run('test()')

In this way you can get a method with all related variables and execute profiling on this method. I'm copying the method in the tree methods:

• before
• after
• test

Then I start with the changes and measuring the time and the call invocations.

A really simple example for testing range performance, list and sets:

import pickle
import cProfile



# before
def before():
    ran = range(0, 10000000)
    for i in range(1, 10):
        if i in ran:
            x = 1

# after
def after():
    ran = set(range(0, 10000000))
    for i in range (1, 10):
        if i in ran:
            x = 1


def test():
    print('test')

# test2()
cProfile.run('before()')
cProfile.run('after()')

the result of this execution is:

         4 function calls in 0.000 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 ProfilingIsolation.py:7(before)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}


         4 function calls in 0.698 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.116    0.116    0.698    0.698 <string>:1(<module>)
        1    0.582    0.582    0.582    0.582 ProfilingIsolation.py:14(after)
        1    0.000    0.000    0.698    0.698 {built-in method builtins.exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

if you need variables you can store them with pickles. For example big maps, objects or any other parameter of the method.