Harness the Power of Python timeit to Optimize Your Code

Hey there! Have you ever wondered how long a block of code takes to run? Or wanted to compare the speeds of different implementations? As Python developers, timing and profiling our code is critical to writing optimized programs.

This comprehensive guide will teach you how to become a Python timing expert using the incredible timeit module!

I‘ll share my top tips and code examples so you can profile your code like a pro. Let‘s get started, friend!

Why Timing Code Matters

Before we jump into the timeit syntax, let‘s first discuss why timing code is so important:

Spot bottlenecks: Timing helps pinpoint slow sections of code that are dragging down performance. You can then focus your optimization efforts on these hot spots.
Compare implementations: Ever debated between a for loop vs list comprehension? Timeit provides concrete speed data to help choose the faster option.
Tune algorithms: Coming up with algorithmic improvements? Use timeit to validate if your new approach actually executes faster.
Identify regressions: You can use timeit to catch sudden performance drops during code changes. Nip regressions in the bud!
Diagnose issues: Changes in execution time can hint at problems like infinite recursion or runaway processes consuming CPU.

As you can see, timing unlocks optimization opportunities at every stage of development. Even basic profiling can provide huge speed boosts.

According to Python experts, a 100x speedup is achievable through optimizations like algorithms, data structures, and micro-optimizations. Timeit helps you get there!

Now let‘s master this game-changing tool.

timeit 101: Key Concepts

The timeit module provides a simple timeit.timeit() API for timing code. Here are the key parameters:

timeit.timeit(stmt, setup, timer, number)

stmt: Required. The code snippet to time, as a string or callable.
setup: Optional. Initialization code to execute once per loop.
timer: Optional. Custom timer function, default is time.perf_counter().
number: Optional. Number of executions, default 1 million.

The statement you want to time is passed via stmt. Use setup for any imports or initializing variables.

Timeit executes stmt for number of times and returns the total duration. The default number is 1 million which gives good accuracy.

Timing Python Expressions

Let‘s start by timing some simple Python expressions:

timeit.timeit(‘"2**10"‘)

This executes 2**10 a million times, taking about 0.06 seconds total on my PC.

We can pass a larger number for better precision with fast statements:

timeit.timeit(‘"min([1,2,3])"‘, number=1_000_000)

Executing min() a million times takes around 0.25 seconds.

You can also time multiple expressions in one call:

timeit.timeit(‘"3**4; 7*9"‘, number=100_000)

This runs 3**4 and 7*9 100,000 times each, in around 0.01 seconds total.

Command Line Usage

You can easily use timeit from the terminal too:

python -m timeit -n 100000 ‘"2**10"‘

It executes the statement 100,000 times and prints the time taken:

100000 loops, best of 5: 4.08 nsec per loop

The -s option lets you specify setup code:

python -m timeit -s "vals = range(100)" "sum(vals)"

This initializes vals and times sum(vals). Convenient for testing snippets!

Timing Code Blocks

In addition to expressions, you can time entire code blocks using timeit.

Pass the block as a multiline string into stmt.

Let‘s time a simple function:

stmt = ‘‘‘
def sum_squares(nums):
    total = 0 
    for x in nums:
        total += x*x
    return total
‘‘‘

print(timeit.timeit(stmt, number=100_000))

Executing sum_squares() 100,000 times takes around 0.11 seconds on my PC.

For larger blocks, define functions in setup and call them from stmt:

setup = ‘‘‘ 
def sum_squares(nums):
    # Function body
‘‘‘

stmt = ‘sum_squares(nums)‘

print(timeit.timeit(stmt, setup, number=100_000))

This avoids redefining sum_squares() on each loop.

Comparing Alternate Implementations

One of timeit‘s most useful applications is comparing different implementations.

Let‘s look at two approaches to check if a list contains any odd numbers:

# Loop based
def has_odd_v1(lst):
    for n in lst:
        if n % 2 != 0:
            return True

    return False


# Using any()
def has_odd_v2(lst):
    return any(num % 2 != 0 for num in lst)

We can time these functions to identify the faster approach:

print(timeit.timeit(‘has_odd_v1(l)‘, ‘l=range(100)‘, number=100_000))
print(timeit.timeit(‘has_odd_v2(l)‘, ‘l=range(100)‘, number=100_000))

On my machine, has_odd_v2 takes around 0.26 seconds while has_odd_v1 takes 0.8 seconds. So the any() version is 3x faster!

Benchmarking Across Inputs

Another useful technique is checking execution time for different input sizes.

Let‘s see how sum() scales from a small to large list:

for i in [10, 100, 1000, 10000]:
  t = timeit.timeit(f‘sum(l)‘, f‘l=range({i})‘, number=1000)
  print(f‘len: {i}  time: {t}‘)

len: 10   time: 5.00e-06  
len: 100  time: 2.52e-05
len: 1000 time: 0.0002038
len: 10000 time: 0.0035067

As expected, time taken grows linearly with input size due to O(n) complexity.

These benchmarks help estimate real-world performance across different workloads.

Optimizing Execution Time

Once you‘ve identified slow areas using timeit, here are some optimization techniques:

Algorithms: Explore faster algorithms like binary search instead of linear search.
Data structures: Use efficient structures like set over list when suitable.
Vectorization: Leverage vectorized operations like NumPy instead of Python loops.
Caching: Add memoization to avoid repeated computations.
Parallelism: Use multiprocessing to distribute work across CPU cores.
Code changes: Simplify logic, remove redundancies, optimize expensive inner loops, etc.
Profiling: Drill-down using cProfile to identify specific hot functions and lines.

With a 100-1000x speedup achievable through optimizations, timing your code is time well spent!

Best Practices for Robust Benchmarking

Here are some tips to ensure accurate and reliable timings using timeit:

Run each statement at least 100,000 times, or even 1 million times for fast functions. This reduces noise.
Use repeat() and run each test 5-10 times. Pick the best time to minimize variability.
Test across different input sizes/params to check scaling. Time with both small and large inputs.
For system calls, measure wall time instead of CPU time which can vary.
Use the same hardware and Python version for comparisons between tests.
Isolate timing from external factors like network I/O, disk access, etc.

Following these practices will result in stable, reproducible measurements from timeit.

Wrapping Up

Let‘s recap what we‘ve learned:

timeit.timeit() provides an easy way to time Python code snippets.
Use appropriate number of runs and repeat() to get robust timings.
Leverage timeit for microbenchmarking, optimization, diagnostics.
Time both simple expressions as well as larger blocks of code.
Compare implementations and algorithmic changes to pick faster alternatives.
Profile code at different input sizes to detect performance issues.

I hope these tips help you become an expert at timing Python code! Optimizing speed can be an iterative process, so integrate timeit into your development workflow.

Happy coding! Let me know if you have any other questions.