PyContest runs a set of functions, the contestants, for increasing sizes of N. The input is generated by calling a user function input(N). For each input, each candidate is benchmarked with warmup, run, and cooldown phases. The result is the mean runtimes of the run phase.
A timeout is set by default to 10 ms. When a candidate surpasses the timeout, it is not run for larger N. When all candidates have bailed out, the contest ends.
The winner is selected by a metric taking the mean of the runtimes divided by N for each input.
The results are plotted in three type of plots, showing:
- the runtimes
- the runtimes divided by N
- the slowdown, the runtimes divided by runtime of the winner
The timing results are also written as a table to text and CSV files, and likewise a ranking table.
Install from Github source:
pip install https://github.com/aiandit/pycontest/blob/master/requirements.txt
pip install https://github.com/aiandit/pycontest/archive/refs/heads/master.zipfrom pycontest import contest
import numpy as np
def fSumAll(l):
return np.sum(l)
def fSumDiag(l):
return np.sum(np.diag(l))
def input(N):
l = np.random.rand(N)
return (np.diag(l),),{}
cfs = [fSumAll, fSumDiag]
res = contest(*cfs, show=False, timeout=1e-3, input=input,
base=2, detail=2, start=8,
name="sum_vs_sumdiag", title="sum() vs. diag(sum())")