Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
279 commits
Select commit Hold shift + click to select a range
09df3d1
v0.3.10
Jelleas Nov 22, 2017
371545b
importerror
Jelleas Nov 29, 2017
7710914
.11
Jelleas Nov 29, 2017
230fe4a
.12 do not let checkpy errors through
Jelleas Nov 30, 2017
bfd266e
TestError
Jelleas Dec 3, 2017
a4119ce
refactored file discovery
Jelleas Dec 3, 2017
c2d7393
support for ipynb
Jelleas Dec 3, 2017
2fc2ad1
.13
Jelleas Dec 3, 2017
ff55fd0
.14
Jelleas Dec 3, 2017
51dfe59
better error message on too many input requests
Jelleas Dec 3, 2017
a9fd3e0
.15
Jelleas Dec 3, 2017
7a8fd3d
function.py
Jelleas Dec 4, 2017
a9b07f8
.16
Jelleas Dec 4, 2017
88bf7c1
-wrapWithExceptionHandler
Jelleas Dec 4, 2017
9610f24
entities
Jelleas Dec 4, 2017
d8707ed
__init__
Jelleas Dec 4, 2017
668f0ee
.17: argv -> moduleAndOutputOf
Jelleas Dec 5, 2017
38739f2
.18
Jelleas Dec 5, 2017
ba87e9f
staticlib?
Jelleas Dec 7, 2017
04a54a0
test.fileName
Jelleas Dec 10, 2017
9a59a4c
.19
Jelleas Dec 10, 2017
67f36f1
.20 (remove magic from .ipynb)
Jelleas Dec 12, 2017
858189e
_Cache is a minimal subclass of dict()
tomkooij Jan 1, 2018
f56dc57
Refactor caches() decorator
tomkooij Jan 1, 2018
ccbde3c
Merge pull request #9 from tomkooij/refactor_caches
Jelleas Jan 2, 2018
fbff926
getFunction() supports extra args, tester sets argv
Jelleas Jan 2, 2018
36d2140
-> .21
Jelleas Jan 2, 2018
7eca7bb
Add fileName
tomkooij Jan 4, 2018
9a0b986
Merge pull request #10 from tomkooij/fix_fileName
Jelleas Jan 4, 2018
1b1308e
-> .22
Jelleas Jan 4, 2018
2e37b56
unittests for lib.py
Jelleas Jan 4, 2018
276ccd6
docs
Jelleas Jan 4, 2018
6f44080
+2
Jelleas Jan 4, 2018
a231a12
start downloader tests, bugfix in upload
Jelleas Jan 4, 2018
0ec34e7
TestList
Jelleas Jan 4, 2018
276fde9
TestClean
Jelleas Jan 4, 2018
18e3941
bugfixes downloader
Jelleas Jan 5, 2018
d019921
tester tests start
Jelleas Jan 5, 2018
7114fec
tester_test update
Jelleas Jan 5, 2018
6fac113
run_tests.py
Jelleas Jan 5, 2018
0929dec
tester.testExists
Jelleas Jan 5, 2018
dc81e71
run_tests buffers print output
Jelleas Jan 5, 2018
33ff76e
test.fileName in readme
Jelleas Jan 11, 2018
259604d
small readme update
Jelleas Jan 11, 2018
24d4acf
small readme update
Jelleas Jan 11, 2018
f50d4bd
small readme update
Jelleas Jan 11, 2018
04ba314
refactoring, lib and assertlib are packages, unify file discovery
Jelleas Jan 15, 2018
98ba542
lib package
Jelleas Jan 15, 2018
e71ec19
rm someTest
Jelleas Jan 15, 2018
df36591
tests/storage folder moved to root again
Jelleas Jan 16, 2018
0c71ac0
-.lower()
Jelleas Jan 16, 2018
2beb13f
lib.require
Jelleas Jan 16, 2018
2fdf965
moved managers into packages
Jelleas Jan 16, 2018
03fd486
require and download tests
Jelleas Jan 16, 2018
ce8f8ee
warning when more than 1 test is found
Jelleas Jan 16, 2018
ba1f459
sandbox
Jelleas Jan 18, 2018
2027303
update unittests
Jelleas Jan 18, 2018
af44d67
bugfix CHECKPYPATH
Jelleas Jan 19, 2018
239aaf6
removed Folder/File from path
Jelleas Jan 19, 2018
8559587
.DS_Store
Jelleas Jan 19, 2018
4dc0962
check whether file exists before copying
Jelleas Jan 19, 2018
9dce172
caputeInput/output + function.printOutput
Jelleas Jan 21, 2018
2ee4695
function tests
Jelleas Jan 21, 2018
d00ff3a
fixed Function.printOutput with nested calls
Jelleas Jan 22, 2018
ca0250c
start path tests
Jelleas Jan 22, 2018
8da5afc
setUp/tearDown
Jelleas Jan 22, 2018
cee479c
copyTo tests
Jelleas Jan 22, 2018
8ed1662
pathFromFolder
Jelleas Jan 22, 2018
49aeab1
path add and sub
Jelleas Jan 23, 2018
deb00c5
remove print
Jelleas Jan 23, 2018
0250d0b
refactored path
Jelleas Jan 23, 2018
b662eed
contains tests
Jelleas Jan 23, 2018
8835343
replaced path.__nonzero__ with path.__len__
Jelleas Jan 23, 2018
f947616
tests passing under windows py2/3
Jelleas Jan 27, 2018
e68359b
path string tests
Jelleas Jan 27, 2018
8b489bf
sandbox integration test
Jelleas Jan 27, 2018
ecfb963
--dev fix
Jelleas Jan 27, 2018
a1e3e44
0.4
Jelleas Jan 27, 2018
1ad743c
checkpy -register
Jelleas Jan 30, 2018
1cd1740
-downloaded
Jelleas Jan 30, 2018
961b4c5
get tests based on isTestCreator
Jelleas Jan 31, 2018
3afd381
only files ending on test.py are considered tests
Jelleas Jan 31, 2018
15d20fb
fix: sandboxing with relative filepath
Jelleas Jan 31, 2018
30ba607
docs
Jelleas Jan 31, 2018
18beebe
start docs
Jelleas Jan 31, 2018
2dfec72
docs in readme
Jelleas Jan 31, 2018
16fbc3c
intro update
Jelleas Feb 7, 2018
36c7ec5
is -> are
Jelleas Feb 7, 2018
3f0d9dd
we -> you
Jelleas Feb 7, 2018
207b81c
docs - functions
Jelleas Feb 7, 2018
98e28e9
extra doc example
Jelleas Feb 8, 2018
fa320a3
arguments in doc
Jelleas Feb 8, 2018
7bd47d0
style fixes
Jelleas Feb 8, 2018
0d49eb4
docs update
Jelleas Feb 13, 2018
7e53827
silent mode, 0.4.1
Jelleas Apr 28, 2018
46aca5d
--json
Jelleas Sep 3, 2018
6ad4f76
version++
Jelleas Sep 3, 2018
9955cb5
--json for -m
Jelleas Sep 3, 2018
64ed440
0.4.4
Jelleas Sep 26, 2018
7cc7933
passed = a boolean
Jelleas Oct 1, 2018
fb2c8f7
0.4.5
Jelleas Oct 1, 2018
7898a3f
try except finally, fix broken stdout
Jelleas Jan 22, 2019
2b03a85
version bump
Jelleas Jan 22, 2019
e193e2b
purge_tables -> drop_tables
Jelleas Sep 16, 2021
9dd88e6
file -> files, adds support for multiple checks in one run
Jelleas Oct 5, 2021
5f5b41b
--gh-auth option
Jelleas Sep 16, 2022
510b69c
0.4.9
Jelleas Sep 16, 2022
241c8d4
capture stderr as well as stdout
stgm Sep 18, 2022
9f8e90f
stderr => /dev/null when capturing stdout
Jelleas Sep 20, 2022
243a7f1
Merge pull request #20 from stgm/master
Jelleas Sep 20, 2022
9709d16
version bump
Jelleas Sep 20, 2022
be1dc5f
report wrong no of arguments without crashing
stgm Oct 3, 2022
8f6002b
Merge branch 'Jelleas:master' into master
stgm Oct 3, 2022
924e82b
neutral
stgm Oct 3, 2022
c4d147e
Merge pull request #21 from stgm/master
Jelleas Oct 3, 2022
54e209c
version bump
Jelleas Oct 3, 2022
ee91301
test attributes dont have to be functions
Jelleas May 27, 2023
4a5d02f
typo
Jelleas May 27, 2023
ee83bf8
allow for docstring Test descriptions
Jelleas Jun 12, 2023
ffa6149
priority and dependencies on testCreator level
Jelleas Jun 22, 2023
2f86fb5
testCreator(test) instead of testCreator(fileName)
Jelleas Jun 23, 2023
be5c8d7
testCreator => TestFunction
Jelleas Jun 23, 2023
0e9d310
priority is optional, test() is optional with failed/passed, failed +…
Jelleas Jun 23, 2023
0bdce9b
description & timeout changes passed to tester
Jelleas Jun 26, 2023
41f0bb1
don't reset description on timeout change
Jelleas Jun 26, 2023
b6450d7
set timeout via test/failed/passed
Jelleas Jun 26, 2023
74c4a03
testFunctions optionally take test as arg
Jelleas Jun 26, 2023
36eb441
use testFunction return value as test outcome
Jelleas Jun 26, 2023
557fa7c
filePath is optional
Jelleas Jun 26, 2023
981272d
filePath is not optional for lib.require and lib.fileExists
Jelleas Jun 27, 2023
14dc135
sandbox rework, no config yet
Jelleas Jun 27, 2023
8d17f1e
sandbox is optional, introducing only/include/exclude/require
Jelleas Jun 28, 2023
538fc5d
conditionalSandbox works on diff included/excluded
Jelleas Jun 28, 2023
588c0fe
set checkpy.file before import module
Jelleas Jul 3, 2023
366b2bb
start on from checkpy import *
Jelleas Jul 3, 2023
92212c4
type hints in test.py, introduce hide=True
Jelleas Jul 3, 2023
479f615
neutral color and straight face on skipped check
Jelleas Jul 3, 2023
d7faeb3
use pytest asserts?
Jelleas Jul 4, 2023
b2c5e64
switch to dessert, move run() to TestFunction
Jelleas Jul 4, 2023
6725729
use pytest's assertrepr_compare as default, introduced lib.explanatio…
Jelleas Jul 5, 2023
8baaba6
simplifyAssertionMessage
Jelleas Jul 5, 2023
dcae391
rm dead (py2) code
Jelleas Jul 17, 2023
d48ba87
always conditionalSandbox
Jelleas Jul 17, 2023
9a49522
tester.getTest() + tester cleanup
Jelleas Jul 17, 2023
a9487d7
tester.getTest -> tester.getActiveTest
Jelleas Jul 17, 2023
5008ab0
rm _ensureCallable, instead overwrite __setattr__
Jelleas Jul 17, 2023
e8ec434
consistent indents in explanations
Jelleas Jul 17, 2023
c8709ac
add dessert to setup.py
Jelleas Jul 17, 2023
21eaa2d
fix dangling open file in database.py
Jelleas Jul 18, 2023
3346417
reorganized checkpy.lib, added deprecationWarnings
Jelleas Jul 18, 2023
48c00ed
add getFunction, getModule, static, monkeypatch to __init__.py
Jelleas Jul 18, 2023
8647367
cleanup namespace polution
Jelleas Jul 18, 2023
ba7f0af
always ignore warnings, +patchMatplotlib, static.getFunctionCalls
Jelleas Jul 19, 2023
1c2f6e9
static.getFunctionDefinitions
Jelleas Jul 19, 2023
bb587c7
add __repr__ to Function
Jelleas Jul 20, 2023
31b7fa8
filter out: Use -v to get the full diff
Jelleas Jul 20, 2023
c0cb75a
fix IndexError on mismatched args in func def / call
Jelleas Jul 20, 2023
c2d38d2
tweak sub regex
Jelleas Jul 20, 2023
ba1953d
fix monkeypatch.documentFunction
Jelleas Jul 20, 2023
90c9617
introduce pytest.approx
Jelleas Jul 20, 2023
63cb30c
fix wrong number of args exception
Jelleas Jul 20, 2023
ffe98fb
static.getNumbersFrom
Jelleas Jul 21, 2023
4023a0b
rm dead code
Jelleas Jul 21, 2023
37855d7
rm python2 from setup.py
Jelleas Jul 21, 2023
0aa1e21
Type
Jelleas Jul 21, 2023
6e04445
add typeguard to setup.py
Jelleas Jul 21, 2023
61b62b6
experimental declarative builder + better Type assertion messages
Jelleas Jul 25, 2023
b052bea
function.stdout.stdoutRegex
Jelleas Jul 26, 2023
71516bd
FunctionBuilder docs
Jelleas Jul 26, 2023
82859ef
FunctionBuilder docs
Jelleas Jul 26, 2023
08a6488
FunctionState docs
Jelleas Jul 26, 2023
dc335a0
builder types
Jelleas Jul 26, 2023
6c6618e
import builder in __init__.py
Jelleas Jul 26, 2023
ecc1c05
builder.function('square', fileName='foo.py')
Jelleas Jul 26, 2023
87d8d44
builder.function.returnType does no assertions
Jelleas Jul 26, 2023
e74d73f
builder.function.returnType phrasing
Jelleas Jul 26, 2023
21b9f47
rm deprecated import imp
Jelleas Jul 26, 2023
dd7275e
FunctionBuilder => function, more do
Jelleas Jul 26, 2023
d81f6ea
introducedstatic.getAstNodes + static.getgetSourceOfDefinitions uses ast
Jelleas Jul 26, 2023
5851a6e
fix static __all__
Jelleas Jul 26, 2023
8468c2d
stdout() accepts Any
Jelleas Jul 26, 2023
92ec83f
getAstNodes phrasing
Jelleas Jul 26, 2023
2cc7c2a
return Self, not 'function'
Jelleas Jul 26, 2023
e6388ed
AbstractSyntaxTree
Jelleas Jul 26, 2023
ada1af1
AbstractSyntaxTree phrasing
Jelleas Jul 26, 2023
0dd3623
builder.function.do deepcopies always
Jelleas Jul 27, 2023
79a4799
builder.function, put .description block first if only name block pre…
Jelleas Jul 27, 2023
4115837
builder.function end with a celebratory message
Jelleas Jul 27, 2023
68b4bcf
phrasings
Jelleas Jul 27, 2023
6dda3a1
entities/path.py is deprecated
Jelleas Jul 27, 2023
ff961c5
typehint all the things
Jelleas Jul 28, 2023
c0315f2
tabs => spaces
Jelleas Jul 28, 2023
1a6e962
builder >> declarative, no more .build()
Jelleas Jul 28, 2023
e543457
space
Jelleas Jul 28, 2023
c76e9df
declarative.function set description back to initial description if c…
Jelleas Jul 28, 2023
273c743
dropped support for return bool, only assert
Jelleas Jul 28, 2023
1d25972
brackets are important, requests broken on 3.10? (https://github.com/…
Jelleas Aug 10, 2023
e8953b8
import requests before assert rewrite
Jelleas Aug 10, 2023
b8b34d7
declarative.function.__call__ takes an optional test
Jelleas Aug 11, 2023
d3dc476
preserve str quotes in getFunctionCallRepr
Jelleas Aug 11, 2023
6ed4cab
fix crash on just decl name check and fix initial descr
Jelleas Aug 11, 2023
f3f95d0
conditionalSandbox replaces sandbox, introduced sandbox.download
Jelleas Aug 13, 2023
2fdcd42
missing import
Jelleas Aug 13, 2023
7c3f4ed
fix hang on full pipe and crash when 'where' appears in assert
Jelleas Aug 15, 2023
2b3685d
README.rst => .md, version bump
Jelleas Aug 30, 2023
04a86b5
quotes
Jelleas Aug 30, 2023
2936d66
phrasing
Jelleas Aug 30, 2023
1d3df8d
Create python-publish.yml
Jelleas Aug 30, 2023
2fe4823
workflow with OIDC
Jelleas Aug 30, 2023
c31d566
gh-action-pypi-publish latest v
Jelleas Aug 30, 2023
c16131c
revert action back to api token
Jelleas Aug 30, 2023
9bad098
getNumbersFrom redesign, now gets all numbers from a string
Jelleas Aug 31, 2023
eda77c0
v2.0.1, fix static.AbstractSyntaxTree on py3.8
Jelleas Sep 5, 2023
e0796d7
error if test's precondition is not a TestFunction
Jelleas Sep 5, 2023
428061a
rm checks for ast.AST types v2.0.2
Jelleas Sep 12, 2023
9f9e2d9
v2.0.3, fix ValueError on single '.' in line
Jelleas Sep 18, 2023
8d20c56
v2.0.4 fix ValueError on single +/- with static.getNumbersFrom
Jelleas Sep 20, 2023
21a7d65
v2.0.5 ensure hasPassed is of type None/bool
Jelleas Sep 28, 2023
3cb9be4
v2.0.6, download calls initSandbox
Jelleas Sep 28, 2023
37b0df5
readme fix
Jelleas Sep 28, 2023
9c186fe
v2.0.7, always assert expected == real in declarative
Jelleas Sep 29, 2023
4e7cecb
reduce overhead of entities.Function, inline _captureStdout
Jelleas Oct 4, 2023
d4d549f
only capture stdout in first entities.Function call
Jelleas Oct 4, 2023
9846d37
v2.0.8, less ovearhead in function calls
Jelleas Oct 5, 2023
9697435
includeFromTests
Jelleas Oct 26, 2023
a631cab
typo
Jelleas Oct 26, 2023
3139a72
bump to v2.0.9
Jelleas Nov 23, 2023
3adccba
fix downloader.py not downloading anything but .py files
Jelleas Nov 23, 2023
afc14b9
fix include/require not overwriting exclude
Jelleas Jan 16, 2024
bd05031
catch MissingRequiredFiles from .require()
Jelleas Jan 16, 2024
f385c67
fix crash on downloading non utf-8 files
Jelleas Jan 16, 2024
3a9689d
v2.0.11
Jelleas Feb 2, 2024
2122db5
v2.0.12, checkpy.interactive.testOffline
Jelleas Mar 21, 2024
5663af7
Use importlib.metadata instead of pkg_resources.get_distribution (dep…
stgm Oct 14, 2024
5e5c9d5
Add double escape to string inside docstring
stgm Oct 14, 2024
db699c5
v2.0.13 replace re.match with re.search in declarative.function.stdou…
Jelleas Oct 17, 2024
9435793
Merge pull request #33 from stgm/escape_typos
Jelleas Feb 17, 2025
7e9926c
Merge pull request #34 from stgm/new_metadata
Jelleas Feb 17, 2025
e5533f7
2.0.14
Jelleas Feb 17, 2025
c9ffcc6
checkpy.context
Jelleas Jul 25, 2025
1cf4c36
start on checkpy.lib.io
Jelleas Jul 25, 2025
52e93a2
fix tests
Jelleas Aug 11, 2025
c894469
add output on getFunction, getModule, outputOf
Jelleas Aug 11, 2025
6030818
formatOutput
Jelleas Aug 11, 2025
3ff14d6
--output-limit
Jelleas Aug 11, 2025
538b196
bump 2.1.0
Jelleas Aug 12, 2025
8f3ebcf
Merge pull request #35 from Jelleas/log
Jelleas Aug 12, 2025
fefd96f
gh releases => commits on default branch
Jelleas Aug 14, 2025
ad90d0d
Merge pull request #36 from Jelleas/log
Jelleas Aug 14, 2025
bb017d7
fix bug multiple stdin in same check
Jelleas Aug 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed .DS_Store
Binary file not shown.
39 changes: 39 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

name: Upload Python Package

on:
release:
types: [published]

permissions:
contents: read

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build package
run: python -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
6 changes: 2 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
studentfiles
checkpy/storage/
checkpy/tests/
readme.md
*.json
.DS_Store

# Byte-compiled / optimized / DLL files
__pycache__/
Expand All @@ -21,7 +20,6 @@ dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
Expand Down Expand Up @@ -93,4 +91,4 @@ ENV/
.spyderproject

# Rope project settings
.ropeproject
.ropeproject
242 changes: 242 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
## checkpy

A Python tool for running tests on Python source files. Intended to be
used by students whom are taking courses from [Programming Lab at the UvA](https://www.proglab.nl).

Check out <https://github.com/spcourse/tests> for examples of tests.

### Installation

pip install checkpy

To download tests, run checkPy with ``-d`` as follows:

checkpy -d YOUR_GITHUB_TESTS_URL

For instance:

checkpy -d spcourse/tests

Here spcourse/tests points to <https://github.com/spcourse/tests>. You can also use the full url. This tests repository contains a test for `hello.py`. Here is how to run it:

$ echo 'print("hello world")' > hello.py
$ checkpy hello
Testing: hello.py
:( prints "Hello, world!"
assert 'hello world\n' == 'Hello, world!\n'
- hello world
? ^
+ Hello, world!
? ^ + +
:) prints exactly 1 line of output

### Usage

usage: checkpy [-h] [-module MODULE] [-download GITHUBLINK] [-register LOCALLINK] [-update] [-list] [-clean] [--dev]
[--silent] [--json] [--gh-auth GH_AUTH]
[files ...]

checkPy: a python testing framework for education. You are running Python version 3.10.6 and checkpy version 2.0.0.

positional arguments:
files names of files to be tested

options:
-h, --help show this help message and exit
-module MODULE provide a module name or path to run all tests from the module, or target a module for a
specific test
-download GITHUBLINK download tests from a Github repository and exit
-register LOCALLINK register a local folder that contains tests and exit
-update update all downloaded tests and exit
-list list all download locations and exit
-clean remove all tests from the tests folder and exit
--dev get extra information to support the development of tests
--silent do not print test results to stdout
--json return output as json, implies silent
--gh-auth GH_AUTH username:personal_access_token for authentication with GitHub.
--output-limit OUTPUTLIMIT
limit the number of characters stored for each test's output field. Default is 1000. Set to 0 to disable this limit.

To test a single file call:

checkpy YOUR_FILE_NAME

### An example

Tests in checkpy are functions with assertions. For instance:

```Py
from checkpy import *

@test()
def printsHello():
"""prints Hello, world!"""
assert outputOf() == "Hello, world!\n"
```

checkpy's `test` decorator marks the function below as a test. The docstring is a short description of the test for the student. This test does just one thing, assert that the output of the student's code matches the expected output exactly. checkpy leverages pytest's assertion rewriting to autmatically create assertion messages. For instance, a student might see the following when running this test:

$ checkpy hello
Testing: hello.py
:( prints Hello, world!
assert 'hello world\n' == 'Hello, world!\n'
- hello world
? ^
+ Hello, world!
? ^ + +

### Writing tests

Tests are discovered by filename. If you want to test a file called ``hello.py``, the corresponding test must be named ``helloTest.py``. These tests must be placed in a folder called `tests`. For instance: `tests/helloTest.py`. Tests are distributed via GitHub repositories, but for development purposes tests can also be registered locally via the `-r` flag. For instance:

mkdir tests
touch tests/helloTest.py
checkpy -r tests/helloTest.py

Once registered, checkpy will start looking in that directory for tests. Now we need a test. A test minimally consists of the following:

```Py
from checkpy import *

@test()
def printsHello():
"""prints Hello, world!"""
assert outputOf() == "Hello, world!\n"
```

A function marked as a test through checkpy's test decorator. The docstring is a short, generally one-line, description of the test for the student. Then at least one assert.

> Quick tip, use only binary expressions in assertions and keep them relatively simple for students to understand. If a binary expression is not possible, or you do not want to spoil the output, raise your own assertionError instead: ```raise AssertionError("Your program did not output the answer to the ultimate question of life, the universe, and everything")```.

While developing, you can run checkpy with the `--dev` flag to get verbose error messages and full tracebacks. So here we might do:

$ checkpy --dev hello
Testing: hello.py
:( prints "Hello, world!"
assert 'hello world\n' == 'Hello, world!\n'
- hello world
? ^
+ Hello, world!
? ^ + +
:) prints exactly 1 line of output

Check out <https://github.com/spcourse/tests> for many examples of checkpy tests.

### Short examples

#### Dependencies between tests

```Py
@test()
def exactHello():
"""prints \"Hello, world!\""""
assert outputOf() == "Hello, world!\n"

@failed(exactHello)
def oneLine():
"""prints exactly 1 line of output"""
assert outputOf().count("\n") == 1

@passed(exactHello)
def allGood():
"""Good job, everything is correct! You are ready to hand in."""
```

#### Test functions

```Py
@test()
def testSquare():
"""square(2) returns 4"""
assert getFunction("square")(4) == 4
```

#### Give hints

```Py
@test()
def testSquare():
"""square(2) returns 4"""
assert getFunction("square")(4) == 4, "did you remember to round your output?"
```

#### Handle randomness with pytest's `approx`

```Py
@test()
def testThrowDice():
"""throw() returns 7 on average"""
throw = getFunction("throw")
avg = sum(throw() for i in range(1000)) / 1000
assert avg == approx(7, abs=0.5)
```

#### Ban language constructs

```Py
import ast

@test()
def testSquare():
"""square(2) returns 4"""
assert ast.While not in static.AbstractSyntaxTree()
assert getFunction("square")(4) == 4
```

#### Check types

```Py
@test()
def testFibonacci():
"""fibonacci(10) returns the first ten fibonacci numbers"""
fibonacci = getFunction("fibonacci")
assert fibonacci(10) == Type(list[int])
assert fibonacci(10) == [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
```

#### Configure which files should be present

> Calling `only`, `include`, `exclude`, `require` or `download` has checkpy create a temporary directory to which the specified files are copied/downloaded. The tests then run in that directory.

```Py
only("sentiment.py")
download("pos_words.txt", "https://github.com/spcourse/text/raw/main/en/sentiment/pos_words.txt")
download("neg_words.txt", "https://github.com/spcourse/text/raw/main/en/sentiment/neg_words.txt")

@test()
def testPositiveSentiment():
"""recognises a positive sentence"""
...
```

#### Change the timeout

```Py
@test(timeout=60)
def exactHello():
"""prints \"Hello, world!\""""
assert outputOf() == "Hello, world!\n"
```

#### Short declarative tests

> This is a new style of tests for simple repetative use cases. Be sure to check out <https://github.com/spcourse/tests> for many more examples. For example [sentimentTest.py](https://github.com/spcourse/tests/blob/676cf5f0d2b0fbc82c7580a76b4359af273b0ca7/tests/text/sentimentTest.py)

```Py
correctForPos = test()(declarative
.function("sentiment_of_text")
.params("text")
.returnType(int)
.call("Coronet has the best lines of all day cruisers.")
.returns(1)
.description("recognises a positive sentence")
)
```

### Distributing tests

checkpy downloads tests directly from Github repos. The requirement is that a folder called ``tests`` exists within the repo that contains only tests and folders (which checkpy treats as modules). checkpy will pull from the default branch. To download tests call checkpy with the optional ``-d`` argument and pass your github repo url. checkpy will automatically keep tests up to date by checking for any new commits on GitHub.

### Testing checkpy

python3 run_tests.py
Loading