refactor: update test-framework.md

src/test-framework.md

@@ -1,69 +1,105 @@
 # Test Framework
 
-Flix comes with a simple built-in test framework. 
-
-A test is a Flix function marked with the `@Test` annotation. That's it. 
-
-A test function can return any value. If it returns a Bool then `true` is interpreted as success and `false` as failure. Any non-Boolean value is interpreted as success. 
-
-The `Assert.eq` function can be used to test for equality between two values that implement the `Eq` and `ToString` traits. The advantage of `Assert.eq` (over `==`) is that it will print the two values if they are unequal. The `Assert.eq` function should not be used outside of unit tests.
+Flix comes with a built-in test framework. A test is a Flix function marked with
+the `@Test` annotation. A test function must take no arguments and return
+`Unit`.
+
+The `Assert` module provides assertion functions for testing. Here are the most commonly used:
+
+| Function                                       | Purpose                              |
+|------------------------------------------------|--------------------------------------|
+| `Assert.assertEq(expected = value, actual)`    | Assert equality between values       |
+| `Assert.assertNeq(unexpected = value, actual)` | Assert inequality between values     |
+| `Assert.assertTrue(cond)`                      | Assert condition is true             |
+| `Assert.assertFalse(cond)`                     | Assert condition is false            |
+| `Assert.assertSome(opt)`                       | Assert Option is Some                |
+| `Assert.assertNone(opt)`                       | Assert Option is None                |
+| `Assert.assertOk(res)`                         | Assert Result is Ok                  |
+| `Assert.assertErr(res)`                        | Assert Result is Err                 |
+| `Assert.assertEmpty(coll)`                     | Assert collection is empty           |
+| `Assert.assertMemberOf(x, coll)`               | Assert element is in collection      |
+| `Assert.fail(msg)`                             | Unconditionally fail with message    |
+| `Assert.success(msg)`                          | Unconditionally succeed with message |
+
+The `assertEq` and `assertNeq` functions require a labelled argument `expected` / `unexpected`.
 
 Here is an example:
 
 ```flix
+use Assert.{assertEq, assertTrue, assertFalse, assertOk, assertErr}
+
 def add(x: Int32, y: Int32): Int32 = x + y
 
+def isEven(x: Int32): Bool = Int32.modulo(x, 2) == 0
+
+def safeDivide(x: Int32, y: Int32): Result[String, Int32] =
+    if (y == 0) Err("Division by zero") else Ok(x / y)
+
 @Test
-def testAdd01(): Bool = 0 == add(0, 0)
+def testAdd01(): Unit \ Assert =
+    assertEq(expected = 5, add(2, 3))
 
 @Test
-def testAdd02(): Bool = Assert.eq(1, add(0, 1))
+def testIsEven01(): Unit \ Assert =
+    assertTrue(isEven(4))
 
 @Test
-def testAdd03(): Bool = Assert.eq(2, add(1, 1))
+def testIsEven02(): Unit \ Assert =
+    assertFalse(isEven(3))
 
 @Test
-def testAdd04(): Bool = Assert.eq(4, add(1, 2))
+def testSafeDivide01(): Unit \ Assert =
+    assertOk(safeDivide(10, 2))
 
-@Test @Skip
-def testAdd05(): Bool = Assert.eq(8, add(2, 3))
+@Test
+def testSafeDivide02(): Unit \ Assert =
+    assertErr(safeDivide(10, 0))
 ```
 
-Running the tests (e.g. with the command `test`) yields:
+Running the tests (e.g. with `flix test`) yields:
 
 ```
 Running 5 tests...
 
-   PASS  testAdd01 237,3us
-   PASS  testAdd02 21,1us
-   PASS  testAdd03 10,3us
-   FAIL  testAdd04 (Assertion Error)
-   SKIP  testAdd05 (SKIPPED)
-
---------------------------------------------------------------------------------
-
-   FAIL  testAdd04
-    Assertion Error
-      Expected: 4
-      Actual:   3
-
-    dev.flix.runtime.HoleError: Hole '?Assert.assertEq' at Assert.flix:32:13    
-        at Assert.Def%eq%174731.invoke(Unknown Source)
-        at Cont%Bool.unwind(Cont%Bool)
-        at Ns.m_testAdd04(Unknown Source)
-        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
-        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
-        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
-        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
-        at ca.uwaterloo.flix.language.phase.jvm.JvmBackend$.$anonfun$link$1(JvmBackend.scala:286)
-        at ca.uwaterloo.flix.language.phase.jvm.JvmBackend$.$anonfun$getCompiledDefs$2(JvmBackend.scala:259)
-        at ca.uwaterloo.flix.tools.Tester$TestRunner.runTest(Tester.scala:182)
-        at ca.uwaterloo.flix.tools.Tester$TestRunner.$anonfun$run$7(Tester.scala:153)
-        at ca.uwaterloo.flix.tools.Tester$TestRunner.$anonfun$run$7$adapted(Tester.scala:152)
-        at scala.collection.immutable.Vector.foreach(Vector.scala:1856)
-        at ca.uwaterloo.flix.tools.Tester$TestRunner.run(Tester.scala:152)
-
---------------------------------------------------------------------------------
-
-Passed: 3, Failed: 1. Skipped: 1. Elapsed: 3,0ms.
+   PASS  testAdd01 1,4ms
+   PASS  testIsEven01 312,5us
+   PASS  testIsEven02 229,8us
+   PASS  testSafeDivide01 366,0us
+   PASS  testSafeDivide02 299,7us
+
+Passed: 5, Failed: 0. Skipped: 0. Elapsed: 3,8ms.
+```
+
+## Assertions with Custom Messages
+
+Most assertions have `WithMsg` variants for custom error messages.
+
+```flix
+use Assert.{assertEqWithMsg, assertTrueWithMsg, assertFalseWithMsg}
+
+@Test
+def testAdd01(): Unit \ Assert =
+    assertEqWithMsg(expected = 5, add(2, 3), "addition should work")
+
+@Test
+def testIsEven01(): Unit \ Assert =
+    assertTrueWithMsg(isEven(4), "4 should be even")
+
+@Test
+def testIsEven02(): Unit \ Assert =
+    assertFalseWithMsg(isEven(3), "3 should be odd")
 ```
+
+## `@Test` Function Signatures
+
+A function marked with `@Test` must have one of the following signatures:
+
+```flix
+@Test
+def test01(): Unit = ...
+def test02(): Unit \ Assert = ...
+def test03(): Unit \ Assert + IO = ...
+```
+
+In addition, a `@Test` function may use any algebraic effect for which there is
+a `@DefaultHandler`.