Problem packages

Why problem packages?

Problem packages are a standardized way to add algorithmic problems to IdLearn. They are saved in the /home/user/.idlearn/problems/ directory and the user can edit these files. This allows for modding and exchange of custom problems between users.

Problem package structure

The problem package is a directory with the following recommended structure:

packageDirectoryName
├── config.yml
├── makefile.in
├── makefile
├── doc
│   ├── statement.txt
│   ├── example1.in
│   ├── example1.out
│   ├── example2.in
│   └── example2.out
└── prog
    ├── generator.cpp
    ├── model.cpp
    └── checker.cpp

Description of components

• config.yml contains important metadata like the title, difficulty, path to the statement and files containing examples, names of files containing input and output and test configuration. All very important and mandatory.
• makefile.in provides various filenames for the scripts. This is entirely modifiable without any negative effects on the application.
• makefile provides scripts for convenient building of executables or auxiliary files, the names of which are in makefile.in.
• doc contains files related to the presentation of the problem: the statement, and an example input and output, although the existence of this folder isn't necessary and is only for structural organisation.
• prog contains files related to testing submitted solutions. Similar to doc, this folder is not necessary.
  • generator takes one argument, a natural number n and outputs the n-th test for that problem to stdout. It is recommended that the tests be generated pseudo-randomly.
  • model reads the input of a test from stdin and outputs useful information to stdout. "Useful information" may be either the correct output to that test, if it's unambiguously defined, or data which will help verify the correctness of the user-submitted program in some other way, if there may be many correct outputs.
  • checker reads the input, user output, and model output from files and decides whether the user-submitted solution is correct on the given test, or not, outputting precisely "OK\n" if and only if the output is correct.

Technical specification

For modifiability purposes, in order to ensure correct application operation and the highest possible efficiency, the following restrictions on user-created problem packages are put in place:

1. It is currently only required that config.yml and makefile are present in the package.

2. The config.yml must provide the following information:

   • The title of the package
   • The estimated difficulty of the problem (Easy/Medium/Hard)
   • Path to the statement
   • Prefix of the path to the input files (testInput)
   • Prefix of the path to the output files (userOutput)
   • IDs of tests to be run (testData)
   • The expectedTime (run your own solution with expected time 1.00, check the result and put that in instead), and expectedMemory usage.

   And optionally

   • A list of examples, each consisting of a path to the input and the output files for that example.

3. The makefile must provide the following goals:

   • make clean must result in a zero exit code (no additional requirements).
   • make all must build all necessary executables.
   • make input TestID=X must create a file called testInputX where testInput is the same path as the one in config.yml containing the input to the X-th test in the problem package (usually this means executing generator X).
   • make modelOut TestID=X must create the output of the model program for testInputX.
   • make check TestID=X -s must use the user output saved in file userOutputX where userOutput is the same path as the one in config.yml, to check the correctness of the solution, outputting the string "OK\n" (without quotation marks) to standard output if and only if the contents of userOutputX are correct for the given testInputX and model output.

4. The test input and output prefix file paths must be consistent between config.yml and makefile (or makefile.in).

5. For purposes of concurrent testing, no files in the package named testInputX or userOutputX, where testInput and userOutput are the same as in config.yml and X is a positive integer, may contain information important for the correct functioning of the package.