TECHNICAL.md

validns technical notes

Goals / development milestones / features for validns:

1. Requirements for an initial public release (missing functionality/doc)

Task/feature/functionality % done Descr

• understand all standard rdtypes 80 - currently missing: AFSDB, APL, CERT, DHCID, DLV, DNAME, HIP, IPSECKEY, KEY, KX, SIG, SPF, TA, TKEY

• initial user documentation 30

2. Performance and other non-critical enhancements

• speed up signature verification 0 - the initial parsing cannot (and possibly other operations) be easily parallelized, via using multiple threads but signature checks can

• add an incremental checks mode 0 - store hashes of succesfully (do not do expensive verified records verifications which were done previously, provided the records did not change)

3. Nice to have features, for post-release

• user-defined policy checks via 0 - lua API shall provide lua embedding (split out convenient means to access syntactical and policy validation) and search records, so that policy checks involving relationships between records can be implemented by the user

• speed up signature verification 0 - requires significan via GPU crypto offload amount of experimentation

Data structures considerations

• the whole parsed zone must be loaded into memory
• some validations work on individual records
• thus, whole zone traversal is needed
• some validations work on records sorted in a particular way
  • the "canonical order" described here http://tools.ietf.org/html/rfc4034#section-6.1
  • thus, this traversal should be in this canonical order
  • Judy is a good way to quickly find and iterate over string-indexed data
  • but it uses normal lexicographic sort order
  • is it possible to map the names in such a way that the result, sorted lexicographically, will correspond to the canonical order?
  • if we agree that labels cannot contain chr(0)
    • this, strictly speaking, is possible
    • but we ignore that this can be seen in practice
  • and, if we agree that labels cannot contain chr(1)
    • same as above - possible, but it's not within "IN" class
  • then we can reverse the name and use chr(1) as the label separator
  • we could just use chr(0) as the label separator, but then we cannot use normal C-style strings, so the code will be somewhat more complex
• some validations apply to given names
• need quick retrieval of all records with a given name
• some validations require complete RR sets
• need quick retrieval of all records in a given RR set

Memory requirements and execution speed

Naturally, memory usage is much higher on 64-bit platforms. For a 4 million records zone, it eats around 700 MB on a 64-bit platform, and only around 400 MB on a 32-bit platform.

It also looks that 32-bit version is somewhat faster than 64-bit one, although I did not do a strict comparison - the tested machines were not the same.

TODO

The todo list is not complete by its nature.

• proper manual page
• a test for every error message
• zone validations specified in RFC 1035
• multiple verboseness levels (-v option repeated)
• include file support
  • -I option
• embedding lua for flexible validations
  • "policy validations"
  • -p policy-file option
  • -r policy-rule option (maybe?)
• better platform support
  • stpcpy() might not be everywhere

DONE

The done list is not complete.

• (./) usage() function
• (./) options support (getopt)
• (./) $TTL support
• (./) $ORIGIN support
  • (./) -z option for initial ORIGIN
• (./) master file support (RFC 1035, section 5)
  • (./) see whether there were changes to it
• -v option for verbose
• (./) -q option for extra quiet
• (./) -f option (die on first error)
• (./) -s option - produce validation summary/statistics
• (./) nice CPAN module for external programs output testing?
  • (./) looks like Test::Command::Simple is what I want
• (./) wire RDATA format
• (./) NSEC3 parsing