Comment handling in Janet

[jmckernon]

Symex makes some efforts to handle comments smartly, but this doesn't currently work for Janet, which uses # to begin comments (which run to the end of the line). For instance, in the following buffer:

(first sexp)

# comment for second sexp
(second sexp)

Deleting the first sexp with symex will delete the comment, too. (Whereas this seems to work if the comment is a semicolon comment.)

Overriding the following functions appears to fix this issue:

(defun symex-comment-line-p ()
  "Check if we're currently at the start of a comment line."
  (save-excursion
    (back-to-indentation)
    (or (looking-at-p ";") (looking-at-p "#"))))

(defun symex-inline-comment-p ()
  "Check if there is an inline comment following point."
  (looking-at-p "[[:space:]]*[;#]"))

However, I think these changes, as they are, would probably break things for other lisps. Not a problem for me as I largely use Janet, but I wouldn't want to put them into the code base, without some mechanism to check conditionally what the currently comment character is or similar, which is beyond my knowledge.

There may also be other functions which would need amending but these are the ones I found for now.

[countvajhula]

Yeah, I agree that specifically looking for these characters may not translate well across languages. A more robust way to do that could potentially be to consult the buffer's syntax table.

Each buffer has an associated syntax table that is typically defined, IIUC, by the major mode. This syntax table specifies the syntactic role of specific characters in the buffer. So for instance, Emacs Lisp major mode would define ; to be a comment character in the syntax table, whereas janet-mode (and, e.g., python mode) would likely define # as a comment character. You can see the current buffer's syntax table using C-h s (M-x describe-syntax).

Instead of directly looking for a ; character, we could potentially consult the syntax table to find out which characters are considered comment characters (perhaps only worrying about single line comments), and look for that character instead. I haven't worked with syntax tables before but I believe they have some specific conventions around how to store and retrieve values from them that make them different from ordinary hash tables. If we can find a way to lookup something like, "what characters in the syntax table have syntax class 'comment'", then I think that would be very promising.

[jmckernon]

I don't know how these things are typically handled in emacs but consulting the syntax table sounds good. It also sounds as if it could potentially help with another issue I was considering reporting, to do with prefix characters for sexps. These are widely used for quotation and disquotation but there are some more unique ones in Janet. ' for quotation (as usual), ~ for quasiquotation, , for disquotation, and ; for splicing. @ is used for marking mutability of data structures (@[] and @{}). | is used for a short lambda notation (|(+ $ $)). Think that's all.

I have discovered symex-quote/unquote-prefix-list but i don't think they affect how prefixes are handled when manipulating sexps.

I will try and investigate consulting the syntax table myself and will share if I make any progress.

Thanks!

[countvajhula]

Though it seems relatively straightforward for comments, I'm not as confident of the use of the syntax table for handling prefixes (though I look forward to your findings).

It also sounds as if it could potentially help with another issue I was considering reporting, to do with prefix characters for sexps. These are widely used for quotation and disquotation but there are some more unique ones in Janet. ' for quotation (as usual), ~ for quasiquotation, , for disquotation, and ; for splicing. @ is used for marking mutability of data structures (@[] and @{}). | is used for a short lambda notation (|(+ $ $)). Think that's all.

Currently, for movement purposes (e.g., entering and leaving symexes via j and k) we use a fairly crude approach to parsing these prefixes, essentially hardcoding each special form like #'(...) and so on, along with their lengths (to determine how many characters forwards or backwards we need to move by upon entering or leaving the expression!).

If the syntax table could be used instead, that would be great.

I have discovered symex-quote/unquote-prefix-list but i don't think they affect how prefixes are handled when manipulating sexps.

That's also an interesting idea, and may be easier, depending on how complicated reading the syntax table for prefixes ends up being.

Currently, as you noticed, these quote/unquote lists are used only in the cycle-prefix feature. In the implementation of that feature, we do some generic things like identifying the longest matching prefix from that list, which could be useful more generally for parsing. That is, instead of hardcoding each prefix case in the parsing functions, we could potentially reuse the longest matching prefix logic + configured prefix quote/unquote lists, in some form.