Comment handling

[wz2b]

Comments are part of the line protocol specification, but the decoder doesn't give you a way to access them. I'm not sure how the decoder here handles comments (if it handles them at all) but it's part of the line protocol specification I use so I'd like to request that some thought be given to how to handle them.

[rogpeppe]

The decoder here does handle comments but it doesn't provide a way to see the contents of them via the API.
Is that what you'd like to do?

[wz2b]

The decoder here does handle comments but it doesn't provide a way to see the contents of them via the API.
Is that what you'd like to do?

That's right, mainly so that they can be captured and logged - at least when in --debug.

[rogpeppe]

Thanks for the feedback. I'll have to think hard about how the API might look. I'd hope to be able to do this without compromising on performance. I can't say when we might get around to working on this, I'm afraid!

[wz2b]

I should explain that my motivation for this request was influxdata/telegraf#9487 which has sparked a lot of conversation/discussion on the slack #telegraf-dev channel. There was mixed reaction to my request to have comments have special meaning. An argument against doing this is that to be consistent you'd need similar comment handling in the other input plugins as well. An alternative that we discussed was to just tell external plugin developers to use os.Stderr. That's fine but as my feature request explains there's still a big gap in interpreting these messages, rather than just assuming they're opaque strings with a severity of Error (I strongly disagree with that).

The current line parser, based on Ragel, actually let me inject a handler for comments by modifying the ragel and go. The next step in my effort was going to be to make a change to telegraf itself to treat these comments as log messages, initially the same way telegraf handles any log message but eventually making it handle them better. I had gotten to a point where this seemed feasible when I learned that the line parser was being completely rewritten so I stopped working on it.

In general, I still think it's useful to have the ability to receive these comment events from the parser, even if we're undecided what you'd do with them at the moment. But the line parser is a library, which can be used in many ways by applications outside of just telegraf. Furthermore, a comment is part of the line protocol spec. Based on these two things, I am still going to argue that it would be a really good thing to be able to dispatch comments as events.

I apologize for being so wordy. It's a little bit of a complicated story and I was not sure how to say all of these things in a few sentences and not lose the nuances.