Journal

Journal

Here is my tear soaked journal with all my secret confessions (about Packet).

Sun Jul 3 15:34:15 UTC 2011

Alternation was first considered for reading. Now I'm considering how to write alternation.

Alternation will occur with the frequency that unions occur in C code. Actually, less freqently, because the alteration is supposed to cover only value. How would you provide a callback with results if alternatives had different numbers of fields? One could use constants to ensure that the matches are equal, and check the constants to see what you recieved.

But, truly different results are going to require a decision in the client code, not in the parser pattern. If you have a switch that would produce an entirely different structure, you probably want to handle that separately.

Or else, you can iterate through the returned arguments, using a constant as a switch to determine what to do at a point of alternation. With CoffeeScript variable arguments, this is easy to express.

With named parameters too, you could end up with a means to switch between different results.

Writing doesn't make sense without reading.

Reading sips a byte or bytes to determine what pattern to apply at the current point in the stream. Reading says, sip a byte, inspect the byte to determine what the data exists at this point it the stream, then put the word bytes and apply a pattern to extract the data.

But, there is no need to specify a sip pattern for writing. The sipping is moot.

With named parameters, you could use the presence or absense of the named parameter in the given map to determine a route for writing.

b8(0-251: b8 | 252/252-0xffff: x8{252}, b16 | 253/0x10000-0xffffff: x8{253}, b24 | 254: x8{254}, b64)
b8(0-251: b8 | 252/0-0xffff: x8{252}, b16 | 253/0-0xffffff: x8{253}, b24 | 254: x8{254}, b64)
b8(&0x80: b16{x1,b15} | b8)/(0-0x7f: b8 | b16{x1,b15})

We need to come to a close with Packet, so we don't want to bulk it up with intensely complicated logic. We can think harder about named elements in future. I've not encountered a structure to parse that is wildly different. The parser is meant to be invoked in stages, reset for different segements, so you don't wnat to build too much logic into this.

Thu Jun 30 05:18:34 UTC 2011

I'm starting to feel like I want to break up packet.coffee into separate modules for the Parser, Serializer, Structure and the default transforms.

Mon Apr 25 03:20:35 CDT 2011

A lot of progress implementing a tar file parser and serializer. Can see how to make them not only streamable, but pausable, but for now I'm considering some pattern language additions to make parsers simpiler.

Primarily, I'd like to be able to build an object instead of calling a call back with the parameters splayed all over the local namespace. As a parser developer the first thing I need to do is gather up all the values into a header for the tar parser user. In the serializers, there is a strange step of exploading the header and then passing it in. I'm repeating myself. It doesn't feel right.

I'm implementing checksums now. For the USTAR format, the header checksum is in the middle of the header, so you need to build the header, then double back and add the checksum, all before streaming. That means I need a buffer, plus a way to write things out, so part of structure or a serializer.

I've made Serializer streaming, but with a switch, because otherwise I'd have to rewrite all of the Serializer tests. I'll have to get past Structure to see if I want to force it all to be one way. If I really do want buffers at times, instead of streams, then I'll want to keep the current API.

Things to do fix:

• The parser or serializer itself is not a parameter to the callback unless the user makes it one.
• The bytesWritten is only for bytes writen, those that go out with a data event.

Thu Apr 21 03:33:35 CDT 2011

About to post an issue on streams, but then thought better of it. Better to journal it. There is no documentation on how to implement pause and resume. In fact, it took a little doing to find the dicussion where pause and resume where introduced.

Pause is meant to send a message to the source of the data, the object with the actual file handle, to stop reading from the file handle, because the destination is returning false on write, meaning that it has no buffer.

Doctor it hurts when I do this. Don't do that.

Stop reading from the file. In the case of a network connection, you're going to back things up all the way to client, but as noted in the discussion, if you have no more I/O buffer memory, it's pretty rough going anyway.

There need to be no more data events after pause. The deliberators feel that it is up to the the last item to do any additional buffering, to hold onto the buffer it couldn't write, but that a message to stop feeding the pipe goes back through the pipe to the source.

node-tar requires streams. I can see how this is something that will occur time and again, and Packet parsers will become trivial to implement if Packet offers a ready implementation of WritableStream and ReadableStream for stream contents. The weight might be felt, but it might not. Most of the functionality it built into Stream itself.