writeFile & readFile

[aemixdp]

What do you think about including ByteString variants of those operations from http://hackage.haskell.org/package/pipes-safe-2.2.2/docs/Pipes-Safe-Prelude.html? It will create dependency on pipes-safe here. As for me, it would be worse to put them into pipes-safe as it will get dependency on pipes-bytestring then. I can make a PR, but I'm not sure, where they should go, and whether there are some other reasons why they are not already here.

[jb55]

No need to inflict a dependency on each other. There could be a Pipes.ByteString.SafeIO library that imports both.

[Gabriella439]

Yeah, I would prefer either a separate library or for pipes-safe to depend on pipes-bytestring (but not the other way around). The pipes-safe library is the one library in the pipes ecosystem that I believe was probably the one I most poorly designed, so I try to avoid dependencies on that library from other pipes libraries.

[mitchellwrosen]

@Gabriel439 I'm curious (and always eager to learn about proper exception handling techniques) - why do you think it's poorly designed?

[Gabriella439]

@mitchellwrosen The main reason is that I feel that pipes is the wrong abstraction for resource-safety. I feel that a more restricted abstraction (like Shell from turtle, for example) is necessary to get predictable resource management guarantees and a nice user experience.

[michaelt]

I was wondering about this. Is it 'bi-directionality' that causes this problem? If so would it be possible to restrict the instances so that only regular pipes-directional Proxys come under them?

[michaelt]

readFile and writeFile are highly desirable operations, I think, if only from a tutorial perspective.

[michaelt]

I was thinking this while looking at an attempt I started to write a version of the conduit tutorial that just went through each example for all familiar streaming libraries at once, rather than just conduit. Frankly the framework dependence of the literature is demoralizing and dumb. The plan of replicating that tutorial isn't quite practical as it stands, but readFile/writeFile would be helpful.

[Gabriella439]

Bidirectionality is not the problem. Even something as simple as ListT has the same issue. The root of the problem is that if you expose the ability to request just one element then the burden of freeing things falls on the user. That's the reason why Shell works and can provide harder resource guarantees because it doesn't provide a next-like function.

[michaelt]

Okay I see people are expecting more than I anticipated under this heading. It explains the resistance of conduit to exposing something like next I suppose? - Which drives me slightly mad ... This does seem like a very disabling constraint. The absence of next and the like is closely bound up the way I feel when using conduit like I'm a captive of a framework with obscure protocols that must be learned from a master and so on. turtle has this effect too, of course, but has a restricted ambition.

[michaelt]

This does suggest that I should scrap the attempt to fit the streaming business with e.g. resourceT, no? - since there's no way I can accept giving up on the inspection of the stream. But there's still some understanding I'm missing. Is this much different from the fact that if e.g. I do rest <- runResourceT $ splitAt 10 $ readFile x, then rest is garbage and will produce an ugly exception if I try to do something with it. I don't mind this at all frankly. What survives runResourceT should be, e.g. a number or ().