Bugfix // Streaming endpoint changes to match specification#48
Bugfix // Streaming endpoint changes to match specification#48stuartjohnpage wants to merge 3 commits intomainfrom
Conversation
stuartjohnpage
requested review from
estreeper,
grossvogel,
jpena925 and
pjdiiori
| |> String.split("\n\n") | ||
| |> Enum.map(fn event -> | ||
| event | ||
| |> String.split("\n") | ||
| |> Enum.filter(fn line -> String.starts_with?(line, "data: ") end) | ||
| |> Enum.map_join("\n", fn line -> String.trim_leading(line, "data: ") end) | ||
| end) | ||
| |> Enum.filter(fn data -> data != "" end) |
There was a problem hiding this comment.
I'm not following how this fixes the problem... not saying it doesn't, I'm just confused. 🫤 Seems like it'd end up being the same in the end since we're splitting on the \n in the Enum.map. is the map_join adding back the newlines that were previously split?
There was a problem hiding this comment.
ye, exactly!
In the old code, you'd get a flat list of independent lines, losing the relationship between lines that were part of the same event.
eg:
data: Line 1
data: Line 2
data: Another event
originally would return: ["Line 1", "Line 2", "Another event"]
but now returns: ["Line 1\nLine 2", "Another event"]
The difference is that in the new approach, lines within the same event stay together with their formatting preserved
we have to split on the \n\n (the event delimited). Then we have each event. Then we want to grab the data bits, while preserving the new lines
|
I'm actually going to wrap this up when I make the new SDK for Revelry.AI 👍 |
2 participants
We recently fixed the prodops API to stream correctly according to the SSE.
This should now handle the splitting of those properly sent new lines correctly.