Gzip will cause react-dom-stream to not stream content

[geekyme]

Tried this out on a test site, I don't see the content being streamed down properly cuz if it did, you would see main-xxx.css start downloading before for-him/ finish downloading.

Am I missing some kind of proper encoding?

[aickin]

Look at the response headers in your browser dev tools. If there's a header called Content-Length, then it's not being streamed. If there's a header called Transfer-Encoding with a value of chunked, then it is being streamed.

If the response is not being streamed, there are two usual culprits: middleware and proxies.

• Middleware, particularly things like gzip, can potentially buffer the responses. I think most middleware can be tuned to buffer less, but it depends what middleware you are using.
• Proxies or CDNs are allowed to turn streaming results into non-streaming, but that's not super common these days in my experience.

[geekyme]

this is awkward. I dont see any Content-Length header. There's a Transfer-Encoding with value of chunked

[geekyme]

You are right. It's gzip that is causing streaming to not work.

Removing the gzip middleware will fix the issue. It's too bad though, gzip is useful. :(

[geekyme]

Renamed the issue title so others may find this useful

[aickin]

Did you use compression?

There are a lot of bug reports of people having a hard time getting compression to stream correctly, but their continuous integration tests show that streaming works with it. I've been poking at it for the last hour or so and have been having a hard time getting it to do the right thing; it could be my code's fault. I'l reopen as a tracking bug to make it work.

[geekyme]

Yes i use compression

Regardless, my servers is fronted by a load balancer which will also gzip content. If either the load balancer or compression is running gzip, then my streaming will not work.

http://stackoverflow.com/questions/5280633/gzip-compression-of-chunked-encoding-response

You gzip the content, and only then apply the chunked encoding:

"Since "chunked" is the only transfer-coding required to be understood by HTTP/1.1 recipients, it plays a crucial role in delimiting messages on a persistent connection. Whenever a transfer-coding is applied to a payload body in a request, the final transfer-coding applied MUST be "chunked". If a transfer-coding is applied to a response payload body, then either the final transfer-coding applied MUST be "chunked" or the message MUST be terminated by closing the connection. When the "chunked" transfer-coding is used, it MUST be the last transfer-coding applied to form the message-body. The "chunked" transfer-coding MUST NOT be applied more than once in a message-body."

[aickin]

Right, but that quote doesn't mean that chunked encoding and gzip are incompatible; it says the opposite. They are compatible and can be used together, and the folks behind compression document it as intending to be compatible with chunked encoding. I think I need to add a few calls to flush, and maybe some guidance in how big a buffer to use in compression.

Of course, if you have a load balancer that doesn't support streaming or that doesn't allow its gzip to be tuned, that's a different issue.

[th0r]

@aickin compression is working for me: the response is chunked and gzipped.
I think you don't need to add flush calls because it's user's responsibility to do so if he really needs it.

@geekyme Try to set content type of the response via res.type('html') before calling any res.write().
From compression docs:

The default filter function uses the compressible module to determine if res.getHeader('Content-Type') is compressible.

It solved my problem.

P.S. @aickin Thanks a LOT for a GREAT module!

[geekyme]

Oh ok I was using res.writeHead to write the content encoding to text/html.

On 22 Oct 2015, at 5:35 PM, Yuriy Grunin notifications@github.com wrote:

@aickin compression is working for me: the response is chunked and gzipped.
I think you don't need to add flush calls because it's user's responsibility to do so if he really needs it.

@geekyme Try to set content type of the response via res.type('html') before calling any res.write().
From compression docs:

The default filter function uses the compressible module to determine if res.getHeader('Content-Type') is compressible.

It solved my problem.

—
Reply to this email directly or view it on GitHub.

[geekyme]

@th0r I just tried res.type('html') with compression on. Doesn't make a difference.

[th0r]

@geekyme Are your js- or css- responses gzipped?
How do your response headers look in the case of chunked response?

[geekyme]

All my responses from my server are gzipped.