Browser freezes when streaming (very) long markdown responses

[JamesHWade]

When streaming long markdown content, the browser can freeze as markdown is re-parsed on every incoming chunk. This is especially noticeable when using LLMs with tool calls, where each tool result can add substantial content to the stream.

Reprex

library(shiny)
library(bslib)
library(shinychat)
library(coro)

# Simulate a long streaming response with "tool call" outputs
fake_stream <- async_generator(function(input) {
  yield("I'll run several analyses for you.\n\n")
  await(async_sleep(0.2))

  for (i in 1:8) {
    yield(sprintf("**Running tool %d: analyze_data**\n\n", i))
    await(async_sleep(0.05))

    # Large code block (simulates tool output)
    code <- paste0("```r\n# Results for dataset ", i, "\n",
      paste(sprintf("row_%02d <- c(%s)\n", 1:40,
        sapply(1:40, function(x) paste(sample(1:99, 8), collapse = ", "))),
        collapse = ""), "```\n\n")

    for (chunk in strsplit(code, "(?<=.{50})", perl = TRUE)[[1]]) {
      yield(chunk)
      await(async_sleep(0.008))
    }

    # Markdown table
    table <- paste0(
      "| Metric | Value | SE | CI Low | CI High | p |\n",
      "|--------|-------|-----|--------|---------|------|\n",
      paste(sprintf("| %s | %.2f | %.2f | %.2f | %.2f | %.4f |\n",
        c("Mean","Median","SD","Var","Skew","Kurt","Min","Max","Range","IQR"),
        runif(10,0,100), runif(10,0,5), runif(10,-5,0),
        runif(10,0,5), runif(10,0,0.1)), collapse = ""), "\n")

    for (char in strsplit(table, "")[[1]]) {
      yield(char)
      await(async_sleep(0.003))
    }
  }
  yield("All analyses complete!")
})

ui <- page_fillable(
  p("Send any message to trigger a long streaming response."),
  chat_ui("chat", fill = TRUE)
)

server <- function(input, output, session) {
  observeEvent(input$chat_user_input, {
    chat_append("chat", fake_stream(input$chat_user_input))
  })
}

shinyApp(ui, server)

The browser becomes increasingly unresponsive as the response grows, eventually freezing for several seconds. In extreme cases, it completely freezes that app.

Proposed solution

I have a local fix that implements incremental markdown rendering. It uses block-based rendering and throttling based on content size.

Would a PR be welcome? If so, should I only modify the .ts or should I also build and copy to both packages?

Abbreviated Session info

shiny     1.12.1.9000
bslib     0.9.0.9002
coro      1.1.0
shinychat 0.3.0.9000

[gadenbuie]

I've wanted to fix this for a while and have made a couple passes at it, but they haven't quite landed because I've always ended up tangled in API details or have been a little too complicated.

The behavior I'd love to see in the end is to have discrete moments in the chat lifecycle where content is no longer re-rendered on update. My current plan is to solve this by breaking up the chat message into sub-streams (<shiny-markdown-stream> components) with every API call (each change-of-turn in the ellmer tool-calling loop).

That said, I would certainly welcome a PR and would be interested to see how you've approached this and if it's a good fit for what I have in mind.

If you want to open a PR so we can take a look and talk through the code, that'd be great! Please don't feel like you need to clean it up a lot or get it into perfect shape; if it's enough to get the conversation started, that'd be great. I don't want you to waste any time if it turns out we have to go in a different direction.

To start, you can modify the .ts files and use make js-build r-update-dist to update the assets in the R package for testing. (Building the assets doesn't happen in CI.)