Skip to content

Sending unicode characters does not play nicely with JavaScript JSON parsing #304

@TheStaticMage

Description

@TheStaticMage

Describe the bug
This is somewhere between a bug and a feature request. Kick is 100% within its rights to send the payload in whatever format you want and expect the receiver to work with that format. I fully acknowledge that the webhook proxy I describe below was the thing that was broken. But I think you can make a fix to minimize this problem for others.

When a Kick payload includes a & it's actually sending \u0026 (at least in the channel rewards payload) and not the actual & character. That presents particular inconvenience for JavaScript developers because when JavaScript parses JSON, either under the hood or implicitly in Express, it parses this to &. Since \u0026 != & this causes a signature mismatch.

To Reproduce

  1. Generate webhook payload that includes an & character. For me this was a & in the description of a channel reward.
  2. In JavaScript, get the string from parsing JSON and then re-stringifying it.
  3. Note that incoming string != restringified string.

Expected behavior
I request that you don't send \uXXXX for common ASCII characters like & and ' in the payloads. Specifically turn off SetEscapeHTML when encoding JSON in payloads.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
I discovered this issue because it broke signature verification because a webhook proxy that I use was parsing and then re-encoding the JavaScript because of the way that Express was set up. I reported this to those devs and they have since started passing the raw payload alongside the header object and payload object, which now works fine. However I still wanted to request this because if it tripped me up, it will probably trip someone else up in the future.

Since it looks like your code is in go based on headers, I recommend:

  buf := &bytes.Buffer{}
  encoder := json.NewEncoder(buf)
  encoder.SetEscapeHTML(false)  // This is the key line
  encoder.Encode(data)

... or ...

  w.Header().Set("Content-Type", "application/json")
  encoder := json.NewEncoder(w)
  encoder.SetEscapeHTML(false)
  encoder.Encode(data)

... as the case may be.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingplannedThis will be worked on

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions