If autolinking would make posts too long, don't

[elyscape]

Summary

Substitutions that increase message length have the potential to grow a post past the ability of the database to store it. This manifests to users as their posts inexplicably failing. To avoid this fail state, we check to see if the post-substitution message is too long and, if so, don't apply the changes.

Unfortunately, the actual maximum post size is not currently accessible to plugins. In the meantime, we use the smallest value for which the server guarantees support, which is exposed through model.POST_MESSAGE_MAX_RUNES_V1 and is in practice 4000 runes.

Ticket Link

None.

[mattermod]

Hello @elyscape,

Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

[hanzei]

Good catch on this issue.:+1:

[hanzei]

For newer Mattermost instances or the one that updates, this will be longer:

This release includes support for post messages longer than the default of 4000 characters, but may require a manual database migration. This migration is entirely optional, and need only be done if you want to enable post messages up to 16383 characters. For many installations, no migration will be required, or the old limit remains sufficient.

https://docs.mattermost.com/administration/important-upgrade-notes.html

I'm hesitant to cap at the old limit as that might render the plugin useless for some instances.

[elyscape]

This is fair, but we currently have no way to find the actual limit from a plug-in, and that excerpt notes that upgrading from 4000 characters may require a manual migration, which we can't expect every deployment to have performed.

[hanzei]

From looking at the server code base the UpdatePost RPC method does return an model.post.is_valid.msg.app_error error. (https://github.com/mattermost/mattermost-server/blob/ee3f986da0d9fbc69769dcec0c66d8493589757a/model/post.go#L299) Can we check for the specific error and return the original message in that case?

[elyscape]

The problem is that this function is effectively the handler for the MessageWillBePosted() and MessageWillBeUpdated() hooks, meaning we can alter the Post object but need to allow persistence to be handled further down the chain.

[elyscape]

I think the ideal solution is to first provide a way for plugins to get the result of GetMaxPostSize(), and then have the plugin:

1. Retrieve that value
2. Update the Post object
3. Call Post.IsValid() with its value
4. Revert the object if the result is false

[hanzei]

That is a concern to me as it unexpectedly renders the plugin useless in some cases that previously worked. I'm 2/5 that it's not worth breaking that cases for the bug fix that the PR includes.

[elyscape]

@hanzei What would you suggest as a path forward?

[hanzei]

From looking at the server code base the UpdatePost RPC method does return an model.post.is_valid.msg.app_error error. (mattermost/mattermost-server@ee3f986/model/post.go#L299) Can we check for the specific error and return the original message in that case?

@elyscape Do you think it's feasible to refactor the code and use that approach?

[elyscape]

I do not. This function is effectively the handler for the MessageWillBePosted and MessageWillBeUpdated hooks, meaning we can alter the Post object but need to allow persistence to be handled further down the chain. From the documentation:

To reject a post, return an non-empty string describing why the post was rejected. To modify the post, return the replacement, non-nil *model.Post and an empty string. To allow the post without modification, return a nil *model.Post and an empty string. To dismiss the post, return a nil *model.Post and the const DismissPostError string.

We could call UpdatePost from MessageHasBeenPosted or MessageHasBeenUpdated, because those are events that fire after a message occurs, but not here.

[elyscape]

@hanzei As a side complication, even once we get access to the result of GetMaxPostSize(), we'll need a fallback path for versions of Mattermost that don't support it.

[levb]

I like the simplicity of the current implementation. 0/5 we add it in "as is". We should probably open a ticket to make the MaxPostSize available to plugins, and to update Autolink once it's done; until the, we document the limitation? cc @aaronrothschild @justinegeffen

[elyscape]

I updated the PR to fix the tests and also include the post ID logged when a message would be too long.

Edit: Well, that broke the build differently because we aren't populating Post.Id in the test. I'll fix that once I figure out if that field is always expected to have a value and, if not, what I should do.

Edit 2: Turns out that the field is never expected to have a value. Oh well.

[elyscape]

Okay, tests fixed.

[levb]

@elyscape @hanzei @iomodo we discussed it offline, and the proposal is to implement both failure modes ("unchanged" and "error") for this feature. It can be triggered by a new Link-level flag (UseOriginalOnError: true? ). The current master behavior should be the default.

[elyscape]

@levb Sounds good. I'll figure out how to implement that in the next week.

[elyscape]

@levb @hanzei @iomodo I started looking into this and realized that I don't think there's actually a good way to implement it. As I understand it, the proposal is to implement this setting on a per-Autolink basis. The problem is that multiple Autolinks can apply to a single message, and processing applies all of them at once. If multiple Autolinks apply to a message but have differing values for UseOriginalOnError, it's unclear what should happen. I could have it try again with only the Autolinks that have UseOriginalOnError set to false, but that seems like it would provide a bad user experience. In my opinion, this should probably be a plugin-level setting. Thoughts?

[mattermod]

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!

/cc @jasonblais @jfrerich @emilyacook

[levb]

0/5 change the semantics, and just let the processing stop at the point where it'd have exceeded the max post size. I understand that it is unpredictable in the sense of what links will be processed, but IMO it can be a simple, sufficent solution?

[hanzei]

@elyscape Coming back to this PR: The solution in #154 (comment) seems good to me. Do you still want to work on it?