Adding support for transcripts, recording, AI summarization and meeting subscription to channels by jespino · Pull Request #377 · mattermost/mattermost-plugin-zoom

jespino · 2024-05-15T17:56:42Z

This PR include 3 important changes:

It allows the user to subscribe channels to meetings. That way, whenever that meeting is started outside mattermost it will post a message with a link to the meeting (identical to the one is create when you click the zoom button).
Whenever a meeting is finish, if there is a recording it is going to report the recording, the transcription and the chat messages in replies to the original zoom post (this works for the meetings in the point 1 and for the meetings that are created using the zoom button).
If you have the mattermost AI Copilot plugin installed and configured, whenever you receive the transcription and the chat, you can summarize them using the AI plugin.

Testing

For testing this, you should test the 3 features independently (there is some degree of overlap, but I going to propose independent test cases). Another important thing is that before you test it you need to configure the zoom application in the zoom marketplace with the right webhook events (it has change, so check out the documentation).

I expect all the test proposed here are executed after properly configuring zoom plugin and AI copilot plugin

Test case 1: Subscription

Go to zoom web interface and create a recurrent meeting.
Go to a channel and execute /zoom subscribe [meeting id], where [meeting id] is the ID of the created meeting in the pervious step (without spaces)
Go to the zoom web interface and initiate the recurrent meeting.
The expected behavior is that in some seconds after that, you see the post in the channel that is subscribe.
Stop the meeting.
After some seconds you should see reflected in the post that the meeting has ended
Go back to the channel and execute /zoom unsubscribe [meeting id], with the same meeting id.
Go back to the zoom web interface and initiate the recurrent meeting again.
The expected behavior is you don't see any post created in the channel.

Test case 2: Recordings, Chats, and Transcriptions (Without AI Copilot enabled)

Go to any channel and click the zoom button to start a meeting.
You will see the zoom post and you get automatically connect to the meeting.
Leave the meeting.
The post in some seconds is going to be mark as terminated.
No other messages are published later (you can skip this step because you can verify it later).
Open another zoom meeting using the same process.
Send a chat message
Start the recording
Say some text to get that in the transcription
Send another chat message
Stop the recording
Leave the meeting.
After this, you will see the post message changing to ended (in some seconds).
Then, after some minutes (maybe around 5), you will receive 2 new posts. one attaching the transcription, and one attaching the chat and including a link to the recording with the password.
Check the transcription and see if that is the one.
Check the chat and verify that only the message sent during the recording is there.
Check the recording link and password (maybe you need to use an incognito window to require the password)

Test case 3: AI Summarization (With AI Copilot enabled)

Follow all the steps of the test case 2 with AI copilot enable
In the transcription message and in the chat message you should see a new button that allows you to summarize the content.
Click that button in the chat history message and see the AI copilot summarizing it
Click that button in the transcription message and see the AI copilot summarizing it
If the text in the chat history or the transcription is too short the AI copilot can send you non-sense, if you want to have a better result with the AI copilot use more messages in the chat and speak more to include more text in the transcription.

Documentation PR: mattermost/docs#7276

jespino · 2024-05-15T18:28:51Z

server/webhook.go

+	var response *http.Response
+	for retries > 0 {
+		var err error
+		response, err = http.DefaultClient.Do(request)


This is because we are handling the incoming webhook from zoom as something secure, it should be signed if you configured it properly

Can these CodeQL errors be suppressed? We don't want this on every PR going forward after merge.

jespino · 2024-05-15T18:28:56Z

server/webhook.go

+					return
+				}
+				request.Header.Set("Authorization", "Bearer "+webhook.DownloadToken)
+				response, err := http.DefaultClient.Do(request)


This is because we are handling the incoming webhook from zoom as something secure, it should be signed if you configured it properly

CodeQL code tracing is 👌

Oh yes, I check the "show paths" and I was really impressed, It is pure gold.

mickmister

Excellent feature here 🚀

I gave the PR a review and added some comments for discussion. Let me know what you think 👍

mickmister · 2024-05-15T19:14:40Z

server/webhook.go

+					return
+				}
+				request.Header.Set("Authorization", "Bearer "+webhook.DownloadToken)
+				response, err := http.DefaultClient.Do(request)


CodeQL code tracing is 👌

mickmister · 2024-05-15T19:49:33Z

server/command.go

 	case actionConnect:
 		return p.runConnectCommand(user, args)
+	case actionSubscribe:
+		return p.runSubscribeCommand(user, args, meetingID)


I like this a lot 👍 Many Zoom meetings do not start in Mattermost, but now they kind of "do" because we can listen for that event in an intentionally subscribed channel. We do need to take access control into account here though. The person running this command should potentially not have access to that meeting

We'll also need to do some exploratory testing in general with this. e.g. Some conflict of meeting ids between starting a meeting in MM with a personal meeting id and a matching subscribed meeting id

Yes, probably the ideal solution is to not allow to subscribe personal meetings, and also ensure that you have access to the meeting before you subscribe it to the channel.

This is done. Now, you can only subscribe to meetings that are not personal meetings, so the only meetings that you know the ID upfront are not ad-hoc meetings, so any meeting created within Mattermost should be a PMI or Ad-hoc created on the fly.

so the only meetings that you know the ID upfront are not ad-hoc meetings

Sure but what if someone knows the meeting id and not e.g. a password, can they start funneling in recordings of the meetings in their own channel? Is "knowing the meeting id ahead of time" enough of a security measure?

@enzowritescode Thoughts?

@wiggin77 This has been on my TODO list for a while but other priorities have taken precedence. I even have a PR draft up from when I had started reviewing the plugin as a whole for better understanding #405.

Maybe with the introduction of Security Partners I can get this wrapped up in the next month or two

webapp/src/components/post_type_chat.jsx

server/command.go

server/webhook.go

…ng-and-ai-summarization

jespino · 2024-05-17T08:13:48Z

@mickmister PTAL

…ng-and-ai-summarization

wiggin77

Leaving this request for change to get this off my review queue until it is ready.

Kshitij-Katiyar · 2024-11-18T08:38:20Z

@jespino Can you sync this PR with master, this fix some of the failing ci

…ng-and-ai-summarization

…dereference

…test

jespino · 2025-01-11T08:15:51Z

@wiggin77 It looks like @enzowritescode is going to start with the security review soon, so feel free to start reviewing the code.

mattermost-build · 2025-05-25T01:07:31Z

This PR has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!

esarafianou · 2025-12-09T15:46:08Z

Removing the security review. Please re-add Product Security if the PR is picked up again.

marianunez · 2026-03-27T15:25:03Z

Closing this PR as this work was moved and completed in #455

Thanks @jespino 👋

jespino added 6 commits May 9, 2024 18:33

Adding support for transcripts, recording and AI summarization

db76929

WIP

a305b43

Adding chat support

dd47bbf

WIP

12815af

Adding support for subscription of meetings to channels

161b660

Removing debug log messages

467399e

jespino requested a review from mickmister as a code owner May 15, 2024 17:56

jespino added the Do Not Merge Should not be merged until this label is removed label May 15, 2024

github-advanced-security AI found potential problems May 15, 2024

View reviewed changes

jespino added 3 commits May 15, 2024 20:15

Fixing tests

e448589

fixing linter errors

df60c28

Fixing linter errors

bc696fd

mickmister requested a review from ayusht2810 May 15, 2024 19:10

mickmister added 2: Dev Review Requires review by a core committer 3: QA Review Requires review by a QA tester labels May 15, 2024

mickmister reviewed May 15, 2024

View reviewed changes

jespino added 5 commits May 16, 2024 10:39

Addressing PR review comments

513006a

Updating webhooks subscriptions needed in the documentation

8b30a5d

Addressing PR review comments

a810064

Addressing PR review comments

9ae260c

Avoid subscriptions to personal meetings

42bfc3a

mickmister requested a review from AayushChaudhary0001 May 16, 2024 19:31

jespino added 5 commits May 16, 2024 21:45

Merge remote-tracking branch 'origin/master' into transcripts-recordi…

4548448

…ng-and-ai-summarization

Migrating to typescript

4f6233f

Migrating to typescript

0cc74d9

Fixing linter

bf0599d

Adding tests for transcript and chat handlers

7777670

jespino requested a review from mickmister May 17, 2024 08:13

Fixing some linter errors

b67ed82

Merge remote-tracking branch 'origin/master' into transcripts-recordi…

30d8b15

…ng-and-ai-summarization

wiggin77 requested changes Oct 28, 2024

View reviewed changes

jespino added 13 commits January 11, 2025 07:35

Merge remote-tracking branch 'origin/master' into transcripts-recordi…

0280fc4

…ng-and-ai-summarization

Fixing linter error

d3067af

fix: Add missing mock expectation for KVGet in test case

fc9d542

fix: Add missing mock expectation for GetUser in webhook test

191ef21

fix: Add missing mock expectation for bot user's Zoom token

0af0204

fix: Add missing KVSetWithExpiry mock for meeting post ID test

d3f70c4

fix: Add missing mock expectation for KVSetWithExpiry in test

70f1af5

test: Add mock expectation for PublishWebSocketEvent in test case

5fe64f0

fix: Initialize plugin client in webhook test to resolve nil pointer …

9dc2a3c

…dereference

fix: Initialize plugin client in TestWebhookHandleRecordingCompleted …

c90d235

…test

Fixing tests

fb6ae2f

Fixing linter error

e9ea52a

test: Add tests for handleMeetingStarted webhook functionality

a35591e

jespino requested review from enzowritescode and wiggin77 and removed request for ayusht2810 January 11, 2025 08:16

mattermost-build added the Lifecycle/1:stale label May 25, 2025

wiggin77 removed their request for review November 8, 2025 06:23

esarafianou removed request for AayushChaudhary0001 and enzowritescode December 9, 2025 15:46

avasconcelos114 mentioned this pull request Feb 26, 2026

MM-67409 Adding support for transcripts, recording, AI summarization and meeting subscription to channels #455

Merged

17 tasks

marianunez closed this Mar 27, 2026

mattermost-build removed the Do Not Merge Should not be merged until this label is removed label Mar 27, 2026

Conversation

jespino commented May 15, 2024 • edited by marianunez Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing

Test case 1: Subscription

Test case 2: Recordings, Chats, and Transcriptions (Without AI Copilot enabled)

Test case 3: AI Summarization (With AI Copilot enabled)

Uh oh!

Check failure

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Check failure

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mickmister left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jespino commented May 17, 2024

Uh oh!

wiggin77 left a comment

Choose a reason for hiding this comment

Uh oh!

Kshitij-Katiyar commented Nov 18, 2024

Uh oh!

jespino commented Jan 11, 2025

Uh oh!

mattermost-build commented May 25, 2025

Uh oh!

esarafianou commented Dec 9, 2025

Uh oh!

marianunez commented Mar 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

11 participants

jespino commented May 15, 2024 •

edited by marianunez

Loading