Skip to content

Comments

[Team B] AWS Transcribe Streaming implementation#272

Merged
FlyingWaffleDev merged 7 commits intodeveloperfrom
teamb-aws-transcribe-implementation
Mar 28, 2025
Merged

[Team B] AWS Transcribe Streaming implementation#272
FlyingWaffleDev merged 7 commits intodeveloperfrom
teamb-aws-transcribe-implementation

Conversation

@FlyingWaffleDev
Copy link
Contributor

@FlyingWaffleDev FlyingWaffleDev commented Mar 27, 2025

I found this library early on in trying to implement AWS: https://github.com/javer/aws_transcribe_streaming
Unfortunately, it hasn't been updated in two years, so it can't be used because of dependency issues. So, I went down three rabbit holes trying to manually implement the bare basics I needed using Amazon's API docs, but that just went nowhere in the time I have.
What I ended up doing was to just copy all the code from that library I linked into our codebase. It still works fine (just hasn't had the dependencies updated) so I was able to make a horrible garbage implementation of AWS side-by-side with our local stuff.
It works, but is just as flaky as our stuff, in new and interesting ways.
This also introduces the need for AWS keys, which I will move to the settings page in my next commit.
I want to add some kind of option to ignore AWS and just use Sherpa if the keys are missing, but that might take a while to get right.

@FlyingWaffleDev FlyingWaffleDev linked an issue Mar 27, 2025 that may be closed by this pull request
@FlyingWaffleDev FlyingWaffleDev force-pushed the teamb-aws-transcribe-implementation branch from 6831881 to 08cc27e Compare March 27, 2025 17:15
@FlyingWaffleDev
Copy link
Contributor Author

New commit should integrate AWS configuration into the settings menu.
Will try to work out the transcription flakiness when I get home with a computer I can actually test on.

@FlyingWaffleDev FlyingWaffleDev marked this pull request as ready for review March 27, 2025 21:51
@FlyingWaffleDev
Copy link
Contributor Author

FlyingWaffleDev commented Mar 27, 2025

All functional now. I would point out, though, that the quality between AWS and Sherpa is very nearly identical. Using this Youtube clip from House, I produced the following two transcripts in parallel:

SHERPA:

Speaker 1: I guess, inflammatory bowel.
Speaker 1: Yeah, is it that bad? Yes.

Speaker 2: It's also written on your chart.

Speaker 1: Bloody diarrhea, gas pain, took self-assalazine that it didn't work. Then I destroyed steroid amyls, oral corticosteroids, 5SA, 6-mercantapeuroy, and pressed.

Speaker 3: by my medical history.

Speaker 1: I will your last doctor charted.

Speaker 3: It's one thing to have to go to the bathroom every hour, but when the kids sit on my lap...
Speaker 3: The store sent me home. They're going to fire me. Can't you put me back on five ASA? Maybe it'll work this time. - Not likely.

Speaker 2: video prescription.
Speaker 2: which is good, insurance company won't pay for it.

Speaker 4: O'Juritis?

Speaker 1: cigarettes.
Speaker 1: One twice a day, and we're no less. Studies have shown that cigarette smoking is one of the most effective ways to control inflammatory bowel.
Speaker 1: well-established that you look thirty percent cooler

Speaker 4: Are you kidding me?

Speaker 1: the looking cooler, yeah.

Speaker 5: The rest is true. Isn't it addictive and dangerous?

Speaker 1: Pretty much all the drugs I prescribe are addictive and dangerous.
Speaker 1: only difference for this one is it's completely legal.

Speaker 5: Merry Christmas.

AWS:

Speaker 1: Let me guess, inflammatory bowel.

Speaker 2: Wow, yeah. Is it that bad? Yes.

Speaker 1: It's also written on your chart.
Speaker 1: Bloody diarrhea, gas, pain, took sulfasalazine, but it didn't work. No, then, then I, uh, tried steroid Emas, oral corticosteroids, 5 ASAs, 6 mercaptopurine unimpressed by my medical history, by how well your last doctor charted.

Speaker 3: It's one thing to have to go to the bathroom every hour, but when the kids sit on my lap, it's the store sent me home. They're going to fire me. Can't you put me back on 5 ASA. Maybe it'll work this time. Not likely. I can't give you a prescription. It's cheap, which is good, your insurance company won't pay for it.

Speaker 2: ger it is cigarettes.

Speaker 1: One twice a day and we're no less. Studies have shown that cigarette smoking is one of the most effective ways to control inflammatory bowel, plus it's been well established that you look 30% cooler. Are you kidding me? About the looking cooler, yeah, the rest is true. Isn't it addictive and dangerous? Pretty much all the drugs I prescribe are addictive and dangerous. The only difference with this one is it's completely legal.
Speaker 1: Merry Christmas.

As you can see, Sherpa has better sentence chunking, AWS has better recognition of weird words, but overall both mess up the speaker count, both mess up the speaker differentiation, etc.

@raeumgc raeumgc self-requested a review March 28, 2025 00:31
@FlyingWaffleDev FlyingWaffleDev force-pushed the teamb-aws-transcribe-implementation branch from 7e31b08 to 594943d Compare March 28, 2025 00:36
@FlyingWaffleDev FlyingWaffleDev merged commit bc7cb82 into developer Mar 28, 2025
1 check failed
@FlyingWaffleDev FlyingWaffleDev deleted the teamb-aws-transcribe-implementation branch March 28, 2025 00:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Team B] AWS Transcribe exploration

3 participants