Slow on Mac M1 pro

[Daniel-Kelvich]

I tried running the demo on 11s audio and it took 4.2s to process.

extractor = PhonemeTimestampAligner(
    preset="en-us",  # Automatically selects best English model
    duration_max=12,
    device='cpu'
)

I have macbook m1 pro.

[Daniel-Kelvich]

balign examples/samples/LJSpeech/LJ001-0001.wav examples/samples/LJSpeech/LJ001-0001.srt.json output.json --debug

this one actually took 80ms

[Daniel-Kelvich]

My bad. Cpu code is till very slow at ~2-4 sec for batterfly.wav example. I tried running it on google colab and got the same numbers. @tabahi any idea why this might be happening?

[tabahi]

I can't think of a good reason why it should be slower on Mac or any CPU when processing a single file. I just ran the following code and got 0.19s. Can you confirm that you are using the same code? If your t0 starts before model loading and audio loading + resampling, then 1 to 10s is expected. The process_sentence function should be around 0.2s on all modern CPUs.

    
import torch
import time
from bournemouth_aligner import PhonemeTimestampAligner

def example_audio_timestamps():

    text_sentence = "butterfly"
    audio_path = "examples/samples/audio/109867__timkahn__butterfly.wav"

    # Try to use the same pre-loaded extractor to avoid reloading delays
    extractor = PhonemeTimestampAligner(preset="en-us", duration_max=10, device='cpu') # loads model from huggingface cache
    

    audio_wav = extractor.load_audio(audio_path) # can replace it with custom audio source # may resample to 16kHz

    t0 = time.time()

    timestamps = extractor.process_sentence(text_sentence, audio_wav, ts_out_path=None, extract_embeddings=False, vspt_path=None, do_groups=True, debug=True)

    t1 = time.time()
    print("Timestamps:")
    print(f"Processing time: {t1 - t0:.2f} seconds")

if __name__ == "__main__":
    torch.random.manual_seed(42)
    example_audio_timestamps()

[Daniel-Kelvich]

import os
os.environ["PHONEMIZER_ESPEAK_LIBRARY"] = "/opt/homebrew/Cellar/espeak-ng/1.52.0/lib/libespeak-ng.1.dylib"
    
import torch
import time
from bournemouth_aligner import PhonemeTimestampAligner

def example_audio_timestamps():

    text_sentence = "butterfly"
    audio_path = "examples/samples/audio/109867__timkahn__butterfly.wav"

    # Try to use the same pre-loaded extractor to avoid reloading delays
    t0 = time.time()
    extractor = PhonemeTimestampAligner(preset="en-us", duration_max=10, device='cpu') # loads model from huggingface cache
    t1 = time.time()
    print(f"Init time: {t1 - t0:.2f} seconds")

    audio_wav = extractor.load_audio(audio_path) # can replace it with custom audio source # may resample to 16kHz

    t0 = time.time()

    timestamps = extractor.process_sentence(text_sentence, audio_wav, ts_out_path=None, extract_embeddings=False, vspt_path=None, do_groups=True, debug=False)

    t1 = time.time()
    print("Timestamps:")
    print(f"Processing time: {t1 - t0:.2f} seconds")

if __name__ == "__main__":
    torch.random.manual_seed(42)
    example_audio_timestamps()

Init time: 1.25 seconds
Processing time: 3.69 seconds