voxtral.c在m1 pro上体验

Posted on Views:

env

0.requirement

依赖xcode,未安装则store -> xcode

1.clone

git clone git@github.com:antirez/voxtral.c.git

2.download mode

  • mistralai/Voxtral-Mini-4B-Realtime-2602
./download_model.sh

⚠️：要开梯子

3.make

mac@GaGa voxtral.c % make
voxtral.c — Voxtral Realtime 4B - Build Targets

Choose a backend:
  make blas     - With BLAS acceleration (Accelerate/OpenBLAS)  # cpu
  make mps      - Apple Silicon with Metal GPU (fastest) # gpu

Other targets:
  make test     - Run regression tests (slow, needs fast GPU)
  make clean    - Remove build artifacts  # clean
  make inspect  - Build safetensors weight inspector
  make info     - Show build configuration

Example: make blas && ./voxtral -d voxtral-model -i audio.wav

make mps

mac@GaGa voxtral.c % make mps
rm -f voxtral.o voxtral_kernels.o voxtral_audio.o voxtral_encoder.o voxtral_decoder.o voxtral_tokenizer.o voxtral_safetensors.o voxtral_mic_macos.o *.mps.o voxtral_metal.o main.o inspect_weights.o voxtral inspect_weights
rm -f voxtral_shaders_source.h
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral.mps.o voxtral.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral_kernels.mps.o voxtral_kernels.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral_audio.mps.o voxtral_audio.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral_encoder.mps.o voxtral_encoder.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral_decoder.mps.o voxtral_decoder.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral_tokenizer.mps.o voxtral_tokenizer.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral_safetensors.mps.o voxtral_safetensors.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o voxtral_mic_macos.mps.o voxtral_mic_macos.c
xxd -i voxtral_shaders.metal > voxtral_shaders_source.h
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -fobjc-arc -c -o voxtral_metal.o voxtral_metal.m
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -c -o main.mps.o main.c
gcc -Wall -Wextra -O3 -march=native -ffast-math -DUSE_BLAS -DUSE_METAL -DACCELERATE_NEW_LAPACK -o voxtral voxtral.mps.o voxtral_kernels.mps.o voxtral_audio.mps.o voxtral_encoder.mps.o voxtral_decoder.mps.o voxtral_tokenizer.mps.o voxtral_safetensors.mps.o voxtral_mic_macos.mps.o voxtral_metal.o main.mps.o -lm -framework Accelerate -framework Metal -framework MetalPerformanceShaders -framework MetalPerformanceShadersGraph -framework Foundation -framework AudioToolbox -framework CoreFoundation

Built with MPS backend (Metal GPU acceleration)

4.test

voxtral.c — Voxtral Realtime 4B speech-to-text

Usage: ./voxtral -d <model_dir> (-i <input.wav> | --stdin | --from-mic) [options]

Required:
  -d <dir>      Model directory (with consolidated.safetensors, tekken.json)  # 模型目录
  -i <file>     Input WAV file (16-bit PCM, any sample rate)    # 从音频文件读取
  --stdin       Read audio from stdin (auto-detect WAV or raw s16le 16kHz mono)
  --from-mic    Capture from default microphone (macOS only, Ctrl+C to stop) # 麦克风读取

Options:
  -I <secs>     Encoder processing interval in seconds (default: 2.0)
  --alt <c>     Show alternative tokens within cutoff distance (0.0-1.0)
  --debug       Debug output (per-layer, per-chunk details)
  --silent      No status output (only transcription on stdout)
  -h            Show this help

# 1.从麦克风读取
./voxtral -d voxtral-model --from-mic

# 2.从音频文件读取
cat teste.mp3 | ./voxtral -d voxtral-model/ --stdin

# 3.
time cat teste.wav | ./voxtral -d voxtral-model/ --stdin

mps vs blas

## mps   gpu
mac@GaGa voxtral.c % ./voxtral -d voxtral-model -i samples/jfk.wav
Loading weights...
Metal GPU: 8429.2 MB
Model loaded.
Audio: 176000 samples (11.0 seconds)
And so, my fellow Americans, ask not what your country can do for you. Ask what you can do for your country.
Encoder: 1496 mel -> 187 tokens (39093 ms)
Decoder: 26 text tokens (149 steps) in 158457 ms (prefill 41151 ms + 792.6 ms/step)

## blas  cpu
mac@GaGa voxtral.c % chmod +x voxtral
mac@GaGa voxtral.c % ./voxtral -d  voxtral-model -i samples/jfk.wav
Loading weights...
Model loaded.
Audio: 176000 samples (11.0 seconds)
And so, my fellow Americans, ask not what your country can do for you. Ask what you can do for your country.
Encoder: 1496 mel -> 187 tokens (42137 ms)
Decoder: 26 text tokens (149 steps) in 117817 ms (prefill 28101 ms + 606.2 ms/step)

建议：m3 max，m1 decode太慢了。