/plushcap/analysis/fly-io/transcribing-on-fly-gpu-machines

Transcribing on Fly GPU Machines

What's this blog post about?

Fly.io has introduced GPU Machines, allowing users to run various workloads such as gaming, Bitcoin mining, NFT management, and AI tasks with just a few API calls. One example is the Whisper Webservice, an OpenAI voice recognition service that can be used for audio transcription. Users can create Fly GPU Machines by spinning up the Whisper Webservice Docker image, which supports Nvidia GPUs. The process involves creating a volume to house and cache models, setting the appropriate gpu_kind flag, and running machines with necessary configurations. Automating this process requires using the Machines API, while making one's own GPU-friendly Docker images can be achieved by installing the correct Nvidia drivers or starting with a base Ubuntu image.

Company
Fly.io

Date published
Nov. 13, 2023

Author(s)
Chris Fidao

Word count
1534

Hacker News points
4

Language
English


By Matt Makai. 2021-2024.