Transcribing on Fly GPU Machines
Fly.io has introduced GPU Machines, allowing users to run various workloads such as gaming, Bitcoin mining, NFT management, and AI tasks with just a few API calls. One example is the Whisper Webservice, an OpenAI voice recognition service that can be used for audio transcription. Users can create Fly GPU Machines by spinning up the Whisper Webservice Docker image, which supports Nvidia GPUs. The process involves creating a volume to house and cache models, setting the appropriate gpu_kind flag, and running machines with necessary configurations. Automating this process requires using the Machines API, while making one's own GPU-friendly Docker images can be achieved by installing the correct Nvidia drivers or starting with a base Ubuntu image.
Company
Fly.io
Date published
Nov. 13, 2023
Author(s)
Chris Fidao
Word count
1534
Language
English
Hacker News points
4