This guide provides instructions on how to fine-tune the Falcon LLM 7B/40B model on a single GPU using LoRA and quantization, enabling data parallelism for linear scaling across multiple GPUs. This allows for impressive performance with commercially viable models like Falcon and MPT, such as performing inference using the Falcon 40B model in 4-bit mode with approximately 27 GB of GPU RAM. The guide is written for Lambda Cloud, but can also be applied to multi-GPU Linux workstations or servers. It includes a conda environment setup and provides example commands for fine-tuning the models. Benchmarking results show that training throughput scales nearly perfectly when scaling from 1x to 8x GPUs.