To set up a continuous delivery system with Jenkins and GitHub, create a new server for deployment and configure it to work with Jenkins. This involves creating a new user on the deployment server, installing an SSH plugin in Jenkins, and setting up credentials for the repository. Next, create a new job in Jenkins that triggers on commits to the repository, builds and tests the code, packages the build, and deploys it to the server. The job can be customized to include additional steps such as restarting services and updating environment variables. Once the job is set up, it can be triggered by commits to different branches, allowing for automated deployment of test, staging, and production versions of the application.