Introduction
We'll be covering 2 main components here, first we will setup GitHub's self-hosted runner, which will enable the communication between our remote server and private repo, next we will create a workflow that will make our repository available on the remote server after pushing changes.
We'll also cover the exclusion of unnecessary files/directories on our remote server, such as .git, .gitignore, and .github, which will speed our workflow and reduce server resource usage.
Pre-requisites
To setup Github with SSH you can follow the official doc here:
https://docs.github.com/en/authentication/connecting-to-github-with-ssh
You can use the following referral link to test out Vultr's hosting with $300 credits:
https://www.vultr.com/?ref=9687233-9J
Setting up the self-hosted runner
The setup is straight forward and can be found on Github's official doc here:
https://docs.github.com/en/actions/how-tos/manage-runners/self-hosted-runners/add-runners
You'll go to settings of the specific repo you are working with, then Actions > Runners > New self-hosted runner, select your servers operating system and follow the instructions for the installation.
Now we'll need to setup the self-hosted runner as a service to run continuously and to start automatically on system reboot. Follow the official doc for the setup here (select your operating system under the title):
https://docs.github.com/en/actions/how-tos/manage-runners/self-hosted-runners/configure-the-application?platform=linux
Now if you visit the Runners section on Github, you should see the self hosted runner Idle with a green dot marking its status.
You might wonder if any specific ports need to be open or IP's whitelisted, but it turns out that the communication is done from the self-hosted runner to Github and not the other way around, therefor all you need is an outbound connection from your remote server which is in most cases open.
Creating the workflow
Inside your local project, create the following yml file:
mkdir -p .github/workflows
nano .github/workflows/deploy.yml
Inside the yml file add the following directives:
name: Deploy to Server
on:
push:
branches:
- main
jobs:
deploy:
runs-on: self-hosted
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Deploy to application directory
run: |
git archive --format=tar.gz --output /tmp/deploy.tar.gz HEAD
cd
mkdir -p test-repo && cd test-repo
rm -rf *
tar -xzf /tmp/deploy.tar.gz
rm /tmp/deploy.tar.gz
Let's break this down:
on:
push:
branches:
- main
The workflow is triggered when someone pushes to the main branch.
jobs:
deploy:
runs-on: self-hosted
Use the self-hosted runner we've setup to run our steps.
steps:
- name: Checkout code
uses: actions/checkout@v4
The actions/checkout@v4 step makes the repository available directly on our remote machine where the self-hosted runner is running.
- name: Deploy to application directory
run: |
git archive --format=tar.gz --output /tmp/deploy.tar.gz HEAD
cd
mkdir -p test-repo && cd test-repo
rm -rf *
tar -xzf /tmp/deploy.tar.gz
rm /tmp/deploy.tar.gz
Thanks to the checkout, we are able to use commands on the checkout repository such as git archive which creates a clean archive of your code without the .git directory which we will sotre in the tmp directory (/tmp/deploy.tar.gz).
Next we are entering into the users home directory, creating the test-repo directory, entering it, removing its contents, extracting the previously archived file inside it, and finally removing the archived file in the tmp directory.
Feel free to modify the commands based on your needs, such as moving the files to the webserver or application and running any necessary commands.
Excluding unnecessary files/directories
By using git archive above, we were able to exclude the .git directory. To exclude additional files when archiving we could use .gitattributes. Create the file:
nano .gitattributes
Next add the following directives to exclude certain files during archive:
.github export-ignore
.gitignore export-ignore
README.md export-ignore
Using a server exclusively for the self-hosted runner
You can host the self-hosted runner on a dedicated server if needed as well, this will require an additional step, the SSH connection between the runner server and application server.
First create SSH keys on your runner server:
cd
ssh-keygen -t rsa -b 2048
Enter the filename when prompted, ex: .ssh/filename.
Keep the passphrase empty when prompted and click enter.
Next we will copy the public key to our application server:
ssh-copy-id -i .ssh/filename.pub user@applicationServerIP
Next we can create an alias that will use our keys to simplify the login. Inside the .ssh directory on the runner server, create the following config file:
cd
nano .ssh/config
Next add the following directives:
Host app
Hostname appIpAddress
User appServerUser
IdentityFile ~/.ssh/filename
ServerAliveInterval 60
ServerAliveCountMax 120
Replace appIpAddress with the application server IP address, replace appServerUser with the application server user, and replace filename with the private key name previously created.
Now you can SSH to your application server using:
ssh app
Let's modify our Github workflow to use this alias to transfer our repo from the runner server to application server:
name: Deploy to Server
on:
push:
branches:
- main
jobs:
deploy:
runs-on: self-hosted
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Deploy to application server
run: |
git archive --format=tar.gz --output deploy.tar.gz HEAD
# Transfer and extract on the server
scp deploy.tar.gz app:/tmp/
ssh app << 'EOF'
cd
mkdir -p test-repo && cd test-repo
rm -rf *
tar -xzf /tmp/deploy.tar.gz
rm /tmp/deploy.tar.gz
EOF
rm deploy.tar.gz
The main difference here is that we are using scp and our alias app to first transfer the archived file to the tmp folder inside our application server, then we SSH the application server using the alias, create the test-repo folder, and extract the archived files into it.