diff options
Diffstat (limited to 'content/blog/2022-07-01-git-server.md')
| -rw-r--r-- | content/blog/2022-07-01-git-server.md | 333 |
1 files changed, 151 insertions, 182 deletions
diff --git a/content/blog/2022-07-01-git-server.md b/content/blog/2022-07-01-git-server.md index 61c4049..64a4a43 100644 --- a/content/blog/2022-07-01-git-server.md +++ b/content/blog/2022-07-01-git-server.md @@ -7,60 +7,54 @@ draft = false # My Approach to Self-Hosting Git -I have often tried to self-host my Git repositories, but have always -fallen short when I tried to find a suitable web interface to show on -the front-end. - -After a few years, I have finally found a combination of methods that -allow me to easily self-host my projects, view them on the web, and -access them from anywhere. - -Before I dive into the details, I want to state a high-level summary of -my self-hosted Git approach: - -- This method uses the `ssh://` (read & write) and - `git://` (read-only) protocols for push and pull access. - - For the `git://` protocol, I create a - `git-daemon-export-ok` file in any repository that I - want to be cloneable by anyone. - - The web interface I am using (`cgit`) allows simple - HTTP cloning by default. I do not disable this setting as I want - beginners to be able to clone one of my repositories even if - they don\'t know the proper method. -- I am not enabling Smart HTTPS for any repositories. Updates to - repositories must be pushed via SSH. -- Beyond the actual repository management, I am using - `cgit` for the front-end web interface. - - If you use the `scan-path=<path>` configuration in - the `cgitrc` configuration file to automatically find - repositories, you can\'t exclude a repository from - `cgit` if it\'s stored within the path that - `cgit` reads. To host private repositories, you\'d - need to set up another directory that `cgit` can\'t - read. +I have often tried to self-host my Git repositories, but have always fallen +short when I tried to find a suitable web interface to show on the front-end. + +After a few years, I have finally found a combination of methods that allow me +to easily self-host my projects, view them on the web, and access them from +anywhere. + +Before I dive into the details, I want to state a high-level summary of my +self-hosted Git approach: + +- This method uses the `ssh://` (read & write) and `git://` (read-only) + protocols for push and pull access. + - For the `git://` protocol, I create a `git-daemon-export-ok` file in any + repository that I want to be cloneable by anyone. + - The web interface I am using (`cgit`) allows simple HTTP cloning by default. + I do not disable this setting as I want beginners to be able to clone one of + my repositories even if they don't know the proper method. +- I am not enabling Smart HTTPS for any repositories. Updates to repositories + must be pushed via SSH. +- Beyond the actual repository management, I am using `cgit` for the front-end + web interface. + - If you use the `scan-path=<path>` configuration in the `cgitrc` + configuration file to automatically find repositories, you can't exclude a + repository from `cgit` if it's stored within the path that `cgit` reads. To + host private repositories, you'd need to set up another directory that + `cgit` can't read. # Assumptions For the purposes of this walkthrough, I am assuming you have a URL -(`git.example.com`) or IP address -(`207.84.26.991`) addressed to the server that you will be -using to host your git repositories. +(`git.example.com`) or IP address (`207.84.26.991`) addressed to the server that +you will be using to host your git repositories. # Adding a Git User -In order to use the SSH method associated with git, we will need to add -a user named `git`. If you have used the SSH method for other -git hosting sites, you are probably used to the following syntax: +In order to use the SSH method associated with git, we will need to add a user +named `git`. If you have used the SSH method for other git hosting sites, you +are probably used to the following syntax: ```sh git clone [user@]server:project.git ``` -The syntax above is an `scp`-like syntax for using SSH on the -`git` user on the server to access your repository. +The syntax above is an `scp`-like syntax for using SSH on the `git` user on the +server to access your repository. -Let\'s delete any remnants of an old `git` user, if any, and -create the new user account: +Let's delete any remnants of an old `git` user, if any, and create the new user +account: ```sh sudo deluser --remove-home git @@ -69,21 +63,19 @@ sudo adduser git ## Import Your SSH Keys to the Git User -Once the `git` user is created, you will need to copy your -public SSH key on your local development machine to the `git` -user on the server. +Once the `git` user is created, you will need to copy your public SSH key on +your local development machine to the `git` user on the server. -If you don\'t have an SSH key yet, create one with this command: +If you don't have an SSH key yet, create one with this command: ```sh ssh-keygen ``` -Once you create the key pair, the public should be saved to -`~/.ssh/id_rsa.pub`. +Once you create the key pair, the public should be saved to `~/.ssh/id_rsa.pub`. -If your server still has password-based authentication available, you -can copy it over to your user\'s home directory like this: +If your server still has password-based authentication available, you can copy +it over to your user's home directory like this: ```sh ssh-copy-id git@server @@ -95,8 +87,8 @@ Otherwise, copy it over to any user that you can access. scp ~/.ssh/id_rsa.pub your_user@your_server: ``` -Once on the server, you will need to copy the contents into the -`git` user\'s `authorized_keys` file: +Once on the server, you will need to copy the contents into the `git` user's +`authorized_keys` file: ```sh cat id_rsa.pub > /home/git/.ssh/authorized_keys @@ -104,16 +96,15 @@ cat id_rsa.pub > /home/git/.ssh/authorized_keys ## (Optional) Disable Password-Based SSH -If you want to lock down your server and ensure that no one can -authenticate in via SSH with a password, you will need to edit your SSH -configuration. +If you want to lock down your server and ensure that no one can authenticate in +via SSH with a password, you will need to edit your SSH configuration. ```sh sudo nano /etc/ssh/sshd_config ``` -Within this file, find the following settings and set them to the values -I am showing below: +Within this file, find the following settings and set them to the values I am +showing below: ``` conf PermitRootLogin no @@ -121,19 +112,17 @@ PasswordAuthentication no AuthenticationMethods publickey ``` -You may have other Authentication Methods required in your personal -set-up, so the key here is just to ensure that -`AuthenticationMethods` does not allow passwords. +You may have other Authentication Methods required in your personal set-up, so +the key here is just to ensure that `AuthenticationMethods` does not allow +passwords. ### Setting up the Base Directory -Now that we have set up a `git` user to handle all transport -methods, we need to set up the directory that we will be using as our -base of all repositories. +Now that we have set up a `git` user to handle all transport methods, we need to +set up the directory that we will be using as our base of all repositories. -In my case, I am using `/git` as my source folder. To create -this folder and assign it to the user we created, execute the following -commands: +In my case, I am using `/git` as my source folder. To create this folder and +assign it to the user we created, execute the following commands: ```sh sudo mkdir /git @@ -142,15 +131,15 @@ sudo chown -R git:git /git ### Creating a Test Repository -On your server, switch over to the `git` user in order to -start managing git files. +On your server, switch over to the `git` user in order to start managing git +files. ```sh su git ``` -Once logged-in as the `git` user, go to your base directory -and create a test repository. +Once logged-in as the `git` user, go to your base directory and create a test +repository. ```sh cd /git @@ -158,9 +147,9 @@ mkdir test.git && cd test.git git init --bare ``` -If you want to make this repo viewable/cloneable to the public via the -`git://` protocol, you need to create a -`git-daemon-export-ok` file inside the repository. +If you want to make this repo viewable/cloneable to the public via the `git://` +protocol, you need to create a `git-daemon-export-ok` file inside the +repository. ```sh touch git-daemon-export-ok @@ -168,34 +157,33 @@ touch git-daemon-export-ok # Change the Login Shell for `git` -To make sure that the `git` user is only used for git -operations and nothing else, you need to change the user\'s login shell. -To do this, simply use the `chsh` command: +To make sure that the `git` user is only used for git operations and nothing +else, you need to change the user's login shell. To do this, simply use the +`chsh` command: ```sh sudo chsh git ``` -The interactive prompt will ask which shell you want the -`git` user to use. You must use the following value: +The interactive prompt will ask which shell you want the `git` user to use. You +must use the following value: ```sh /usr/bin/git-shell ``` -Once done, no one will be able to SSH to the `git` user or -execute commands other than the standard git commands. +Once done, no one will be able to SSH to the `git` user or execute commands +other than the standard git commands. # Opening the Firewall -Don\'t forget to open up ports on the device firewall and network -firewall if you want to access these repositories publicly. If you\'re -using default ports, forward ports `22` (ssh) and -`9418` (git) from your router to your server\'s IP address. +Don't forget to open up ports on the device firewall and network firewall if +you want to access these repositories publicly. If you're using default ports, +forward ports `22` (ssh) and `9418` (git) from your router to your server's IP +address. -If your server also has a firewall, ensure that the firewall allows the -same ports that are forwarded from the router. For example, if you use -`ufw`: +If your server also has a firewall, ensure that the firewall allows the same +ports that are forwarded from the router. For example, if you use `ufw`: ```sh sudo ufw allow 22 @@ -204,12 +192,12 @@ sudo ufw allow 9418 ## Non-Standard SSH Ports -If you use a non-standard port for SSH, such as `9876`, you -will need to create an SSH configuration file on your local development -machine in order to connect to your server\'s git repositories. +If you use a non-standard port for SSH, such as `9876`, you will need to create +an SSH configuration file on your local development machine in order to connect +to your server's git repositories. -To do this, you\'ll need to define your custom port on your client -machine in your `~/.ssh/config` file: +To do this, you'll need to define your custom port on your client machine in +your `~/.ssh/config` file: ```sh nano ~/.ssh/config @@ -227,11 +215,11 @@ Host git.example.com There are two main syntaxes you can use to manage git over SSH: -- `git clone [user@]server:project.git` -- `git clone ssh://[user@]server/project.git` +- `git clone [user@]server:project.git` +- `git clone ssh://[user@]server/project.git` -I prefer the first, which is an `scp`-like syntax. To test -it, try to clone the test repository you set up on the server: +I prefer the first, which is an `scp`-like syntax. To test it, try to clone the +test repository you set up on the server: ```sh git clone git@git.example.com:/git/test.git @@ -239,9 +227,8 @@ git clone git@git.example.com:/git/test.git # Enabling Read-Only Access -If you want people to be able to clone any repository where you\'ve -placed a `git-daemon-export-ok` file, you will need to start -the git daemon. +If you want people to be able to clone any repository where you've placed a +`git-daemon-export-ok` file, you will need to start the git daemon. To do this on a system with `systemd`, create a service file: @@ -279,8 +266,7 @@ sudo systemctl enable git-daemon.service sudo systemctl start git-daemon.service ``` -To clone read-only via the `git://` protocol, you can use the -following syntax: +To clone read-only via the `git://` protocol, you can use the following syntax: ```sh git clone git://git.example.com/test.git @@ -291,9 +277,9 @@ git clone git://git.example.com/test.git At this point, we have a working git server that works with both SSH and read-only access. -For each of the repositories I had hosted a different provider, I -executed the following commands in order to place a copy on my server as -my new source of truth: +For each of the repositories I had hosted a different provider, I executed the +following commands in order to place a copy on my server as my new source of +truth: Server: @@ -316,23 +302,20 @@ git push # Optional Web View: `cgit` -If you want a web viewer for your repositories, you can use various -tools, such as `gitweb`, `cgit`, or -`klaus`. I chose `cgit` due to its simple -interface and fairly easy set-up (compared to others). Not to mention -that the [Linux kernel uses `cgit`](https://git.kernel.org/). +If you want a web viewer for your repositories, you can use various tools, such +as `gitweb`, `cgit`, or `klaus`. I chose `cgit` due to its simple interface and +fairly easy set-up (compared to others). Not to mention that the [Linux kernel +uses `cgit`](https://git.kernel.org/). ## Docker Compose -Instead of using my previous method of using a `docker run` -command, I\'ve updated this section to use `docker-compose` -instead for an easier installation and simpler management and -configuration. +Instead of using my previous method of using a `docker run` command, I've +updated this section to use `docker-compose` instead for an easier installation +and simpler management and configuration. -In order to use Docker Compose, you will set up a -`docker-compose.yml` file to automatically connect resources -like the repositories, `cgitrc`, and various files or folders -to the `cgit` container you\'re creating: +In order to use Docker Compose, you will set up a `docker-compose.yml` file to +automatically connect resources like the repositories, `cgitrc`, and various +files or folders to the `cgit` container you're creating: ```sh mkdir ~/cgit && cd ~/cgit @@ -363,17 +346,16 @@ Then, just start the container: sudo docker-compose up -d ``` -Once it\'s finished installing, you can access the site at -`<SERVER_IP>:8763` or use a reverse-proxy service to forward -`cgit` to a URL, such as `git.example.com`. See -the next section for more details on reverse proxying a URL to a local -port. +Once it's finished installing, you can access the site at `<SERVER_IP>:8763` or +use a reverse-proxy service to forward `cgit` to a URL, such as +`git.example.com`. See the next section for more details on reverse proxying a +URL to a local port. ## Nginx Reverse Proxy -I am using Nginx as my reverse proxy so that the `cgit` -Docker container can use `git.example.com` as its URL. To do -so, I simply created the following configuration file: +I am using Nginx as my reverse proxy so that the `cgit` Docker container can use +`git.example.com` as its URL. To do so, I simply created the following +configuration file: ```sh sudo nano /etc/nginx/sites-available/git.example.com @@ -422,29 +404,26 @@ sudo ln -s /etc/nginx/sites-available/git.example.com /etc/nginx/sites-enabled/ sudo systemctl restart nginx.service ``` -As we can see below, my site at `git.example.com` is -available and running: +As we can see below, my site at `git.example.com` is available and running: ## Settings Up Git Details -Once you have `cgit` running, you can add some small details, -such as repository owners and descriptions by editing the following -files within each repository. +Once you have `cgit` running, you can add some small details, such as repository +owners and descriptions by editing the following files within each repository. -Alternatively, you can use the `cgitrc` file to edit these -details if you only care to edit them for the purpose of seeing them on -your website. +Alternatively, you can use the `cgitrc` file to edit these details if you only +care to edit them for the purpose of seeing them on your website. -The `description` file within the repository on your server -will display the description online. +The `description` file within the repository on your server will display the +description online. ```sh cd /git/example.git nano description ``` -You can add a `[gitweb]` block to the `config` -file in order to display the owner of the repository. +You can add a `[gitweb]` block to the `config` file in order to display the +owner of the repository. ```sh cd /git/example.git @@ -456,21 +435,20 @@ nano config owner = "YourName" ``` -Note that you can ignore the configuration within each repository and -simply set up this information in the `cgitrc` file, if you -want to do it that way. +Note that you can ignore the configuration within each repository and simply set +up this information in the `cgitrc` file, if you want to do it that way. ## Editing `cgit` -In order to edit certain items within `cgit`, you need to -edit the `cgitrc` file. +In order to edit certain items within `cgit`, you need to edit the `cgitrc` +file. ```sh nano ~/cgit/cgitrc ``` -Below is an example configuration for `cgitrc`. You can find -all the configuration options within the \[configuration manual\] +Below is an example configuration for `cgitrc`. You can find all the +configuration options within the [configuration manual] (<https://git.zx2c4.com/cgit/plain/cgitrc.5.txt>). ``` conf @@ -549,17 +527,14 @@ repo.desc=An example repository! ## Final Fixes: Syntax Highlighting & README Rendering -After completing my initial install and playing around with it for a few -days, I noticed two issues: +After completing my initial install and playing around with it for a few days, I +noticed two issues: -1. Syntax highlighting did not work when viewing the source code within - a file. -2. The `about` tab within a repository was not rendered to - HTML. +1. Syntax highlighting did not work when viewing the source code within a file. +2. The `about` tab within a repository was not rendered to HTML. -The following process fixes these issues. To start, let\'s go to the -`cgit` directory where we were editing our configuration file -earlier. +The following process fixes these issues. To start, let's go to the `cgit` +directory where we were editing our configuration file earlier. ```sh cd ~/cgit @@ -589,15 +564,13 @@ curl https://git.zx2c4.com/cgit/plain/filters/html-converters/md2html > md2html chmod 755 md2html ``` -If you need other filters or html-converters found within [the cgit -project files](https://git.zx2c4.com/cgit/tree/filters), repeat the -`curl` and `chmod` process above for whichever -files you need. +If you need other filters or html-converters found within [the cgit project +files](https://git.zx2c4.com/cgit/tree/filters), repeat the `curl` and `chmod` +process above for whichever files you need. -However, formatting will not work quite yet since the Docker cgit -container we\'re using doesn\'t have the formatting package installed. -You can install this easily by install Python 3+ and the -`pygments` package: +However, formatting will not work quite yet since the Docker cgit container +we're using doesn't have the formatting package installed. You can install +this easily by install Python 3+ and the `pygments` package: ```sh # Enter the container's command line @@ -613,13 +586,11 @@ pip3 install markdown pygments && \ exit ``` -**You will need to enter the cgit docker container and re-run these -`yum` commands every time you kill and restart the -container!** +**You will need to enter the cgit docker container and re-run these `yum` +commands every time you kill and restart the container!** -If not done already, we need to add the following variables to our -`cgitrc` file in order for `cgit` to know where -our filtering files are: +If not done already, we need to add the following variables to our `cgitrc` file +in order for `cgit` to know where our filtering files are: ``` conf # Highlight source code with python pygments-based highlighter @@ -630,26 +601,24 @@ source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh ``` -Now you should see that syntax highlighting and README rendering to the -`about` tab is fixed. +Now you should see that syntax highlighting and README rendering to the `about` +tab is fixed. ## Theming -I won\'t go into much detail in this section, but you can fully theme -your installation of `cgit` since you have access to the -`cgit.css` file in your web root. This is another file you -can add as a volume to the `docker-compose.yml` file if you -want to edit this without entering the container\'s command line. +I won't go into much detail in this section, but you can fully theme your +installation of `cgit` since you have access to the `cgit.css` file in your web +root. This is another file you can add as a volume to the `docker-compose.yml` +file if you want to edit this without entering the container's command line. ### :warning: Remember to Back Up Your Data! -The last thing to note is that running services on your own equipment -means that you\'re assuming a level of risk that exists regarding data -loss, catastrophes, etc. In order to reduce the impact of any such -occurrence, I suggest backing up your data regularly. +The last thing to note is that running services on your own equipment means that +you're assuming a level of risk that exists regarding data loss, catastrophes, +etc. In order to reduce the impact of any such occurrence, I suggest backing up +your data regularly. -Backups can be automated via `cron`, by hooking your base -directory up to a cloud provider, or even setting up hooks to push all -repository info to git mirrors on other git hosts. Whatever the method, -make sure that your data doesn\'t vanish in the event that your drives -or servers fail. +Backups can be automated via `cron`, by hooking your base directory up to a +cloud provider, or even setting up hooks to push all repository info to git +mirrors on other git hosts. Whatever the method, make sure that your data +doesn't vanish in the event that your drives or servers fail. |