diff options
Diffstat (limited to 'content/blog/2022-07-01-git-server.md')
| -rw-r--r-- | content/blog/2022-07-01-git-server.md | 624 |
1 files changed, 0 insertions, 624 deletions
diff --git a/content/blog/2022-07-01-git-server.md b/content/blog/2022-07-01-git-server.md deleted file mode 100644 index 5299fdb..0000000 --- a/content/blog/2022-07-01-git-server.md +++ /dev/null @@ -1,624 +0,0 @@ -+++ -date = 2022-07-01 -title = "Self-Hosting a Personal Git Server" -description = "" -draft = false -+++ - -# My Approach to Self-Hosting Git - -I have often tried to self-host my Git repositories, but have always fallen -short when I tried to find a suitable web interface to show on the front-end. - -After a few years, I have finally found a combination of methods that allow me -to easily self-host my projects, view them on the web, and access them from -anywhere. - -Before I dive into the details, I want to state a high-level summary of my -self-hosted Git approach: - -- This method uses the `ssh://` (read & write) and `git://` (read-only) - protocols for push and pull access. - - For the `git://` protocol, I create a `git-daemon-export-ok` file in any - repository that I want to be cloneable by anyone. - - The web interface I am using (`cgit`) allows simple HTTP cloning by - default. I do not disable this setting as I want beginners to be able to - clone one of my repositories even if they don't know the proper method. -- I am not enabling Smart HTTPS for any repositories. Updates to repositories - must be pushed via SSH. -- Beyond the actual repository management, I am using `cgit` for the front-end - web interface. - - If you use the `scan-path=<path>` configuration in the `cgitrc` - configuration file to automatically find repositories, you can't exclude - a repository from `cgit` if it's stored within the path that `cgit` - reads. To host private repositories, you'd need to set up another - directory that `cgit` can't read. - -# Assumptions - -For the purposes of this walkthrough, I am assuming you have a URL -(`git.example.com`) or IP address (`207.84.26.991`) addressed to the server that -you will be using to host your git repositories. - -# Adding a Git User - -In order to use the SSH method associated with git, we will need to add a user -named `git`. If you have used the SSH method for other git hosting sites, you -are probably used to the following syntax: - -```sh -git clone [user@]server:project.git -``` - -The syntax above is an `scp`-like syntax for using SSH on the `git` user on the -server to access your repository. - -Let's delete any remnants of an old `git` user, if any, and create the new user -account: - -```sh -sudo deluser --remove-home git -sudo adduser git -``` - -## Import Your SSH Keys to the Git User - -Once the `git` user is created, you will need to copy your public SSH key on -your local development machine to the `git` user on the server. - -If you don't have an SSH key yet, create one with this command: - -```sh -ssh-keygen -``` - -Once you create the key pair, the public should be saved to `~/.ssh/id_rsa.pub`. - -If your server still has password-based authentication available, you can copy -it over to your user's home directory like this: - -```sh -ssh-copy-id git@server -``` - -Otherwise, copy it over to any user that you can access. - -```sh -scp ~/.ssh/id_rsa.pub your_user@your_server: -``` - -Once on the server, you will need to copy the contents into the `git` user's -`authorized_keys` file: - -```sh -cat id_rsa.pub > /home/git/.ssh/authorized_keys -``` - -## (Optional) Disable Password-Based SSH - -If you want to lock down your server and ensure that no one can authenticate in -via SSH with a password, you will need to edit your SSH configuration. - -```sh -sudo nano /etc/ssh/sshd_config -``` - -Within this file, find the following settings and set them to the values I am -showing below: - -```conf -PermitRootLogin no -PasswordAuthentication no -AuthenticationMethods publickey -``` - -You may have other Authentication Methods required in your personal set-up, so -the key here is just to ensure that `AuthenticationMethods` does not allow -passwords. - -### Setting up the Base Directory - -Now that we have set up a `git` user to handle all transport methods, we need to -set up the directory that we will be using as our base of all repositories. - -In my case, I am using `/git` as my source folder. To create this folder and -assign it to the user we created, execute the following commands: - -```sh -sudo mkdir /git -sudo chown -R git:git /git -``` - -### Creating a Test Repository - -On your server, switch over to the `git` user in order to start managing git -files. - -```sh -su git -``` - -Once logged-in as the `git` user, go to your base directory and create a test -repository. - -```sh -cd /git -mkdir test.git && cd test.git -git init --bare -``` - -If you want to make this repo viewable/cloneable to the public via the `git://` -protocol, you need to create a `git-daemon-export-ok` file inside the -repository. - -```sh -touch git-daemon-export-ok -``` - -# Change the Login Shell for `git` - -To make sure that the `git` user is only used for git operations and nothing -else, you need to change the user's login shell. To do this, simply use the -`chsh` command: - -```sh -sudo chsh git -``` - -The interactive prompt will ask which shell you want the `git` user to use. You -must use the following value: - -```sh -/usr/bin/git-shell -``` - -Once done, no one will be able to SSH to the `git` user or execute commands -other than the standard git commands. - -# Opening the Firewall - -Don't forget to open up ports on the device firewall and network firewall if you -want to access these repositories publicly. If you're using default ports, -forward ports `22` (ssh) and `9418` (git) from your router to your server's IP -address. - -If your server also has a firewall, ensure that the firewall allows the same -ports that are forwarded from the router. For example, if you use `ufw`: - -```sh -sudo ufw allow 22 -sudo ufw allow 9418 -``` - -## Non-Standard SSH Ports - -If you use a non-standard port for SSH, such as `9876`, you will need to create -an SSH configuration file on your local development machine in order to connect -to your server's git repositories. - -To do this, you'll need to define your custom port on your client machine in -your `~/.ssh/config` file: - -```sh -nano ~/.ssh/config -``` - -```conf -Host git.example.com - # HostName can be a URL or an IP address - HostName git.example.com - Port 9876 - User git -``` - -## Testing SSH - -There are two main syntaxes you can use to manage git over SSH: - -- `git clone [user@]server:project.git` -- `git clone ssh://[user@]server/project.git` - -I prefer the first, which is an `scp`-like syntax. To test it, try to clone the -test repository you set up on the server: - -```sh -git clone git@git.example.com:/git/test.git -``` - -# Enabling Read-Only Access - -If you want people to be able to clone any repository where you've placed a -`git-daemon-export-ok` file, you will need to start the git daemon. - -To do this on a system with `systemd`, create a service file: - -```sh -sudo nano /etc/systemd/system/git-daemon.service -``` - -Inside the `git-daemon.service` file, paste the following: - -```conf -[Unit] -Description=Start Git Daemon - -[Service] -ExecStart=/usr/bin/git daemon --reuseaddr --base-path=/git/ /git/ - -Restart=always -RestartSec=500ms - -StandardOutput=syslog -StandardError=syslog -SyslogIdentifier=git-daemon - -User=git -Group=git - -[Install] -WantedBy=multi-user.target -``` - -Once created, enable and start the service: - -```sh -sudo systemctl enable git-daemon.service -sudo systemctl start git-daemon.service -``` - -To clone read-only via the `git://` protocol, you can use the following syntax: - -```sh -git clone git://git.example.com/test.git -``` - -# Migrating Repositories - -At this point, we have a working git server that works with both SSH and -read-only access. - -For each of the repositories I had hosted a different provider, I executed the -following commands in order to place a copy on my server as my new source of -truth: - -Server: - -```sh -su git -mkdir /git/<REPOSITORY_NAME>.git && cd /git/<REPOSITORY_NAME>.git -git init --bare - -# If you want to make this repo viewable/cloneable to the public -touch git-daemon-export-ok -``` - -Client: - -```sh -git clone git@<PREVIOUS_HOST>:<REPOSITORY_NAME> -git remote set-url origin git@git.EXAMPLE.COM:/git/<REPOSITORY_NAME>.git -git push -``` - -# Optional Web View: `cgit` - -If you want a web viewer for your repositories, you can use various tools, such -as `gitweb`, `cgit`, or `klaus`. I chose `cgit` due to its simple interface and -fairly easy set-up (compared to others). Not to mention that the [Linux kernel -uses `cgit`](https://git.kernel.org/). - -## Docker Compose - -Instead of using my previous method of using a `docker run` command, I've -updated this section to use `docker-compose` instead for an easier installation -and simpler management and configuration. - -In order to use Docker Compose, you will set up a `docker-compose.yml` file to -automatically connect resources like the repositories, `cgitrc`, and various -files or folders to the `cgit` container you're creating: - -```sh -mkdir ~/cgit && cd ~/cgit -nano docker-compose.yml -``` - -```conf -# docker-compose.yml -version: '3' - -services: - cgit: - image: invokr/cgit - volumes: - - /git:/git - - ./cgitrc:/etc/cgitrc - - ./logo.png:/var/www/htdocs/cgit/logo.png - - ./favicon.png:/var/www/htdocs/cgit/favicon.png - - ./filters:/var/www/htdocs/cgit/filters - ports: - - "8763:80" - restart: always -``` - -Then, just start the container: - -```sh -sudo docker-compose up -d -``` - -Once it's finished installing, you can access the site at `<SERVER_IP>:8763` or -use a reverse-proxy service to forward `cgit` to a URL, such as -`git.example.com`. See the next section for more details on reverse proxying a -URL to a local port. - -## Nginx Reverse Proxy - -I am using Nginx as my reverse proxy so that the `cgit` Docker container can use -`git.example.com` as its URL. To do so, I simply created the following -configuration file: - -```sh -sudo nano /etc/nginx/sites-available/git.example.com -``` - -```conf -server { - listen 80; - server_name git.example.com; - - if ($host = git.example.com) { - return 301 https://$host$request_uri; - } - - return 404; -} - -server { - server_name git.example.com; - listen 443 ssl http2; - - location / { - # The final `/` is important. - proxy_pass http://localhost:8763/; - add_header X-Frame-Options SAMEORIGIN; - add_header X-XSS-Protection "1; mode=block"; - proxy_redirect off; - proxy_buffering off; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header X-Forwarded-Port $server_port; - } - - # INCLUDE ANY SSL CERTS HERE - include /etc/letsencrypt/options-ssl-nginx.conf; - ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; -} -``` - -Once created, symlink it and restart the web server. - -```sh -sudo ln -s /etc/nginx/sites-available/git.example.com /etc/nginx/sites-enabled/ -sudo systemctl restart nginx.service -``` - -As we can see below, my site at `git.example.com` is available and running: - -## Settings Up Git Details - -Once you have `cgit` running, you can add some small details, such as repository -owners and descriptions by editing the following files within each repository. - -Alternatively, you can use the `cgitrc` file to edit these details if you only -care to edit them for the purpose of seeing them on your website. - -The `description` file within the repository on your server will display the -description online. - -```sh -cd /git/example.git -nano description -``` - -You can add a `[gitweb]` block to the `config` file in order to display the -owner of the repository. - -```sh -cd /git/example.git -nano config -``` - -```conf -[gitweb] - owner = "YourName" -``` - -Note that you can ignore the configuration within each repository and simply set -up this information in the `cgitrc` file, if you want to do it that way. - -## Editing `cgit` - -In order to edit certain items within `cgit`, you need to edit the `cgitrc` -file. - -```sh -nano ~/cgit/cgitrc -``` - -Below is an example configuration for `cgitrc`. You can find all the -configuration options within the [configuration manual] -(<https://git.zx2c4.com/cgit/plain/cgitrc.5.txt>). - -```conf -css=/cgit.css -logo=/logo.png -favicon=/favicon.png -robots=noindex, nofollow - -enable-index-links=1 -enable-commit-graph=1 -enable-blame=1 -enable-log-filecount=1 -enable-log-linecount=1 -enable-git-config=1 - -clone-url=git://git.example.com/$CGIT_REPO_URL ssh://git@git.example.com:/git/$CGIT_REPO_URL - -root-title=My Git Website -root-desc=My personal git repositories. - -# Allow download of tar.gz, tar.bz2 and zip-files -snapshots=tar.gz tar.bz2 zip - -## -## List of common mimetypes -## -mimetype.gif=image/gif -mimetype.html=text/html -mimetype.jpg=image/jpeg -mimetype.jpeg=image/jpeg -mimetype.pdf=application/pdf -mimetype.png=image/png -mimetype.svg=image/svg+xml - -# Highlight source code -# source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.sh -source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py - -# Format markdown, restructuredtext, manpages, text files, and html files -# through the right converters -about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh - -## -## Search for these files in the root of the default branch of repositories -## for coming up with the about page: -## -readme=:README.md -readme=:readme.md -readme=:README.mkd -readme=:readme.mkd -readme=:README.rst -readme=:readme.rst -readme=:README.html -readme=:readme.html -readme=:README.htm -readme=:readme.htm -readme=:README.txt -readme=:readme.txt -readme=:README -readme=:readme - -# Repositories - -# Uncomment the following line to scan a path instead of adding repositories manually -# scan-path=/git - -## Test Section -section=git/test-section - -repo.url=test.git -repo.path=/git/test.git -repo.readme=:README.md -repo.owner=John Doe -repo.desc=An example repository! -``` - -## Final Fixes: Syntax Highlighting & README Rendering - -After completing my initial install and playing around with it for a few days, I -noticed two issues: - -1. Syntax highlighting did not work when viewing the source code within a file. -2. The `about` tab within a repository was not rendered to HTML. - -The following process fixes these issues. To start, let's go to the `cgit` -directory where we were editing our configuration file earlier. - -```sh -cd ~/cgit -``` - -In here, create two folders that will hold our syntax files: - -```sh -mkdir filters && mkdir filters/html-converters && cd filters -``` - -Next, download the default filters: - -```sh -curl https://git.zx2c4.com/cgit/plain/filters/about-formatting.sh > about-formatting.sh -chmod 755 about-formatting.sh -curl https://git.zx2c4.com/cgit/plain/filters/syntax-highlighting.py > syntax-highlighting.py -chmod 755 syntax-highlighting.py -``` - -Finally, download the HTML conversion files you need. The example below -downloads the Markdown converter: - -```sh -cd html-converters -curl https://git.zx2c4.com/cgit/plain/filters/html-converters/md2html > md2html -chmod 755 md2html -``` - -If you need other filters or html-converters found within [the cgit project -files](https://git.zx2c4.com/cgit/tree/filters), repeat the `curl` and `chmod` -process above for whichever files you need. - -However, formatting will not work quite yet since the Docker cgit container -we're using doesn't have the formatting package installed. You can install this -easily by install Python 3+ and the `pygments` package: - -```sh -# Enter the container's command line -sudo docker exec -it cgit bash -``` - -```sh -# Install the necessary packages and then exit -yum update -y && \ -yum upgrade -y && \ -yum install python3 python3-pip -y && \ -pip3 install markdown pygments && \ -exit -``` - -**You will need to enter the cgit docker container and re-run these `yum` -commands every time you kill and restart the container!** - -If not done already, we need to add the following variables to our `cgitrc` file -in order for `cgit` to know where our filtering files are: - -```conf -# Highlight source code with python pygments-based highlighter -source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py - -# Format markdown, restructuredtext, manpages, text files, and html files -# through the right converters -about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh -``` - -Now you should see that syntax highlighting and README rendering to the `about` -tab is fixed. - -## Theming - -I won't go into much detail in this section, but you can fully theme your -installation of `cgit` since you have access to the `cgit.css` file in your web -root. This is another file you can add as a volume to the `docker-compose.yml` -file if you want to edit this without entering the container's command line. - -### :warning: Remember to Back Up Your Data! - -The last thing to note is that running services on your own equipment means that -you're assuming a level of risk that exists regarding data loss, catastrophes, -etc. In order to reduce the impact of any such occurrence, I suggest backing up -your data regularly. - -Backups can be automated via `cron`, by hooking your base directory up to a -cloud provider, or even setting up hooks to push all repository info to git -mirrors on other git hosts. Whatever the method, make sure that your data -doesn't vanish in the event that your drives or servers fail. |