path: root/content/blog/2022-07-01-git-server.org

#+date:        <2022-07-01 Fri 00:00:00>
#+title:       Procedures for Establishing a Personal Git Repository Server
#+description: Comprehensive instructions for secure and efficient setup, hosting, and management of personal Git repositories to facilitate collaborative development and remote access.
#+slug:        git-server
#+filetags:    :git:server:self-hosting:

* My Approach to Self-Hosting Git

I have often tried to self-host my Git repositories, but have always
fallen short when I tried to find a suitable web interface to show on
the front-end.

After a few years, I have finally found a combination of methods that
allow me to easily self-host my projects, view them on the web, and
access them from anywhere.

Before I dive into the details, I want to state a high-level summary of
my self-hosted Git approach:

- This method uses the =ssh://= (read & write) and =git://= (read-only)
  protocols for push and pull access.
  - For the =git://= protocol, I create a =git-daemon-export-ok= file in
    any repository that I want to be cloneable by anyone.
  - The web interface I am using (=cgit=) allows simple HTTP cloning by
    default. I do not disable this setting as I want beginners to be
    able to clone one of my repositories even if they don't know the
    proper method.
- I am not enabling Smart HTTPS for any repositories. Updates to
  repositories must be pushed via SSH.
- Beyond the actual repository management, I am using =cgit= for the
  front-end web interface.
  - If you use the =scan-path=<path>= configuration in the =cgitrc=
    configuration file to automatically find repositories, you can't
    exclude a repository from =cgit= if it's stored within the path that
    =cgit= reads. To host private repositories, you'd need to set up
    another directory that =cgit= can't read.

* Assumptions

For the purposes of this walkthrough, I am assuming you have a URL
(=git.example.com=) or IP address (=207.84.26.991=) addressed to the
server that you will be using to host your git repositories.

* Adding a Git User

In order to use the SSH method associated with git, we will need to add
a user named =git=. If you have used the SSH method for other git
hosting sites, you are probably used to the following syntax:

#+begin_src sh
git clone [user@]server:project.git
#+end_src

The syntax above is an =scp=-like syntax for using SSH on the =git= user
on the server to access your repository.

Let's delete any remnants of an old =git= user, if any, and create the
new user account:

#+begin_src sh
sudo deluser --remove-home git
sudo adduser git
#+end_src

** Import Your SSH Keys to the Git User

Once the =git= user is created, you will need to copy your public SSH
key on your local development machine to the =git= user on the server.

If you don't have an SSH key yet, create one with this command:

#+begin_src sh
ssh-keygen
#+end_src

Once you create the key pair, the public should be saved to
=~/.ssh/id_rsa.pub=.

If your server still has password-based authentication available, you
can copy it over to your user's home directory like this:

#+begin_src sh
ssh-copy-id git@server
#+end_src

Otherwise, copy it over to any user that you can access.

#+begin_src sh
scp ~/.ssh/id_rsa.pub your_user@your_server:
#+end_src

Once on the server, you will need to copy the contents into the =git=
user's =authorized_keys= file:

#+begin_src sh
cat id_rsa.pub > /home/git/.ssh/authorized_keys
#+end_src

** (Optional) Disable Password-Based SSH

If you want to lock down your server and ensure that no one can
authenticate in via SSH with a password, you will need to edit your SSH
configuration.

#+begin_src sh
sudo nano /etc/ssh/sshd_config
#+end_src

Within this file, find the following settings and set them to the values
I am showing below:

#+begin_src conf
PermitRootLogin no
PasswordAuthentication no
AuthenticationMethods publickey
#+end_src

You may have other Authentication Methods required in your personal
set-up, so the key here is just to ensure that =AuthenticationMethods=
does not allow passwords.

*** Setting up the Base Directory

Now that we have set up a =git= user to handle all transport methods, we
need to set up the directory that we will be using as our base of all
repositories.

In my case, I am using =/git= as my source folder. To create this folder
and assign it to the user we created, execute the following commands:

#+begin_src sh
sudo mkdir /git
sudo chown -R git:git /git
#+end_src

*** Creating a Test Repository

On your server, switch over to the =git= user in order to start managing
git files.

#+begin_src sh
su git
#+end_src

Once logged-in as the =git= user, go to your base directory and create a
test repository.

#+begin_src sh
cd /git
mkdir test.git && cd test.git
git init --bare
#+end_src

If you want to make this repo viewable/cloneable to the public via the
=git://= protocol, you need to create a =git-daemon-export-ok= file
inside the repository.

#+begin_src sh
touch git-daemon-export-ok
#+end_src

* Change the Login Shell for =git=

To make sure that the =git= user is only used for git operations and
nothing else, you need to change the user's login shell. To do this,
simply use the =chsh= command:

#+begin_src sh
sudo chsh git
#+end_src

The interactive prompt will ask which shell you want the =git= user to
use. You must use the following value:

#+begin_src sh
/usr/bin/git-shell
#+end_src

Once done, no one will be able to SSH to the =git= user or execute
commands other than the standard git commands.

* Opening the Firewall

Don't forget to open up ports on the device firewall and network
firewall if you want to access these repositories publicly. If you're
using default ports, forward ports =22= (ssh) and =9418= (git) from your
router to your server's IP address.

If your server also has a firewall, ensure that the firewall allows the
same ports that are forwarded from the router. For example, if you use
=ufw=:

#+begin_src sh
sudo ufw allow 22
sudo ufw allow 9418
#+end_src

** Non-Standard SSH Ports

If you use a non-standard port for SSH, such as =9876=, you will need to
create an SSH configuration file on your local development machine in
order to connect to your server's git repositories.

To do this, you'll need to define your custom port on your client
machine in your =~/.ssh/config= file:

#+begin_src sh
nano ~/.ssh/config
#+end_src

#+begin_src conf
Host git.example.com
  # HostName can be a URL or an IP address
  HostName git.example.com
  Port 9876
  User git
#+end_src

** Testing SSH

There are two main syntaxes you can use to manage git over SSH:

- =git clone [user@]server:project.git=
- =git clone ssh://[user@]server/project.git=

I prefer the first, which is an =scp=-like syntax. To test it, try to
clone the test repository you set up on the server:

#+begin_src sh
git clone git@git.example.com:/git/test.git
#+end_src

* Enabling Read-Only Access

If you want people to be able to clone any repository where you've
placed a =git-daemon-export-ok= file, you will need to start the git
daemon.

To do this on a system with =systemd=, create a service file:

#+begin_src sh
sudo nano /etc/systemd/system/git-daemon.service
#+end_src

Inside the =git-daemon.service= file, paste the following:

#+begin_src conf
[Unit]
Description=Start Git Daemon

[Service]
ExecStart=/usr/bin/git daemon --reuseaddr --base-path=/git/ /git/

Restart=always
RestartSec=500ms

StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=git-daemon

User=git
Group=git

[Install]
WantedBy=multi-user.target
#+end_src

Once created, enable and start the service:

#+begin_src sh
sudo systemctl enable git-daemon.service
sudo systemctl start git-daemon.service
#+end_src

To clone read-only via the =git://= protocol, you can use the following
syntax:

#+begin_src sh
git clone git://git.example.com/test.git
#+end_src

* Migrating Repositories

At this point, we have a working git server that works with both SSH and
read-only access.

For each of the repositories I had hosted a different provider, I
executed the following commands in order to place a copy on my server as
my new source of truth:

Server:

#+begin_src sh
su git
mkdir /git/<REPOSITORY_NAME>.git && cd /git/<REPOSITORY_NAME>.git
git init --bare

# If you want to make this repo viewable/cloneable to the public
touch git-daemon-export-ok
#+end_src

Client:

#+begin_src sh
git clone git@<PREVIOUS_HOST>:<REPOSITORY_NAME>
git remote set-url origin git@git.EXAMPLE.COM:/git/<REPOSITORY_NAME>.git
git push
#+end_src

* Optional Web View: =cgit=

If you want a web viewer for your repositories, you can use various
tools, such as =gitweb=, =cgit=, or =klaus=. I chose =cgit= due to its
simple interface and fairly easy set-up (compared to others). Not to
mention that the [[https://git.kernel.org/][Linux kernel uses =cgit=]].

** Docker Compose

Instead of using my previous method of using a =docker run= command,
I've updated this section to use =docker-compose= instead for an easier
installation and simpler management and configuration.

In order to use Docker Compose, you will set up a =docker-compose.yml=
file to automatically connect resources like the repositories, =cgitrc=,
and various files or folders to the =cgit= container you're creating:

#+begin_src sh
mkdir ~/cgit && cd ~/cgit
nano docker-compose.yml
#+end_src

#+begin_src conf
# docker-compose.yml
version: '3'

services:
  cgit:
    image: invokr/cgit
    volumes:
      - /git:/git
      - ./cgitrc:/etc/cgitrc
      - ./logo.png:/var/www/htdocs/cgit/logo.png
      - ./favicon.png:/var/www/htdocs/cgit/favicon.png
      - ./filters:/var/www/htdocs/cgit/filters
    ports:
      - "8763:80"
    restart: always
#+end_src

Then, just start the container:

#+begin_src sh
sudo docker-compose up -d
#+end_src

Once it's finished installing, you can access the site at
=<SERVER_IP>:8763= or use a reverse-proxy service to forward =cgit= to a
URL, such as =git.example.com=. See the next section for more details on
reverse proxying a URL to a local port.

** Nginx Reverse Proxy

I am using Nginx as my reverse proxy so that the =cgit= Docker container
can use =git.example.com= as its URL. To do so, I simply created the
following configuration file:

#+begin_src sh
sudo nano /etc/nginx/sites-available/git.example.com
#+end_src

#+begin_src conf
server {
        listen 80;
          server_name git.example.com;

        if ($host = git.example.com) {
                return 301 https://$host$request_uri;
          }

          return 404;
}

server {
        server_name git.example.com;
        listen 443 ssl http2;

        location / {
                # The final `/` is important.
                    proxy_pass http://localhost:8763/;
                add_header X-Frame-Options SAMEORIGIN;
                add_header X-XSS-Protection "1; mode=block";
                proxy_redirect off;
                proxy_buffering off;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                proxy_set_header X-Forwarded-Port $server_port;
        }

        # INCLUDE ANY SSL CERTS HERE
        include /etc/letsencrypt/options-ssl-nginx.conf;
        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
}
#+end_src

Once created, symlink it and restart the web server.

#+begin_src sh
sudo ln -s /etc/nginx/sites-available/git.example.com /etc/nginx/sites-enabled/
sudo systemctl restart nginx.service
#+end_src

As we can see below, my site at =git.example.com= is available and
running:

** Settings Up Git Details

Once you have =cgit= running, you can add some small details, such as
repository owners and descriptions by editing the following files within
each repository.

Alternatively, you can use the =cgitrc= file to edit these details if
you only care to edit them for the purpose of seeing them on your
website.

The =description= file within the repository on your server will display
the description online.

#+begin_src sh
cd /git/example.git
nano description
#+end_src

You can add a =[gitweb]= block to the =config= file in order to display
the owner of the repository.

#+begin_src sh
cd /git/example.git
nano config
#+end_src

#+begin_src conf
[gitweb]
    owner = "YourName"
#+end_src

Note that you can ignore the configuration within each repository and
simply set up this information in the =cgitrc= file, if you want to do
it that way.

** Editing =cgit=

In order to edit certain items within =cgit=, you need to edit the
=cgitrc= file.

#+begin_src sh
nano ~/cgit/cgitrc
#+end_src

Below is an example configuration for =cgitrc=. You can find all the
configuration options within the [configuration manual]
([[https://git.zx2c4.com/cgit/plain/cgitrc.5.txt]]).

#+begin_src conf
css=/cgit.css
logo=/logo.png
favicon=/favicon.png
robots=noindex, nofollow

enable-index-links=1
enable-commit-graph=1
enable-blame=1
enable-log-filecount=1
enable-log-linecount=1
enable-git-config=1

clone-url=git://git.example.com/$CGIT_REPO_URL ssh://git@git.example.com:/git/$CGIT_REPO_URL

root-title=My Git Website
root-desc=My personal git repositories.

# Allow download of tar.gz, tar.bz2 and zip-files
snapshots=tar.gz tar.bz2 zip

##
## List of common mimetypes
##
mimetype.gif=image/gif
mimetype.html=text/html
mimetype.jpg=image/jpeg
mimetype.jpeg=image/jpeg
mimetype.pdf=application/pdf
mimetype.png=image/png
mimetype.svg=image/svg+xml

# Highlight source code
# source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.sh
source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py

# Format markdown, restructuredtext, manpages, text files, and html files
# through the right converters
about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh

##
## Search for these files in the root of the default branch of repositories
## for coming up with the about page:
##
readme=:README.md
readme=:readme.md
readme=:README.mkd
readme=:readme.mkd
readme=:README.rst
readme=:readme.rst
readme=:README.html
readme=:readme.html
readme=:README.htm
readme=:readme.htm
readme=:README.txt
readme=:readme.txt
readme=:README
readme=:readme

# Repositories

# Uncomment the following line to scan a path instead of adding repositories manually
# scan-path=/git

## Test Section
section=git/test-section

repo.url=test.git
repo.path=/git/test.git
repo.readme=:README.md
repo.owner=John Doe
repo.desc=An example repository!
#+end_src

** Final Fixes: Syntax Highlighting & README Rendering

After completing my initial install and playing around with it for a few
days, I noticed two issues:

1. Syntax highlighting did not work when viewing the source code within
   a file.
2. The =about= tab within a repository was not rendered to HTML.

The following process fixes these issues. To start, let's go to the
=cgit= directory where we were editing our configuration file earlier.

#+begin_src sh
cd ~/cgit
#+end_src

In here, create two folders that will hold our syntax files:

#+begin_src sh
mkdir filters && mkdir filters/html-converters && cd filters
#+end_src

Next, download the default filters:

#+begin_src sh
curl https://git.zx2c4.com/cgit/plain/filters/about-formatting.sh > about-formatting.sh
chmod 755 about-formatting.sh
curl https://git.zx2c4.com/cgit/plain/filters/syntax-highlighting.py > syntax-highlighting.py
chmod 755 syntax-highlighting.py
#+end_src

Finally, download the HTML conversion files you need. The example below
downloads the Markdown converter:

#+begin_src sh
cd html-converters
curl https://git.zx2c4.com/cgit/plain/filters/html-converters/md2html > md2html
chmod 755 md2html
#+end_src

If you need other filters or html-converters found within
[[https://git.zx2c4.com/cgit/tree/filters][the cgit project files]],
repeat the =curl= and =chmod= process above for whichever files you
need.

However, formatting will not work quite yet since the Docker cgit
container we're using doesn't have the formatting package installed. You
can install this easily by install Python 3+ and the =pygments= package:

#+begin_src sh
# Enter the container's command line
sudo docker exec -it cgit bash
#+end_src

#+begin_src sh
# Install the necessary packages and then exit
yum update -y &&                      \
yum upgrade -y &&                     \
yum install python3 python3-pip -y && \
pip3 install markdown pygments &&     \
exit
#+end_src

*You will need to enter the cgit docker container and re-run these =yum=
commands every time you kill and restart the container!*

If not done already, we need to add the following variables to our
=cgitrc= file in order for =cgit= to know where our filtering files are:

#+begin_src conf
# Highlight source code with python pygments-based highlighter
source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py

# Format markdown, restructuredtext, manpages, text files, and html files
# through the right converters
about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh
#+end_src

Now you should see that syntax highlighting and README rendering to the
=about= tab is fixed.

** Theming

I won't go into much detail in this section, but you can fully theme
your installation of =cgit= since you have access to the =cgit.css= file
in your web root. This is another file you can add as a volume to the
=docker-compose.yml= file if you want to edit this without entering the
container's command line.

*** :warning: Remember to Back Up Your Data!

The last thing to note is that running services on your own equipment
means that you're assuming a level of risk that exists regarding data
loss, catastrophes, etc. In order to reduce the impact of any such
occurrence, I suggest backing up your data regularly.

Backups can be automated via =cron=, by hooking your base directory up
to a cloud provider, or even setting up hooks to push all repository
info to git mirrors on other git hosts. Whatever the method, make sure
that your data doesn't vanish in the event that your drives or servers
fail.