initial migration to test org-mode

1 files changed, 617 insertions, 0 deletions

diff --git a/blog/git-server/index.org b/blog/git-server/index.org
new file mode 100644
index 0000000..c716484
--- /dev/null
+++ b/blog/git-server/index.org
@@ -0,0 +1,617 @@
+#+title: Self-Hosting a Personal Git Server
+#+date: 2022-07-01
+#+description: A guide to self-hosting a Git server on your own server.
+#+filetags: :selfhosting:
+
+* My Approach to Self-Hosting Git
+I have often tried to self-host my Git repositories, but have always
+fallen short when I tried to find a suitable web interface to show on
+the front-end.
+
+After a few years, I have finally found a combination of methods that
+allow me to easily self-host my projects, view them on the web, and
+access them from anywhere.
+
+Before I dive into the details, I want to state a high-level summary of
+my self-hosted Git approach:
+
+- This method uses the =ssh://= (read & write) and =git://= (read-only)
+  protocols for push and pull access.
+  - For the =git://= protocol, I create a =git-daemon-export-ok= file in
+    any repository that I want to be cloneable by anyone.
+  - The web interface I am using (=cgit=) allows simple HTTP cloning by
+    default. I do not disable this setting as I want beginners to be
+    able to clone one of my repositories even if they don't know the
+    proper method.
+- I am not enabling Smart HTTPS for any repositories. Updates to
+  repositories must be pushed via SSH.
+- Beyond the actual repository management, I am using =cgit= for the
+  front-end web interface.
+  - If you use the =scan-path=<path>= configuration in the =cgitrc=
+    configuration file to automatically find repositories, you can't
+    exclude a repository from =cgit= if it's stored within the path that
+    =cgit= reads. To host private repositories, you'd need to set up
+    another directory that =cgit= can't read.
+
+* Assumptions
+For the purposes of this walkthrough, I am assuming you have a URL
+(=git.example.com=) or IP address (=207.84.26.991=) addressed to the
+server that you will be using to host your git repositories.
+
+* Adding a Git User
+In order to use the SSH method associated with git, we will need to add
+a user named =git=. If you have used the SSH method for other git
+hosting sites, you are probably used to the following syntax:
+
+#+begin_src sh
+git clone [user@]server:project.git
+#+end_src
+
+The syntax above is an =scp=-like syntax for using SSH on the =git= user
+on the server to access your repository.
+
+Let's delete any remnants of an old =git= user, if any, and create the
+new user account:
+
+#+begin_src sh
+sudo deluser --remove-home git
+sudo adduser git
+#+end_src
+
+** Import Your SSH Keys to the Git User
+Once the =git= user is created, you will need to copy your public SSH
+key on your local development machine to the =git= user on the server.
+
+If you don't have an SSH key yet, create one with this command:
+
+#+begin_src sh
+ssh-keygen
+#+end_src
+
+Once you create the key pair, the public should be saved to
+=~/.ssh/id_rsa.pub=.
+
+If your server still has password-based authentication available, you
+can copy it over to your user's home directory like this:
+
+#+begin_src sh
+ssh-copy-id git@server
+#+end_src
+
+Otherwise, copy it over to any user that you can access.
+
+#+begin_src sh
+scp ~/.ssh/id_rsa.pub your_user@your_server:
+#+end_src
+
+Once on the server, you will need to copy the contents into the =git=
+user's =authorized_keys= file:
+
+#+begin_src sh
+cat id_rsa.pub > /home/git/.ssh/authorized_keys
+#+end_src
+
+** (Optional) Disable Password-Based SSH
+If you want to lock down your server and ensure that no one can
+authenticate in via SSH with a password, you will need to edit your SSH
+configuration.
+
+#+begin_src sh
+sudo nano /etc/ssh/sshd_config
+#+end_src
+
+Within this file, find the following settings and set them to the values
+I am showing below:
+
+#+begin_src conf
+PermitRootLogin no
+PasswordAuthentication no
+AuthenticationMethods publickey
+#+end_src
+
+You may have other Authentication Methods required in your personal
+set-up, so the key here is just to ensure that =AuthenticationMethods=
+does not allow passwords.
+
+*** Setting up the Base Directory
+Now that we have set up a =git= user to handle all transport methods, we
+need to set up the directory that we will be using as our base of all
+repositories.
+
+In my case, I am using =/git= as my source folder. To create this folder
+and assign it to the user we created, execute the following commands:
+
+#+begin_src sh
+sudo mkdir /git
+sudo chown -R git:git /git
+#+end_src
+
+*** Creating a Test Repository
+On your server, switch over to the =git= user in order to start managing
+git files.
+
+#+begin_src sh
+su git
+#+end_src
+
+Once logged-in as the =git= user, go to your base directory and create a
+test repository.
+
+#+begin_src sh
+cd /git
+mkdir test.git && cd test.git
+git init --bare
+#+end_src
+
+If you want to make this repo viewable/cloneable to the public via the
+=git://= protocol, you need to create a =git-daemon-export-ok= file
+inside the repository.
+
+#+begin_src sh
+touch git-daemon-export-ok
+#+end_src
+
+* Change the Login Shell for =git=
+To make sure that the =git= user is only used for git operations and
+nothing else, you need to change the user's login shell. To do this,
+simply use the =chsh= command:
+
+#+begin_src sh
+sudo chsh git
+#+end_src
+
+The interactive prompt will ask which shell you want the =git= user to
+use. You must use the following value:
+
+#+begin_src sh
+/usr/bin/git-shell
+#+end_src
+
+Once done, no one will be able to SSH to the =git= user or execute
+commands other than the standard git commands.
+
+* Opening the Firewall
+Don't forget to open up ports on the device firewall and network
+firewall if you want to access these repositories publicly. If you're
+using default ports, forward ports =22= (ssh) and =9418= (git) from your
+router to your server's IP address.
+
+If your server also has a firewall, ensure that the firewall allows the
+same ports that are forwarded from the router. For example, if you use
+=ufw=:
+
+#+begin_src sh
+sudo ufw allow 22
+sudo ufw allow 9418
+#+end_src
+
+** Non-Standard SSH Ports
+If you use a non-standard port for SSH, such as =9876=, you will need to
+create an SSH configuration file on your local development machine in
+order to connect to your server's git repositories.
+
+To do this, you'll need to define your custom port on your client
+machine in your =~/.ssh/config= file:
+
+#+begin_src sh
+nano ~/.ssh/config
+#+end_src
+
+#+begin_src conf
+Host git.example.com
+  # HostName can be a URL or an IP address
+  HostName git.example.com
+  Port 9876
+  User git
+#+end_src
+
+** Testing SSH
+There are two main syntaxes you can use to manage git over SSH:
+
+- =git clone [user@]server:project.git=
+- =git clone ssh://[user@]server/project.git=
+
+I prefer the first, which is an =scp=-like syntax. To test it, try to
+clone the test repository you set up on the server:
+
+#+begin_src sh
+git clone git@git.example.com:/git/test.git
+#+end_src
+
+* Enabling Read-Only Access
+If you want people to be able to clone any repository where you've
+placed a =git-daemon-export-ok= file, you will need to start the git
+daemon.
+
+To do this on a system with =systemd=, create a service file:
+
+#+begin_src sh
+sudo nano /etc/systemd/system/git-daemon.service
+#+end_src
+
+Inside the =git-daemon.service= file, paste the following:
+
+#+begin_src conf
+[Unit]
+Description=Start Git Daemon
+
+[Service]
+ExecStart=/usr/bin/git daemon --reuseaddr --base-path=/git/ /git/
+
+Restart=always
+RestartSec=500ms
+
+StandardOutput=syslog
+StandardError=syslog
+SyslogIdentifier=git-daemon
+
+User=git
+Group=git
+
+[Install]
+WantedBy=multi-user.target
+#+end_src
+
+Once created, enable and start the service:
+
+#+begin_src sh
+sudo systemctl enable git-daemon.service
+sudo systemctl start git-daemon.service
+#+end_src
+
+To clone read-only via the =git://= protocol, you can use the following
+syntax:
+
+#+begin_src sh
+git clone git://git.example.com/test.git
+#+end_src
+
+* Migrating Repositories
+At this point, we have a working git server that works with both SSH and
+read-only access.
+
+For each of the repositories I had hosted a different provider, I
+executed the following commands in order to place a copy on my server as
+my new source of truth:
+
+Server:
+
+#+begin_src sh
+su git
+mkdir /git/<REPOSITORY_NAME>.git && cd /git/<REPOSITORY_NAME>.git
+git init --bare
+
+# If you want to make this repo viewable/cloneable to the public
+touch git-daemon-export-ok
+#+end_src
+
+Client:
+
+#+begin_src sh
+git clone git@<PREVIOUS_HOST>:<REPOSITORY_NAME>
+git remote set-url origin git@git.EXAMPLE.COM:/git/<REPOSITORY_NAME>.git
+git push
+#+end_src
+
+* Optional Web View: =cgit=
+If you want a web viewer for your repositories, you can use various
+tools, such as =gitweb=, =cgit=, or =klaus=. I chose =cgit= due to its
+simple interface and fairly easy set-up (compared to others). Not to
+mention that the [[https://git.kernel.org/][Linux kernel uses =cgit=]].
+
+** Docker Compose
+Instead of using my previous method of using a =docker run= command,
+I've updated this section to use =docker-compose= instead for an easier
+installation and simpler management and configuration.
+
+In order to use Docker Compose, you will set up a =docker-compose.yml=
+file to automatically connect resources like the repositories, =cgitrc=,
+and various files or folders to the =cgit= container you're creating:
+
+#+begin_src sh
+mkdir ~/cgit && cd ~/cgit
+nano docker-compose.yml
+#+end_src
+
+#+begin_src conf
+# docker-compose.yml
+version: '3'
+
+services:
+  cgit:
+    image: invokr/cgit
+    volumes:
+      - /git:/git
+      - ./cgitrc:/etc/cgitrc
+      - ./logo.png:/var/www/htdocs/cgit/logo.png
+      - ./favicon.png:/var/www/htdocs/cgit/favicon.png
+      - ./filters:/var/www/htdocs/cgit/filters
+    ports:
+      - "8763:80"
+    restart: always
+#+end_src
+
+Then, just start the container:
+
+#+begin_src sh
+sudo docker-compose up -d
+#+end_src
+
+Once it's finished installing, you can access the site at
+=<SERVER_IP>:8763= or use a reverse-proxy service to forward =cgit= to a
+URL, such as =git.example.com=. See the next section for more details on
+reverse proxying a URL to a local port.
+
+** Nginx Reverse Proxy
+I am using Nginx as my reverse proxy so that the =cgit= Docker container
+can use =git.example.com= as its URL. To do so, I simply created the
+following configuration file:
+
+#+begin_src sh
+sudo nano /etc/nginx/sites-available/git.example.com
+#+end_src
+
+#+begin_src conf
+server {
+        listen 80;
+          server_name git.example.com;
+
+        if ($host = git.example.com) {
+                return 301 https://$host$request_uri;
+          }
+
+          return 404;
+}
+
+server {
+        server_name git.example.com;
+        listen 443 ssl http2;
+
+        location / {
+                # The final `/` is important.
+                    proxy_pass http://localhost:8763/;
+                add_header X-Frame-Options SAMEORIGIN;
+                add_header X-XSS-Protection "1; mode=block";
+                proxy_redirect off;
+                proxy_buffering off;
+                proxy_set_header Host $host;
+                proxy_set_header X-Real-IP $remote_addr;
+                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+                proxy_set_header X-Forwarded-Proto $scheme;
+                proxy_set_header X-Forwarded-Port $server_port;
+        }
+
+        # INCLUDE ANY SSL CERTS HERE
+        include /etc/letsencrypt/options-ssl-nginx.conf;
+        ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
+}
+#+end_src
+
+Once created, symlink it and restart the web server.
+
+#+begin_src sh
+sudo ln -s /etc/nginx/sites-available/git.example.com /etc/nginx/sites-enabled/
+sudo systemctl restart nginx.service
+#+end_src
+
+As we can see below, my site at =git.example.com= is available and
+running:
+
+** Settings Up Git Details
+Once you have =cgit= running, you can add some small details, such as
+repository owners and descriptions by editing the following files within
+each repository.
+
+Alternatively, you can use the =cgitrc= file to edit these details if
+you only care to edit them for the purpose of seeing them on your
+website.
+
+The =description= file within the repository on your server will display
+the description online.
+
+#+begin_src sh
+cd /git/example.git
+nano description
+#+end_src
+
+You can add a =[gitweb]= block to the =config= file in order to display
+the owner of the repository.
+
+#+begin_src sh
+cd /git/example.git
+nano config
+#+end_src
+
+#+begin_src conf
+[gitweb]
+    owner = "YourName"
+#+end_src
+
+Note that you can ignore the configuration within each repository and
+simply set up this information in the =cgitrc= file, if you want to do
+it that way.
+
+** Editing =cgit=
+In order to edit certain items within =cgit=, you need to edit the
+=cgitrc= file.
+
+#+begin_src sh
+nano ~/cgit/cgitrc
+#+end_src
+
+Below is an example configuration for =cgitrc=. You can find all the
+configuration options within the [configuration manual]
+([[https://git.zx2c4.com/cgit/plain/cgitrc.5.txt]]).
+
+#+begin_src conf
+css=/cgit.css
+logo=/logo.png
+favicon=/favicon.png
+robots=noindex, nofollow
+
+enable-index-links=1
+enable-commit-graph=1
+enable-blame=1
+enable-log-filecount=1
+enable-log-linecount=1
+enable-git-config=1
+
+clone-url=git://git.example.com/$CGIT_REPO_URL ssh://git@git.example.com:/git/$CGIT_REPO_URL
+
+root-title=My Git Website
+root-desc=My personal git repositories.
+
+# Allow download of tar.gz, tar.bz2 and zip-files
+snapshots=tar.gz tar.bz2 zip
+
+##
+## List of common mimetypes
+##
+mimetype.gif=image/gif
+mimetype.html=text/html
+mimetype.jpg=image/jpeg
+mimetype.jpeg=image/jpeg
+mimetype.pdf=application/pdf
+mimetype.png=image/png
+mimetype.svg=image/svg+xml
+
+# Highlight source code
+# source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.sh
+source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py
+
+# Format markdown, restructuredtext, manpages, text files, and html files
+# through the right converters
+about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh
+
+##
+## Search for these files in the root of the default branch of repositories
+## for coming up with the about page:
+##
+readme=:README.md
+readme=:readme.md
+readme=:README.mkd
+readme=:readme.mkd
+readme=:README.rst
+readme=:readme.rst
+readme=:README.html
+readme=:readme.html
+readme=:README.htm
+readme=:readme.htm
+readme=:README.txt
+readme=:readme.txt
+readme=:README
+readme=:readme
+
+# Repositories
+
+# Uncomment the following line to scan a path instead of adding repositories manually
+# scan-path=/git
+
+## Test Section
+section=git/test-section
+
+repo.url=test.git
+repo.path=/git/test.git
+repo.readme=:README.md
+repo.owner=John Doe
+repo.desc=An example repository!
+#+end_src
+
+** Final Fixes: Syntax Highlighting & README Rendering
+After completing my initial install and playing around with it for a few
+days, I noticed two issues:
+
+1. Syntax highlighting did not work when viewing the source code within
+   a file.
+2. The =about= tab within a repository was not rendered to HTML.
+
+The following process fixes these issues. To start, let's go to the
+=cgit= directory where we were editing our configuration file earlier.
+
+#+begin_src sh
+cd ~/cgit
+#+end_src
+
+In here, create two folders that will hold our syntax files:
+
+#+begin_src sh
+mkdir filters && mkdir filters/html-converters && cd filters
+#+end_src
+
+Next, download the default filters:
+
+#+begin_src sh
+curl https://git.zx2c4.com/cgit/plain/filters/about-formatting.sh > about-formatting.sh
+chmod 755 about-formatting.sh
+curl https://git.zx2c4.com/cgit/plain/filters/syntax-highlighting.py > syntax-highlighting.py
+chmod 755 syntax-highlighting.py
+#+end_src
+
+Finally, download the HTML conversion files you need. The example below
+downloads the Markdown converter:
+
+#+begin_src sh
+cd html-converters
+curl https://git.zx2c4.com/cgit/plain/filters/html-converters/md2html > md2html
+chmod 755 md2html
+#+end_src
+
+If you need other filters or html-converters found within
+[[https://git.zx2c4.com/cgit/tree/filters][the cgit project files]],
+repeat the =curl= and =chmod= process above for whichever files you
+need.
+
+However, formatting will not work quite yet since the Docker cgit
+container we're using doesn't have the formatting package installed. You
+can install this easily by install Python 3+ and the =pygments= package:
+
+#+begin_src sh
+# Enter the container's command line
+sudo docker exec -it cgit bash
+#+end_src
+
+#+begin_src sh
+# Install the necessary packages and then exit
+yum update -y &&                      \
+yum upgrade -y &&                     \
+yum install python3 python3-pip -y && \
+pip3 install markdown pygments &&     \
+exit
+#+end_src
+
+*You will need to enter the cgit docker container and re-run these =yum=
+commands every time you kill and restart the container!*
+
+If not done already, we need to add the following variables to our
+=cgitrc= file in order for =cgit= to know where our filtering files are:
+
+#+begin_src conf
+# Highlight source code with python pygments-based highlighter
+source-filter=/var/www/htdocs/cgit/filters/syntax-highlighting.py
+
+# Format markdown, restructuredtext, manpages, text files, and html files
+# through the right converters
+about-filter=/var/www/htdocs/cgit/filters/about-formatting.sh
+#+end_src
+
+Now you should see that syntax highlighting and README rendering to the
+=about= tab is fixed.
+
+** Theming
+I won't go into much detail in this section, but you can fully theme
+your installation of =cgit= since you have access to the =cgit.css= file
+in your web root. This is another file you can add as a volume to the
+=docker-compose.yml= file if you want to edit this without entering the
+container's command line.
+
+*** :warning: Remember to Back Up Your Data!
+The last thing to note is that running services on your own equipment
+means that you're assuming a level of risk that exists regarding data
+loss, catastrophes, etc. In order to reduce the impact of any such
+occurrence, I suggest backing up your data regularly.
+
+Backups can be automated via =cron=, by hooking your base directory up
+to a cloud provider, or even setting up hooks to push all repository
+info to git mirrors on other git hosts. Whatever the method, make sure
+that your data doesn't vanish in the event that your drives or servers
+fail.