Update builds.sr.hts installation/config pages

3 files changed, 195 insertions, 159 deletions

diff --git a/builds.sr.ht/configuration.md b/builds.sr.ht/configuration.md
new file mode 100644
index 0000000..aa14291
--- /dev/null
+++ b/builds.sr.ht/configuration.md
@@ -0,0 +1,171 @@
+---
+title: builds.sr.ht Configuration
+---
+
+This document covers the configuration process for builds.sr.ht.
+
+# Security Model
+
+Let's start with a brief overview of the security model of builds.sr.ht.
+Since builds.sr.ht runs arbitrary user code (and allows users to utilize
+root), it's important to carefully secure the build environments.
+
+To that end, our build jobs run in a sandbox that consists of:
+
+- A [KVM](https://www.linux-kvm.org) virtual machine (via
+  [QEMU](https://www.qemu.org)), which
+- runs inside of an otherwise empty Docker image, which
+- runs as an unprivledged user on
+- a server that is physically separate from anything important, uses its own
+  isolated Redis instance, and has minimal database access.
+
+We suggest you take similar precautions if your servers may run untrusted
+builds.
+
+<div class="alert alert-warning">
+  <strong>Warning:</strong> Even if you only build your own software,
+  integration with other services may cause you to run untrusted builds (e.g.
+  automatic testing of patches via lists.sr.ht).
+</div>
+
+# Master Server
+
+## Web Service
+
+The master server requires *two* Redis servers – one that runners should have
+access to, and one that they should not have access to. For the former,
+insert connection details into build.sr.ht's configuration file under the `redis`
+key.
+
+Each runner also requires a local Redis instance running.
+
+<div class="alert alert-info">
+  <strong>Note:</strong> In a deployment where all services are on the same
+  server, running only trusted builds, you can get away with a single Redis
+  instance.
+</div>
+
+# Database
+
+Create two users, one for the master and one for the runners (or one for each
+runner if you prefer). They need the following permissions:
+
+- **master** should have ownership over the database and full read/write/alter
+  access to all table
+- **runner** should have read/write access to the job table and artifact table.
+
+If you are running the master and runners on the same server, you will only be
+able to use one user - the master user. Configure both the web service and build
+runner with this account. Otherwise, two separate accounts is recommended.
+
+Note: in the future runners will not have database access.
+
+# Install images
+
+On the runner, install the `builds.sr.ht-images` package (if building from
+source, this package is simply the `images` directory copied to
+`/var/lib/images`), as well as docker. Build the docker image like so:
+
+    $ cd /var/lib/images
+    $ docker build -t qemu -f qemu/Dockerfile .
+
+This will build a docker image named `qemu` which contains a statically linked
+build of qemu and nothing else.
+
+## Bootstrapping our images
+
+A `genimg` script is provided for each image which can be run from a working
+image of that guest to produce a new image. You need to manually prepare a
+working guest of each image type (that is, to build the Arch Linux image you
+need a working Arch Linux installation to bootstrap from). Then you can run
+the provided `genimg` to produce the disk image. You should read the genimg
+script to determine what dependencies need to be installed before it can be
+run to completion.
+
+The directory structure for bootable images should have the format
+images/$distro/$release/$arch/ with the root.img.qcow2 file within the $arch
+directory.
+
+A `build.yml` file is also provided for each image to build itself on your
+build infrastructure once you have it set up, which you should customize as
+necessary. It's recommended that you set up cron jobs to build fresh images
+frequently - a script at `contrib/submit_image_build` is provided for this
+purpose.
+
+**Note**: it is recommended that you modify our `build.yml` files to suit your
+instance's needs, then run it on *our* hosted builds.sr.ht instance to bootstrap
+your images. This is the fastest and most convenient way to bootstrap the images
+you need.
+
+**Note**: You will need nested virtualization enabled in order to build images
+from within a pre-existing build image (i.e. via the `build.yml` file). If you
+run into issues with `modprobe kvm_intel` within the genimg script, you can fix
+this by removing the module and then re-inserting it with `insmod kvm_intel.ko
+nested=1` in the directory containing the kernel module.
+
+## Creating new images
+
+If you require additional images, study the `control` script to understand how
+the top-level boot process works. You should then prepare a disk image for your
+new system (name it `root.img.qcow2`) and write a `functions` file. The only
+required function is `boot`, which should call `_boot` with any additional
+arguments you want to pass to qemu. If your image will boot up with no
+additional qemu arguments, this function will likely just call `_boot`. You can
+optionally provide a number of other functions in your `functions` file to
+enable various features:
+
+- To enable installing packages specified in the build manifest, write an
+  `install` function with the following usage:
+  `install [ssh port] [packages...]`
+- To enable adding third-party package repositories, write an `add_repository`
+  function: `add_repository [ssh port] [name] [source]`. The `source` is usually
+  vendor-specific, you can make this any format you want to encode repo URLs,
+  package signing keys, etc.
+
+In order to run builds, we require the following:
+
+- The disk should be able to boot itself up, make sure to install a bootloader
+  and set up partitions however you like.
+- Networking configured with IPv4 address `10.0.2.15/25` and gateway `10.0.2.2`.
+  Don't forget to configure DNS, too.
+- SSH listening on port 22 (the standard port) with passwordless login *enabled*
+- A user named `build` to log into SSH with, preferrably with uid 1000
+- git config setting user.name to builds.sr.ht and user.email to builds@sr.ht
+- Bash (temporary - we'll make this more generic at some point)
+
+Not strictly necessary, but recommended:
+
+- Set the hostname to `build`
+- Configure NTP and set the timezone to UTC
+- Add the build user to the sudoers file with `NOPASSWD: ALL`
+- In your `functions` file, set `poweroff_cmd` to a command we can SSH into the
+  box and use to shut off the machine. If you don't, we'll just kill the qemu
+  process.
+- It is also recommended to write a `sanity_check` function which takes no
+  arguments, but boots up the image and runs any tests necessary to verify
+  everything is working and return a nonzero status code if not.
+
+You will likely find it useful to read the scripts for existing build images as
+a reference. Once you have a new image, email the scripts to
+[`~sircmpwn/sr.ht-dev@lists.sr.ht`](https://lists.sr.ht/~sircmpwn/sr.ht-dev) so
+we can integrate them upstream!
+
+# Additional configuration
+
+Write an `/etc/sr.ht/builds.ini` configuration file similar to the one you wrote
+on the master server. Only the `[sr.ht]` and `[builds.sr.ht]` sections are
+required for the runners. `images` should be set to the installation path of
+your images (`/var/lib/images`) and `buildlogs` should be set to the path where
+the runner should write its build logs (the runner user should be able to create
+files and directories here). Set `runner` to the hostname of the build runner.
+You will need to configure nginx to serve the build logs directory at
+http://RUNNER-HOSTNAME/logs/ in order for build logs to appear correctly on the
+website.
+
+Once all of this is done, make sure the worker is compiled (with go 1.11 or
+later) by running `go build` in the worker/ directory, start the
+`builds.sr.ht-worker` service and it's off to the races. Submit builds on the
+master server and they should run correctly at this point.
+
+For SSH access to (failed) builds you will need to install `git.sr.ht` and
+configure `[git.sr.ht::dispatch]` for `buildsrht-keys`.
diff --git a/builds.sr.ht/configuration_reference.md b/builds.sr.ht/configuration_reference.md
new file mode 100644
index 0000000..4b5b5e1
--- /dev/null
+++ b/builds.sr.ht/configuration_reference.md
@@ -0,0 +1,5 @@
+---
+title: builds.sr.ht Configuration Reference
+---
+
+This document covers the configuration options for the builds.sr.ht service.
diff --git a/builds.sr.ht/installation.md b/builds.sr.ht/installation.md
index 2ed6cd1..111bc60 100644
--- a/builds.sr.ht/installation.md
+++ b/builds.sr.ht/installation.md
@@ -1,169 +1,29 @@
 ---
-title: builds.sr.ht installation
+title: builds.sr.ht Installation
 ---
 
-There are two components to builds.sr.ht: the job runner and the master server.
-Typically installations will have one master and many runners distributed on
-many servers, but both can be installed on the same server for small
-installations (though [not without risk](#security-model)). We'll start by
-setting up the master server.
+This document covers the installation steps for builds.sr.ht, a continuous
+integration service.
 
-# Web service
+builds.sr.ht is comprised of two components: a **master server** and **job
+runners**. Typically, deployments have one master and many runners, which are
+distributed across multiple servers.
 
-The master server is a standard sr.ht web service and can be [installed as
-such](/installation.md). However, it is important that you configure *two* Redis
-servers - one that the runners should have access to, and one that they should
-not. Insert connection details for the former into build.sr.ht's configuration
-file under the `redis` key. Each build runner will also need a local redis
-instance running. In an insecure deployment (all services on the same server,
-running only trusted builds), you can get away with a single Redis instance.
+<div class="alert alert-info">
+  <strong>Note:</strong> For smaller deployments, job runners can be installed
+  alongside the master server, but
+  <a href="/builds.sr.ht/configuration.md#security-model"
+  class="alert-link">not without risk</a>.
+</div>
 
-# Security model
+# Installation
 
-Let's start with a brief overview of the security model of builds.sr.ht.
-Because builds.sr.ht runs arbitrary user code (and allows users to utilize
-root), it's important to carefully secure the build environments. To this end,
-builds run in a sandbox which consists of:
+On the master server, install the `builds.sr.ht` package.
 
-- A KVM virtual machine via qemu
-- Inside of an otherwise empty docker image
-- Running as an unprivledged user
-- On a server which is isolated by:
-    - Being on physically separate server from anything important
-    - Using its own isolated redis instance
-    - Having minimal database access
+On each server hosting a job runner, install the `builds.sr.ht-worker` and
+`builds.sr.ht-images` packages.
 
-We suggest you take similar precautions if your servers could be running
-untrusted builds. Remember that if you build only your own software, integration
-with other services could end up running untrusted builds (for example,
-automatic testing of patches via lists.sr.ht).
+## Daemons
 
-# Package installation
-
-On each runner, install the builds.sr.ht-images and builds.sr.ht-worker
-packages.
-
-# Database configuration
-
-Create two users, one for the master and one for the runners (or one for each
-runner if you prefer). They need the following permissions:
-
-- **master** should have ownership over the database and full read/write/alter
-  access to all table
-- **runner** should have read/write access to the job table and artifact table.
-
-If you are running the master and runners on the same server, you will only be
-able to use one user - the master user. Configure both the web service and build
-runner with this account. Otherwise, two separate accounts is recommended.
-
-Note: in the future runners will not have database access.
-
-# Install images
-
-On the runner, install the `builds.sr.ht-images` package (if building from
-source, this package is simply the `images` directory copied to
-`/var/lib/images`), as well as docker. Build the docker image like so:
-
-    $ cd /var/lib/images
-    $ docker build -t qemu -f qemu/Dockerfile .
-
-This will build a docker image named `qemu` which contains a statically linked
-build of qemu and nothing else.
-
-## Bootstrapping our images
-
-A `genimg` script is provided for each image which can be run from a working
-image of that guest to produce a new image. You need to manually prepare a
-working guest of each image type (that is, to build the Arch Linux image you
-need a working Arch Linux installation to bootstrap from). Then you can run
-the provided `genimg` to produce the disk image. You should read the genimg
-script to determine what dependencies need to be installed before it can be
-run to completion.
-
-The directory structure for bootable images should have the format
-images/$distro/$release/$arch/ with the root.img.qcow2 file within the $arch
-directory.
-
-A `build.yml` file is also provided for each image to build itself on your
-build infrastructure once you have it set up, which you should customize as
-necessary. It's recommended that you set up cron jobs to build fresh images
-frequently - a script at `contrib/submit_image_build` is provided for this
-purpose.
-
-**Note**: it is recommended that you modify our `build.yml` files to suit your
-instance's needs, then run it on *our* hosted builds.sr.ht instance to bootstrap
-your images. This is the fastest and most convenient way to bootstrap the images
-you need.
-
-**Note**: You will need nested virtualization enabled in order to build images
-from within a pre-existing build image (i.e. via the `build.yml` file). If you
-run into issues with `modprobe kvm_intel` within the genimg script, you can fix
-this by removing the module and then re-inserting it with `insmod kvm_intel.ko
-nested=1` in the directory containing the kernel module.
-
-## Creating new images
-
-If you require additional images, study the `control` script to understand how
-the top-level boot process works. You should then prepare a disk image for your
-new system (name it `root.img.qcow2`) and write a `functions` file. The only
-required function is `boot`, which should call `_boot` with any additional
-arguments you want to pass to qemu. If your image will boot up with no
-additional qemu arguments, this function will likely just call `_boot`. You can
-optionally provide a number of other functions in your `functions` file to
-enable various features:
-
-- To enable installing packages specified in the build manifest, write an
-  `install` function with the following usage:
-  `install [ssh port] [packages...]`
-- To enable adding third-party package repositories, write an `add_repository`
-  function: `add_repository [ssh port] [name] [source]`. The `source` is usually
-  vendor-specific, you can make this any format you want to encode repo URLs,
-  package signing keys, etc.
-
-In order to run builds, we require the following:
-
-- The disk should be able to boot itself up, make sure to install a bootloader
-  and set up partitions however you like.
-- Networking configured with IPv4 address `10.0.2.15/25` and gateway `10.0.2.2`.
-  Don't forget to configure DNS, too.
-- SSH listening on port 22 (the standard port) with passwordless login *enabled*
-- A user named `build` to log into SSH with, preferrably with uid 1000
-- git config setting user.name to builds.sr.ht and user.email to builds@sr.ht
-- Bash (temporary - we'll make this more generic at some point)
-
-Not strictly necessary, but recommended:
-
-- Set the hostname to `build`
-- Configure NTP and set the timezone to UTC
-- Add the build user to the sudoers file with `NOPASSWD: ALL`
-- In your `functions` file, set `poweroff_cmd` to a command we can SSH into the
-  box and use to shut off the machine. If you don't, we'll just kill the qemu
-  process.
-- It is also recommended to write a `sanity_check` function which takes no
-  arguments, but boots up the image and runs any tests necessary to verify
-  everything is working and return a nonzero status code if not.
-
-You will likely find it useful to read the scripts for existing build images as
-a reference. Once you have a new image, email the scripts to
-[`~sircmpwn/sr.ht-dev@lists.sr.ht`](https://lists.sr.ht/~sircmpwn/sr.ht-dev) so
-we can integrate them upstream!
-
-# Additional configuration
-
-Write an `/etc/sr.ht/builds.ini` configuration file similar to the one you wrote
-on the master server. Only the `[sr.ht]` and `[builds.sr.ht]` sections are
-required for the runners. `images` should be set to the installation path of
-your images (`/var/lib/images`) and `buildlogs` should be set to the path where
-the runner should write its build logs (the runner user should be able to create
-files and directories here). Set `runner` to the hostname of the build runner.
-You will need to configure nginx to serve the build logs directory at
-http://RUNNER-HOSTNAME/logs/ in order for build logs to appear correctly on the
-website.
-
-Once all of this is done, make sure the worker is compiled (with go 1.11 or
-later) by running `go build` in the worker/ directory, start the
-`builds.sr.ht-worker` service and it's off to the races. Submit builds on the
-master server and they should run correctly at this point.
-
-For SSH access to (failed) builds you will need to install `git.sr.ht` and
-configure `[git.sr.ht::dispatch]` for `buildsrht-keys`.
+- `builds.sr.ht` - The web service (master server).
+- `builds.sr.ht-worker` - The job runner.