Add shared installation documentation

3 files changed, 198 insertions, 85 deletions

diff --git a/builds.sr.ht/installation.md b/builds.sr.ht/installation.md
index b7201f6..7eb0fe0 100644
--- a/builds.sr.ht/installation.md
+++ b/builds.sr.ht/installation.md
@@ -4,14 +4,22 @@ many servers, but both can be installed on the same server for small
 installations (though [not without risk](#security-model)). We'll start by
 setting up the master server.
 
-# Prerequisites
+# Web service
 
-- [meta.sr.ht](/meta.sr.ht/installation.md)
-- [An OAuth client ID and secret for meta.sr.ht](https://meta.sr.ht/oauth)
-- PostgreSQL server
-- Two redis servers
+The master server is a standard sr.ht web service and can be [installed as
+such](/installation.md). However, it is important that you configure *two* Redis
+servers - one that the runners should have access to, and one that they should
+not. Insert connection details for the former into build.sr.ht's configuration
+file under the `redis` key. Each build runner will also need a local redis
+instance running. In an insecure deployment (all services on the same server)
+you can get away with a single Redis instance.
 
-## Security model
+We suggest using an SSH tunnel to share the slave Redis instance between job
+runners and the master server, but you can use any method you prefer. If you use
+an SSH tunnel, you will likely want to use a reverse tunnel initiated from the
+master server, so the slaves are unable to SSH into the master server.
+
+# Security model
 
 Let's start with a brief overview of the security model of builds.sr.ht.
 Because builds.sr.ht runs arbitrary user code (and allows users to utilize
@@ -31,91 +39,26 @@ untrusted builds. Remember that if you build only your own software, integration
 with other services could end up running untrusted builds (for example,
 automatic testing of patches via lists.sr.ht).
 
-## PostgreSQL configuration
+# Package installation
 
-Start by setting your config.ini's connection string to a superuser, then run
-the following commands to create the initial schema:
+On each runner, install the builds.sr.ht and builds.sr.ht-images packages.
 
-    $ python3
-    >>> from buildsrht.app import db
-    >>> db.create()
+# Database configuration
 
-Then create two users, one for the master and one for the runners (or one for
-each runner if you prefer). They need the following permissions:
+Create two users, one for the master and one for the runners (or one for each
+runner if you prefer). They need the following permissions:
 
 - **master** should have ownership over the database and full read/write/alter
   access to all table
 - **runner** should have read/write access to the jobs table
 
 If you are running the master and runners on the same server, you will only be
-able to use one user.
+able to use one user - the master user. Configure both the web service and build
+runner with this account. Otherwise, two separate accounts is recommended.
 
 Note: in the future runners will not have database access.
 
-## Redis configuration
-
-The master server should have two redis servers, one for its own use (as part of
-the standard sr.ht configuration) and one for communication with the runners.
-Runners should be provided access to only their redis instance - if distributed,
-we recommend setting up an SSH tunnel from the master to the runner to offer
-access to the appropriate Redis instance.
-
-## Package installation
-
-On the master server, install builds.sr.ht either [from
-source](https://git.sr.ht/~sircmpwn/builds.sr.ht) or [from
-packages](/packages.md). On the runners, install builds.sr.ht and
-builds.sr.ht-images, the latter containing templates for our upstream build
-images and glue code for running them.
-
-Place a config file in `/etc/sr.ht/builds.ini` based on
-[config.ini.example](https://git.sr.ht/~sircmpwn/builds.sr.ht/tree/config.ini.example).
-[Register an OAuth client](https://meta.sr.ht/oauth) and include its client ID
-and secret in the config file. Under the `[builds.sr.ht]` section, configure
-`redis` on the master to point to the *runner* redis - it will use this to
-submit jobs to the runners. The other options are relevant to the build runner,
-which we will set up later.
-
-Start builds.sr.ht (`systemctl start builds.sr.ht` or `service builds.sr.ht
-start`) and configure a reverse proxy - here's our nginx configuration for
-builds.sr.ht upstream:
-
-    server {
-        listen 80;
-        server_name builds.sr.ht;
-
-        location / {
-            return 302 https://$server_name$request_uri;
-        }
-
-        location ^~ /.well-known {
-            root /var/www;
-        }
-    }
-
-    server {
-        listen 443 ssl;
-        listen [::]:443 ssl;
-        server_name builds.sr.ht;
-        client_max_body_size 100M;
-        ssl_certificate /var/lib/acme/live/builds.sr.ht/fullchain;
-        ssl_certificate_key /var/lib/acme/live/builds.sr.ht/privkey;
-        ssl_trusted_certificate /var/lib/acme/live/builds.sr.ht/fullchain;
-
-        location / {
-            proxy_pass http://127.0.0.1:5002;
-        }
-
-        location /static {
-            root /usr/lib/python3.6/site-packages/buildsrht;
-        }
-
-        location ^~ /.well-known {
-            root /var/www;
-        }
-    }
-
-## Runner setup
+# Install images
 
 On the runner, install the `builds.sr.ht-images` package (if building from
 source, this package is simply the `images` directory copied to
@@ -136,9 +79,10 @@ build of qemu and nothing else. Next, build any of the images you need:
     ... and so on ...
 
 Note: some images may not build on your host system without the installation of
-extra tools, which may not be availalbe. However, all images can be built on
-themselves, and a build.yml file is typically provided which can do the job if
-run in a similar environment as the target image.
+extra tools, which may or may not be available for your host system. However,
+all images can be built on themselves, and a build.yml file is typically
+provided which can do the job if run in a similar environment as the target
+image.
 
 ## Creating new images
 
@@ -150,7 +94,7 @@ should be possible to make virtually any kind of image (including non-Linux
 operating systems - the only hard requirement is an ssh daemon) with enough
 effort.
 
-## Configuring the runner
+# Additional configuration
 
 Write an `/etc/sr.ht/builds.ini` configuration file similar to the one you wrote
 on the master server. Only the `[sr.ht]` and `[builds.sr.ht]` sections are
diff --git a/installation.md b/installation.md
new file mode 100644
index 0000000..f0b37bd
--- /dev/null
+++ b/installation.md
@@ -0,0 +1,162 @@
+Installation of sr.ht web services uses a mostly consistent procedure across the
+network. Specific details to each service are available on each service's
+installation page:
+
+- [build.sr.ht](/builds.sr.ht/installation.md)
+- [git.sr.ht](/git.sr.ht/installation.md)
+- [man.sr.ht](/man.sr.ht/installation.md)
+- [meta.sr.ht](/meta.sr.ht/installation.md)
+- [todo.sr.ht](/todo.sr.ht/installation.md)
+
+General installation instructions follow.
+
+# Packages
+
+sr.ht provides a number of Linux distribution package repositories for your
+use. For details specific to your distribution, see [packages.md](/packages.md).
+Even if you wish to install sr.ht services from source, your distribution may
+not include some of our dependencies and using our package repositories is
+recommended.
+
+Installation from packages will also give you distro-specific daemon services
+and will handle database migrations automatically during system updates.
+
+# Prerequisites
+
+You will need at least:
+
+- A PostgreSQL server
+- A Redis server
+- A mail server
+- A cron daemon
+
+In order to use most sr.ht services.
+
+# sr.ht core
+
+sr.ht core is a Python package that provides shared functionality across all
+sr.ht services. It also contains the default templates and stylesheets that give
+sr.ht a consistent look and feel.
+
+The core package is listed as a dependency of the official `*.sr.ht` packages,
+so installing it explicitly is not necessary if you are using our package
+repositories. If you are not using our packages, obtain the [source
+code](https://git.sr.ht/~sircmpwn/srht) and install it like a typical Python
+package (`./setup.py install --prefix=/usr`). You will need to install its
+dependencies beforehand, for an up-to-date list see [the
+package](https://git.sr.ht/~sircmpwn/sr.ht-pkgbuilds/tree/python-srht/PKGBUILD).
+
+# Package Installation
+
+Packages are named as you would expect: meta.sr.ht is called `meta.sr.ht`. On
+package managers where dots are not permitted in package names, dashes are used.
+Underscores are used on systems where dashes are not permitted.
+
+If installing from source, see [the
+packages](https://git.sr.ht/~sircmpwn/sr.ht-pkgbuilds) for an up-to-date list of
+dependencies and install it like any other Python package: `./setup.py install
+--prefix=/usr`. When installing from source, daemon service files are not
+provided; you must write one appropriate to your system.
+
+# Site Configuration
+
+Each site will search for its config file in two locations: `./config.ini` and
+`/etc/sr.ht/subdomain.ini` (for example, meta.sr.ht looks for
+`/etc/sr.ht/meta.ini`). The exact nature of this config file varies depending on
+which service you're setting up - from source, see `config.ini.example` and from
+packages see `/etc/sr.ht/subdomain.ini.example`.
+
+# Database Configuration
+
+Start by setting your config.ini's connection string to a superuser, then run
+the following commands to create the initial schema:
+
+    $ python3
+    >>> from [module].app import db
+    >>> db.create()
+
+Substitute `[module]` for the specific application's module, such as `metasrht`
+or `buildsrht`. When complete, you may update your connection string to use a
+less privileged user.
+
+## Schema Upgrades
+
+We use [alembic](http://alembic.zzzcomputing.com/en/latest/) to manage schema
+migrations. A sample `alembic.ini` is provided (`alembic.ini.example` from
+source and `/etc/sr.ht/alembic/subdomain.ini.example` from packages), which you
+should modify to suit your needs - you will probably only need to set the
+connection string.
+
+Run `alembic stamp head` once to tell alembic the schema is up-to-date. Future
+upgrades will be managed automatically by the package, or if you're using the
+source code you can run `alembic upgrade head` when you pull the latest version
+down.
+
+# Compile static assets
+
+This step is only necessary for users configuring sr.ht from source. Run `make`
+in the root of the repository to compile static assets.
+
+# Start the service
+
+A service file is included in the packages and uses the same name as the
+package. On Arch Linux, for example, you can run `systemctl enable --now
+meta.sr.ht` to start up meta.sr.ht, or `rc-update add meta.sr.ht && service
+meta.sr.ht start` on Alpine. The service will be running at the port specified
+in its config file, and you must now prepare to proxy to it.
+
+# Proxy configuration
+
+The exact nginx configuration you use will vary depending on your needs. Here is
+an example for meta.sr.ht:
+
+    server {
+        listen 80;
+        server_name meta.sr.ht;
+
+        location / {
+            return 302 https://$server_name$request_uri;
+        }
+
+        location ^~ /.well-known {
+            root /var/www;
+        }
+    }
+
+    server {
+        listen 443 ssl;
+        listen [::]:443 ssl;
+        server_name meta.sr.ht;
+        client_max_body_size 100M;
+        ssl_certificate /var/lib/acme/live/meta.sr.ht/fullchain;
+        ssl_certificate_key /var/lib/acme/live/meta.sr.ht/privkey;
+        ssl_trusted_certificate /var/lib/acme/live/meta.sr.ht/fullchain;
+
+        location / {
+            proxy_pass http://127.0.0.1:5002;
+        }
+
+        location /static {
+            root /usr/lib/python3.6/site-packages/metasrht;
+        }
+
+        location ^~ /.well-known {
+            root /var/www;
+        }
+    }
+
+Once the proxy is configured, you should be able to log into your new service.
+
+# Hacking on sr.ht
+
+If you just want to get the codebase running to hack on it, follow these steps:
+
+1. [Prerequisites](#prerequisites)
+2. [sr.ht core](#sr.ht-core), however, you can simply clone it somewhere and add
+   it to your Python path. Also export `SRHT_PATH=/path/to/sr.ht-core` to use
+   your development repository for generating static assets.
+3. [Site configuration](#site-configuration), but you can just put `config.ini`
+   in the working directory and sr.ht will read it there.
+4. [Database configuration](#database-configuration)
+5. [Compile static assets](#compile-static-assets)
+6. `./run.py` will start the development server.
diff --git a/meta.sr.ht/installation.md b/meta.sr.ht/installation.md
index 90ade71..1ec1224 100644
--- a/meta.sr.ht/installation.md
+++ b/meta.sr.ht/installation.md
@@ -1,2 +1,9 @@
-meta.sr.ht is the only hard requirement of an sr.ht network, which is why it'd
-be great if someone actually wrote installation instructions here.
+meta.sr.ht is the central autentication service and service bus for the sr.ht
+network, and a working meta.sr.ht is a basic requirement of running any other
+sr.ht software.
+
+# Installation
+
+meta.sr.ht is a standard sr.ht web service and can be installed through the
+[standard procedure](/installation.md). Additionally, you will need to set up a
+cron job to run `meta-audit-cleanup` daily.