1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
|
---
title: Installation
---
sr.ht is a distributed system. Each service, such as git.sr.ht and builds.sr.ht,
runs independently, with its own database and resources. They communicate with
each other using mainly their respective APIs and webhooks, and are mostly
tolerant to the temporary or permanent absence of their peers. The system is
fairly complex, and is designed more for deployment at scale than for
small-scale installations. However, you can choose a subset of these services to
install as your needs demand, and it will work correctly.
Developers wishing to hack on or contribute to sr.ht, see
[hacking.md](/hacking.md).
This page documents details common to the installation of all packages, but many
services have specific needs. Specific details to each service are available on
each service's installation page:
- [builds.sr.ht](/builds.sr.ht/installation.md)
- [dispatch.sr.ht](/dispatch.sr.ht/installation.md)
- [git.sr.ht](/git.sr.ht/installation.md)
- [hg.sr.ht](/hg.sr.ht/installation.md)
- [lists.sr.ht](/lists.sr.ht/installation.md)
- [man.sr.ht](/man.sr.ht/installation.md)
- [meta.sr.ht](/meta.sr.ht/installation.md)
- [paste.sr.ht](/paste.sr.ht/installation.md)
- [todo.sr.ht](/todo.sr.ht/installation.md)
All sr.ht sysadmins are encouraged to subscribe to the
[sr.ht-admins](https://lists.sr.ht/~sircmpwn/sr.ht-admins) mailing list, a
low-volume list of sysadmin-oriented announcements regarding breaking changes,
security vulnerabilities, and so on.
# Packages
The only supported installation is Alpine Linux hosts using the SourceHut
[Alpine Linux package repository](/packages.md). There are also
community-maintained SourceHut distributions for [Debian and Arch
Linux](/packages.md), which are preferred before installing SourceHut from
source.
SourceHut is still in alpha, and has no stable releases. If you are interested
in packaging it for your distribution, please reach out to us for help in
automating the maintenance of a third-party distribution which is automatically
updated following our (frequent) upstream rolling releases. Please refrain from
packaging SourceHut for your upstream distribution repositories until we have
shipped stable versions of our software.
Packages also ship with the correct users and groups rigged up, and with
whichever daemon configurations are particular to your distribution's init
system (e.g. OpenRC scripts or systemd units).
# Prerequisites
Generally speaking, most services require the following services:
- A PostgreSQL server: persistent storage
- A Redis server: ephemeral storage, caching, work distribution
- A mail server: incoming and outgoing mail
- A cron daemon: running scheduled tasks
Additionally, many services are able to integrate with some optional tools:
- [Prometheus](https://prometheus.io/) for monitoring
- Any S3-compatible object storage, such as [Minio](https://min.io/) or
[Ceph](https://ceph.io/)
# Installing SourceHut services
These instructions apply generally to all services. Consult service-specific
documentation for amendments to these procedures.
## sr.ht core
[core.sr.ht](https://git.sr.ht/~sircmpwn/core.sr.ht) is a Python package which
provides common functionality for all services. For users are installing from
packages, it is not necessary to concern yourself with this, as it is
automatically pulled in as a dependency. For users building from source, you
must build this package first. Be aware that the dependencies in `setup.py` are
not generally kept up-to-date, consult the latest
[Alpine package][alpine package] for an up-to-date and comprehensive list.
[alpine package]: https://git.sr.ht/~sircmpwn/sr.ht-apkbuilds/tree/master/sr.ht/py3-srht/APKBUILD
## meta.sr.ht
meta.sr.ht is the only service which is required. It is responsible for storing
your account details and profile, handling logins, storing SSH and PGP keys,
keeping a secure audit log, and providing some management interfaces for
administrators.
Consult the [meta.sr.ht installation guide](/meta.sr.ht/installation.md) for
details on the installation procedure.
## Service configuration
All services use a shared configuration file at `/etc/sr.ht/config.ini`. Each
site provides an example configuration in `config.example.ini` in their
respective source code repositories. It is the administrator's responsibility to
consult these examples to produce a unified configuration file which is
applicable to all of the services which they intend to operate.
Some configuration options are applicable to all services.
### Service keys
Encryption is used throughout SourceHut to encrypt and validate communications.
There are two main keys which you need to generate: service keys, the network
key, and the webhook key.
Service keys are used to encrypt session cookies, and if you configure
load-balancing, it must be consistent between all nodes of that service. They
can be generated with `srht-keygen service`.
The network key needs to be consistent throughout all services and nodes in your
sr.ht installation. It is used to encrypt and sign internal communications
between services. This key is generated with `srht-keygen network`.
The webhook key is also consistent throughout all services and nodes, but is an
asymmetric key. It is used to sign webhook payloads to validate the authenticity
of the request. You can generate this with `srht-keygen webhook`, store the
private key in your config file and distribute to the public key to any parties
interested in authenticating webhook payloads from your services. This is also
used to validate webhooks used internally; it is not optional.
### Mail configuration
Outgoing emails from sr.ht are configured in the `[mail]` section. If you fill
out the `error-to` address, the services will send exceptions to this address
for diagnostics. You must also generate a PGP key (without a password) for sr.ht
services to sign outgoing emails with (and optionally encrypt emails to each
recipient).
## Database Configuration
Each service requires its own database, though they can co-exist on the same
server. It is also recommended to give each service its own database login, with
full access rights to that database so that it may manage its own schema
migrations.
After you populate your `config.ini`'s connection string, you may use the
`[module]-initdb` script to populate the schema and stamp the latest revision
for migrations, for example `metasrht-initdb` to set up meta.sr.ht's database.
### Schema Upgrades
We use [alembic](http://alembic.zzzcomputing.com/en/latest/) to manage schema
migrations. We use custom scripts to automatically retrieve database credentials
from your main sr.ht config file. If you have installed from distribution
packages, set `[service] migrate-on-upgrade=yes` (where service is e.g.
`[meta.sr.ht]`) to have migrations automatically performed during normal service
upgrades.
Otherwise, you may use `srht-migrate <service> upgrade head` to run updates for
core.sr.ht migrations, and `<service>-migrate upgrade head` to run
service-specific upgrades. For example, to upgrade the database schema for
git.sr.ht, run `srht-migrate git.sr.ht upgrade head`, then `gitsrht-migrate
upgrade head`. Other alembic commands are available, use `gitsrht-migrate
--help` for more information.
## Upgrade procedure
1. Stop all services
2. Run your distro's update commands (e.g. `apk update && apk upgrade`)
3. Resume all services
We source your alembic config from your main sr.ht `config.ini` -
no need to write an `alembic.ini` file. Run `srht-migrate <service> stamp head
&& <service>-migrate stamp head` once to tell alembic the schema is up-to-date
(e.g. `srht-migrate man.sr.ht stamp head && mansrht-migrate stamp head`).
Future upgrades will be managed automatically by the package, or if you're
using the source code you can run `srht-migrate <service> upgrade head &&
<service>-migrate upgrade head` when you pull the latest version down.
## Start the daemons
Daemon configuration is provided in the distribution packages. Use the service
manager appropriate to your distro to start the daemons, e.g. `service git.sr.ht
start` or `systemctl start git.sr.ht`.
## Proxy configuration
The exact configuration you use will vary depending on your needs. Here is an
example nginx configuration for meta.sr.ht:
server {
listen 80;
server_name meta.sr.ht;
location / {
return 302 https://$server_name$request_uri;
}
location ^~ /.well-known {
root /var/www;
}
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name meta.sr.ht;
client_max_body_size 100M;
ssl_certificate /etc/ssl/uacme/meta.sr.ht/cert.pem;
ssl_certificate_key /etc/ssl/uacme/private/meta.sr.ht/key.pem;
location / {
proxy_pass http://127.0.0.1:5002;
}
location /static {
root /usr/lib/python3.6/site-packages/metasrht;
}
}
Once the proxy is configured, you should be able to access your new service.
## OAuth configuration
For services other than meta.sr.ht, you have to create and configure an OAuth
client before users can log into your service. To do this, visit your profile
on your meta.sr.ht instance, select the OAuth tab and register a new client.
Append the path `/oauth/callback` to the URL of your service instance and
choose this value as the base redirect URI (for example
`https://git.sr.ht/oauth/callback`). Update your service configuration file with
the generated client ID and secret.
You then need to configure that client as "preauthorized", i.e. first-party.
This skips the OAuth consent screen that third-party applications are subject to
and enables some extra API features. Log into the meta.sr.ht database and run
the following query (adjusting the client ID as appropriate):
```sql
update oauthclient set preauthorized = true where client_id = 'your client ID';
```
Now you should be able to log into your new service.
Alternatively, you can generally run most SourceHut services by pointing them at
*our* meta.sr.ht instance and registering them as an OAuth client. For example,
you could run a custom builds.sr.ht instance which logs in with hosted
meta.sr.ht accounts by registering for an OAuth client there and skipping the
preauthorized step. YMMV.
|
