path: root/ops/hg.sr.ht-migration.md

hg.sr.ht migration plan

1. Build & install the new server (sakuya1) as VM host gen 2
1. Announce planned outage a week in advance
1. Spin up an hg.sr.ht stack and restore from the last backup. This is a
   good opportunity to test our backups in action.
1. Test everything!
1. Set up pgbouncer on hg.sr.ht¹
1. Copy over host keys from hg.sr.ht¹ to hg.sr.ht²
1. Move root image back to ZFS on sakuya1
1. Await planned outage date
1. Set hg.sr.ht¹ to read-only mode (via pgbouncer, probably, and disable
   the hg SSH login account)
1. rsync any changes which have occured between steps 3 and 6 to
   hg.sr.ht²
1. Cut DNS over to hg.sr.ht² and monitor as users get transferred over, reset
   TTL to default
1. Disable cronjobs on hg.sr.ht¹
1. Monitor hg.sr.ht¹ and shut it off when traffic is more or less done
   hitting it
1. Remove backup credentials for hg.sr.ht¹ from konapku.sr.ht
1. Remove SQL access from pg_hba.conf for hg.sr.ht¹
1. Finish configuration of acme cronjob for SSL certs on hg.sr.ht²
1. Wait 2 weeks and then decommission hg.sr.ht¹

Things to double check on hg.sr.ht²:

- Is monitoring working? Double check node exporter
- Are backups working?
- Are ZFS snapshots being taken correctly?
- Are ZFS scrubs being run? Double check on the 1st