# hg.sr.ht security incident 2021-11-16
On November 16th, we were alerted to a security vulnerability in hg.sr.ht.
Unauthenticated users could clone private hg repositories via https, which is
normally not permitted, provided they knew the URL in advance.
During the period in which hg.sr.ht was vulnerable, we identified 24 private Hg
repositories which had been cloned in this way, representing a total of 5 users,
from 7 IP addresses. We believe that this activity was benign, but have notified
the affected users nevertheless.
Third-party instances of hg.sr.ht are not expected to be affected by this issue.
## Full details
git.sr.ht serves git repositories over HTTPs via nginx for improved performance.
To secure private repositories, we use the nginx "auth_request" feature to make
an HTTP request to our web service to determine if the clone should be
permitted. The configuration looks like this:
```
location = /authorize {
proxy_pass http://127.0.0.1:5001;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
}
location ~ ^/([^/]+)/([^/]+)/(HEAD|info/refs|objects/info/.*|git-upload-pack).*$ {
auth_request /authorize;
# ...
```
However, hg.sr.ht uses a somewhat different approach. HTTP requests from the
Mercurial client are routed directly to the web application, which runs a shim
over hgweb to service the request. This shim also calls the authorize endpoint,
which is implemented in code shared between git.sr.ht and hg.sr.ht, to determine
if it should allow the clone to proceed. The relevant hg.sr.ht code is:
```
def _get_hgweb_config(self, req):
# Check that this repository is authorized for public https
# cloning. The authorize endpoint is implemented in scmsrht.
auth_endpoint = req.url_root + 'authorize'
headers = {'X-Original-URI': req.url}
auth_resp = requests.get(auth_endpoint, headers=headers)
if auth_resp.status_code != 200:
return None
```
On October 27th, as part of a broader initiative to bring our nginx
configurations under version control, the configuration details for git.sr.ht's
authorize endpoint were copied into hg.sr.ht's nginx configuration. Because we
do not use the "auth_request" feature in hg.sr.ht, this was not necessary.
Moreover, it introduced a vulnerability by causing nginx to overwrite the
X-Original-URI header set by this Python code to "/authorize" -- the value of
$request_uri. The authorize endpoint is designed to permit access for any routes
other than a repository, which caused all requests to be permitted regardless of
the URI.
## Mitigations
The issue was corrected by removing the erroneously copied authorize route from
the hg.sr.ht nginx configuration.
To catch similar issues sooner, we have also added a cronjob which periodically
attempts to clone private repositories without authorization. Should it succeed,
operators will be notified.
To identify potentially affected users, we consulted HTTP logs from the
vulnerable time period for hg clone requests. We identified all Hg repositories
which had been cloned over HTTP in this period, narrowed the list down to
private repositories, and sent the following email to the owners of these
repositories:
```
To: $user
From: Drew DeVault <sir@cmpwn.com>
Subject: Notice of potential disclosure of private repository data on hg.sr.ht
It has come to our attention that your private repositories on hg.sr.ht
may have been affected by a recent vulnerability in hg.sr.ht. Assuming
they knew the URL in advance, this vulnerability allowed arbitrary users
to clone your private hg.sr.ht repositories.
During the time period when this vulnerability was present, we recorded
HTTPs clone activity for the following private repositories on your
account:
$repos
The following IP addresses were implicated in this activity:
$addresses
If one or more of these addresses is not associated with your own
activity, it is possible that someone has obtained a copy of your
private repository.
I apologise for the oversight. If you have any questions, please reply
to this email and I will answer them to the best of my ability.
```
## Timeline
**2021-10-27**: Updates to our nginx configuration introduce the vulnerability.
**2021-11-16 9:04 PM**: User reaches out to us to responsibly disclose the issue.
**2021-11-16 9:26 PM**: SourceHut confirms the bug, starts investigation.
**2021-11-16 9:48 PM**: A fix is identified and applied. SourceHut begins to
identify potentially affected users.
**2021-11-16 10:54 PM**: Potentially affected users are notified.
