Restore the previous behaviour where the Prometheus /metrics endpoint
required auth if auth was enabled.
A new -prometheus-no-auth flag allows you to override this and disable
auth for that specific endpoint.
- Helper method for internal server errors with consistent logging.
- Add PanicOnError option to panic on internal server errors. This
makes it easier to traces where the condition was hit in testing.
- Do not allow '.' as path component, because it undermines depth
checks, and add tests
- Fix GiB reporting
- Fix metrics label
- Helper function for http errors
This contains all the glue to make Server use the new repo.Handler:
- Remove all old handlers
- Add ServeHTTP to make Server a single http.Handler
- Remove Goji routing and replace by net/http and custom routing logic
Additionally, this implements two-level backup repositories.
Refactor the old HTTP handlers to fit the purpose of the new
http.Handler:
- repo.New function to instantiate a handler for a single repo (can be done
dynamically for every request)
- Single ServeHTTP entrypoint
- Move quota management to two methods that will be implemented later
(stubs for now)
- Move metrics update to an external function (BlobMetricFunc type)
- Use constants and options for file modes
Goji routes incoming requests without first URL decoding the path, so
'%2F' in a URL will not be decoded to a '/' before routing. But by the
time that we perform the path checks for private urls on r.URL.Path,
these characters have been decoded.
As a consequence, a user 'foo' could use 'foo%2Fbar' as the repo name.
The private repo check would see that the path starts with 'foo/' and
allow it, and rest-server would happily create a 'foo/bar' repo. Other
more harmful variants are possible.
To resolve this issue, we now reject any name part that contains a '/'.
Additionally, we immediately reject a few other characters that are
disallowed under some operating systems or filesystems.
In order to prevent users from accidentally exposing rest-server without
authentication, rest-server now defaults to requiring a .htpasswd. If
you want to disable authentication, you need to explicitly pass the new
--no-auth flag.
Exposes a few metrics for Prometheus under /metrics if started with --prometheus.
Example:
# HELP rest_server_blob_read_bytes_total Total number of bytes read from blobs
# TYPE rest_server_blob_read_bytes_total counter
rest_server_blob_read_bytes_total{repo="test",type="data"} 2.13557024e+09
rest_server_blob_read_bytes_total{repo="test",type="index"} 1.198653e+06
rest_server_blob_read_bytes_total{repo="test",type="keys"} 5388
rest_server_blob_read_bytes_total{repo="test",type="locks"} 1975
rest_server_blob_read_bytes_total{repo="test",type="snapshots"} 10018
# HELP rest_server_blob_read_total Total number of blobs read
# TYPE rest_server_blob_read_total counter
rest_server_blob_read_total{repo="test",type="data"} 3985
rest_server_blob_read_total{repo="test",type="index"} 21
rest_server_blob_read_total{repo="test",type="keys"} 12
rest_server_blob_read_total{repo="test",type="locks"} 12
rest_server_blob_read_total{repo="test",type="snapshots"} 32
# HELP rest_server_blob_write_bytes_total Total number of bytes written to blobs
# TYPE rest_server_blob_write_bytes_total counter
rest_server_blob_write_bytes_total{repo="test",type="data"} 1.063726179e+09
rest_server_blob_write_bytes_total{repo="test",type="index"} 395586
rest_server_blob_write_bytes_total{repo="test",type="locks"} 1975
rest_server_blob_write_bytes_total{repo="test",type="snapshots"} 1933
# HELP rest_server_blob_write_total Total number of blobs written
# TYPE rest_server_blob_write_total counter
rest_server_blob_write_total{repo="test",type="data"} 226
rest_server_blob_write_total{repo="test",type="index"} 6
rest_server_blob_write_total{repo="test",type="locks"} 12
rest_server_blob_write_total{repo="test",type="snapshots"} 6
If htpasswd was modified, it will be automatically reloaded. This check
happens at most once per second and only on incoming requests.
Note that this removes the public `NewHtpasswd()` function.
Konrad Wojas