Compare commits

...

851 Commits

Author SHA1 Message Date
Matt Holt 6bad878a22 httpcaddyfile: Improve detection of indistinguishable TLS automation policies (#5120)
* httpcaddyfile: Skip some logic if auto_https off

* Try removing this check altogether...

* Refine test timeouts slightly, sigh

* caddyhttp: Assume udp for unrecognized network type

Seems like the reasonable thing to do if a plugin registers its own
network type.

* Add comment to document my lack of knowledge

* Clean up and prepare to merge

Add comments to try to explain what happened
2022-10-13 11:30:57 -06:00
Matt Holt 3e1fd2a8d4 httpcaddyfile: Wrap site block in subroute if host matcher used (#5130)
* httpcaddyfile: Wrap site block in subroute if host matcher used (fix #5124)

* Correct boolean logic (oops)
2022-10-12 09:27:08 -06:00
Abdussamet Koçak 33f60da9f2 fileserver: stop listing dir when request context is cancelled (#5131)
Prevents caddy from performing disk IO needlessly when the request is cancelled before the listing is finished.

Closes #5129
2022-10-08 12:56:35 -06:00
Kévin Dunglas b4e28af953 replacer: working directory global placeholder (#5127) 2022-10-07 05:54:41 -04:00
Francis Lavoie d46ba2e27f httpcaddyfile: Fix metrics global option parsing (#5126) 2022-10-06 19:40:08 -06:00
Cory Cooper 498f32bab9 caddyconfig: Implement retries into HTTPLoader (#5077)
* httploader: Add max_retries

* caddyconfig: dependency-free http config loading retries

* caddyconfig: support `retry_delay` in http loader

* httploader: Implement retries

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-10-05 22:34:49 -06:00
Ioannis Cherouvim ed118f2b09 Fix typo in comment (#5121) 2022-10-05 12:36:06 -06:00
Francis Lavoie 99ffe93388 logging: Fix skip_hosts with wildcards (#5102)
Fix #4859
2022-10-05 12:14:13 -06:00
Matthew Holt e07a267276 caddytest: Revise sleep durations
Attempt to reduce flakiness a bit more

Test suite needs to be rewritten.
2022-10-05 11:40:41 -06:00
Adam Weinberger e4fac1294f core: Set version manually via CustomVersion (#5072)
* Allow version to be set manually

When Caddy is built from a release tarball (as downloaded from GitHub),
`caddy version` returns an empty string. This causes confusion for
downstream packagers.

With this commit, VersionString can be set with eg.
  go build (...) -ldflags '-X (...).VersionString=v1.2.3'
Then the short form version will be "v1.2.3", and the full version
string will begin with "v1.2.3 ".

* Prefer embedded version, then CustomVersion

* Prefer "unknown" for full version over empty

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-10-05 10:59:57 -06:00
Matt Holt 2153a81ec8 forwardauth: Canonicalize header fields (fix #5038) (#5097) 2022-10-05 01:37:01 -04:00
Francis Lavoie ea58d51907 logging: Perform filtering on arrays of strings (where possible) (#5101)
* logging: Perform filtering on arrays of strings (where possible)

* Add test for ip_mask filter

* Oops, need to continue when it's not an IP

* Test for invalid IPs
2022-10-04 23:21:23 -06:00
Francis Lavoie 9e1d964bd6 logging: Add time_local option to use local time instead of UTC (#5108) 2022-10-05 00:23:14 -04:00
xufanglu 2be56c526c fileserver: Treat invalid file path as NotFound (#5099)
treat invalid file path as notFound so that PassThru can work
2022-10-04 21:32:40 -06:00
Francis Lavoie 01e192edc9 logging: Better console encoder defaults (#5109)
This is something that has bothered me for a while, so I figured I'd do something about it now since I'm playing in the logging code lately.

The `console` encoder doesn't actually match the defaults that zap's default logger uses. This makes it match better with the rest of the logs when using the `console` encoder alongside somekind of filter, which requires you to configure an encoder to wrap.
2022-10-04 21:18:48 -06:00
Francis Lavoie 2808de1e30 httpcaddyfile: Skip automate when auto_https off is specified (#5110) 2022-10-04 20:58:19 -06:00
Tobias Gruetzmacher 253d97c93d core: Chdir to executable location on Windows (#5115)
Since all Windows services are run from the Windows system directory,
make it easier for users by switching to our program directory right
after the start.
2022-10-04 11:04:02 -06:00
Mohammed Al Sahaf c28cd29fe7 ci: enhance the CI/CD flow (#5118) 2022-10-04 17:03:10 +03:00
Tobias Gruetzmacher da24f57dac Fix inverted logic in Windows service detection (#5106) 2022-10-02 16:56:54 -04:00
iliana etaoin b1d04f5b39 fileserver: better dark mode visited link contrast (#5105)
PR #4066 added a dark color scheme to the file_server browse template.
PR #4356 later set the links for the `:visited` pseudo-class, but did
not set anything for the dark mode, resulting in poor contrast. I
selected some new colors by feel.

This commit also adds an `a:visited:hover` for both, to go along with
the normal blue hover colors.
2022-10-01 18:14:27 -06:00
Matthew Holt fe91de67b6 go.mod: Upgrade select dependencies 2022-09-30 13:39:37 -06:00
Matthew Holt 9873ff9918 caddyhttp: Remote IP prefix placeholders
See https://github.com/mholt/caddy-ratelimit/issues/12
2022-09-30 13:29:33 -06:00
Matt Holt 5e52bbb136 map: Remove infinite recursion check (#5094)
It was not accurate. Placeholders could be used in outputs that are
defined in the same mapping as long as that placeholder does not do the
same.

A more general solution would be to detect it at run-time in the
replacer directly, but that's a bit tedious
and will require allocations I think.

A better implementation of this check could still be done, but I don't
know if it would always be accurate. Could be a "best-effort" thing?
But I've also never heard of an actual case where someone configured
infinite recursion...
2022-09-29 12:46:38 -06:00
Matthew Holt fcdbc69fab Fix comment
I apparently read the diff backwards in
2a8c458ffe
2022-09-29 12:38:36 -06:00
Matthew Holt 2a8c458ffe reverseproxy: Parse humanized byte size (fix #5095) 2022-09-29 12:37:06 -06:00
Cory Cooper 037dc23cad admin: Use replacer on listen addresses (#5071)
* admin: use replacer on listen address

* admin: consolidate replacer logic
2022-09-29 11:24:52 -06:00
Matthew Holt ab720fb768 core: Fix ListenQUIC listener key conflict
Reported on commit e3e8aabbcf

Abused this change in some bash for loops to rapidly reload config
while making requests and didn't observe any memory or resource leaks.
2022-09-29 10:32:02 -06:00
Matt Holt e2991eb019 reverseproxy: On 103 don't delete own headers (#5091)
See #5074
2022-09-29 08:19:56 -06:00
Matt Holt 897a38958c Merge pull request #5076 from caddyserver/fastcgi-redir
fastcgi: Redirect using original URI path (fix #5073) and rewrite: Only trim prefix if matched
2022-09-28 15:22:45 -06:00
Will Norris 61822f129b caddyhttp: replace placeholders in map defaults (#5081)
This updates the map directive to replace placeholders in default values
in the same way as matched values.
2022-09-28 13:38:20 -06:00
Matt Holt e3e8aabbcf core: Refactor and improve listener logic (#5089)
* core: Refactor, improve listener logic

Deprecate:
- caddy.Listen
- caddy.ListenTimeout
- caddy.ListenPacket

Prefer caddy.NetworkAddress.Listen() instead.

Change:
- caddy.ListenQUIC (hopefully to remove later)
- caddy.ListenerFunc signature (add context and ListenConfig)

- Don't emit Alt-Svc header advertising h3 over HTTP/3

- Use quic.ListenEarly instead of quic.ListenEarlyAddr; this gives us
more flexibility (e.g. possibility of HTTP/3 over UDS) but also
introduces a new issue:
https://github.com/lucas-clemente/quic-go/issues/3560#issuecomment-1258959608

- Unlink unix socket before and after use

* Appease the linter

* Keep ListenAll
2022-09-28 13:35:51 -06:00
Matthew Holt 013b510352 rewrite: Only trim prefix if matched
See #5073
2022-09-28 00:13:12 -06:00
lemmi d0556929a4 reverseproxy: fix upstream scheme handling in command (#5088)
e338648fed introduced multiple upstream
addresses. A comment notes that mixing schemes isn't supported and
therefore the first valid scheme is supposed to be used.

Fixes setting the first scheme.

fixes #5087
2022-09-27 13:03:30 -06:00
Mohammed Al Sahaf b5727b9c44 ci: fix integration tests (#5079) 2022-09-24 19:00:55 +00:00
Matthew Holt 7041970059 headers: Support repeated WriteHeader if 1xx (fix #5074) 2022-09-23 17:11:53 -06:00
Matthew Holt e747a9bb12 Fix tests 2022-09-23 16:47:59 -06:00
Matthew Holt f7c1a51efb fastcgi: Redirect using original URI path (fix #5073) 2022-09-23 14:36:38 -06:00
Mohammed Al Sahaf eead00f54a ci: extend goreleaser timeout to 1-hour (#5067) 2022-09-22 15:09:18 +00:00
Matthew Holt 9206e8a738 Tweak some comments 2022-09-21 12:59:44 -06:00
Matt Holt 1426c97da5 core: Reuse unix sockets (UDS) and don't try to serve HTTP/3 over UDS (#5063)
* core: Reuse unix sockets

* Don't serve HTTP/3 over unix sockets

This requires upstream support, if even useful

* Don't use unix build tag... yet

* Fix build tag

* Allow ErrNotExist when unlinking socket
2022-09-21 12:55:23 -06:00
WeidiDeng 44ad0cedaf encode: don't WriteHeader unless called (#5060) 2022-09-21 08:30:42 -06:00
Matthew Holt beb7dcbf2a fileserver: Reinstate --debug flag
I think it got lost during a rebase or something
2022-09-20 16:56:02 -06:00
Francis Lavoie 821a08a6e3 httpcaddyfile: Fix protocols global option parsing (#5054)
* httpcaddyfile: Fix `protocols` global option parsing

When checking for a block, the current nesting must be used, otherwise it returns the wrong thing.

* Adjust adapt test to cover the broken behaviour that is now fixed

* Fix some admin tests which suddenly run even with -short
2022-09-20 08:09:04 -06:00
Francis Lavoie e3d04ff86b caddyhttp: Skip inserting HTTP->HTTPS redir if catch-all for both exist (#5051) 2022-09-19 22:11:19 -06:00
Matt Holt da8b7fe58f caddyhttp: Honor grace period in background (#5043)
* caddyhttp: Honor grace period in background

This avoids blocking during config reloads.

* Don't quit process until servers shut down

* Make tests more likely to pass on fast CI (#5045)

* caddyhttp: Even faster shutdowns

Simultaneously shut down all HTTP servers, rather than one at a time.

In practice there usually won't be more than 1 that lingers. But this
code ensures that they all Shutdown() in their own goroutine
and then we wait for them at the end (if exiting).

We also wait for them to start up so we can be fairly confident the
shutdowns have begun; i.e. old servers no longer
accepting new connections.

* Fix comment typo

* Pull functions out of loop, for readability
2022-09-19 21:54:47 -06:00
Matthew Holt 0950ba4f0b events: Make event data exported
This could lead to bugs if handlers are not careful, but it is surely
useful. We'll see how it goes, what the feedback is like, etc.
2022-09-19 16:20:58 -06:00
WeidiDeng c7a6bc5934 caddyhttp: responseRecorder save status in all cases (#5049) 2022-09-17 18:47:53 -06:00
Matthew Holt 00beec2e34 caddyhttp: Fix write header on responseRecorder 2022-09-17 11:28:13 -06:00
Mohammed Al Sahaf b4643994d5 ci: fix the name template of singing certificate and sboms (#5046) 2022-09-17 08:54:50 -06:00
Matthew Holt e43b6d8178 core: Variadic Context.Logger(); soft deprecation
Ideally I'd just remove the parameter to caddy.Context.Logger(), but
this would break most Caddy plugins.

Instead, I'm making it variadic and marking it as partially deprecated.
In the future, I might completely remove the parameter once most
plugins have updated.
2022-09-16 16:55:36 -06:00
WeidiDeng bffc258732 caddyhttp: Support configuring Server from handler provisioning (#4933)
* configuring http.Server from handlers.

* Minor tweaks

* Run gofmt

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-09-16 14:48:55 -06:00
David Manouchehri 616418281b caddyhttp: Support TLS key logging for debugging (#4808)
* Add SSL key logging.

* Resolve merge conflict with master

* Add Caddyfile support; various fixes

* Also commit go.mod and go.sum, oops

* Appease linter

* Minor tweaks

* Add doc comment

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-16 14:05:37 -06:00
Matt Holt 74547f5bed caddyhttp: Make metrics opt-in (#5042)
* caddyhttp: Make metrics opt-in

Related to #4644

* Make configurable in Caddyfile
2022-09-16 13:32:49 -06:00
Matthew Holt 258071d857 caddytls: Debug log on implicit tailscale error (#5041) 2022-09-16 09:42:05 -06:00
Matthew Holt b6cec37893 caddyhttp: Add --debug flag to commands
file-server and reverse-proxy

This might be useful!
2022-09-15 23:10:16 -06:00
WeidiDeng 48d723c07c encode: Fix Accept-Ranges header; HEAD requests (#5039)
* fix encode handler header manipulation
also avoid implementing ReadFrom because it breaks when io.Copied to directly

* strconv.Itoa should be tried as a last resort
WriteHeader during Close
2022-09-15 16:05:08 -06:00
Matthew Holt f1f7a22674 Reject absurdly long duration strings (fix #4175) 2022-09-15 14:25:29 -06:00
Matthew Holt 49b7a25264 Fix #4169 (correct e6c58fd) 2022-09-15 14:13:58 -06:00
Matthew Holt e6c58fdc08 caddyfile: Prevent infinite nesting on fmt (fix #4175) 2022-09-15 14:12:53 -06:00
Matthew Holt 2dc747cf2d Limit unclosed placeholder tolerance (fix #4170) 2022-09-15 13:36:08 -06:00
Isaac Parker e338648fed reverseproxy: Support repeated --to flags in command (#4693)
* feat: Multiple 'to' upstreams in reverse-proxy cmd

* Repeat --to for multiple upstreams, rather than comma-separating in a single flag

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-15 12:35:38 -06:00
Francis Lavoie 9ad0ebc956 caddyhttp: Add 'skip_log' var to omit request from logs (#4691)
* caddyhttp: Implement `skip_log` handler

* Refactor to use vars middleware

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-15 10:05:36 -06:00
Michael Stapelberg a1ad20e472 httpcaddyfile: Fix bind when IPv6 is specified with network (#4950)
* fix listening on IPv6 addresses: use net.JoinHostPort

Commit 1e18afb5c8 broke my caddy setup.
This commit fixes it.

* Refactor solution; simplify, add descriptive comment

* Move network to host, not copy

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-09-15 08:03:24 -06:00
Matthew Holt 62b0685375 cmd: Improve error message if config missing 2022-09-14 23:24:16 -06:00
Matthew Holt 0b3161aeea cmd: Customizable user agent (close #2795) 2022-09-13 17:21:04 -06:00
Matthew Holt 754fe4f7b4 httpcaddyfile: Fix sorting of repeated directives
Fixes #5037
2022-09-13 13:43:21 -06:00
Matthew Holt 20d487be57 caddyhttp: Very minor optimization to path matcher
If * is in the matcher it will always match so we can just put it first.
2022-09-13 11:26:10 -06:00
Francis Lavoie 61c75f74de caddyhttp: Explicitly disallow multiple regexp matchers (#5030)
* caddyhttp: Explicitly disallow multiple regexp matchers

Fix #5028

Since the matchers would overwrite eachother, we should error out to tell the user their config doesn't make sense.

* Update modules/caddyhttp/matchers.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-13 11:18:37 -06:00
Matthew Holt d35f618b10 caddytls: Error if placeholder is empty in 'ask'
Fixes #5036
2022-09-13 08:59:03 -06:00
Mohammed Al Sahaf 9fe4f93bc7 supplychain: publish signing cert, sbom, and signatures of sbom (#5027) 2022-09-12 22:59:53 +00:00
Matthew Holt c5df7bb6bd go.mod: Update truststore 2022-09-10 21:44:35 -06:00
Matthew Holt 076a8b8095 Very minor tweaks 2022-09-08 13:10:40 -06:00
Matthew Holt 50748e19c3 core: Check error on ListenQUIC 2022-09-08 12:36:31 -06:00
Matthew Holt c19f207237 fileserver: Ignore EOF when browsing empty dir
Thanks to @WeidiDeng for reporting this
2022-09-07 21:14:11 -06:00
fleandro dd9813c65b caddyhttp: ensure ResponseWriterWrapper and ResponseRecorder use ReadFrom if the underlying response writer implements it. (#5022)
Doing so allows for splice/sendfile optimizations when available.
Fixes #4731

Co-authored-by: flga <flga@users.noreply.github.com>
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-09-07 21:13:35 +01:00
Matthew Holt 1c9c8f6a13 cmd: Enhance some help text 2022-09-06 14:19:58 -06:00
Francis Lavoie 8cc8f9fddd httpcaddyfile: Add a couple more placeholder shortcuts (#5015)
This adds:
- `{file.*}` -> `{http.request.uri.path.file.*}`
- `{file_match.*}` -> `{http.matchers.file.*}`

This is a follow-up to #4993 which introduces the new URI file placeholders, and a shortcut for using `file` matcher output.

For example, where the `try_files` directive is a shortcut for this:

```
@try_files file <files...>
rewrite @try_files {http.matchers.file.relative}
```

It could instead be:
```
@try_files file <files...>
rewrite @try_files {file_match.relative}
```
2022-09-05 21:41:48 -06:00
Dave Henderson 8f6a88e2b0 Merge pull request #5018 from hairyhenderson/allow-fs.FS-for-virtual-filesystems
Drop requirement for filesystems to implement fs.StatFS
2022-09-05 20:10:48 -04:00
Dave Henderson fded2644f8 Drop requirement for filesystems to implement fs.StatFS
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-09-05 19:25:34 -04:00
Mohammed Al Sahaf 487217519c ci: grant the release workflow the write permission to contents (#5017) 2022-09-05 21:35:47 +00:00
Mohammed Al Sahaf 0499d9c1c4 ci: add id-token permission and update the signing command (#5016) 2022-09-05 20:57:27 +00:00
Matthew Holt 5dfa08174a go.mod: Upgrade CertMagic (v0.17.1) 2022-09-05 13:55:48 -06:00
Matt Holt d5ea43fb4b fileserver: Support glob expansion in file matcher (#4993)
* fileserver: Support glob expansion in file matcher

* Fix tests

* Fix bugs and tests

* Attempt Windows fix, sigh

* debug Windows, WIP

* Continue debugging Windows

* Another attempt at Windows

* Plz Windows

* Cmon...

* Clean up, hope I didn't break anything
2022-09-05 13:53:41 -06:00
Matt Holt ca4fae64d9 caddyhttp: Support respond with HTTP 103 Early Hints (#5006)
* caddyhttp: Support sending HTTP 103 Early Hints

This adds support for early hints in the static_response handler.

* caddyhttp: Don't record 1xx responses
2022-09-05 13:50:44 -06:00
Matthew Holt ad69503aef Remove unnecessary error check 2022-09-05 13:42:59 -06:00
Francis Lavoie 6e3063b15a caddyauth: Speed up basicauth provision, deprecate scrypt (#4720)
* caddyauth: Speed up basicauth provisioning, precalculate fake password

* Deprecate scrypt, allow using decoded bcrypt hashes

* Add TODO note

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-05 13:32:58 -06:00
Mohammed Al Sahaf d6b3c7d262 ci: generate SBOM and sign artifacts using cosign (#4910)
* ci: sign artifacts using cosign

* include SBOM
2022-09-03 03:37:10 +03:00
Matt Holt 66476d8c8f reverseproxy: Close hijacked conns on reload/quit (#4895)
* reverseproxy: Close hijacked conns on reload/quit

We also send a Close control message to both ends of
WebSocket connections. I have tested this many times in
my dev environment with consistent success, although
the variety of scenarios was limited.

* Oops... actually call Close() this time

* CloseMessage --> closeMessage

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Use httpguts, duh

* Use map instead of sync.Map

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-09-02 17:01:55 -06:00
Matt Holt d3c3fa10bd core: Refactor listeners; use SO_REUSEPORT on Unix (#4705)
* core: Refactor listeners; use SO_REUSEPORT on Unix

Just an experiment for now

* Fix lint by logging error

* TCP Keepalive configuration (#4865)

* initial attempt at TCP Keepalive configuration

* core: implement tcp-keepalive for linux

* move canSetKeepAlive interface

* Godoc for keepalive server parameter

* handle return values

* log keepalive errors

* Clean up after bad merge

* Merge in pluggable network types

From 1edc1a45e3

* Slight refactor, fix from recent merge conflict

Co-authored-by: Karmanyaah Malhotra <karmanyaah.gh@malhotra.cc>
2022-09-02 16:59:11 -06:00
WeidiDeng 83b26975bd fastcgi: Optimize FastCGI transport (#4978)
* break up code and use lazy reading and pool bufio.Writer

* close underlying connection when operation failed

* allocate bufWriter and streamWriter only once

* refactor record writing

* rebase from master

* handle err

* Fix type assertion

Also reduce some duplication

* Refactor client and clientCloser for logging

Should reduce allocations

* Minor cosmetic adjustments; apply Apache license

* Appease the linter

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-09-02 16:57:55 -06:00
Matthew Holt 005c5a6382 Minor style adjustments for HTTP redir logging 2022-09-02 13:04:31 -06:00
Matthew Holt 6c0d0511ba Update readme 2022-09-02 10:26:31 -06:00
Matthew Holt 5c7ae5e505 Minor fix of error log 2022-09-02 10:19:51 -06:00
Matthew Holt 59286d2c7e notify: Don't send ready after error (fix #5003)
Also simplify the notify package quite a bit.
Also move stop notification into better place.
Add ability to send status or error.
2022-09-02 09:24:05 -06:00
Avdhut 66959d9f18 templates: Document httpError function (#4972)
* added the httpError function into the document

* Update templates.go

* Update templates.go

* Fix gofmt

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-09-01 22:07:52 -06:00
fleandro f2a7e7c966 fastcgi: allow users to log stderr output (#4967) (#5004)
Co-authored-by: flga <flga@users.noreply.github.com>
2022-09-02 00:02:48 -04:00
Matthew Holt ec2a5762b0 cmd: Don't print long help text on error 2022-09-01 21:43:23 -06:00
Matthew Holt e77992dd99 Fix failing test 2022-09-01 21:43:23 -06:00
Mohammed Al Sahaf aefd821ae0 dist: deb package manpages and bash completion scripts (#5007) 2022-09-01 23:39:18 -04:00
Francis Lavoie d062fb4020 caddyhttp: Copy logger config to HTTP server during AutoHTTPS (#4990) 2022-09-01 23:31:54 -04:00
Matthew Holt 73d4a8ba02 map: Coerce val to string, fix #4987
Also prevent infinite recursion, and enforce placeholder syntax.
2022-09-01 21:15:44 -06:00
Francis Lavoie 7d5108d132 httpcaddyfile: Add shortcut for expression matchers (#4976) 2022-09-01 23:12:37 -04:00
Matthew Holt 7c35bfa57c caddyhttp: Accept placeholders in vars matcher key
Until now, the vars matcher has unintentionally lacked parity with the
map directive: the destination placeholders of the map directive would
be expressed as placeholders, i.e. {foo}. But the vars matcher would
not use { }: vars foo value

This looked weird, and was confusing, since it implied that the key
could be dynamic, which doesn't seem helpful here.

I think this is a proper bug fix, since we're not used to accessing
placeholders literally without { } in the Caddyfile.
2022-09-01 16:49:18 -06:00
Matt Holt 1edc1a45e3 core: Plugins can register listener networks (#5002)
* core: Plugins can register listener networks

This can be useful for custom listeners.

This feature/API is experimental and may change!

* caddyhttp: Expose server listeners
2022-09-01 16:30:03 -06:00
Matthew Holt cb849bd664 caddyhttp: Disable draft versions of QUIC
See comment in #4996
2022-08-31 18:49:34 -06:00
Matthew Holt 3cd7437b3d events: Tune logging and context cancellation 2022-08-31 18:48:46 -06:00
Francis Lavoie d4d8bbcfc6 events: Implement event system (#4912)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-31 15:01:30 -06:00
Francis Lavoie 68d8ac9802 httpcaddyfile: Add {cookie.*} placeholder shortcut (#5001) 2022-08-31 10:18:29 -06:00
Matt Holt 2d5a30b908 caddyhttp: Set Content-Type for static response (#4999) 2022-08-31 09:43:46 -06:00
Matthew Holt 687a4b9e81 cmd: Enhance CLI docs 2022-08-30 19:15:52 -06:00
Mohammed Al Sahaf d605ebe75a cmd: add completion command (#4994)
* cmd: add completion command

* error check
2022-08-30 23:24:05 +00:00
Mohammed Al Sahaf 258bc82b69 cmd: Migrate to spf13/cobra, remove single-dash arg support (#4565)
* cmd: migrate to spf13/cobra

* add `manpage` command

* limit Caddy tagline to root `help` only

* hard-code the manpage section to 8
2022-08-30 22:38:38 +00:00
Matthew Holt 8cb3cf540c Minor cleanup, resolve a couple lint warnings 2022-08-29 12:31:56 -06:00
Abirdcfly e1801fdb19 Remove duplicate words in comments (#4986) 2022-08-27 14:39:26 -06:00
Dávid Szabó 0c57facc67 reverseproxy: Add upstreams healthy metrics (#4935) 2022-08-27 12:30:23 -06:00
WeidiDeng 4c282e86da admin: Don't stop old server if new one fails (#4964)
Fixes #4954

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-08-25 22:17:52 -06:00
Matthew Holt 5fb5b81439 reverseproxy: Multiple dynamic upstreams
This allows users to, for example, get upstreams from multiple SRV
endpoints in order (such as primary and secondary clusters).

Also, gofmt went to town on the comments, sigh
2022-08-25 21:42:48 -06:00
Matthew Holt 2cc5d38229 Fix comment indentation 2022-08-25 13:28:58 -06:00
Simon Legner 66596f2d74 zstd: fix typo in comment (#4985) 2022-08-25 12:00:05 +03:00
Ben Burkert b540f195b1 httpcaddyfile: Add ocsp_interval global option (#4980) 2022-08-24 11:22:56 -06:00
Matthew Holt 3aabbc49a2 caddytls: Log error if ask request fails
Errors returned from the DecisionFunc (whether to get a cert on-demand)
are used as a signal whether to allow a cert or not; *any* error
will forbid cert issuance.

We bubble up the error all the way to the caller, but that caller is the
Go standard library which might gobble it up.
Now we explicitly log connection errors so sysadmins can
ensure their ask endpoints are working.

Thanks to our sponsor AppCove for reporting this!
2022-08-23 22:28:15 -06:00
Matt Holt bbc923d66b ci: Increase linter timeout (#4981) 2022-08-23 14:26:19 -06:00
jedy e289ba6187 templates: cap of slice should not be smaller than length (#4975) 2022-08-23 08:26:02 -06:00
Francis Lavoie a22c08a638 caddyhttp: Fix for nil handlerErr.Err (#4977) 2022-08-23 08:17:46 -06:00
Francis Lavoie 72541f1cb8 caddyhttp: Set http.error.message to the HandlerError message (#4971) 2022-08-22 23:31:07 -06:00
Matthew Holt fe5f5dfd6a go.mod: Upgrade CertMagic to v0.16.3 2022-08-18 10:56:27 -06:00
WilczyńskiT c7772588bd core: Change net.IP to netip.Addr; use netip.Prefix (#4966)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-08-17 16:10:57 -06:00
Matthew Holt a944de4ab7 Clean up metrics test code
No need to use != for booleans
2022-08-16 10:03:19 -06:00
Matt Holt a479943acd caddyhttp: Smarter path matching and rewriting (#4948)
Co-authored-by: RussellLuo <luopeng.he@gmail.com>
2022-08-16 08:48:57 -06:00
Abdussamet Koçak dc62d468e9 fileserver: reset buffer before using it (#4962) (#4963) 2022-08-15 22:31:45 -06:00
Matt Holt c79c08627d caddyhttp: Enable HTTP/3 by default (#4707) 2022-08-15 12:01:58 -06:00
Francis Lavoie e2a5e2293a reverseproxy: Add unix+h2c Caddyfile network shortcut (#4953) 2022-08-12 17:09:18 -04:00
Matt Holt f5dce84a70 reverseproxy: Ignore context cancel in stream mode (#4952) 2022-08-12 13:15:41 -06:00
Francis Lavoie 922d9f5c25 reverseproxy: Fix H2C dialer using new stdlib DialTLSContext (#4951) 2022-08-12 13:11:13 -06:00
Matthew Holt 91ab0e6066 httpcaddyfile: redir with "html" emits 200, no Location (fix #4940)
The intent of "html" is to redirect browser clients only, or those which can evaluate JS and/or meta tags. So return HTTP 200 and no Location header. See #4940.
2022-08-09 11:12:09 -06:00
Kévin Dunglas 085df25c7e reverseproxy: Support 1xx status codes (HTTP early hints) (#4882) 2022-08-09 10:53:24 -06:00
Francis Lavoie fe61209df2 logging: Fix cookie filter (#4943) 2022-08-08 19:11:02 -06:00
lewandowski-stripe 7f6a328b47 go.mod: Upgrade OpenTelemetry dependencies (#4937) 2022-08-08 15:04:18 -06:00
Matthew Holt 7ab61f46f0 fileserver: Better fix for Etag of compressed files 2022-08-08 13:09:57 -06:00
Matthew Holt 8c72f34357 fileserver: Generate Etag from sidecar file
Don't use the primary/uncompressed file for Etag when serving sidecars.

This was just overlooked initially.
2022-08-08 12:50:06 -06:00
Matthew Holt b9618b8b98 Improve docs for ZeroSSL issuer 2022-08-08 12:50:06 -06:00
Chirag Maheshwari d26559316f Replace strings.Index with strings.Cut (#4932) 2022-08-06 22:03:37 -06:00
WilczyńskiT 2642bd72b7 Replace strings.Index usages with strings.Cut (#4930) 2022-08-04 11:17:35 -06:00
Matt Holt 17ae5acaba cmd: Use newly-available version information (#4931) 2022-08-04 11:16:59 -06:00
Matt Holt 1960a0dc11 httpserver: Configurable shutdown delay (#4906) 2022-08-03 11:04:51 -06:00
Matthew Holt 63c7720e84 go.mod: Upgrade CertMagic and acmez 2022-08-02 15:35:19 -06:00
Francis Lavoie 141872ed80 chore: Bump up to Go 1.19, minimum 1.18 (#4925) 2022-08-02 16:39:09 -04:00
Matthew Holt db1aa5b5bc Oops (sigh)
Forgot to remove this redundant line
2022-08-01 13:40:09 -06:00
Matt Holt f783290f40 caddyhttp: Implement caddy respond command (#4870) 2022-08-01 13:36:22 -06:00
Matthew Holt ebd6abcbd5 fileserver: Support virtual file system in Caddyfile 2022-07-31 21:41:26 -06:00
Matt Holt 6668271661 fileserver: Support virtual file systems (#4909)
* fileserver: Support virtual file systems (close #3720)

This change replaces the hard-coded use of os.Open() and os.Stat() with
the use of the new (Go 1.16) io/fs APIs, enabling virtual file systems.
It introduces a new module namespace, caddy.fs, for such file systems.

Also improve documentation for the file server. I realized it was one of
the first modules written for Caddy 2, and the docs hadn't really been
updated since!

* Virtualize FS for file matcher; minor tweaks

* Fix tests and rename dirFS -> osFS

(Since we do not use a root directory, it is dynamic.)
2022-07-30 13:07:44 -06:00
Matthew Holt 07ed3e7c30 Minor docs clarification
Related to #4565
2022-07-29 16:56:02 -06:00
WingLim 1e0cdc54f8 core: Windows service integration (#4790)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-29 14:06:54 -06:00
Francis Lavoie 2f43aa0629 chore: Add .gitattributes to force *.go to LF (#4919)
* chore: Add .gitattributes to force *.go to LF

* What if I remove this flag
2022-07-29 08:46:45 -04:00
Matthew Holt 56c139f003 Fix compilation on Windows 2022-07-28 15:44:36 -06:00
Matthew Holt 35a81d7c5b Ignore linter warnings
Use of non-cryptographic random numbers in the load balancing
is intentional.
2022-07-28 15:40:23 -06:00
Matthew Holt 2e70d1d3bf Fix deprecation notice by using UTF16PtrFromString 2022-07-28 15:24:08 -06:00
Francis Lavoie ff2ba6de8a caddyhttp: Clear out matcher error immediately after grabbing it (#4916)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-28 15:19:48 -06:00
Matthew Holt 4fced0b6e1 Finish fixing lint errors from ea8df6ff
Follows up #4915
2022-07-28 15:16:36 -06:00
Matthew Holt 1bdd451913 caddytls: Remove PreferServerCipherSuites
It has been deprecated by Go
2022-07-28 14:50:51 -06:00
Matthew Holt ea8df6ff11 caddyhttp: Use new CEL APIs (fix #4915)
Hahaha this is the ultimate "I have no idea what I'm doing" commit but it
compiles and the tests pass and I declare victory!

... probably broke something, should be tested more.

It is nice that the protobuf dependency becomes indirect now.
2022-07-28 14:50:28 -06:00
Y.Horie c833e3b249 ci: Run golangci-lint on multiple os(#4875) (#4913) 2022-07-27 09:27:18 -04:00
Matthew Holt 7991cd1250 go.mod: Upgrade dependencies 2022-07-26 11:07:20 -06:00
Matthew Holt 1e18afb5c8 httpcaddyfile: Detect ambiguous site definitions (fix #4635)
Previously, our "duplicate key in server block" logic was flawed because
it did not account for the site's bind address. We defer this check to
when the listener addresses have been assigned, but before we commit
a server block to its listener.

Also refined how network address parsing and joining works, which was
necessary for a less convoluted fix.
2022-07-25 17:28:20 -06:00
Matthew Holt 0bebea0d4c caddyhttp: Log shutdown errors, don't return (fix #4908) 2022-07-25 10:39:59 -06:00
Matt Holt a379fa4c6c reverseproxy: Implement read & write timeouts for HTTP transport (#4905) 2022-07-23 22:38:41 -06:00
Francis Lavoie abad9bc256 cmd: Fix reload with stdin (#4900) 2022-07-20 18:14:33 -06:00
Matthew Holt 8bdee04651 caddyhttp: Enhance comment 2022-07-16 23:33:49 -06:00
Francis Lavoie 7d1f7771c9 reverseproxy: Implement retry count, alternative to try_duration (#4756)
* reverseproxy: Implement retry count, alternative to try_duration

* Add Caddyfile support for `retry_match`

* Refactor to deduplicate matcher parsing logic

* Fix lint
2022-07-13 14:15:00 -06:00
Matthew Holt 04a14ee37a caddyhttp: Make query matcher more efficient
Only parse query string once
2022-07-13 12:20:00 -06:00
Matthew Holt c2bbe42fc3 reverseproxy: Export SetScheme() again
Turns out the NTLM transport uses it. Oops.
2022-07-13 08:52:30 -06:00
jhwz ad3a83fb91 admin: expect quoted ETags (#4879)
* expect quoted etags

* admin: Minor refactor of etag facilities

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-07-12 12:23:55 -06:00
Francis Lavoie 53c4d788d4 headers: Only replace known placeholders (#4880) 2022-07-12 12:16:03 -06:00
Matthew Holt d6bc9e0b5c reverseproxy: Err 503 if all upstreams unavailable 2022-07-08 13:01:32 -06:00
Francis Lavoie 54d1923ccb reverseproxy: Adjust new TLS Caddyfile directive names (#4872) 2022-07-08 13:04:22 -04:00
Matthew Holt c0f76e9ed4 fileserver: Use safe redirects in file browser 2022-07-07 14:10:19 -06:00
jhwz f259ed52bb admin: support ETag on config endpoints (#4579)
* admin: support ETags

* support etags

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-07-06 13:50:07 -06:00
Francis Lavoie 8bac134f26 go.mod: Bump up quic-go to v0.28.0, fixes for BC breaks (#4867) 2022-07-06 12:14:32 -06:00
Matt Holt 412dcc07d3 caddytls: Reuse issuer between PreCheck and Issue (#4866)
This enables EAB reuse for ZeroSSLIssuer (which is now supported by ZeroSSL).
2022-07-05 18:12:25 -06:00
Matt Holt 660c59b6f3 admin: Implement /adapt endpoint (close #4465) (#4846) 2022-06-29 00:43:57 -04:00
Francis Lavoie 58e05cab15 forwardauth: Fix case when copy_headers is omitted (#4856)
See https://caddy.community/t/using-forward-auth-and-writing-my-own-authenticator-in-php/16410, apparently it didn't work when `copy_headers` wasn't used. This is because we were skipping adding a handler to the routes in the "good response handler", but this causes the logic in `reverseproxy.go` to ignore the response handler since it's empty. Instead, we can just always put in the `header` handler, even with an empty `Set` operation, it's just a no-op, but it fixes that condition in the proxy code.
2022-06-28 19:23:30 -06:00
Tristan Swadell 10f85558ea Expose several Caddy HTTP Matchers to the CEL Matcher (#4715)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-06-22 18:53:46 -04:00
Francis Lavoie 98468af8b6 reverseproxy: Fix double headers in response handlers (#4847) 2022-06-22 15:10:14 -04:00
Francis Lavoie 25f10511e7 reverseproxy: Fix panic when TLS is not configured (#4848)
* reverseproxy: Fix panic when TLS is not configured

* Refactor and simplify setScheme

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-06-22 15:01:57 -04:00
Kiss Károly Pál b6e96fa3c5 reverseproxy: Skip TLS for certain configured ports (#4843)
* Make reverse proxy TLS server name replaceable for SNI upstreams.

* Reverted previous TLS server name replacement, and implemented thread safe version.

* Move TLS servername replacement into it's own function

* Moved SNI servername replacement into httptransport.

* Solve issue when dynamic upstreams use wrong protocol upstream.

* Revert previous commit.

Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6

* Added SkipTLSPorts option to http transport.

* Fix typo in test config file.

* Rename config option as suggested by Matt

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update code to match renamed config option.

* Fix typo in config option name.

* Fix another typo that I missed.

* Tests not completing because of apparent wrong ordering of options.

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-20 11:51:42 -06:00
Matthew Holt 56013934a4 go.mod: Update some dependencies 2022-06-20 10:50:50 -06:00
Francis Lavoie 0b6f764356 forwardauth: Support renaming copied headers, block support (#4783) 2022-06-16 14:28:11 -06:00
Matthew Holt 050d6e0aeb Add comment about xcaddy to main 2022-06-15 15:20:59 -06:00
Matt Holt 0bcd02d5f6 headers: Support wildcards for delete ops (close #4830) (#4831) 2022-06-15 09:57:43 -06:00
Kiss Károly Pál c82fe91104 reverseproxy: Dynamic ServerName for TLS upstreams (#4836)
* Make reverse proxy TLS server name replaceable for SNI upstreams.

* Reverted previous TLS server name replacement, and implemented thread safe version.

* Move TLS servername replacement into it's own function

* Moved SNI servername replacement into httptransport.

* Solve issue when dynamic upstreams use wrong protocol upstream.

* Revert previous commit.

Old commit was: Solve issue when dynamic upstreams use wrong protocol upstream.
Id: 3c9806ccb6

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-14 21:53:05 -06:00
Matthew Holt f9b42c3772 reverseproxy: Make TLS renegotiation optional 2022-06-14 09:05:25 -06:00
Yaacov Akiba Slama aaf6794b31 reverseproxy: Add renegotiation param in TLS client (#4784)
* Add renegotiation option in reverseproxy tls client

* Update modules/caddyhttp/reverseproxy/httptransport.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-10 09:33:35 -06:00
Matthew Holt 1498132ea3 caddyhttp: Log error from CEL evaluation (fix #4832) 2022-06-08 16:42:24 -06:00
Francis Lavoie 7f9b1f43c9 reverseproxy: Correct the tls_server_name docs (#4827)
* reverseproxy: Correct the `tls_server_name` docs

* Update modules/caddyhttp/reverseproxy/httptransport.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-06-06 12:37:09 -06:00
Matt Holt 5e729c1e85 reverseproxy: HTTP 504 for upstream timeouts (#4824)
Closes #4823
2022-06-03 14:13:47 -06:00
Gr33nbl00d 0a14f97e49 caddytls: Make peer certificate verification pluggable (#4389)
* caddytls: Adding ClientCertValidator for custom client cert validations

* caddytls: Cleanups for ClientCertValidator changes

caddytls: Cleanups for ClientCertValidator changes

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update modules/caddytls/connpolicy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Unexported field Validators, corrected renaming of LeafVerificationValidator to LeafCertClientAuth

* admin: Write proper status on invalid requests (#4569) (fix #4561)

* Apply suggestions from code review

* Register module; fix compilation

* Add log for deprecation notice

Co-authored-by: Roettges Florian <roettges.florian@scheidt-bachmann.de>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
2022-06-02 14:25:07 -06:00
Matthew Holt 9864b138fb reverseproxy: api: Remove misleading 'healthy' value
In v2.5.0, upstream health was fixed such that whether an upstream is
considered healthy or not is mostly up to each individual handler's
config. Since "healthy" is an opinion, it is not a global value.

I unintentionally left in the "healthy" field in the API endpoint for
checking upstreams, and it is now misleading (see #4792).

However, num_requests and fails remains, so health can be determined by
the API client, rather than having it be opaquely (and unhelpfully)
determined for the client.

If we do restore this value later on, it'd need to be replicated once
per reverse_proxy handler according to their individual configs.
2022-06-02 12:32:23 -06:00
Matthew Holt 3d18bc56b9 go.mod: Update go-yaml to v3 2022-06-01 15:15:20 -06:00
Matthew Holt 886ba84baa Fix #4822 and fix #4779
The fix for 4822 is the change at the top of the file, and
4779's fix is toward the bottom of the file.
2022-06-01 15:12:57 -06:00
Alexander M a9267791c4 reverseproxy: Add --internal-certs CLI flag #3589 (#4817)
added flag --internal-certs
when set, for non-local domains the internal CA will be used for cert generation
2022-05-29 14:33:01 -06:00
Francis Lavoie ef0aaca0d6 ci: Fix build caching on Windows (#4811)
* ci: Fix build caching on Windows

I was getting tired of Windows being slow as molasses in our CI jobs, so I went to look at our trusty source of github actions + golang information, and found a somewhat recent commit that actually fixed it. See https://github.com/mvdan/github-actions-golang/commit/4b754729baa709da219a5889c459010d4eda1888

I'll do a 2nd empty commit to re-trigger CI shortly to confirm that it actually fixes it.

* Retrigger CI
2022-05-25 11:56:39 -06:00
Aleks 6891f7f421 templates: Add humanize function (#4767)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-05-24 19:47:08 -04:00
Kévin Dunglas 499ad6d182 core: Micro-optim in run() (#4810) 2022-05-24 13:52:50 -06:00
Matthew Holt 8e6bc36084 go.mod: Upgrade some dependencies 2022-05-24 12:44:16 -06:00
Francis Lavoie 58970cae92 httpcaddyfile: Add {err.*} placeholder shortcut (#4798) 2022-05-24 10:06:46 -06:00
David Larlet 9e760e2e0c templates: Documentation consistency (#4796) 2022-05-17 18:56:40 -04:00
世界 4b4e99bdb2 chore: Bump quic-go to v0.27.0 (#4782) 2022-05-12 01:25:17 -04:00
Matt Holt 57d27c1b58 reverseproxy: Support http1.1>h2c (close #4777) (#4778) 2022-05-10 17:25:58 -04:00
Matthew Holt 693e9b5283 rewrite: Handle fragment before query (fix #4775) 2022-05-09 11:09:42 -06:00
Francis Lavoie b687d7b967 httpcaddyfile: Support multiple values for default_bind (#4774)
* httpcaddyfile: Support multiple values for `default_bind`

* Fix ordering of server blocks
2022-05-08 21:32:10 -04:00
Francis Lavoie f7be0ee101 map: Prevent output destinations overlap with Caddyfile shorthands (#4657) 2022-05-06 10:25:31 -06:00
Francis Lavoie f6900fcf53 reverseproxy: Support performing pre-check requests (#4739) 2022-05-06 10:50:26 -04:00
Francis Lavoie ec86a2f7a3 caddyfile: Shortcut for remote_ip for private IP ranges (#4753) 2022-05-04 12:42:37 -06:00
Francis Lavoie e7fbee8c82 reverseproxy: Permit resolver addresses to not specify a port (#4760)
Context: https://caddy.community/t/caddy-2-5-dynamic-upstreams-and-consul-srv-dns/15839

I realized it probably makes sense to allow `:53` to be omitted, since it's the default port for DNS.
2022-05-04 12:40:39 -06:00
Tyler Kropp e84e19a04e templates: Add custom template function registration (#4757)
* Add custom template function registration

* Rename TemplateFunctions to CustomFunctions

* Add documentation

* Document CustomFunctions interface

* Preallocate custom functions map list

* Fix interface name in error message
2022-05-02 14:55:34 -06:00
Francis Lavoie 4a223f5203 reverseproxy: Fix Caddyfile support for replace_status (#4754) 2022-05-02 11:44:28 -06:00
Francis Lavoie af7321511c httpcaddyfile: Fix duplicate access log when debug is on (#4746) 2022-04-28 12:16:25 -04:00
Francis Lavoie 0be3d99543 logging: Implement rename filter, changes field key names (#4745) 2022-04-28 11:38:44 -04:00
Francis Lavoie 3017b245c9 logging: Use RedirectStdLog to capture more stdlib logs (#4732)
* logging: Use `RedirectStdLog`

* .gitignore a file pattern that I'm constantly using for testing
2022-04-28 08:42:30 -06:00
Francis Lavoie 2e4c09155a cmd: Fix unix socket addresses for admin API requests (#4742)
Fixes a regression in c2327161f7
2022-04-28 08:31:59 -06:00
Francis Lavoie dcc98da4d2 caddyhttp: Improve listen addr error message for IPv6 (#4740) 2022-04-28 08:18:45 -06:00
Marco Kaufmann 3ab648382d templates: Add missing backticks in docs (#4737) 2022-04-27 11:41:37 -06:00
Matt Holt 40b193fb79 reverseproxy: Improve hashing LB policies with HRW (#4724)
* reverseproxy: Improve hashing LB policies with HRW

Previously, if a list of upstreams changed, hash-based LB policies
would be greatly affected because the hash relied on the position of
upstreams in the pool. Highest Random Weight or "rendezvous" hashing
is apparently robust to pool changes. It runs in O(n) instead of
O(log n), but n is very small usually.

* Fix bug and update tests
2022-04-27 10:39:22 -06:00
Francis Lavoie d543ad1ffd caddypki: Fix caddy trust command to use the correct API endpoint (#4730) 2022-04-25 22:00:39 -06:00
Francis Lavoie a8bb4a665a httpcaddyfile: Add {vars.*} placeholder shortcut, reverse vars sort order (#4726)
* httpcaddyfile: Add `{vars.*}` placeholder shortcut

I'm yoinking this from my https://github.com/caddyserver/caddy/pull/4657 PR because I think we should get this in ASAP for v2.5.0 along with the new `vars` directive.

* Sort vars by matchers in reverse
2022-04-25 10:47:12 -06:00
Francis Lavoie 3a1e0dbf47 httpcaddyfile: Deprecate paths in site addresses; use zap logs (#4728) 2022-04-25 10:12:10 -06:00
Francis Lavoie 77a77c0219 caddytls: Add propagation_delay, support propagation_timeout -1 (#4723) 2022-04-22 16:09:11 -06:00
Matthew Holt db62942d63 Make file modes consistent
No need to have executable bit on .go or .txt files
2022-04-21 15:06:55 -06:00
Matthew Holt dadd4b59b0 Update smallstep/certificates 2022-04-20 11:32:33 -06:00
Mohammed Al Sahaf d230b33007 ci: use latest Go version on macOS (#4708) 2022-04-15 13:58:48 -04:00
Matthew Holt 0d13173071 ci: Fix typo 2022-04-13 14:11:03 -06:00
Francis Lavoie c3a82f53d5 ci: Ensure we always check for latest version of Go (#4703)
* ci: Ensure we always check for latest version of Go

* Try to force 1.18.1, 1.17.9

* Use includes for the actual go semver

* Use `~` for semver here, apparently

* Try to make tests still run on 1.18.0 for Mac, for now
2022-04-13 14:03:38 -06:00
Matthew Holt 30b6d1f47a cmd: Enhance .env (dotenv) file parsing
Basic support for quoted values, newlines in quoted values, and comments.

Does not support variable or command expansion.
2022-04-13 11:38:20 -06:00
Francis Lavoie bc15b4b0e7 caddypki: Load intermediate for signing on-the-fly (#4669)
* caddypki: Load intermediate for signing on-the-fly

Fixes #4517

Big thanks to @maraino for adding an API in `smallstep/certificates` so that we can fix this

* Debug log

* Trying a hunch, does it need to be a pointer receiver?

* Clarify pointer receiver

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-04-13 10:20:42 -06:00
cui fliter e2535233bb fix typo (#4702)
Signed-off-by: cuishuang <imcusg@gmail.com>
2022-04-13 10:13:28 -06:00
Francis Lavoie 00234c8ac2 templates: Switch to BurntSushi/toml (#4700) 2022-04-12 13:48:42 -06:00
Francis Lavoie 6512832f9f cmd: Add --diff option for caddy fmt (#4695) 2022-04-12 14:49:19 -04:00
Francis Lavoie 3e3bb00265 reverseproxy: Add _ms placeholders for proxy durations (#4666)
* reverseproxy: Add `_ms` placeholders for proxy durations

* Add http.request.duration_ms

Also add comments, and change duration_sec to duration_ms

* Add response.duration_ms for consistency

* Add missing godoc comment

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-04-11 13:04:05 -06:00
Francis Lavoie e4ce40f8ff reverseproxy: Sync up handleUpgradeResponse with stdlib (#4664)
* reverseproxy: Sync up `handleUpgradeResponse` with stdlib

I had left this as a TODO for when we bump to minimum 1.17, but I should've realized it was under `internal` so it couldn't be used directly.

Copied the functions we needed for parity. Hopefully this is ok!

* Add tests and fix godoc comments

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-04-11 12:49:56 -06:00
Y.Horie afca242111 staticfiles: Expand placeholder for index files (#4679) 2022-04-07 15:01:09 -06:00
Francis Lavoie 7d229665ed logging: Caddyfile support for duration_format (#4684)
Somehow, this was missed. Oops!
2022-04-07 11:23:28 -06:00
Francis Lavoie 22d8edb984 cmd: Fix defaulting admin address if empty in config, fixes reload (#4674) 2022-04-03 12:04:33 -04:00
Francis Lavoie 734acc776a chore: Fix for xcaddy builds (#4665)
* chore: Attempt fix for xcaddy builds

* Upgrade smallstep/certificates which avoids the problem
2022-03-28 15:07:43 -06:00
Francis Lavoie b4f1a71397 chore: Bump minimum Go to 1.17 (#4662) 2022-03-25 14:56:29 -04:00
Matthew Holt d06d0e79f8 go.mod: Upgrade CertMagic to v0.16.0
Includes several breaking changes; code base updated accordingly.

- Added lots of context arguments
- Use fs.ErrNotExist
- Rename ACMEManager -> ACMEIssuer; CertificateManager -> Manager
2022-03-25 11:28:54 -06:00
Francis Lavoie a58f240d3e httpcaddyfile: Fix #4640 (auto-HTTPS edgecase) (#4661)
Guh, this is complicated.

Fixes #4640

This also follows up on #4398 (reverting it) which made a change that technically worked, but was incorrect. It changed the condition in `hostsFromKeysNotHTTP` from `&&` to `||`, but then the function no longer did what its name said it would do, and it would return hosts even if they were marked with `http://`, if they used a non-HTTP port. That wasn't the intent of it. The test added in there was kept though, because it is a valid usecase.

The actual fix is to check _earlier_ whether all the addresses explicitly have `http://`, and if so we can short circuit and skip considering the rest.
2022-03-24 22:54:03 -06:00
Francis Lavoie 4b75f3e2f0 chore: Clean up adapt test line endings (#4660)
Lots of the files were using CRLF instead of LF. Mostly my fault cause sometimes I make the files on Windows and VSCode for some reason kept making them with the wrong line endings. Sigh.

Since .txt files typically default to spaces for indentation, I'm also adding an .editorconfig to ensure they use tabs instead
2022-03-24 22:48:45 -06:00
Matthew Holt b8dbecb841 reverseproxy: Include port in A upstreams cache
Should fix #4659
2022-03-24 10:44:36 -06:00
Francis Lavoie 134b805644 caddyfile: Prevent bad block opening tokens (#4655)
* caddyfile: Prevent bad block opening tokens

* Clarifying comments
2022-03-23 12:34:13 -06:00
Artem Mikheev c9b5e7f77b Fix http3 servers dying after reload (#4654) 2022-03-22 19:47:57 -04:00
Matthew Holt 79cbe7bfd0 httpcaddyfile: Add 'vars' directive
See discussion in #4650
2022-03-22 10:47:21 -06:00
Matthew Holt 55b4c12e04 map: Evaluate placeholders in output vals (#4650) 2022-03-21 17:05:38 -06:00
Matthew Holt 2196c92c0e reverseproxy: Don't clear name in SRV upstreams
Fix for dc4d147388
2022-03-21 08:33:24 -06:00
Matthew Holt c2327161f7 cmd: Set Origin header properly on API requests
Ref. https://caddy.community/t/bug-in-enforce-origin/15417
2022-03-19 22:51:32 -06:00
Francis Lavoie c5fffb4ac2 caddyfile: Support for raw token values; improve map, expression (#4643)
* caddyfile: Support for raw token values, improve `map`, `expression`

* Applied code review comments

* Rename RawVal to ValRaw

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-18 15:08:23 -06:00
Matthew Holt dc4d147388 reverseproxy: Expand SRV/A addrs for cache key
Hopefully fix #4645
2022-03-18 13:42:29 -06:00
Matthew Holt 93c99f6734 map: Support numeric and bool types with Caddyfile
Based on caddyserver/website#221
2022-03-17 17:53:32 -06:00
Francis Lavoie 4e9fbee1e2 ci: Build on Go 1.18, bump actions versions (#4637)
* ci: Build on Go 1.18, bump actions versions

* Revert linter version bump for now

* Try linter again
2022-03-15 22:09:19 +00:00
Francis Lavoie a9c7e94a38 chore: Comment fixes (#4634) 2022-03-13 01:38:11 -05:00
Matthew Holt 3d616e8c6d requestbody: Return HTTP 413 (fix #4558) 2022-03-11 12:34:55 -07:00
Mohammed Al Sahaf b82e22b459 caddyhttp: retain all values of vars matcher when specified multiple times (#4629) 2022-03-11 10:55:37 -05:00
Matthew Holt bf6a1b7538 go.mod: Upgrade some dependencies
Fixes bug in yuin/goldmark
https://github.com/caddyserver/website/issues/217
2022-03-10 11:40:03 -07:00
Francis Lavoie c7d6c4cbb9 reverseproxy: copy_response and copy_response_headers for handle_response routes (#4391)
* reverseproxy: New `copy_response` handler for `handle_response` routes

Followup to #4298 and #4388.

This adds a new `copy_response` handler which may only be used in `reverse_proxy`'s `handle_response` routes, which can be used to actually copy the proxy response downstream. 

Previously, if `handle_response` was used (with routes, not the status code mode), it was impossible to use the upstream's response body at all, because we would always close the body, expecting the routes to write a new body from scratch.

To implement this, I had to refactor `h.reverseProxy()` to move all the code that came after the `HandleResponse` loop into a new function. This new function `h.finalizeResponse()` takes care of preparing the response by removing extra headers, dealing with trailers, then copying the headers and body downstream.

Since basically what we want `copy_response` to do is invoke `h.finalizeResponse()` at a configurable point in time, we need to pass down the proxy handler, the response, and some other state via a new `req.WithContext(ctx)`. Wrapping a new context is pretty much the only way we have to jump a few layers in the HTTP middleware chain and let a handler pick up this information. Feels a bit dirty, but it works.

Also fixed a bug with the `http.reverse_proxy.upstream.duration` placeholder, it always had the same duration as `http.reverse_proxy.upstream.latency`, but the former was meant to be the time taken for the roundtrip _plus_ copying/writing the response.

* Delete the "Content-Length" header if we aren't copying

Fixes a bug where the Content-Length will mismatch the actual bytes written if we skipped copying the response, so we get a message like this when using curl:

```
curl: (18) transfer closed with 18 bytes remaining to read
```

To replicate:

```
{
	admin off
	debug
}

:8881 {
	reverse_proxy 127.0.0.1:8882 {
		@200 status 200
		handle_response @200 {
			header Foo bar
		}
	}
}

:8882 {
	header Content-Type application/json
	respond `{"hello": "world"}` 200
}
```

* Implement `copy_response_headers`, with include/exclude list support

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-03-09 11:00:51 -07:00
Andrii Kushch d0b608af31 tracing: New OpenTelemetry module (#4361)
* opentelemetry: create a new module

* fix imports

* fix test

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddyhttp/opentelemetry/tracer.go

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* rename error ErrUnsupportedTracesProtocol

* replace spaces with tabs in the test data

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* replace spaces with tabs in the README.md

* use default values for a propagation and exporter protocol

* set http attributes with helper

* simplify code

* Cleanup modules/caddyhttp/opentelemetry/README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update documentation in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update link to naming spec in README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename module from opentelemetry to tracing

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Rename span_name to span

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Simplify otel resource creation

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* handle extra attributes

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel/semconv to 1.7.0

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.opentelemetry.io/otel version

* remove environment variable handling

* always use tracecontext,baggage as propagators

* extract tracer name into variable

* rename OpenTelemetry to Tracing

* simplify resource creation

* update go.mod

* rename package from opentelemetry to tracing

* cleanup tests

* update Caddyfile example in README.md

* update README.md

* fix test

* fix module name in README.md

* fix module name in README.md

* change names in README.md and tests

* order imports

* remove redundant tests

* Update documentation README.md

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Fix grammar

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* Update comments

Co-authored-by: Dave Henderson <dhenderson@gmail.com>

* update go.sum

* update go.sum

* Add otelhttp instrumentation, update OpenTelemetry libraries.

* Use otelhttp instrumentation for instrumenting HTTP requests.

This change uses context.WithValue to inject the next handler into the
request context via a "nextCall" carrier struct, and pass it on to a
standard Go HTTP handler returned by otelhttp.NewHandler. The
underlying handler will extract the next handler from the context,
call it and pass the returned error to the carrier struct.

* use zap.Error() for the error log

* remove README.md

* update dependencies

* clean up the code

* change comment

* move serveHTTP method from separate file

* add syntax to the UnmarshalCaddyfile comment

* go import the file

* admin: Write proper status on invalid requests (#4569) (fix #4561)

* update dependencies

Co-authored-by: Dave Henderson <dhenderson@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Vibhav Pant <vibhavp@gmail.com>
Co-authored-by: Alok Naushad <alokme123@gmail.com>
Co-authored-by: Cedric Ziel <cedric@cedric-ziel.com>
2022-03-08 12:18:32 -07:00
Ran Chen d9b1d46325 caddytls: dns_challenge_override_domain for challenge delegation (#4596)
* Add a override_domain option to allow DNS chanllenge delegation

CNAME can be used to delegate answering the chanllenge to another DNS
zone. One usage is to reduce the exposure of the DNS credential [1].
Based on the discussion in caddy/certmagic#160, we are adding an option
to allow the user explicitly specify the domain to delegate, instead of
following the CNAME chain.

This needs caddy/certmagic#160.

* rename override_domain to dns_challenge_override_domain

* Update CertMagic; fix spelling

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-08 12:03:43 -07:00
Francis Lavoie c8f2834b51 fastcgi: Protect against requests with null bytes in the path (#4614) 2022-03-07 10:06:33 -07:00
Matt Holt ab0455922a reverseproxy: Dynamic upstreams (with SRV and A/AAAA support) (#4470)
* reverseproxy: Begin refactor to enable dynamic upstreams

Streamed here: https://www.youtube.com/watch?v=hj7yzXb11jU

* Implement SRV and A/AAA upstream sources

Also get upstreams at every retry loop iteration instead of just once
before the loop. See #4442.

* Minor tweaks from review

* Limit size of upstreams caches

* Add doc notes deprecating LookupSRV

* Provision dynamic upstreams

Still WIP, preparing to preserve health checker functionality

* Rejigger health checks

Move active health check results into handler-specific Upstreams.

Improve documentation regarding health checks and upstreams.

* Deprecation notice

* Add Caddyfile support, use `caddy.Duration`

* Interface guards

* Implement custom resolvers, add resolvers to http transport Caddyfile

* SRV: fix Caddyfile `name` inline arg, remove proto condition

* Use pointer receiver

* Add debug logs

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-03-06 17:43:39 -07:00
Francis Lavoie c50094fc9d reverseproxy: Implement trusted proxies for X-Forwarded-* headers (#4507) 2022-03-06 18:51:55 -05:00
Francis Lavoie d058dee11d reverseproxy: Refactor dial address parsing, augment command parsing (#4616) 2022-03-05 16:34:19 -07:00
Francis Lavoie 09ba9e994e fileserver: Add pass_thru Caddyfile option (#4613) 2022-03-04 20:50:05 -07:00
Matthew Holt be82cc7aca Appease the linter 2022-03-04 20:26:37 -07:00
Matt Holt 2bb8550a4c caddyhttp: Honor wildcard hosts in log SkipHosts (#4606) 2022-03-04 13:44:59 -07:00
Matthew Holt a72acd21b0 core: Retry dynamic config load if config unchanged
(see discussion in #4603)
2022-03-03 21:41:51 -07:00
Matthew Holt a6199cf814 templates: Fix docs for .Args 2022-03-03 11:12:37 -07:00
Matthew Holt ceef70dbc5 core: Retry dynamic config load if error or no-op (#4603)
Also fix ineffectual assignment (unrelated)
2022-03-03 10:58:15 -07:00
Francis Lavoie f5e104944e reverseproxy: Make shallow-ish clone of the request (#4551)
* reverseproxy: Make shallow-ish clone of the request

* Refactor request cloning into separate function

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2022-03-03 09:54:45 -07:00
Matthew Holt 6b385a36f9 caddyhttp: Don't attempt to manage Tailscale certs
If .ts.net domains are explicitly added to config,
don't try to manage a cert for them (it will fail, and our
implicit Tailscale module will
get those certs at run-time).
2022-03-02 13:42:38 -07:00
Matthew Holt 9b7cdfa2f2 caddypki: Try to fix lint warnings 2022-03-02 13:38:05 -07:00
Matthew Holt 78e381b29f caddypki: Refactor /pki/ admin endpoints
Remove /pki/certificates/<ca> endpoint and split into two endpoints:

- GET /pki/ca/<id> to get CA info and certs in JSON format
- GET /pki/ca/<id>/certificates to get cert in PEM chain
2022-03-02 13:00:37 -07:00
ttys3 de490c7cad fastcgi: Set SERVER_PORT to 80 or 443 depending on scheme (#4572) 2022-03-02 11:24:16 -07:00
Francis Lavoie bbad6931e3 pki: Implement API endpoints for certs and caddy trust (#4443)
* admin: Implement /pki/certificates/<id> API

* pki: Lower "skip_install_trust" log level to INFO

See https://github.com/caddyserver/caddy/issues/4058#issuecomment-976132935

It's not necessary to warn about this, because this was an option explicitly configured by the user. Still useful to log, but we don't need to be so loud about it.

* cmd: Export functions needed for PKI app, return API response to caller

* pki: Rewrite `caddy trust` command to use new admin endpoint instead

* pki: Rewrite `caddy untrust` command to support using admin endpoint

* Refactor cmd and pki packages for determining admin API endpoint
2022-03-02 11:08:36 -07:00
Francis Lavoie 5bd96a6ac2 httpcaddyfile: Support explicitly turning off strict_sni_host (#4592) 2022-03-01 20:02:39 -05:00
BitWuehler ac14b64e08 caddyhttp: Support zone identifiers in remote_ip matcher (#4597)
* Update matchers.go

* Update matchers.go

* implementation of zone_id handling

* last changes in zone handling

* give return true values instead of bool

* Apply suggestions from code review

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* changes as suggested

* Apply suggestions from code review

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Update matchers.go

* shortened the Match function

* changed mazcher handling

* Update matchers.go

* delete space

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-03-01 15:50:12 -07:00
Francis Lavoie 15c95e9d5b fileserver: Canonical redir when whole path is stripped (#4549) 2022-03-01 15:32:39 -07:00
Matthew Holt bc447e307f core: Config LoadInterval -> LoadDelay for clarity
And improve/clarify docs about this feature

See #4577
2022-03-01 15:05:12 -07:00
Francis Lavoie 87a1f228b4 reverseproxy: Move status replacement intercept to replace_status (#4300) 2022-03-01 14:12:43 -07:00
Matthew Holt acbee94708 core: Revert 7f364c7; simplify dynamic config load
Fixes #4577
2022-03-01 13:00:14 -07:00
Noorain Panjwani 7ea5b2a818 core: Config load interval only reloads if changed (#4603) 2022-03-01 11:32:33 -07:00
Francis Lavoie 186fdba916 caddyhttp: Move HTTP redirect listener to an optional module (#4585) 2022-02-19 15:36:36 -07:00
Mohammed Al Sahaf 7778912d4e ci: update goreleaser (#4582) 2022-02-19 15:16:11 -07:00
Francis Lavoie c921e08296 logging: Add roll_local_time Caddyfile option (#4583) 2022-02-19 15:12:28 -07:00
Francis Lavoie ddbb234d91 caddyhttp: Always log handled errors at debug level (#4584) 2022-02-19 15:10:49 -07:00
Francis Lavoie 0de51593a6 go.mod: Revert version bump of CEL (#4587) 2022-02-19 15:09:09 -07:00
Francis Lavoie 26d633baf8 httpcaddyfile: Disabling OCSP stapling for both managed and unmanaged (#4589) 2022-02-19 14:20:38 -07:00
Matthew Holt ff137d17d0 caddyconfig: Support placeholders in HTTP loader 2022-02-17 22:58:25 -07:00
Matt Holt 57a708d189 caddytls: Support external certificate Managers (like Tailscale) (#4541)
Huge thank-you to Tailscale (https://tailscale.com) for making this change possible!
This is a great feature for Caddy and Tailscale is a great fit for a standard implementation.

* caddytls: GetCertificate modules; Tailscale

* Caddyfile support for get_certificate

Also fix AP provisioning in case of empty subject list (persist loaded
module on struct, much like Issuers, to surive reprovisioning).

And implement start of HTTP cert getter, still WIP.

* Update modules/caddytls/automation.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Use tsclient package, check status for name

* Implement HTTP cert getter

And use reuse CertMagic's PEM functions for private keys.

* Remove cache option from Tailscale getter

Tailscale does its own caching and we don't need the added complexity...
for now, at least.

* Several updates

- Option to disable cert automation in auto HTTPS
- Support multiple cert managers
- Remove cache feature from cert manager modules
- Minor improvements to auto HTTPS logging

* Run go mod tidy

* Try to get certificates from Tailscale implicitly

Only for domains ending in .ts.net.

I think this is really cool!

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2022-02-17 15:40:34 -07:00
Alok Naushad 32aad90938 admin: Write proper status on invalid requests (#4569) (fix #4561) 2022-02-15 12:13:33 -07:00
Matthew Holt 40b54434f3 admin: Enforce and refactor origin checking
Using URLs seems a little cleaner and more correct

cf: https://caddy.community/t/protect-admin-endpoint/15114

(This used to work. Something must have changed recently.)
2022-02-15 12:08:12 -07:00
Francis Lavoie 1d0425b26f templates: Elaborate on what's supported by the markdown function (#4564) 2022-02-06 22:14:41 -07:00
Francis Lavoie 7557d1d922 reverseproxy: Avoid returning a nil error during GetClientCertificate (#4550) 2022-02-01 23:33:36 -07:00
Matthew Holt ff74a0aa09 go.mod: Upgrade dependencies
Including crucial CertMagic upgrade
2022-02-01 21:00:23 -07:00
Matthew Holt 599c81d753 Interrim upgrade CertMagic
For auto-replace certificate on revocation for on-demand mode,
until a proper release is made.
2022-01-30 22:46:25 -07:00
Dave Henderson 741b0502ee Merge pull request #4545 from hairyhenderson/metrics-restrict-http-methods
metrics: Enforce smaller set of method labels
2022-01-25 15:34:35 -05:00
Dave Henderson 7ca5921a87 move common metrics-related funcs to internal package
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-01-25 15:07:17 -05:00
Francis Lavoie da4a759bad Update modules/caddyhttp/metrics_test.go 2022-01-25 15:07:17 -05:00
Dave Henderson 042abeb431 other is not uppercase
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-01-25 15:07:17 -05:00
Dave Henderson eb891d4683 metrics: Enforce smaller set of method labels
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2022-01-25 15:07:17 -05:00
Kevin Daudt 44e5e9e43f caddyhttp: Fix test when /tmp/etc already exists (#4544)
The TestFileListing test in tplcontext_test has one test that verifies
if directory traversal is not happening. The context root is set to
'/tmp' and then it tries to open '../../../../../etc', which gets
normalized to '/tmp/etc'.

The test then expects an error to be returned, assuming that '/tmp/etc'
does not exist on the system. When it does exist, it results in a test
failure:

```
--- FAIL: TestFileListing (0.00s)
    tplcontext_test.go:422: Test 4: Expected error but had none
    FAIL
    FAIL
    github.com/caddyserver/caddy/v2/modules/caddyhttp/templates	0.042s
```

Instead of using '/tmp' as root, use a dedicated directory created with
`os.MkdirTemp()` instead. That way, we know that the directory is empty.
2022-01-24 14:41:08 -07:00
Matt Holt bf380d00ab caddyhttp: Reject absurd methods (#4538)
* caddyhttp: Reject absurdly long methods

* Limit method to 32 chars and truncate

* Just reject the request and debug-log it

* Log remote address
2022-01-19 13:44:09 -07:00
Vojtech Vitek 94035c1797 Improve the reverse-proxy CLI --to flag help message (#4535) 2022-01-19 14:51:46 -05:00
Forest Johnson b3f7ce34b4 More explanatory error message from Listen (#4534)
* explain cryptic unix socket listener error related to process kill

https://github.com/caddyserver/caddy/pull/4533

* less ambiguous wording: clean up -> delete

* shorten error message explanation

* link back to pull request in comment for later archeaology
2022-01-19 12:26:44 -07:00
Francis Lavoie a79b4055e5 caddytls: Add internal Caddyfile lifetime, sign_with_root opts (#4513) 2022-01-18 12:19:50 -07:00
Francis Lavoie 5a07156894 httpcaddyfile: Add pki app root and intermediate cert/key config (#4514) 2022-01-18 12:18:31 -07:00
Francis Lavoie bcb7a19cd3 rewrite: Add method Caddyfile directive (#4528) 2022-01-18 12:17:35 -07:00
Francis Lavoie 6e6ce2be6b caddyhttp: Fix HTTP->HTTPS redir not preferring HTTPS port if ambiguous (#4530) 2022-01-18 11:56:00 -07:00
Francis Lavoie 1b7ff5d76c httpcaddyfile: Add default_bind global option (#4531) 2022-01-18 11:29:07 -07:00
Francis Lavoie 93a7a45e7e httpcaddyfile: Fix incorrect handling of IPv6 bind addresses (#4532)
The `net.JoinHostPort()` function has some naiive logic for handling IPv6, it just checks if the host part has a `:` and if so it wraps the host part with `[ ]` but this causes our network type prefix to get wrapped as well, which is invalid for `caddy.NetworkAddress`. Instead, we can just concatenate the host and port manually here to avoid this side-effect.
2022-01-18 11:27:43 -07:00
Matthew Holt 1a7a78a1f2 cmd: Print error if fmt overwrite fails (fix #4524) 2022-01-16 17:30:14 -07:00
Francis Lavoie 1feb65952a rewrite: Fix a double-encode issue when using the {uri} placeholder (#4516) 2022-01-13 12:17:15 -05:00
GallopingKylin 66de438a98 caddytls: Fix MatchRemoteIP provisoning with multiple CIDR ranges (#4522) 2022-01-13 11:56:18 -05:00
rayjlinden 850e1605df caddyhttp: Return HTTP 421 for mismatched Host header (#4023)
Potential fix for #4017 although the consensus is unclear.

Made change to return status code 421 instead of 403 when StrictSNIHost matching is on.
2022-01-12 14:24:22 -07:00
Matthew Holt af1ac9cd2e Fix lint warnings 2022-01-10 23:27:39 -07:00
Matthew Holt 64a3218f5c core: Simplify shared listeners, fix deadline bug
When this listener code was first written, UsagePool didn't exist. We
can simplify much of the wrapped listener logic by utilizing UsagePool.

This also fixes a bug where new servers were able to clear deadlines
set by old servers, even if the old server didn't get booted out of its
Accept() call yet. And with the deadline cleared, they never would.
(Sometimes. Based on reports and difficulty of reproducing the bug,
this behavior was extremely rare.) I don't know why that happened
exactly, maybe some polling mechanism in the kernel and if the timings
worked out just wrong it would expose the bug.

Anyway, now we ensure that only the closer that set the deadline is the
same one that clears it, ensuring that old servers always return out of
Accept(), because the deadline doesn't get cleared until they do.

Of course, all this hinges on the hope that my suspicions in the middle
of the night are correct and that kernels work the way I think they do
in my head.

Also minor enhancement to UsagePool where if a value errors upon
construction (a very real possibility with listeners), it is removed from
the pool. Not 100% sure the sync logic is correct there, or maybe we
don't have to even put it in the pool until after construction, but it's
subtle either way and I think this is safe... right?
2022-01-10 23:24:58 -07:00
Matthew Holt c634bbe9cc caddypki: Return error if no PEM data found
Best guess for https://caddy.community/t/on-fly-certificate-generation-based-on-sni/14639/4
2022-01-07 10:55:11 -07:00
Francis Lavoie 4b9849c792 httpcaddyfile: Support configuring pki app names via global options (#4450) 2022-01-05 22:45:41 -05:00
Francis Lavoie 80d7a356b3 caddyhttp: Redirect HTTP requests on the HTTPS port to https:// (#4313)
* caddyhttp: Redirect HTTP requests on the HTTPS port to https://

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2022-01-05 18:01:15 -07:00
Matthew Holt b4bfa29be2 admin: Require identity for remote (fix #4478) 2022-01-05 17:55:09 -07:00
Matthew Holt 6cadb60fa2 templates: Document .OriginalReq
Close caddyserver/website#91
2022-01-05 13:59:59 -07:00
Денис Телюх 2e46c2ac1d admin, reverseproxy: Stop timers if canceled to avoid goroutine leak (#4482) 2022-01-04 12:14:18 -07:00
Francis Lavoie 249adc1c87 logging: Support turning off roll compression via Caddyfile (#4505) 2022-01-04 12:11:27 -07:00
Francis Lavoie e9dde23024 headers: Fix + in Caddyfile to properly append rather than set (#4506) 2022-01-04 10:10:11 -07:00
Francis Lavoie 3fe2c73dd0 caddyhttp: Fix MatchPath sanitizing (#4499)
This is a followup to #4407, in response to a report on the forums: https://caddy.community/t/php-fastcgi-phishing-redirection/14542

Turns out that doing `TrimRight` to remove trailing dots, _before_ cleaning the path, will cause double-dots at the end of the path to not be cleaned away as they should. We should instead remove the dots _after_ cleaning.
2021-12-30 04:15:48 -05:00
Francis Lavoie 5333c3528b reverseproxy: Fix incorrect health_headers Caddyfile parsing (#4485)
Fixes #4481
2021-12-17 08:53:11 -07:00
Rainer Borene 180ae0cc48 caddyhttp: Implement http.request.uuid placeholder (#4285) 2021-12-15 00:17:53 -07:00
Matthew Holt a1c41210d3 caddypki: Minor tweak, don't use context pointer 2021-12-13 16:13:38 -07:00
Matt Holt ecac03cdcb caddyhttp: Enhance vars matcher (#4433)
* caddyhttp: Enhance vars matcher

Enable "or" logic for multiple values.
Fall back to checking placeholders if not a var name.

* Fix tests (thanks @mohammed90 !)
2021-12-13 13:59:58 -07:00
Francis Lavoie c04d24cafa pki: Avoid provisioning the local CA when not necessary (#4463)
* pki: Avoid provisioning the `local` CA when not necessary

* pki: Refactor CA loading to keep the logic in the PKI app
2021-12-13 12:25:35 -07:00
Francis Lavoie 81ee34e962 httpcaddyfile: Fix sorting edgecase for nested handle_path (#4477) 2021-12-13 13:42:08 -05:00
Mohammed Al Sahaf 78b5356f2b fileserver: do not double-escape paths (#4447) 2021-12-11 09:26:21 -05:00
Francis Lavoie 6f9b6ad78e go.mod: Update smallstep/certificates, no longer need replace (#4475) 2021-12-10 14:58:53 -05:00
Francis Lavoie 4906b9357a go.mod: Update smallstep/truststore, fix build on FreeBSD (#4473) 2021-12-09 15:57:26 -05:00
Runzhi He e90d751732 caddyfile: impove fmt warning message (#4444)
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-12-07 10:03:58 -07:00
Adam Burgess dce81e85d5 docs: use backticks to not italicise glob path (#4460) 2021-12-05 23:48:40 -07:00
Kévin Dunglas a1b417c832 logging: add support for hashing data (#4434)
* logging: add support for hashing data

* Update modules/logging/filters.go

Co-authored-by: wiese <wiese@users.noreply.github.com>

* Update modules/logging/filters.go

Co-authored-by: wiese <wiese@users.noreply.github.com>

Co-authored-by: wiese <wiese@users.noreply.github.com>
2021-12-02 13:51:37 -07:00
Francis Lavoie 5bf0adad87 caddyhttp: Make logging of credential headers opt-in (#4438) 2021-12-02 13:26:24 -07:00
Francis Lavoie 8e5aafa5cd fastcgi: Fix a TODO, prevent zap using reflection for logging env (#4437)
* fastcgi: Fix a TODO, prevent zap using reflection for logging env

* Update modules/caddyhttp/reverseproxy/fastcgi/fastcgi.go

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
2021-12-02 13:23:19 -07:00
Francis Lavoie c133153447 go.mod: Update to latest smallstep/truststore, support FreeBSD (#4453) 2021-11-29 17:15:41 -07:00
Tim Culverhouse ec14ccdd40 templates: fix inconsistent nested includes (#4452) 2021-11-29 12:29:40 -05:00
Francis Lavoie f55b123d63 caddyhttp: Split up logged remote address into IP and port (#4403) 2021-11-29 01:18:35 -05:00
Matt Holt 0eb0b60f47 logging: Remove common_log field and single_field encoder (#4149) (#4282) 2021-11-29 01:08:52 -05:00
Rainer Borene 5e5af50e64 caddyfile: make renew_interval option configurable (#4451) 2021-11-28 17:22:26 -05:00
Francis Lavoie 9ee68c1bd5 reverseproxy: Adjust defaults, document defaults (#4436)
* reverseproxy: Adjust defaults, document defaults

Related to some of the issues in https://github.com/caddyserver/caddy/issues/4245, a complaint about the proxy transport defaults not being properly documented in https://caddy.community/t/default-values-for-directives/14254/6.

- Dug into the stdlib to find the actual defaults for some of the timeouts and buffer limits, documenting them in godoc so the JSON docs get them next release.

- Moved the keep-alive and dial-timeout defaults from `reverseproxy.go` to `httptransport.go`. It doesn't make sense to set defaults in the proxy, because then any time the transport is configured with non-defaults, the keep-alive and dial-timeout defaults are lost!

- Sped up the dial timeout from 10s to 3s, in practice it rarely makes sense to wait a whole 10s for dialing. A shorter timeout helps a lot with the load balancer retries, so using something lower helps with user experience.

* reverseproxy: Make keepalive interval configurable via Caddyfile

* fastcgi: DialTimeout default for fastcgi transport too
2021-11-24 01:32:25 -05:00
Kévin Dunglas 789efa5dee logging: add a regexp filter (#4426) 2021-11-23 10:00:20 -07:00
Kévin Dunglas 8887adb027 logging: add a filter for cookies (#4425)
* feat(logging): add a filter for cookies

* Improve godoc and add validation
2021-11-23 09:40:20 -07:00
Kévin Dunglas bcac2beee7 logging: add a filter for query parameters (#4424)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-11-23 04:01:43 -05:00
Mohammed Al Sahaf 1e10f6f725 fileserver: browse: do not encode the paths in breadcrumbs and page title (#4410) 2021-11-23 03:13:09 -05:00
Jeremy Lin c8b5a81607 fileserver: Fix handling of symlink sizes in directory listings (#4415) 2021-11-22 14:59:09 -07:00
Francis Lavoie eead337324 caddyhttp: Log non-500 handler errors at debug level (#4429)
Fixes #4428

It's best to still log handler errors at debug level so that they're hidden by default, but still accessible if additional details are necessary.
2021-11-22 11:58:25 -07:00
Matthew Holt 7d5047c1f1 caddyhttp: Log empty value for typical password headers
Work around for common misconfiguration
2021-11-22 11:31:50 -07:00
Matthew Holt 7f364c777a core: Load config at interval instead of just once 2021-11-16 13:08:22 -07:00
Matthew Holt b47af6ef04 caddyfile: Copy input before parsing (fix #4422) 2021-11-15 14:41:19 -07:00
Jeremy Lin e81369e220 fileserver: Move default browse template into a separate file (#4417)
This makes it easier for users to find the default browse template if they
want to create a custom template based on that. It also makes it easier to
view the template with proper syntax highlighting.
2021-11-15 11:53:54 -07:00
Francis Lavoie e7457b43e4 caddyhttp: Sanitize the path before evaluating path matchers (#4407) 2021-11-08 13:45:03 -07:00
Matthew Holt f376a38b25 go.mod: Update ACMEz and CertMagic 2021-11-08 13:08:50 -07:00
Francis Lavoie 749e55c738 caddycmd: Add --keep-backup to upgrade commands (#4387)
* caddycmd: Add `--skip-cleanup` to upgrade commands

This is a partial fix for https://github.com/caddyserver/caddy/issues/4057, making it possible to retain the old build of Caddy, in case something went wrong.

* caddycmd: Fix duplicate error message

The error message "download succeeded, but unable to execute" was repeated, because it was both in the `listModules`/`showVersion` functions and in the calling `upgradeBuild` function. Oversight when this was refactored.

* caddycmd: Implement fix for performing cleanup on Windows

Without this, the cleanup operation would fail with an error message like this:

upgrade: download succeeded, but unable to clean up backup binary: remove C:\caddy\caddy.exe.tmp: Access is denied.

* caddycmd: Rename to `--keep-backup`, simplify build constraints
2021-11-08 11:35:46 -07:00
Matt Holt 24fda7514d caddytls: Mark storage clean timestamp at end of routine (#4401)
See discussion on 42b7134ffa
2021-11-02 08:27:25 -06:00
Matthew Holt 3385856966 Fix lint message in metrics tests 2021-10-27 13:44:46 -06:00
Francis Lavoie f73f55dba7 reverseproxy: Sanitize scheme and host on incoming requests (#4237)
* caddyhttp: Sanitize scheme and host on incoming requests

* reverseproxy: Sanitize the URL scheme and host before proxying

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-10-26 14:41:28 -06:00
Marc Easen 012d235314 httpcaddyfile: Empty tls policy for internal http localhost (#4398)
* test: replicated empty tls automation policy issue

* fix: empty tls policy for an http:// endpoint running on a non-standard http port
2021-10-26 13:54:19 -06:00
Matthew Holt 997e41deae go.mod: Replace promptui with Apache-compatible fork (fix #4394)
Ideally this needs to be fixed upstream in github.com/manifoldco/promptui, but it appears unmaintained. Our dependency is extremely indirect:

    $ go mod why github.com/juju/ansiterm
    # github.com/juju/ansiterm
    github.com/caddyserver/caddy/v2/modules/caddypki
    github.com/smallstep/certificates/authority
    go.step.sm/cli-utils/ui
    github.com/manifoldco/promptui
    github.com/juju/ansiterm

And it appears that all dependencies in this chain are in conflict with the LGPL license.

Ref:
- https://github.com/manifoldco/promptui/issues/173
- https://github.com/manifoldco/promptui/pull/181

/cc @maraino
2021-10-21 13:44:16 -06:00
Matthew Holt 0ffb2229b0 httpcaddyfile: Preserve IPv6 addresses through normalization (fix #4381)
Remove unnecessary Key() method and improve related tests
2021-10-20 10:27:59 -06:00
Klaus Helenius a21d5a001f fileserver: Prevent focusing filter from scrolling on page load (#4393) 2021-10-20 12:15:58 -04:00
Matthew Holt a2119c09e9 map: Fix 95c03506 (avoid repeated expansions) 2021-10-19 12:25:36 -06:00
Francis Lavoie 062657d0d8 caddycmd: Add --skip-standard to list-modules command, quieter output (#4386)
* caddycmd: Add --skip-standard to list-modules command, quieter output

* caddycmd: Also quiet `caddy upgrade` output, redundant information
2021-10-18 12:19:04 -06:00
Francis Lavoie b092061591 reverseproxy: Prevent copying the response if a response handler ran (#4388) 2021-10-18 14:00:43 -04:00
Y.Horie 64f8b557b1 fileserver: Fix compression breaks using httpInclude (#4352) (#4358) 2021-10-16 11:09:16 -04:00
Matthew Holt 95c035060f map: Fix regex mappings
It didn't really make sense how we were doing them before. See https://caddy.community/t/map-directive-and-regular-expressions/13866/6?u=matt
2021-10-13 17:58:20 -06:00
Matthew Holt c4790d7f9d go.mod: Carefully upgrade some dependencies (fix #4251)
The upgrade of smallstep/certificates fixes #4251. The upgrade of CertMagic fixes an issue reported in the forum that a longer timeout was confirmed to resolve (without any particular explanation, but oh well). Other upgrades have minor improvements and seem safe.
2021-10-12 01:08:28 -06:00
Simão Gomes Viana 837cdc566d caddyhttp: reverseproxy: clarify warning for -insecure (#4379)
The question would only receive bad answers so it's better
to just say what the option actually does.
2021-10-11 16:15:00 -06:00
M. Ángel Jimeno be5f77e84d caddycmd: fix caddy validate/fmt help message (#4377)
* caddycmd: fix caddy validate help message

Fixes #4376

* caddycmd: fix caddy fmt help message
2021-10-11 11:56:03 -04:00
Oleg cbb045a121 caddyhttp: Placeholder for client cert in DER + base64 format (#4241)
* client.certificate_pem_encoded in base64 format

* base64-encoding without pem encoding;naming change

* fix cert.Raw instead of block.bytes
2021-10-01 16:27:29 -06:00
KallyDev c48fadc4a7 Move from deprecated ioutil to os and io packages (#4364) 2021-09-29 11:17:48 -06:00
Matthew Holt 059fc32f00 Revert 3336faf2 (close #4360)
Debug log is correct level for this
2021-09-27 12:06:06 -06:00
Matthew Holt e2d964ea30 Add explanation for project name to readme 2021-09-27 10:33:32 -06:00
Matthew Holt 501da21f20 General minor improvements to docs 2021-09-24 18:31:01 -06:00
Matthew Holt 3336faf254 reverseproxy: Log error at error level (fix #4360) 2021-09-24 18:29:23 -06:00
Tim Culverhouse 16f752125f templates: Add tests for funcInclude and funcImport (#4357)
* Update tplcontext.go

Add {{ render "/path/to/file.ext" $data }} via funcRender

* Update tplcontext.go

* Refactor funcInclude, add funcImport to enable {{block}} and {{template}}

* Fix funcImport return of nil showing up in html

* Update godocs for  and

* Add tests for funcInclude

* Add tests for funcImport

* os.RemoveAll -> os.Remove for TestFuncInclude and TestFuncImport
2021-09-20 12:29:37 -06:00
Slavik 0a5f7a677f fileserver: Make file listing links purple once visited (#4356) 2021-09-19 22:01:11 -06:00
HayatoShiba d3a0259944 fileserver: Fix displayed file size if it is symlink (#4354)
* Fix file size if it is symlink

* change the variable name for readability
2021-09-18 05:51:59 -06:00
Tim Culverhouse 5fda9610f9 templates: Add 'import' action (#4321)
Related to (closed) Issue #2094 on template inheritance. This PR adds a new function called "import" which works like "include", except it only takes one argument and passes it to the referenced file to be used as "." in that file.

* Update tplcontext.go

Add {{ render "/path/to/file.ext" $data }} via funcRender

* Update tplcontext.go

* Refactor funcInclude, add funcImport to enable {{block}} and {{template}}

* Fix funcImport return of nil showing up in html

* Update godocs for  and
2021-09-17 13:00:36 -06:00
Francis Lavoie 3f2c3ecf85 fastcgi: Implement try_files override in Caddyfile directive (#4347) 2021-09-17 08:23:06 -06:00
Francis Lavoie 907e2d8d3a caddyhttp: Add support for triggering errors from try_files (#4346)
* caddyhttp: Add support for triggering errors from `try_files`

* caddyhttp: Use vars instead of placeholders/replacer for matcher errors

* caddyhttp: Add comment for matcher error var key
2021-09-17 00:52:32 -06:00
Mohammed Al Sahaf 33c70f418f fileserver: properly handle escaped/non-ascii paths (#4332)
* fileserver: properly handle escaped/non-ascii paths

* fileserver: tests: accommodate Windows hate of colons in files names
2021-09-16 20:40:31 +00:00
Matthew Holt 2ebfda1ae9 Make copyright notice more consistent
Some files had the old copyright or were missing the license comment entirely.

Also change Light Code Labs to Dyanim in security contact and releases.
2021-09-16 12:50:32 -06:00
Matthew Holt 2392478bd3 templates: Propagate httpError to HTTP response
Now possible with Go 1.17.
See https://github.com/golang/go/issues/34201.
2021-09-15 09:55:57 -06:00
Matthew Holt a437206643 headers: Canonicalize case in replace (fix #4330) 2021-09-13 10:13:32 -06:00
Francis Lavoie a779e1b383 fastcgi: Fix Caddyfile parsing when handle_response is used (#4342) 2021-09-11 14:12:21 -06:00
Matthew Holt 46ab93be51 go.mod: Update CertMagic
Adds one more debug log
2021-09-03 11:42:13 -06:00
Mohammed Al Sahaf e0fc46a911 ci: revert workaround implemented in #4306 (#4328) 2021-09-03 10:05:04 -04:00
peymaneh 9f6393c64c cmd: export CaddyVersion(), Commands() (#4316)
* cmd: Export CaddyVersion()

* cmd: Add getter Commands()
2021-09-01 18:08:02 -06:00
Francis Lavoie 105dac8c2a ci: Only test cross-build on latest Go version (#4319)
This generated way too many test jobs, which weren't really that useful. Cross-build is just to keep us posted on which architectures are building okay, so it's not necessary to do it twice. Only plan9 is not working at this point (see https://github.com/caddyserver/caddy/issues/3615)
2021-08-31 13:44:07 -06:00
Steffen Brüheim 4ebf100f09 encode: ignore flushing until after first write (#4318)
* encode: ignore flushing until after first write (fix #4314)

The first write will determine if encoding has to be done and will add an Content-Encoding. Until then Flushing has to be delayed so the Content-Encoding header can be added before headers and status code is written. (A passthrough flush would write header and status code)

* Update modules/caddyhttp/encode/encode.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-08-31 13:36:36 -06:00
Matthew Holt f43fd6f388 go.mod: Upgrade CertMagic to v0.14.4
Adds more debug logging
2021-08-30 13:14:42 -06:00
Matthew Holt 84b906a248 go.mod: Upgrade some dependencies 2021-08-26 15:00:25 -06:00
Francis Lavoie 403732c433 httpcaddyfile: Reorder some directives (#4311)
We realized we made some mistakes with the directive ordering, so we're making some minor adjustments.

`abort` and `error` don't really make sense to be after other handler directives, because you would expect to be able to "fail-fast" and throw an error before falling through to some `file_server` or `respond` typically. So we're moving them up to just before `respond`, i.e. before the common handler directives. 

This is also more consistent with our existing examples in the docs, which actually didn't work due to the directive ordering. See https://caddyserver.com/docs/caddyfile/directives/error#examples

Also, `push` doesn't quite make sense to be after `handle`/`route`, since its job is to read from response headers to push additional resources if necessary, and `handle`/`route` may be terminal so push would not be reached if it was declared outside those. And also, it would make sense to be _before_ `templates` because a template _could_ add a `Link` header to the response dynamically.
2021-08-26 14:31:55 -06:00
Francis Lavoie f6d5ec2fd6 chore: Upgrade smallstep libs (#4307)
See https://github.com/smallstep/nosql/issues/12 for context.
2021-08-25 12:16:55 -06:00
Mohammed Al Sahaf 19a55d6aeb chore: promote creating 'caddy-build' to the release action (#4306)
The commit goreleaser/goreleaser@013bd69126 of GoReleaser is now checking the `go version` prior to executing any of the pre-hooks, which involves setting the current dir of the command to the `build.dir` of the build config. At the time of version check, the buil dir does not exist. It's created in the pre-hook. As a workaround, the build-dir is now created in the Github Action prior to executing goreleaser action.
2021-08-25 17:30:24 +00:00
Matthew Holt bfbc459c0a httpcaddyfile: Improve unrecognized directive errors 2021-08-25 10:30:39 -06:00
Francis Lavoie f70a7578fa reverseproxy: Remove redundant flushing (#4299)
From reading through the code, I think this code path is now obsoleted by the changes made in https://github.com/caddyserver/caddy/pull/4266.

Basically, `h.flushInterval()` will set the flush interval to `-1` if we're in a bi-directional stream, and the recent PR ensured that `h.copyResponse()` properly flushes headers immediately when the flush interval is non-zero. So now there should be no need to call Flush before calling `h.copyResponse()`.
2021-08-23 11:54:28 -06:00
Francis Lavoie 51f125bd44 caddyfile: Better error message for missing site block braces (#4301)
Some new users mistakenly try to define two sites without braces around each. Doing this can yield a confusing error message saying that their site address is an "unknown directive".

We can do better by keeping track of whether the current site block was parsed with or without a brace, then changing the error message later based on that.

For example, now this invalid config:

```
foo.example.com
respond "foo"

bar.example.com
respond "bar"
```

Will yield this error message:

```
$ caddy adapt
2021/08/22 19:21:31.028 INFO    using adjacent Caddyfile
adapt: Caddyfile:4: unrecognized directive: bar.example.com
Did you mean to define a second site? If so, you must use curly braces around each site to separate their configurations.
```
2021-08-23 11:53:27 -06:00
Francis Lavoie d74913f871 caddyfile: Error on invalid site addresses containing comma (#4302)
Some users forget to use a comma between their site addresses. This is invalid (commas aren't a valid character in domains) and later parts of the code like certificate automation will try to use this otherwise, which doesn't make sense. Best to error as early as possible.

Example thread on the forums where this happened: https://caddy.community/t/simplify-caddyfile/13281/9
2021-08-23 11:26:07 -06:00
Pascal Zarrad ce5a45db45 cmd: Fix paths when using an env file (#4296)
* core: Fix paths when using an env file

* refactor: move path logic to loadFromEnv
2021-08-20 15:51:31 -06:00
Adam Weinberger e0a6a1efff chore: Update quic-go for go 1.17 support (#4297)
* Update quic-go for go 1.17 support

* Complete quic-go update (go mod tidy)
2021-08-20 10:19:16 -06:00
Scott Mebberson c1cd192ee7 caddyhttp: Updated the documentation for MatchQuery (#4295) 2021-08-19 22:44:28 -06:00
Francis Lavoie a056fcd7ba chore: Upgrade smallstep libs (#4291)
See https://github.com/smallstep/nosql/issues/12 for context.
2021-08-19 16:08:19 -06:00
M. Ángel Jimeno 9e333c39da cmd: use net.ErrClosed for matching returned error (#4289)
Implements #3805
2021-08-18 12:58:19 -06:00
Matthew Holt 8a974a4f8f logging: Warn for deprecated single_field encoder 2021-08-17 10:51:26 -06:00
Francis Lavoie 6bc87ea2ff ci: Start testing on Go 1.17, drop 1.15 (#4283) 2021-08-16 21:56:20 -06:00
Rainer Borene 1b1e625c20 core: Unix ns and Unix ms time placeholders (#4280) 2021-08-16 15:06:44 -06:00
Steven Angles a10910f398 admin: Sync server variables (fix #4260) (#4274)
* Synchronize server assignment/references to avoid data race

* only hold lock during var reassignment
2021-08-16 15:04:47 -06:00
Francis Lavoie ab32440b21 httpcaddyfile: Add shortcut for proxy hostport placeholder (#4263)
* httpcaddyfile: Add shortcut for proxy hostport placeholder

I've noticed that it's a pretty common pattern to write a proxy like this, when needing to proxy over HTTPS:

```
reverse_proxy https://example.com {
	header_up Host {http.reverse_proxy.upstream.hostport}
}
```

I find it pretty hard to remember the exact placeholder to use for this, and I continually need to refer to the docs when I need it. I think a simple fix for this is to add another Caddyfile placeholder for this one to shorten it:

```
reverse_proxy https://example.com {
	header_up Host {proxy_hostport}
}
```

* Switch the shortcut name
2021-08-12 12:08:37 -06:00
Francis Lavoie e6c29ce081 reverseproxy: Incorporate latest proxy changes from stdlib (#4266)
I went through the commits that touched stdlib's `reverseproxy.go` file, and copied over all the changes that are to code that was copied into Caddy.

The commits I pulled changes from:

- https://github.com/golang/go/commit/2cc347382f4df3fb40d8d81ec9331f0748b1c394
- https://github.com/golang/go/commit/a5cea062b305c8502bdc959c0eec279dbcd4391f
- https://github.com/golang/go/commit/ecdbffd4ec68b509998792f120868fec319de59b
- https://github.com/golang/go/commit/21898524f66c075d7cfb64a38f17684140e57675
-https://github.com/golang/go/commit/ca3c0df1f8e07337ba4048b191bf905118ebe251
- https://github.com/golang/go/commit/9c017ff30dd21bbdcdb11f39458d3944db530d7e

This may also fix https://github.com/caddyserver/caddy/issues/4247 because of the change to `copyResponse` to set `mlw.flushPending = true` right away.
2021-08-12 10:48:24 -06:00
Oleg 68c5c71659 cmd: New add-package and remove-package commands (#4226)
* adding package command

* add-package command name

* refactoring duplicate code

* fixed by review

* fixed by review

* remove-package command

* commands in different files, common utils

* fix add, remove, upgrade packages in 1 file

* copyright and downloadPath moved

* refactor

* downloadPath do no export

* adding/removing multiple packages

* addPackages/removePackages, comments, command-desc

* add-package, process case len(args) == 0

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-08-11 17:31:41 -06:00
Frederik Ring 569ecdbd02 httpcaddyfile: Ensure hosts to skip for logs can always be collected (#4258)
* httpcaddyfile: ensure hosts to skip can always be collected

Previously, some hosts that should be skipped in logging would
be missed as the current logic would only collect them after
encountering the first server that would log. This change makes sure
the ServerLogConfig is initialized before iterating over the server
blocks.

* httpcaddyfile: add test case for skip hosts behavior
2021-08-02 14:15:27 -06:00
王清雨 c131339c5c admin: Implement load_interval to pull config on a timer (#4246)
* feat: implement a simple timer to pull config

mostly referenced to the issue

re #4106

* Update admin.go

use `caddy.Duration`

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update caddy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update admin.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* fix: sync load config when no pull interval provided

try not to make break change

* fix: change PullInterval to LoadInterval

* fix: change pull_interval to load_interval

* Update caddy.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-07-28 15:39:08 -06:00
Ggicci b6f51254ea caddyfile: keep error chain info in Dispenser.Errf (#4233)
* caddyfile: Errf enable error chain unwrapping

* refactor: remove parseError
2021-07-19 08:35:14 -06:00
Francis Lavoie 124ba1ba71 logging: Prep for common_log removal (#4149)
See https://github.com/caddyserver/caddy/issues/4148#issuecomment-833207811
2021-07-14 11:07:38 -06:00
Francis Lavoie 1c6c7714a3 caddyhttp: Fix edgecase with auto HTTP->HTTPS logic (#4243) 2021-07-14 10:49:34 -06:00
Leo Di Donato 46d99aba85 logging: Add missing interface guards for replace filter (#4244)
Signed-off-by: Leonardo Di Donato <leodidonato@gmail.com>
2021-07-12 11:13:01 -04:00
diamondburned 9e16e80f3c fileserver: Fix browse name_dir_first sorting (#4218)
This commit fixes the `sortByNameDirFirst` variable inside fileserver to
match what browse's default template has.

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-07-07 02:54:54 -04:00
elnoro d882211080 reverseproxy: Keep path to unix socket as dial address (#4232) 2021-07-06 23:43:45 -04:00
hmol233 42e140b1b2 caddyhttp: Fix incorrect determination of gRPC protocol (#4236) 2021-07-06 12:09:44 -04:00
mritd 4245ceb67d fileserver: Add disable_canonical_uris Caddyfile subdirective (#4222)
* feat(fileserver): add 'canonical_uris' parameter to caddyfile

add 'canonical_uris' parameter to caddyfile

reference #2741

Signed-off-by: mritd <mritd@linux.com>

* feat(file_server): rename subdirective canonical_uris to disable_canonical_uris

rename subdirective canonical_uris to disable_canonical_uris

Signed-off-by: mritd <mritd@linux.com>

* test(caddyfile_adapt): add disable_canonical_uris subdirective test file

add disable_canonical_uris subdirective test file

Signed-off-by: mritd <mritd@linux.com>
2021-07-01 17:22:16 -06:00
Matthew Holt 0bdb8aa82d acmeserver: Don't set host for directory links by default
This makes the server more easily proxied.
2021-07-01 17:20:51 -06:00
Matthew Holt 191dc86f9e fileserver: Clarify docs about canonicalization
Related to https://github.com/caddyserver/caddy/issues/4205.
2021-06-25 11:33:18 -06:00
Matthew Holt 81e5318021 caddytls: Remove "IssuerRaw" field
Has been deprecated and printing warnings for about 8 months now.
Replaced by "IssuersRaw" field in v2.3.0.
2021-06-25 11:29:56 -06:00
Matthew Holt b3d35a4995 httpcaddyfile: Don't put localhost in public APs (fix #4220)
If an email is specified in global options, a site called 'localhost' shouldn't be bunched together with public DNS names in the automation policies, which get the default, public-CA issuers. Fix old test that did this.

I also noticed that these two:

    localhost {
    }
    example.com {
    }

and

    localhost, example.com {
    }

produce slightly different TLS automation policies. The former is what the new test case covers, and we have logic that removes the empty automation policy for localhost so that auto-HTTPS can implicitly create one. (We prefer that whenever possible.) But the latter case produces two automation policies, with the second one being for localhost, with an explicit internal issuer. It's not wrong, just more explicit than it needs to be.

I'd really like to completely rewrite the code from scratch that generates automation policies, hopefully there is a simpler, more correct algorithm.
2021-06-25 11:28:32 -06:00
Matthew Holt 2de7e14e1c acmeserver: Trim slashes from path prefix
See https://caddy.community/t/mtls-tls-internal-error/12807
2021-06-21 11:56:41 -06:00
Matthew Holt 885a9aaf48 go.mod: Update dependencies (close #4216) 2021-06-18 12:02:47 -06:00
Klaus Post 69c914483d encode: Tweak compression settings (#4215)
* Tweak compression settings

zstd: Limit window sizes to 128K to keep memory in control both server and client size.
zstd: Write 0 length frames. This may be needed for compatibility.
zstd: Create fewer encoders. Small memory improvement.
gzip: Allow -2 (Huffman only) and -3 (stateless) compression modes.

* Update modules/caddyhttp/encode/zstd/zstd.go

Update docs.

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-06-18 11:49:49 -06:00
Matt Holt 9d4ed3a323 caddyhttp: Refactor and export SanitizedPathJoin for use in fastcgi (#4207) 2021-06-17 09:59:08 -06:00
Matthew Holt fbd6560976 fileserver: Only redirect if filename not rewritten (fix #4205)
This is the more correct implementation of  23dadc0d86 (#4179)... I think. This commit effectively undoes the revert in 8848df9c5d, but with corrections to the logic.

We *do* need to use the original request path (the path the browser knows) for redirects, since they are external, and rewrites are only internal.

However, if the path was rewritten to a non-canonical path, we should not redirect to canonicalize that, since rewrites are intentional by the site owner. Canonicalizing the path involves modifying only the suffix (base element, or filename) of the path. Thus, if a rewrite involves only the prefix (like how handle_path strips a path prefix), then we can (hopefully!) safely redirect using the original URI since the filename was not rewritten.

So basically, if rewrites modify the filename, we should not canonicalize those requests. If rewrites only modify another part of the path (commonly a prefix), we should be OK to redirect.
2021-06-17 09:55:49 -06:00
Matthew Holt 238914d70b Some misc. cleanup
The fastcgi changes came from v1 which don't make sense in v2.

Fix comment about default value in reverse proxy keep alive.
2021-06-16 14:29:42 -06:00
Matthew Holt e8ae80adca fileserver: Don't persist parsed template (fix #4202)
Templates are parsed at request-time (like they are in the templates middleware) to allow live changes to the template while the server is running. Fixes race condition.

Also refactored use of a buffer so a buffer put back in the pool will not continue to be used (written to client) in the meantime.

A couple of benchmarks removed due to refactor, which is fine, since we know pooling helps here.
2021-06-16 14:28:34 -06:00
Matthew Holt 32c284b54a reverseproxy: Adjust test related to #4201
Commit 7c68809f4e
2021-06-15 15:02:22 -06:00
Matthew Holt 7c68809f4e reverseproxy: Fix overwriting of max_idle_conns_per_host (closes #4201)
Also split the Caddyfile subdirective keepalive_idle_conns into two properties so the conns and conns_per_host can be set separately.

This is technically a breaking change, but probably anyone who this breaks already had a broken config anyway, and silently fixing it won't help them fix their configs.
2021-06-15 14:54:48 -06:00
Matthew Holt 6d25261c22 Expand and clarify security policy
While the Caddy project has had very few valid security bug reports over the years, we have a low signal-to-noise ratio with them (lots of invalid reports). Most are out of scope, and it can take too much valuable time for us to determine that. We would prefer researchers do this first. Hopefully these paragraphs spell out much more clearly what we do and don't accept.
2021-06-14 14:00:43 -06:00
Matthew Holt 8848df9c5d Revert "fileserver: Redirect within the original URL (#4179)"
This reverts commit f9b54454a1.
/cc @diamondburned (see #4205)
2021-06-14 09:04:30 -06:00
Matt Holt 89aa3a5ef3 go.mod: Use CertMagic v0.14.0 (fix #4191)
* Force auto-renew for OCSP revoked status (maybe) (fix #4191)

* Use latest commit

* go.mod: Use CertMagic v0.14.0 (fix #4191)

Correctly replaces revoked certificates
2021-06-12 14:44:32 -06:00
Matthew Holt 05656a60b3 httpcaddyfile: Don't add HTTP hosts to TLS APs (fix #4176 and fix #4198)
In the Caddyfile, hosts specified for HTTP sockets (either scheme is "http" or it is on the HTTP port) should not be used as subjects in TLS automation policies (APs).
2021-06-09 14:35:09 -06:00
Klooven 1e92258dd6 httpcaddyfile: Add preferred_chains global option and issuer subdirective (#4192)
* Added preferred_chains option to Caddyfile

* Caddyfile adapt tests for preferred_chains
2021-06-08 14:10:37 -06:00
diamondburned 76913b19ff fileserver: Fix browse not redirecting query parameters (#4196)
This commit is a follow up to PR #4179 that introduced a bug where
browse redirections to the right URL would not preserve query
parameters.
2021-06-07 17:33:54 -06:00
Peter Magnusson 4c2da18841 caddytls: Add Caddyfile support for propagation_timeout (#4178)
* add propagation_timeout to UnmarshalCaddyfile

- Closes #4177

* added caddyfile_adapt test
2021-06-07 12:25:12 -06:00
diamondburned f9b54454a1 fileserver: Redirect within the original URL (#4179)
This commit changes the file_server directive to redirect using the
original request's URL instead of the possibly trimmed URL. This should
make file_server work with handle_path.

This fix is taken from mholt's comment in
https://caddy.community/t/file-servers-on-different-paths-not-working/11698/11.
2021-06-07 12:20:08 -06:00
Francis Lavoie 658772ff24 httpcaddyfile: Add skip_install_trust global option (#4153)
Fixes https://github.com/caddyserver/caddy/issues/4002
2021-06-07 12:18:49 -06:00
Matthew Holt 323ffd2076 admin: Replace admin cert cache when reloading (fix #4184) 2021-06-05 11:47:44 -06:00
Matthew Holt 2a8109468c reverseproxy: Always remove hop-by-hop headers
See golang/go#46313

Based on https://github.com/golang/go/commit/950fa11c4cb01a145bb07eeb167d90a1846061b3
2021-06-04 15:21:16 -06:00
Francis Lavoie 94b712009a logging: Actually use level_key (#4189) 2021-06-04 14:15:43 -06:00
Dave Henderson 7b500e74b4 metrics: use buildinfo collector from new collectors pkg (#4187)
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2021-06-04 00:19:16 -04:00
Matthew Holt ecd5eeab38 go.mod: Update direct dependencies 2021-06-03 12:18:25 -06:00
Matt Holt b4cef492cc Update .goreleaser.yml
Ubuntu's package updater doesn't show the name of the package, so just adding "Caddy" to the description helps a lot
2021-05-24 16:21:53 -06:00
Matt Holt e3c369d452 logging: Implement dial timeout for net writer (fix #4083) (#4172)
* logging: Implement dial timeout for net writer (fix #4083)

* Limit how often redials are attempted

This should cause dial blocking to occur only once every 10 seconds at most, but it also means the logger connection might be down for up to 10 seconds after it comes back online; oh well. We shouldn't block for DialTimeout at every single log emission.

* Clarify offline behavior
2021-05-19 15:14:03 -06:00
Matthew Holt c052162203 Merge branch '2.4' 2021-05-19 10:47:06 -06:00
Matthew Holt 7f26a6b3e5 admin: Reinstate internal redirect for /id/ requests
Fix regression from ab80ff4fd2 (probably a mistake when rebasing)

See https://caddy.community/t/id-selector-is-not-working-after-upgrade-to-2-4-0/12513?u=matt
2021-05-19 10:27:25 -06:00
Francis Lavoie b82db994f3 caddyfile: Add parse error on site address with trailing { (#4163)
* caddyfile: Add parse error on site address in `{`

This is an incredibly common mistake made by users, so we should catch it earlier in the parser and give a more friendly message. Often it ends up adapting but with mistakes, or erroring out later due to other site addresses being read as directives.

There's not really ever a situation where a lone '{' is valid at the end of a site address (but I suppose there are edgecases where the user wants to use a path matcher where it ends specifically in `{`, but... why?), so this should be fine.

* Update caddyconfig/caddyfile/parse.go
2021-05-12 16:18:44 -06:00
Francis Lavoie aef8d4decc reverseproxy: Set the headers in the replacer before handle_response (#4165)
Turns out this was an oversight, we assumed we could use `{http.response.header.*}` but that doesn't work because those are grabbed from the response writer, and we haven't copied any headers into the response writer yet.

So the fix is to set all the response headers into the replacer at a new namespace before running the handlers.

This adds the `{http.reverse_proxy.header.*}` replacer.

See https://caddy.community/t/empty-http-response-header-x-accel-redirect/12447
2021-05-12 14:19:08 -06:00
Francis Lavoie 37718560c1 ci: Run CI on PRs targeting minor version branches (#4164)
We decided that we'll use branches like `2.4` as the target for any changes that we might want to release in a `2.4.x` version like `2.4.1`, so that we can continue to merge changes targeting the next minor release (e.g. `2.5.0`) on master.

Our CI config wasn't set up for this to work properly though, since it was only running checks on PRs targeting master. This should fix it.

I couldn't find a way to do a pattern to only match digits for the branch names from Github's docs, it just looks like a pretty generic glob syntax. But this should do until we get to 3.0
2021-05-12 00:26:16 -04:00
Mohammed Al Sahaf 2aefe15686 cmd: upgrade: inherit the permissions of the original executable (#4160) 2021-05-11 16:11:27 -06:00
Matthew Holt dbe164d98a httpcaddyfile: Fix automation policy consolidation again (fix #4161)
Also fix a previous test that asserted incorrect behavior.
2021-05-11 15:26:07 -06:00
Matthew Penner bc22102478 caddyfile: Fix caddy fmt nesting not decrementing (#4157)
* caddyfile(formatter): fix nesting not decrementing

This is an extremely weird edge-case where if you had a environment variable {}
on one line, a comment on the next line, and the closing of the block on the
following line; the rest of the Caddyfile would be indented further than it
should've been.

ref; https://github.com/matthewpi/vscode-caddyfile-support/issues/13

* run gofmt

* fmt: better way of handling edge case
2021-05-10 12:01:27 -06:00
Francis Lavoie f5db41ce1d encode: Drop prefer from Caddyfile (#4156)
Followup to #4150, #4151 /cc @ueffel @polarathene

After a bit of discussion with @mholt, we decided to remove `prefer` as a subdirective and just go with using the order implicitly always. Simpler config, simpler docs, etc.

Effectively changes 7776471 and reverts a small part of f35a7fa.
2021-05-10 11:12:59 -06:00
Francis Lavoie 77764714ad encode: Default to order the formats are enabled for prefer in Caddyfile (#4151) 2021-05-10 10:06:38 -06:00
Francis Lavoie 61642b766b caddytls: Run replacer on ask URL, for env vars (#4154)
Fixes #3922
2021-05-08 22:37:27 -06:00
Francis Lavoie 3cf443f0fe httpcaddyfile: Add grace_period global option (#4152)
See https://caddyserver.com/docs/json/apps/http/#grace_period
2021-05-07 16:18:17 -06:00
Francis Lavoie d4b2f1bcee caddyhttp: Fix fallback for the error handler chain (#4131)
* caddyhttp: Fix fallback for the error handler chain

The fix I went with in the end (after realizing some mistaken assumptions in #4131) is to just make the routes fall back to errorEmptyHandler instead of the non-error empty handler, if Terminal is true, making the routes error-aware. Ultimately this was probably just an oversight when errors was implemented at some point in the early betas of v2.

See https://caddy.community/t/problem-with-basicauth-handle-errors/12243/9 for context.

* Revert "caddyhttp: Fix fallback for the error handler chain"

This reverts commit 95b6ac44a6.

* caddyhttp: Fix via `routes.go`
2021-05-05 15:55:40 -06:00
Matthew Holt a17c3b568d reverseproxy: Minor logging improvements 2021-05-05 14:52:24 -06:00
Francis Lavoie 74f5d66c48 fileserver: Fix file matcher with empty try_files (#4147)
* fileserver: Fix `file` matcher with empty `try_files`

Fixes https://github.com/caddyserver/caddy/issues/4146

If `TryFiles` is empty, we fill it with `r.URL.Path`. In this case, this is `/`. Then later, in `prepareFilePath()`, we run the replacer (which turns `{path}` into `/` at that point) but `file` remains the original value (and the placeholder is still the placeholder there).

So then `strings.HasSuffix(file, "/")` will be `false` for the placeholder, but `true` for the empty `TryFiles` codepath, because `file` was `/` due to being set to the actual request value beforehand.

This means that `suffix` becomes `//` in that case, so after `sanitizedPathJoin`, it becomes `./`, so `strictFileExists`'s `strings.HasSuffix(file, separator)` codepath will return true.

I think we should change the `m.TryFiles == nil` codepath to `m.TryFiles = []string{"{http.request.uri.path}"}` for consistency. (And maybe consider hoisting this to `Provision` cause there's no point doing this on every request). I don't think this "optimization" of directly using `r.URL.Path` is so valuable, cause it causes this edgecase with directories.

* Update modules/caddyhttp/fileserver/matcher.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-05-04 09:49:13 -06:00
Matthew Holt efe84497d7 go.mod: CertMagic v0.13.1 2021-05-03 11:10:18 -06:00
Francis Lavoie e4a22de9d1 reverseproxy: Add handle_response blocks to reverse_proxy (#3710) (#4021)
* reverseproxy: Add `handle_response` blocks to `reverse_proxy` (#3710)

* reverseproxy: complete handle_response test

* reverseproxy: Change handle_response matchers to use named matchers

reverseproxy: Add support for changing status code

* fastcgi: Remove obsolete TODO

We already have d.Err("transport already specified") in the reverse_proxy parsing code which covers this case

* reverseproxy: Fix support for "4xx" type status codes

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* caddyhttp: Reorganize response matchers

* reverseproxy: Reintroduce caddyfile.Unmarshaler

* reverseproxy: Add comment mentioning Finalize should be called

Co-authored-by: Maxime Soulé <btik-git@scoubidou.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-05-02 12:39:06 -06:00
Jesse e6f6d3a476 cmd: Add --envfile flag to start command (#4141)
* add envfile in start cmd

* fix commandfuncs

* fix commandfuncs

* fix cmdStart envfile from bool to string

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2021-05-02 12:38:16 -06:00
Francis Lavoie ef7f15f3a4 httpcaddyfile: Add auto_https ignore_loaded_certs (#4077) 2021-05-02 12:11:27 -06:00
Francis Lavoie 6e0e3e1537 httpcaddyfile: Add global option for storage_clean_interval (#4134)
Followup to https://github.com/caddyserver/caddy/commit/42b7134ffa3bf3e9e86514c82407979c2627a5ab
2021-05-02 11:57:28 -06:00
Calvin Xiao 53ececda21 caddyhttp: performance improvement in HeaderRE Matcher (#4143)
Below is the report using `benchstat` and cmd:

`go test -run=BenchmarkHeaderREMatcher -bench=BenchmarkHeaderREMatcher -benchmem -count=10`

```
name                old time/op    new time/op    delta
HeaderREMatcher-16     869ns ± 1%     658ns ± 0%  -24.29%  (p=0.000 n=10+10)

name                old alloc/op   new alloc/op   delta
HeaderREMatcher-16      144B ± 0%      112B ± 0%  -22.22%  (p=0.000 n=10+10)

name                old allocs/op  new allocs/op  delta
HeaderREMatcher-16      7.00 ± 0%      5.00 ± 0%  -28.57%  (p=0.000 n=10+10)
```
2021-05-02 10:35:28 -06:00
Jason Du 637fd8f67b fileserver: Share template logic for both templates and file_server browse (#4093)
Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2021-04-30 22:17:23 -04:00
Matt Holt 956f01163d caddytls: Implement remote IP connection matcher (#4123)
* caddytls: Implement remote IP connection matcher

* Implement IP range negation

If both Ranges and NotRanges are specified, both must match.
2021-04-30 10:14:52 -06:00
Alban Lecocq ff6ca577ec httpcaddyfile: Fix unexpectedly removed policy (#4128)
* httpcaddyfile: Fix unexpectedly removed policy

When user set on_demand tls option in a catch-all (:443) policy,
we expect other policies to not have the on_demand enabled
See ex in tls_automation_policies_5.txt

Btw, we can remove policies if they are **all** empty.

* Update caddyconfig/httpcaddyfile/tlsapp.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2021-04-29 10:56:01 -06:00
Simão Gomes Viana 9017557169 reverseproxy: fix hash selection policy (#4137)
* caddyhttp: reverseproxy: fix hash selection policy

Fixes: #4135
Test: go test './...' -count=1

* caddyhttp: reverseproxy: add test to catch #4135

If you revert the last commit, the test will fail.
2021-04-29 10:52:22 -06:00
Francis Lavoie 3a1e81dbf6 fileserver: Better handling of HTTP status override (#4132) 2021-04-29 02:01:48 -04:00
Francis Lavoie a8d45277ca caddyfile: Fix import replacing unrelated placeholders (#4129)
* caddyfile: Fix `import` replacing unrelated placeholders

See https://caddy.community/t/snippet-issue-works-outside-snippet/12231

So it turns out that `NewReplacer()` gives a replacer with some global defaults (like `{env.*}` and some system and time placeholders), which is not ideal when running `import` because we just want to replace `{args.*}` only, and nothing else.

* caddyfile: Add test
2021-04-22 18:29:04 -06:00
Francis Lavoie 1e218e1d2e caddytls: Add load_storage module (#4055)
An idea that came up in https://caddy.community/t/save-internally-issued-wildcard-certificate-in-consul/11740, this a simple module that might be useful for anyone who uses storage modules that aren't filesystem, to let them load certs/keys externally issued for use by Caddy.

Bit goofy, since we need to fetch the certmagic.Storage during provisioning, it needs a wrapping struct instead of just being an array like `load_files`.

Future work might involve adding Caddyfile support via a subdirective of the `tls` directive maybe?
2021-04-21 17:05:55 -06:00
Francis Lavoie 4d0474e3b8 reverseproxy: Admin endpoint for reporting upstream statuses (#4125) 2021-04-21 13:43:34 -06:00
Francis Lavoie d789596bc0 caddyhttp: Implement better logic for inserting the HTTP->HTTPS redirs (#4033)
* caddyhttp: Implement better logic for inserting the HTTP->HTTPS redirs

* caddyhttp: Add integration test
2021-04-19 19:54:12 -06:00
Matthew Holt 96bb365929 httpcaddyfile: Take into account host scheme/port (fix #4113) 2021-04-16 11:17:22 -06:00
Mohammed Al Sahaf 00e12aa918 fuzz: fix the FuzzFormat comparison (#4117) 2021-04-15 23:03:54 +00:00
Matthew Holt 2250920e1d caddytls: Disable OCSP stapling for manual certs (#4064) 2021-04-12 16:09:02 -06:00
Matthew Holt 42b7134ffa caddytls: Configurable storage clean interval
Can drastically reduce costs on storage backends where scans are expensive.

Also reduced default interval to 24h.

See https://github.com/silinternational/certmagic-storage-dynamodb/issues/18
2021-04-12 15:41:22 -06:00
Mohammed Al Sahaf 3903642aa7 caddyfile: reject cyclic imports (#4022)
* caddyfile: reject recursive self-imports

* caddyfile: detect and reject cyclic imports of snippets and files

* caddyfile: do not be stickler about connected nodes not being connected already

* caddyfile: include missing test artifacts of cyclic imports

* address review comments
2021-04-09 12:06:25 -06:00
Mohammed Al Sahaf 03b5debd95 ci: fuzz: add 4 more fuzzing targets (#4105) 2021-04-08 11:45:19 -06:00
Francis Lavoie 3f6283b385 fileserver: Add status code override (#4076)
After reading a question about the `handle_response` feature of `reverse_proxy`, I realized that we didn't have a way of serving an arbitrary file with a status code other than 200. This is an issue in situations where you want to serve a custom error page in routes that are not errors, like the aforementioned `handle_response`, where you may want to retain the status code returned by the proxy but write a response with content from a file.

This feature is super simple, basically if a status code is configured (can be a status code number, or a placeholder string) then that status will be written out before serving the file - if we write the status code first, then the stdlib won't write its own (only the first HTTP status header wins).
2021-04-08 11:09:12 -06:00
Carl George 45fb7202ac notify: Send all sd_notify signals from main caddy process (#4060)
Initial sd_notify support was added in #3963, but that sent signals from
both cmdRun and cmdReload.  This approach has two drawbacks:

- Reloads initiated via the API do not send signals.
- The signals are sent from different processes, which requires the
  `NotifyAccess=exec` directive in the unit file.

This change moves the NotifyReloading and NotifyReadiness invocations to
Load, which address both of those drawbacks.  It also adds a
complimentary NotifyStopping method which is invoked from handleStop.
All the notify methods are defined in a notify package to avoid an
import loop.
2021-04-05 14:01:20 -06:00
Marten Seemann 66783eb4d9 go.mod: Update quic-go to v0.20.1 (#4075) 2021-04-05 13:09:30 -06:00
Francis Lavoie 1455d6bb69 httpcaddyfile: Fix panic in automation policy consolidation (#4104)
* httpcaddyfile: Add reproduce test

* httpcaddyfile: Don't allow `i` to go below zero
2021-04-02 16:47:04 -06:00
Francis Lavoie 3401f91dbe caddyfile: Normalize line endings before comparing fmt result (#4103) 2021-04-02 11:55:34 -06:00
Mohammed Al Sahaf eb3955a960 ci: accommodate go1.16 changes to go mod (#4102)
As of go1.16, the `go` commands will no longer make automatic changes to go.{mod,sum} files (see: https://blog.golang.org/go116-module-changes). This broke the release script which relied on `go mod download` and/or `go build` to automatically generate the go.sum file. This commit explicitly invokes `go mod tidy` to have the go.sum file generated.
2021-04-01 16:07:32 -06:00
Matthew Holt d21e88ae3a Minor tweaks 2021-04-01 12:49:51 -06:00
Matthew Holt a0a7c60cb9 go.mod: Use latest CertMagic 2021-04-01 12:48:59 -06:00
Matthew Holt 7da9241fd7 Use 600 instead of 644 for UUID file
Are you happy, linter?
2021-04-01 12:12:28 -06:00
Matthew Holt e68dbe9cf8 Change os to ioutil for now
My editor automatically changed ioutil.ReadFile() to os.ReadFile() in accordance
with Go 1.16 changes. I didn't notice this until pushing.

But we still have to support Go 1.15 for a little while.
2021-03-30 15:42:04 -06:00
Dimitri Masson bd357bf005 reverseproxy: Set cookie path to / when using cookie lb_policy (#4096) 2021-03-30 15:29:00 -06:00
Matthew Holt aac1ccf12d caddy: Add InstanceID() method
Caddy can now generate and persist its own instance ID, a UUID that is stored in
the data directory.

This makes it possible to differentiate it from other instances in a cluster.
2021-03-30 14:15:20 -06:00
Steffen Brüheim f35a7fa466 encode,staticfiles: Content negotiation, precompressed files (#4045)
* encode: implement prefer setting

* encode: minimum_length configurable via caddyfile

* encode: configurable content-types which to encode

* file_server: support precompressed files

* encode: use ReponseMatcher for conditional encoding of content

* linting error & documentation of encode.PrecompressedOrder

* encode: allow just one response matcher

also change the namespace of the encoders back, I accidently changed to precompressed >.>
default matchers include a *  to match to any charset, that may be appended

* rounding of the PR

* added integration tests for new caddyfile directives
* improved various doc strings (punctuation and typos)
* added json tag for file_server precompress order and encode matcher

* file_server: add vary header, remove accept-ranges when serving precompressed files

* encode: move Suffix implementation to precompressed modules
2021-03-29 18:47:19 -06:00
Francis Lavoie 75f797debd reverseproxy: Implement health_uri, deprecate health_path, supports query (#4050)
* reverseproxy: Implement health_uri, replaces health_path, supports query

Also fixes a bug with `health_status` Caddyfile parsing , it would always only take the first character of the status code even if it didn't end with "xx".

* reverseproxy: Rename to URI, named logger, warn in Provision (for JSON)
2021-03-29 18:36:40 -06:00
Simão Gomes Viana 1c8ea00828 go.mod: Migrate to golang.org/x/term (#4073)
golang.org/x/crypto/ssh/terminal is deprecated in favor of golang.org/x/term

See https://github.com/caddyserver/caddy/pull/4073/checks?check_run_id=2152150495
Error: SA1019: package golang.org/x/crypto/ssh/terminal is deprecated: this package moved to golang.org/x/term.  (staticcheck)

See https://github.com/caddyserver/caddy/pull/4073/checks?check_run_id=2152228516
Error: SA1019: package golang.org/x/crypto/ssh/terminal is deprecated: this package moved to golang.org/x/term.  (staticcheck)

Test: go test -count=1 './...'
2021-03-29 12:39:08 -06:00
Simão Gomes Viana d63d5ae1ce caddyhttp: improve grammar of comment for AllowH2C (#4072) 2021-03-29 12:04:25 -06:00
Simão Gomes Viana a6bc58153b sigtrap_posix: add missing comma to SIGTERM info (#4078)
Was missing a comma, so added it
2021-03-29 11:04:22 -06:00
Francis Lavoie 911c8a371a cmd: Use formatted logger for config adapter warnings (#4080) 2021-03-29 11:03:50 -06:00
Simão Gomes Viana 87fbc0783a cmd: main: fix minor doc typos (#4082)
Fixed typos in the documentation
2021-03-29 11:02:21 -06:00
Francis Lavoie f1c36680fc headers: Fix Caddyfile parsing for request_header with matchers (#4085) 2021-03-29 10:55:29 -06:00
Simão Gomes Viana a87f757fcc .gitignore: add IDE files (#4087)
These tend to clutter up repositories.
2021-03-29 10:54:38 -06:00
Francis Lavoie 0018b9be0d fileserver: Add a few more debug lines (#4063) 2021-03-19 11:42:26 -06:00
rai a48c6205b7 fileserver: Browse listing supports dark mode (#4066)
* Add dark color scheme media query

* Theme search box, make everything less contrasting

* Further contrast tweaks
2021-03-19 11:41:02 -06:00
Simão Gomes Viana 28a4159933 CONTRIBUTING: fix spelling (#4070)
Minor spelling fixes to make this document even better
2021-03-19 07:37:43 -04:00
Francis Lavoie 0d7fe36007 httpcaddyfile: Add error directive for the existing handler (#4034)
* httpcaddyfile: Add `error` directive for the existing handler

* httpcaddyfile: Move `error` to the end of the order
2021-03-12 13:25:49 -07:00
Aaron Taylor f137b82227 logging: add replace filter for static value replacement (#4029)
This filter is intended to be useful in scenarios where you may want to
redact a value with a static string, giving you information that the
field did previously exist and was present, but not revealing the value
itself in the logs.

This was inspired by work on adding more complete support for removing
sensitive values from logs [1]. An example use case would be the
Authorization header in request log output, for which the value should
usually not be logged, but it may be quite useful for debugging to
confirm that the header was present in the request.

[1] https://github.com/caddyserver/caddy/issues/3958
2021-03-12 13:01:34 -07:00
Aaron Taylor 2a127ac3d1 caddyconfig: add global option for configuring loggers (#4028)
This change is aimed at enhancing the logging module within the
Caddyfile directive to allow users to configure logs other than the HTTP
access log stream, which is the current capability of the Caddyfile [1].
The intent here is to leverage the same syntax as the server log
directive at a global level, so that similar customizations can be added
without needing to resort to a JSON-based configuration.

Discussion for this approach happened in the referenced issue.

Closes https://github.com/caddyserver/caddy/issues/3958

[1] https://caddyserver.com/docs/caddyfile/directives/log
2021-03-12 13:00:02 -07:00
Rajat Jain 802f80c382 map: Accept regex substitution in outputs (#3991)
* Replace placeholders with regex groups

* using Matcher methods

* test added

* linting fix

* Revert "linting fix"

This reverts commit cafd7296f4.

* Revert "test added"

This reverts commit 3a76cc7b0b.

* Revert "using Matcher methods"

This reverts commit cc34337b8e.

* tests added
2021-03-10 14:22:33 -07:00
Francis Lavoie 51f35ba03f reverseproxy: Fix upstreams with placeholders with no port (#4046) 2021-03-03 10:12:31 -07:00
Matthew Holt ad8d01cb66 rewrite: Implement regex path replacements
https://caddy.community/t/collapsing-multiple-forward-slashes-in-path-only/11626
2021-03-01 18:27:59 -07:00
Matthew Holt 5bf0a55df4 fileserver: Don't replace in request paths (fix #4027) 2021-03-01 13:49:13 -07:00
Matthew Holt ec309c6d52 caddypki: Add SignWithRoot option for ACME server
See https://caddy.community/t/setting-up-a-caddy-pki-based-on-a-windows-
root-ca-was-getting-pki-config/11616/7

Also improved a godoc comment in the caddytls package.
2021-02-26 19:27:58 -07:00
Matthew Holt ce5a0934a8 reverseproxy: Fix round robin data race (#4038) 2021-02-25 09:41:52 -07:00
Matthew Holt b54fa41239 Update docs; commit setcap.sh 2021-02-24 11:55:56 -07:00
Matthew Holt 427bbe99d0 go.mod: Latest CertMagic (updated libdns conventions) 2021-02-24 11:50:23 -07:00
Matthew Holt a8fdc0a998 core: Initialize logging before admin
We'll see how this goes.

https://caddy.community/t/calling-the-admin-api-from-within-a-route-route-causes-a-deadlock/11319
2021-02-23 12:57:10 -07:00
Matthew Holt f6bb02b303 caddytls: Remove old asset migration code (close #3894) 2021-02-22 15:19:35 -07:00
Matt Holt 6722ae3a83 reverseproxy: Add duration/latency placeholders (close #4012) (#4013)
* reverseproxy: Add duration/latency placeholders (close #4012) (and #2268)

Adds 4 placeholders, one is actually outside reverse proxy though:

{http.request.duration} is how long since the server decoded the HTTP request (headers).
{http.reverse_proxy.upstream.latency} is how long it took a proxy upstream to write the response header.
{http.reverse_proxy.upstream.duration} is total time proxying to the upstream, including writing response body to client.
{http.reverse_proxy.duration} is total time spent proxying, including selecting an upstream and retries.

Obviously, most of these are only useful at the end of a request, like when writing response headers or logs.

See also: https://caddy.community/t/any-equivalent-of-request-time-and-upstream-header-time-from-nginx/11418

* Add new placeholders to documentation
2021-02-22 11:57:21 -07:00
Matthew Holt edb362aa96 httpcaddyfile: Fix catch-all site block sorting
A site block that has a catch-all and the shortest address is now sorted better.

https://caddy.community/t/caddy-suddenly-directs-my-site-to-the-wrong-directive/11597/2
2021-02-22 11:15:11 -07:00
Francis Lavoie 5376e5113e ci: Build and test on Go 1.16, bump minimum to 1.15 (#4024)
* ci: Build and test on Go 1.16

* ci: Drop Go 1.14 support
2021-02-18 07:09:49 -05:00
Matthew Holt ec3ac840cf caddy: Support SetReadBuffer and SyscallConn for QUIC (fix #3998)
Supersedes #3999
2021-02-16 14:05:31 -07:00
Matthew Holt fbd00e4b53 Improve security warnings 2021-02-16 14:05:31 -07:00
Francis Lavoie bafb562991 httpcaddyfile: Configure other apps from global options (#3990) 2021-02-15 20:10:27 -07:00
Matthew Holt ed678235a4 cmd: Clean up build-info and upgrade output 2021-02-15 12:07:55 -07:00
Matthew Holt cc63c5805e caddyhttp: Support placeholders in header matcher values (close #3916) 2021-02-11 16:27:09 -07:00
Matthew Holt 51e3fdba77 caddytls: Save email with account if not already specified
I'm pretty sure this fixes a bug when the default email is used...
2021-02-10 19:49:23 -07:00
Matthew Holt 5ef76ff3e6 reverseproxy: Response buffering & configurable buffer size
Proxy response bodies can now be buffered, and the size of the request body and
response body buffer can be limited. Any remaining content that doesn't fit in the
buffer will remain on the wire until it can be read; i.e. bodies are not truncated,
even if the buffer is not big enough.

This fulfills a customer requirement. This was made possible by their sponsorship!
2021-02-09 14:15:04 -07:00
Matthew Holt 653a0d3f6b httpcaddyfile: Fix automation policies
Fixes a bug introduced in #3862
2021-02-08 11:06:19 -07:00
Mohammed Al Sahaf 0aefa7b047 ci: deflake integration tests (#3966)
* ci: deflake integration tests by pulling Caddy for the running config until new config is loaded
2021-02-05 15:36:52 +00:00
Matthew Holt 8c291298c9 httpcaddyfile: Add resolvers subdir of tls (close #4008)
Allows conveniently setting the resolvers for the DNS challenge using a TLS subdirective, which applies to default issuers, rather than having to explicitly define the issuers and overwrite the defaults.
2021-02-02 23:07:50 -07:00
Matthew Holt bf50d7010a acmeserver: Support custom CAs from Caddyfile
The HTTP Caddyfile adapter can now configure the PKI app, and the acme_server directive can now be used to specify a custom CA used for issuing certificates. More customization options can follow later as needed.
2021-02-02 17:23:52 -07:00
Matthew Holt 8ec90f1c40 caddyhttp: Check for invalid subdirectives of static_response
Ref: https://caddy.community/t/acme-server-implementation/11256/
2021-02-02 16:19:58 -07:00
Matthew Holt 90284e8017 httpcaddyfile: Fix default issuers when email provided
If `tls <email>` is used, we should apply that to all applicable default issuers, not drop them. This refactoring applies implicit ACME issuer settings from the tls directive to all default ACME issuers, like ZeroSSL.

We also consolidate some annoying logic and improve config validity checks.

Ref: https://caddy.community/t/error-obtaining-certificate-after-caddy-restart/11335/8
2021-02-02 16:17:26 -07:00
Matthew Holt 2772ede43c cmd: Add --force flag to reload command (close #4005)
Can be useful if user wants to reload manual certificates, for example.
2021-02-01 18:14:03 -07:00
Matthew Holt c986110678 httpcaddyfile: Warn if site address uses unspecified IP (close #4004) 2021-02-01 17:02:01 -07:00
Matthew Holt 55e49ff5c8 httpcaddyfile: Sort catch-all site blocks properly (fix #4003) 2021-02-01 11:45:28 -07:00
Mohammed Al Sahaf e2940c8c03 ci: update the command to run tests on the s390x machine (#3995) 2021-01-28 22:40:36 +00:00
Tyler Kropp bef80cd806 caddyhttp: Fix redir html status code, improve flow (#3987)
* Fix html redir code, improve flow

* Fix integer check error and add tests
2021-01-28 12:59:50 -07:00
Matt Holt e2c5c28597 caddyhttp: Implement handler abort; new 'abort' directive (close #3871) (#3983)
* caddyhttp: Implement handler abort; new 'abort' directive (close #3871)

* Move abort directive ordering; clean up redirects

Seems logical for the end-all of handlers to go at the... end.

The Connection header no longer needs to be set there, since Close is
true, and the static_response handler now does that.
2021-01-28 12:54:55 -07:00
Matt Holt ab80ff4fd2 admin: Identity management, remote admin, config loaders (#3994)
This commits dds 3 separate, but very related features:

1. Automated server identity management

How do you know you're connecting to the server you think you are? How do you know the server connecting to you is the server instance you think it is? Mutually-authenticated TLS (mTLS) answers both of these questions. Using TLS to authenticate requires a public/private key pair (and the peer must trust the certificate you present to it).

Fortunately, Caddy is really good at managing certificates by now. We tap into that power to make it possible for Caddy to obtain and renew its own identity credentials, or in other words, a certificate that can be used for both server verification when clients connect to it, and client verification when it connects to other servers. Its associated private key is essentially its identity, and TLS takes care of possession proofs.

This configuration is simply a list of identifiers and an optional list of custom certificate issuers. Identifiers are things like IP addresses or DNS names that can be used to access the Caddy instance. The default issuers are ZeroSSL and Let's Encrypt, but these are public CAs, so they won't issue certs for private identifiers. Caddy will simply manage credentials for these, which other parts of Caddy can use, for example: remote administration or dynamic config loading (described below).

2. Remote administration over secure connection

This feature adds generic remote admin functionality that is safe to expose on a public interface.

- The "remote" (or "secure") endpoint is optional. It does not affect the standard/local/plaintext endpoint.
- It's the same as the [API endpoint on localhost:2019](https://caddyserver.com/docs/api), but over TLS.
- TLS cannot be disabled on this endpoint.
- TLS mutual auth is required, and cannot be disabled.
- The server's certificate _must_ be obtained and renewed via automated means, such as ACME. It cannot be manually loaded.
- The TLS server takes care of verifying the client.
- The admin handler takes care of application-layer permissions (methods and paths that each client is allowed to use).\
- Sensible defaults are still WIP.
- Config fields subject to change/renaming.

3. Dyanmic config loading at startup

Since this feature was planned in tandem with remote admin, and depends on its changes, I am combining them into one PR.

Dynamic config loading is where you tell Caddy how to load its config, and then it loads and runs that. First, it will load the config you give it (and persist that so it can be optionally resumed later). Then, it will try pulling its _actual_ config using the module you've specified (dynamically loaded configs are _not_ persisted to storage, since resuming them doesn't make sense).

This PR comes with a standard config loader module called `caddy.config_loaders.http`.

Caddyfile config for all of this can probably be added later.

COMMITS:

* admin: Secure socket for remote management

Functional, but still WIP.

Optional secure socket for the admin endpoint is designed
for remote management, i.e. to be exposed on a public
port. It enforces TLS mutual authentication which cannot
be disabled. The default port for this is :2021. The server
certificate cannot be specified manually, it MUST be
obtained from a certificate issuer (i.e. ACME).

More polish and sensible defaults are still in development.

Also cleaned up and consolidated the code related to
quitting the process.

* Happy lint

* Implement dynamic config loading; HTTP config loader module

This allows Caddy to load a dynamic config when it starts.

Dynamically-loaded configs are intentionally not persisted to storage.

Includes an implementation of the standard config loader, HTTPLoader.
Can be used to download configs over HTTP(S).

* Refactor and cleanup; prevent recursive config pulls

Identity management is now separated from remote administration.

There is no need to enable remote administration if all you want is identity
management, but you will need to configure identity management
if you want remote administration.

* Fix lint warnings

* Rename identities->identifiers for consistency
2021-01-27 16:16:04 -07:00
Matt Holt 3366384d93 caddycmd: Add upgrade command (#3972)
Replaces the current Caddy executable with a new one from the build server. Honors custom builds, as long as plugins are registered on the Caddy website. Requires permissions to replace current executable, of course.

This is an experimental command that may get changed or removed later.
2021-01-19 18:45:49 -07:00
Matthew Holt 1ac6351705 Revert "requestbody: Allow overwriting remote address"
This reverts commit 0bf2046da7.

No actual use case.
2021-01-19 18:43:01 -07:00
Matthew Holt 160d199999 caddytest: Update Caddyfile tests for formatting, HTTP-only blocks
Previous commit improved the Caddyfile adapter so it doesn't unnecessarily add names to "skip" in "auto_https" when the server is already HTTP-only.

This commit updates the tests to reflect that change, while also fixing the Caddyfile formatting in many of the tests.

We also print the line number of the divergence between input and formatted version in Caddyfile adapt warnings - very useful for finding initial formatting problems.
2021-01-19 14:21:11 -07:00
Matthew Holt d68cff8eb6 httpcaddyfile: Skip TLS APs for HTTP-only hosts (fix #3977)
This is probably an invasive change, but existing tests continue to pass.
It seems to make sense this way. There is likely an edge case I haven't
considered.
2021-01-19 14:16:06 -07:00
Matthew Holt 8f6f9865d4 cmd: Print more detailed version with --environ 2021-01-16 12:52:33 -07:00
Matthew Holt 58e83a811b map: Add missing json struct tag 2021-01-16 09:56:06 -07:00
Sven Dowideit f0c0f38ba5 tests: use actual admin port value in error message (#3973)
Signed-off-by: Sven Dowideit <sven.dowideit@csiro.au>
2021-01-13 18:30:49 -07:00
Stefan Tatschner 59071ea15d cmd: Implement sd_notify() to notify systemd about readiness (#3963)
Issue: #3786
Based on Gaurav Dhameeja's work in #3908.
2021-01-12 14:38:53 -07:00
Matthew Holt 14f50d9dfb templates: Add fileExists and httpError template actions
The httpError function isn't particularly useful until https://github.com/golang/go/issues/34201 is fixed in the Go standard lib.
2021-01-11 13:49:20 -07:00
Matthew Holt 0bf2046da7 requestbody: Allow overwriting remote address
An experimental feature, let's see if it's useful.
2021-01-11 13:35:12 -07:00
go-d 88a38bd00d rewrite: Use RawPath instead of Path (fix #3596) (#3918)
Prevent information loss, i.e. the encoded form that was sent by the
client, when using URL strip/replace.
2021-01-11 09:18:53 -07:00
Matthew Holt 4f64105fbb Update docs 2021-01-08 16:00:36 -07:00
Matthew Holt 09432ba64d caddytls: Configurable OCSP stapling; global option (closes #3714)
Allows user to disable OCSP stapling (including support in the Caddyfile via the ocsp_stapling global option) or overriding responder URLs. Useful in environments where responders are not reachable due to firewalls.
2021-01-07 15:52:58 -07:00
Matthew Holt ef54483249 logging: Remove logfmt encoder (close #3575)
Has been deprecated for about 6 months now because it is broken.
2021-01-07 14:29:19 -07:00
Matthew Holt c2b91dbd65 httpcaddyfile: Support repeated use of cert_issuer global option
This changes the signature of UnmarshalGlobalFunc but this is probably OK since it's only used by this repo as far as we know.

We need this change in order to "remember" the previous value in case a global option appears more than once, which is now a possibility with the cert_issuer option since Caddy now supports multiple issuers in the order defined by the user.

Bonus: the issuer subdirective of tls now supports one-liner for "acme" when all you need to set is the directory:

issuer acme <dir>
2021-01-07 11:02:06 -07:00
Jordi Masip 8b6fdc04da caddytls: add 'key_type' subdirective (#3956)
* caddytls: add 'key_type' subdirective

* Suggested change

* *string -> string

* test
2021-01-06 12:02:58 -07:00
Matthew Holt f0216967dc caddyfile: Refactor unmarshaling of module tokens
Eliminates a fair amount of repeated code
2021-01-05 14:39:30 -07:00
Matthew Holt b1bec8c899 go.mod: Update CertMagic and acmez (improved IDN support) 2021-01-04 15:59:48 -07:00
yaxin 3c9256a1be reverseproxy: Caddyfile health check headers, host header support (#3948)
* reverse_proxy: 1.health check headers can be set through Caddyfile using health_headers directive; 2.health check header host can be set properly

* reverse_proxy:
replace example with syntax definition
inline health_headers directive parse function

* bugfix: change caddyfile_adapt testcase file from space to tab

* reverseproxy: modify health_header value document as optional and add more test cases
2021-01-04 11:26:18 -07:00
Matthew Holt 7846bc1e06 httpcaddyfile: Adjust iterator when removing AP (fix #3953) 2021-01-04 11:25:41 -07:00
Matt Holt 144b65cf99 cmd: Organize list-modules output; --packages flag (#3925) 2021-01-04 11:11:56 -07:00
Matt Holt c8557dc00b caddyfile: Introduce basic linting and fmt check (#3923)
* caddyfile: Introduce basic linting and fmt check

This will help encourage people to keep their Caddyfiles tidy.

* Remove unrelated tests

I am not sure that testing the output of warnings here is quite the
right idea; these tests are just for syntax and parsing success.
2021-01-04 11:11:36 -07:00
Mohammed Al Sahaf 1b453dd4fb ci: force fetch the upstream tags (#3947) 2020-12-30 21:02:54 +00:00
Dave Henderson ebc278ec98 metrics: allow disabling OpenMetrics negotiation (#3944)
* metrics: allow disabling OpenMetrics negotiation

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* fixup! metrics: allow disabling OpenMetrics negotiation
2020-12-30 11:44:02 -07:00
Francis Lavoie 79f3af9927 ci: Add pushing to cloudsmith (#3941)
* ci: Add pushing to cloudsmith

* ci: Update comments, remove env TODO

* ci: Fix Cloudsmith installation by setting PATH

* docs: Add Cloudsmith attribution to README

* ci: Switch to keeping armv7 as the armhf .deb
2020-12-30 10:54:58 -07:00
Matthew Holt d8bcf5be4e fileserver: Fix "go up" links in browse listings (closes #3942)
At some point we changed how paths are represented down the function calls of browse listings and forgot to update the canGoUp logic. I think this is right? It's simpler now.
2020-12-30 08:05:01 -07:00
Francis Lavoie 38a83ca6f8 ci: Update goreleaser to fix deprecation notices (#3945)
See https://goreleaser.com/deprecations#nfpmsfiles and https://goreleaser.com/deprecations#nfpmsconfig_files
2020-12-30 09:28:20 -05:00
Mohammed Al Sahaf 2b90cdba52 ci: reject tags if not signed by Matthew Holt's key (#3932)
* ci: reject tags if not signed by Matthew Holt's key

* ci: don't reject tags if an intermediate commits are not signed
2020-12-29 12:52:13 -07:00
Matthew Holt 635f075f18 caddyfile: Fix minor bug in formatter 2020-12-16 15:22:16 -07:00
Matthew Holt e384f07a3c caddytls: Improve alt chain preference settings
This allows for finer-grained control when choosing alternate chains than
simply the previous/Certbot-esque behavior of "choose first chain that
contains an issuer's common name." This update allows you to sort by
length (if optimizing for efficiency on the wire) and also to select the
chain with a specific root CommonName.
2020-12-15 12:16:04 -07:00
Matthew Holt 132525de3b reverseproxy: Minor lint fixes 2020-12-14 15:30:55 -07:00
Matthew Holt deedf8abb0 caddyhttp: Optionally use forwarded IP for remote_ip matcher
The remote_ip matcher was reading the X-Forwarded-For header by default, but this behavior was not documented in anything that was released. This is also a less secure default, as it is trivially easy to spoof request headers. Reading IPs from that header should be optional, and it should not be the default.

This is technically a breaking change, but anyone relying on the undocumented behavior was just doing so by coincidence/luck up to this point since it was never in any released documentation. We'll still add a mention in the release notes about this.
2020-12-10 16:09:30 -07:00
Matthew Holt 63bda6a0dc caddyhttp: Clean up internal auto-HTTPS redirect code
Refactor redirect route creation into own function.

Improve condition for appending port.
Fixes a bug manifested through new test case:
TestAutoHTTPRedirectsWithHTTPListenerFirstInAddresses
2020-12-10 14:36:46 -07:00
Matthew Holt b8a799df9f caddyhttp: Document that remote_ip reads X-Forwarded-For header
https://caddy.community/t/remote-ip-behaviour/10762?u=matt
2020-12-09 13:07:11 -07:00
Matthew Holt a748151666 go.mod: Update CertMagic (fix #3911) 2020-12-09 13:07:11 -07:00
Jack Baron c898a37f40 httpcaddyfile: support matching headers that do not exist (#3909)
* add integration test for null header matcher

* implement null header matcher syntax

* avoid repeating magic !

* check for field following ! character
2020-12-09 11:28:14 -07:00
Matthew Holt 31fbcd7401 go.mod: Upgrade some dependencies 2020-12-08 14:06:52 -07:00
Matthew Holt 7e719157d9 httpcaddyfile: Decrement counter when removing conn policy (fix #3906) 2020-12-07 14:22:47 -07:00
Francis Lavoie 6e9ac248dd fastcgi: Set PATH_INFO to file matcher remainder as fallback (#3739)
* fastcgi: Set PATH_INFO to file matcher remainder as fallback

* fastcgi: Avoid changing scriptName when not necessary

* Stylistic tweaks

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-12-04 17:12:13 -07:00
Marten Seemann 5643dc3fb9 go.mod: update quic-go to v0.19.3 (#3901) 2020-12-04 08:49:26 -07:00
Matthew Holt 3d0e046238 caddyauth: Use structured log 2020-12-03 11:33:55 -07:00
Matthew Holt bac82073d0 Merge remote-tracking branch 'origin/master' 2020-12-03 11:33:18 -07:00
Jordi Masip e7a5a3850f cmd: add ability to read config from stdin (#3898) 2020-12-03 10:02:18 -07:00
Matthew Holt aca7ef0d4c Add setcap script to gitignore 2020-12-02 13:48:13 -07:00
Matthew Holt 792fca40f1 Minor comments 2020-12-02 13:27:08 -07:00
Matthew Holt 9157051f45 caddyhttp: Optimize large host matchers 2020-12-02 13:26:28 -07:00
Cuong Manh Le 4cff36d731 caddyauth: Use buffered channel passed to signal.Notify (#3895)
The docs at os/signal.Notify warn about this signal delivery loss bug at
https://golang.org/pkg/os/signal/#Notify, which says:

    Package signal will not block sending to c: the caller must ensure
    that c has sufficient buffer space to keep up with the expected signal
    rate. For a channel used for notification of just one signal value,
    a buffer of size 1 is sufficient.

Caught by a static analysis tool from Orijtech, Inc. called "sigchanyzer"
2020-12-01 08:27:46 -07:00
Francis Lavoie a26f70a12b headers: Fix Caddyfile parsing with request matcher (#3892) 2020-11-30 10:20:30 -07:00
Francis Lavoie 4afcdc49d1 docs: Mention {http.auth.user.id} placeholder in basicauth JSON docs (#3886) 2020-11-26 22:31:25 -05:00
Matthew Holt 7d7434c9ce fileserver: Add debug logging 2020-11-26 09:37:42 -07:00
Daniel Santos 53aa60afff reverseproxy: Handle "operation was canceled" errors (#3816)
* fix(caddy): Avoid "operation was canceled" errors

- Also add error handling for StatusGatewayTimeout

* revert(caddy): Revert 504 handling

- This will potentially break load balancing and health checks

* Handle client cancellation as different error

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-11-25 10:54:23 -07:00
Matt Holt b0f8fc7aae caddytls: Configure trusted CAs from PEM files (#3882)
Closes #3563
2020-11-25 10:53:00 -07:00
Matthew Holt 03d853e2ec httpcaddyfile: Fix test on Windows 2020-11-24 18:04:37 -07:00
Matthew Holt 63afffc2e3 httpcaddyfile: Proper log config with catch-all blocks (fix #3878) 2020-11-24 16:36:58 -07:00
Matthew Holt 2d5498ee6f Update readme 2020-11-24 12:57:12 -07:00
Matthew Holt 0a7721dcfe fileserver: Preserve transformed root (fix #3838) 2020-11-24 12:24:44 -07:00
Ian c5197f5999 acme_server: fix reload of acme database (#3874)
* acme_server: Refactor database creation apart from authority creation

This is a WIP commit that doesn't really offer anything other than
setting us up for using a UsagePool to gracefully reload acme_server
configs.

* Implement UsagePool

* Remove unused context

* Fix initializing non-ACME CA

This will handle cases where a DB is not provided

* Sanitize acme db path and clean debug logs

* Move regex to package level to prevent recompiling
2020-11-23 13:58:26 -07:00
Ian 06ba006f9b acme_server: switch to bbolt storage (#3868)
* acme_server: switch to bbolt storage

There have been some issues with the badger storage engine
being used by the embedded acme_server. This will replace
the storage engine with bbolt

* Switch database path back to acme_server/db and remove if directory
2020-11-23 13:03:58 -07:00
Francis Lavoie c6dec30535 caddyfile: Add support for env var defaults; add tests (#3682)
* caddyfile: Add support for env var defaults, tests

* caddyfile: Use ?? instead, fix redundant cast, remove env chaining

* caddyfile: Use : instead
2020-11-23 12:51:35 -07:00
Francis Lavoie 3cfefeb0f7 httpcaddyfile: Configure servers via global options (#3836)
* httpcaddyfile: First pass at implementing server options

* httpcaddyfile: Add listener wrapper support

* httpcaddyfile: Sort sbaddrs to make adapt output more deterministic

* httpcaddyfile: Add server options adapt tests

* httpcaddyfile: Windows line endings lol

* caddytest: More windows line endings lol (sorry Matt)

* Update caddyconfig/httpcaddyfile/serveroptions.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* httpcaddyfile: Reword listener address "matcher"

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* httpcaddyfile: Deprecate experimental_http3 option (moved to servers)

* httpcaddyfile: Remove validation step, no longer needed

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-11-23 12:46:50 -07:00
Francis Lavoie 4a641f6c6f reverseproxy: Add Caddyfile scheme shorthand for h2c (#3629)
* reverseproxy: Add Caddyfile scheme shorthand for h2c

* reverseproxy: Use parentheses for condition

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-11-23 12:18:26 -07:00
Dave Henderson bd17eb205d ci: Use golangci's github action for linting (#3794)
* ci: Use golangci's github action for linting

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix most of the staticcheck lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the prealloc lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the misspell lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the varcheck lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the errcheck lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the bodyclose lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the deadcode lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the unused lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the gosec lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the gosimple lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the ineffassign lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Fix the staticcheck lint errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Revert the misspell change, use a neutral English

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Remove broken golangci-lint CI job

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Re-add errantly-removed weakrand initialization

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* don't break the loop and return

* Removing extra handling for null rootKey

* unignore RegisterModule/RegisterAdapter

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>

* single-line log message

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Fix lint after a1808b0dbf209c615e438a496d257ce5e3acdce2 was merged

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Revert ticker change, ignore it instead

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Ignore some of the write errors

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Remove blank line

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Use lifetime

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* close immediately

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Preallocate configVals

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Update modules/caddytls/distributedstek/distributedstek.go

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-11-22 14:50:29 -07:00
Marten Seemann 1e480b818b go.mod: update quic-go to v0.19.2 (#3880) 2020-11-21 14:54:11 -07:00
Francis Lavoie 96058538f0 reverseproxy: Logging for streaming and upgrades (#3689)
* reverseproxy: Enable error logging for connection upgrades

* reverseproxy: Change some of the error levels, unsugar

* Use unsugared log in one spot

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-11-20 14:24:58 -07:00
Dimitri Masson 6e0849d4c2 reverseproxy: Implement cookie hash selection policy (#3809)
* add CookieHashSelection for session affinity

* add CookieHashSelection for session affinity

* register module

* reverse_proxy: Add and fix cookie lb_policy

* reverse_proxy: Manage hmac.write error on cookie hash selection

* reverse_proxy: fix some comments

* reverse_proxy: variable `cookieValue` is inside the else block

* reverse_proxy: Abstract duplicate nuanced logic of reservoir sampling into a function

* reverse_proxy: Set a default secret is indeed useless

* reverse_proxy: add configuration syntax for cookie lb_policy

* reverse_proxy: doc typo and improvement

Co-authored-by: utick <123liuqingdong@163.com>
2020-11-20 12:39:26 -07:00
Gilbert Gilb's b0d5c2c8ae headers: Support default header values in Caddyfile with '?' (#3807)
* implement default values for header directive

closes #3804

* remove `set_default` header op and rely on "require" handler instead

This has the following advantages over the previous attempt:

- It does not introduce a new operation for headers, but rather nicely
  extends over an existing feature in the header handler.
- It removes the need to specify the header as "deferred" because it is
  already implicitely deferred by the use of the require handler. This
  should be less confusing to the user.

* add integration test for header directive in caddyfile

* bubble up errors when parsing caddyfile header directive

* don't export unnecessarily and don't canonicalize headers unnecessarily

* fix response headers not passed in blocks

* caddyfile: fix clash when using default header in block

Each header is now set in a separate handler so that it doesn't clash
with other headers set/added/deleted in the same block.

* caddyhttp: New idle_timeout default of 5m

* reverseproxy: fix random hangs on http/2 requests with server push (#3875)

see https://github.com/golang/go/issues/42534

* Refactor and cleanup with improvements

* More specific link

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
Co-authored-by: Денис Телюх <telyukh.denis@gmail.com>
2020-11-20 12:38:16 -07:00
Matthew Holt 12cc69ab7a Merge branch 'master' of https://github.com/caddyserver/caddy 2020-11-18 16:14:55 -07:00
Matthew Holt 349457cc1b caddyhttp: Return error if error handling error
Before, if there was an error in the error handler, we would not write a
status code, which resulted in Go writing a 200 for us by default, which
does not make sense when there's an error. Now we write the second
error's status if available, otherwise 500.
2020-11-18 16:14:50 -07:00
Денис Телюх 6ea6f3ebe0 reverseproxy: fix random hangs on http/2 requests with server push (#3875)
see https://github.com/golang/go/issues/42534
2020-11-18 11:53:43 -07:00
Matthew Holt 1438e4dbc8 caddyhttp: New idle_timeout default of 5m 2020-11-18 10:57:54 -07:00
Matthew Holt 4fc570711e caddyhttp: Fix header matcher when using nil
Uncovered in #3807
2020-11-17 11:29:43 -07:00
Dimitri Masson 99b8f44486 reverse_proxy: Fix random_choose selection policy (#3811) 2020-11-16 12:47:15 -07:00
Nicola Piccinini 670b723e38 requestbody: Add Caddyfile support (#3859)
* Add Caddyfile support for request_body:

```
  request_body {
    max_size 10000000
  }
```

* Improve Caddyfile parser for request_body module

* Remove unnecessary `continue`

* Add sample for caddyfile_adapt_test
2020-11-16 11:43:39 -07:00
Matt Holt 13781e67ab caddytls: Support multiple issuers (#3862)
* caddytls: Support multiple issuers

Defaults are Let's Encrypt and ZeroSSL.

There are probably bugs.

* Commit updated integration tests, d'oh

* Update go.mod
2020-11-16 11:05:55 -07:00
Aurelia 7a3d9d81fe basicauth: Minor internal improvements (#3861)
* nitpicks and small improvements in basicauth module

1:
roll two if statements into one, since err will be nil in the second case anyhow

2:
unlock cache mutex after reading the key, as this happens by-value and reduces code complexity

3:
switch cache sync.Mutex to sync.RWMutex for better concurrency on cache fast track

* allocate the right kind of mutex
2020-11-13 15:28:21 -07:00
Matthew Holt 95af4262a8 caddytls: Support ACME alt cert chain preferences 2020-11-12 15:03:07 -07:00
Matthew Holt 3db60e6cba Update contact info 2020-11-12 15:03:07 -07:00
Gaurav Dhameeja 7c28ecb5f4 httpcaddyfile: Add certificate_pem placeholder short, add to godoc (#3846)
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2020-11-04 13:37:41 -05:00
Mohammed Al Sahaf 9e28f60aab ci: remove the continuous fuzzing job (#3845)
Between Github Actions deprecting a command we use[0] and Fuzzit planning to deprecate their standalone service after being acquired by Gitlab[1][2], there are no reasons to keep this job.

[0] https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/

[1] https://about.gitlab.com/press/releases/2020-06-11-gitlab-acquires-peach-tech-and-fuzzit-to-expand-devsecops-offering.html

[2] https://fuzzit.dev/2020/06/11/news-fuzzit-is-acquired-by-gitlab/
2020-11-04 16:32:07 +00:00
Francis Lavoie b4f49e2962 caddyhttp: Merge query matchers in Caddyfile (#3839)
Also, turns out that `Add` on headers will work even if there's nothing there yet, so we can remove the condition I introduced in #3832
2020-11-02 16:05:01 -07:00
Christoph Kluge dd26875ffc logging: Fix for IP filtering 2020-11-02 16:01:58 -07:00
Francis Lavoie eda9a1b377 fastcgi: Add timeouts support to Caddyfile adapter (#3842)
* fastcgi: Add timeouts support to Caddyfile adapter

* fastcgi: Use tabs instead of spaces
2020-11-02 15:11:17 -07:00
Francis Lavoie 860cc6adfe reverseproxy: Wire up some http transport options in Caddyfile (#3843) 2020-11-02 14:59:02 -07:00
Matt Holt 8d038ca515 fileserver: Improve and clarify file hiding logic (#3844)
* fileserver: Improve and clarify file hiding logic

* Oops, forgot to run integration tests

* Make this one integration test OS-agnostic

* See if this appeases the Windows gods

* D'oh
2020-11-02 14:20:12 -07:00
Matthew Holt 937ec34201 caddyauth: Prevent user enumeration by timing
Always follow the code path of hashing and comparing a plaintext
password even if the account is not found by the given username; this
ensures that similar CPU cycles are spent for both valid and invalid
usernames.

Thanks to @tylerlm for helping and looking into this!
2020-10-31 10:51:05 -06:00
Francis Lavoie 966d5e6b42 caddyhttp: Merge header matchers in Caddyfile (#3832) 2020-10-31 10:27:01 -06:00
Francis Lavoie b66099379d reverseproxy: Add max_idle_conns_per_host; fix godocs (#3829) 2020-10-30 12:05:21 -06:00
Jason McCallister c9fdff9976 reverseproxy: caddyfile: Don't add port if upstream has placeholder (#3819)
* check if the host is a placeholder

* Update modules/caddyhttp/reverseproxy/caddyfile.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-10-29 13:51:42 -06:00
Matt Holt db4f1c0277 httpcaddyfile: Revise automation policy generation (#3824)
* httpcaddyfile: Revise automation policy generation

This should fix a frustrating edge case where wildcard subjects are
used, which potentially get shadowed by more specific versions of
themselves; see the new tests for an example. This change is motivated
by an actual customer requirement.

Although all the tests pass, this logic is incredibly complex and
nuanced, and I'm worried it is not correct. But it took me about 4 days
to get this far on a solution. I did my best.

* Fix typo
2020-10-28 20:36:00 -06:00
Matthew Holt b6e96d6f4a go.mod: Update CertMagic 2020-10-22 12:42:06 -06:00
Matthew Holt b6686a54d8 httpcaddyfile: Improve AP logic with OnDemand
We have users that have site blocks like *.*.tld with on-demand TLS
enabled. While *.*.tld does not qualify for a publicly-trusted cert due
to its wildcards, On-Demand TLS does not actually obtain a cert with
those wildcards, since it uses the actual hostname on the handshake.

This improves on that logic, but I am still not 100% satisfied with the
result since I think we need to also check if another site block is more
specific, like foo.example.tld, which might not have on-demand TLS
enabled, and make sure an automation policy gets created before the
more general policy with on-demand...
2020-10-22 12:40:23 -06:00
Matthew Holt 97caf368ee readme: Add zerossl 2020-10-19 10:44:46 -06:00
Matt Holt 385adf5d87 caddyhttp: Restore original request params before error handlers (#3781)
* caddyhttp: Restore original request params before error handlers

Fixes #3717

* Add comment
2020-10-13 10:52:39 -06:00
Matt Holt c7efb0307d reverseproxy: Fix dial placeholders, SRV, active health checks (#3780)
* reverseproxy: Fix dial placeholders, SRV, active health checks

Supercedes #3776
Partially reverts or updates #3756, #3693, and #3695

* reverseproxy: add integration tests

Co-authored-by: Mohammed Al Sahaf <msaa1990@gmail.com>
2020-10-13 10:35:20 -06:00
AJ ONeal e34d9f1244 readme: Add link to website for download instructions (#3785)
* add Webi as install method

* link to install page
2020-10-09 11:32:09 -06:00
Matthew Holt ef8a372a1c map: Bug fixes; null literal with hyphen in Caddyfile 2020-10-02 16:08:28 -06:00
Matthew Holt 0fc47e8357 map: Apply default if mapped output is nil 2020-10-02 15:23:52 -06:00
Matthew Holt 25d2b4bf29 map: Reimplement; multiple outputs; optimize 2020-10-02 14:23:56 -06:00
Matt Holt 023d702f30 Update SECURITY.md 2020-10-01 17:11:10 -06:00
Mohammed Al Sahaf 6722426f1a reverseproxy: allow no port for SRV; fix regression in d55d50b (#3756)
* reverseproxy: fix breakage in handling SRV lookup introduced by 3695

* reverseproxy: validate against incompatible config options with lookup_srv

* reverseproxy: add integration test cases for validations involving lookup_srv

* reverseproxy: clarify the reason for skipping an iteration

* grammar.. Oxford comma

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

Fixes #3753
2020-10-01 14:05:39 -06:00
Aleksei 3b9eae70c9 reverseproxy: Change 500 error to 502 for lookup_srv config (#3771)
Fixes #3763
2020-10-01 14:02:31 -06:00
Mohammed Al Sahaf aa9c3eb732 reverseproxy: default to port 80 for upstreams in Caddyfile (#3772)
* reverseproxy: default to port 80 for port-less upstream dial addresses

* reverseproxy: replace integration test with an adapter test

Fixes #3761
2020-10-01 13:53:19 -06:00
Christian Flach fdfdc03339 reverseproxy: Ignore RFC 1521 params in Content-Type header (#3758)
Without this change, a Content-Type header like "text/event-stream;charset=utf-8"
would not trigger the immediate flushing.

Fixes #3765
2020-10-01 12:15:45 -06:00
Dave Henderson dadfe1933b metrics: fix handler to not run the next route (#3769)
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2020-10-01 10:57:14 -06:00
Dave Henderson 85152679ce admin: lower log level to Debug for /metrics requests (#3749)
* admin: lower log level to Debug for /metrics requests

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-09-25 18:00:08 -06:00
Mohammed Al Sahaf a33e4b5426 caddyfile: Add support for vars and vars_regexp matchers (#3730)
* caddyfile: support vars and vars_regexp matchers in the caddyfile

* caddyfile: matchers: Brian Kernighan said printf is good debugging tool but didn't say keep them around
2020-09-25 17:50:26 -06:00
Dave Henderson f197cec7f3 metrics: Always track method label in uppercase (#3742)
* metrics: Always track method label in uppercase

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Just use strings.ToUpper for clarity

Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2020-09-22 20:10:34 -06:00
Francis Lavoie be6daa5fd4 httpcaddyfile: Fix panic when parsing route with matchers (#3746)
Fixes #3745
2020-09-22 17:37:15 -06:00
Francis Lavoie fe27f9cf0c httpcaddyfile: Disallow args on route/handle directive family (#3740) 2020-09-21 13:44:41 -06:00
Dave Henderson b1d456d8ab metrics: Fix panic when headers aren't written (#3737)
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2020-09-21 13:42:47 -06:00
Dave Henderson d16ede358a metrics: Fix hidden panic while observing with bad exemplars (#3733)
* metrics: Fixing panic while observing with bad exemplars

Signed-off-by: Dave Henderson <dhenderson@gmail.com>

* Minor cleanup

The server is already added to the context. So, we can simply use that
to get the server name, which is a field on the server.

* Add integration test for auto HTTP->HTTPS redirects

A test like this would have caught the problem in the first place

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-09-17 21:46:24 -06:00
Matthew Holt c82c231ba7 caddyhttp: Remove server name from metrics
For some reason this breaks automatic HTTP->HTTPS redirects. I am not
sure why yet, but as a hotfix remove this until we understand it better.
2020-09-17 17:23:58 -06:00
Matthew Holt 3ee663dee1 go.mod: Upgrade dependencies 2020-09-17 12:35:25 -06:00
Dave Henderson 8ec51bbede metrics: Initial integration of Prometheus metrics (#3709)
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2020-09-17 12:01:20 -06:00
Mohammed Al Sahaf bc453fa6ae reverseproxy: Correct alternate port for active health checks (#3693)
* reverseproxy: construct active health-check transport from scratch (Fixes #3691)

* reverseproxy: do upstream health-check on the correct alternative port

* reverseproxy: add integration test for health-check on alternative port

* reverseproxy: put back the custom transport for health-check http client

* reverseproxy: cleanup health-check integration test

* reverseproxy: fix health-check of unix socket upstreams

* reverseproxy: skip unix socket tests on Windows

* tabs > spaces

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* make the linter (and @francislavoie) happy

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* One more lint fix

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2020-09-17 10:25:34 -06:00
Francis Lavoie e3324aa6de httpcaddyfile: Ensure handle_path is sorted equally to handle (#3676)
* httpcaddyfile: Ensure handle_path is sorted as equal to handle

* httpcaddyfile: Make mutual exclusivity grouping deterministic (I hope)

* httpcaddyfile: Add comment linking to the issue being fixed

* httpcaddyfile: Typo fix, comment clarity

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update caddyconfig/httpcaddyfile/httptype.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-09-16 20:01:22 -06:00
Mohammed Al Sahaf d55d50b3b3 reverseproxy: Enforce port range size of 1 at provision (#3695)
* reverse_proxy: ensure upstream address has port range of only 1

* reverse_proxy: don't log the error if upstream range size is more than 1
2020-09-16 19:48:37 -06:00
Francis Lavoie b95b87381a fileserver: Fix try_files for directories; windows fix (#3684)
* fileserver: Fix try_files for directories, windows fix

* fileserver: Add new file type placeholder, refactoring, tests

* fileserver: Review cleanup

* fileserver: Flip the return args order
2020-09-16 18:09:28 -06:00
Gaurav Dhameeja b01bb275b3 caddyhttp: New placeholder for PEM of client certificate (#3662)
* Fix-3585: added placeholder for a PEM encoded value of the certificate

* Update modules/caddyhttp/replacer.go

Change type of block and empty headers removed

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* fixed tests

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-09-16 15:06:51 -06:00
Francis Lavoie 309c1fec62 logging: Implement Caddyfile support for filter encoder (#3578)
* logging: Implement Caddyfile support for filter encoder

* logging: Add support for parsing IP masks from strings


wip

* logging: Implement Caddyfile support for ip_mask

* logging: Get rid of unnecessary logic to allow strings, not that useful

* logging: Add adapt test
2020-09-15 12:37:41 -06:00
Matthew Penner b88e2b6a49 cmd: Allow caddy fmt to read from stdin (#3680)
* Allow 'caddy fmt' to read from stdin

* fmt: use '-' as the file name for reading from stdin

* Minor adjustments

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-09-14 12:30:12 -06:00
Matthew Holt 4217217bad httpcaddyfile: Properly record whether we added catch-all conn policy
We recently introduced `if !cp.SettingsEmpty()` which conditionally
adds the connection policy to the list. If the condition evaluates to
false, the policy wouldn't actually be added, even if
hasCatchAllTLSConnPolicy was set to true on the previous line.

Now we set that variable in accordance with whether we actually add
the policy.

While debugging this I noticed that catch-all policies added early in
that loop (i.e. not at the end if we later determine we need one) are
not always at the end of the list. They should be, though, since they
are selected by which one matches first, and having a catch-all first
would nullify any more specific ones later in the list. So I added a
sort in consolidateConnPolicies to take care of that.

Should fix #3670 and
https://caddy.community/t/combining-on-demand-tls-with-custom-ssl-certs-doesnt-seem-to-work-in-2-1-1/9719
but I won't know for sure until somebody verifies it, since at least in
the GitHub issue there is not yet enough information (the configs are
redacted).
2020-09-11 13:45:21 -06:00
Matt Holt 1c5969b576 fileserver: Fix new file hide tests on Windows (#3719) 2020-09-11 13:09:16 -06:00
Matthew Holt 0ee4378227 fileserver: Improve file hiding logic for directories and prefixes
Now, a filename to hide that is specified without a path separator will
count as hidden if it appears in any component of the file path (not
only the last component); semantically, this means hiding a file by only
its name (without any part of a path) will hide both files and folders,
e.g. hiding ".git" will hide "/.git" and also "/.git/foo".

We also do prefix matching so that hiding "/.git" will hide "/.git"
and "/.git/foo" but not "/.gitignore".

The remaining logic is a globular match like before.
2020-09-11 12:20:39 -06:00
Matthew Holt 9859ab8148 caddytls: Fix resolvers option of acme issuer (Caddyfile)
Reported in:
https://caddy.community/t/dns-challenge-with-namecheap-and-split-horizon-dns/9611/17?u=matt
2020-09-09 10:21:59 -06:00
Francis Lavoie 00e6b77fe4 caddytls: Add dns config to acmeissuer (#3701) 2020-09-08 11:36:46 -06:00
Mohammed Al Sahaf d4f249741e browse: align template to struct field renames from 4940325 (#3706) 2020-09-08 10:45:48 -06:00
Francis Lavoie 04f50a9759 caddyhttp: Wrap http.Server logging with zap (#3668) 2020-09-08 10:44:58 -06:00
Francis Lavoie 4cd7ae35b3 reverseproxy: Add buffer_requests option to reverse_proxy directive (#3710) 2020-09-08 10:37:46 -06:00
Matthew Holt 24f34780b6 caddytls: Customize DNS resolvers for DNS challenge with Caddyfile 2020-08-31 13:23:26 -06:00
Matthew Holt 724b74d981 reverseproxy: Abort active health checks on context cancellation 2020-08-31 13:22:34 -06:00
Matthew Holt 4940325844 fileserver: Fix inconsistencies in browse JSON 2020-08-31 12:33:43 -06:00
Matthew Holt 744d04c258 caddytls: Configure custom DNS resolvers for DNS challenge (close #2476)
And #3391

Maybe also related: #3664
2020-08-21 20:30:14 -06:00
Francis Lavoie ecbc1f85c5 ci: Tweaks for multi go version tests (#3673) 2020-08-20 22:40:26 -04:00
Matthew Holt 997ef522bc go.mod: Use v0.15(.1) of smallstep libs
Update internal issuer for compatibility -- yay simpler code!

The .1 version also fixes non-critical SAN extensions that caused trust
issues on several clients.
2020-08-20 19:28:25 -06:00
Francis Lavoie 0279a57ac4 ci: Upgrade to Go 1.15 (#3642)
* ci: Try Go 1.15 RC1 out of curiosity

* Go 1.15 was released; let's try it

* Update to latest quic-go

* Attempt at fixing broken test

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-08-20 14:04:10 -06:00
Matthew Holt c94f5bb7dd reverseproxy: Make default buffer size const 2020-08-17 16:17:16 -06:00
Francis Lavoie 0afbab8667 httpcaddyfile: Improve directive sorting logic (#3658)
* httpcaddyfile: Flip `root` directive sort order

* httpcaddyfile: Sort directives with any matcher before those with none

* httpcaddyfile: Generalize reverse sort directives, improve logic

* httpcaddyfile: Fix "spelling" issue

* httpcaddyfile: Turns out the second change precludes the first


httpcaddyfile: Delete test that no longer makes sense

* httpcaddyfile: Shorten logic

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-08-17 16:15:51 -06:00
Francis Lavoie fc65320e9c reverseproxy: Support header selection policy on Host field (#3653) 2020-08-17 15:14:46 -06:00
Matthew Holt e385be9225 Update comment and Caddy 1 EOL 2020-08-11 11:26:19 -06:00
Matt Holt 66863aad3b caddytls: Add support for ZeroSSL; add Caddyfile support for issuers (#3633)
* caddytls: Add support for ZeroSSL; add Caddyfile support for issuers

Configuring issuers explicitly in a Caddyfile is not easily compatible
with existing ACME-specific parameters such as email or acme_ca which
infer the kind of issuer it creates (this is complicated now because
the ZeroSSL issuer wraps the ACME issuer)... oh well, we can revisit
that later if we need to.

New Caddyfile global option:

    {
        cert_issuer <name> ...
    }

Or, alternatively, as a tls subdirective:

    tls {
        issuer <name> ...
    }

For example, to use ZeroSSL with an API key:

    {
        cert_issuser zerossl API_KEY
    }

For now, that still uses ZeroSSL's ACME endpoint; it fetches EAB
credentials for you. You can also provide the EAB credentials directly
just like any other ACME endpoint:

    {
        cert_issuer acme {
            eab KEY_ID MAC_KEY
        }
    }

All these examples use the new global option (or tls subdirective). You
can still use traditional/existing options with ZeroSSL, since it's
just another ACME endpoint:

    {
        acme_ca  https://acme.zerossl.com/v2/DV90
        acme_eab KEY_ID MAC_KEY
    }

That's all there is to it. You just can't mix-and-match acme_* options
with cert_issuer, because it becomes confusing/ambiguous/complicated to
merge the settings.

* Fix broken test

This test was asserting buggy behavior, oops - glad this branch both
discovers and fixes the bug at the same time!

* Fix broken test (post-merge)

* Update modules/caddytls/acmeissuer.go

Fix godoc comment

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* Add support for ZeroSSL's EAB-by-email endpoint

Also transform the ACMEIssuer into ZeroSSLIssuer implicitly if set to
the ZeroSSL endpoint without EAB (the ZeroSSLIssuer is needed to
generate EAB if not already provided); this is now possible with either
an API key or an email address.

* go.mod: Use latest certmagic, acmez, and x/net

* Wrap underlying logic rather than repeating it

Oops, duh

* Form-encode email info into request body for EAB endpoint

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
2020-08-11 08:58:06 -06:00
Matthew Holt c42bfaf31e go.mod: Bump CertMagic 2020-08-08 08:42:01 -06:00
Matthew Holt e2f913bb7f reverseproxy: Minor fixes and cleanup
Now use context cancellation to stop active health checker, which is
simpler than and just as effective as using a separate stop channel.
2020-08-07 18:02:24 -06:00
Matt Holt 65a09524c3 caddyhttp: Add TLS client cert info to logs (#3640) 2020-08-07 12:12:29 -06:00
Matthew Holt c6d6a775a1 go.mod: Update some dependencies
We can't update smallstep/nosql and klauspost/cpuid yet because of
upstream breakage.
2020-08-06 14:36:21 -06:00
Matt Holt 4accf737a6 ci: Ignore s390x failures (#3644)
As of early August 2020 the VM has been down for several days due to
lack of power due related to bad weather at the data center... sigh.
2020-08-06 14:17:40 -06:00
Matthew Holt ff19bddac5 httpcaddyfile: Avoid repeated subjects in APs (fix #3618)
When consolidating automation policies, ensure same subject names do not
get appended to list.
2020-08-06 13:56:23 -06:00
Francis Lavoie 584eba94a4 httpcaddyfile: Allow named matchers in route blocks (#3632) 2020-08-05 13:42:29 -06:00
Kevin Lin 904f149e5b reverse_proxy: fix bidirectional streams with encodings (fix #3606) (#3620)
* reverse_proxy: fix bi-h2stream breaking gzip encode handle(#3606).

* reverse_proxy: check http version of both sides to avoid affecting non-h2 upstream.

* Minor cleanup; apply review suggestions

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-08-03 20:50:38 -06:00
Ye Zhihao 8b80a3201f httpcaddyfile: Bring enforce_origin and origins to admin config (#3595)
* Bring `ensure_origin` and `origins` to caddyfile admin config

* Add unit test for caddyfile admin config update

* Add caddyfile adapt test for typical admin setup

* httpcaddyfile: Replace admin config error message when there's more arguments than needed

Replace d.Err() to d.ArgErr() since the latter provides similarly informative error message

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-08-03 13:44:38 -06:00
Matthew Holt 68529e2f9e cmd: Print caddy version with environ or --environ (#3627) 2020-08-03 10:42:42 -06:00
Mohammed Al Sahaf 399eff415c ci: Include tracking of GOOS for which Caddy fails to build (#3617)
* ci: include tracking of GOOS for which Caddy fails to build

* ci: split cross-build check into separate workflow

* ci: cross-build check: make it clear the cross-build check is not a blocker

* ci: cross-build check: set annotation instead of failing the build

* ci: cross-build check: explicitly set continue-on-error to force success marker

* ci: cross-build check: send stderr to /dev/null

* ci: Simplify workflow names

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-08-01 20:23:22 +00:00
Matt Holt c054a818a1 fileserver: Fix newly-introduced failing test on Linux (#3625)
* fileserver: First attempt to fix failing test on Linux

I think I updated the wrong test case before

* Make new test function

I guess what we really are trying to test is the case insensitivity of
firstSplit. So a new test function is better for that.
2020-08-01 12:43:30 -06:00
Bart af5c148ed1 admin,templates,core: Minor enhancements and error handling (#3607)
* fix 2 possible bugs

* handle unhandled errors
2020-07-31 16:54:18 -06:00
v-rosa 514eef33fe caddyhttp: Add support to resolve DN in CEL expression (#3608) 2020-07-31 15:06:30 -06:00
Matthew Holt 3860b235d0 fileserver: Don't assume len(str) == len(ToLower(str)) (fix #3623)
We can't use a positional index on an original string that we got from
its lower-cased equivalent. Implement our own IndexFold() function b/c
the std lib does not have one.
2020-07-31 13:55:01 -06:00
Ye Zhihao 6f73a358f4 httpcaddyfile: Add compression to http transport config (#3624)
* httpcaddyfile: Add `compression` to http transport config

* Add caddyfile adapt test for typical h2c setup
2020-07-31 11:30:20 -06:00
Matt Holt 6a14e2c2a8 caddytls: Replace lego with acmez (#3621)
* Replace lego with acmez; upgrade CertMagic

* Update integration test
2020-07-30 15:18:14 -06:00
Patrick Hein 2bc30bb780 templates: Implement placeholders function (#3324)
* caddyhttp, httpcaddyfile: Implement placeholders in template

* caddyhttp, httpcaddyfile: Remove support for placeholder shorthands in templates

* Update modules/caddyhttp/templates/templates.go

updates JSON doc

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update modules/caddyhttp/templates/tplcontext.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-07-20 17:17:38 -06:00
Matthew Holt 28d870c193 go.mod: Update quic-go, truststore, and goldmark 2020-07-20 14:57:40 -06:00
Francis Lavoie fb9d874fa9 caddyfile: Export Tokenize function for lexing (#3549) 2020-07-20 13:55:51 -06:00
Matt Holt 6cea1f239d push: Implement HTTP/2 server push (#3573)
* push: Implement HTTP/2 server push (close #3551)

* push: Abstract header ops by embedding into new struct type

This will allow us to add more fields to customize headers in
push-specific ways in the future.

* push: Ensure Link resources are pushed before response is written

* Change header name from X-Caddy-Push to Caddy-Push
2020-07-20 12:28:40 -06:00
Manuel Dalla Lana 2ae8c11927 fastcgi: Add resolve_root_symlink (#3587) 2020-07-20 12:16:13 -06:00
Kevin Lin e9b1d7dcb4 reverse_proxy: flush HTTP/2 response when ContentLength is unknown (#3561)
* reverse proxy: Support more h2 stream scenarios (#3556)

* reverse proxy: add integration test for better h2 stream (#3556)

* reverse proxy: adjust comments as francislavoie suggests

* link to issue #3556 in the comments
2020-07-20 12:14:46 -06:00
Mohammed Al Sahaf bd9d796e6e reverseproxy: add support for custom DNS resolver (#3479)
* reverse proxy: add support for custom resolver

* reverse proxy: don't pollute the global resolver with bootstrap resolver setup

* Improve documentation of reverseproxy.UpstreamResolver fields

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* reverse proxy: clarify the name resolution conventions of upstream resolvers and bootstrap resolver

* remove support for bootstraper of resolver

* godoc and code-style changes

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-07-18 15:00:00 -06:00
Matthew Holt 246a31aacd reverseproxy: Restore request's original host and header (fix #3509)
We already restore them within the retry loop, but after successful
proxy we didn't reset them, so as handlers bubble back up, they would
see the values used for proxying.

Thanks to @ziddey for identifying the cause.
2020-07-17 17:54:58 -06:00
Francis Lavoie 0665a86eb7 fastcgi: Ensure leading slash, omit SERVER_PORT if empty for compliance (#3570)
See https://tools.ietf.org/html/rfc3875#section-4.1.13 for SCRIPT_NAME requiring leading slash
See https://tools.ietf.org/html/rfc3875#section-4.1.15 for SERVER_PORT requiring omission if empty
2020-07-17 14:48:50 -06:00
Francis Lavoie 3fdaf50785 fastcgi: Fill REMOTE_USER with http.auth.user.id placeholder (#3577)
Completing a TODO!
2020-07-17 13:33:40 -06:00
Francis Lavoie 19cc2bd3c3 reverseproxy: Fix Caddyfile parsing for empty non-http transports (#3576)
* reverseproxy: Fix Caddyfile parsing for empty non-http transports

* Update modules/caddyhttp/reverseproxy/caddyfile.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Rename empty transport test

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-07-17 13:18:32 -06:00
Matthew Holt 705de11bef readme: Minor tweaks 2020-07-17 12:53:48 -06:00
Matthew Holt 8a0fff58aa caddyauth: hash-password: Set bcrypt cost to 14 (#3580) 2020-07-17 12:20:53 -06:00
Matthew Holt 6f0f159ba5 caddyhttp: Add {http.request.body} placeholder 2020-07-16 19:25:37 -06:00
Matthew Holt 6eafd4e82f readme: Update badges 2020-07-16 15:29:06 -06:00
Matthew Holt eda54c22a6 logging: ⚠️ Deprecate logfmt encoder
It is essentially broken because it occludes many log fields.

See: https://github.com/caddyserver/caddy/issues/3575
2020-07-13 16:18:34 -06:00
Matthew Holt 2c71fb116b chore: Rename file to be consistent 2020-07-11 17:53:33 -06:00
Kévin Dunglas 724613a1be docs: Remove extra word in README.md (#3564) 2020-07-10 10:00:24 -04:00
snu-ceyda 735c86658d fileserver: Enable browse pagination with offset parameter (#3542)
* Update browse.go

* Update browselisting.go

* Update browsetpl.go

* fix linter err

* Update modules/caddyhttp/fileserver/browse.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update modules/caddyhttp/fileserver/browselisting.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update browsetpl.go

change from -> offset

* Update browse.go

* Update browselisting.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-07-08 23:56:15 -06:00
Matthew Holt a2dae1d43f templates: Fix front matter closing fence search
This makes it choose first matching closing fence instead of last one,
which could appear in document body.
2020-07-08 16:46:56 -06:00
Matthew Holt efc0cc5e85 caddytls: Move initial storage clean op into goroutine
Sometimes this operation can take a while (we observed 7 minutes
recently, with a large, globally-distributed storage backend).
2020-07-08 10:59:49 -06:00
Matthew Holt 0bf2565c37 caddyhttp: Reorder some access log fields; add host matcher test case
This field order reads a little more naturally.
2020-07-07 08:11:35 -06:00
Matthew Holt 7bfe5b6c95 httpcaddyfile: Reorder automation policy logic (close #3550) 2020-07-07 08:10:37 -06:00
Matthew Holt 2a5599e2ad go.mod: Upgrade and downgrade smallstep, quic-go, and cpuid
Closes #3537 and fixes #3535
2020-07-06 12:10:35 -06:00
Greg Anders c35820012b templates: Disable hard wraps in Markdown rendering (#3553) 2020-07-06 11:53:40 -06:00
Francis Lavoie 2d0f8831f8 ci: Fix another oops with publish workflow (#3536) 2020-06-30 15:36:17 -04:00
Mohammed Al Sahaf d7dbf85525 cel: fix validation of expression result type (#3526)
* cel: fix validation of expression result type

The earlier code used the proto.Equals from github.com/gogo/protobuf, which failed to compare two messages of the same type for some reason. Switching to proto.Equal from the canonical github.com/golang/protobuf fixes the issue.

* deps: remove deprecated github.com/golang/protobuf in favor of google.golang.org/protobuf

* downgrade github.com/smallstep/nosql to resolve warning pb.proto warning
2020-06-30 11:53:29 -06:00
Matthew Holt 77f233a484 caddyhttp: Corrected host label index check (fix #3502) 2020-06-30 11:43:01 -06:00
James Birtles ddd690de4c caddyhttp: Support placeholders in query matcher (#3521) 2020-06-26 15:14:47 -06:00
Mark Sargent 6004d3f779 caddyhttp: Add 'map' handler (#3199)
* inital map implementation

* resolve the value during middleware execution

* use regex instead

* pr feedback

* renamed mmap to maphandler

* refactored GetString implementation

* fixed mispelling

* additional feedback
2020-06-26 15:12:37 -06:00
Francis Lavoie caca55e582 ci: Fix release publish trigger (#3524)
Looks like event payloads need to be prefixed with `github.event` to get the actual payload contents. Didn't dig deep enough.

https://help.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#github-context
2020-06-26 16:00:54 -04:00
Matthew Holt c9049bdc24 go.mod: Minor dependency updates 2020-06-26 12:19:51 -06:00
Matt Holt 21c00a3cd2 caddyhttp: Better host matching for logger names (fix #3488) (#3522)
First try an exact lookup like before, but if it fails, strip the port
and try again. example.com:1234 should still use a logger keyed for
example.com if there is no key example.com:1234.
2020-06-26 12:01:50 -06:00
Francis Lavoie 61b7002d26 ci: Apparently only single-quote strings are supported (#3523)
https://help.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#literals

https://github.com/caddyserver/caddy/actions/runs/147953515
2020-06-25 21:58:44 +00:00
Francis Lavoie b1480eb52f fastcgi: Fix php_fastcgi matcher regression (#3512) 2020-06-22 11:45:18 -06:00
Xiuming Chen 5bc4777be9 chore: Fix typo in reverse-proxy subcommand help message (#3513) 2020-06-22 00:40:54 -04:00
Matthew Holt 3af15c0725 caddyhttp: Empty, not nil, query matcher matches empty query string 2020-06-16 12:02:23 -06:00
Matthew Holt 6db3615547 caddyhttp: Enable matching empty query string
Caddyfile syntax: query ""

Or a nil matcher in the JSON should also match an empty query string.

See https://caddy.community/t/v2-match-empty-query/8708?u=matt
2020-06-16 10:41:37 -06:00
Matthew Holt 32cafbb630 httpcaddyfile: Fix ordering of catch-all site blocks
Catch-alls should always go last. Normally this is the case, but we have
a special case for comparing one wildcard-host site block to another
non-wildcard host site block; and a catch-all site block is also a
non-wildcard host site block, so now we have to special-case the
catch-all site block. Sigh.

This could be reproduced with a Caddyfile that has two site blocks:
":80" and "*.example.com", in that order.
2020-06-16 10:02:06 -06:00
Francis Lavoie 003403ecbc templates: Add support for dots to close yaml frontmatter (#3498)
* templates: Add support for dots to close yaml frontmatter

* templates: Fix regression in body output
2020-06-15 12:38:51 -06:00
Mohammed Al Sahaf 5b48f784ae ci: don't run s390x tests on PRs of forks (#3494)
* ci: don't run s390x tests on PRs of forks

* ci: check if fork by matchinging name from event against name of repo
2020-06-12 19:51:04 +00:00
Chris Ortman d84a5d8427 httpcaddyfile: New acme_eab option (#3492)
* Adds global options for external account bindings

* Maybe other people use ctags too?

* Use nested block to configure external account

* go format files

* Restore acme_ca directive in test file

* Change Caddyfile config syntax for acme_eab

* Update test

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-06-12 13:37:56 -06:00
Mohammed Al Sahaf 7da32f493a ci: skip s390x tests on forks (#3493) 2020-06-12 18:03:29 +00:00
Matthew Holt cb0d9838cb go.mod: Update quic-go to 0.17.1 (draft 29) and certmagic 0.11.2 (eab) 2020-06-12 11:52:12 -06:00
Matthew Holt d81a69ef16 Merge branch 'eab-fix' 2020-06-12 11:49:45 -06:00
Mohammed Al Sahaf 99dcc10f31 ci: add CI on s390x (#3463)
* ci: lay out foundation for s390x tests

* ci: uncomment the s390x test script & replace placeholders with real values

* ci: amend the s390x test job name to be more consistent with others
2020-06-12 17:11:46 +00:00
Wynn Wolf Arbor fa4cdde7d8 fastcgi: Make sure splitPos handles empty SplitPath correctly (#3491)
In commit f2ce81c, support for multiple path splitters was added. The
type of SplitPath changed from string to []string, and splitPos was
changed to loop through all values in SplitPath.

Before that commit, if SplitPath was empty, strings.Index returned 0 and
PATH_INFO was set correctly in buildEnv.

Currently, however, splitPos returns -1 for empty values of SplitPath,
behaving as if a split position could not be found at all. PATH_INFO is
then never set in buildEnv and remains empty.

Restore the old behaviour by explicitly checking whether SplitPath is
empty and returning 0 in splitPos.

Closes #3490
2020-06-12 10:07:59 -06:00
Matthew Holt d55c3b31eb caddyhttp: Add client cert SAN placeholders 2020-06-11 16:19:07 -06:00
Matthew Holt 6d03fb48f9 caddytls: Don't decode HMAC
https://caddy.community/t/trouble-with-external-account-hmac/8600?u=matt
2020-06-11 15:33:27 -06:00
Matthew Holt b3bff13f7d reverseproxy: Close websocket conn if req context cancels
This is a recent patch in the Go standard library
2020-06-11 15:25:26 -06:00
Francis Lavoie 7211101c52 ci: Fix gemfury upload condition, move triggers to publish event (#3483) 2020-06-08 12:21:20 -06:00
Mohammed Al Sahaf 90dba172cb ci: fix an oopsie in the release script (#3482) 2020-06-08 11:10:28 -06:00
Matthew Holt 4b10ae5ce6 reverseproxy: Add Caddyfile support for ClientCertificateAutomate 2020-06-08 10:30:26 -06:00
NWHirschfeld 1dfb11486e httpcaddyfile: Add client_auth options to tls directive (#3335)
* reading client certificate config from Caddyfile

Signed-off-by: NWHirschfeld <Niclas@NWHirschfeld.de>

* Update caddyconfig/httpcaddyfile/builtins.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* added adapt test for parsing client certificate configuration from Caddyfile

Signed-off-by: NWHirschfeld <Niclas@NWHirschfeld.de>

* read client ca and leaf certificates from file https://github.com/caddyserver/caddy/pull/3335#discussion_r421633844

Signed-off-by: NWHirschfeld <Niclas@NWHirschfeld.de>

* Update modules/caddytls/connpolicy.go

* Make review adjustments

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-06-05 12:19:36 -06:00
Matthew Holt 11a132d48b caddytls: Configurable cache size limit 2020-06-05 11:14:39 -06:00
Matthew Holt 9dafa63933 go.mod: Update dependencies 2020-06-05 11:14:09 -06:00
Francis Lavoie 21c1da101c ci: Disable publishing .deb on beta tags (#3473) 2020-06-05 10:23:15 -06:00
Matthew Holt 7a99835dab reverseproxy: Enable changing only the status code (close #2920) 2020-06-04 12:06:38 -06:00
Matthew Holt 7b0962ba4d caddyhttp: Default to error status if found in context
This is just a convenience if using a static_response handler in an
error route, by setting the default status code to the same one as
the error status.
2020-06-04 10:32:01 -06:00
Matthew Holt 2d1f7b9da8 caddyhttp: Auto-redirects from all bind addresses (fix #3443) 2020-06-03 10:56:26 -06:00
Matthew Holt a285fe4129 caddypki: Add 'acme_server' Caddyfile directive 2020-06-03 09:59:36 -06:00
Matthew Holt 97e61c16a3 httpcaddyfile: Sort site blocks with wildcards last (fix #3410) 2020-06-03 09:35:13 -06:00
Matthew Holt 83551edf3e cmd: Only stop admin server on signal if it exists (fix #3470) 2020-06-03 07:31:31 -06:00
Matthew Holt e18c373064 caddytls: Actually use configured test CA 2020-06-02 11:13:44 -06:00
Matt Holt 9a7756c6e4 caddyauth: Cache basicauth results (fixes #3462) (#3465)
Cache capacity is currently hard-coded at 1000 with random eviction.
It is enabled by default from Caddyfile configurations because I assume
this is the most common preference.
2020-06-01 23:56:47 -06:00
Francis Lavoie fdf2a77feb caddyfile: Add args on imports (#3423)
* caddyfile: Add support for args on imports

* caddyfile: Add more import args tests
2020-06-01 10:43:06 -06:00
Georges Haidar a496308f6e httpcaddyfile: Let modules add listener wrappers (#3397)
* httpcaddyfile: allow modules to customize listener wrappers

* Update caddyconfig/httpcaddyfile/httptype.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update caddyconfig/httpcaddyfile/httptype.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update caddyconfig/httpcaddyfile/httptype.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* Update caddyconfig/httpcaddyfile/httptype.go

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-06-01 09:50:00 -06:00
Matthew Holt d5d7fb5954 go.mod: Update dependencies 2020-06-01 09:31:08 -06:00
Matt Holt 996af0915d cmd: Support admin endpoint on unix socket (#3320) 2020-05-29 14:21:55 -06:00
Matthew Holt 6c051cd27d caddyconfig: Minor internal and godoc tweaks 2020-05-29 11:49:25 -06:00
Matt Holt 9415feca7c logging: Net writer redials if write fails (#3453)
* logging: Net writer redials if write fails

https://caddy.community/t/v2-log-output-net-does-not-reconnect-after-lost-connection/8386?u=matt

* Only replace connection if redial succeeds

* Fix error handling
2020-05-28 10:40:14 -06:00
Matthew Holt 881b826fb5 reverseproxy: Pool copy buffers (minor optimization) 2020-05-27 11:42:19 -06:00
Matthew Holt 538ddb8587 reverseproxy: Enable response interception (#1447, #2920)
It's a raw, low-level implementation for now, but it's very flexible.
More sugar-coating can be added after error handling is more developed.
2020-05-27 10:17:45 -06:00
Francis Lavoie 69b5643130 chore: Fix typo in dispenser.go (#3456) 2020-05-27 08:13:57 -06:00
Matthew Holt e5bbed1046 caddyhttp: Refactor header matching
This allows response matchers to benefit from the same matching logic
as the request header matchers (mainly prefix/suffix wildcards).
2020-05-26 17:35:27 -06:00
Matthew Holt 294910c68c caddyhttp: Add client.public_key(_sha256) placeholders 2020-05-26 15:52:53 -06:00
Francis Lavoie 8c5d00b2bc httpcaddyfile: New handle_path directive (#3281)
* caddyconfig: WIP implementation of handle_path

* caddyconfig: Complete the implementation - h.NewRoute was key

* caddyconfig: Add handle_path integration test

* caddyhttp: Use the path matcher as-is, strip the trailing *, update test
2020-05-26 15:27:51 -06:00
Rui Lopes aa20878887 cmd: file-server: add --access-log flag (#3454) 2020-05-26 15:04:04 -06:00
Francis Lavoie c1e5c09294 reverseproxy: Improve error message when using scheme+placeholder (#3393)
* reverseproxy: Improve error message when using scheme+placeholder

* reverseproxy: Simplify error message

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-05-26 14:13:15 -06:00
Francis Lavoie ffc125d6f5 caddyfile: Move NewTestDispenser into non-test file (#3439) 2020-05-26 13:45:22 -06:00
AndyBan 22055c5e0f reverseproxy: Fix https active health checks #3450 (#3451) 2020-05-26 12:40:57 -06:00
Mohammed Al Sahaf dfe802aed3 chore: forego the use of deprecated cel func NewIdent in favor of NewVar (#3444) 2020-05-25 03:59:38 +00:00
Mohammed Al Sahaf 7a365af5df chore: simplify goreleaser flow, add bash completions to .deb (#3436) 2020-05-22 15:13:31 -04:00
Matthew Holt 0cbf467b3f caddyhttp: Add time.now placeholder and update cel-go (closes #2594) 2020-05-21 18:19:01 -06:00
Francis Lavoie bb67e19d7b cmd: hash-password: Fix broken terminal state on SIGINT (#3416)
* caddyauth: Fix hash-password broken terminal state on SIGINT

* caddycmd: Move TrapSignals calls to only subcommands that run long
2020-05-21 13:09:49 -06:00
Matthew Holt 1dc4ec2d77 admin: Disallow websockets
No currently-known exploit here, just being conservative
2020-05-21 12:29:19 -06:00
Matt Holt 452d4726f7 Update SECURITY.md 2020-05-20 14:24:47 -06:00
Matthew Holt 2a8a198568 reverseproxy: Don't overwrite existing X-Forwarded-Proto header
Correct behavior is not well defined because this is a non-standard
header field. This could be a "hop-by-hop" field much like
X-Forwarded-For is, but even our X-Forwarded-For implementation
preserves prior entries. Or, it could be best to preserve the original
value from the first hop, representing the protocol as facing the
client.

Let's try it the other way for a bit and see how it goes.

See https://caddy.community/t/caddy2-w-wordpress-behind-nginx-reverse-proxy/8174/3?u=matt
2020-05-20 11:33:17 -06:00
Francis Lavoie cc8fb488d3 httpcaddyfile: Improve error on matcher declared outside site block (#3431) 2020-05-20 10:37:48 -06:00
Francis Lavoie fae064262d httpcaddyfile: Add auto_https global option (#3284) 2020-05-19 16:59:51 -06:00
Matthew Holt 9ee01dceac reverseproxy: Make debug log safe if error occurs 2020-05-18 14:08:11 -06:00
Matthew Holt 812278acd8 reverseproxy: Emit debug log before checking error (#3425)
This way the upstream request will always be available even if it failed
2020-05-18 13:50:46 -06:00
Matthew Holt c47ddbeffb pki: Add docs to some struct fields 2020-05-18 13:50:46 -06:00
Thorkild Gregersen 483e31b978 templates: trim windows whitespace in SplitFrontMatter; fix #3386 (#3387)
* add test case for SplitFrontMatter showing issue with windows newline

* fix issue with windows newline when using SplitFrontMatter

* Update modules/caddyhttp/templates/frontmatter.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>

* make it mere explicit what is trimmed from firstLine

* Update modules/caddyhttp/templates/frontmatter.go

Co-authored-by: Francis Lavoie <lavofr@gmail.com>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-05-18 13:01:04 -06:00
Francis Lavoie 41a682ddde caddyauth: Add realm to basicauth Caddyfile directive (#3315) 2020-05-18 12:19:28 -06:00
Francis Lavoie 7243454a96 fastcgi: php_fastcgi subdirectives to override shortcut behaviour (#3255)
* fastcgi: Add new php_fastcgi subdirectives to override the shortcut

* fastcgi: Support "index off" to disable redir and try_files

* fastcgi: Remove whitespace to satisfy linter

* fastcgi: Run gofmt

* fastcgi: Make a new dispenser instead of using rewind

* fastcgi: Some fmt

* fastcgi: Add a couple adapt tests

* fastcgi: Clean up for loops

* fastcgi: Move adapt tests to separate files
2020-05-18 12:15:38 -06:00
Matthew Holt 3fb2c394d1 go.mod: Update dependencies
Notably, this adds Caddyfile syntax highlighting in markdown rendering
2020-05-17 17:12:34 -06:00
Francis Lavoie 21de227fe9 httpcaddyfile: Be stricter about log syntax (#3419) 2020-05-15 15:57:16 -06:00
elcore 62c9f2cf3e cmd: Add --envfile flag to run command (#3278)
* run: Add the possibility to load an env file

* run: change envfile flag var

* run: do not ignore err values

* Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-05-15 15:49:51 -06:00
Francis Lavoie bde3823b76 caddytest: Refactor Caddyfile adapt tests to separate files (#3398) 2020-05-14 17:53:28 -04:00
Matthew Holt 4df56c77e3 cmd: Add pidfile support (closes #3235) 2020-05-13 11:28:15 -06:00
Mohammed Al Sahaf cee5589b98 docs: link to CEL standard definitions (#3407)
* docs: link to CEL standard definitions

* Rephrase the anchor to CEL standard definitions

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-05-13 17:11:31 +00:00
Matt Holt 90c7b4b0a1 reverseproxy: Apply response header ops before copying it (fix #3382) (#3401) 2020-05-13 09:52:20 -06:00
Matthew Holt aef560c7fc all: Recover from panics in goroutines 2020-05-12 11:36:20 -06:00
linquize 44536a7594 cmd: reverse-proxy: add --insecure flag (with warning) (#3389) 2020-05-12 10:43:18 -06:00
Francis Lavoie ea7e4b4024 httpcaddyfile: Shorthands for parameterized placeholders (#3305)
* httpcaddyfile: Add shorthands for parameterized placeholders


httpcaddyfile: Now with regexp instead


httpcaddyfile: Allow dashes, gofmt


httpcaddyfile: Compile regexp only once


httpcaddyfile: Cleanup struct


httpcaddyfile: Optimize the replacers, pull out of the loop


httpcaddyfile: Add `{port}` shorthand

* httpcaddyfile: Switch `r.` to `re.`
2020-05-11 16:50:49 -06:00
Francis Lavoie ef6e53bb5f core: Add support for d duration unit (#3323)
* caddy: Add support for `d` duration unit

* Improvements to ParseDuration; add unit tests

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-05-11 16:41:11 -06:00
Francis Lavoie 35e1d92d58 ci: Delete .travis.yml (#3396)
Too flaky. We'll explore different avenues to testing s390x and ppc64le.

See discussion here: https://github.com/caddyserver/caddy/pull/3355

/cc @grooverdan, @Mohammed90 said he'll reach out to Elizabeth as you suggested.
2020-05-11 15:07:02 -06:00
Francis Lavoie dc9f4f13fc httpcaddyfile: Make global options pluggable (#3265)
* httpcaddyfile: Make global options pluggable

* httpcaddyfile: Add a global options adapt test

* httpcaddyfile: Wrap err

Co-Authored-By: Dave Henderson <dhenderson@gmail.com>

* httpcaddyfile: Revert wrap err

Co-authored-by: Dave Henderson <dhenderson@gmail.com>
2020-05-11 15:00:35 -06:00
Francis Lavoie 4c55d26f11 caddyhttp: Fix merging of Caddyfile matchers in not blocks (#3379) 2020-05-11 14:38:33 -06:00
Gregory Dosh d534162556 caddyhttp: Match hostnames with wildcards to loggers (#3378)
* adding wildcard matching of logger names

* reordering precedence for more specific loggers to match first

* removing dependence on certmagic and extra loop

Co-authored-by: GregoryDosh <GregoryDosh@users.noreply.github.com>
2020-05-11 14:17:59 -06:00
Andrew Zhou 5bde8d705b cmd: hash-password: Support reading from stdin (#3373)
Closes #3365 

* http: Add support in hash-password for reading from terminals/stdin

* FIXUP: Run gofmt -s

* FIXUP

* FIXUP: Apply suggestions from code review

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>

* FIXUP

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-05-11 14:10:47 -06:00
Matthew Holt 7960b4259d caddyhttp: Minor refactoring for preparing requests
While building a layer4 app for Caddy, I discovered that we need the
ability to fill a request's context just like the HTTP server does,
hence this exported function PrepareRequest().
2020-05-11 12:14:47 -06:00
Mark Sargent 2c91688f39 fix testharness, dumps the current config, only if the config was successfully loaded (#3385) 2020-05-10 08:11:35 +12:00
Chandler Swift 513e0240fd docs: Fix TOC/section header mismatch (#3380) 2020-05-08 19:46:40 -04:00
Jeremy Lin bf8c3c25c1 log: improve rounding logic for log rolling directives (#3367)
* For `roll_size` and `roll_keep_for` directives, round up instead of down.
  For example, if a user wants to be able to look back on 36 hours of logs,
  but you must round to a 24-hour multiple, then it's better to round up to
  48 hours (which includes the desired 36 hours) instead of down to 24 hours.

* `roll_size` had an off-by-one error that caused the size to be as much as
  1 MB larger than requested. For example, requests of `1MB` and `1.1MB`
  both became 2 MB. Now `1MB` means 1 MB, and `1.1MB` is rounded up to 2 MB.
2020-05-07 13:06:00 -06:00
Matthew Holt c8da8ca673 Update readme 2020-05-07 13:01:33 -06:00
Jose Donizetti 43fba378d6 docs: Fix command.Func documentation (#3371) 2020-05-07 09:31:58 -06:00
Matthew Holt cd9317e5df httpcaddyfile: Fix route ordering bug
https://caddy.community/t/cant-get-simple-alias-to-work/7911/8?u=matt

This removes an optimization where we amortized path matcher decoding.
The decoded matchers were index by... position... which obviously
changes during sorting. Duh.

Anyway, sorting is sliiightly slower now but the Caddyfile is not
really CPU-sensitive, so this is fine.
2020-05-06 19:41:37 -06:00
Matthew Holt 8dbc5f70a5 Update dependencies and get rid of placeholder hacks in CA code
With the latest commit on smallstep/certificates, placeholders in config
are no longer needed.
2020-05-06 16:02:21 -06:00
Francis Lavoie 07c6076ea0 ci: Add release tagged event triggers to sister repos (#3321) 2020-05-06 16:42:55 -04:00
Matthew Holt 28ab0bfb13 core: Support loading modules from [][]json.RawMessage fields 2020-05-06 13:18:56 -06:00
Matthew Holt 1c17e6c6bb reverseproxy: Allow using TLS for port 80 upstreams (see #3361)
An upstream like https://localhost:80 is still forbidden, but an addr of
localhost:80 can be used while explicitly enabling TLS as an override;
we just don't allow the implicit behavior to be ambiguous.
2020-05-06 12:37:44 -06:00
Karol Będkowski b814c0af9c tls/client auth: verify first certificates in client request (#3344)
When client certificate is enabled Caddy check only last certificate from
request. When this cert is not in list of trusted leaf certificates,
connection is rejected. According to RFC TLS1.x the sender's certificate
must come first in the list.  Each following certificate must directly
certify the one preceding it.

This patch fix this problem - first certificate is checked instead of last.
2020-05-06 10:07:13 -06:00
Dave Henderson 9e5d9e2530 ci: Add linux-armv5 builds (#3356)
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2020-05-05 19:13:56 -06:00
Dave Henderson 9408dacc27 Fixing goreleaser syntax error (#3355)
Signed-off-by: Dave Henderson <dhenderson@gmail.com>
2020-05-06 00:50:30 +00:00
Daniel Black 12cfc19487 ci: add s390x and ppc64le builds for linux (#3325) 2020-05-05 12:41:08 -06:00
Francis Lavoie afecd90a6c reverseproxy: Add tls_server_name option to Caddyfile (#3322) 2020-05-05 12:39:39 -06:00
Matt Holt 2f59467ac3 httpcaddyfile: Only append TLS conn policy if it's non-empty (#3319)
This can lead to nicer, smaller JSON output for Caddyfiles like this:

	a {
		tls internal
	}
	b {
		tls foo@bar.com
	}

i.e. where the tls directive only configures automation policies, and
is merely meant to enable TLS on a server block (if it wasn't implied).
This helps keeps implicit config implicit.

Needs a little more testing to ensure it doesn't break anything
important.
2020-05-05 12:37:52 -06:00
Matt Holt 184e8e9f71 pki: Embedded ACME server (#3198)
* pki: Initial commit of embedded ACME server (#3021)

* reverseproxy: Support auto-managed TLS client certificates (#3021)

* A little cleanup after today's review session
2020-05-05 12:35:32 -06:00
Matt Holt 1e8c9764df file_server: Accept files args in one-liner of Caddyfile matcher (#3298)
Previously, matching by trying files other than the actual path of the
URI was:

    file {
        try_files <files...>
    }

Now, the same can be done in one line:

    file <files...>

As before, an empty file matcher:

    file

still matches if the request URI exists as a file in the site root.
2020-05-05 12:34:58 -06:00
Matt Holt 41c7bd27b4 httpserver: Add experimental H2C support (#3289)
* reverse_proxy: Initial attempt at H2C transport/client support (#3218)

I have not tested this yet

* Experimentally enabling H2C server support (closes #3227)

See also #3218

I have not tested this

* reverseproxy: Clean up H2C transport a bit

* caddyhttp: Update godoc for h2c server; clarify experimental status

* caddyhttp: Fix trailers when recording responses (fixes #3236)

* caddyhttp: Tweak h2c config settings and docs
2020-05-05 12:33:21 -06:00
Francis Lavoie 96d6d277a4 caddyconfig: Don't start comments in middle of tokens (#3267)
* caddyconfig: Only parse # as start of comment if preceded by space

* caddyconfig: Simplify # logic using len(val), add a test
2020-05-05 12:32:12 -06:00
Francis Lavoie 26e559662d httpcaddyfile: Support single-line matchers (#3263)
* httpcaddyfile: Support single-line matchers

* httpcaddyfile: Add single-line matcher test

* httpcaddyfile: Add a matcher syntax adapt test
2020-05-05 12:29:21 -06:00
Matt Holt 52305618df caddyfile: Support backticks as quotes (closes #2591) (#3242) 2020-05-05 12:27:49 -06:00
350 changed files with 39624 additions and 7545 deletions
+5
View File
@@ -0,0 +1,5 @@
[*]
end_of_line = lf
[caddytest/integration/caddyfile_adapt/*.txt]
indent_style = tab
+1
View File
@@ -0,0 +1 @@
*.go text eol=lf
+17 -14
View File
@@ -23,13 +23,13 @@ Other menu items:
### Contributing code
You can have a huge impact on the project by helping with its code. To contribute code to Caddy, open a [pull request](https://github.com/caddyserver/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with Caddy's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/caddyserver/caddy/-/search).
You can have a huge impact on the project by helping with its code. To contribute code to Caddy, first submit or comment in an issue to discuss your contribution, then open a [pull request](https://github.com/caddyserver/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with Caddy's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/caddyserver/caddy).
We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions&mdash;even if it seems small or insignificant. Please don't take it personally. :blue_heart: If your change is on the right track, we can guide you to make it mergable.
Here are some of the expectations we have of contributors:
- **Open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that any changes are in-line with the project's goals and the best interests of its users. We can also discuss the best possible implementation. If there's already an issue about it, comment on the existing issue to claim it.
- **Open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that any changes are in-line with the project's goals and the best interests of its users. We can also discuss the best possible implementation. If there's already an issue about it, comment on the existing issue to claim it. A lot of valuable time can be saved by discussing a proposal first.
- **Keep pull requests small.** Smaller PRs are more likely to be merged because they are easier to review! We might ask you to break up large PRs into smaller ones. [An example of what we want to avoid.](https://twitter.com/iamdevloper/status/397664295875805184)
@@ -45,16 +45,18 @@ Here are some of the expectations we have of contributors:
- **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious.
- **Pull requests may still get closed.** The longer a PR stays open and idle, the more likely it is to be closed. If we haven't reviewed it in a while, it probably means the change is not a priority. Please don't take this personally, we're trying to balance a lot of tasks! If nobody else has commented or reacted to the PR, it likely means your change is useful only to you. The reality is this happens quite a bit. We don't tend to accept PRs that aren't generally helpful. For these reasons or others, the PR may get closed even after a review. We are not obligated to accept all proposed changes, even if the best justification we can give is something vague like, "It doesn't sit right." Sometimes PRs are just the wrong thing or the wrong time. Because it is open source, you can always build your own modified version of Caddy with a change you need, even if we reject it in the official repo.
We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base!
#### HOW TO MAKE A PULL REQUEST TO CADDY
Contributing to Go projects on GitHub is fun and easy. We recommend the following workflow:
Contributing to Go projects on GitHub is fun and easy. After you have proposed your change in an issue, we recommend the following workflow:
1. [Fork this repo](https://github.com/caddyserver/caddy). This makes a copy of the code you can write to.
2. If you don't already have this repo (caddyserver/caddy.git) repo on your computer, get it with `go get github.com/caddyserver/caddy/v2`.
2. If you don't already have this repo (caddyserver/caddy.git) repo on your computer, clone it down: `git clone https://github.com/caddyserver/caddy.git`
3. Tell git that it can push the caddyserver/caddy.git repo to your fork by adding a remote: `git remote add myfork https://github.com/<your-username>/caddy.git`
@@ -85,9 +87,9 @@ Many people on the forums could benefit from your experience and expertise, too.
Like every software, Caddy has its flaws. If you find one, [search the issues](https://github.com/caddyserver/caddy/issues) to see if it has already been reported. If not, [open a new issue](https://github.com/caddyserver/caddy/issues/new) and describe the bug, and somebody will look into it! (This repository is only for Caddy and its standard modules.)
**You can help stop bugs in their tracks!** Speed up the patch by identifying the bug in the code. This can sometimes be done by adding `fmt.Println()` statements (or similar) in relevant code paths to narrow down where the problem may be. It's a good way to [introduce yourself to the Go language](https://tour.golang.org), too.
**You can help us fix bugs!** Speed up the patch by identifying the bug in the code. This can sometimes be done by adding `fmt.Println()` statements (or similar) in relevant code paths to narrow down where the problem may be. It's a good way to [introduce yourself to the Go language](https://tour.golang.org), too.
Please follow the issue template so we have all the needed information. Unredacted&mdash;yes, actual values matter. We need to be able to repeat the bug using your instructions. Please simplify the issue as much as possible. The burden is on you to convince us that it is actually a bug in Caddy. This is easiest to do when you write clear, concise instructions so we can reproduce the behavior (even if it seems obvious). The more detailed and specific you are, the faster we will be able to help you!
We may reply with an issue template. Please follow the template so we have all the needed information. Unredacted&mdash;yes, actual values matter. We need to be able to repeat the bug using your instructions. Please simplify the issue as much as possible. If you don't, we might close your report. The burden is on you to make it easily reproducible and to convince us that it is actually a bug in Caddy. This is easiest to do when you write clear, concise instructions so we can reproduce the behavior (even if it seems obvious). The more detailed and specific you are, the faster we will be able to help you!
We suggest reading [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
@@ -98,11 +100,12 @@ Please be kind. :smile: Remember that Caddy comes at no cost to you, and you're
Maintainers---or more generally, developers---need three things to act on bugs:
1. To agree or be convinced that it's a bug (reporter's responsibility).
- A bug is undesired or surprising behavior which violates documentation or the spec.
- A bug is unintentional, undesired, or surprising behavior which violates documentation or relevant spec. It might be either a mistake in the documentation or a bug in the code.
- This project usually does not work around bugs in other software, systems, and dependencies; instead, we recommend that those bugs are fixed at their source. This sometimes means we close issues or reject PRs that attempt to fix, workaround, or hide bugs in other projects.
2. To be able to understand what is happening (mostly reporter's responsibility).
- If the reporter can provide satisfactory instructions such that a developer can reproduce the bug, the developer will likely be able to understand the bug, write a test case, and implement a fix.
- Otherwise, the burden is on the reporter to test possible solutions. This is discouraged because it loosens the feedback loop, slows down debugging efforts, obscures the true nature of the problem from the developers, and is unlikely to result in new test cases.
- If the reporter can provide satisfactory instructions such that a developer can reproduce the bug, the developer will likely be able to understand the bug, write a test case, and implement a fix. This is the least amount of work for everyone and path to the fastest resolution.
- Otherwise, the burden is on the reporter to test possible solutions. This is less preferable because it loosens the feedback loop, slows down debugging efforts, obscures the true nature of the problem from the developers, and is unlikely to result in new test cases.
3. A solution, or ideas toward a solution (mostly maintainer's responsibility).
- Sometimes the best solution is a documentation change.
@@ -112,7 +115,7 @@ Maintainers---or more generally, developers---need three things to act on bugs:
Thus, at the very least, the reporter is expected to:
1. Convince the reader that it's a bug (if it's not obvious).
1. Convince the reader that it's a bug in Caddy (if it's not obvious).
2. Reduce the problem down to the minimum specific steps required to reproduce it.
The maintainer is usually able to do the rest; but of course the reporter may invest additional effort to speed up the process.
@@ -123,7 +126,7 @@ The maintainer is usually able to do the rest; but of course the reporter may in
First, [search to see if your feature has already been requested](https://github.com/caddyserver/caddy/issues). If it has, you can add a :+1: reaction to vote for it. If your feature idea is new, open an issue to request the feature. Please describe your idea thoroughly so that we know how to implement it! Really vague requests may not be helpful or actionable and, without clarification, will have to be closed.
While we really do value your requests and implement many of them, not all features are a good fit for Caddy. Most of those [make good modules](#writing-a-caddy-module), which can be made by anyone! But if a feature is not in the best interest of the Caddy project or its users in general, we may politely decline to implement it into Caddy core.
While we really do value your requests and implement many of them, not all features are a good fit for Caddy. Most of those [make good modules](#writing-a-caddy-module), which can be made by anyone! But if a feature is not in the best interest of the Caddy project or its users in general, we may politely decline to implement it into Caddy core. Additionally, some features are bad ideas altogether (for either obvious or non-obvious reasons) which may be rejected. We'll try to explain why we reject a feature, but sometimes the best we can do is, "It's not a good fit for the project."
### Improving documentation
@@ -132,11 +135,11 @@ Caddy's documentation is available at [https://caddyserver.com/docs](https://cad
Note that third-party module documentation is not hosted by the Caddy website, other than basic usage examples. They are managed by the individual module authors, and you will have to contact them to change their documentation.
Our documentation is scoped to the Caddy project only: it is not for describing how other software or systems work, even if they relate to Caddy or web servers. That kind of content [can be found in our community wiki](https://caddy.community/c/wiki/13), however.
## Collaborator Instructions
Collaborators have push rights to the repository. We grant this permission after one or more successful, high-quality PRs are merged! We thank them for their help.The expectations we have of collaborators are:
Collaborators have push rights to the repository. We grant this permission after one or more successful, high-quality PRs are merged! We thank them for their help. The expectations we have of collaborators are:
- **Help review pull requests.** Be meticulous, but also kind. We love our contributors, but we critique the contribution to make it better. Multiple, thorough reviews make for the best contributions! Here are some questions to consider:
- Can the change be made more elegant?
@@ -167,7 +170,7 @@ Collaborators have push rights to the repository. We grant this permission after
## Values
## Values (WIP)
- A person is always more important than code. People don't like being handled "efficiently". But we can still process issues and pull requests efficiently while being kind, patient, and considerate.
+39 -7
View File
@@ -2,26 +2,58 @@
The Caddy project would like to make sure that it stays on top of all practically-exploitable vulnerabilities.
Some security problems are more the result of interplay between different components of the Web, rather than a vulnerability in the web server itself. Please report only vulnerabilities in the web server itself, as we cannot coerce the rest of the Web to be fixed (for example, we do not consider IP spoofing or BGP hijacks a vulnerability in the Caddy web server).
Please note that we consider publicly-registered domain names to be public information. This necessary in order to maintain the integrity of certificate transparency, public DNS, and other public trust systems.
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 2.x | :white_check_mark: |
| 1.x | :white_check_mark: (deprecating soon) |
| 1.x | :x: |
| < 1.x | :x: |
## Acceptable Scope
A security report must demonstrate a security bug in the source code from this repository.
Some security problems are the result of interplay between different components of the Web, rather than a vulnerability in the web server itself. Please only report vulnerabilities in the web server itself, as we cannot coerce the rest of the Web to be fixed (for example, we do not consider IP spoofing, BGP hijacks, or missing/misconfigured HTTP headers a vulnerability in the Caddy web server).
Vulnerabilities caused by misconfigurations are out of scope. Yes, it is entirely possible to craft and use a configuration that is unsafe, just like with every other web server; we recommend against doing that.
We do not accept reports if the steps imply or require a compromised system or third-party software, as we cannot control those. We expect that users secure their own systems and keep all their software patched. For example, if untrusted users are able to upload/write/host arbitrary files in the web root directory, it is NOT a security bug in Caddy if those files get served to clients; however, it _would_ be a valid report if a bug in Caddy's source code unintentionally gave unauthorized users the ability to upload unsafe files or delete files without relying on an unpatched system or piece of software.
Client-side exploits are out of scope. In other words, it is not a bug in Caddy if the web browser does something unsafe, even if the downloaded content was served by Caddy. (Those kinds of exploits can generally be mitigated by proper configuration of HTTP headers.) As a general rule, the content served by Caddy is not considered in scope because content is configurable by the site owner or the associated web application.
Security bugs in code dependencies are out of scope. Instead, if a dependency has patched a relevant security bug, please feel free to open a public issue or pull request to update that dependency in our code.
## Reporting a Vulnerability
Please email Matt Holt (the author) directly: Matthew dot Holt at Gmail.
We get a lot of difficult reports that turn out to be invalid. Clear, obvious reports tend to be the most credible (but are also rare).
We'll need enough information to verify the bug and make a patch. It will speed things up if you suggest a working patch, such as a code diff, and explain why and how it works. Reports that are not actionable, do not contain enough information, are too pushy/demanding, or are not able to convince us that it is a viable and practical attack on the web server itself may be deferred to a later time or possibly ignored, resources permitting. Priority will be given to credible, responsible reports that are constructive, specific, and actionable. Thank you for understanding.
First please ensure your report falls within the accepted scope of security bugs (above).
We'll need enough information to verify the bug and make a patch. To speed things up, please include:
- Most minimal possible config (without redactions!)
- Command(s)
- Precise HTTP requests (`curl -v` and its output please)
- Full log output (please enable debug mode)
- Specific minimal steps to reproduce the issue from scratch
- A working patch
Please DO NOT use containers, VMs, cloud instances or services, or any other complex infrastructure in your steps. Always prefer `curl` instead of web browsers.
We consider publicly-registered domain names to be public information. This necessary in order to maintain the integrity of certificate transparency, public DNS, and other public trust systems. Do not redact domain names from your reports. The actual content of your domain name affects Caddy's behavior, so we need the exact domain name(s) to reproduce with, or your report will be ignored.
It will speed things up if you suggest a working patch, such as a code diff, and explain why and how it works. Reports that are not actionable, do not contain enough information, are too pushy/demanding, or are not able to convince us that it is a viable and practical attack on the web server itself may be deferred to a later time or possibly ignored, depending on available resources. Priority will be given to credible, responsible reports that are constructive, specific, and actionable. (We get a lot of invalid reports.) Thank you for understanding.
When you are ready, please email Matt Holt (the author) directly: matt at dyanim dot com.
Please don't encrypt the email body. It only makes the process more complicated.
Please also understand that due to our nature as an open source project, we do not have a budget to award security bounties. We can only thank you.
If your report is valid and a patch is released, we will not reveal your identity by default. If you wish to be credited, please give us the name to use.
If your report is valid and a patch is released, we will not reveal your identity by default. If you wish to be credited, please give us the name to use and/or your GitHub username. If you don't provide this we can't credit you.
Thanks for responsibly helping Caddy&mdash;and thousands of websites&mdash;be more secure!
+70 -22
View File
@@ -1,14 +1,16 @@
# Used as inspiration: https://github.com/mvdan/github-actions-golang
name: Cross-Platform
name: Tests
on:
push:
branches:
branches:
- master
- 2.*
pull_request:
branches:
branches:
- master
- 2.*
jobs:
test:
@@ -17,12 +19,20 @@ jobs:
fail-fast: false
matrix:
os: [ ubuntu-latest, macos-latest, windows-latest ]
go-version: [ 1.14.x ]
go: [ '1.18', '1.19' ]
include:
# Set the minimum Go patch version for the given Go minor
# Usable via ${{ matrix.GO_SEMVER }}
- go: '1.18'
GO_SEMVER: '~1.18.4'
- go: '1.19'
GO_SEMVER: '~1.19.0'
# Set some variables per OS, usable via ${{ matrix.VAR }}
# CADDY_BIN_PATH: the path to the compiled Caddy binary, for artifact publishing
# SUCCESS: the typical value for $? per OS (Windows/pwsh returns 'True')
include:
- os: ubuntu-latest
CADDY_BIN_PATH: ./cmd/caddy/caddy
SUCCESS: 0
@@ -39,12 +49,13 @@ jobs:
steps:
- name: Install Go
uses: actions/setup-go@v1
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
go-version: ${{ matrix.GO_SEMVER }}
check-latest: true
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v3
# These tools would be useful if we later decide to reinvestigate
# publishing test/coverage reports to some tool for easier consumption
@@ -64,17 +75,26 @@ jobs:
go env
printf "\n\nSystem environment:\n\n"
env
printf "Git version: $(git version)\n\n"
# Calculate the short SHA1 hash of the git commit
echo "::set-output name=short_sha::$(git rev-parse --short HEAD)"
echo "::set-output name=go_cache::$(go env GOCACHE)"
- name: Cache the build cache
uses: actions/cache@v1
uses: actions/cache@v2
with:
path: ${{ steps.vars.outputs.go_cache }}
key: ${{ runner.os }}-go-ci-${{ hashFiles('**/go.sum') }}
# In order:
# * Module download cache
# * Build cache (Linux)
# * Build cache (Mac)
# * Build cache (Windows)
path: |
~/go/pkg/mod
~/.cache/go-build
~/Library/Caches/go-build
~\AppData\Local\go-build
key: ${{ runner.os }}-${{ matrix.go }}-go-ci-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-ci
${{ runner.os }}-${{ matrix.go }}-go-ci
- name: Get dependencies
run: |
@@ -91,7 +111,7 @@ jobs:
- name: Publish Build Artifact
uses: actions/upload-artifact@v1
with:
name: caddy_v2_${{ runner.os }}_${{ steps.vars.outputs.short_sha }}
name: caddy_${{ runner.os }}_go${{ matrix.go }}_${{ steps.vars.outputs.short_sha }}
path: ${{ matrix.CADDY_BIN_PATH }}
# Commented bits below were useful to allow the job to continue
@@ -120,16 +140,44 @@ jobs:
# echo "step_test ${{ steps.step_test.outputs.status }}\n"
# exit 1
# From https://github.com/reviewdog/action-golangci-lint
golangci-lint:
name: runner / golangci-lint
s390x-test:
name: test (s390x on IBM Z)
runs-on: ubuntu-latest
if: github.event.pull_request.head.repo.full_name == github.repository
continue-on-error: true # August 2020: s390x VM is down due to weather and power issues
steps:
- name: Checkout code into the Go module directory
uses: actions/checkout@v2
uses: actions/checkout@v3
- name: Run Tests
run: |
mkdir -p ~/.ssh && echo -e "${SSH_KEY//_/\\n}" > ~/.ssh/id_ecdsa && chmod og-rwx ~/.ssh/id_ecdsa
- name: Run golangci-lint
uses: reviewdog/action-golangci-lint@v1
# uses: docker://reviewdog/action-golangci-lint:v1 # pre-build docker image
# short sha is enough?
short_sha=$(git rev-parse --short HEAD)
# The environment is fresh, so there's no point in keeping accepting and adding the key.
rsync -arz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress --delete --exclude '.git' . "$CI_USER"@ci-s390x.caddyserver.com:/var/tmp/"$short_sha"
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -t "$CI_USER"@ci-s390x.caddyserver.com "cd /var/tmp/$short_sha; go version; go env; printf "\n\n";CGO_ENABLED=0 go test -v ./..."
test_result=$?
# There's no need leaving the files around
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null "$CI_USER"@ci-s390x.caddyserver.com "rm -rf /var/tmp/'$short_sha'"
echo "Test exit code: $test_result"
exit $test_result
env:
SSH_KEY: ${{ secrets.S390X_SSH_KEY }}
CI_USER: ${{ secrets.CI_USER }}
goreleaser-check:
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v3
- uses: goreleaser/goreleaser-action@v2
with:
github_token: ${{ secrets.github_token }}
version: latest
args: check
env:
TAG: ${{ steps.vars.outputs.version_tag }}
+74
View File
@@ -0,0 +1,74 @@
name: Cross-Build
on:
push:
branches:
- master
- 2.*
pull_request:
branches:
- master
- 2.*
jobs:
cross-build-test:
strategy:
fail-fast: false
matrix:
goos: ['android', 'linux', 'solaris', 'illumos', 'dragonfly', 'freebsd', 'openbsd', 'plan9', 'windows', 'darwin', 'netbsd']
go: [ '1.19' ]
include:
# Set the minimum Go patch version for the given Go minor
# Usable via ${{ matrix.GO_SEMVER }}
- go: '1.19'
GO_SEMVER: '~1.19.0'
runs-on: ubuntu-latest
continue-on-error: true
steps:
- name: Install Go
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.GO_SEMVER }}
check-latest: true
- name: Print Go version and environment
id: vars
run: |
printf "Using go at: $(which go)\n"
printf "Go version: $(go version)\n"
printf "\n\nGo environment:\n\n"
go env
printf "\n\nSystem environment:\n\n"
env
- name: Cache the build cache
uses: actions/cache@v2
with:
# In order:
# * Module download cache
# * Build cache (Linux)
path: |
~/go/pkg/mod
~/.cache/go-build
key: cross-build-go${{ matrix.go }}-${{ matrix.goos }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
cross-build-go${{ matrix.go }}-${{ matrix.goos }}
- name: Checkout code into the Go module directory
uses: actions/checkout@v3
- name: Run Build
env:
CGO_ENABLED: 0
GOOS: ${{ matrix.goos }}
shell: bash
continue-on-error: true
working-directory: ./cmd/caddy
run: |
GOOS=$GOOS go build -trimpath -o caddy-"$GOOS"-amd64 2> /dev/null
if [ $? -ne 0 ]; then
echo "::warning ::$GOOS Build Failed"
exit 0
fi
-73
View File
@@ -1,73 +0,0 @@
name: Fuzzing
on:
# Daily midnight fuzzing
schedule:
- cron: '0 0 * * *'
jobs:
fuzzing:
name: Fuzzing
strategy:
matrix:
os: [ ubuntu-latest ]
go-version: [ 1.14.x ]
runs-on: ${{ matrix.os }}
steps:
- name: Install Go
uses: actions/setup-go@v1
with:
go-version: ${{ matrix.go-version }}
- name: Checkout code
uses: actions/checkout@v2
- name: Download go-fuzz tools and the Fuzzit CLI, move Fuzzit CLI to GOBIN
# If we decide we need to prevent this from running on forks, we can use this line:
# if: github.repository == 'caddyserver/caddy'
run: |
go get -v github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build
wget -q -O fuzzit https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.77/fuzzit_Linux_x86_64
chmod a+x fuzzit
mv fuzzit $(go env GOPATH)/bin
echo "::add-path::$(go env GOPATH)/bin"
- name: Generate fuzzers & submit them to Fuzzit
continue-on-error: true
env:
FUZZIT_API_KEY: ${{ secrets.FUZZIT_API_KEY }}
SYSTEM_PULLREQUEST_SOURCEBRANCH: ${{ github.ref }}
BUILD_SOURCEVERSION: ${{ github.sha }}
run: |
# debug
echo "PR Source Branch: $SYSTEM_PULLREQUEST_SOURCEBRANCH"
echo "Source version: $BUILD_SOURCEVERSION"
declare -A fuzzers_funcs=(\
["./caddyconfig/httpcaddyfile/addresses_fuzz.go"]="FuzzParseAddress" \
["./listeners_fuzz.go"]="FuzzParseNetworkAddress" \
["./replacer_fuzz.go"]="FuzzReplacer" \
)
declare -A fuzzers_targets=(\
["./caddyconfig/httpcaddyfile/addresses_fuzz.go"]="parse-address" \
["./listeners_fuzz.go"]="parse-network-address" \
["./replacer_fuzz.go"]="replacer" \
)
fuzz_type="fuzzing"
for f in $(find . -name \*_fuzz.go); do
FUZZER_DIRECTORY=$(dirname "$f")
echo "go-fuzz-build func ${fuzzers_funcs[$f]} residing in $f"
go-fuzz-build -func "${fuzzers_funcs[$f]}" -o "$FUZZER_DIRECTORY/${fuzzers_targets[$f]}.zip" "$FUZZER_DIRECTORY"
fuzzit create job --engine go-fuzz caddyserver/"${fuzzers_targets[$f]}" "$FUZZER_DIRECTORY"/"${fuzzers_targets[$f]}.zip" --api-key "${FUZZIT_API_KEY}" --type "${fuzz_type}" --branch "${SYSTEM_PULLREQUEST_SOURCEBRANCH}" --revision "${BUILD_SOURCEVERSION}"
echo "Completed $f"
done
+35
View File
@@ -0,0 +1,35 @@
name: Lint
on:
push:
branches:
- master
- 2.*
pull_request:
branches:
- master
- 2.*
jobs:
# From https://github.com/golangci/golangci-lint-action
golangci:
name: lint
strategy:
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- uses: actions/setup-go@v3
with:
go-version: '~1.18.4'
check-latest: true
- name: golangci-lint
uses: golangci/golangci-lint-action@v3
with:
version: v1.47
# Windows times out frequently after about 5m50s if we don't set a longer timeout.
args: --timeout 10m
# Optional: show only new issues if it's a pull request. The default value is `false`.
# only-new-issues: true
+132 -16
View File
@@ -11,21 +11,45 @@ jobs:
strategy:
matrix:
os: [ ubuntu-latest ]
go-version: [ 1.14.x ]
go: [ '1.19' ]
include:
# Set the minimum Go patch version for the given Go minor
# Usable via ${{ matrix.GO_SEMVER }}
- go: '1.19'
GO_SEMVER: '~1.19.0'
runs-on: ${{ matrix.os }}
# https://github.com/sigstore/cosign/issues/1258#issuecomment-1002251233
# https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#adding-permissions-settings
permissions:
id-token: write
# https://docs.github.com/en/rest/overview/permissions-required-for-github-apps#permission-on-contents
# "Releases" is part of `contents`, so it needs the `write`
contents: write
steps:
- name: Install Go
uses: actions/setup-go@v1
uses: actions/setup-go@v3
with:
go-version: ${{ matrix.go-version }}
go-version: ${{ matrix.GO_SEMVER }}
check-latest: true
- name: Checkout code
uses: actions/checkout@v2
uses: actions/checkout@v3
with:
fetch-depth: 0
# So GoReleaser can generate the changelog properly
- name: Unshallowify the repo clone
run: git fetch --prune --unshallow
# Force fetch upstream tags -- because 65 minutes
# tl;dr: actions/checkout@v3 runs this line:
# git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +ebc278ec98bb24f2852b61fde2a9bf2e3d83818b:refs/tags/
# which makes its own local lightweight tag, losing all the annotations in the process. Our earlier script ran:
# git fetch --prune --unshallow
# which doesn't overwrite that tag because that would be destructive.
# Credit to @francislavoie for the investigation.
# https://github.com/actions/checkout/issues/290#issuecomment-680260080
- name: Force fetch upstream tags
run: git fetch --tags --force
# https://github.community/t5/GitHub-Actions/How-to-get-just-the-tag-name/m-p/32167/highlight/true#M1027
- name: Print Go version and environment
@@ -39,30 +63,122 @@ jobs:
env
echo "::set-output name=version_tag::${GITHUB_REF/refs\/tags\//}"
echo "::set-output name=short_sha::$(git rev-parse --short HEAD)"
echo "::set-output name=go_cache::$(go env GOCACHE)"
# Add "pip install" CLI tools to PATH
echo ~/.local/bin >> $GITHUB_PATH
# Parse semver
TAG=${GITHUB_REF/refs\/tags\//}
SEMVER_RE='[^0-9]*\([0-9]*\)[.]\([0-9]*\)[.]\([0-9]*\)\([0-9A-Za-z\.-]*\)'
TAG_MAJOR=`echo ${TAG#v} | sed -e "s#$SEMVER_RE#\1#"`
TAG_MINOR=`echo ${TAG#v} | sed -e "s#$SEMVER_RE#\2#"`
TAG_PATCH=`echo ${TAG#v} | sed -e "s#$SEMVER_RE#\3#"`
TAG_SPECIAL=`echo ${TAG#v} | sed -e "s#$SEMVER_RE#\4#"`
echo "::set-output name=tag_major::${TAG_MAJOR}"
echo "::set-output name=tag_minor::${TAG_MINOR}"
echo "::set-output name=tag_patch::${TAG_PATCH}"
echo "::set-output name=tag_special::${TAG_SPECIAL}"
# Cloudsmith CLI tooling for pushing releases
# See https://help.cloudsmith.io/docs/cli
- name: Install Cloudsmith CLI
run: pip install --upgrade cloudsmith-cli
- name: Validate commits and tag signatures
run: |
# Import Matt Holt's key
curl 'https://github.com/mholt.gpg' | gpg --import
echo "Verifying the tag: ${{ steps.vars.outputs.version_tag }}"
# tags are only accepted if signed by Matt's key
git verify-tag "${{ steps.vars.outputs.version_tag }}" || exit 1
- name: Cache the build cache
uses: actions/cache@v1
uses: actions/cache@v2
with:
path: ${{ steps.vars.outputs.go_cache }}
key: ${{ runner.os }}-go-release-${{ hashFiles('**/go.sum') }}
# In order:
# * Module download cache
# * Build cache (Linux)
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-go${{ matrix.go }}-release-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-release
${{ runner.os }}-go${{ matrix.go }}-release
- name: Install Cosign
uses: sigstore/cosign-installer@main
- name: Cosign version
run: cosign version
- name: Install Syft
uses: anchore/sbom-action/download-syft@main
- name: Syft version
run: syft version
# GoReleaser will take care of publishing those artifacts into the release
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v1
uses: goreleaser/goreleaser-action@v2
with:
version: latest
args: release --rm-dist
args: release --rm-dist --timeout 60m
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TAG: ${{ steps.vars.outputs.version_tag }}
COSIGN_EXPERIMENTAL: 1
# Only publish on non-special tags (e.g. non-beta)
# We will continue to push to Gemfury for the foreseeable future, although
# Cloudsmith is probably better, to not break things for existing users of Gemfury.
# See https://gemfury.com/caddy/deb:caddy
- name: Publish .deb to Gemfury
if: ${{ steps.vars.outputs.tag_special == '' }}
env:
GEMFURY_PUSH_TOKEN: ${{ secrets.GEMFURY_PUSH_TOKEN }}
run: |
for filename in dist/*.deb; do
# armv6 and armv7 are both "armhf" so we can skip the duplicate
if [[ "$filename" == *"armv6"* ]]; then
echo "Skipping $filename"
continue
fi
curl -F package=@"$filename" https://${GEMFURY_PUSH_TOKEN}:@push.fury.io/caddy/
done
done
# Publish only special tags (unstable/beta/rc) to the "testing" repo
# See https://cloudsmith.io/~caddy/repos/testing/
- name: Publish .deb to Cloudsmith (special tags)
if: ${{ steps.vars.outputs.tag_special != '' }}
env:
CLOUDSMITH_API_KEY: ${{ secrets.CLOUDSMITH_API_KEY }}
run: |
for filename in dist/*.deb; do
# armv6 and armv7 are both "armhf" so we can skip the duplicate
if [[ "$filename" == *"armv6"* ]]; then
echo "Skipping $filename"
continue
fi
echo "Pushing $filename to 'testing'"
cloudsmith push deb caddy/testing/any-distro/any-version $filename
done
# Publish stable tags to Cloudsmith to both repos, "stable" and "testing"
# See https://cloudsmith.io/~caddy/repos/stable/
- name: Publish .deb to Cloudsmith (stable tags)
if: ${{ steps.vars.outputs.tag_special == '' }}
env:
CLOUDSMITH_API_KEY: ${{ secrets.CLOUDSMITH_API_KEY }}
run: |
for filename in dist/*.deb; do
# armv6 and armv7 are both "armhf" so we can skip the duplicate
if [[ "$filename" == *"armv6"* ]]; then
echo "Skipping $filename"
continue
fi
echo "Pushing $filename to 'stable'"
cloudsmith push deb caddy/stable/any-distro/any-version $filename
echo "Pushing $filename to 'testing'"
cloudsmith push deb caddy/testing/any-distro/any-version $filename
done
+34
View File
@@ -0,0 +1,34 @@
name: Release Published
# Event payload: https://developer.github.com/webhooks/event-payloads/#release
on:
release:
types: [published]
jobs:
release:
name: Release Published
strategy:
matrix:
os: [ ubuntu-latest ]
runs-on: ${{ matrix.os }}
steps:
# See https://github.com/peter-evans/repository-dispatch
- name: Trigger event on caddyserver/dist
uses: peter-evans/repository-dispatch@v1
with:
token: ${{ secrets.REPO_DISPATCH_TOKEN }}
repository: caddyserver/dist
event-type: release-tagged
client-payload: '{"tag": "${{ github.event.release.tag_name }}"}'
- name: Trigger event on caddyserver/caddy-docker
uses: peter-evans/repository-dispatch@v1
with:
token: ${{ secrets.REPO_DISPATCH_TOKEN }}
repository: caddyserver/caddy-docker
event-type: release-tagged
client-payload: '{"tag": "${{ github.event.release.tag_name }}"}'
+7 -2
View File
@@ -1,13 +1,14 @@
_gitignore/
*.log
Caddyfile
Caddyfile.*
!caddyfile/
# artifacts from pprof tooling
*.prof
*.test
# build artifacts
# build artifacts and helpers
cmd/caddy/caddy
cmd/caddy/caddy.exe
@@ -20,4 +21,8 @@ vendor
# goreleaser artifacts
dist
caddy-build
caddy-dist
caddy-dist
# IDE files
.idea/
.vscode/
+52 -5
View File
@@ -1,21 +1,68 @@
linters-settings:
errcheck:
ignore: fmt:.*,io/ioutil:^Read.*,github.com/caddyserver/caddy/v2/caddyconfig:RegisterAdapter,github.com/caddyserver/caddy/v2:RegisterModule
ignore: fmt:.*,go.uber.org/zap/zapcore:^Add.*
ignoretests: true
misspell:
locale: US
linters:
disable-all: true
enable:
- bodyclose
- prealloc
- unconvert
- deadcode
- errcheck
- gofmt
- goimports
- gosec
- gosimple
- govet
- ineffassign
- misspell
- prealloc
- staticcheck
- structcheck
- typecheck
- unconvert
- unused
- varcheck
# these are implicitly disabled:
# - asciicheck
# - depguard
# - dogsled
# - dupl
# - exhaustive
# - exportloopref
# - funlen
# - gci
# - gochecknoglobals
# - gochecknoinits
# - gocognit
# - goconst
# - gocritic
# - gocyclo
# - godot
# - godox
# - goerr113
# - gofumpt
# - goheader
# - golint
# - gomnd
# - gomodguard
# - goprintffuncname
# - interfacer
# - lll
# - maligned
# - nakedret
# - nestif
# - nlreturn
# - noctx
# - nolintlint
# - rowserrcheck
# - scopelint
# - sqlclosecheck
# - stylecheck
# - testpackage
# - unparam
# - whitespace
# - wsl
run:
# default concurrency is a available CPU number.
+68 -13
View File
@@ -1,14 +1,25 @@
before:
hooks:
# The build is done in this particular way to build Caddy in a designated directory named in .gitignore.
# This is so we can run goreleaser on tag without Git complaining of being dirty. The main.go in cmd/caddy directory
# cannot be built within that directory due to changes necessary for the build causing Git to be dirty, which
# subsequently causes gorleaser to refuse running.
- rm -rf caddy-build caddy-dist
- mkdir -p caddy-build
- cp cmd/caddy/main.go caddy-build/main.go
- cp ./go.mod caddy-build/go.mod
- sed -i.bkp 's|github.com/caddyserver/caddy/v2|caddy|g' ./caddy-build/go.mod
- /bin/sh -c 'cd ./caddy-build && go mod init caddy'
# GoReleaser doesn't seem to offer {{.Tag}} at this stage, so we have to embed it into the env
# so we run: TAG=$(git describe --abbrev=0) goreleaser release --rm-dist --skip-publish --skip-validate
- go mod edit -require=github.com/caddyserver/caddy/v2@{{.Env.TAG}} ./caddy-build/go.mod
# as of Go 1.16, `go` commands no longer automatically change go.{mod,sum}. We now have to explicitly
# run `go mod tidy`. The `/bin/sh -c '...'` is because goreleaser can't find cd in PATH without shell invocation.
- /bin/sh -c 'cd ./caddy-build && go mod tidy'
- git clone --depth 1 https://github.com/caddyserver/dist caddy-dist
- mkdir -p caddy-dist/man
- go mod download
- go run cmd/caddy/main.go manpage --directory ./caddy-dist/man
- gzip -r ./caddy-dist/man/
- /bin/sh -c 'go run cmd/caddy/main.go completion bash > ./caddy-dist/scripts/bash-completion'
builds:
- env:
@@ -26,17 +37,47 @@ builds:
- amd64
- arm
- arm64
- s390x
- ppc64le
goarm:
- 6
- 7
- "5"
- "6"
- "7"
ignore:
- goos: darwin
goarch: arm
- goos: darwin
goarch: ppc64le
- goos: darwin
goarch: s390x
- goos: windows
goarch: ppc64le
- goos: windows
goarch: s390x
- goos: freebsd
goarch: ppc64le
- goos: freebsd
goarch: s390x
- goos: freebsd
goarch: arm
goarm: "5"
flags:
- -trimpath
- -mod=readonly
ldflags:
- -s -w
signs:
- cmd: cosign
signature: "${artifact}.sig"
certificate: '{{ trimsuffix (trimsuffix .Env.artifact ".zip") ".tar.gz" }}.pem'
args: ["sign-blob", "--output-signature=${signature}", "--output-certificate", "${certificate}", "${artifact}"]
artifacts: all
sboms:
- artifacts: binary
documents:
- '{{ .Binary }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{if .Arm}}v{{ .Arm }}{{end}}.sbom'
cmd: syft
args: ["$artifact", "--file", "${document}", "--output", "cyclonedx-json"]
archives:
- format_overrides:
- goos: windows
@@ -50,11 +91,11 @@ nfpms:
- id: default
package_name: caddy
vendor: Light Code Labs
vendor: Dyanim
homepage: https://caddyserver.com
maintainer: Matthew Holt <mholt@users.noreply.github.com>
description: |
Powerful, enterprise-ready, open source web server with automatic HTTPS written in Go
Caddy - Powerful, enterprise-ready, open source web server with automatic HTTPS written in Go
license: Apache 2.0
formats:
@@ -62,12 +103,25 @@ nfpms:
# - rpm
bindir: /usr/bin
files:
./caddy-dist/init/caddy.service: /lib/systemd/system/caddy.service
./caddy-dist/init/caddy-api.service: /lib/systemd/system/caddy-api.service
./caddy-dist/welcome/index.html: /usr/share/caddy/index.html
config_files:
./caddy-dist/config/Caddyfile: /etc/caddy/Caddyfile
contents:
- src: ./caddy-dist/init/caddy.service
dst: /lib/systemd/system/caddy.service
- src: ./caddy-dist/init/caddy-api.service
dst: /lib/systemd/system/caddy-api.service
- src: ./caddy-dist/welcome/index.html
dst: /usr/share/caddy/index.html
- src: ./caddy-dist/scripts/bash-completion
dst: /etc/bash_completion.d/caddy
- src: ./caddy-dist/config/Caddyfile
dst: /etc/caddy/Caddyfile
type: config
- src: ./caddy-dist/man/*
dst: /usr/share/man/man8/
scripts:
postinstall: ./caddy-dist/scripts/postinstall.sh
@@ -89,5 +143,6 @@ changelog:
- '^chore:'
- '^ci:'
- '^docs?:'
- '^readme:'
- '^tests?:'
- '^\w+\s+' # a hack to remove commit messages without colons thus don't correspond to a package
-24
View File
@@ -1,24 +0,0 @@
branches:
only:
- master
language: go
go:
- 1.14.x
dist: bionic
arch:
- ppc64le
- s390x
env:
- GO111MODULE=on CGO_ENABLED=0
cache:
directories:
- $GOPATH/pkg/mod
- /home/travis/.cache/go-build
git:
depth: 1
install:
- go get -v -t -d ./...
script:
- go test -v -short ./...
- go build -trimpath -ldflags="-w -s" -v -o cmd/caddy/caddy cmd/caddy/main.go
+56 -20
View File
@@ -1,21 +1,25 @@
<p align="center">
<a href="https://caddyserver.com"><img src="https://user-images.githubusercontent.com/1128849/36338535-05fb646a-136f-11e8-987b-e6901e717d5a.png" alt="Caddy" width="450"></a>
<br>
<h3 align="center">a <a href="https://zerossl.com"><img src="https://caddyserver.com/resources/images/zerossl-logo.svg" height="28" valign="middle"></a> project</h3>
</p>
<hr>
<h3 align="center">Every site on HTTPS</h3>
<p align="center">Caddy is an extensible server platform that uses TLS by default.</p>
<p align="center">
<a href="https://github.com/caddyserver/caddy/actions?query=workflow%3ACross-Platform"><img src="https://github.com/caddyserver/caddy/workflows/Cross-Platform/badge.svg"></a>
<a href="https://pkg.go.dev/github.com/caddyserver/caddy/v2"><img src="https://img.shields.io/badge/godoc-reference-blue.svg"></a>
<a href="https://app.fuzzit.dev/orgs/caddyserver-gh/dashboard"><img src="https://app.fuzzit.dev/badge?org_id=caddyserver-gh"></a>
<a href="https://pkg.go.dev/github.com/caddyserver/caddy/v2"><img src="https://img.shields.io/badge/godoc-reference-%23007d9c.svg"></a>
<br>
<a href="https://twitter.com/caddyserver" title="@caddyserver on Twitter"><img src="https://img.shields.io/badge/twitter-@caddyserver-55acee.svg" alt="@caddyserver on Twitter"></a>
<a href="https://caddy.community" title="Caddy Forum"><img src="https://img.shields.io/badge/community-forum-ff69b4.svg" alt="Caddy Forum"></a>
<br>
<a href="https://sourcegraph.com/github.com/caddyserver/caddy?badge" title="Caddy on Sourcegraph"><img src="https://sourcegraph.com/github.com/caddyserver/caddy/-/badge.svg" alt="Caddy on Sourcegraph"></a>
<a href="https://cloudsmith.io/~caddy/repos/"><img src="https://img.shields.io/badge/OSS%20hosting%20by-cloudsmith-blue?logo=cloudsmith" alt="Cloudsmith"></a>
</p>
<p align="center">
<a href="https://github.com/caddyserver/caddy/releases">Download</a> ·
<a href="https://github.com/caddyserver/caddy/releases">Releases</a> ·
<a href="https://caddyserver.com/docs/">Documentation</a> ·
<a href="https://caddy.community">Community</a>
<a href="https://caddy.community">Get Help</a>
</p>
@@ -23,10 +27,11 @@
### Menu
- [Features](#features)
- [Install](#install)
- [Build from source](#build-from-source)
- [For development](#for-development)
- [With version information and/or plugins](#with-version-information-andor-plugins)
- [Getting started](#getting-started)
- [Quick start](#quick-start)
- [Overview](#overview)
- [Full documentation](#full-documentation)
- [Getting help](#getting-help)
@@ -39,49 +44,71 @@
</p>
## Features
## [Features](https://caddyserver.com/v2)
- **Easy configuration** with the [Caddyfile](https://caddyserver.com/docs/caddyfile)
- **Powerful configuration** with its [native JSON config](https://caddyserver.com/docs/json/)
- **Dynamic configuration** with the [JSON API](https://caddyserver.com/docs/api)
- [**Config adapters**](https://caddyserver.com/docs/config-adapters) if you don't like JSON
- **Automatic HTTPS** by default
- [Let's Encrypt](https://letsencrypt.org) for public sites
- [ZeroSSL](https://zerossl.com) and [Let's Encrypt](https://letsencrypt.org) for public names
- Fully-managed local CA for internal names & IPs
- Can coordinate with other Caddy instances in a cluster
- Multi-issuer fallback
- **Stays up when other servers go down** due to TLS/OCSP/certificate-related issues
- **HTTP/1.1, HTTP/2, and experimental HTTP/3** support
- **Production-ready** after serving trillions of requests and managing millions of TLS certificates
- **Scales to hundreds of thousands of sites** as proven in production
- **HTTP/1.1, HTTP/2, and HTTP/3** supported all by default
- **Highly extensible** [modular architecture](https://caddyserver.com/docs/architecture) lets Caddy do anything without bloat
- **Runs anywhere** with **no external dependencies** (not even libc)
- Written in Go, a language with higher **memory safety guarantees** than other servers
- Actually **fun to use**
- So, so much more to discover
- So much more to [discover](https://caddyserver.com/v2)
## Install
The simplest, cross-platform way to get started is to download Caddy from [GitHub Releases](https://github.com/caddyserver/caddy/releases) and place the executable file in your PATH.
See [our online documentation](https://caddyserver.com/docs/install) for other install instructions.
## Build from source
Requirements:
- [Go 1.14 or newer](https://golang.org/dl/)
- Do NOT disable [Go modules](https://github.com/golang/go/wiki/Modules) (`export GO111MODULE=on`)
- [Go 1.18 or newer](https://golang.org/dl/)
### For development
_**Note:** These steps [will not embed proper version information](https://github.com/golang/go/issues/29228). For that, please follow the instructions in the next section._
```bash
$ git clone "https://github.com/caddyserver/caddy.git"
$ cd caddy/cmd/caddy/
$ go build
```
_**Note:** These steps [will not embed proper version information](https://github.com/golang/go/issues/29228). For that, please follow the instructions below._
When you run Caddy, it may try to bind to low ports unless otherwise specified in your config. If your OS requires elevated privileges for this, you will need to give your new binary permission to do so. On Linux, this can be done easily with: `sudo setcap cap_net_bind_service=+ep ./caddy`
If you prefer to use `go run` which only creates temporary binaries, you can still do this with the included `setcap.sh` like so:
```bash
$ go run -exec ./setcap.sh main.go
```
If you don't want to type your password for `setcap`, use `sudo visudo` to edit your sudoers file and allow your user account to run that command without a password, for example:
```
username ALL=(ALL:ALL) NOPASSWD: /usr/sbin/setcap
```
replacing `username` with your actual username. Please be careful and only do this if you know what you are doing! We are only qualified to document how to use Caddy, not Go tooling or your computer, and we are providing these instructions for convenience only; please learn how to use your own computer at your own risk and make any needful adjustments.
### With version information and/or plugins
Using [our builder tool](https://github.com/caddyserver/xcaddy)...
Using [our builder tool, `xcaddy`](https://github.com/caddyserver/xcaddy)...
```
$ xcaddy build <caddy_version>
$ xcaddy build
```
...the following steps are automated:
@@ -90,8 +117,9 @@ $ xcaddy build <caddy_version>
2. Change into it: `cd caddy`
3. Copy [Caddy's main.go](https://github.com/caddyserver/caddy/blob/master/cmd/caddy/main.go) into the empty folder. Add imports for any custom plugins you want to add.
4. Initialize a Go module: `go mod init caddy`
5. Pin Caddy version: `go get github.com/caddyserver/caddy/v2@TAG` replacing `TAG` with a git tag or commit. You can also pin any plugin versions similarly.
6. Compile: `go build`
5. (Optional) Pin Caddy version: `go get github.com/caddyserver/caddy/v2@version` replacing `version` with a git tag, commit, or branch name.
6. (Optional) Add plugins by adding their import: `_ "import/path/here"`
7. Compile: `go build`
@@ -100,7 +128,7 @@ $ xcaddy build <caddy_version>
The [Caddy website](https://caddyserver.com/docs/) has documentation that includes tutorials, quick-start guides, reference, and more.
**We recommend that all users do our [Getting Started](https://caddyserver.com/docs/getting-started) guide to become familiar with using Caddy.**
**We recommend that all users -- regardless of experience level -- do our [Getting Started](https://caddyserver.com/docs/getting-started) guide to become familiar with using Caddy.**
If you've only got a minute, [the website has several quick-start tutorials](https://caddyserver.com/docs/quick-starts) to choose from! However, after finishing a quick-start tutorial, please read more documentation to understand how the software works. 🙂
@@ -119,7 +147,7 @@ The primary way to configure Caddy is through [its API](https://caddyserver.com/
Caddy exposes an unprecedented level of control compared to any web server in existence. In Caddy, you are usually setting the actual values of the initialized types in memory that power everything from your HTTP handlers and TLS handshakes to your storage medium. Caddy is also ridiculously extensible, with a powerful plugin system that makes vast improvements over other web servers.
To wield the power of this design, you need to know how the config document is structured. Please see the [our documentation site](https://caddyserver.com/docs/) for details about [Caddy's config structure](https://caddyserver.com/docs/json/).
To wield the power of this design, you need to know how the config document is structured. Please see [our documentation site](https://caddyserver.com/docs/) for details about [Caddy's config structure](https://caddyserver.com/docs/json/).
Nearly all of Caddy's configuration is contained in a single config document, rather than being scattered across CLI flags and env variables and a configuration file as with other web servers. This makes managing your server config more straightforward and reduces hidden variables/factors.
@@ -136,7 +164,9 @@ The docs are also open source. You can contribute to them here: https://github.c
## Getting help
- We **strongly recommend** that all professionals or companies using Caddy get a support contract through [Ardan Labs](https://www.ardanlabs.com/my/contact-us?dd=caddy) before help is needed.
- We advise companies using Caddy to secure a support contract through [Ardan Labs](https://www.ardanlabs.com/my/contact-us?dd=caddy) before help is needed.
- A [sponsorship](https://github.com/sponsors/mholt) goes a long way! We can offer private help to sponsors. If Caddy is benefitting your company, please consider a sponsorship. This not only helps fund full-time work to ensure the longevity of the project, it provides your company the resources, support, and discounts you need; along with being a great look for your company to your customers and potential customers!
- Individuals can exchange help for free on our community forum at https://caddy.community. Remember that people give help out of their spare time and good will. The best way to get help is to give it first!
@@ -146,7 +176,13 @@ Please use our [issue tracker](https://github.com/caddyserver/caddy/issues) only
## About
**The name "Caddy" is trademarked.** The name of the software is "Caddy", not "Caddy Server" or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the Caddy web server". Caddy is a registered trademark of Light Code Labs, LLC.
Matthew Holt began developing Caddy in 2014 while studying computer science at Brigham Young University. (The name "Caddy" was chosen because this software helps with the tedious, mundane tasks of serving the Web, and is also a single place for multiple things to be organized together.) It soon became the first web server to use HTTPS automatically and by default, and now has hundreds of contributors and has served trillions of HTTPS requests.
**The name "Caddy" is trademarked.** The name of the software is "Caddy", not "Caddy Server" or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the Caddy web server". Caddy is a registered trademark of Stack Holdings GmbH.
- _Project on Twitter: [@caddyserver](https://twitter.com/caddyserver)_
- _Author on Twitter: [@mholt6](https://twitter.com/mholt6)_
Caddy is a project of [ZeroSSL](https://zerossl.com), a Stack Holdings company.
Debian package repository hosting is graciously provided by [Cloudsmith](https://cloudsmith.com). Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence.
+713 -188
View File
File diff suppressed because it is too large Load Diff
+88 -21
View File
@@ -16,10 +16,31 @@ package caddy
import (
"encoding/json"
"fmt"
"net/http"
"reflect"
"sync"
"testing"
)
var testCfg = []byte(`{
"apps": {
"http": {
"servers": {
"myserver": {
"listen": ["tcp/localhost:8080-8084"],
"read_timeout": "30s"
},
"yourserver": {
"listen": ["127.0.0.1:5000"],
"read_header_timeout": "15s"
}
}
}
}
}
`)
func TestUnsyncedConfigAccess(t *testing.T) {
// each test is performed in sequence, so
// each change builds on the previous ones;
@@ -94,7 +115,7 @@ func TestUnsyncedConfigAccess(t *testing.T) {
}
// decode the expected config so we can do a convenient DeepEqual
var expectedDecoded interface{}
var expectedDecoded any
err = json.Unmarshal([]byte(tc.expect), &expectedDecoded)
if err != nil {
t.Fatalf("Test %d: Unmarshaling expected config: %v", i, err)
@@ -108,25 +129,71 @@ func TestUnsyncedConfigAccess(t *testing.T) {
}
}
func BenchmarkLoad(b *testing.B) {
for i := 0; i < b.N; i++ {
cfg := []byte(`{
"apps": {
"http": {
"servers": {
"myserver": {
"listen": ["tcp/localhost:8080-8084"],
"read_timeout": "30s"
},
"yourserver": {
"listen": ["127.0.0.1:5000"],
"read_header_timeout": "15s"
}
}
}
}
}
`)
Load(cfg, true)
// TestLoadConcurrent exercises Load under concurrent conditions
// and is most useful under test with `-race` enabled.
func TestLoadConcurrent(t *testing.T) {
var wg sync.WaitGroup
for i := 0; i < 100; i++ {
wg.Add(1)
go func() {
_ = Load(testCfg, true)
wg.Done()
}()
}
wg.Wait()
}
type fooModule struct {
IntField int
StrField string
}
func (fooModule) CaddyModule() ModuleInfo {
return ModuleInfo{
ID: "foo",
New: func() Module { return new(fooModule) },
}
}
func (fooModule) Start() error { return nil }
func (fooModule) Stop() error { return nil }
func TestETags(t *testing.T) {
RegisterModule(fooModule{})
if err := Load([]byte(`{"admin": {"listen": "localhost:2999"}, "apps": {"foo": {"strField": "abc", "intField": 0}}}`), true); err != nil {
t.Fatalf("loading: %s", err)
}
const key = "/" + rawConfigKey + "/apps/foo"
// try update the config with the wrong etag
err := changeConfig(http.MethodPost, key, []byte(`{"strField": "abc", "intField": 1}}`), fmt.Sprintf(`"/%s not_an_etag"`, rawConfigKey), false)
if apiErr, ok := err.(APIError); !ok || apiErr.HTTPStatus != http.StatusPreconditionFailed {
t.Fatalf("expected precondition failed; got %v", err)
}
// get the etag
hash := etagHasher()
if err := readConfig(key, hash); err != nil {
t.Fatalf("reading: %s", err)
}
// do the same update with the correct key
err = changeConfig(http.MethodPost, key, []byte(`{"strField": "abc", "intField": 1}`), makeEtag(key, hash), false)
if err != nil {
t.Fatalf("expected update to work; got %v", err)
}
// now try another update. The hash should no longer match and we should get precondition failed
err = changeConfig(http.MethodPost, key, []byte(`{"strField": "abc", "intField": 2}`), makeEtag(key, hash), false)
if apiErr, ok := err.(APIError); !ok || apiErr.HTTPStatus != http.StatusPreconditionFailed {
t.Fatalf("expected precondition failed; got %v", err)
}
}
func BenchmarkLoad(b *testing.B) {
for i := 0; i < b.N; i++ {
Load(testCfg, true)
}
}
+527 -99
View File
@@ -17,10 +17,11 @@ package caddy
import (
"bytes"
"context"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
@@ -30,9 +31,12 @@ import (
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/caddyserver/caddy/v2/notify"
"github.com/caddyserver/certmagic"
"github.com/google/uuid"
"go.uber.org/zap"
)
@@ -99,16 +103,50 @@ func Run(cfg *Config) error {
// if it is different from the current config or
// forceReload is true.
func Load(cfgJSON []byte, forceReload bool) error {
return changeConfig(http.MethodPost, "/"+rawConfigKey, cfgJSON, forceReload)
if err := notify.Reloading(); err != nil {
Log().Error("unable to notify service manager of reloading state", zap.Error(err))
}
// after reload, notify system of success or, if
// failure, update with status (error message)
var err error
defer func() {
if err != nil {
if notifyErr := notify.Error(err, 0); notifyErr != nil {
Log().Error("unable to notify to service manager of reload error",
zap.Error(notifyErr),
zap.String("reload_err", err.Error()))
}
return
}
if err := notify.Ready(); err != nil {
Log().Error("unable to notify to service manager of ready state", zap.Error(err))
}
}()
err = changeConfig(http.MethodPost, "/"+rawConfigKey, cfgJSON, "", forceReload)
if errors.Is(err, errSameConfig) {
err = nil // not really an error
}
return err
}
// changeConfig changes the current config (rawCfg) according to the
// method, traversed via the given path, and uses the given input as
// the new value (if applicable; i.e. "DELETE" doesn't have an input).
// If the resulting config is the same as the previous, no reload will
// occur unless forceReload is true. This function is safe for
// occur unless forceReload is true. If the config is unchanged and not
// forcefully reloaded, then errConfigUnchanged This function is safe for
// concurrent use.
func changeConfig(method, path string, input []byte, forceReload bool) error {
// The ifMatchHeader can optionally be given a string of the format:
//
// "<path> <hash>"
//
// where <path> is the absolute path in the config and <hash> is the expected hash of
// the config at that path. If the hash in the ifMatchHeader doesn't match
// the hash of the config, then an APIError with status 412 will be returned.
func changeConfig(method, path string, input []byte, ifMatchHeader string, forceReload bool) error {
switch method {
case http.MethodGet,
http.MethodHead,
@@ -118,8 +156,42 @@ func changeConfig(method, path string, input []byte, forceReload bool) error {
return fmt.Errorf("method not allowed")
}
currentCfgMu.Lock()
defer currentCfgMu.Unlock()
currentCtxMu.Lock()
defer currentCtxMu.Unlock()
if ifMatchHeader != "" {
// expect the first and last character to be quotes
if len(ifMatchHeader) < 2 || ifMatchHeader[0] != '"' || ifMatchHeader[len(ifMatchHeader)-1] != '"' {
return APIError{
HTTPStatus: http.StatusBadRequest,
Err: fmt.Errorf("malformed If-Match header; expect quoted string"),
}
}
// read out the parts
parts := strings.Fields(ifMatchHeader[1 : len(ifMatchHeader)-1])
if len(parts) != 2 {
return APIError{
HTTPStatus: http.StatusBadRequest,
Err: fmt.Errorf("malformed If-Match header; expect format \"<path> <hash>\""),
}
}
// get the current hash of the config
// at the given path
hash := etagHasher()
err := unsyncedConfigAccess(http.MethodGet, parts[0], nil, hash)
if err != nil {
return err
}
if hex.EncodeToString(hash.Sum(nil)) != parts[1] {
return APIError{
HTTPStatus: http.StatusPreconditionFailed,
Err: fmt.Errorf("If-Match header did not match current config hash"),
}
}
}
err := unsyncedConfigAccess(method, path, input, nil)
if err != nil {
@@ -130,15 +202,15 @@ func changeConfig(method, path string, input []byte, forceReload bool) error {
newCfg, err := json.Marshal(rawCfg[rawConfigKey])
if err != nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("encoding new config: %v", err),
HTTPStatus: http.StatusBadRequest,
Err: fmt.Errorf("encoding new config: %v", err),
}
}
// if nothing changed, no need to do a whole reload unless the client forces it
if !forceReload && bytes.Equal(rawCfgJSON, newCfg) {
Log().Named("admin.api").Info("config is unchanged")
return nil
Log().Info("config is unchanged")
return errSameConfig
}
// find any IDs in this config and index them
@@ -146,21 +218,21 @@ func changeConfig(method, path string, input []byte, forceReload bool) error {
err = indexConfigObjects(rawCfg[rawConfigKey], "/"+rawConfigKey, idx)
if err != nil {
return APIError{
Code: http.StatusInternalServerError,
Err: fmt.Errorf("indexing config: %v", err),
HTTPStatus: http.StatusInternalServerError,
Err: fmt.Errorf("indexing config: %v", err),
}
}
// load this new config; if it fails, we need to revert to
// our old representation of caddy's actual config
err = unsyncedDecodeAndRun(newCfg)
err = unsyncedDecodeAndRun(newCfg, true)
if err != nil {
if len(rawCfgJSON) > 0 {
// restore old config state to keep it consistent
// with what caddy is still running; we need to
// unmarshal it again because it's likely that
// pointers deep in our rawCfg map were modified
var oldCfg interface{}
var oldCfg any
err2 := json.Unmarshal(rawCfgJSON, &oldCfg)
if err2 != nil {
err = fmt.Errorf("%v; additionally, restoring old config: %v", err, err2)
@@ -185,18 +257,18 @@ func changeConfig(method, path string, input []byte, forceReload bool) error {
// readConfig traverses the current config to path
// and writes its JSON encoding to out.
func readConfig(path string, out io.Writer) error {
currentCfgMu.RLock()
defer currentCfgMu.RUnlock()
currentCtxMu.RLock()
defer currentCtxMu.RUnlock()
return unsyncedConfigAccess(http.MethodGet, path, nil, out)
}
// indexConfigObjects recursively searches ptr for object fields named
// "@id" and maps that ID value to the full configPath in the index.
// This function is NOT safe for concurrent access; obtain a write lock
// on currentCfgMu.
func indexConfigObjects(ptr interface{}, configPath string, index map[string]string) error {
// on currentCtxMu.
func indexConfigObjects(ptr any, configPath string, index map[string]string) error {
switch val := ptr.(type) {
case map[string]interface{}:
case map[string]any:
for k, v := range val {
if k == idKey {
switch idVal := v.(type) {
@@ -215,7 +287,7 @@ func indexConfigObjects(ptr interface{}, configPath string, index map[string]str
return err
}
}
case []interface{}:
case []any:
// traverse each element of the array recursively
for i := range val {
err := indexConfigObjects(val[i], path.Join(configPath, strconv.Itoa(i)), index)
@@ -233,8 +305,10 @@ func indexConfigObjects(ptr interface{}, configPath string, index map[string]str
// it as the new config, replacing any other current config.
// It does NOT update the raw config state, as this is a
// lower-level function; most callers will want to use Load
// instead. A write lock on currentCfgMu is required!
func unsyncedDecodeAndRun(cfgJSON []byte) error {
// instead. A write lock on currentCtxMu is required! If
// allowPersist is false, it will not be persisted to disk,
// even if it is configured to.
func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error {
// remove any @id fields from the JSON, which would cause
// loading to break since the field wouldn't be recognized
strippedCfgJSON := RemoveMetaFields(cfgJSON)
@@ -245,21 +319,36 @@ func unsyncedDecodeAndRun(cfgJSON []byte) error {
return err
}
// prevent recursive config loads; that is a user error, and
// although frequent config loads should be safe, we cannot
// guarantee that in the presence of third party plugins, nor
// do we want this error to go unnoticed (we assume it was a
// pulled config if we're not allowed to persist it)
if !allowPersist &&
newCfg != nil &&
newCfg.Admin != nil &&
newCfg.Admin.Config != nil &&
newCfg.Admin.Config.LoadRaw != nil &&
newCfg.Admin.Config.LoadDelay <= 0 {
return fmt.Errorf("recursive config loading detected: pulled configs cannot pull other configs without positive load_delay")
}
// run the new config and start all its apps
err = run(newCfg, true)
ctx, err := run(newCfg, true)
if err != nil {
return err
}
// swap old config with the new one
oldCfg := currentCfg
currentCfg = newCfg
// swap old context (including its config) with the new one
oldCtx := currentCtx
currentCtx = ctx
// Stop, Cleanup each old app
unsyncedStop(oldCfg)
unsyncedStop(oldCtx)
// autosave a non-nil config, if not disabled
if newCfg != nil &&
if allowPersist &&
newCfg != nil &&
(newCfg.Admin == nil ||
newCfg.Admin.Config == nil ||
newCfg.Admin.Config.Persist == nil ||
@@ -271,9 +360,9 @@ func unsyncedDecodeAndRun(cfgJSON []byte) error {
zap.String("dir", dir),
zap.Error(err))
} else {
err := ioutil.WriteFile(ConfigAutosavePath, cfgJSON, 0600)
err := os.WriteFile(ConfigAutosavePath, cfgJSON, 0600)
if err == nil {
Log().Info("autosaved config", zap.String("file", ConfigAutosavePath))
Log().Info("autosaved config (load with --resume flag)", zap.String("file", ConfigAutosavePath))
} else {
Log().Error("unable to autosave config",
zap.String("file", ConfigAutosavePath),
@@ -299,7 +388,7 @@ func unsyncedDecodeAndRun(cfgJSON []byte) error {
// This is a low-level function; most callers
// will want to use Run instead, which also
// updates the config's raw state.
func run(newCfg *Config, start bool) error {
func run(newCfg *Config, start bool) (Context, error) {
// because we will need to roll back any state
// modifications if this function errors, we
// keep a single error value and scope all
@@ -309,21 +398,10 @@ func run(newCfg *Config, start bool) error {
// been set by a short assignment
var err error
// start the admin endpoint (and stop any prior one)
if start {
err = replaceAdmin(newCfg)
if err != nil {
return fmt.Errorf("starting caddy administration endpoint: %v", err)
}
}
if newCfg == nil {
return nil
newCfg = new(Config)
}
// prepare the new config for use
newCfg.apps = make(map[string]App)
// create a context within which to load
// modules - essentially our new config's
// execution environment; be sure that
@@ -341,8 +419,8 @@ func run(newCfg *Config, start bool) error {
cancel()
// also undo any other state changes we made
if currentCfg != nil {
certmagic.Default.Storage = currentCfg.storage
if currentCtx.cfg != nil {
certmagic.Default.Storage = currentCtx.cfg.storage
}
}
}()
@@ -354,9 +432,20 @@ func run(newCfg *Config, start bool) error {
}
err = newCfg.Logging.openLogs(ctx)
if err != nil {
return err
return ctx, err
}
// start the admin endpoint (and stop any prior one)
if start {
err = replaceLocalAdminServer(newCfg)
if err != nil {
return ctx, fmt.Errorf("starting caddy administration endpoint: %v", err)
}
}
// prepare the new config for use
newCfg.apps = make(map[string]App)
// set up global storage and make it CertMagic's default storage, too
err = func() error {
if newCfg.StorageRaw != nil {
@@ -379,7 +468,7 @@ func run(newCfg *Config, start bool) error {
return nil
}()
if err != nil {
return err
return ctx, err
}
// Load and Provision each app and their submodules
@@ -392,16 +481,23 @@ func run(newCfg *Config, start bool) error {
return nil
}()
if err != nil {
return err
return ctx, err
}
if !start {
return nil
return ctx, nil
}
// Provision any admin routers which may need to access
// some of the other apps at runtime
err = newCfg.Admin.provisionAdminRouters(ctx)
if err != nil {
return ctx, err
}
// Start
return func() error {
var started []string
err = func() error {
started := make([]string, 0, len(newCfg.apps))
for name, a := range newCfg.apps {
err := a.Start()
if err != nil {
@@ -420,6 +516,108 @@ func run(newCfg *Config, start bool) error {
}
return nil
}()
if err != nil {
return ctx, err
}
// now that the user's config is running, finish setting up anything else,
// such as remote admin endpoint, config loader, etc.
return ctx, finishSettingUp(ctx, newCfg)
}
// finishSettingUp should be run after all apps have successfully started.
func finishSettingUp(ctx Context, cfg *Config) error {
// establish this server's identity (only after apps are loaded
// so that cert management of this endpoint doesn't prevent user's
// servers from starting which likely also use HTTP/HTTPS ports;
// but before remote management which may depend on these creds)
err := manageIdentity(ctx, cfg)
if err != nil {
return fmt.Errorf("provisioning remote admin endpoint: %v", err)
}
// replace any remote admin endpoint
err = replaceRemoteAdminServer(ctx, cfg)
if err != nil {
return fmt.Errorf("provisioning remote admin endpoint: %v", err)
}
// if dynamic config is requested, set that up and run it
if cfg != nil && cfg.Admin != nil && cfg.Admin.Config != nil && cfg.Admin.Config.LoadRaw != nil {
val, err := ctx.LoadModule(cfg.Admin.Config, "LoadRaw")
if err != nil {
return fmt.Errorf("loading config loader module: %s", err)
}
logger := Log().Named("config_loader").With(
zap.String("module", val.(Module).CaddyModule().ID.Name()),
zap.Int("load_delay", int(cfg.Admin.Config.LoadDelay)))
runLoadedConfig := func(config []byte) error {
logger.Info("applying dynamically-loaded config")
err := changeConfig(http.MethodPost, "/"+rawConfigKey, config, "", false)
if errors.Is(err, errSameConfig) {
return err
}
if err != nil {
logger.Error("failed to run dynamically-loaded config", zap.Error(err))
return err
}
logger.Info("successfully applied dynamically-loaded config")
return nil
}
if cfg.Admin.Config.LoadDelay > 0 {
go func() {
// the loop is here to iterate ONLY if there is an error, a no-op config load,
// or an unchanged config; in which case we simply wait the delay and try again
for {
timer := time.NewTimer(time.Duration(cfg.Admin.Config.LoadDelay))
select {
case <-timer.C:
loadedConfig, err := val.(ConfigLoader).LoadConfig(ctx)
if err != nil {
logger.Error("failed loading dynamic config; will retry", zap.Error(err))
continue
}
if loadedConfig == nil {
logger.Info("dynamically-loaded config was nil; will retry")
continue
}
err = runLoadedConfig(loadedConfig)
if errors.Is(err, errSameConfig) {
logger.Info("dynamically-loaded config was unchanged; will retry")
continue
}
case <-ctx.Done():
if !timer.Stop() {
<-timer.C
}
logger.Info("stopping dynamic config loading")
}
break
}
}()
} else {
// if no LoadDelay is provided, will load config synchronously
loadedConfig, err := val.(ConfigLoader).LoadConfig(ctx)
if err != nil {
return fmt.Errorf("loading dynamic config from %T: %v", val, err)
}
// do this in a goroutine so current config can finish being loaded; otherwise deadlock
go func() { _ = runLoadedConfig(loadedConfig) }()
}
}
return nil
}
// ConfigLoader is a type that can load a Caddy config. If
// the return value is non-nil, it must be valid Caddy JSON;
// if nil or with non-nil error, it is considered to be a
// no-op load and may be retried later.
type ConfigLoader interface {
LoadConfig(Context) ([]byte, error)
}
// Stop stops running the current configuration.
@@ -429,10 +627,10 @@ func run(newCfg *Config, start bool) error {
// stop the others. Stop should only be called
// if not replacing with a new config.
func Stop() error {
currentCfgMu.Lock()
defer currentCfgMu.Unlock()
unsyncedStop(currentCfg)
currentCfg = nil
currentCtxMu.Lock()
defer currentCtxMu.Unlock()
unsyncedStop(currentCtx)
currentCtx = Context{}
rawCfgJSON = nil
rawCfgIndex = nil
rawCfg[rawConfigKey] = nil
@@ -445,13 +643,13 @@ func Stop() error {
// it is logged and the function continues stopping
// the next app. This function assumes all apps in
// cfg were successfully started first.
func unsyncedStop(cfg *Config) {
if cfg == nil {
func unsyncedStop(ctx Context) {
if ctx.cfg == nil {
return
}
// stop each app
for name, a := range cfg.apps {
for name, a := range ctx.cfg.apps {
err := a.Stop()
if err != nil {
log.Printf("[ERROR] stop %s: %v", name, err)
@@ -459,34 +657,103 @@ func unsyncedStop(cfg *Config) {
}
// clean up all modules
cfg.cancelFunc()
}
// stopAndCleanup calls stop and cleans up anything
// else that is expedient. This should only be used
// when stopping and not replacing with a new config.
func stopAndCleanup() error {
if err := Stop(); err != nil {
return err
}
certmagic.CleanUpOwnLocks()
return nil
ctx.cfg.cancelFunc()
}
// Validate loads, provisions, and validates
// cfg, but does not start running it.
func Validate(cfg *Config) error {
err := run(cfg, false)
_, err := run(cfg, false)
if err == nil {
cfg.cancelFunc() // call Cleanup on all modules
}
return err
}
// exitProcess exits the process as gracefully as possible,
// but it always exits, even if there are errors doing so.
// It stops all apps, cleans up external locks, removes any
// PID file, and shuts down admin endpoint(s) in a goroutine.
// Errors are logged along the way, and an appropriate exit
// code is emitted.
func exitProcess(ctx context.Context, logger *zap.Logger) {
// let the rest of the program know we're quitting
atomic.StoreInt32(exiting, 1)
// give the OS or service/process manager our 2 weeks' notice: we quit
if err := notify.Stopping(); err != nil {
Log().Error("unable to notify service manager of stopping state", zap.Error(err))
}
if logger == nil {
logger = Log()
}
logger.Warn("exiting; byeee!! 👋")
exitCode := ExitCodeSuccess
// stop all apps
if err := Stop(); err != nil {
logger.Error("failed to stop apps", zap.Error(err))
exitCode = ExitCodeFailedQuit
}
// clean up certmagic locks
certmagic.CleanUpOwnLocks(ctx, logger)
// remove pidfile
if pidfile != "" {
err := os.Remove(pidfile)
if err != nil {
logger.Error("cleaning up PID file:",
zap.String("pidfile", pidfile),
zap.Error(err))
exitCode = ExitCodeFailedQuit
}
}
// shut down admin endpoint(s) in goroutines so that
// if this function was called from an admin handler,
// it has a chance to return gracefully
// use goroutine so that we can finish responding to API request
go func() {
defer func() {
logger = logger.With(zap.Int("exit_code", exitCode))
if exitCode == ExitCodeSuccess {
logger.Info("shutdown complete")
} else {
logger.Error("unclean shutdown")
}
os.Exit(exitCode)
}()
if remoteAdminServer != nil {
err := stopAdminServer(remoteAdminServer)
if err != nil {
exitCode = ExitCodeFailedQuit
logger.Error("failed to stop remote admin server gracefully", zap.Error(err))
}
}
if localAdminServer != nil {
err := stopAdminServer(localAdminServer)
if err != nil {
exitCode = ExitCodeFailedQuit
logger.Error("failed to stop local admin server gracefully", zap.Error(err))
}
}
}()
}
var exiting = new(int32) // accessed atomically
// Exiting returns true if the process is exiting.
// EXPERIMENTAL API: subject to change or removal.
func Exiting() bool { return atomic.LoadInt32(exiting) == 1 }
// Duration can be an integer or a string. An integer is
// interpreted as nanoseconds. If a string, it is a Go
// time.Duration value such as `300ms`, `1.5h`, or `2h45m`;
// valid units are `ns`, `us`/`µs`, `ms`, `s`, `m`, and `h`.
// valid units are `ns`, `us`/`µs`, `ms`, `s`, `m`, `h`, and `d`.
type Duration time.Duration
// UnmarshalJSON satisfies json.Unmarshaler.
@@ -497,7 +764,7 @@ func (d *Duration) UnmarshalJSON(b []byte) error {
var dur time.Duration
var err error
if b[0] == byte('"') && b[len(b)-1] == byte('"') {
dur, err = time.ParseDuration(strings.Trim(string(b), `"`))
dur, err = ParseDuration(strings.Trim(string(b), `"`))
} else {
err = json.Unmarshal(b, &dur)
}
@@ -505,36 +772,188 @@ func (d *Duration) UnmarshalJSON(b []byte) error {
return err
}
// GoModule returns the build info of this Caddy
// build from debug.BuildInfo (requires Go modules).
// If no version information is available, a non-nil
// value will still be returned, but with an
// unknown version.
func GoModule() *debug.Module {
var mod debug.Module
return goModule(&mod)
// ParseDuration parses a duration string, adding
// support for the "d" unit meaning number of days,
// where a day is assumed to be 24h. The maximum
// input string length is 1024.
func ParseDuration(s string) (time.Duration, error) {
if len(s) > 1024 {
return 0, fmt.Errorf("parsing duration: input string too long")
}
var inNumber bool
var numStart int
for i := 0; i < len(s); i++ {
ch := s[i]
if ch == 'd' {
daysStr := s[numStart:i]
days, err := strconv.ParseFloat(daysStr, 64)
if err != nil {
return 0, err
}
hours := days * 24.0
hoursStr := strconv.FormatFloat(hours, 'f', -1, 64)
s = s[:numStart] + hoursStr + "h" + s[i+1:]
i--
continue
}
if !inNumber {
numStart = i
}
inNumber = (ch >= '0' && ch <= '9') || ch == '.' || ch == '-' || ch == '+'
}
return time.ParseDuration(s)
}
// goModule holds the actual implementation of GoModule.
// Allocating debug.Module in GoModule() and passing a
// reference to goModule enables mid-stack inlining.
func goModule(mod *debug.Module) *debug.Module {
mod.Version = "unknown"
// InstanceID returns the UUID for this instance, and generates one if it
// does not already exist. The UUID is stored in the local data directory,
// regardless of storage configuration, since each instance is intended to
// have its own unique ID.
func InstanceID() (uuid.UUID, error) {
uuidFilePath := filepath.Join(AppDataDir(), "instance.uuid")
uuidFileBytes, err := os.ReadFile(uuidFilePath)
if os.IsNotExist(err) {
uuid, err := uuid.NewRandom()
if err != nil {
return uuid, err
}
err = os.WriteFile(uuidFilePath, []byte(uuid.String()), 0600)
return uuid, err
} else if err != nil {
return [16]byte{}, err
}
return uuid.ParseBytes(uuidFileBytes)
}
// CustomVersion is an optional string that overrides Caddy's
// reported version. It can be helpful when downstream packagers
// need to manually set Caddy's version. If no other version
// information is available, the short form version (see
// Version()) will be set to CustomVersion, and the full version
// will include CustomVersion at the beginning.
//
// Set this variable during `go build` with `-ldflags`:
//
// -ldflags '-X github.com/caddyserver/caddy/v2.CustomVersion=v2.6.2'
//
// for example.
var CustomVersion string
// Version returns the Caddy version in a simple/short form, and
// a full version string. The short form will not have spaces and
// is intended for User-Agent strings and similar, but may be
// omitting valuable information. Note that Caddy must be compiled
// in a special way to properly embed complete version information.
// First this function tries to get the version from the embedded
// build info provided by go.mod dependencies; then it tries to
// get info from embedded VCS information, which requires having
// built Caddy from a git repository. If no version is available,
// this function returns "(devel)" because Go uses that, but for
// the simple form we change it to "unknown". If still no version
// is available (e.g. no VCS repo), then it will use CustomVersion;
// CustomVersion is always prepended to the full version string.
//
// See relevant Go issues: https://github.com/golang/go/issues/29228
// and https://github.com/golang/go/issues/50603.
//
// This function is experimental and subject to change or removal.
func Version() (simple, full string) {
// the currently-recommended way to build Caddy involves
// building it as a dependency so we can extract version
// information from go.mod tooling; once the upstream
// Go issues are fixed, we should just be able to use
// bi.Main... hopefully.
var module *debug.Module
bi, ok := debug.ReadBuildInfo()
if ok {
mod.Path = bi.Main.Path
// The recommended way to build Caddy involves
// creating a separate main module, which
// TODO: track related Go issue: https://github.com/golang/go/issues/29228
// once that issue is fixed, we should just be able to use bi.Main... hopefully.
// find the Caddy module in the dependency list
for _, dep := range bi.Deps {
if dep.Path == ImportPath {
return dep
module = dep
break
}
}
return &bi.Main
}
return mod
if module != nil {
simple, full = module.Version, module.Version
if module.Sum != "" {
full += " " + module.Sum
}
if module.Replace != nil {
full += " => " + module.Replace.Path
if module.Replace.Version != "" {
simple = module.Replace.Version + "_custom"
full += "@" + module.Replace.Version
}
if module.Replace.Sum != "" {
full += " " + module.Replace.Sum
}
}
}
if full == "" {
var vcsRevision string
var vcsTime time.Time
var vcsModified bool
for _, setting := range bi.Settings {
switch setting.Key {
case "vcs.revision":
vcsRevision = setting.Value
case "vcs.time":
vcsTime, _ = time.Parse(time.RFC3339, setting.Value)
case "vcs.modified":
vcsModified, _ = strconv.ParseBool(setting.Value)
}
}
if vcsRevision != "" {
var modified string
if vcsModified {
modified = "+modified"
}
full = fmt.Sprintf("%s%s (%s)", vcsRevision, modified, vcsTime.Format(time.RFC822))
simple = vcsRevision
// use short checksum for simple, if hex-only
if _, err := hex.DecodeString(simple); err == nil {
simple = simple[:8]
}
// append date to simple since it can be convenient
// to know the commit date as part of the version
if !vcsTime.IsZero() {
simple += "-" + vcsTime.Format("20060102")
}
}
}
if full == "" {
if CustomVersion != "" {
full = CustomVersion
} else {
full = "unknown"
}
} else if CustomVersion != "" {
full = CustomVersion + " " + full
}
if simple == "" || simple == "(devel)" {
if CustomVersion != "" {
simple = CustomVersion
} else {
simple = "unknown"
}
}
return
}
// ActiveContext returns the currently-active context.
// This function is experimental and might be changed
// or removed in the future.
func ActiveContext() Context {
currentCtxMu.RLock()
defer currentCtxMu.RUnlock()
return currentCtx
}
// CtxKey is a value type for use with context.WithValue.
@@ -542,18 +961,21 @@ type CtxKey string
// This group of variables pertains to the current configuration.
var (
// currentCfgMu protects everything in this var block.
currentCfgMu sync.RWMutex
// currentCtxMu protects everything in this var block.
currentCtxMu sync.RWMutex
// currentCfg is the currently-running configuration.
currentCfg *Config
// currentCtx is the root context for the currently-running
// configuration, which can be accessed through this value.
// If the Config contained in this value is not nil, then
// a config is currently active/running.
currentCtx Context
// rawCfg is the current, generic-decoded configuration;
// we initialize it as a map with one field ("config")
// to maintain parity with the API endpoint and to avoid
// the special case of having to access/mutate the variable
// directly without traversing into it.
rawCfg = map[string]interface{}{
rawCfg = map[string]any{
rawConfigKey: nil,
}
@@ -566,5 +988,11 @@ var (
rawCfgIndex map[string]string
)
// errSameConfig is returned if the new config is the same
// as the old one. This isn't usually an actual, actionable
// error; it's mostly a sentinel value.
var errSameConfig = errors.New("config is unchanged")
// ImportPath is the package import path for Caddy core.
// This identifier may be removed in the future.
const ImportPath = "github.com/caddyserver/caddy/v2"
+74
View File
@@ -0,0 +1,74 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"testing"
"time"
)
func TestParseDuration(t *testing.T) {
const day = 24 * time.Hour
for i, tc := range []struct {
input string
expect time.Duration
}{
{
input: "3h",
expect: 3 * time.Hour,
},
{
input: "1d",
expect: day,
},
{
input: "1d30m",
expect: day + 30*time.Minute,
},
{
input: "1m2d",
expect: time.Minute + day*2,
},
{
input: "1m2d30s",
expect: time.Minute + day*2 + 30*time.Second,
},
{
input: "1d2d",
expect: 3 * day,
},
{
input: "1.5d",
expect: time.Duration(1.5 * float64(day)),
},
{
input: "4m1.25d",
expect: 4*time.Minute + time.Duration(1.25*float64(day)),
},
{
input: "-1.25d12h",
expect: time.Duration(-1.25*float64(day)) - 12*time.Hour,
},
} {
actual, err := ParseDuration(tc.input)
if err != nil {
t.Errorf("Test %d ('%s'): Got error: %v", i, tc.input, err)
continue
}
if actual != tc.expect {
t.Errorf("Test %d ('%s'): Expected=%s Actual=%s", i, tc.input, tc.expect, actual)
}
}
}
+65 -7
View File
@@ -15,6 +15,7 @@
package caddyfile
import (
"bytes"
"encoding/json"
"fmt"
@@ -28,12 +29,12 @@ type Adapter struct {
}
// Adapt converts the Caddyfile config in body to Caddy JSON.
func (a Adapter) Adapt(body []byte, options map[string]interface{}) ([]byte, []caddyconfig.Warning, error) {
func (a Adapter) Adapt(body []byte, options map[string]any) ([]byte, []caddyconfig.Warning, error) {
if a.ServerType == nil {
return nil, nil, fmt.Errorf("no server type")
}
if options == nil {
options = make(map[string]interface{})
options = make(map[string]any)
}
filename, _ := options["filename"].(string)
@@ -51,15 +52,46 @@ func (a Adapter) Adapt(body []byte, options map[string]interface{}) ([]byte, []c
return nil, warnings, err
}
marshalFunc := json.Marshal
if options["pretty"] == "true" {
marshalFunc = caddyconfig.JSONIndent
// lint check: see if input was properly formatted; sometimes messy files files parse
// successfully but result in logical errors (the Caddyfile is a bad format, I'm sorry)
if warning, different := formattingDifference(filename, body); different {
warnings = append(warnings, warning)
}
result, err := marshalFunc(cfg)
result, err := json.Marshal(cfg)
return result, warnings, err
}
// formattingDifference returns a warning and true if the formatted version
// is any different from the input; empty warning and false otherwise.
// TODO: also perform this check on imported files
func formattingDifference(filename string, body []byte) (caddyconfig.Warning, bool) {
// replace windows-style newlines to normalize comparison
normalizedBody := bytes.Replace(body, []byte("\r\n"), []byte("\n"), -1)
formatted := Format(normalizedBody)
if bytes.Equal(formatted, normalizedBody) {
return caddyconfig.Warning{}, false
}
// find where the difference is
line := 1
for i, ch := range normalizedBody {
if i >= len(formatted) || ch != formatted[i] {
break
}
if ch == '\n' {
line++
}
}
return caddyconfig.Warning{
File: filename,
Line: line,
Message: "Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies",
}, true
}
// Unmarshaler is a type that can unmarshal
// Caddyfile tokens to set itself up for a
// JSON encoding. The goal of an unmarshaler
@@ -84,7 +116,33 @@ type ServerType interface {
// (e.g. CLI flags) and creates a Caddy
// config, along with any warnings or
// an error.
Setup([]ServerBlock, map[string]interface{}) (*caddy.Config, []caddyconfig.Warning, error)
Setup([]ServerBlock, map[string]any) (*caddy.Config, []caddyconfig.Warning, error)
}
// UnmarshalModule instantiates a module with the given ID and invokes
// UnmarshalCaddyfile on the new value using the immediate next segment
// of d as input. In other words, d's next token should be the first
// token of the module's Caddyfile input.
//
// This function is used when the next segment of Caddyfile tokens
// belongs to another Caddy module. The returned value is often
// type-asserted to the module's associated type for practical use
// when setting up a config.
func UnmarshalModule(d *Dispenser, moduleID string) (Unmarshaler, error) {
mod, err := caddy.GetModule(moduleID)
if err != nil {
return nil, d.Errf("getting module named '%s': %v", moduleID, err)
}
inst := mod.New()
unm, ok := inst.(Unmarshaler)
if !ok {
return nil, d.Errf("module %s is not a Caddyfile unmarshaler; is %T", mod.ID, inst)
}
err = unm.UnmarshalCaddyfile(d.NewFromNextSegment())
if err != nil {
return nil, err
}
return unm, nil
}
// Interface guard
+143 -10
View File
@@ -17,6 +17,9 @@ package caddyfile
import (
"errors"
"fmt"
"io"
"log"
"strconv"
"strings"
)
@@ -37,6 +40,16 @@ func NewDispenser(tokens []Token) *Dispenser {
}
}
// NewTestDispenser parses input into tokens and creates a new
// Dispenser for test purposes only; any errors are fatal.
func NewTestDispenser(input string) *Dispenser {
tokens, err := allTokens("Testfile", []byte(input))
if err != nil && err != io.EOF {
log.Fatalf("getting all tokens from input: %v", err)
}
return NewDispenser(tokens)
}
// Next loads the next token. Returns true if a token
// was loaded; false otherwise. If false, all tokens
// have been consumed.
@@ -133,15 +146,15 @@ func (d *Dispenser) NextLine() bool {
//
// Proper use of this method looks like this:
//
// for nesting := d.Nesting(); d.NextBlock(nesting); {
// }
// for nesting := d.Nesting(); d.NextBlock(nesting); {
// }
//
// However, in simple cases where it is known that the
// Dispenser is new and has not already traversed state
// by a loop over NextBlock(), this will do:
//
// for d.NextBlock(0) {
// }
// for d.NextBlock(0) {
// }
//
// As with other token parsing logic, a loop over
// NextBlock() should be contained within a loop over
@@ -189,6 +202,43 @@ func (d *Dispenser) Val() string {
return d.tokens[d.cursor].Text
}
// ValRaw gets the raw text of the current token (including quotes).
// If there is no token loaded, it returns empty string.
func (d *Dispenser) ValRaw() string {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return ""
}
quote := d.tokens[d.cursor].wasQuoted
if quote > 0 {
return string(quote) + d.tokens[d.cursor].Text + string(quote) // string literal
}
return d.tokens[d.cursor].Text
}
// ScalarVal gets value of the current token, converted to the closest
// scalar type. If there is no token loaded, it returns nil.
func (d *Dispenser) ScalarVal() any {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return nil
}
quote := d.tokens[d.cursor].wasQuoted
text := d.tokens[d.cursor].Text
if quote > 0 {
return text // string literal
}
if num, err := strconv.Atoi(text); err == nil {
return num
}
if num, err := strconv.ParseFloat(text, 64); err == nil {
return num
}
if bool, err := strconv.ParseBool(text); err == nil {
return bool
}
return text
}
// Line gets the line number of the current token.
// If there is no token loaded, it returns 0.
func (d *Dispenser) Line() int {
@@ -237,6 +287,19 @@ func (d *Dispenser) AllArgs(targets ...*string) bool {
return true
}
// CountRemainingArgs counts the amount of remaining arguments
// (tokens on the same line) without consuming the tokens.
func (d *Dispenser) CountRemainingArgs() int {
count := 0
for d.NextArg() {
count++
}
for i := 0; i < count; i++ {
d.Prev()
}
return count
}
// RemainingArgs loads any more arguments (tokens on the same line)
// into a slice and returns them. Open curly brace tokens also indicate
// the end of arguments, and the curly brace is not included in
@@ -249,6 +312,18 @@ func (d *Dispenser) RemainingArgs() []string {
return args
}
// RemainingArgsRaw loads any more arguments (tokens on the same line,
// retaining quotes) into a slice and returns them. Open curly brace
// tokens also indicate the end of arguments, and the curly brace is
// not included in the return value nor is it loaded.
func (d *Dispenser) RemainingArgsRaw() []string {
var args []string
for d.NextArg() {
args = append(args, d.ValRaw())
}
return args
}
// NewFromNextSegment returns a new dispenser with a copy of
// the tokens from the current token until the end of the
// "directive" whether that be to the end of the line or
@@ -333,13 +408,17 @@ func (d *Dispenser) EOFErr() error {
// Err generates a custom parse-time error with a message of msg.
func (d *Dispenser) Err(msg string) error {
msg = fmt.Sprintf("%s:%d - Error during parsing: %s", d.File(), d.Line(), msg)
return errors.New(msg)
return d.Errf(msg)
}
// Errf is like Err, but for formatted error messages
func (d *Dispenser) Errf(format string, args ...interface{}) error {
return d.Err(fmt.Sprintf(format, args...))
func (d *Dispenser) Errf(format string, args ...any) error {
return d.WrapErr(fmt.Errorf(format, args...))
}
// WrapErr takes an existing error and adds the Caddyfile file and line number.
func (d *Dispenser) WrapErr(err error) error {
return fmt.Errorf("%s:%d - Error during parsing: %w", d.File(), d.Line(), err)
}
// Delete deletes the current token and returns the updated slice
@@ -379,6 +458,60 @@ func (d *Dispenser) isNewLine() bool {
if d.cursor > len(d.tokens)-1 {
return false
}
return d.tokens[d.cursor-1].File != d.tokens[d.cursor].File ||
d.tokens[d.cursor-1].Line+d.numLineBreaks(d.cursor-1) < d.tokens[d.cursor].Line
prev := d.tokens[d.cursor-1]
curr := d.tokens[d.cursor]
// If the previous token is from a different file,
// we can assume it's from a different line
if prev.File != curr.File {
return true
}
// The previous token may contain line breaks if
// it was quoted and spanned multiple lines. e.g:
//
// dir "foo
// bar
// baz"
prevLineBreaks := d.numLineBreaks(d.cursor - 1)
// If the previous token (incl line breaks) ends
// on a line earlier than the current token,
// then the current token is on a new line
return prev.Line+prevLineBreaks < curr.Line
}
// isNextOnNewLine determines whether the current token is on a different
// line (higher line number) than the next token. It handles imported
// tokens correctly. If there isn't a next token, it returns true.
func (d *Dispenser) isNextOnNewLine() bool {
if d.cursor < 0 {
return false
}
if d.cursor >= len(d.tokens)-1 {
return true
}
curr := d.tokens[d.cursor]
next := d.tokens[d.cursor+1]
// If the next token is from a different file,
// we can assume it's from a different line
if curr.File != next.File {
return true
}
// The current token may contain line breaks if
// it was quoted and spanned multiple lines. e.g:
//
// dir "foo
// bar
// baz"
currLineBreaks := d.numLineBreaks(d.cursor)
// If the current token (incl line breaks) ends
// on a line earlier than the next token,
// then the next token is on a new line
return curr.Line+currLineBreaks < next.Line
}
+5 -10
View File
@@ -15,8 +15,7 @@
package caddyfile
import (
"io"
"log"
"errors"
"reflect"
"strings"
"testing"
@@ -305,14 +304,10 @@ func TestDispenser_ArgErr_Err(t *testing.T) {
if !strings.Contains(err.Error(), "foobar") {
t.Errorf("Expected error message with custom message in it ('foobar'); got '%v'", err)
}
}
// NewTestDispenser parses input into tokens and creates a new
// Disenser for test purposes only; any errors are fatal.
func NewTestDispenser(input string) *Dispenser {
tokens, err := allTokens("Testfile", []byte(input))
if err != nil && err != io.EOF {
log.Fatalf("getting all tokens from input: %v", err)
var ErrBarIsFull = errors.New("bar is full")
bookingError := d.Errf("unable to reserve: %w", ErrBarIsFull)
if !errors.Is(bookingError, ErrBarIsFull) {
t.Errorf("Errf(): should be able to unwrap the error chain")
}
return NewDispenser(tokens)
}
+7 -4
View File
@@ -78,6 +78,9 @@ func Format(input []byte) []byte {
if comment {
if ch == '\n' {
comment = false
space = true
nextLine()
continue
} else {
write(ch)
continue
@@ -131,9 +134,6 @@ func Format(input []byte) []byte {
//////////////////////////////////////////////////////////
if ch == '#' {
if !spacePrior && !beginningOfLine {
write(' ')
}
comment = true
}
@@ -153,7 +153,10 @@ func Format(input []byte) []byte {
openBraceWritten = true
nextLine()
newLines = 0
nesting++
// prevent infinite nesting from ridiculous inputs (issue #4169)
if nesting < 10 {
nesting++
}
}
switch {
+27
View File
@@ -0,0 +1,27 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build gofuzz
package caddyfile
import "bytes"
func FuzzFormat(input []byte) int {
formatted := Format(input)
if bytes.Equal(formatted, Format(formatted)) {
return 1
}
return 0
}
+61 -2
View File
@@ -179,6 +179,11 @@ d {
{$F}
}`,
},
{
description: "env var placeholders with port",
input: `:{$PORT}`,
expect: `:{$PORT}`,
},
{
description: "comments",
input: `#a "\n"
@@ -201,7 +206,7 @@ c
}
d {
e #f
e#f
# g
}
@@ -229,7 +234,7 @@ bar"
j {
"\"k\" l m"
}`,
expect: `"a \"b\" " #c
expect: `"a \"b\" "#c
d
e {
@@ -305,6 +310,60 @@ bar "{\"key\":34}"`,
baz`,
},
{
description: "hash within string is not a comment",
input: `redir / /some/#/path`,
expect: `redir / /some/#/path`,
},
{
description: "brace does not fold into comment above",
input: `# comment
{
foo
}`,
expect: `# comment
{
foo
}`,
},
{
description: "matthewpi/vscode-caddyfile-support#13",
input: `{
email {$ACMEEMAIL}
#debug
}
block {
}
`,
expect: `{
email {$ACMEEMAIL}
#debug
}
block {
}
`,
},
{
description: "matthewpi/vscode-caddyfile-support#13 - bad formatting",
input: `{
email {$ACMEEMAIL}
#debug
}
block {
}
`,
expect: `{
email {$ACMEEMAIL}
#debug
}
block {
}
`,
},
} {
// the formatter should output a trailing newline,
// even if the tests aren't written to expect that
+127
View File
@@ -0,0 +1,127 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"fmt"
)
type adjacency map[string][]string
type importGraph struct {
nodes map[string]bool
edges adjacency
}
func (i *importGraph) addNode(name string) {
if i.nodes == nil {
i.nodes = make(map[string]bool)
}
if _, exists := i.nodes[name]; exists {
return
}
i.nodes[name] = true
}
func (i *importGraph) addNodes(names []string) {
for _, name := range names {
i.addNode(name)
}
}
func (i *importGraph) removeNode(name string) {
delete(i.nodes, name)
}
func (i *importGraph) removeNodes(names []string) {
for _, name := range names {
i.removeNode(name)
}
}
func (i *importGraph) addEdge(from, to string) error {
if !i.exists(from) || !i.exists(to) {
return fmt.Errorf("one of the nodes does not exist")
}
if i.willCycle(to, from) {
return fmt.Errorf("a cycle of imports exists between %s and %s", from, to)
}
if i.areConnected(from, to) {
// if connected, there's nothing to do
return nil
}
if i.nodes == nil {
i.nodes = make(map[string]bool)
}
if i.edges == nil {
i.edges = make(adjacency)
}
i.edges[from] = append(i.edges[from], to)
return nil
}
func (i *importGraph) addEdges(from string, tos []string) error {
for _, to := range tos {
err := i.addEdge(from, to)
if err != nil {
return err
}
}
return nil
}
func (i *importGraph) areConnected(from, to string) bool {
al, ok := i.edges[from]
if !ok {
return false
}
for _, v := range al {
if v == to {
return true
}
}
return false
}
func (i *importGraph) willCycle(from, to string) bool {
collector := make(map[string]bool)
var visit func(string)
visit = func(start string) {
if !collector[start] {
collector[start] = true
for _, v := range i.edges[start] {
visit(v)
}
}
}
for _, v := range i.edges[from] {
visit(v)
}
for k := range collector {
if to == k {
return true
}
}
return false
}
func (i *importGraph) exists(key string) bool {
_, exists := i.nodes[key]
return exists
}
Executable → Regular
+48 -14
View File
@@ -1,4 +1,4 @@
// Copyright 2015 Light Code Labs, LLC
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -16,6 +16,7 @@ package caddyfile
import (
"bufio"
"bytes"
"io"
"unicode"
)
@@ -34,9 +35,12 @@ type (
// Token represents a single parsable unit.
Token struct {
File string
Line int
Text string
File string
Line int
Text string
wasQuoted rune // enclosing quote character, if any
inSnippet bool
snippetName string
}
)
@@ -73,10 +77,11 @@ func (l *lexer) load(input io.Reader) error {
// a token was loaded; false otherwise.
func (l *lexer) next() bool {
var val []rune
var comment, quoted, escaped bool
var comment, quoted, btQuoted, escaped bool
makeToken := func() bool {
makeToken := func(quoted rune) bool {
l.token.Text = string(val)
l.token.wasQuoted = quoted
return true
}
@@ -84,7 +89,7 @@ func (l *lexer) next() bool {
ch, _, err := l.reader.ReadRune()
if err != nil {
if len(val) > 0 {
return makeToken()
return makeToken(0)
}
if err == io.EOF {
return false
@@ -92,13 +97,13 @@ func (l *lexer) next() bool {
panic(err)
}
if !escaped && ch == '\\' {
if !escaped && !btQuoted && ch == '\\' {
escaped = true
continue
}
if quoted {
if escaped {
if quoted || btQuoted {
if quoted && escaped {
// all is literal in quoted area,
// so only escape quotes
if ch != '"' {
@@ -106,8 +111,11 @@ func (l *lexer) next() bool {
}
escaped = false
} else {
if ch == '"' {
return makeToken()
if quoted && ch == '"' {
return makeToken('"')
}
if btQuoted && ch == '`' {
return makeToken('`')
}
}
if ch == '\n' {
@@ -133,12 +141,12 @@ func (l *lexer) next() bool {
comment = false
}
if len(val) > 0 {
return makeToken()
return makeToken(0)
}
continue
}
if ch == '#' {
if ch == '#' && len(val) == 0 {
comment = true
}
if comment {
@@ -151,6 +159,10 @@ func (l *lexer) next() bool {
quoted = true
continue
}
if ch == '`' {
btQuoted = true
continue
}
}
if escaped {
@@ -161,3 +173,25 @@ func (l *lexer) next() bool {
val = append(val, ch)
}
}
// Tokenize takes bytes as input and lexes it into
// a list of tokens that can be parsed as a Caddyfile.
// Also takes a filename to fill the token's File as
// the source of the tokens, which is important to
// determine relative paths for `import` directives.
func Tokenize(input []byte, filename string) ([]Token, error) {
l := lexer{}
if err := l.load(bytes.NewReader(input)); err != nil {
return nil, err
}
var tokens []Token
for l.next() {
l.token.File = filename
tokens = append(tokens, l.token)
}
return tokens, nil
}
func (t Token) Quoted() bool {
return t.wasQuoted > 0
}
+28
View File
@@ -0,0 +1,28 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build gofuzz
package caddyfile
func FuzzTokenize(input []byte) int {
tokens, err := Tokenize(input, "Caddyfile")
if err != nil {
return 0
}
if len(tokens) == 0 {
return -1
}
return 1
}
+83 -41
View File
@@ -1,4 +1,4 @@
// Copyright 2015 Light Code Labs, LLC
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -15,37 +15,35 @@
package caddyfile
import (
"log"
"strings"
"testing"
)
type lexerTestCase struct {
input string
input []byte
expected []Token
}
func TestLexer(t *testing.T) {
testCases := []lexerTestCase{
{
input: `host:123`,
input: []byte(`host:123`),
expected: []Token{
{Line: 1, Text: "host:123"},
},
},
{
input: `host:123
input: []byte(`host:123
directive`,
directive`),
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 3, Text: "directive"},
},
},
{
input: `host:123 {
input: []byte(`host:123 {
directive
}`,
}`),
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 1, Text: "{"},
@@ -54,7 +52,7 @@ func TestLexer(t *testing.T) {
},
},
{
input: `host:123 { directive }`,
input: []byte(`host:123 { directive }`),
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 1, Text: "{"},
@@ -63,12 +61,12 @@ func TestLexer(t *testing.T) {
},
},
{
input: `host:123 {
input: []byte(`host:123 {
#comment
directive
# comment
foobar # another comment
}`,
}`),
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 1, Text: "{"},
@@ -78,8 +76,28 @@ func TestLexer(t *testing.T) {
},
},
{
input: `a "quoted value" b
foobar`,
input: []byte(`host:123 {
# hash inside string is not a comment
redir / /some/#/path
}`),
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 1, Text: "{"},
{Line: 3, Text: "redir"},
{Line: 3, Text: "/"},
{Line: 3, Text: "/some/#/path"},
{Line: 4, Text: "}"},
},
},
{
input: []byte("# comment at beginning of file\n# comment at beginning of line\nhost:123"),
expected: []Token{
{Line: 3, Text: "host:123"},
},
},
{
input: []byte(`a "quoted value" b
foobar`),
expected: []Token{
{Line: 1, Text: "a"},
{Line: 1, Text: "quoted value"},
@@ -88,7 +106,7 @@ func TestLexer(t *testing.T) {
},
},
{
input: `A "quoted \"value\" inside" B`,
input: []byte(`A "quoted \"value\" inside" B`),
expected: []Token{
{Line: 1, Text: "A"},
{Line: 1, Text: `quoted "value" inside`},
@@ -96,7 +114,7 @@ func TestLexer(t *testing.T) {
},
},
{
input: "An escaped \"newline\\\ninside\" quotes",
input: []byte("An escaped \"newline\\\ninside\" quotes"),
expected: []Token{
{Line: 1, Text: "An"},
{Line: 1, Text: "escaped"},
@@ -105,7 +123,7 @@ func TestLexer(t *testing.T) {
},
},
{
input: "An escaped newline\\\noutside quotes",
input: []byte("An escaped newline\\\noutside quotes"),
expected: []Token{
{Line: 1, Text: "An"},
{Line: 1, Text: "escaped"},
@@ -115,7 +133,7 @@ func TestLexer(t *testing.T) {
},
},
{
input: "line1\\\nescaped\nline2\nline3",
input: []byte("line1\\\nescaped\nline2\nline3"),
expected: []Token{
{Line: 1, Text: "line1"},
{Line: 1, Text: "escaped"},
@@ -124,7 +142,7 @@ func TestLexer(t *testing.T) {
},
},
{
input: "line1\\\nescaped1\\\nescaped2\nline4\nline5",
input: []byte("line1\\\nescaped1\\\nescaped2\nline4\nline5"),
expected: []Token{
{Line: 1, Text: "line1"},
{Line: 1, Text: "escaped1"},
@@ -134,34 +152,34 @@ func TestLexer(t *testing.T) {
},
},
{
input: `"unescapable\ in quotes"`,
input: []byte(`"unescapable\ in quotes"`),
expected: []Token{
{Line: 1, Text: `unescapable\ in quotes`},
},
},
{
input: `"don't\escape"`,
input: []byte(`"don't\escape"`),
expected: []Token{
{Line: 1, Text: `don't\escape`},
},
},
{
input: `"don't\\escape"`,
input: []byte(`"don't\\escape"`),
expected: []Token{
{Line: 1, Text: `don't\\escape`},
},
},
{
input: `un\escapable`,
input: []byte(`un\escapable`),
expected: []Token{
{Line: 1, Text: `un\escapable`},
},
},
{
input: `A "quoted value with line
input: []byte(`A "quoted value with line
break inside" {
foobar
}`,
}`),
expected: []Token{
{Line: 1, Text: "A"},
{Line: 1, Text: "quoted value with line\n\t\t\t\t\tbreak inside"},
@@ -171,13 +189,13 @@ func TestLexer(t *testing.T) {
},
},
{
input: `"C:\php\php-cgi.exe"`,
input: []byte(`"C:\php\php-cgi.exe"`),
expected: []Token{
{Line: 1, Text: `C:\php\php-cgi.exe`},
},
},
{
input: `empty "" string`,
input: []byte(`empty "" string`),
expected: []Token{
{Line: 1, Text: `empty`},
{Line: 1, Text: ``},
@@ -185,7 +203,7 @@ func TestLexer(t *testing.T) {
},
},
{
input: "skip those\r\nCR characters",
input: []byte("skip those\r\nCR characters"),
expected: []Token{
{Line: 1, Text: "skip"},
{Line: 1, Text: "those"},
@@ -194,30 +212,54 @@ func TestLexer(t *testing.T) {
},
},
{
input: "\xEF\xBB\xBF:8080", // test with leading byte order mark
input: []byte("\xEF\xBB\xBF:8080"), // test with leading byte order mark
expected: []Token{
{Line: 1, Text: ":8080"},
},
},
{
input: []byte("simple `backtick quoted` string"),
expected: []Token{
{Line: 1, Text: `simple`},
{Line: 1, Text: `backtick quoted`},
{Line: 1, Text: `string`},
},
},
{
input: []byte("multiline `backtick\nquoted\n` string"),
expected: []Token{
{Line: 1, Text: `multiline`},
{Line: 1, Text: "backtick\nquoted\n"},
{Line: 3, Text: `string`},
},
},
{
input: []byte("nested `\"quotes inside\" backticks` string"),
expected: []Token{
{Line: 1, Text: `nested`},
{Line: 1, Text: `"quotes inside" backticks`},
{Line: 1, Text: `string`},
},
},
{
input: []byte("reverse-nested \"`backticks` inside\" quotes"),
expected: []Token{
{Line: 1, Text: `reverse-nested`},
{Line: 1, Text: "`backticks` inside"},
{Line: 1, Text: `quotes`},
},
},
}
for i, testCase := range testCases {
actual := tokenize(testCase.input)
actual, err := Tokenize(testCase.input, "")
if err != nil {
t.Errorf("%v", err)
}
lexerCompare(t, i, testCase.expected, actual)
}
}
func tokenize(input string) (tokens []Token) {
l := lexer{}
if err := l.load(strings.NewReader(input)); err != nil {
log.Printf("[ERROR] load failed: %v", err)
}
for l.next() {
tokens = append(tokens, l.token)
}
return
}
func lexerCompare(t *testing.T, n int, expected, actual []Token) {
if len(expected) != len(actual) {
t.Errorf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
Executable → Regular
+137 -43
View File
@@ -1,4 +1,4 @@
// Copyright 2015 Light Code Labs, LLC
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -16,11 +16,15 @@ package caddyfile
import (
"bytes"
"io/ioutil"
"log"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/caddyserver/caddy/v2"
"go.uber.org/zap"
)
// Parse parses the input just enough to group tokens, in
@@ -33,15 +37,43 @@ import (
// Environment variables in {$ENVIRONMENT_VARIABLE} notation
// will be replaced before parsing begins.
func Parse(filename string, input []byte) ([]ServerBlock, error) {
tokens, err := allTokens(filename, input)
// unfortunately, we must copy the input because parsing must
// remain a read-only operation, but we have to expand environment
// variables before we parse, which changes the underlying array (#4422)
inputCopy := make([]byte, len(input))
copy(inputCopy, input)
tokens, err := allTokens(filename, inputCopy)
if err != nil {
return nil, err
}
p := parser{Dispenser: NewDispenser(tokens)}
p := parser{
Dispenser: NewDispenser(tokens),
importGraph: importGraph{
nodes: make(map[string]bool),
edges: make(adjacency),
},
}
return p.parseAll()
}
// allTokens lexes the entire input, but does not parse it.
// It returns all the tokens from the input, unstructured
// and in order. It may mutate input as it expands env vars.
func allTokens(filename string, input []byte) ([]Token, error) {
inputCopy, err := replaceEnvVars(input)
if err != nil {
return nil, err
}
tokens, err := Tokenize(inputCopy, filename)
if err != nil {
return nil, err
}
return tokens, nil
}
// replaceEnvVars replaces all occurrences of environment variables.
// It mutates the underlying array and returns the updated slice.
func replaceEnvVars(input []byte) ([]byte, error) {
var offset int
for {
@@ -57,52 +89,42 @@ func replaceEnvVars(input []byte) ([]byte, error) {
end += begin + len(spanOpen) // make end relative to input, not begin
// get the name; if there is no name, skip it
envVarName := input[begin+len(spanOpen) : end]
if len(envVarName) == 0 {
envString := input[begin+len(spanOpen) : end]
if len(envString) == 0 {
offset = end + len(spanClose)
continue
}
// split the string into a key and an optional default
envParts := strings.SplitN(string(envString), envVarDefaultDelimiter, 2)
// do a lookup for the env var, replace with the default if not found
envVarValue, found := os.LookupEnv(envParts[0])
if !found && len(envParts) == 2 {
envVarValue = envParts[1]
}
// get the value of the environment variable
envVarValue := []byte(os.ExpandEnv(os.Getenv(string(envVarName))))
// note that this causes one-level deep chaining
envVarBytes := []byte(envVarValue)
// splice in the value
input = append(input[:begin],
append(envVarValue, input[end+len(spanClose):]...)...)
append(envVarBytes, input[end+len(spanClose):]...)...)
// continue at the end of the replacement
offset = begin + len(envVarValue)
offset = begin + len(envVarBytes)
}
return input, nil
}
// allTokens lexes the entire input, but does not parse it.
// It returns all the tokens from the input, unstructured
// and in order.
func allTokens(filename string, input []byte) ([]Token, error) {
input, err := replaceEnvVars(input)
if err != nil {
return nil, err
}
l := new(lexer)
err = l.load(bytes.NewReader(input))
if err != nil {
return nil, err
}
var tokens []Token
for l.next() {
l.token.File = filename
tokens = append(tokens, l.token)
}
return tokens, nil
}
type parser struct {
*Dispenser
block ServerBlock // current server block being parsed
eof bool // if we encounter a valid EOF in a hard place
definedSnippets map[string][]Token
nesting int
importGraph importGraph
}
func (p *parser) parseAll() ([]ServerBlock, error) {
@@ -158,6 +180,15 @@ func (p *parser) begin() error {
if err != nil {
return err
}
// Just as we need to track which file the token comes from, we need to
// keep track of which snippets do the tokens come from. This is helpful
// in tracking import cycles across files/snippets by namespacing them. Without
// this we end up with false-positives in cycle-detection.
for k, v := range tokens {
v.inSnippet = true
v.snippetName = name
tokens[k] = v
}
p.definedSnippets[name] = tokens
// empty block keys so we don't save this block as a real server.
p.block.Keys = nil
@@ -187,9 +218,20 @@ func (p *parser) addresses() error {
if expectingAnother {
return p.Errf("Expected another address but had '%s' - check for extra comma", tkn)
}
// Mark this server block as being defined with braces.
// This is used to provide a better error message when
// the user may have tried to define two server blocks
// without having used braces, which are required in
// that case.
p.block.HasBraces = true
break
}
// Users commonly forget to place a space between the address and the '{'
if strings.HasSuffix(tkn, "{") {
return p.Errf("Site addresses cannot end with a curly brace: '%s' - put a space between the token and the brace", tkn)
}
if tkn != "" { // empty token possible if user typed ""
// Trailing comma indicates another address will follow, which
// may possibly be on the next line
@@ -200,6 +242,13 @@ func (p *parser) addresses() error {
expectingAnother = false // but we may still see another one on this line
}
// If there's a comma here, it's probably because they didn't use a space
// between their two domains, e.g. "foo.com,bar.com", which would not be
// parsed as two separate site addresses.
if strings.Contains(tkn, ",") {
return p.Errf("Site addresses cannot contain a comma ',': '%s' - put a space after the comma to separate site addresses", tkn)
}
p.block.Keys = append(p.block.Keys, tkn)
}
@@ -292,17 +341,30 @@ func (p *parser) doImport() error {
if importPattern == "" {
return p.Err("Import requires a non-empty filepath")
}
if p.NextArg() {
return p.Err("Import takes only one argument (glob pattern or file)")
// grab remaining args as placeholder replacements
args := p.RemainingArgs()
// add args to the replacer
repl := caddy.NewEmptyReplacer()
for index, arg := range args {
repl.Set("args."+strconv.Itoa(index), arg)
}
// splice out the import directive and its argument (2 tokens total)
tokensBefore := p.tokens[:p.cursor-1]
// splice out the import directive and its arguments
// (2 tokens, plus the length of args)
tokensBefore := p.tokens[:p.cursor-1-len(args)]
tokensAfter := p.tokens[p.cursor+1:]
var importedTokens []Token
var nodes []string
// first check snippets. That is a simple, non-recursive replacement
if p.definedSnippets != nil && p.definedSnippets[importPattern] != nil {
importedTokens = p.definedSnippets[importPattern]
if len(importedTokens) > 0 {
// just grab the first one
nodes = append(nodes, fmt.Sprintf("%s:%s", importedTokens[0].File, importedTokens[0].snippetName))
}
} else {
// make path relative to the file of the _token_ being processed rather
// than current working directory (issue #867) and then use glob to get
@@ -331,14 +393,13 @@ func (p *parser) doImport() error {
}
if len(matches) == 0 {
if strings.ContainsAny(globPattern, "*?[]") {
log.Printf("[WARNING] No files matching import glob pattern: %s", importPattern)
caddy.Log().Warn("No files matching import glob pattern", zap.String("pattern", importPattern))
} else {
return p.Errf("File to import not found: %s", importPattern)
}
}
// collect all the imported tokens
for _, importFile := range matches {
newTokens, err := p.doSingleImport(importFile)
if err != nil {
@@ -346,12 +407,34 @@ func (p *parser) doImport() error {
}
importedTokens = append(importedTokens, newTokens...)
}
nodes = matches
}
nodeName := p.File()
if p.Token().inSnippet {
nodeName += fmt.Sprintf(":%s", p.Token().snippetName)
}
p.importGraph.addNode(nodeName)
p.importGraph.addNodes(nodes)
if err := p.importGraph.addEdges(nodeName, nodes); err != nil {
p.importGraph.removeNodes(nodes)
return err
}
// copy the tokens so we don't overwrite p.definedSnippets
tokensCopy := make([]Token, len(importedTokens))
copy(tokensCopy, importedTokens)
// run the argument replacer on the tokens
for index, token := range tokensCopy {
token.Text = repl.ReplaceKnown(token.Text, "")
tokensCopy[index] = token
}
// splice the imported tokens in the place of the import statement
// and rewind cursor so Next() will land on first imported token
p.tokens = append(tokensBefore, append(importedTokens, tokensAfter...)...)
p.cursor--
p.tokens = append(tokensBefore, append(tokensCopy, tokensAfter...)...)
p.cursor -= len(args) + 1
return nil
}
@@ -371,7 +454,7 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
return nil, p.Errf("Could not import %s: is a directory", importFile)
}
input, err := ioutil.ReadAll(file)
input, err := io.ReadAll(file)
if err != nil {
return nil, p.Errf("Could not read imported file %s: %v", importFile, err)
}
@@ -411,6 +494,13 @@ func (p *parser) directive() error {
for p.Next() {
if p.Val() == "{" {
p.nesting++
if !p.isNextOnNewLine() && p.Token().wasQuoted == 0 {
return p.Err("Unexpected next token after '{' on same line")
}
} else if p.Val() == "{}" {
if p.isNextOnNewLine() && p.Token().wasQuoted == 0 {
return p.Err("Unexpected '{}' at end of line")
}
} else if p.isNewLine() && p.nesting == 0 {
p.cursor-- // read too far
break
@@ -501,8 +591,9 @@ func (p *parser) snippetTokens() ([]Token, error) {
// head of the server block with tokens, which are
// grouped by segments.
type ServerBlock struct {
Keys []string
Segments []Segment
HasBraces bool
Keys []string
Segments []Segment
}
// DispenseDirective returns a dispenser that contains
@@ -533,4 +624,7 @@ func (s Segment) Directive() string {
// spanOpen and spanClose are used to bound spans that
// contain the name of an environment variable.
var spanOpen, spanClose = []byte{'{', '$'}, []byte{'}'}
var (
spanOpen, spanClose = []byte{'{', '$'}, []byte{'}'}
envVarDefaultDelimiter = ":"
)
+69 -9
View File
@@ -1,4 +1,4 @@
// Copyright 2015 Light Code Labs, LLC
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
@@ -16,7 +16,6 @@ package caddyfile
import (
"bytes"
"io/ioutil"
"os"
"path/filepath"
"testing"
@@ -160,6 +159,10 @@ func TestParseOneAndImport(t *testing.T) {
"localhost",
}, []int{}},
{`localhost{
dir1
}`, true, []string{}, []int{}},
{`localhost
dir1 {
nested {
@@ -182,14 +185,31 @@ func TestParseOneAndImport(t *testing.T) {
"host1",
}, []int{1, 2}},
{`import testdata/import_test1.txt testdata/import_test2.txt`, true, []string{}, []int{}},
{`import testdata/not_found.txt`, true, []string{}, []int{}},
{`""`, false, []string{}, []int{}},
{``, false, []string{}, []int{}},
// Unexpected next token after '{' on same line
{`localhost
dir1 { a b }`, true, []string{"localhost"}, []int{}},
// Workaround with quotes
{`localhost
dir1 "{" a b "}"`, false, []string{"localhost"}, []int{5}},
// Unexpected '{}' at end of line
{`localhost
dir1 {}`, true, []string{"localhost"}, []int{}},
// Workaround with quotes
{`localhost
dir1 "{}"`, false, []string{"localhost"}, []int{2}},
// import with args
{`import testdata/import_args0.txt a`, false, []string{"a"}, []int{}},
{`import testdata/import_args1.txt a b`, false, []string{"a", "b"}, []int{}},
{`import testdata/import_args*.txt a b`, false, []string{"a"}, []int{2}},
// test cases found by fuzzing!
{`import }{$"`, true, []string{}, []int{}},
{`import /*/*.txt`, true, []string{}, []int{}},
@@ -210,6 +230,7 @@ func TestParseOneAndImport(t *testing.T) {
t.Errorf("Test %d: Expected no error, but got: %v", i, err)
}
// t.Logf("%+v\n", result)
if len(result.Keys) != len(test.keys) {
t.Errorf("Test %d: Expected %d keys, got %d",
i, len(test.keys), len(result.Keys))
@@ -272,7 +293,7 @@ func TestRecursiveImport(t *testing.T) {
}
// test relative recursive import
err = ioutil.WriteFile(recursiveFile1, []byte(
err = os.WriteFile(recursiveFile1, []byte(
`localhost
dir1
import recursive_import_test2`), 0644)
@@ -281,7 +302,7 @@ func TestRecursiveImport(t *testing.T) {
}
defer os.Remove(recursiveFile1)
err = ioutil.WriteFile(recursiveFile2, []byte("dir2 1"), 0644)
err = os.WriteFile(recursiveFile2, []byte("dir2 1"), 0644)
if err != nil {
t.Fatal(err)
}
@@ -306,7 +327,7 @@ func TestRecursiveImport(t *testing.T) {
}
// test absolute recursive import
err = ioutil.WriteFile(recursiveFile1, []byte(
err = os.WriteFile(recursiveFile1, []byte(
`localhost
dir1
import `+recursiveFile2), 0644)
@@ -362,7 +383,7 @@ func TestDirectiveImport(t *testing.T) {
t.Fatal(err)
}
err = ioutil.WriteFile(directiveFile, []byte(`prop1 1
err = os.WriteFile(directiveFile, []byte(`prop1 1
prop2 2`), 0644)
if err != nil {
t.Fatal(err)
@@ -440,6 +461,28 @@ func TestParseAll(t *testing.T) {
{`import notfound/*`, false, [][]string{}}, // glob needn't error with no matches
{`import notfound/file.conf`, true, [][]string{}}, // but a specific file should
// recursive self-import
{`import testdata/import_recursive0.txt`, true, [][]string{}},
{`import testdata/import_recursive3.txt
import testdata/import_recursive1.txt`, true, [][]string{}},
// cyclic imports
{`(A) {
import A
}
:80
import A
`, true, [][]string{}},
{`(A) {
import B
}
(B) {
import A
}
:80
import A
`, true, [][]string{}},
} {
p := testParser(test.input)
blocks, err := p.parseAll()
@@ -474,6 +517,7 @@ func TestParseAll(t *testing.T) {
func TestEnvironmentReplacement(t *testing.T) {
os.Setenv("FOOBAR", "foobar")
os.Setenv("CHAINED", "$FOOBAR")
for i, test := range []struct {
input string
@@ -519,6 +563,22 @@ func TestEnvironmentReplacement(t *testing.T) {
input: "{$FOOBAR}{$FOOBAR}",
expect: "foobarfoobar",
},
{
input: "{$CHAINED}",
expect: "$FOOBAR", // should not chain env expands
},
{
input: "{$FOO:default}",
expect: "default",
},
{
input: "foo{$BAR:bar}baz",
expect: "foobarbaz",
},
{
input: "foo{$BAR:$FOOBAR}baz",
expect: "foo$FOOBARbaz", // should not chain env expands
},
{
input: "{$FOOBAR",
expect: "{$FOOBAR",
@@ -586,7 +646,7 @@ func TestSnippets(t *testing.T) {
}
func writeStringToTempFileOrDie(t *testing.T, str string) (pathToFile string) {
file, err := ioutil.TempFile("", t.Name())
file, err := os.CreateTemp("", t.Name())
if err != nil {
panic(err) // get a stack trace so we know where this was called from.
}
+1
View File
@@ -0,0 +1 @@
{args.0}
+1
View File
@@ -0,0 +1 @@
{args.0} {args.1}
View File
View File
View File
+1
View File
@@ -0,0 +1 @@
import import_recursive0.txt
+1
View File
@@ -0,0 +1 @@
import import_recursive2.txt
+1
View File
@@ -0,0 +1 @@
import import_recursive3.txt
+1
View File
@@ -0,0 +1 @@
import import_recursive1.txt
View File
View File
+19 -18
View File
@@ -24,7 +24,7 @@ import (
// Adapter is a type which can adapt a configuration to Caddy JSON.
// It returns the results and any warnings, or an error.
type Adapter interface {
Adapt(body []byte, options map[string]interface{}) ([]byte, []Warning, error)
Adapt(body []byte, options map[string]any) ([]byte, []Warning, error)
}
// Warning represents a warning or notice related to conversion.
@@ -35,12 +35,20 @@ type Warning struct {
Message string `json:"message,omitempty"`
}
func (w Warning) String() string {
var directive string
if w.Directive != "" {
directive = fmt.Sprintf(" (%s)", w.Directive)
}
return fmt.Sprintf("%s:%d%s: %s", w.File, w.Line, directive, w.Message)
}
// JSON encodes val as JSON, returning it as a json.RawMessage. Any
// marshaling errors (which are highly unlikely with correct code)
// are converted to warnings. This is convenient when filling config
// structs that require a json.RawMessage, without having to worry
// about errors.
func JSON(val interface{}, warnings *[]Warning) json.RawMessage {
func JSON(val any, warnings *[]Warning) json.RawMessage {
b, err := json.Marshal(val)
if err != nil {
if warnings != nil {
@@ -51,15 +59,14 @@ func JSON(val interface{}, warnings *[]Warning) json.RawMessage {
return b
}
// JSONModuleObject is like JSON, except it marshals val into a JSON object
// and then adds a key to that object named fieldName with the value fieldVal.
// This is useful for JSON-encoding module values where the module name has to
// be described within the object by a certain key; for example,
// "responder": "file_server" for a file server HTTP responder. The val must
// encode into a map[string]interface{} (i.e. it must be a struct or map),
// and any errors are converted into warnings, so this can be conveniently
// used when filling a struct. For correct code, there should be no errors.
func JSONModuleObject(val interface{}, fieldName, fieldVal string, warnings *[]Warning) json.RawMessage {
// JSONModuleObject is like JSON(), except it marshals val into a JSON object
// with an added key named fieldName with the value fieldVal. This is useful
// for encoding module values where the module name has to be described within
// the object by a certain key; for example, `"handler": "file_server"` for a
// file server HTTP handler (fieldName="handler" and fieldVal="file_server").
// The val parameter must encode into a map[string]any (i.e. it must be
// a struct or map). Any errors are converted into warnings.
func JSONModuleObject(val any, fieldName, fieldVal string, warnings *[]Warning) json.RawMessage {
// encode to a JSON object first
enc, err := json.Marshal(val)
if err != nil {
@@ -70,7 +77,7 @@ func JSONModuleObject(val interface{}, fieldName, fieldVal string, warnings *[]W
}
// then decode the object
var tmp map[string]interface{}
var tmp map[string]any
err = json.Unmarshal(enc, &tmp)
if err != nil {
if warnings != nil {
@@ -94,12 +101,6 @@ func JSONModuleObject(val interface{}, fieldName, fieldVal string, warnings *[]W
return result
}
// JSONIndent is used to JSON-marshal the final resulting Caddy
// configuration in a consistent, human-readable way.
func JSONIndent(val interface{}) ([]byte, error) {
return json.MarshalIndent(val, "", "\t")
}
// RegisterAdapter registers a config adapter with the given name.
// This should usually be done at init-time. It panics if the
// adapter cannot be registered successfully.
+61 -42
View File
@@ -17,7 +17,9 @@ package httpcaddyfile
import (
"fmt"
"net"
"net/netip"
"reflect"
"sort"
"strconv"
"strings"
"unicode"
@@ -34,12 +36,12 @@ import (
// server block that share the same address stay grouped together so the config
// isn't repeated unnecessarily. For example, this Caddyfile:
//
// example.com {
// bind 127.0.0.1
// }
// www.example.com, example.net/path, localhost:9999 {
// bind 127.0.0.1 1.2.3.4
// }
// example.com {
// bind 127.0.0.1
// }
// www.example.com, example.net/path, localhost:9999 {
// bind 127.0.0.1 1.2.3.4
// }
//
// has two server blocks to start with. But expressed in this Caddyfile are
// actually 4 listener addresses: 127.0.0.1:443, 1.2.3.4:443, 127.0.0.1:9999,
@@ -75,7 +77,7 @@ import (
// multiple addresses to the same lists of server blocks (a many:many mapping).
// (Doing this is essentially a map-reduce technique.)
func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBlock,
options map[string]interface{}) (map[string][]serverBlock, error) {
options map[string]any) (map[string][]serverBlock, error) {
sbmap := make(map[string][]serverBlock)
for i, sblock := range originalServerBlocks {
@@ -101,12 +103,20 @@ func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBloc
}
}
// make a slice of the map keys so we can iterate in sorted order
addrs := make([]string, 0, len(addrToKeys))
for k := range addrToKeys {
addrs = append(addrs, k)
}
sort.Strings(addrs)
// now that we know which addresses serve which keys of this
// server block, we iterate that mapping and create a list of
// new server blocks for each address where the keys of the
// server block are only the ones which use the address; but
// the contents (tokens) are of course the same
for addr, keys := range addrToKeys {
for _, addr := range addrs {
keys := addrToKeys[addr]
// parse keys so that we only have to do it once
parsedKeys := make([]Address, 0, len(keys))
for _, key := range keys {
@@ -160,14 +170,24 @@ func (st *ServerType) consolidateAddrMappings(addrToServerBlocks map[string][]se
delete(addrToServerBlocks, otherAddr)
}
}
sort.Strings(a.addresses)
sbaddrs = append(sbaddrs, a)
}
// sort them by their first address (we know there will always be at least one)
// to avoid problems with non-deterministic ordering (makes tests flaky)
sort.Slice(sbaddrs, func(i, j int) bool {
return sbaddrs[i].addresses[0] < sbaddrs[j].addresses[0]
})
return sbaddrs
}
// listenerAddrsForServerBlockKey essentially converts the Caddyfile
// site addresses to Caddy listener addresses for each server block.
func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key string,
options map[string]interface{}) ([]string, error) {
options map[string]any) ([]string, error) {
addr, err := ParseAddress(key)
if err != nil {
return nil, fmt.Errorf("parsing key: %v", err)
@@ -199,24 +219,42 @@ func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key str
return nil, fmt.Errorf("[%s] scheme and port violate convention", key)
}
// the bind directive specifies hosts, but is optional
lnHosts := make([]string, 0, len(sblock.pile))
// the bind directive specifies hosts (and potentially network), but is optional
lnHosts := make([]string, 0, len(sblock.pile["bind"]))
for _, cfgVal := range sblock.pile["bind"] {
lnHosts = append(lnHosts, cfgVal.Value.([]string)...)
}
if len(lnHosts) == 0 {
lnHosts = []string{""}
if defaultBind, ok := options["default_bind"].([]string); ok {
lnHosts = defaultBind
} else {
lnHosts = []string{""}
}
}
// use a map to prevent duplication
listeners := make(map[string]struct{})
for _, host := range lnHosts {
addr, err := caddy.ParseNetworkAddress(host)
if err == nil && addr.IsUnixNetwork() {
listeners[host] = struct{}{}
} else {
listeners[net.JoinHostPort(host, lnPort)] = struct{}{}
for _, lnHost := range lnHosts {
// normally we would simply append the port,
// but if lnHost is IPv6, we need to ensure it
// is enclosed in [ ]; net.JoinHostPort does
// this for us, but lnHost might also have a
// network type in front (e.g. "tcp/") leading
// to "[tcp/::1]" which causes parsing failures
// later; what we need is "tcp/[::1]", so we have
// to split the network and host, then re-combine
network, host, ok := strings.Cut(lnHost, "/")
if !ok {
host = network
network = ""
}
host = strings.Trim(host, "[]") // IPv6
networkAddr := caddy.JoinNetworkAddress(network, host, lnPort)
addr, err := caddy.ParseNetworkAddress(networkAddr)
if err != nil {
return nil, fmt.Errorf("parsing network address: %v", err)
}
listeners[addr.String()] = struct{}{}
}
// now turn map into list
@@ -224,6 +262,7 @@ func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key str
for lnStr := range listeners {
listenersList = append(listenersList, lnStr)
}
sort.Strings(listenersList)
return listenersList, nil
}
@@ -328,8 +367,10 @@ func (a Address) Normalize() Address {
// ensure host is normalized if it's an IP address
host := strings.TrimSpace(a.Host)
if ip := net.ParseIP(host); ip != nil {
host = ip.String()
if ip, err := netip.ParseAddr(host); err == nil {
if ip.Is6() && !ip.Is4() && !ip.Is4In6() {
host = ip.String()
}
}
return Address{
@@ -341,28 +382,6 @@ func (a Address) Normalize() Address {
}
}
// Key returns a string form of a, much like String() does, but this
// method doesn't add anything default that wasn't in the original.
func (a Address) Key() string {
res := ""
if a.Scheme != "" {
res += a.Scheme + "://"
}
if a.Host != "" {
res += a.Host
}
// insert port only if the original has its own explicit port
if a.Port != "" &&
len(a.Original) >= len(res) &&
strings.HasPrefix(a.Original[len(res):], ":"+a.Port) {
res += ":" + a.Port
}
if a.Path != "" {
res += a.Path
}
return res
}
// lowerExceptPlaceholders lowercases s except within
// placeholders (substrings in non-escaped '{ }' spans).
// See https://github.com/caddyserver/caddy/issues/3264
+1 -1
View File
@@ -12,7 +12,7 @@
// See the License for the specific language governing permissions and
// limitations under the License.
// +build gofuzz
//go:build gofuzz
package httpcaddyfile
+102 -32
View File
@@ -106,67 +106,128 @@ func TestAddressString(t *testing.T) {
func TestKeyNormalization(t *testing.T) {
testCases := []struct {
input string
expect string
expect Address
}{
{
input: "example.com",
expect: "example.com",
input: "example.com",
expect: Address{
Host: "example.com",
},
},
{
input: "http://host:1234/path",
expect: "http://host:1234/path",
input: "http://host:1234/path",
expect: Address{
Scheme: "http",
Host: "host",
Port: "1234",
Path: "/path",
},
},
{
input: "HTTP://A/ABCDEF",
expect: "http://a/ABCDEF",
input: "HTTP://A/ABCDEF",
expect: Address{
Scheme: "http",
Host: "a",
Path: "/ABCDEF",
},
},
{
input: "A/ABCDEF",
expect: "a/ABCDEF",
input: "A/ABCDEF",
expect: Address{
Host: "a",
Path: "/ABCDEF",
},
},
{
input: "A:2015/Path",
expect: "a:2015/Path",
input: "A:2015/Path",
expect: Address{
Host: "a",
Port: "2015",
Path: "/Path",
},
},
{
input: "sub.{env.MY_DOMAIN}",
expect: "sub.{env.MY_DOMAIN}",
input: "sub.{env.MY_DOMAIN}",
expect: Address{
Host: "sub.{env.MY_DOMAIN}",
},
},
{
input: "sub.ExAmPle",
expect: "sub.example",
input: "sub.ExAmPle",
expect: Address{
Host: "sub.example",
},
},
{
input: "sub.\\{env.MY_DOMAIN\\}",
expect: "sub.\\{env.my_domain\\}",
input: "sub.\\{env.MY_DOMAIN\\}",
expect: Address{
Host: "sub.\\{env.my_domain\\}",
},
},
{
input: "sub.{env.MY_DOMAIN}.com",
expect: "sub.{env.MY_DOMAIN}.com",
input: "sub.{env.MY_DOMAIN}.com",
expect: Address{
Host: "sub.{env.MY_DOMAIN}.com",
},
},
{
input: ":80",
expect: ":80",
input: ":80",
expect: Address{
Port: "80",
},
},
{
input: ":443",
expect: ":443",
input: ":443",
expect: Address{
Port: "443",
},
},
{
input: ":1234",
expect: ":1234",
input: ":1234",
expect: Address{
Port: "1234",
},
},
{
input: "",
expect: "",
expect: Address{},
},
{
input: ":",
expect: "",
expect: Address{},
},
{
input: "[::]",
expect: "::",
input: "[::]",
expect: Address{
Host: "::",
},
},
{
input: "127.0.0.1",
expect: Address{
Host: "127.0.0.1",
},
},
{
input: "[2001:db8:85a3:8d3:1319:8a2e:370:7348]:1234",
expect: Address{
Host: "2001:db8:85a3:8d3:1319:8a2e:370:7348",
Port: "1234",
},
},
{
// IPv4 address in IPv6 form (#4381)
input: "[::ffff:cff4:e77d]:1234",
expect: Address{
Host: "::ffff:cff4:e77d",
Port: "1234",
},
},
{
input: "::ffff:cff4:e77d",
expect: Address{
Host: "::ffff:cff4:e77d",
},
},
}
for i, tc := range testCases {
@@ -175,9 +236,18 @@ func TestKeyNormalization(t *testing.T) {
t.Errorf("Test %d: Parsing address '%s': %v", i, tc.input, err)
continue
}
if actual := addr.Normalize().Key(); actual != tc.expect {
t.Errorf("Test %d: Input '%s': Expected '%s' but got '%s'", i, tc.input, tc.expect, actual)
actual := addr.Normalize()
if actual.Scheme != tc.expect.Scheme {
t.Errorf("Test %d: Input '%s': Expected Scheme='%s' but got Scheme='%s'", i, tc.input, tc.expect.Scheme, actual.Scheme)
}
if actual.Host != tc.expect.Host {
t.Errorf("Test %d: Input '%s': Expected Host='%s' but got Host='%s'", i, tc.input, tc.expect.Host, actual.Host)
}
if actual.Port != tc.expect.Port {
t.Errorf("Test %d: Input '%s': Expected Port='%s' but got Port='%s'", i, tc.input, tc.expect.Port, actual.Port)
}
if actual.Path != tc.expect.Path {
t.Errorf("Test %d: Input '%s': Expected Path='%s' but got Path='%s'", i, tc.input, tc.expect.Path, actual.Path)
}
}
}
+433 -126
View File
@@ -15,10 +15,14 @@
package httpcaddyfile
import (
"encoding/base64"
"encoding/pem"
"fmt"
"html"
"net/http"
"os"
"reflect"
"strconv"
"strings"
"github.com/caddyserver/caddy/v2"
@@ -26,6 +30,8 @@ import (
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/caddyserver/caddy/v2/modules/caddytls"
"github.com/caddyserver/certmagic"
"github.com/mholt/acmez/acme"
"go.uber.org/zap/zapcore"
)
@@ -33,18 +39,21 @@ func init() {
RegisterDirective("bind", parseBind)
RegisterDirective("tls", parseTLS)
RegisterHandlerDirective("root", parseRoot)
RegisterHandlerDirective("vars", parseVars)
RegisterHandlerDirective("redir", parseRedir)
RegisterHandlerDirective("respond", parseRespond)
RegisterHandlerDirective("abort", parseAbort)
RegisterHandlerDirective("error", parseError)
RegisterHandlerDirective("route", parseRoute)
RegisterHandlerDirective("handle", parseHandle)
RegisterDirective("handle_errors", parseHandleErrors)
RegisterDirective("log", parseLog)
RegisterHandlerDirective("skip_log", parseSkipLog)
}
// parseBind parses the bind directive. Syntax:
//
// bind <addresses...>
//
// bind <addresses...>
func parseBind(h Helper) ([]ConfigValue, error) {
var lnHosts []string
for h.Next() {
@@ -55,25 +64,38 @@ func parseBind(h Helper) ([]ConfigValue, error) {
// parseTLS parses the tls directive. Syntax:
//
// tls [<email>|internal]|[<cert_file> <key_file>] {
// protocols <min> [<max>]
// ciphers <cipher_suites...>
// curves <curves...>
// alpn <values...>
// load <paths...>
// ca <acme_ca_endpoint>
// ca_root <pem_file>
// dns <provider_name>
// on_demand
// }
//
// tls [<email>|internal]|[<cert_file> <key_file>] {
// protocols <min> [<max>]
// ciphers <cipher_suites...>
// curves <curves...>
// client_auth {
// mode [request|require|verify_if_given|require_and_verify]
// trusted_ca_cert <base64_der>
// trusted_ca_cert_file <filename>
// trusted_leaf_cert <base64_der>
// trusted_leaf_cert_file <filename>
// }
// alpn <values...>
// load <paths...>
// ca <acme_ca_endpoint>
// ca_root <pem_file>
// dns <provider_name> [...]
// on_demand
// eab <key_id> <mac_key>
// issuer <module_name> [...]
// get_certificate <module_name> [...]
// insecure_secrets_log <log_file>
// }
func parseTLS(h Helper) ([]ConfigValue, error) {
cp := new(caddytls.ConnectionPolicy)
var fileLoader caddytls.FileLoader
var folderLoader caddytls.FolderLoader
var certSelector caddytls.CustomCertSelectionPolicy
var acmeIssuer *caddytls.ACMEIssuer
var keyType string
var internalIssuer *caddytls.InternalIssuer
var issuers []certmagic.Issuer
var certManagers []certmagic.Manager
var onDemand bool
for h.Next() {
@@ -107,10 +129,10 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
// must load each cert only once; otherwise, they each get a
// different tag... since a cert loaded twice has the same
// bytes, it will overwrite the first one in the cache, and
// only the last cert (and its tag) will survive, so a any conn
// policy that is looking for any tag but the last one to be
// loaded won't find it, and TLS handshakes will fail (see end)
// of issue #3004)
// only the last cert (and its tag) will survive, so any conn
// policy that is looking for any tag other than the last one
// to be loaded won't find it, and TLS handshakes will fail
// (see end of issue #3004)
//
// tlsCertTags maps certificate filenames to their tag.
// This is used to remember which tag is used for each
@@ -143,7 +165,7 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
}
var hasBlock bool
for h.NextBlock(0) {
for nesting := h.Nesting(); h.NextBlock(nesting); {
hasBlock = true
switch h.Val() {
@@ -181,6 +203,57 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
cp.Curves = append(cp.Curves, h.Val())
}
case "client_auth":
cp.ClientAuthentication = &caddytls.ClientAuthentication{}
for nesting := h.Nesting(); h.NextBlock(nesting); {
subdir := h.Val()
switch subdir {
case "mode":
if !h.Args(&cp.ClientAuthentication.Mode) {
return nil, h.ArgErr()
}
if h.NextArg() {
return nil, h.ArgErr()
}
case "trusted_ca_cert",
"trusted_leaf_cert":
if !h.NextArg() {
return nil, h.ArgErr()
}
if subdir == "trusted_ca_cert" {
cp.ClientAuthentication.TrustedCACerts = append(cp.ClientAuthentication.TrustedCACerts, h.Val())
} else {
cp.ClientAuthentication.TrustedLeafCerts = append(cp.ClientAuthentication.TrustedLeafCerts, h.Val())
}
case "trusted_ca_cert_file",
"trusted_leaf_cert_file":
if !h.NextArg() {
return nil, h.ArgErr()
}
filename := h.Val()
certDataPEM, err := os.ReadFile(filename)
if err != nil {
return nil, err
}
block, _ := pem.Decode(certDataPEM)
if block == nil || block.Type != "CERTIFICATE" {
return nil, h.Errf("no CERTIFICATE pem block found in %s", h.Val())
}
if subdir == "trusted_ca_cert_file" {
cp.ClientAuthentication.TrustedCACerts = append(cp.ClientAuthentication.TrustedCACerts,
base64.StdEncoding.EncodeToString(block.Bytes))
} else {
cp.ClientAuthentication.TrustedLeafCerts = append(cp.ClientAuthentication.TrustedLeafCerts,
base64.StdEncoding.EncodeToString(block.Bytes))
}
default:
return nil, h.Errf("unknown subdirective for client_auth: %s", subdir)
}
}
case "alpn":
args := h.RemainingArgs()
if len(args) == 0 {
@@ -201,6 +274,58 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
}
acmeIssuer.CA = arg[0]
case "key_type":
arg := h.RemainingArgs()
if len(arg) != 1 {
return nil, h.ArgErr()
}
keyType = arg[0]
case "eab":
arg := h.RemainingArgs()
if len(arg) != 2 {
return nil, h.ArgErr()
}
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
acmeIssuer.ExternalAccount = &acme.EAB{
KeyID: arg[0],
MACKey: arg[1],
}
case "issuer":
if !h.NextArg() {
return nil, h.ArgErr()
}
modName := h.Val()
modID := "tls.issuance." + modName
unm, err := caddyfile.UnmarshalModule(h.Dispenser, modID)
if err != nil {
return nil, err
}
issuer, ok := unm.(certmagic.Issuer)
if !ok {
return nil, h.Errf("module %s (%T) is not a certmagic.Issuer", modID, unm)
}
issuers = append(issuers, issuer)
case "get_certificate":
if !h.NextArg() {
return nil, h.ArgErr()
}
modName := h.Val()
modID := "tls.get_certificate." + modName
unm, err := caddyfile.UnmarshalModule(h.Dispenser, modID)
if err != nil {
return nil, err
}
certManager, ok := unm.(certmagic.Manager)
if !ok {
return nil, h.Errf("module %s (%T) is not a certmagic.CertificateManager", modID, unm)
}
certManagers = append(certManagers, certManager)
case "dns":
if !h.NextArg() {
return nil, h.ArgErr()
@@ -211,20 +336,48 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
}
if acmeIssuer.Challenges == nil {
acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
}
if acmeIssuer.Challenges.DNS == nil {
acmeIssuer.Challenges.DNS = new(caddytls.DNSChallengeConfig)
}
dnsProvModule, err := caddy.GetModule("dns.providers." + provName)
modID := "dns.providers." + provName
unm, err := caddyfile.UnmarshalModule(h.Dispenser, modID)
if err != nil {
return nil, h.Errf("getting DNS provider module named '%s': %v", provName, err)
return nil, err
}
dnsProvModuleInstance := dnsProvModule.New()
if unm, ok := dnsProvModuleInstance.(caddyfile.Unmarshaler); ok {
err = unm.UnmarshalCaddyfile(h.NewFromNextSegment())
if err != nil {
return nil, err
}
acmeIssuer.Challenges.DNS.ProviderRaw = caddyconfig.JSONModuleObject(unm, "name", provName, h.warnings)
case "resolvers":
args := h.RemainingArgs()
if len(args) == 0 {
return nil, h.ArgErr()
}
acmeIssuer.Challenges.DNS.ProviderRaw = caddyconfig.JSONModuleObject(dnsProvModuleInstance, "name", provName, h.warnings)
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
if acmeIssuer.Challenges == nil {
acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
}
if acmeIssuer.Challenges.DNS == nil {
acmeIssuer.Challenges.DNS = new(caddytls.DNSChallengeConfig)
}
acmeIssuer.Challenges.DNS.Resolvers = args
case "dns_challenge_override_domain":
arg := h.RemainingArgs()
if len(arg) != 1 {
return nil, h.ArgErr()
}
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
if acmeIssuer.Challenges == nil {
acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
}
if acmeIssuer.Challenges.DNS == nil {
acmeIssuer.Challenges.DNS = new(caddytls.DNSChallengeConfig)
}
acmeIssuer.Challenges.DNS.OverrideDomain = arg[0]
case "ca_root":
arg := h.RemainingArgs()
@@ -242,6 +395,12 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
}
onDemand = true
case "insecure_secrets_log":
if !h.NextArg() {
return nil, h.ArgErr()
}
cp.InsecureSecretsLog = h.Val()
default:
return nil, h.Errf("unknown subdirective: %s", h.Val())
}
@@ -254,50 +413,80 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
}
// begin building the final config values
var configVals []ConfigValue
configVals := []ConfigValue{}
// certificate loaders
if len(fileLoader) > 0 {
configVals = append(configVals, ConfigValue{
Class: "tls.certificate_loader",
Class: "tls.cert_loader",
Value: fileLoader,
})
}
if len(folderLoader) > 0 {
configVals = append(configVals, ConfigValue{
Class: "tls.certificate_loader",
Class: "tls.cert_loader",
Value: folderLoader,
})
}
// issuer
if acmeIssuer != nil && internalIssuer != nil {
// the logic to support this would be complex
return nil, h.Err("cannot use both ACME and internal issuers in same server block")
// some tls subdirectives are shortcuts that implicitly configure issuers, and the
// user can also configure issuers explicitly using the issuer subdirective; the
// logic to support both would likely be complex, or at least unintuitive
if len(issuers) > 0 && (acmeIssuer != nil || internalIssuer != nil) {
return nil, h.Err("cannot mix issuer subdirective (explicit issuers) with other issuer-specific subdirectives (implicit issuers)")
}
if acmeIssuer != nil {
// fill in global defaults, if configured
if email := h.Option("email"); email != nil && acmeIssuer.Email == "" {
acmeIssuer.Email = email.(string)
}
if acmeCA := h.Option("acme_ca"); acmeCA != nil && acmeIssuer.CA == "" {
acmeIssuer.CA = acmeCA.(string)
}
if caPemFile := h.Option("acme_ca_root"); caPemFile != nil {
acmeIssuer.TrustedRootsPEMFiles = append(acmeIssuer.TrustedRootsPEMFiles, caPemFile.(string))
if acmeIssuer != nil && internalIssuer != nil {
return nil, h.Err("cannot create both ACME and internal certificate issuers")
}
// now we should either have: explicitly-created issuers, or an implicitly-created
// ACME or internal issuer, or no issuers at all
switch {
case len(issuers) > 0:
for _, issuer := range issuers {
configVals = append(configVals, ConfigValue{
Class: "tls.cert_issuer",
Value: issuer,
})
}
configVals = append(configVals, ConfigValue{
Class: "tls.cert_issuer",
Value: acmeIssuer,
})
} else if internalIssuer != nil {
case acmeIssuer != nil:
// implicit ACME issuers (from various subdirectives) - use defaults; there might be more than one
defaultIssuers := caddytls.DefaultIssuers()
// if a CA endpoint was set, override multiple implicit issuers since it's a specific one
if acmeIssuer.CA != "" {
defaultIssuers = []certmagic.Issuer{acmeIssuer}
}
for _, issuer := range defaultIssuers {
switch iss := issuer.(type) {
case *caddytls.ACMEIssuer:
issuer = acmeIssuer
case *caddytls.ZeroSSLIssuer:
iss.ACMEIssuer = acmeIssuer
}
configVals = append(configVals, ConfigValue{
Class: "tls.cert_issuer",
Value: issuer,
})
}
case internalIssuer != nil:
configVals = append(configVals, ConfigValue{
Class: "tls.cert_issuer",
Value: internalIssuer,
})
}
// certificate key type
if keyType != "" {
configVals = append(configVals, ConfigValue{
Class: "tls.key_type",
Value: keyType,
})
}
// on-demand TLS
if onDemand {
configVals = append(configVals, ConfigValue{
@@ -305,6 +494,12 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
Value: true,
})
}
for _, certManager := range certManagers {
configVals = append(configVals, ConfigValue{
Class: "tls.cert_manager",
Value: certManager,
})
}
// custom certificate selection
if len(certSelector.AnyTag) > 0 {
@@ -326,8 +521,7 @@ func parseTLS(h Helper) ([]ConfigValue, error) {
// parseRoot parses the root directive. Syntax:
//
// root [<matcher>] <path>
//
// root [<matcher>] <path>
func parseRoot(h Helper) (caddyhttp.MiddlewareHandler, error) {
var root string
for h.Next() {
@@ -342,10 +536,22 @@ func parseRoot(h Helper) (caddyhttp.MiddlewareHandler, error) {
return caddyhttp.VarsMiddleware{"root": root}, nil
}
// parseVars parses the vars directive. See its UnmarshalCaddyfile method for syntax.
func parseVars(h Helper) (caddyhttp.MiddlewareHandler, error) {
v := new(caddyhttp.VarsMiddleware)
err := v.UnmarshalCaddyfile(h.Dispenser)
return v, err
}
// parseRedir parses the redir directive. Syntax:
//
// redir [<matcher>] <to> [<code>]
// redir [<matcher>] <to> [<code>]
//
// <code> can be "permanent" for 301, "temporary" for 302 (default),
// a placeholder, or any number in the 3xx range or 401. The special
// code "html" can be used to redirect only browser clients (will
// respond with HTTP 200 and no Location header; redirect is performed
// with JS and a meta tag).
func parseRedir(h Helper) (caddyhttp.MiddlewareHandler, error) {
if !h.Next() {
return nil, h.ArgErr()
@@ -360,14 +566,15 @@ func parseRedir(h Helper) (caddyhttp.MiddlewareHandler, error) {
if h.NextArg() {
code = h.Val()
}
if code == "permanent" {
code = "301"
}
if code == "temporary" || code == "" {
code = "302"
}
var body string
if code == "html" {
var hdr http.Header
switch code {
case "permanent":
code = "301"
case "temporary", "":
code = "302"
case "html":
// Script tag comes first since that will better imitate a redirect in the browser's
// history, but the meta tag is a fallback for most non-JS clients.
const metaRedir = `<!DOCTYPE html>
@@ -382,11 +589,37 @@ func parseRedir(h Helper) (caddyhttp.MiddlewareHandler, error) {
`
safeTo := html.EscapeString(to)
body = fmt.Sprintf(metaRedir, safeTo, safeTo, safeTo, safeTo)
code = "200" // don't redirect non-browser clients
default:
// Allow placeholders for the code
if strings.HasPrefix(code, "{") {
break
}
// Try to validate as an integer otherwise
codeInt, err := strconv.Atoi(code)
if err != nil {
return nil, h.Errf("Not a supported redir code type or not valid integer: '%s'", code)
}
// Sometimes, a 401 with Location header is desirable because
// requests made with XHR will "eat" the 3xx redirect; so if
// the intent was to redirect to an auth page, a 3xx won't
// work. Responding with 401 allows JS code to read the
// Location header and do a window.location redirect manually.
// see https://stackoverflow.com/a/2573589/846934
// see https://github.com/oauth2-proxy/oauth2-proxy/issues/1522
if codeInt < 300 || (codeInt > 399 && codeInt != 401) {
return nil, h.Errf("Redir code not in the 3xx range or 401: '%v'", codeInt)
}
}
// don't redirect non-browser clients
if code != "200" {
hdr = http.Header{"Location": []string{to}}
}
return caddyhttp.StaticResponse{
StatusCode: caddyhttp.WeakString(code),
Headers: http.Header{"Location": []string{to}},
Headers: hdr,
Body: body,
}, nil
}
@@ -401,40 +634,46 @@ func parseRespond(h Helper) (caddyhttp.MiddlewareHandler, error) {
return sr, nil
}
// parseAbort parses the abort directive.
func parseAbort(h Helper) (caddyhttp.MiddlewareHandler, error) {
h.Next() // consume directive
for h.Next() || h.NextBlock(0) {
return nil, h.ArgErr()
}
return &caddyhttp.StaticResponse{Abort: true}, nil
}
// parseError parses the error directive.
func parseError(h Helper) (caddyhttp.MiddlewareHandler, error) {
se := new(caddyhttp.StaticError)
err := se.UnmarshalCaddyfile(h.Dispenser)
if err != nil {
return nil, err
}
return se, nil
}
// parseRoute parses the route directive.
func parseRoute(h Helper) (caddyhttp.MiddlewareHandler, error) {
sr := new(caddyhttp.Subroute)
for h.Next() {
for nesting := h.Nesting(); h.NextBlock(nesting); {
dir := h.Val()
allResults, err := parseSegmentAsConfig(h)
if err != nil {
return nil, err
}
dirFunc, ok := registeredDirectives[dir]
if !ok {
return nil, h.Errf("unrecognized directive: %s", dir)
}
subHelper := h
subHelper.Dispenser = h.NewFromNextSegment()
results, err := dirFunc(subHelper)
if err != nil {
return nil, h.Errf("parsing caddyfile tokens for '%s': %v", dir, err)
}
for _, result := range results {
switch handler := result.Value.(type) {
case caddyhttp.Route:
sr.Routes = append(sr.Routes, handler)
case caddyhttp.Subroute:
// directives which return a literal subroute instead of a route
// means they intend to keep those handlers together without
// them being reordered; we're doing that anyway since we're in
// the route directive, so just append its handlers
sr.Routes = append(sr.Routes, handler.Routes...)
default:
return nil, h.Errf("%s directive returned something other than an HTTP route or subroute: %#v (only handler directives can be used in routes)", dir, result.Value)
}
}
for _, result := range allResults {
switch handler := result.Value.(type) {
case caddyhttp.Route:
sr.Routes = append(sr.Routes, handler)
case caddyhttp.Subroute:
// directives which return a literal subroute instead of a route
// means they intend to keep those handlers together without
// them being reordered; we're doing that anyway since we're in
// the route directive, so just append its handlers
sr.Routes = append(sr.Routes, handler.Routes...)
default:
return nil, h.Errf("%s directive returned something other than an HTTP route or subroute: %#v (only handler directives can be used in routes)", result.directive, result.Value)
}
}
@@ -442,11 +681,11 @@ func parseRoute(h Helper) (caddyhttp.MiddlewareHandler, error) {
}
func parseHandle(h Helper) (caddyhttp.MiddlewareHandler, error) {
return parseSegmentAsSubroute(h)
return ParseSegmentAsSubroute(h)
}
func parseHandleErrors(h Helper) ([]ConfigValue, error) {
subroute, err := parseSegmentAsSubroute(h)
subroute, err := ParseSegmentAsSubroute(h)
if err != nil {
return nil, err
}
@@ -460,15 +699,58 @@ func parseHandleErrors(h Helper) ([]ConfigValue, error) {
// parseLog parses the log directive. Syntax:
//
// log {
// output <writer_module> ...
// format <encoder_module> ...
// level <level>
// }
//
// log {
// output <writer_module> ...
// format <encoder_module> ...
// level <level>
// }
func parseLog(h Helper) ([]ConfigValue, error) {
return parseLogHelper(h, nil)
}
// parseLogHelper is used both for the parseLog directive within Server Blocks,
// as well as the global "log" option for configuring loggers at the global
// level. The parseAsGlobalOption parameter is used to distinguish any differing logic
// between the two.
func parseLogHelper(h Helper, globalLogNames map[string]struct{}) ([]ConfigValue, error) {
// When the globalLogNames parameter is passed in, we make
// modifications to the parsing behavior.
parseAsGlobalOption := globalLogNames != nil
var configValues []ConfigValue
for h.Next() {
// Logic below expects that a name is always present when a
// global option is being parsed.
var globalLogName string
if parseAsGlobalOption {
if h.NextArg() {
globalLogName = h.Val()
// Only a single argument is supported.
if h.NextArg() {
return nil, h.ArgErr()
}
} else {
// If there is no log name specified, we
// reference the default logger. See the
// setupNewDefault function in the logging
// package for where this is configured.
globalLogName = caddy.DefaultLoggerName
}
// Verify this name is unused.
_, used := globalLogNames[globalLogName]
if used {
return nil, h.Err("duplicate global log option for: " + globalLogName)
}
globalLogNames[globalLogName] = struct{}{}
} else {
// No arguments are supported for the server block log directive
if h.NextArg() {
return nil, h.ArgErr()
}
}
cl := new(caddy.CustomLog)
for h.NextBlock(0) {
@@ -492,21 +774,15 @@ func parseLog(h Helper) ([]ConfigValue, error) {
case "discard":
wo = caddy.DiscardWriter{}
default:
mod, err := caddy.GetModule("caddy.logging.writers." + moduleName)
if err != nil {
return nil, h.Errf("getting log writer module named '%s': %v", moduleName, err)
}
unm, ok := mod.New().(caddyfile.Unmarshaler)
if !ok {
return nil, h.Errf("log writer module '%s' is not a Caddyfile unmarshaler", mod)
}
err = unm.UnmarshalCaddyfile(h.NewFromNextSegment())
modID := "caddy.logging.writers." + moduleName
unm, err := caddyfile.UnmarshalModule(h.Dispenser, modID)
if err != nil {
return nil, err
}
var ok bool
wo, ok = unm.(caddy.WriterOpener)
if !ok {
return nil, h.Errf("module %s is not a WriterOpener", mod)
return nil, h.Errf("module %s (%T) is not a WriterOpener", modID, unm)
}
}
cl.WriterRaw = caddyconfig.JSONModuleObject(wo, "output", moduleName, h.warnings)
@@ -516,21 +792,14 @@ func parseLog(h Helper) ([]ConfigValue, error) {
return nil, h.ArgErr()
}
moduleName := h.Val()
mod, err := caddy.GetModule("caddy.logging.encoders." + moduleName)
if err != nil {
return nil, h.Errf("getting log encoder module named '%s': %v", moduleName, err)
}
unm, ok := mod.New().(caddyfile.Unmarshaler)
if !ok {
return nil, h.Errf("log encoder module '%s' is not a Caddyfile unmarshaler", mod)
}
err = unm.UnmarshalCaddyfile(h.NewFromNextSegment())
moduleID := "caddy.logging.encoders." + moduleName
unm, err := caddyfile.UnmarshalModule(h.Dispenser, moduleID)
if err != nil {
return nil, err
}
enc, ok := unm.(zapcore.Encoder)
if !ok {
return nil, h.Errf("module %s is not a zapcore.Encoder", mod)
return nil, h.Errf("module %s (%T) is not a zapcore.Encoder", moduleID, unm)
}
cl.EncoderRaw = caddyconfig.JSONModuleObject(enc, "format", moduleName, h.warnings)
@@ -543,22 +812,48 @@ func parseLog(h Helper) ([]ConfigValue, error) {
return nil, h.ArgErr()
}
case "include":
// This configuration is only allowed in the global options
if !parseAsGlobalOption {
return nil, h.ArgErr()
}
for h.NextArg() {
cl.Include = append(cl.Include, h.Val())
}
case "exclude":
// This configuration is only allowed in the global options
if !parseAsGlobalOption {
return nil, h.ArgErr()
}
for h.NextArg() {
cl.Exclude = append(cl.Exclude, h.Val())
}
default:
return nil, h.Errf("unrecognized subdirective: %s", h.Val())
}
}
var val namedCustomLog
// Skip handling of empty logging configs
if !reflect.DeepEqual(cl, new(caddy.CustomLog)) {
logCounter, ok := h.State["logCounter"].(int)
if !ok {
logCounter = 0
if parseAsGlobalOption {
// Use indicated name for global log options
val.name = globalLogName
val.log = cl
} else {
// Construct a log name for server log streams
logCounter, ok := h.State["logCounter"].(int)
if !ok {
logCounter = 0
}
val.name = fmt.Sprintf("log%d", logCounter)
cl.Include = []string{"http.log.access." + val.name}
val.log = cl
logCounter++
h.State["logCounter"] = logCounter
}
val.name = fmt.Sprintf("log%d", logCounter)
cl.Include = []string{"http.log.access." + val.name}
val.log = cl
logCounter++
h.State["logCounter"] = logCounter
}
configValues = append(configValues, ConfigValue{
Class: "custom_log",
@@ -567,3 +862,15 @@ func parseLog(h Helper) ([]ConfigValue, error) {
}
return configValues, nil
}
// parseSkipLog parses the skip_log directive. Syntax:
//
// skip_log [<matcher>]
func parseSkipLog(h Helper) (caddyhttp.MiddlewareHandler, error) {
for h.Next() {
if h.NextArg() {
return nil, h.ArgErr()
}
}
return caddyhttp.VarsMiddleware{"skip_log": true}, nil
}
+215
View File
@@ -0,0 +1,215 @@
package httpcaddyfile
import (
"testing"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
_ "github.com/caddyserver/caddy/v2/modules/logging"
)
func TestLogDirectiveSyntax(t *testing.T) {
for i, tc := range []struct {
input string
output string
expectError bool
}{
{
input: `:8080 {
log
}
`,
output: `{"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{}}}}}}`,
expectError: false,
},
{
input: `:8080 {
log {
output file foo.log
}
}
`,
output: `{"logging":{"logs":{"default":{"exclude":["http.log.access.log0"]},"log0":{"writer":{"filename":"foo.log","output":"file"},"include":["http.log.access.log0"]}}},"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{"default_logger_name":"log0"}}}}}}`,
expectError: false,
},
{
input: `:8080 {
log {
format filter {
wrap console
fields {
request>remote_ip ip_mask {
ipv4 24
ipv6 32
}
}
}
}
}
`,
output: `{"logging":{"logs":{"default":{"exclude":["http.log.access.log0"]},"log0":{"encoder":{"fields":{"request\u003eremote_ip":{"filter":"ip_mask","ipv4_cidr":24,"ipv6_cidr":32}},"format":"filter","wrap":{"format":"console"}},"include":["http.log.access.log0"]}}},"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{"default_logger_name":"log0"}}}}}}`,
expectError: false,
},
{
input: `:8080 {
log invalid {
output file foo.log
}
}
`,
expectError: true,
},
} {
adapter := caddyfile.Adapter{
ServerType: ServerType{},
}
out, _, err := adapter.Adapt([]byte(tc.input), nil)
if err != nil != tc.expectError {
t.Errorf("Test %d error expectation failed Expected: %v, got %s", i, tc.expectError, err)
continue
}
if string(out) != tc.output {
t.Errorf("Test %d error output mismatch Expected: %s, got %s", i, tc.output, out)
}
}
}
func TestRedirDirectiveSyntax(t *testing.T) {
for i, tc := range []struct {
input string
expectError bool
}{
{
input: `:8080 {
redir :8081
}`,
expectError: false,
},
{
input: `:8080 {
redir * :8081
}`,
expectError: false,
},
{
input: `:8080 {
redir /api/* :8081 300
}`,
expectError: false,
},
{
input: `:8080 {
redir :8081 300
}`,
expectError: false,
},
{
input: `:8080 {
redir /api/* :8081 399
}`,
expectError: false,
},
{
input: `:8080 {
redir :8081 399
}`,
expectError: false,
},
{
input: `:8080 {
redir /old.html /new.html
}`,
expectError: false,
},
{
input: `:8080 {
redir /old.html /new.html temporary
}`,
expectError: false,
},
{
input: `:8080 {
redir https://example.com{uri} permanent
}`,
expectError: false,
},
{
input: `:8080 {
redir /old.html /new.html permanent
}`,
expectError: false,
},
{
input: `:8080 {
redir /old.html /new.html html
}`,
expectError: false,
},
{
// this is now allowed so a Location header
// can be written and consumed by JS
// in the case of XHR requests
input: `:8080 {
redir * :8081 401
}`,
expectError: false,
},
{
input: `:8080 {
redir * :8081 402
}`,
expectError: true,
},
{
input: `:8080 {
redir * :8081 {http.reverse_proxy.status_code}
}`,
expectError: false,
},
{
input: `:8080 {
redir /old.html /new.html htlm
}`,
expectError: true,
},
{
input: `:8080 {
redir * :8081 200
}`,
expectError: true,
},
{
input: `:8080 {
redir * :8081 temp
}`,
expectError: true,
},
{
input: `:8080 {
redir * :8081 perm
}`,
expectError: true,
},
{
input: `:8080 {
redir * :8081 php
}`,
expectError: true,
},
} {
adapter := caddyfile.Adapter{
ServerType: ServerType{},
}
_, _, err := adapter.Adapt([]byte(tc.input), nil)
if err != nil != tc.expectError {
t.Errorf("Test %d error expectation failed Expected: %v, got %s", i, tc.expectError, err)
continue
}
}
}
+235 -119
View File
@@ -37,32 +37,48 @@ import (
// The header directive goes second so that headers
// can be manipulated before doing redirects.
var directiveOrder = []string{
"tracing",
"map",
"vars",
"root",
"skip_log",
"header",
"copy_response_headers", // only in reverse_proxy's handle_response
"request_body",
"redir",
"rewrite",
// URI manipulation
// incoming request manipulation
"method",
"rewrite",
"uri",
"try_files",
// middleware handlers; some wrap responses
"basicauth",
"forward_auth",
"request_header",
"encode",
"push",
"templates",
// special routing directives
// special routing & dispatching directives
"handle",
"handle_path",
"route",
// handlers that typically respond to requests
"abort",
"error",
"copy_response", // only in reverse_proxy's handle_response
"respond",
"metrics",
"reverse_proxy",
"php_fastcgi",
"file_server",
"acme_server",
}
// directiveIsOrdered returns true if dir is
@@ -97,20 +113,11 @@ func RegisterHandlerDirective(dir string, setupFunc UnmarshalHandlerFunc) {
return nil, h.ArgErr()
}
matcherSet, ok, err := h.MatcherToken()
matcherSet, err := h.ExtractMatcherSet()
if err != nil {
return nil, err
}
if ok {
// strip matcher token; we don't need to
// use the return value here because a
// new dispenser should have been made
// solely for this directive's tokens,
// with no other uses of same slice
h.Dispenser.Delete()
}
h.Dispenser.Reset() // pretend this lookahead never happened
val, err := setupFunc(h)
if err != nil {
return nil, err
@@ -120,13 +127,24 @@ func RegisterHandlerDirective(dir string, setupFunc UnmarshalHandlerFunc) {
})
}
// RegisterGlobalOption registers a unique global option opt with
// an associated unmarshaling (setup) function. When the global
// option opt is encountered in a Caddyfile, setupFunc will be
// called to unmarshal its tokens.
func RegisterGlobalOption(opt string, setupFunc UnmarshalGlobalFunc) {
if _, ok := registeredGlobalOptions[opt]; ok {
panic("global option " + opt + " already registered")
}
registeredGlobalOptions[opt] = setupFunc
}
// Helper is a type which helps setup a value from
// Caddyfile tokens.
type Helper struct {
*caddyfile.Dispenser
// State stores intermediate variables during caddyfile adaptation.
State map[string]interface{}
options map[string]interface{}
State map[string]any
options map[string]any
warnings *[]caddyconfig.Warning
matcherDefs map[string]caddy.ModuleMap
parentBlock caddyfile.ServerBlock
@@ -134,7 +152,7 @@ type Helper struct {
}
// Option gets the option keyed by name.
func (h Helper) Option(name string) interface{} {
func (h Helper) Option(name string) any {
return h.options[name]
}
@@ -158,7 +176,7 @@ func (h Helper) Caddyfiles() []string {
}
// JSON converts val into JSON. Any errors are added to warnings.
func (h Helper) JSON(val interface{}) json.RawMessage {
func (h Helper) JSON(val any) json.RawMessage {
return caddyconfig.JSON(val, h.warnings)
}
@@ -184,7 +202,12 @@ func (h Helper) ExtractMatcherSet() (caddy.ModuleMap, error) {
return nil, err
}
if hasMatcher {
h.Dispenser.Delete() // strip matcher token
// strip matcher token; we don't need to
// use the return value here because a
// new dispenser should have been made
// solely for this directive's tokens,
// with no other uses of same slice
h.Dispenser.Delete()
}
h.Dispenser.Reset() // pretend this lookahead never happened
return matcherSet, nil
@@ -250,6 +273,94 @@ func (h Helper) NewBindAddresses(addrs []string) []ConfigValue {
return []ConfigValue{{Class: "bind", Value: addrs}}
}
// WithDispenser returns a new instance based on d. All others Helper
// fields are copied, so typically maps are shared with this new instance.
func (h Helper) WithDispenser(d *caddyfile.Dispenser) Helper {
h.Dispenser = d
return h
}
// ParseSegmentAsSubroute parses the segment such that its subdirectives
// are themselves treated as directives, from which a subroute is built
// and returned.
func ParseSegmentAsSubroute(h Helper) (caddyhttp.MiddlewareHandler, error) {
allResults, err := parseSegmentAsConfig(h)
if err != nil {
return nil, err
}
return buildSubroute(allResults, h.groupCounter)
}
// parseSegmentAsConfig parses the segment such that its subdirectives
// are themselves treated as directives, including named matcher definitions,
// and the raw Config structs are returned.
func parseSegmentAsConfig(h Helper) ([]ConfigValue, error) {
var allResults []ConfigValue
for h.Next() {
// don't allow non-matcher args on the first line
if h.NextArg() {
return nil, h.ArgErr()
}
// slice the linear list of tokens into top-level segments
var segments []caddyfile.Segment
for nesting := h.Nesting(); h.NextBlock(nesting); {
segments = append(segments, h.NextSegment())
}
// copy existing matcher definitions so we can augment
// new ones that are defined only in this scope
matcherDefs := make(map[string]caddy.ModuleMap, len(h.matcherDefs))
for key, val := range h.matcherDefs {
matcherDefs[key] = val
}
// find and extract any embedded matcher definitions in this scope
for i := 0; i < len(segments); i++ {
seg := segments[i]
if strings.HasPrefix(seg.Directive(), matcherPrefix) {
// parse, then add the matcher to matcherDefs
err := parseMatcherDefinitions(caddyfile.NewDispenser(seg), matcherDefs)
if err != nil {
return nil, err
}
// remove the matcher segment (consumed), then step back the loop
segments = append(segments[:i], segments[i+1:]...)
i--
}
}
// with matchers ready to go, evaluate each directive's segment
for _, seg := range segments {
dir := seg.Directive()
dirFunc, ok := registeredDirectives[dir]
if !ok {
return nil, h.Errf("unrecognized directive: %s - are you sure your Caddyfile structure (nesting and braces) is correct?", dir)
}
subHelper := h
subHelper.Dispenser = caddyfile.NewDispenser(seg)
subHelper.matcherDefs = matcherDefs
results, err := dirFunc(subHelper)
if err != nil {
return nil, h.Errf("parsing caddyfile tokens for '%s': %v", dir, err)
}
dir = normalizeDirectiveName(dir)
for _, result := range results {
result.directive = dir
allResults = append(allResults, result)
}
}
}
return allResults, nil
}
// ConfigValue represents a value to be added to the final
// configuration, or a value to be consulted when building
// the final configuration.
@@ -265,7 +376,7 @@ type ConfigValue struct {
// The value to be used when building the config.
// Generally its type is associated with the
// name of the Class.
Value interface{}
Value any
directive string
}
@@ -276,113 +387,78 @@ func sortRoutes(routes []ConfigValue) {
dirPositions[dir] = i
}
// while we are sorting, we will need to decode a route's path matcher
// in order to sub-sort by path length; we can amortize this operation
// for efficiency by storing the decoded matchers in a slice
decodedMatchers := make([]caddyhttp.MatchPath, len(routes))
sort.SliceStable(routes, func(i, j int) bool {
// if the directives are different, just use the established directive order
iDir, jDir := routes[i].directive, routes[j].directive
if iDir == jDir {
// directives are the same; sub-sort by path matcher length
// if there's only one matcher set and one path (common case)
iRoute, ok := routes[i].Value.(caddyhttp.Route)
if !ok {
return false
}
jRoute, ok := routes[j].Value.(caddyhttp.Route)
if !ok {
return false
}
// use already-decoded matcher, or decode if it's the first time seeing it
iPM, jPM := decodedMatchers[i], decodedMatchers[j]
if iPM == nil && len(iRoute.MatcherSetsRaw) == 1 {
var pathMatcher caddyhttp.MatchPath
_ = json.Unmarshal(iRoute.MatcherSetsRaw[0]["path"], &pathMatcher)
decodedMatchers[i] = pathMatcher
iPM = pathMatcher
}
if jPM == nil && len(jRoute.MatcherSetsRaw) == 1 {
var pathMatcher caddyhttp.MatchPath
_ = json.Unmarshal(jRoute.MatcherSetsRaw[0]["path"], &pathMatcher)
decodedMatchers[j] = pathMatcher
jPM = pathMatcher
}
// sort by longer path (more specific) first; missing
// path matchers are treated as zero-length paths
var iPathLen, jPathLen int
if iPM != nil {
iPathLen = len(iPM[0])
}
if jPM != nil {
jPathLen = len(jPM[0])
}
return iPathLen > jPathLen
if iDir != jDir {
return dirPositions[iDir] < dirPositions[jDir]
}
return dirPositions[iDir] < dirPositions[jDir]
// directives are the same; sub-sort by path matcher length if there's
// only one matcher set and one path (this is a very common case and
// usually -- but not always -- helpful/expected, oh well; user can
// always take manual control of order using handler or route blocks)
iRoute, ok := routes[i].Value.(caddyhttp.Route)
if !ok {
return false
}
jRoute, ok := routes[j].Value.(caddyhttp.Route)
if !ok {
return false
}
// decode the path matchers if there is just one matcher set
var iPM, jPM caddyhttp.MatchPath
if len(iRoute.MatcherSetsRaw) == 1 {
_ = json.Unmarshal(iRoute.MatcherSetsRaw[0]["path"], &iPM)
}
if len(jRoute.MatcherSetsRaw) == 1 {
_ = json.Unmarshal(jRoute.MatcherSetsRaw[0]["path"], &jPM)
}
// if there is only one path in the path matcher, sort by longer path
// (more specific) first; missing path matchers or multi-matchers are
// treated as zero-length paths
var iPathLen, jPathLen int
if len(iPM) == 1 {
iPathLen = len(iPM[0])
}
if len(jPM) == 1 {
jPathLen = len(jPM[0])
}
// some directives involve setting values which can overwrite
// each other, so it makes most sense to reverse the order so
// that the lease specific matcher is first; everything else
// has most-specific matcher first
if iDir == "vars" {
// we can only confidently compare path lengths if both
// directives have a single path to match (issue #5037)
if iPathLen > 0 && jPathLen > 0 {
// sort least-specific (shortest) path first
return iPathLen < jPathLen
}
// if both directives don't have a single path to compare,
// sort whichever one has no matcher first; if both have
// no matcher, sort equally (stable sort preserves order)
return len(iRoute.MatcherSetsRaw) == 0 && len(jRoute.MatcherSetsRaw) > 0
} else {
// we can only confidently compare path lengths if both
// directives have a single path to match (issue #5037)
if iPathLen > 0 && jPathLen > 0 {
// sort most-specific (longest) path first
return iPathLen > jPathLen
}
// if both directives don't have a single path to compare,
// sort whichever one has a matcher first; if both have
// a matcher, sort equally (stable sort preserves order)
return len(iRoute.MatcherSetsRaw) > 0 && len(jRoute.MatcherSetsRaw) == 0
}
})
}
// parseSegmentAsSubroute parses the segment such that its subdirectives
// are themselves treated as directives, from which a subroute is built
// and returned.
func parseSegmentAsSubroute(h Helper) (caddyhttp.MiddlewareHandler, error) {
var allResults []ConfigValue
for h.Next() {
// slice the linear list of tokens into top-level segments
var segments []caddyfile.Segment
for nesting := h.Nesting(); h.NextBlock(nesting); {
segments = append(segments, h.NextSegment())
}
// copy existing matcher definitions so we can augment
// new ones that are defined only in this scope
matcherDefs := make(map[string]caddy.ModuleMap, len(h.matcherDefs))
for key, val := range h.matcherDefs {
matcherDefs[key] = val
}
// find and extract any embedded matcher definitions in this scope
for i, seg := range segments {
if strings.HasPrefix(seg.Directive(), matcherPrefix) {
err := parseMatcherDefinitions(caddyfile.NewDispenser(seg), matcherDefs)
if err != nil {
return nil, err
}
segments = append(segments[:i], segments[i+1:]...)
}
}
// with matchers ready to go, evaluate each directive's segment
for _, seg := range segments {
dir := seg.Directive()
dirFunc, ok := registeredDirectives[dir]
if !ok {
return nil, h.Errf("unrecognized directive: %s", dir)
}
subHelper := h
subHelper.Dispenser = caddyfile.NewDispenser(seg)
subHelper.matcherDefs = matcherDefs
results, err := dirFunc(subHelper)
if err != nil {
return nil, h.Errf("parsing caddyfile tokens for '%s': %v", dir, err)
}
for _, result := range results {
result.directive = dir
allResults = append(allResults, result)
}
}
}
return buildSubroute(allResults, h.groupCounter)
}
// serverBlock pairs a Caddyfile server block with
// a "pile" of config values, keyed by class name,
// as well as its parsed keys for convenience.
@@ -435,6 +511,27 @@ func (sb serverBlock) hostsFromKeys(loggerMode bool) []string {
return sblockHosts
}
func (sb serverBlock) hostsFromKeysNotHTTP(httpPort string) []string {
// ensure each entry in our list is unique
hostMap := make(map[string]struct{})
for _, addr := range sb.keys {
if addr.Host == "" {
continue
}
if addr.Scheme != "http" && addr.Port != httpPort {
hostMap[addr.Host] = struct{}{}
}
}
// convert map to slice
sblockHosts := make([]string, 0, len(hostMap))
for host := range hostMap {
sblockHosts = append(sblockHosts, host)
}
return sblockHosts
}
// hasHostCatchAllKey returns true if sb has a key that
// omits a host portion, i.e. it "catches all" hosts.
func (sb serverBlock) hasHostCatchAllKey() bool {
@@ -446,6 +543,17 @@ func (sb serverBlock) hasHostCatchAllKey() bool {
return false
}
// isAllHTTP returns true if all sb keys explicitly specify
// the http:// scheme
func (sb serverBlock) isAllHTTP() bool {
for _, addr := range sb.keys {
if addr.Scheme != "http" {
return false
}
}
return true
}
type (
// UnmarshalFunc is a function which can unmarshal Caddyfile
// tokens into zero or more config values using a Helper type.
@@ -462,6 +570,14 @@ type (
// for you. These are passed to a call to
// RegisterHandlerDirective.
UnmarshalHandlerFunc func(h Helper) (caddyhttp.MiddlewareHandler, error)
// UnmarshalGlobalFunc is a function which can unmarshal Caddyfile
// tokens from a global option. It is passed the tokens to parse and
// existing value from the previous instance of this global option
// (if any). It returns the value to associate with this global option.
UnmarshalGlobalFunc func(d *caddyfile.Dispenser, existingVal any) (any, error)
)
var registeredDirectives = make(map[string]UnmarshalFunc)
var registeredGlobalOptions = make(map[string]UnmarshalGlobalFunc)
+480 -152
View File
@@ -18,6 +18,7 @@ import (
"encoding/json"
"fmt"
"reflect"
"regexp"
"sort"
"strconv"
"strings"
@@ -26,37 +27,44 @@ import (
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/caddyserver/caddy/v2/modules/caddypki"
"github.com/caddyserver/caddy/v2/modules/caddytls"
"go.uber.org/zap"
)
func init() {
caddyconfig.RegisterAdapter("caddyfile", caddyfile.Adapter{ServerType: ServerType{}})
}
// App represents the configuration for a non-standard
// Caddy app module (e.g. third-party plugin) which was
// parsed from a global options block.
type App struct {
// The JSON key for the app being configured
Name string
// The raw app config as JSON
Value json.RawMessage
}
// ServerType can set up a config from an HTTP Caddyfile.
type ServerType struct {
}
// Setup makes a config from the tokens.
func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
options map[string]interface{}) (*caddy.Config, []caddyconfig.Warning, error) {
options map[string]any) (*caddy.Config, []caddyconfig.Warning, error) {
var warnings []caddyconfig.Warning
gc := counter{new(int)}
state := make(map[string]interface{})
state := make(map[string]any)
// load all the server blocks and associate them with a "pile"
// of config values; also prohibit duplicate keys because they
// can make a config confusing if more than one server block is
// chosen to handle a request - we actually will make each
// server block's route terminal so that only one will run
sbKeys := make(map[string]struct{})
// load all the server blocks and associate them with a "pile" of config values
originalServerBlocks := make([]serverBlock, 0, len(inputServerBlocks))
for i, sblock := range inputServerBlocks {
for _, sblock := range inputServerBlocks {
for j, k := range sblock.Keys {
if _, ok := sbKeys[k]; ok {
return nil, warnings, fmt.Errorf("duplicate site address not allowed: '%s' in %v (site block %d, key %d)", k, sblock.Keys, i, j)
if j == 0 && strings.HasPrefix(k, "@") {
return nil, warnings, fmt.Errorf("cannot define a matcher outside of a site block: '%s'", k)
}
sbKeys[k] = struct{}{}
}
originalServerBlocks = append(originalServerBlocks, serverBlock{
block: sblock,
@@ -71,34 +79,40 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
return nil, warnings, err
}
// replace shorthand placeholders (which are convenient
// when writing a Caddyfile) with their actual placeholder
// identifiers or variable names
replacer := strings.NewReplacer(placeholderShorthands()...)
// these are placeholders that allow a user-defined final
// parameters, but we still want to provide a shorthand
// for those, so we use a regexp to replace
regexpReplacements := []struct {
search *regexp.Regexp
replace string
}{
{regexp.MustCompile(`{header\.([\w-]*)}`), "{http.request.header.$1}"},
{regexp.MustCompile(`{cookie\.([\w-]*)}`), "{http.request.cookie.$1}"},
{regexp.MustCompile(`{labels\.([\w-]*)}`), "{http.request.host.labels.$1}"},
{regexp.MustCompile(`{path\.([\w-]*)}`), "{http.request.uri.path.$1}"},
{regexp.MustCompile(`{file\.([\w-]*)}`), "{http.request.uri.path.file.$1}"},
{regexp.MustCompile(`{query\.([\w-]*)}`), "{http.request.uri.query.$1}"},
{regexp.MustCompile(`{re\.([\w-]*)\.([\w-]*)}`), "{http.regexp.$1.$2}"},
{regexp.MustCompile(`{vars\.([\w-]*)}`), "{http.vars.$1}"},
{regexp.MustCompile(`{rp\.([\w-\.]*)}`), "{http.reverse_proxy.$1}"},
{regexp.MustCompile(`{err\.([\w-\.]*)}`), "{http.error.$1}"},
{regexp.MustCompile(`{file_match\.([\w-]*)}`), "{http.matchers.file.$1}"},
}
for _, sb := range originalServerBlocks {
// replace shorthand placeholders (which are
// convenient when writing a Caddyfile) with
// their actual placeholder identifiers or
// variable names
replacer := strings.NewReplacer(
"{dir}", "{http.request.uri.path.dir}",
"{file}", "{http.request.uri.path.file}",
"{host}", "{http.request.host}",
"{hostport}", "{http.request.hostport}",
"{method}", "{http.request.method}",
"{path}", "{http.request.uri.path}",
"{query}", "{http.request.uri.query}",
"{remote}", "{http.request.remote}",
"{remote_host}", "{http.request.remote.host}",
"{remote_port}", "{http.request.remote.port}",
"{scheme}", "{http.request.scheme}",
"{uri}", "{http.request.uri}",
"{tls_cipher}", "{http.request.tls.cipher_suite}",
"{tls_version}", "{http.request.tls.version}",
"{tls_client_fingerprint}", "{http.request.tls.client.fingerprint}",
"{tls_client_issuer}", "{http.request.tls.client.issuer}",
"{tls_client_serial}", "{http.request.tls.client.serial}",
"{tls_client_subject}", "{http.request.tls.client.subject}",
)
for _, segment := range sb.block.Segments {
for i := 0; i < len(segment); i++ {
// simple string replacements
segment[i].Text = replacer.Replace(segment[i].Text)
// complex regexp replacements
for _, r := range regexpReplacements {
segment[i].Text = r.search.ReplaceAllString(segment[i].Text, r.replace)
}
}
}
@@ -130,7 +144,11 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
dirFunc, ok := registeredDirectives[dir]
if !ok {
tkn := segment[0]
return nil, warnings, fmt.Errorf("%s:%d: unrecognized directive: %s", tkn.File, tkn.Line, dir)
message := "%s:%d: unrecognized directive: %s"
if !sb.block.HasBraces {
message += "\nDid you mean to define a second site? If so, you must use curly braces around each site to separate their configurations."
}
return nil, warnings, fmt.Errorf(message, tkn.File, tkn.Line, dir)
}
h := Helper{
@@ -147,6 +165,9 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
if err != nil {
return nil, warnings, fmt.Errorf("parsing caddyfile tokens for '%s': %v", dir, err)
}
dir = normalizeDirectiveName(dir)
for _, result := range results {
result.directive = dir
sb.pile[result.Class] = append(sb.pile[result.Class], result)
@@ -172,9 +193,11 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
// now that each server is configured, make the HTTP app
httpApp := caddyhttp.App{
HTTPPort: tryInt(options["http_port"], &warnings),
HTTPSPort: tryInt(options["https_port"], &warnings),
Servers: servers,
HTTPPort: tryInt(options["http_port"], &warnings),
HTTPSPort: tryInt(options["https_port"], &warnings),
GracePeriod: tryDuration(options["grace_period"], &warnings),
ShutdownDelay: tryDuration(options["shutdown_delay"], &warnings),
Servers: servers,
}
// then make the TLS app
@@ -183,31 +206,32 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
return nil, warnings, err
}
// if experimental HTTP/3 is enabled, enable it on each server
if enableH3, ok := options["experimental_http3"].(bool); ok && enableH3 {
for _, srv := range httpApp.Servers {
srv.ExperimentalHTTP3 = true
}
// then make the PKI app
pkiApp, warnings, err := st.buildPKIApp(pairings, options, warnings)
if err != nil {
return nil, warnings, err
}
// extract any custom logs, and enforce configured levels
var customLogs []namedCustomLog
var hasDefaultLog bool
for _, p := range pairings {
for _, sb := range p.serverBlocks {
for _, clVal := range sb.pile["custom_log"] {
ncl := clVal.Value.(namedCustomLog)
if ncl.name == "" {
continue
}
if ncl.name == "default" {
hasDefaultLog = true
}
if _, ok := options["debug"]; ok && ncl.log.Level == "" {
ncl.log.Level = "DEBUG"
}
customLogs = append(customLogs, ncl)
}
addCustomLog := func(ncl namedCustomLog) {
if ncl.name == "" {
return
}
if ncl.name == caddy.DefaultLoggerName {
hasDefaultLog = true
}
if _, ok := options["debug"]; ok && ncl.log.Level == "" {
ncl.log.Level = zap.DebugLevel.CapitalString()
}
customLogs = append(customLogs, ncl)
}
// Apply global log options, when set
if options["log"] != nil {
for _, logValue := range options["log"].([]ConfigValue) {
addCustomLog(logValue.Value.(namedCustomLog))
}
}
@@ -216,32 +240,51 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
// configure it with any applicable options
if _, ok := options["debug"]; ok {
customLogs = append(customLogs, namedCustomLog{
name: "default",
log: &caddy.CustomLog{Level: "DEBUG"},
name: caddy.DefaultLoggerName,
log: &caddy.CustomLog{Level: zap.DebugLevel.CapitalString()},
})
}
}
// Apply server-specific log options
for _, p := range pairings {
for _, sb := range p.serverBlocks {
for _, clVal := range sb.pile["custom_log"] {
addCustomLog(clVal.Value.(namedCustomLog))
}
}
}
// annnd the top-level config, then we're done!
cfg := &caddy.Config{AppsRaw: make(caddy.ModuleMap)}
// loop through the configured options, and if any of
// them are an httpcaddyfile App, then we insert them
// into the config as raw Caddy apps
for _, opt := range options {
if app, ok := opt.(App); ok {
cfg.AppsRaw[app.Name] = app.Value
}
}
// insert the standard Caddy apps into the config
if len(httpApp.Servers) > 0 {
cfg.AppsRaw["http"] = caddyconfig.JSON(httpApp, &warnings)
}
if !reflect.DeepEqual(tlsApp, &caddytls.TLS{CertificatesRaw: make(caddy.ModuleMap)}) {
cfg.AppsRaw["tls"] = caddyconfig.JSON(tlsApp, &warnings)
}
if !reflect.DeepEqual(pkiApp, &caddypki.PKI{CAs: make(map[string]*caddypki.CA)}) {
cfg.AppsRaw["pki"] = caddyconfig.JSON(pkiApp, &warnings)
}
if storageCvtr, ok := options["storage"].(caddy.StorageConverter); ok {
cfg.StorageRaw = caddyconfig.JSONModuleObject(storageCvtr,
"module",
storageCvtr.(caddy.Module).CaddyModule().ID.Name(),
&warnings)
}
if adminConfig, ok := options["admin"].(string); ok && adminConfig != "" {
if adminConfig == "off" {
cfg.Admin = &caddy.AdminConfig{Disabled: true}
} else {
cfg.Admin = &caddy.AdminConfig{Listen: adminConfig}
}
if adminConfig, ok := options["admin"].(*caddy.AdminConfig); ok && adminConfig != nil {
cfg.Admin = adminConfig
}
if len(customLogs) > 0 {
if cfg.Logging == nil {
@@ -256,11 +299,11 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
// most users seem to prefer not writing access logs
// to the default log when they are directed to a
// file or have any other special customization
if len(ncl.log.Include) > 0 {
defaultLog, ok := cfg.Logging.Logs["default"]
if ncl.name != caddy.DefaultLoggerName && len(ncl.log.Include) > 0 {
defaultLog, ok := cfg.Logging.Logs[caddy.DefaultLoggerName]
if !ok {
defaultLog = new(caddy.CustomLog)
cfg.Logging.Logs["default"] = defaultLog
cfg.Logging.Logs[caddy.DefaultLoggerName] = defaultLog
}
defaultLog.Exclude = append(defaultLog.Exclude, ncl.log.Include...)
}
@@ -274,50 +317,74 @@ func (st ServerType) Setup(inputServerBlocks []caddyfile.ServerBlock,
// which is expected to be the first server block if it has zero
// keys. It returns the updated list of server blocks with the
// global options block removed, and updates options accordingly.
func (ServerType) evaluateGlobalOptionsBlock(serverBlocks []serverBlock, options map[string]interface{}) ([]serverBlock, error) {
func (ServerType) evaluateGlobalOptionsBlock(serverBlocks []serverBlock, options map[string]any) ([]serverBlock, error) {
if len(serverBlocks) == 0 || len(serverBlocks[0].block.Keys) > 0 {
return serverBlocks, nil
}
for _, segment := range serverBlocks[0].block.Segments {
dir := segment.Directive()
var val interface{}
opt := segment.Directive()
var val any
var err error
disp := caddyfile.NewDispenser(segment)
switch dir {
case "debug":
val = true
case "http_port":
val, err = parseOptHTTPPort(disp)
case "https_port":
val, err = parseOptHTTPSPort(disp)
case "default_sni":
val, err = parseOptSingleString(disp)
case "order":
val, err = parseOptOrder(disp)
case "experimental_http3":
val, err = parseOptExperimentalHTTP3(disp)
case "storage":
val, err = parseOptStorage(disp)
case "acme_ca", "acme_dns", "acme_ca_root":
val, err = parseOptSingleString(disp)
case "email":
val, err = parseOptSingleString(disp)
case "admin":
val, err = parseOptAdmin(disp)
case "on_demand_tls":
val, err = parseOptOnDemand(disp)
case "local_certs":
val = true
case "key_type":
val, err = parseOptSingleString(disp)
default:
return nil, fmt.Errorf("unrecognized parameter name: %s", dir)
optFunc, ok := registeredGlobalOptions[opt]
if !ok {
tkn := segment[0]
return nil, fmt.Errorf("%s:%d: unrecognized global option: %s", tkn.File, tkn.Line, opt)
}
val, err = optFunc(disp, options[opt])
if err != nil {
return nil, fmt.Errorf("%s: %v", dir, err)
return nil, fmt.Errorf("parsing caddyfile tokens for '%s': %v", opt, err)
}
// As a special case, fold multiple "servers" options together
// in an array instead of overwriting a possible existing value
if opt == "servers" {
existingOpts, ok := options[opt].([]serverOptions)
if !ok {
existingOpts = []serverOptions{}
}
serverOpts, ok := val.(serverOptions)
if !ok {
return nil, fmt.Errorf("unexpected type from 'servers' global options: %T", val)
}
options[opt] = append(existingOpts, serverOpts)
continue
}
// Additionally, fold multiple "log" options together into an
// array so that multiple loggers can be configured.
if opt == "log" {
existingOpts, ok := options[opt].([]ConfigValue)
if !ok {
existingOpts = []ConfigValue{}
}
logOpts, ok := val.([]ConfigValue)
if !ok {
return nil, fmt.Errorf("unexpected type from 'log' global options: %T", val)
}
options[opt] = append(existingOpts, logOpts...)
continue
}
options[opt] = val
}
// If we got "servers" options, we'll sort them by their listener address
if serverOpts, ok := options["servers"].([]serverOptions); ok {
sort.Slice(serverOpts, func(i, j int) bool {
return len(serverOpts[i].ListenerAddress) > len(serverOpts[j].ListenerAddress)
})
// Reject the config if there are duplicate listener address
seen := make(map[string]bool)
for _, entry := range serverOpts {
if _, alreadySeen := seen[entry.ListenerAddress]; alreadySeen {
return nil, fmt.Errorf("cannot have 'servers' global options with duplicate listener addresses: %s", entry.ListenerAddress)
}
seen[entry.ListenerAddress] = true
}
options[dir] = val
}
return serverBlocks[1:], nil
@@ -327,7 +394,7 @@ func (ServerType) evaluateGlobalOptionsBlock(serverBlocks []serverBlock, options
// to server blocks. Each pairing is essentially a server definition.
func (st *ServerType) serversFromPairings(
pairings []sbAddrAssociation,
options map[string]interface{},
options map[string]any,
warnings *[]caddyconfig.Warning,
groupCounter counter,
) (map[string]*caddyhttp.Server, error) {
@@ -342,12 +409,59 @@ func (st *ServerType) serversFromPairings(
if hsp, ok := options["https_port"].(int); ok {
httpsPort = strconv.Itoa(hsp)
}
autoHTTPS := "on"
if ah, ok := options["auto_https"].(string); ok {
autoHTTPS = ah
}
for i, p := range pairings {
// detect ambiguous site definitions: server blocks which
// have the same host bound to the same interface (listener
// address), otherwise their routes will improperly be added
// to the same server (see issue #4635)
for j, sblock1 := range p.serverBlocks {
for _, key := range sblock1.block.Keys {
for k, sblock2 := range p.serverBlocks {
if k == j {
continue
}
if sliceContains(sblock2.block.Keys, key) {
return nil, fmt.Errorf("ambiguous site definition: %s", key)
}
}
}
}
srv := &caddyhttp.Server{
Listen: p.addresses,
}
// handle the auto_https global option
if autoHTTPS != "on" {
srv.AutoHTTPS = new(caddyhttp.AutoHTTPSConfig)
switch autoHTTPS {
case "off":
srv.AutoHTTPS.Disabled = true
case "disable_redirects":
srv.AutoHTTPS.DisableRedir = true
case "disable_certs":
srv.AutoHTTPS.DisableCerts = true
case "ignore_loaded_certs":
srv.AutoHTTPS.IgnoreLoadedCerts = true
}
}
// Using paths in site addresses is deprecated
// See ParseAddress() where parsing should later reject paths
// See https://github.com/caddyserver/caddy/pull/4728 for a full explanation
for _, sblock := range p.serverBlocks {
for _, addr := range sblock.keys {
if addr.Path != "" {
caddy.Log().Named("caddyfile").Warn("Using a path in a site address is deprecated; please use the 'handle' directive instead", zap.String("address", addr.String()))
}
}
}
// sort server blocks by their keys; this is important because
// only the first matching site should be evaluated, and we should
// attempt to match most specific site first (host and path), in
@@ -358,7 +472,11 @@ func (st *ServerType) serversFromPairings(
// but I don't expect many blocks will have THAT many keys...
var iLongestPath, jLongestPath string
var iLongestHost, jLongestHost string
var iWildcardHost, jWildcardHost bool
for _, addr := range p.serverBlocks[i].keys {
if strings.Contains(addr.Host, "*") || addr.Host == "" {
iWildcardHost = true
}
if specificity(addr.Host) > specificity(iLongestHost) {
iLongestHost = addr.Host
}
@@ -367,6 +485,9 @@ func (st *ServerType) serversFromPairings(
}
}
for _, addr := range p.serverBlocks[j].keys {
if strings.Contains(addr.Host, "*") || addr.Host == "" {
jWildcardHost = true
}
if specificity(addr.Host) > specificity(jLongestHost) {
jLongestHost = addr.Host
}
@@ -374,6 +495,20 @@ func (st *ServerType) serversFromPairings(
jLongestPath = addr.Path
}
}
// catch-all blocks (blocks with no hostname) should always go
// last, even after blocks with wildcard hosts
if specificity(iLongestHost) == 0 {
return false
}
if specificity(jLongestHost) == 0 {
return true
}
if iWildcardHost != jWildcardHost {
// site blocks that have a key with a wildcard in the hostname
// must always be less specific than blocks without one; see
// https://github.com/caddyserver/caddy/issues/3410
return jWildcardHost && !iWildcardHost
}
if specificity(iLongestHost) == specificity(jLongestHost) {
return len(iLongestPath) > len(jLongestPath)
}
@@ -381,7 +516,17 @@ func (st *ServerType) serversFromPairings(
})
var hasCatchAllTLSConnPolicy, addressQualifiesForTLS bool
autoHTTPSWillAddConnPolicy := true
autoHTTPSWillAddConnPolicy := autoHTTPS != "off"
// if needed, the ServerLogConfig is initialized beforehand so
// that all server blocks can populate it with data, even when not
// coming with a log directive
for _, sblock := range p.serverBlocks {
if len(sblock.pile["custom_log"]) != 0 {
srv.Logs = new(caddyhttp.ServerLogConfig)
break
}
}
// create a subroute for each site in the server block
for _, sblock := range p.serverBlocks {
@@ -392,6 +537,13 @@ func (st *ServerType) serversFromPairings(
hosts := sblock.hostsFromKeys(false)
// emit warnings if user put unspecified IP addresses; they probably want the bind directive
for _, h := range hosts {
if h == "0.0.0.0" || h == "::" {
caddy.Log().Named("caddyfile").Warn("Site block has an unspecified IP address which only matches requests having that Host header; you probably want the 'bind' directive to configure the socket", zap.String("address", h))
}
}
// tls: connection policies
if cpVals, ok := sblock.pile["tls.connection_policy"]; ok {
// tls connection policies
@@ -413,24 +565,31 @@ func (st *ServerType) serversFromPairings(
}
} else {
cp.DefaultSNI = defaultSNI
hasCatchAllTLSConnPolicy = true
}
srv.TLSConnPolicies = append(srv.TLSConnPolicies, cp)
// only append this policy if it actually changes something
if !cp.SettingsEmpty() {
srv.TLSConnPolicies = append(srv.TLSConnPolicies, cp)
hasCatchAllTLSConnPolicy = len(hosts) == 0
}
}
}
for _, addr := range sblock.keys {
// exclude any hosts that were defined explicitly with "http://"
// in the key from automated cert management (issue #2998)
if addr.Scheme == "http" && addr.Host != "" {
if srv.AutoHTTPS == nil {
srv.AutoHTTPS = new(caddyhttp.AutoHTTPSConfig)
}
if !sliceContains(srv.AutoHTTPS.Skip, addr.Host) {
srv.AutoHTTPS.Skip = append(srv.AutoHTTPS.Skip, addr.Host)
// if server only uses HTTP port, auto-HTTPS will not apply
if listenersUseAnyPortOtherThan(srv.Listen, httpPort) {
// exclude any hosts that were defined explicitly with "http://"
// in the key from automated cert management (issue #2998)
if addr.Scheme == "http" && addr.Host != "" {
if srv.AutoHTTPS == nil {
srv.AutoHTTPS = new(caddyhttp.AutoHTTPSConfig)
}
if !sliceContains(srv.AutoHTTPS.Skip, addr.Host) {
srv.AutoHTTPS.Skip = append(srv.AutoHTTPS.Skip, addr.Host)
}
}
}
// we'll need to remember if the address qualifies for auto-HTTPS, so we
// can add a TLS conn policy if necessary
if addr.Scheme == "https" ||
@@ -443,6 +602,20 @@ func (st *ServerType) serversFromPairings(
(addr.Port == httpsPort || (addr.Port != httpPort && addr.Host != ""))
}
// Look for any config values that provide listener wrappers on the server block
for _, listenerConfig := range sblock.pile["listener_wrapper"] {
listenerWrapper, ok := listenerConfig.Value.(caddy.ListenerWrapper)
if !ok {
return nil, fmt.Errorf("config for a listener wrapper did not provide a value that implements caddy.ListenerWrapper")
}
jsonListenerWrapper := caddyconfig.JSONModuleObject(
listenerWrapper,
"wrapper",
listenerWrapper.(caddy.Module).CaddyModule().ID.Name(),
warnings)
srv.ListenerWrappersRaw = append(srv.ListenerWrappersRaw, jsonListenerWrapper)
}
// set up each handler directive, making sure to honor directive order
dirRoutes := sblock.pile["route"]
siteSubroute, err := buildSubroute(dirRoutes, groupCounter)
@@ -469,9 +642,6 @@ func (st *ServerType) serversFromPairings(
sblockLogHosts := sblock.hostsFromKeys(true)
for _, cval := range sblock.pile["custom_log"] {
ncl := cval.Value.(namedCustomLog)
if srv.Logs == nil {
srv.Logs = new(caddyhttp.ServerLogConfig)
}
if sblock.hasHostCatchAllKey() {
// all requests for hosts not able to be listed should use
// this log because it's a catch-all-hosts server block
@@ -479,18 +649,10 @@ func (st *ServerType) serversFromPairings(
} else {
// map each host to the user's desired logger name
for _, h := range sblockLogHosts {
// if the custom logger name is non-empty, add it to
// the map; otherwise, only map to an empty logger
// name if the server block has a catch-all host (in
// which case only requests with mapped hostnames will
// be access-logged, so it'll be necessary to add them
// to the map even if they use default logger)
if ncl.name != "" || len(hosts) == 0 {
if srv.Logs.LoggerNames == nil {
srv.Logs.LoggerNames = make(map[string]string)
}
srv.Logs.LoggerNames[h] = ncl.name
if srv.Logs.LoggerNames == nil {
srv.Logs.LoggerNames = make(map[string]string)
}
srv.Logs.LoggerNames[h] = ncl.name
}
}
}
@@ -542,10 +704,15 @@ func (st *ServerType) serversFromPairings(
servers[fmt.Sprintf("srv%d", i)] = srv
}
err := applyServerOptions(servers, options, warnings)
if err != nil {
return nil, err
}
return servers, nil
}
func detectConflictingSchemes(srv *caddyhttp.Server, serverBlocks []serverBlock, options map[string]interface{}) error {
func detectConflictingSchemes(srv *caddyhttp.Server, serverBlocks []serverBlock, options map[string]any) error {
httpPort := strconv.Itoa(caddyhttp.DefaultHTTPPort)
if hp, ok := options["http_port"].(int); ok {
httpPort = strconv.Itoa(hp)
@@ -603,9 +770,15 @@ func detectConflictingSchemes(srv *caddyhttp.Server, serverBlocks []serverBlock,
return nil
}
// consolidateConnPolicies removes empty TLS connection policies and combines
// equivalent ones for a cleaner overall output.
// consolidateConnPolicies sorts any catch-all policy to the end, removes empty TLS connection
// policies, and combines equivalent ones for a cleaner overall output.
func consolidateConnPolicies(cps caddytls.ConnectionPolicies) (caddytls.ConnectionPolicies, error) {
// catch-all policies (those without any matcher) should be at the
// end, otherwise it nullifies any more specific policies
sort.SliceStable(cps, func(i, j int) bool {
return cps[j].MatchersRaw == nil && cps[i].MatchersRaw != nil
})
for i := 0; i < len(cps); i++ {
// compare it to the others
for j := 0; j < len(cps); j++ {
@@ -734,11 +907,32 @@ func appendSubrouteToRouteList(routeList caddyhttp.RouteList,
return routeList
}
// No need to wrap the handlers in a subroute if this is the only server block
// and there is no matcher for it (doing so would produce unnecessarily nested
// JSON), *unless* there is a host matcher within this site block; if so, then
// we still need to wrap in a subroute because otherwise the host matcher from
// the inside of the site block would be a top-level host matcher, which is
// subject to auto-HTTPS (cert management), and using a host matcher within
// a site block is a valid, common pattern for excluding domains from cert
// management, leading to unexpected behavior; see issue #5124.
wrapInSubroute := true
if len(matcherSetsEnc) == 0 && len(p.serverBlocks) == 1 {
// no need to wrap the handlers in a subroute if this is
// the only server block and there is no matcher for it
routeList = append(routeList, subroute.Routes...)
} else {
var hasHostMatcher bool
outer:
for _, route := range subroute.Routes {
for _, ms := range route.MatcherSetsRaw {
for matcherName := range ms {
if matcherName == "host" {
hasHostMatcher = true
break outer
}
}
}
}
wrapInSubroute = hasHostMatcher
}
if wrapInSubroute {
route := caddyhttp.Route{
// the semantics of a site block in the Caddyfile dictate
// that only the first matching one is evaluated, since
@@ -756,7 +950,10 @@ func appendSubrouteToRouteList(routeList caddyhttp.RouteList,
if len(route.MatcherSetsRaw) > 0 || len(route.HandlersRaw) > 0 {
routeList = append(routeList, route)
}
} else {
routeList = append(routeList, subroute.Routes...)
}
return routeList
}
@@ -765,7 +962,7 @@ func appendSubrouteToRouteList(routeList caddyhttp.RouteList,
func buildSubroute(routes []ConfigValue, groupCounter counter) (*caddyhttp.Subroute, error) {
for _, val := range routes {
if !directiveIsOrdered(val.directive) {
return nil, fmt.Errorf("directive '%s' is not ordered, so it cannot be used here", val.directive)
return nil, fmt.Errorf("directive '%s' is not an ordered HTTP handler, so it cannot be used here", val.directive)
}
}
@@ -798,7 +995,18 @@ func buildSubroute(routes []ConfigValue, groupCounter counter) (*caddyhttp.Subro
// root directives would overwrite previously-matched ones; they should not cascade
"root": {},
}
for meDir, info := range mutuallyExclusiveDirs {
// we need to deterministically loop over each of these directives
// in order to keep the group numbers consistent
keys := make([]string, 0, len(mutuallyExclusiveDirs))
for k := range mutuallyExclusiveDirs {
keys = append(keys, k)
}
sort.Strings(keys)
for _, meDir := range keys {
info := mutuallyExclusiveDirs[meDir]
// see how many instances of the directive there are
for _, r := range routes {
if r.directive == meDir {
@@ -859,6 +1067,19 @@ func buildSubroute(routes []ConfigValue, groupCounter counter) (*caddyhttp.Subro
return subroute, nil
}
// normalizeDirectiveName ensures directives that should be sorted
// at the same level are named the same before sorting happens.
func normalizeDirectiveName(directive string) string {
// As a special case, we want "handle_path" to be sorted
// at the same level as "handle", so we force them to use
// the same directive name after their parsing is complete.
// See https://github.com/caddyserver/caddy/issues/3675#issuecomment-678042377
if directive == "handle_path" {
directive = "handle"
}
return directive
}
// consolidateRoutes combines routes with the same properties
// (same matchers, same Terminal and Group settings) for a
// cleaner overall output.
@@ -989,6 +1210,7 @@ func (st *ServerType) compileEncodedMatcherSets(sblock serverBlock) ([]caddy.Mod
func parseMatcherDefinitions(d *caddyfile.Dispenser, matchers map[string]caddy.ModuleMap) error {
for d.Next() {
// this is the "name" for "named matchers"
definitionName := d.Val()
if _, ok := matchers[definitionName]; ok {
@@ -996,16 +1218,9 @@ func parseMatcherDefinitions(d *caddyfile.Dispenser, matchers map[string]caddy.M
}
matchers[definitionName] = make(caddy.ModuleMap)
// in case there are multiple instances of the same matcher, concatenate
// their tokens (we expect that UnmarshalCaddyfile should be able to
// handle more than one segment); otherwise, we'd overwrite other
// instances of the matcher in this set
tokensByMatcherName := make(map[string][]caddyfile.Token)
for nesting := d.Nesting(); d.NextBlock(nesting); {
matcherName := d.Val()
tokensByMatcherName[matcherName] = append(tokensByMatcherName[matcherName], d.NextSegment()...)
}
for matcherName, tokens := range tokensByMatcherName {
// given a matcher name and the tokens following it, parse
// the tokens as a matcher module and record it
makeMatcher := func(matcherName string, tokens []caddyfile.Token) error {
mod, err := caddy.GetModule("http.matchers." + matcherName)
if err != nil {
return fmt.Errorf("getting matcher module '%s': %v", matcherName, err)
@@ -1023,6 +1238,39 @@ func parseMatcherDefinitions(d *caddyfile.Dispenser, matchers map[string]caddy.M
return fmt.Errorf("matcher module '%s' is not a request matcher", matcherName)
}
matchers[definitionName][matcherName] = caddyconfig.JSON(rm, nil)
return nil
}
// if the next token is quoted, we can assume it's not a matcher name
// and that it's probably an 'expression' matcher
if d.NextArg() {
if d.Token().Quoted() {
err := makeMatcher("expression", []caddyfile.Token{d.Token()})
if err != nil {
return err
}
continue
}
// if it wasn't quoted, then we need to rewind after calling
// d.NextArg() so the below properly grabs the matcher name
d.Prev()
}
// in case there are multiple instances of the same matcher, concatenate
// their tokens (we expect that UnmarshalCaddyfile should be able to
// handle more than one segment); otherwise, we'd overwrite other
// instances of the matcher in this set
tokensByMatcherName := make(map[string][]caddyfile.Token)
for nesting := d.Nesting(); d.NextArg() || d.NextBlock(nesting); {
matcherName := d.Val()
tokensByMatcherName[matcherName] = append(tokensByMatcherName[matcherName], d.NextSegment()...)
}
for matcherName, tokens := range tokensByMatcherName {
err := makeMatcher(matcherName, tokens)
if err != nil {
return err
}
}
}
return nil
@@ -1040,9 +1288,61 @@ func encodeMatcherSet(matchers map[string]caddyhttp.RequestMatcher) (caddy.Modul
return msEncoded, nil
}
// placeholderShorthands returns a slice of old-new string pairs,
// where the left of the pair is a placeholder shorthand that may
// be used in the Caddyfile, and the right is the replacement.
func placeholderShorthands() []string {
return []string{
"{dir}", "{http.request.uri.path.dir}",
"{file}", "{http.request.uri.path.file}",
"{host}", "{http.request.host}",
"{hostport}", "{http.request.hostport}",
"{port}", "{http.request.port}",
"{method}", "{http.request.method}",
"{path}", "{http.request.uri.path}",
"{query}", "{http.request.uri.query}",
"{remote}", "{http.request.remote}",
"{remote_host}", "{http.request.remote.host}",
"{remote_port}", "{http.request.remote.port}",
"{scheme}", "{http.request.scheme}",
"{uri}", "{http.request.uri}",
"{tls_cipher}", "{http.request.tls.cipher_suite}",
"{tls_version}", "{http.request.tls.version}",
"{tls_client_fingerprint}", "{http.request.tls.client.fingerprint}",
"{tls_client_issuer}", "{http.request.tls.client.issuer}",
"{tls_client_serial}", "{http.request.tls.client.serial}",
"{tls_client_subject}", "{http.request.tls.client.subject}",
"{tls_client_certificate_pem}", "{http.request.tls.client.certificate_pem}",
"{tls_client_certificate_der_base64}", "{http.request.tls.client.certificate_der_base64}",
"{upstream_hostport}", "{http.reverse_proxy.upstream.hostport}",
}
}
// WasReplacedPlaceholderShorthand checks if a token string was
// likely a replaced shorthand of the known Caddyfile placeholder
// replacement outputs. Useful to prevent some user-defined map
// output destinations from overlapping with one of the
// predefined shorthands.
func WasReplacedPlaceholderShorthand(token string) string {
prev := ""
for i, item := range placeholderShorthands() {
// only look at every 2nd item, which is the replacement
if i%2 == 0 {
prev = item
continue
}
if strings.Trim(token, "{}") == strings.Trim(item, "{}") {
// we return the original shorthand so it
// can be used for an error message
return prev
}
}
return ""
}
// tryInt tries to convert val to an integer. If it fails,
// it downgrades the error to a warning and returns 0.
func tryInt(val interface{}, warnings *[]caddyconfig.Warning) int {
func tryInt(val any, warnings *[]caddyconfig.Warning) int {
intVal, ok := val.(int)
if val != nil && !ok && warnings != nil {
*warnings = append(*warnings, caddyconfig.Warning{Message: "not an integer type"})
@@ -1050,7 +1350,7 @@ func tryInt(val interface{}, warnings *[]caddyconfig.Warning) int {
return intVal
}
func tryString(val interface{}, warnings *[]caddyconfig.Warning) string {
func tryString(val any, warnings *[]caddyconfig.Warning) string {
stringVal, ok := val.(string)
if val != nil && !ok && warnings != nil {
*warnings = append(*warnings, caddyconfig.Warning{Message: "not a string type"})
@@ -1058,6 +1358,14 @@ func tryString(val interface{}, warnings *[]caddyconfig.Warning) string {
return stringVal
}
func tryDuration(val any, warnings *[]caddyconfig.Warning) caddy.Duration {
durationVal, ok := val.(caddy.Duration)
if val != nil && !ok && warnings != nil {
*warnings = append(*warnings, caddyconfig.Warning{Message: "not a duration type"})
}
return durationVal
}
// sliceContains returns true if needle is in haystack.
func sliceContains(haystack []string, needle string) bool {
for _, s := range haystack {
@@ -1068,6 +1376,26 @@ func sliceContains(haystack []string, needle string) bool {
return false
}
// listenersUseAnyPortOtherThan returns true if there are any
// listeners in addresses that use a port which is not otherPort.
// Mostly borrowed from unexported method in caddyhttp package.
func listenersUseAnyPortOtherThan(addresses []string, otherPort string) bool {
otherPortInt, err := strconv.Atoi(otherPort)
if err != nil {
return false
}
for _, lnAddr := range addresses {
laddrs, err := caddy.ParseNetworkAddress(lnAddr)
if err != nil {
continue
}
if uint(otherPortInt) > laddrs.EndPort || uint(otherPortInt) < laddrs.StartPort {
return true
}
}
return false
}
// specificity returns len(s) minus any wildcards (*) and
// placeholders ({...}). Basically, it's a length count
// that penalizes the use of wildcards and placeholders.
+64 -22
View File
@@ -9,7 +9,6 @@ import (
func TestMatcherSyntax(t *testing.T) {
for i, tc := range []struct {
input string
expectWarn bool
expectError bool
}{
{
@@ -18,7 +17,6 @@ func TestMatcherSyntax(t *testing.T) {
query showdebug=1
}
`,
expectWarn: false,
expectError: false,
},
{
@@ -27,7 +25,6 @@ func TestMatcherSyntax(t *testing.T) {
query bad format
}
`,
expectWarn: false,
expectError: true,
},
{
@@ -38,7 +35,6 @@ func TestMatcherSyntax(t *testing.T) {
}
}
`,
expectWarn: false,
expectError: false,
},
{
@@ -47,21 +43,29 @@ func TestMatcherSyntax(t *testing.T) {
not path /somepath*
}
`,
expectWarn: false,
expectError: false,
},
{
input: `http://localhost
@debug not path /somepath*
`,
expectError: false,
},
{
input: `@matcher {
path /matcher-not-allowed/outside-of-site-block/*
}
http://localhost
`,
expectError: true,
},
} {
adapter := caddyfile.Adapter{
ServerType: ServerType{},
}
_, warnings, err := adapter.Adapt([]byte(tc.input), nil)
if len(warnings) > 0 != tc.expectWarn {
t.Errorf("Test %d warning expectation failed Expected: %v, got %v", i, tc.expectWarn, warnings)
continue
}
_, _, err := adapter.Adapt([]byte(tc.input), nil)
if err != nil != tc.expectError {
t.Errorf("Test %d error expectation failed Expected: %v, got %s", i, tc.expectError, err)
@@ -103,7 +107,6 @@ func TestSpecificity(t *testing.T) {
func TestGlobalOptions(t *testing.T) {
for i, tc := range []struct {
input string
expectWarn bool
expectError bool
}{
{
@@ -113,7 +116,6 @@ func TestGlobalOptions(t *testing.T) {
}
:80
`,
expectWarn: false,
expectError: false,
},
{
@@ -123,7 +125,6 @@ func TestGlobalOptions(t *testing.T) {
}
:80
`,
expectWarn: false,
expectError: false,
},
{
@@ -133,7 +134,6 @@ func TestGlobalOptions(t *testing.T) {
}
:80
`,
expectWarn: false,
expectError: false,
},
{
@@ -145,7 +145,54 @@ func TestGlobalOptions(t *testing.T) {
}
:80
`,
expectWarn: false,
expectError: true,
},
{
input: `
{
admin {
enforce_origin
origins 192.168.1.1:2020 127.0.0.1:2020
}
}
:80
`,
expectError: false,
},
{
input: `
{
admin 127.0.0.1:2020 {
enforce_origin
origins 192.168.1.1:2020 127.0.0.1:2020
}
}
:80
`,
expectError: false,
},
{
input: `
{
admin 192.168.1.1:2020 127.0.0.1:2020 {
enforce_origin
origins 192.168.1.1:2020 127.0.0.1:2020
}
}
:80
`,
expectError: true,
},
{
input: `
{
admin off {
enforce_origin
origins 192.168.1.1:2020 127.0.0.1:2020
}
}
:80
`,
expectError: true,
},
} {
@@ -154,12 +201,7 @@ func TestGlobalOptions(t *testing.T) {
ServerType: ServerType{},
}
_, warnings, err := adapter.Adapt([]byte(tc.input), nil)
if len(warnings) > 0 != tc.expectWarn {
t.Errorf("Test %d warning expectation failed Expected: %v, got %v", i, tc.expectWarn, warnings)
continue
}
_, _, err := adapter.Adapt([]byte(tc.input), nil)
if err != nil != tc.expectError {
t.Errorf("Test %d error expectation failed Expected: %v, got %s", i, tc.expectError, err)
+257 -34
View File
@@ -16,14 +16,49 @@ package httpcaddyfile
import (
"strconv"
"time"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddytls"
"github.com/caddyserver/certmagic"
"github.com/mholt/acmez/acme"
)
func parseOptHTTPPort(d *caddyfile.Dispenser) (int, error) {
func init() {
RegisterGlobalOption("debug", parseOptTrue)
RegisterGlobalOption("http_port", parseOptHTTPPort)
RegisterGlobalOption("https_port", parseOptHTTPSPort)
RegisterGlobalOption("default_bind", parseOptStringList)
RegisterGlobalOption("grace_period", parseOptDuration)
RegisterGlobalOption("shutdown_delay", parseOptDuration)
RegisterGlobalOption("default_sni", parseOptSingleString)
RegisterGlobalOption("order", parseOptOrder)
RegisterGlobalOption("storage", parseOptStorage)
RegisterGlobalOption("storage_clean_interval", parseOptDuration)
RegisterGlobalOption("renew_interval", parseOptDuration)
RegisterGlobalOption("ocsp_interval", parseOptDuration)
RegisterGlobalOption("acme_ca", parseOptSingleString)
RegisterGlobalOption("acme_ca_root", parseOptSingleString)
RegisterGlobalOption("acme_dns", parseOptACMEDNS)
RegisterGlobalOption("acme_eab", parseOptACMEEAB)
RegisterGlobalOption("cert_issuer", parseOptCertIssuer)
RegisterGlobalOption("skip_install_trust", parseOptTrue)
RegisterGlobalOption("email", parseOptSingleString)
RegisterGlobalOption("admin", parseOptAdmin)
RegisterGlobalOption("on_demand_tls", parseOptOnDemand)
RegisterGlobalOption("local_certs", parseOptTrue)
RegisterGlobalOption("key_type", parseOptSingleString)
RegisterGlobalOption("auto_https", parseOptAutoHTTPS)
RegisterGlobalOption("servers", parseServerOptions)
RegisterGlobalOption("ocsp_stapling", parseOCSPStaplingOptions)
RegisterGlobalOption("log", parseLogOptions)
RegisterGlobalOption("preferred_chains", parseOptPreferredChains)
}
func parseOptTrue(d *caddyfile.Dispenser, _ any) (any, error) { return true, nil }
func parseOptHTTPPort(d *caddyfile.Dispenser, _ any) (any, error) {
var httpPort int
for d.Next() {
var httpPortStr string
@@ -39,7 +74,7 @@ func parseOptHTTPPort(d *caddyfile.Dispenser) (int, error) {
return httpPort, nil
}
func parseOptHTTPSPort(d *caddyfile.Dispenser) (int, error) {
func parseOptHTTPSPort(d *caddyfile.Dispenser, _ any) (any, error) {
var httpsPort int
for d.Next() {
var httpsPortStr string
@@ -55,11 +90,7 @@ func parseOptHTTPSPort(d *caddyfile.Dispenser) (int, error) {
return httpsPort, nil
}
func parseOptExperimentalHTTP3(d *caddyfile.Dispenser) (bool, error) {
return true, nil
}
func parseOptOrder(d *caddyfile.Dispenser) ([]string, error) {
func parseOptOrder(d *caddyfile.Dispenser, _ any) (any, error) {
newOrder := directiveOrder
for d.Next() {
@@ -135,34 +166,110 @@ func parseOptOrder(d *caddyfile.Dispenser) ([]string, error) {
return newOrder, nil
}
func parseOptStorage(d *caddyfile.Dispenser) (caddy.StorageConverter, error) {
func parseOptStorage(d *caddyfile.Dispenser, _ any) (any, error) {
if !d.Next() { // consume option name
return nil, d.ArgErr()
}
if !d.Next() { // get storage module name
return nil, d.ArgErr()
}
modName := d.Val()
mod, err := caddy.GetModule("caddy.storage." + modName)
if err != nil {
return nil, d.Errf("getting storage module '%s': %v", modName, err)
}
unm, ok := mod.New().(caddyfile.Unmarshaler)
if !ok {
return nil, d.Errf("storage module '%s' is not a Caddyfile unmarshaler", mod.ID)
}
err = unm.UnmarshalCaddyfile(d.NewFromNextSegment())
modID := "caddy.storage." + d.Val()
unm, err := caddyfile.UnmarshalModule(d, modID)
if err != nil {
return nil, err
}
storage, ok := unm.(caddy.StorageConverter)
if !ok {
return nil, d.Errf("module %s is not a StorageConverter", mod.ID)
return nil, d.Errf("module %s is not a caddy.StorageConverter", modID)
}
return storage, nil
}
func parseOptSingleString(d *caddyfile.Dispenser) (string, error) {
func parseOptDuration(d *caddyfile.Dispenser, _ any) (any, error) {
if !d.Next() { // consume option name
return nil, d.ArgErr()
}
if !d.Next() { // get duration value
return nil, d.ArgErr()
}
dur, err := caddy.ParseDuration(d.Val())
if err != nil {
return nil, err
}
return caddy.Duration(dur), nil
}
func parseOptACMEDNS(d *caddyfile.Dispenser, _ any) (any, error) {
if !d.Next() { // consume option name
return nil, d.ArgErr()
}
if !d.Next() { // get DNS module name
return nil, d.ArgErr()
}
modID := "dns.providers." + d.Val()
unm, err := caddyfile.UnmarshalModule(d, modID)
if err != nil {
return nil, err
}
prov, ok := unm.(certmagic.ACMEDNSProvider)
if !ok {
return nil, d.Errf("module %s (%T) is not a certmagic.ACMEDNSProvider", modID, unm)
}
return prov, nil
}
func parseOptACMEEAB(d *caddyfile.Dispenser, _ any) (any, error) {
eab := new(acme.EAB)
for d.Next() {
if d.NextArg() {
return nil, d.ArgErr()
}
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "key_id":
if !d.NextArg() {
return nil, d.ArgErr()
}
eab.KeyID = d.Val()
case "mac_key":
if !d.NextArg() {
return nil, d.ArgErr()
}
eab.MACKey = d.Val()
default:
return nil, d.Errf("unrecognized parameter '%s'", d.Val())
}
}
}
return eab, nil
}
func parseOptCertIssuer(d *caddyfile.Dispenser, existing any) (any, error) {
var issuers []certmagic.Issuer
if existing != nil {
issuers = existing.([]certmagic.Issuer)
}
for d.Next() { // consume option name
if !d.Next() { // get issuer module name
return nil, d.ArgErr()
}
modID := "tls.issuance." + d.Val()
unm, err := caddyfile.UnmarshalModule(d, modID)
if err != nil {
return nil, err
}
iss, ok := unm.(certmagic.Issuer)
if !ok {
return nil, d.Errf("module %s (%T) is not a certmagic.Issuer", modID, unm)
}
issuers = append(issuers, iss)
}
return issuers, nil
}
func parseOptSingleString(d *caddyfile.Dispenser, _ any) (any, error) {
d.Next() // consume parameter name
if !d.Next() {
return "", d.ArgErr()
@@ -174,21 +281,52 @@ func parseOptSingleString(d *caddyfile.Dispenser) (string, error) {
return val, nil
}
func parseOptAdmin(d *caddyfile.Dispenser) (string, error) {
if d.Next() {
var listenAddress string
if !d.AllArgs(&listenAddress) {
return "", d.ArgErr()
}
if listenAddress == "" {
listenAddress = caddy.DefaultAdminListen
}
return listenAddress, nil
func parseOptStringList(d *caddyfile.Dispenser, _ any) (any, error) {
d.Next() // consume parameter name
val := d.RemainingArgs()
if len(val) == 0 {
return "", d.ArgErr()
}
return "", nil
return val, nil
}
func parseOptOnDemand(d *caddyfile.Dispenser) (*caddytls.OnDemandConfig, error) {
func parseOptAdmin(d *caddyfile.Dispenser, _ any) (any, error) {
adminCfg := new(caddy.AdminConfig)
for d.Next() {
if d.NextArg() {
listenAddress := d.Val()
if listenAddress == "off" {
adminCfg.Disabled = true
if d.Next() { // Do not accept any remaining options including block
return nil, d.Err("No more option is allowed after turning off admin config")
}
} else {
adminCfg.Listen = listenAddress
if d.NextArg() { // At most 1 arg is allowed
return nil, d.ArgErr()
}
}
}
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "enforce_origin":
adminCfg.EnforceOrigin = true
case "origins":
adminCfg.Origins = d.RemainingArgs()
default:
return nil, d.Errf("unrecognized parameter '%s'", d.Val())
}
}
}
if adminCfg.Listen == "" && !adminCfg.Disabled {
adminCfg.Listen = caddy.DefaultAdminListen
}
return adminCfg, nil
}
func parseOptOnDemand(d *caddyfile.Dispenser, _ any) (any, error) {
var ond *caddytls.OnDemandConfig
for d.Next() {
if d.NextArg() {
@@ -209,7 +347,7 @@ func parseOptOnDemand(d *caddyfile.Dispenser) (*caddytls.OnDemandConfig, error)
if !d.NextArg() {
return nil, d.ArgErr()
}
dur, err := time.ParseDuration(d.Val())
dur, err := caddy.ParseDuration(d.Val())
if err != nil {
return nil, err
}
@@ -247,3 +385,88 @@ func parseOptOnDemand(d *caddyfile.Dispenser) (*caddytls.OnDemandConfig, error)
}
return ond, nil
}
func parseOptAutoHTTPS(d *caddyfile.Dispenser, _ any) (any, error) {
d.Next() // consume parameter name
if !d.Next() {
return "", d.ArgErr()
}
val := d.Val()
if d.Next() {
return "", d.ArgErr()
}
if val != "off" && val != "disable_redirects" && val != "disable_certs" && val != "ignore_loaded_certs" {
return "", d.Errf("auto_https must be one of 'off', 'disable_redirects', 'disable_certs', or 'ignore_loaded_certs'")
}
return val, nil
}
func parseServerOptions(d *caddyfile.Dispenser, _ any) (any, error) {
return unmarshalCaddyfileServerOptions(d)
}
func parseOCSPStaplingOptions(d *caddyfile.Dispenser, _ any) (any, error) {
d.Next() // consume option name
var val string
if !d.AllArgs(&val) {
return nil, d.ArgErr()
}
if val != "off" {
return nil, d.Errf("invalid argument '%s'", val)
}
return certmagic.OCSPConfig{
DisableStapling: val == "off",
}, nil
}
// parseLogOptions parses the global log option. Syntax:
//
// log [name] {
// output <writer_module> ...
// format <encoder_module> ...
// level <level>
// include <namespaces...>
// exclude <namespaces...>
// }
//
// When the name argument is unspecified, this directive modifies the default
// logger.
func parseLogOptions(d *caddyfile.Dispenser, existingVal any) (any, error) {
currentNames := make(map[string]struct{})
if existingVal != nil {
innerVals, ok := existingVal.([]ConfigValue)
if !ok {
return nil, d.Errf("existing log values of unexpected type: %T", existingVal)
}
for _, rawVal := range innerVals {
val, ok := rawVal.Value.(namedCustomLog)
if !ok {
return nil, d.Errf("existing log value of unexpected type: %T", existingVal)
}
currentNames[val.name] = struct{}{}
}
}
var warnings []caddyconfig.Warning
// Call out the same parser that handles server-specific log configuration.
configValues, err := parseLogHelper(
Helper{
Dispenser: d,
warnings: &warnings,
},
currentNames,
)
if err != nil {
return nil, err
}
if len(warnings) > 0 {
return nil, d.Errf("warnings found in parsing global log options: %+v", warnings)
}
return configValues, nil
}
func parseOptPreferredChains(d *caddyfile.Dispenser, _ any) (any, error) {
d.Next()
return caddytls.ParseCaddyfilePreferredChainsOptions(d)
}
+64
View File
@@ -0,0 +1,64 @@
package httpcaddyfile
import (
"testing"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
_ "github.com/caddyserver/caddy/v2/modules/logging"
)
func TestGlobalLogOptionSyntax(t *testing.T) {
for i, tc := range []struct {
input string
output string
expectError bool
}{
// NOTE: Additional test cases of successful Caddyfile parsing
// are present in: caddytest/integration/caddyfile_adapt/
{
input: `{
log default
}
`,
output: `{}`,
expectError: false,
},
{
input: `{
log example {
output file foo.log
}
log example {
format json
}
}
`,
expectError: true,
},
{
input: `{
log example /foo {
output file foo.log
}
}
`,
expectError: true,
},
} {
adapter := caddyfile.Adapter{
ServerType: ServerType{},
}
out, _, err := adapter.Adapt([]byte(tc.input), nil)
if err != nil != tc.expectError {
t.Errorf("Test %d error expectation failed Expected: %v, got %v", i, tc.expectError, err)
continue
}
if string(out) != tc.output {
t.Errorf("Test %d error output mismatch Expected: %s, got %s", i, tc.output, out)
}
}
}
+218
View File
@@ -0,0 +1,218 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package httpcaddyfile
import (
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddypki"
)
func init() {
RegisterGlobalOption("pki", parsePKIApp)
}
// parsePKIApp parses the global log option. Syntax:
//
// pki {
// ca [<id>] {
// name <name>
// root_cn <name>
// intermediate_cn <name>
// root {
// cert <path>
// key <path>
// format <format>
// }
// intermediate {
// cert <path>
// key <path>
// format <format>
// }
// }
// }
//
// When the CA ID is unspecified, 'local' is assumed.
func parsePKIApp(d *caddyfile.Dispenser, existingVal any) (any, error) {
pki := &caddypki.PKI{CAs: make(map[string]*caddypki.CA)}
for d.Next() {
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "ca":
pkiCa := new(caddypki.CA)
if d.NextArg() {
pkiCa.ID = d.Val()
if d.NextArg() {
return nil, d.ArgErr()
}
}
if pkiCa.ID == "" {
pkiCa.ID = caddypki.DefaultCAID
}
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "name":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.Name = d.Val()
case "root_cn":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.RootCommonName = d.Val()
case "intermediate_cn":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.IntermediateCommonName = d.Val()
case "root":
if pkiCa.Root == nil {
pkiCa.Root = new(caddypki.KeyPair)
}
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "cert":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.Root.Certificate = d.Val()
case "key":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.Root.PrivateKey = d.Val()
case "format":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.Root.Format = d.Val()
default:
return nil, d.Errf("unrecognized pki ca root option '%s'", d.Val())
}
}
case "intermediate":
if pkiCa.Intermediate == nil {
pkiCa.Intermediate = new(caddypki.KeyPair)
}
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "cert":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.Intermediate.Certificate = d.Val()
case "key":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.Intermediate.PrivateKey = d.Val()
case "format":
if !d.NextArg() {
return nil, d.ArgErr()
}
pkiCa.Intermediate.Format = d.Val()
default:
return nil, d.Errf("unrecognized pki ca intermediate option '%s'", d.Val())
}
}
default:
return nil, d.Errf("unrecognized pki ca option '%s'", d.Val())
}
}
pki.CAs[pkiCa.ID] = pkiCa
default:
return nil, d.Errf("unrecognized pki option '%s'", d.Val())
}
}
}
return pki, nil
}
func (st ServerType) buildPKIApp(
pairings []sbAddrAssociation,
options map[string]any,
warnings []caddyconfig.Warning,
) (*caddypki.PKI, []caddyconfig.Warning, error) {
skipInstallTrust := false
if _, ok := options["skip_install_trust"]; ok {
skipInstallTrust = true
}
falseBool := false
// Load the PKI app configured via global options
var pkiApp *caddypki.PKI
unwrappedPki, ok := options["pki"].(*caddypki.PKI)
if ok {
pkiApp = unwrappedPki
} else {
pkiApp = &caddypki.PKI{CAs: make(map[string]*caddypki.CA)}
}
for _, ca := range pkiApp.CAs {
if skipInstallTrust {
ca.InstallTrust = &falseBool
}
pkiApp.CAs[ca.ID] = ca
}
// Add in the CAs configured via directives
for _, p := range pairings {
for _, sblock := range p.serverBlocks {
// find all the CAs that were defined and add them to the app config
// i.e. from any "acme_server" directives
for _, caCfgValue := range sblock.pile["pki.ca"] {
ca := caCfgValue.Value.(*caddypki.CA)
if skipInstallTrust {
ca.InstallTrust = &falseBool
}
// the CA might already exist from global options, so
// don't overwrite it in that case
if _, ok := pkiApp.CAs[ca.ID]; !ok {
pkiApp.CAs[ca.ID] = ca
}
}
}
}
// if there was no CAs defined in any of the servers,
// and we were requested to not install trust, then
// add one for the default/local CA to do so
if len(pkiApp.CAs) == 0 && skipInstallTrust {
ca := new(caddypki.CA)
ca.ID = caddypki.DefaultCAID
ca.InstallTrust = &falseBool
pkiApp.CAs[ca.ID] = ca
}
return pkiApp, warnings, nil
}
+282
View File
@@ -0,0 +1,282 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package httpcaddyfile
import (
"encoding/json"
"fmt"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/dustin/go-humanize"
)
// serverOptions collects server config overrides parsed from Caddyfile global options
type serverOptions struct {
// If set, will only apply these options to servers that contain a
// listener address that matches exactly. If empty, will apply to all
// servers that were not already matched by another serverOptions.
ListenerAddress string
// These will all map 1:1 to the caddyhttp.Server struct
ListenerWrappersRaw []json.RawMessage
ReadTimeout caddy.Duration
ReadHeaderTimeout caddy.Duration
WriteTimeout caddy.Duration
IdleTimeout caddy.Duration
KeepAliveInterval caddy.Duration
MaxHeaderBytes int
Protocols []string
StrictSNIHost *bool
ShouldLogCredentials bool
Metrics *caddyhttp.Metrics
}
func unmarshalCaddyfileServerOptions(d *caddyfile.Dispenser) (any, error) {
serverOpts := serverOptions{}
for d.Next() {
if d.NextArg() {
serverOpts.ListenerAddress = d.Val()
if d.NextArg() {
return nil, d.ArgErr()
}
}
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "listener_wrappers":
for nesting := d.Nesting(); d.NextBlock(nesting); {
modID := "caddy.listeners." + d.Val()
unm, err := caddyfile.UnmarshalModule(d, modID)
if err != nil {
return nil, err
}
listenerWrapper, ok := unm.(caddy.ListenerWrapper)
if !ok {
return nil, fmt.Errorf("module %s (%T) is not a listener wrapper", modID, unm)
}
jsonListenerWrapper := caddyconfig.JSONModuleObject(
listenerWrapper,
"wrapper",
listenerWrapper.(caddy.Module).CaddyModule().ID.Name(),
nil,
)
serverOpts.ListenerWrappersRaw = append(serverOpts.ListenerWrappersRaw, jsonListenerWrapper)
}
case "timeouts":
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "read_body":
if !d.NextArg() {
return nil, d.ArgErr()
}
dur, err := caddy.ParseDuration(d.Val())
if err != nil {
return nil, d.Errf("parsing read_body timeout duration: %v", err)
}
serverOpts.ReadTimeout = caddy.Duration(dur)
case "read_header":
if !d.NextArg() {
return nil, d.ArgErr()
}
dur, err := caddy.ParseDuration(d.Val())
if err != nil {
return nil, d.Errf("parsing read_header timeout duration: %v", err)
}
serverOpts.ReadHeaderTimeout = caddy.Duration(dur)
case "write":
if !d.NextArg() {
return nil, d.ArgErr()
}
dur, err := caddy.ParseDuration(d.Val())
if err != nil {
return nil, d.Errf("parsing write timeout duration: %v", err)
}
serverOpts.WriteTimeout = caddy.Duration(dur)
case "idle":
if !d.NextArg() {
return nil, d.ArgErr()
}
dur, err := caddy.ParseDuration(d.Val())
if err != nil {
return nil, d.Errf("parsing idle timeout duration: %v", err)
}
serverOpts.IdleTimeout = caddy.Duration(dur)
default:
return nil, d.Errf("unrecognized timeouts option '%s'", d.Val())
}
}
case "keepalive_interval":
if !d.NextArg() {
return nil, d.ArgErr()
}
dur, err := caddy.ParseDuration(d.Val())
if err != nil {
return nil, d.Errf("parsing keepalive interval duration: %v", err)
}
serverOpts.KeepAliveInterval = caddy.Duration(dur)
case "max_header_size":
var sizeStr string
if !d.AllArgs(&sizeStr) {
return nil, d.ArgErr()
}
size, err := humanize.ParseBytes(sizeStr)
if err != nil {
return nil, d.Errf("parsing max_header_size: %v", err)
}
serverOpts.MaxHeaderBytes = int(size)
case "log_credentials":
if d.NextArg() {
return nil, d.ArgErr()
}
serverOpts.ShouldLogCredentials = true
case "protocols":
protos := d.RemainingArgs()
for _, proto := range protos {
if proto != "h1" && proto != "h2" && proto != "h2c" && proto != "h3" {
return nil, d.Errf("unknown protocol '%s': expected h1, h2, h2c, or h3", proto)
}
if sliceContains(serverOpts.Protocols, proto) {
return nil, d.Errf("protocol %s specified more than once", proto)
}
serverOpts.Protocols = append(serverOpts.Protocols, proto)
}
if nesting := d.Nesting(); d.NextBlock(nesting) {
return nil, d.ArgErr()
}
case "strict_sni_host":
if d.NextArg() && d.Val() != "insecure_off" && d.Val() != "on" {
return nil, d.Errf("strict_sni_host only supports 'on' or 'insecure_off', got '%s'", d.Val())
}
boolVal := true
if d.Val() == "insecure_off" {
boolVal = false
}
serverOpts.StrictSNIHost = &boolVal
case "metrics":
if d.NextArg() {
return nil, d.ArgErr()
}
if nesting := d.Nesting(); d.NextBlock(nesting) {
return nil, d.ArgErr()
}
serverOpts.Metrics = new(caddyhttp.Metrics)
// TODO: DEPRECATED. (August 2022)
case "protocol":
caddy.Log().Named("caddyfile").Warn("DEPRECATED: protocol sub-option will be removed soon")
for nesting := d.Nesting(); d.NextBlock(nesting); {
switch d.Val() {
case "allow_h2c":
caddy.Log().Named("caddyfile").Warn("DEPRECATED: allow_h2c will be removed soon; use protocols option instead")
if d.NextArg() {
return nil, d.ArgErr()
}
if sliceContains(serverOpts.Protocols, "h2c") {
return nil, d.Errf("protocol h2c already specified")
}
serverOpts.Protocols = append(serverOpts.Protocols, "h2c")
case "strict_sni_host":
caddy.Log().Named("caddyfile").Warn("DEPRECATED: protocol > strict_sni_host in this position will be removed soon; move up to the servers block instead")
if d.NextArg() && d.Val() != "insecure_off" && d.Val() != "on" {
return nil, d.Errf("strict_sni_host only supports 'on' or 'insecure_off', got '%s'", d.Val())
}
boolVal := true
if d.Val() == "insecure_off" {
boolVal = false
}
serverOpts.StrictSNIHost = &boolVal
default:
return nil, d.Errf("unrecognized protocol option '%s'", d.Val())
}
}
default:
return nil, d.Errf("unrecognized servers option '%s'", d.Val())
}
}
}
return serverOpts, nil
}
// applyServerOptions sets the server options on the appropriate servers
func applyServerOptions(
servers map[string]*caddyhttp.Server,
options map[string]any,
warnings *[]caddyconfig.Warning,
) error {
serverOpts, ok := options["servers"].([]serverOptions)
if !ok {
return nil
}
for _, server := range servers {
// find the options that apply to this server
opts := func() *serverOptions {
for _, entry := range serverOpts {
if entry.ListenerAddress == "" {
return &entry
}
for _, listener := range server.Listen {
if entry.ListenerAddress == listener {
return &entry
}
}
}
return nil
}()
// if none apply, then move to the next server
if opts == nil {
continue
}
// set all the options
server.ListenerWrappersRaw = opts.ListenerWrappersRaw
server.ReadTimeout = opts.ReadTimeout
server.ReadHeaderTimeout = opts.ReadHeaderTimeout
server.WriteTimeout = opts.WriteTimeout
server.IdleTimeout = opts.IdleTimeout
server.KeepAliveInterval = opts.KeepAliveInterval
server.MaxHeaderBytes = opts.MaxHeaderBytes
server.Protocols = opts.Protocols
server.StrictSNIHost = opts.StrictSNIHost
server.Metrics = opts.Metrics
if opts.ShouldLogCredentials {
if server.Logs == nil {
server.Logs = &caddyhttp.ServerLogConfig{}
}
server.Logs.ShouldLogCredentials = opts.ShouldLogCredentials
}
}
return nil
}
+441 -226
View File
@@ -21,219 +21,237 @@ import (
"reflect"
"sort"
"strconv"
"strings"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/caddyserver/caddy/v2/modules/caddytls"
"github.com/caddyserver/certmagic"
"github.com/mholt/acmez/acme"
)
func (st ServerType) buildTLSApp(
pairings []sbAddrAssociation,
options map[string]interface{},
options map[string]any,
warnings []caddyconfig.Warning,
) (*caddytls.TLS, []caddyconfig.Warning, error) {
tlsApp := &caddytls.TLS{CertificatesRaw: make(caddy.ModuleMap)}
var certLoaders []caddytls.CertificateLoader
httpsPort := strconv.Itoa(caddyhttp.DefaultHTTPSPort)
if hsp, ok := options["https_port"].(int); ok {
httpsPort = strconv.Itoa(hsp)
httpPort := strconv.Itoa(caddyhttp.DefaultHTTPPort)
if hp, ok := options["http_port"].(int); ok {
httpPort = strconv.Itoa(hp)
}
autoHTTPS := "on"
if ah, ok := options["auto_https"].(string); ok {
autoHTTPS = ah
}
// count how many server blocks have a TLS-enabled key with
// no host, and find all hosts that share a server block with
// a hostless key, so that they don't get forgotten/omitted
// by auto-HTTPS (since they won't appear in route matchers)
var serverBlocksWithTLSHostlessKey int
hostsSharedWithHostlessKey := make(map[string]struct{})
for _, pair := range pairings {
for _, sb := range pair.serverBlocks {
for _, addr := range sb.keys {
if addr.Host == "" {
// this address has no hostname, but if it's explicitly set
// to HTTPS, then we need to count it as being TLS-enabled
if addr.Scheme == "https" || addr.Port == httpsPort {
serverBlocksWithTLSHostlessKey++
}
// this server block has a hostless key, now
// go through and add all the hosts to the set
for _, otherAddr := range sb.keys {
if otherAddr.Original == addr.Original {
continue
}
if otherAddr.Host != "" {
hostsSharedWithHostlessKey[otherAddr.Host] = struct{}{}
// find all hosts that share a server block with a hostless
// key, so that they don't get forgotten/omitted by auto-HTTPS
// (since they won't appear in route matchers)
httpsHostsSharedWithHostlessKey := make(map[string]struct{})
if autoHTTPS != "off" {
for _, pair := range pairings {
for _, sb := range pair.serverBlocks {
for _, addr := range sb.keys {
if addr.Host == "" {
// this server block has a hostless key, now
// go through and add all the hosts to the set
for _, otherAddr := range sb.keys {
if otherAddr.Original == addr.Original {
continue
}
if otherAddr.Host != "" && otherAddr.Scheme != "http" && otherAddr.Port != httpPort {
httpsHostsSharedWithHostlessKey[otherAddr.Host] = struct{}{}
}
}
break
}
break
}
}
}
}
// a catch-all automation policy is used as a "default" for all subjects that
// don't have custom configuration explicitly associated with them; this
// is only to add if the global settings or defaults are non-empty
catchAllAP, err := newBaseAutomationPolicy(options, warnings, false)
if err != nil {
return nil, warnings, err
}
if catchAllAP != nil {
if tlsApp.Automation == nil {
tlsApp.Automation = new(caddytls.AutomationConfig)
}
tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, catchAllAP)
}
for _, p := range pairings {
// avoid setting up TLS automation policies for a server that is HTTP-only
if !listenersUseAnyPortOtherThan(p.addresses, httpPort) {
continue
}
for _, sblock := range p.serverBlocks {
// check the scheme of all the site addresses,
// skip building AP if they all had http://
if sblock.isAllHTTP() {
continue
}
// get values that populate an automation policy for this block
var ap *caddytls.AutomationPolicy
ap, err := newBaseAutomationPolicy(options, warnings, true)
if err != nil {
return nil, warnings, err
}
sblockHosts := sblock.hostsFromKeys(false)
if len(sblockHosts) == 0 {
if len(sblockHosts) == 0 && catchAllAP != nil {
ap = catchAllAP
}
// on-demand tls
if _, ok := sblock.pile["tls.on_demand"]; ok {
if ap == nil {
var err error
ap, err = newBaseAutomationPolicy(options, warnings, true)
if err != nil {
return nil, warnings, err
}
}
ap.OnDemand = true
}
if keyTypeVals, ok := sblock.pile["tls.key_type"]; ok {
ap.KeyType = keyTypeVals[0].Value.(string)
}
// certificate issuers
if issuerVals, ok := sblock.pile["tls.cert_issuer"]; ok {
var issuers []certmagic.Issuer
for _, issuerVal := range issuerVals {
issuer := issuerVal.Value.(certmagic.Issuer)
if ap == nil {
var err error
ap, err = newBaseAutomationPolicy(options, warnings, true)
if err != nil {
return nil, warnings, err
}
}
if ap == catchAllAP && !reflect.DeepEqual(ap.Issuer, issuer) {
return nil, warnings, fmt.Errorf("automation policy from site block is also default/catch-all policy because of key without hostname, and the two are in conflict: %#v != %#v", ap.Issuer, issuer)
}
ap.Issuer = issuer
issuers = append(issuers, issuerVal.Value.(certmagic.Issuer))
}
if ap == catchAllAP && !reflect.DeepEqual(ap.Issuers, issuers) {
// this more correctly implements an error check that was removed
// below; try it with this config:
//
// :443 {
// bind 127.0.0.1
// }
//
// :443 {
// bind ::1
// tls {
// issuer acme
// }
// }
return nil, warnings, fmt.Errorf("automation policy from site block is also default/catch-all policy because of key without hostname, and the two are in conflict: %#v != %#v", ap.Issuers, issuers)
}
ap.Issuers = issuers
}
// certificate managers
if certManagerVals, ok := sblock.pile["tls.cert_manager"]; ok {
for _, certManager := range certManagerVals {
certGetterName := certManager.Value.(caddy.Module).CaddyModule().ID.Name()
ap.ManagersRaw = append(ap.ManagersRaw, caddyconfig.JSONModuleObject(certManager.Value, "via", certGetterName, &warnings))
}
}
// custom bind host
for _, cfgVal := range sblock.pile["bind"] {
// either an existing issuer is already configured (and thus, ap is not
// nil), or we need to configure an issuer, so we need ap to be non-nil
if ap == nil {
ap, err = newBaseAutomationPolicy(options, warnings, true)
if err != nil {
return nil, warnings, err
for _, iss := range ap.Issuers {
// if an issuer was already configured and it is NOT an ACME issuer,
// skip, since we intend to adjust only ACME issuers; ensure we
// include any issuer that embeds/wraps an underlying ACME issuer
var acmeIssuer *caddytls.ACMEIssuer
if acmeWrapper, ok := iss.(acmeCapable); ok {
acmeIssuer = acmeWrapper.GetACMEIssuer()
}
if acmeIssuer == nil {
continue
}
}
// if an issuer was already configured and it is NOT an ACME
// issuer, skip, since we intend to adjust only ACME issuers
var acmeIssuer *caddytls.ACMEIssuer
if ap.Issuer != nil {
var ok bool
if acmeIssuer, ok = ap.Issuer.(*caddytls.ACMEIssuer); !ok {
break
// proceed to configure the ACME issuer's bind host, without
// overwriting any existing settings
if acmeIssuer.Challenges == nil {
acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
}
if acmeIssuer.Challenges.BindHost == "" {
// only binding to one host is supported
var bindHost string
if bindHosts, ok := cfgVal.Value.([]string); ok && len(bindHosts) > 0 {
bindHost = bindHosts[0]
}
acmeIssuer.Challenges.BindHost = bindHost
}
}
// proceed to configure the ACME issuer's bind host, without
// overwriting any existing settings
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
if acmeIssuer.Challenges == nil {
acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
}
if acmeIssuer.Challenges.BindHost == "" {
// only binding to one host is supported
var bindHost string
if bindHosts, ok := cfgVal.Value.([]string); ok && len(bindHosts) > 0 {
bindHost = bindHosts[0]
}
acmeIssuer.Challenges.BindHost = bindHost
}
ap.Issuer = acmeIssuer // we'll encode it later
}
if ap != nil {
if ap.Issuer != nil {
// encode issuer now that it's all set up
issuerName := ap.Issuer.(caddy.Module).CaddyModule().ID.Name()
ap.IssuerRaw = caddyconfig.JSONModuleObject(ap.Issuer, "module", issuerName, &warnings)
}
// we used to ensure this block is allowed to create an automation policy;
// doing so was forbidden if it has a key with no host (i.e. ":443")
// and if there is a different server block that also has a key with no
// host -- since a key with no host matches any host, we need its
// associated automation policy to have an empty Subjects list, i.e. no
// host filter, which is indistinguishable between the two server blocks
// because automation is not done in the context of a particular server...
// this is an example of a poor mapping from Caddyfile to JSON but that's
// the least-leaky abstraction I could figure out -- however, this check
// was preventing certain listeners, like those provided by plugins, from
// being used as desired (see the Tailscale listener plugin), so I removed
// the check: and I think since I originally wrote the check I added a new
// check above which *properly* detects this ambiguity without breaking the
// listener plugin; see the check above with a commented example config
if len(sblockHosts) == 0 && catchAllAP == nil {
// this server block has a key with no hosts, but there is not yet
// a catch-all automation policy (probably because no global options
// were set), so this one becomes it
catchAllAP = ap
}
// first make sure this block is allowed to create an automation policy;
// doing so is forbidden if it has a key with no host (i.e. ":443")
// and if there is a different server block that also has a key with no
// host -- since a key with no host matches any host, we need its
// associated automation policy to have an empty Subjects list, i.e. no
// host filter, which is indistinguishable between the two server blocks
// because automation is not done in the context of a particular server...
// this is an example of a poor mapping from Caddyfile to JSON but that's
// the least-leaky abstraction I could figure out
if len(sblockHosts) == 0 {
if serverBlocksWithTLSHostlessKey > 1 {
// this server block and at least one other has a key with no host,
// making the two indistinguishable; it is misleading to define such
// a policy within one server block since it actually will apply to
// others as well
return nil, warnings, fmt.Errorf("cannot make a TLS automation policy from a server block that has a host-less address when there are other TLS-enabled server block addresses lacking a host")
// associate our new automation policy with this server block's hosts
ap.Subjects = sblock.hostsFromKeysNotHTTP(httpPort)
sort.Strings(ap.Subjects) // solely for deterministic test results
// if a combination of public and internal names were given
// for this same server block and no issuer was specified, we
// need to separate them out in the automation policies so
// that the internal names can use the internal issuer and
// the other names can use the default/public/ACME issuer
var ap2 *caddytls.AutomationPolicy
if len(ap.Issuers) == 0 {
var internal, external []string
for _, s := range ap.Subjects {
if !certmagic.SubjectQualifiesForCert(s) {
return nil, warnings, fmt.Errorf("subject does not qualify for certificate: '%s'", s)
}
if catchAllAP == nil {
// this server block has a key with no hosts, but there is not yet
// a catch-all automation policy (probably because no global options
// were set), so this one becomes it
catchAllAP = ap
// we don't use certmagic.SubjectQualifiesForPublicCert() because of one nuance:
// names like *.*.tld that may not qualify for a public certificate are actually
// fine when used with OnDemand, since OnDemand (currently) does not obtain
// wildcards (if it ever does, there will be a separate config option to enable
// it that we would need to check here) since the hostname is known at handshake;
// and it is unexpected to switch to internal issuer when the user wants to get
// regular certificates on-demand for a class of certs like *.*.tld.
if subjectQualifiesForPublicCert(ap, s) {
external = append(external, s)
} else {
internal = append(internal, s)
}
}
// associate our new automation policy with this server block's hosts,
// unless, of course, the server block has a key with no hosts, in which
// case its automation policy becomes or blends with the default/global
// automation policy because, of necessity, it applies to all hostnames
// (i.e. it has no Subjects filter) -- in that case, we'll append it last
if ap != catchAllAP {
ap.Subjects = sblockHosts
// if a combination of public and internal names were given
// for this same server block and no issuer was specified, we
// need to separate them out in the automation policies so
// that the internal names can use the internal issuer and
// the other names can use the default/public/ACME issuer
var ap2 *caddytls.AutomationPolicy
if ap.Issuer == nil {
var internal, external []string
for _, s := range ap.Subjects {
if certmagic.SubjectQualifiesForPublicCert(s) {
external = append(external, s)
} else {
internal = append(internal, s)
}
}
if len(external) > 0 && len(internal) > 0 {
ap.Subjects = external
apCopy := *ap
ap2 = &apCopy
ap2.Subjects = internal
ap2.IssuerRaw = caddyconfig.JSONModuleObject(caddytls.InternalIssuer{}, "module", "internal", &warnings)
}
}
if tlsApp.Automation == nil {
tlsApp.Automation = new(caddytls.AutomationConfig)
}
tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, ap)
if ap2 != nil {
tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, ap2)
}
if len(external) > 0 && len(internal) > 0 {
ap.Subjects = external
apCopy := *ap
ap2 = &apCopy
ap2.Subjects = internal
ap2.IssuersRaw = []json.RawMessage{caddyconfig.JSONModuleObject(caddytls.InternalIssuer{}, "module", "internal", &warnings)}
}
}
if tlsApp.Automation == nil {
tlsApp.Automation = new(caddytls.AutomationConfig)
}
tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, ap)
if ap2 != nil {
tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, ap2)
}
// certificate loaders
if clVals, ok := sblock.pile["tls.certificate_loader"]; ok {
if clVals, ok := sblock.pile["tls.cert_loader"]; ok {
for _, clVal := range clVals {
certLoaders = append(certLoaders, clVal.Value.(caddytls.CertificateLoader))
}
@@ -277,6 +295,35 @@ func (st ServerType) buildTLSApp(
tlsApp.Automation.OnDemand = onDemand
}
// set the storage clean interval if configured
if storageCleanInterval, ok := options["storage_clean_interval"].(caddy.Duration); ok {
if tlsApp.Automation == nil {
tlsApp.Automation = new(caddytls.AutomationConfig)
}
tlsApp.Automation.StorageCleanInterval = storageCleanInterval
}
// set the expired certificates renew interval if configured
if renewCheckInterval, ok := options["renew_interval"].(caddy.Duration); ok {
if tlsApp.Automation == nil {
tlsApp.Automation = new(caddytls.AutomationConfig)
}
tlsApp.Automation.RenewCheckInterval = renewCheckInterval
}
// set the OCSP check interval if configured
if ocspCheckInterval, ok := options["ocsp_interval"].(caddy.Duration); ok {
if tlsApp.Automation == nil {
tlsApp.Automation = new(caddytls.AutomationConfig)
}
tlsApp.Automation.OCSPCheckInterval = ocspCheckInterval
}
// set whether OCSP stapling should be disabled for manually-managed certificates
if ocspConfig, ok := options["ocsp_stapling"].(certmagic.OCSPConfig); ok {
tlsApp.DisableOCSPStapling = ocspConfig.DisableStapling
}
// if any hostnames appear on the same server block as a key with
// no host, they will not be used with route matchers because the
// hostless key matches all hosts, therefore, it wouldn't be
@@ -286,12 +333,14 @@ func (st ServerType) buildTLSApp(
// get internal certificates by default rather than ACME
var al caddytls.AutomateLoader
internalAP := &caddytls.AutomationPolicy{
IssuerRaw: json.RawMessage(`{"module":"internal"}`),
IssuersRaw: []json.RawMessage{json.RawMessage(`{"module":"internal"}`)},
}
for h := range hostsSharedWithHostlessKey {
al = append(al, h)
if !certmagic.SubjectQualifiesForPublicCert(h) {
internalAP.Subjects = append(internalAP.Subjects, h)
if autoHTTPS != "off" {
for h := range httpsHostsSharedWithHostlessKey {
al = append(al, h)
if !certmagic.SubjectQualifiesForPublicCert(h) {
internalAP.Subjects = append(internalAP.Subjects, h)
}
}
}
if len(al) > 0 {
@@ -304,23 +353,60 @@ func (st ServerType) buildTLSApp(
tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, internalAP)
}
// if there is a global/catch-all automation policy, ensure it goes last
if catchAllAP != nil {
// first, encode its issuer, if there is one
if catchAllAP.Issuer != nil {
issuerName := catchAllAP.Issuer.(caddy.Module).CaddyModule().ID.Name()
catchAllAP.IssuerRaw = caddyconfig.JSONModuleObject(catchAllAP.Issuer, "module", issuerName, &warnings)
}
// if there are any global options set for issuers (ACME ones in particular), make sure they
// take effect in every automation policy that does not have any issuers
if tlsApp.Automation != nil {
globalEmail := options["email"]
globalACMECA := options["acme_ca"]
globalACMECARoot := options["acme_ca_root"]
globalACMEDNS := options["acme_dns"]
globalACMEEAB := options["acme_eab"]
globalPreferredChains := options["preferred_chains"]
hasGlobalACMEDefaults := globalEmail != nil || globalACMECA != nil || globalACMECARoot != nil || globalACMEDNS != nil || globalACMEEAB != nil || globalPreferredChains != nil
if hasGlobalACMEDefaults {
for i := 0; i < len(tlsApp.Automation.Policies); i++ {
ap := tlsApp.Automation.Policies[i]
if len(ap.Issuers) == 0 && automationPolicyHasAllPublicNames(ap) {
// for public names, create default issuers which will later be filled in with configured global defaults
// (internal names will implicitly use the internal issuer at auto-https time)
ap.Issuers = caddytls.DefaultIssuers()
// then append it to the end of the policies list
if tlsApp.Automation == nil {
tlsApp.Automation = new(caddytls.AutomationConfig)
// if a specific endpoint is configured, can't use multiple default issuers
if globalACMECA != nil {
if strings.Contains(globalACMECA.(string), "zerossl") {
ap.Issuers = []certmagic.Issuer{&caddytls.ZeroSSLIssuer{ACMEIssuer: new(caddytls.ACMEIssuer)}}
} else {
ap.Issuers = []certmagic.Issuer{new(caddytls.ACMEIssuer)}
}
}
}
}
}
tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, catchAllAP)
}
// do a little verification & cleanup
// finalize and verify policies; do cleanup
if tlsApp.Automation != nil {
for i, ap := range tlsApp.Automation.Policies {
// ensure all issuers have global defaults filled in
for j, issuer := range ap.Issuers {
err := fillInGlobalACMEDefaults(issuer, options)
if err != nil {
return nil, warnings, fmt.Errorf("filling in global issuer defaults for AP %d, issuer %d: %v", i, j, err)
}
}
// encode all issuer values we created, so they will be rendered in the output
if len(ap.Issuers) > 0 && ap.IssuersRaw == nil {
for _, iss := range ap.Issuers {
issuerName := iss.(caddy.Module).CaddyModule().ID.Name()
ap.IssuersRaw = append(ap.IssuersRaw, caddyconfig.JSONModuleObject(iss, "module", issuerName, &warnings))
}
}
}
// consolidate automation policies that are the exact same
tlsApp.Automation.Policies = consolidateAutomationPolicies(tlsApp.Automation.Policies)
// ensure automation policies don't overlap subjects (this should be
// an error at provision-time as well, but catch it in the adapt phase
// for convenience)
@@ -334,27 +420,74 @@ func (st ServerType) buildTLSApp(
}
}
// consolidate automation policies that are the exact same
tlsApp.Automation.Policies = consolidateAutomationPolicies(tlsApp.Automation.Policies)
// if nothing remains, remove any excess values to clean up the resulting config
if len(tlsApp.Automation.Policies) == 0 {
tlsApp.Automation.Policies = nil
}
if reflect.DeepEqual(tlsApp.Automation, new(caddytls.AutomationConfig)) {
tlsApp.Automation = nil
}
}
return tlsApp, warnings, nil
}
type acmeCapable interface{ GetACMEIssuer() *caddytls.ACMEIssuer }
func fillInGlobalACMEDefaults(issuer certmagic.Issuer, options map[string]any) error {
acmeWrapper, ok := issuer.(acmeCapable)
if !ok {
return nil
}
acmeIssuer := acmeWrapper.GetACMEIssuer()
if acmeIssuer == nil {
return nil
}
globalEmail := options["email"]
globalACMECA := options["acme_ca"]
globalACMECARoot := options["acme_ca_root"]
globalACMEDNS := options["acme_dns"]
globalACMEEAB := options["acme_eab"]
globalPreferredChains := options["preferred_chains"]
if globalEmail != nil && acmeIssuer.Email == "" {
acmeIssuer.Email = globalEmail.(string)
}
if globalACMECA != nil && acmeIssuer.CA == "" {
acmeIssuer.CA = globalACMECA.(string)
}
if globalACMECARoot != nil && !sliceContains(acmeIssuer.TrustedRootsPEMFiles, globalACMECARoot.(string)) {
acmeIssuer.TrustedRootsPEMFiles = append(acmeIssuer.TrustedRootsPEMFiles, globalACMECARoot.(string))
}
if globalACMEDNS != nil && (acmeIssuer.Challenges == nil || acmeIssuer.Challenges.DNS == nil) {
acmeIssuer.Challenges = &caddytls.ChallengesConfig{
DNS: &caddytls.DNSChallengeConfig{
ProviderRaw: caddyconfig.JSONModuleObject(globalACMEDNS, "name", globalACMEDNS.(caddy.Module).CaddyModule().ID.Name(), nil),
},
}
}
if globalACMEEAB != nil && acmeIssuer.ExternalAccount == nil {
acmeIssuer.ExternalAccount = globalACMEEAB.(*acme.EAB)
}
if globalPreferredChains != nil && acmeIssuer.PreferredChains == nil {
acmeIssuer.PreferredChains = globalPreferredChains.(*caddytls.ChainPreference)
}
return nil
}
// newBaseAutomationPolicy returns a new TLS automation policy that gets
// its values from the global options map. It should be used as the base
// for any other automation policies. A nil policy (and no error) will be
// returned if there are no default/global options. However, if always is
// true, a non-nil value will always be returned (unless there is an error).
func newBaseAutomationPolicy(options map[string]interface{}, warnings []caddyconfig.Warning, always bool) (*caddytls.AutomationPolicy, error) {
acmeCA, hasACMECA := options["acme_ca"]
acmeDNS, hasACMEDNS := options["acme_dns"]
acmeCARoot, hasACMECARoot := options["acme_ca_root"]
email, hasEmail := options["email"]
localCerts, hasLocalCerts := options["local_certs"]
func newBaseAutomationPolicy(options map[string]any, warnings []caddyconfig.Warning, always bool) (*caddytls.AutomationPolicy, error) {
issuers, hasIssuers := options["cert_issuer"]
_, hasLocalCerts := options["local_certs"]
keyType, hasKeyType := options["key_type"]
ocspStapling, hasOCSPStapling := options["ocsp_stapling"]
hasGlobalAutomationOpts := hasACMECA || hasACMEDNS || hasACMECARoot || hasEmail || hasLocalCerts || hasKeyType
hasGlobalAutomationOpts := hasIssuers || hasLocalCerts || hasKeyType || hasOCSPStapling
// if there are no global options related to automation policies
// set, then we can just return right away
@@ -366,40 +499,24 @@ func newBaseAutomationPolicy(options map[string]interface{}, warnings []caddycon
}
ap := new(caddytls.AutomationPolicy)
if hasKeyType {
ap.KeyType = keyType.(string)
}
if localCerts != nil {
// internal issuer enabled trumps any ACME configurations; useful in testing
ap.Issuer = new(caddytls.InternalIssuer) // we'll encode it later
} else {
if acmeCA == nil {
acmeCA = ""
}
if email == nil {
email = ""
}
mgr := &caddytls.ACMEIssuer{
CA: acmeCA.(string),
Email: email.(string),
}
if acmeDNS != nil {
provName := acmeDNS.(string)
dnsProvModule, err := caddy.GetModule("dns.providers." + provName)
if err != nil {
return nil, fmt.Errorf("getting DNS provider module named '%s': %v", provName, err)
}
mgr.Challenges = &caddytls.ChallengesConfig{
DNS: &caddytls.DNSChallengeConfig{
ProviderRaw: caddyconfig.JSONModuleObject(dnsProvModule.New(), "name", provName, &warnings),
},
}
}
if acmeCARoot != nil {
mgr.TrustedRootsPEMFiles = []string{acmeCARoot.(string)}
}
if keyType != nil {
ap.KeyType = keyType.(string)
}
ap.Issuer = mgr // we'll encode it later
if hasIssuers && hasLocalCerts {
return nil, fmt.Errorf("global options are ambiguous: local_certs is confusing when combined with cert_issuer, because local_certs is also a specific kind of issuer")
}
if hasIssuers {
ap.Issuers = issuers.([]certmagic.Issuer)
} else if hasLocalCerts {
ap.Issuers = []certmagic.Issuer{new(caddytls.InternalIssuer)}
}
if hasOCSPStapling {
ocspConfig := ocspStapling.(certmagic.OCSPConfig)
ap.DisableOCSPStapling = ocspConfig.DisableStapling
ap.OCSPOverrides = ocspConfig.ResponderOverrides
}
return ap, nil
@@ -408,17 +525,50 @@ func newBaseAutomationPolicy(options map[string]interface{}, warnings []caddycon
// consolidateAutomationPolicies combines automation policies that are the same,
// for a cleaner overall output.
func consolidateAutomationPolicies(aps []*caddytls.AutomationPolicy) []*caddytls.AutomationPolicy {
for i := 0; i < len(aps); i++ {
for j := 0; j < len(aps); j++ {
if j == i {
continue
}
// sort from most specific to least specific; we depend on this ordering
sort.SliceStable(aps, func(i, j int) bool {
if automationPolicyIsSubset(aps[i], aps[j]) {
return true
}
if automationPolicyIsSubset(aps[j], aps[i]) {
return false
}
return len(aps[i].Subjects) > len(aps[j].Subjects)
})
emptyAPCount := 0
origLenAPs := len(aps)
// compute the number of empty policies (disregarding subjects) - see #4128
emptyAP := new(caddytls.AutomationPolicy)
for i := 0; i < len(aps); i++ {
emptyAP.Subjects = aps[i].Subjects
if reflect.DeepEqual(aps[i], emptyAP) {
emptyAPCount++
if !automationPolicyHasAllPublicNames(aps[i]) {
// if this automation policy has internal names, we might as well remove it
// so auto-https can implicitly use the internal issuer
aps = append(aps[:i], aps[i+1:]...)
i--
}
}
}
// If all policies are empty, we can return nil, as there is no need to set any policy
if emptyAPCount == origLenAPs {
return nil
}
// remove or combine duplicate policies
outer:
for i := 0; i < len(aps); i++ {
// compare only with next policies; we sorted by specificity so we must not delete earlier policies
for j := i + 1; j < len(aps); j++ {
// if they're exactly equal in every way, just keep one of them
if reflect.DeepEqual(aps[i], aps[j]) {
aps = append(aps[:j], aps[j+1:]...)
// must re-evaluate current i against next j; can't skip it!
// even if i decrements to -1, will be incremented to 0 immediately
i--
break
continue outer
}
// if the policy is the same, we can keep just one, but we have
@@ -427,30 +577,95 @@ func consolidateAutomationPolicies(aps []*caddytls.AutomationPolicy) []*caddytls
// otherwise the one without any subjects (a catch-all) would be
// eaten up by the one with subjects; and if both have subjects, we
// need to combine their lists
if bytes.Equal(aps[i].IssuerRaw, aps[j].IssuerRaw) &&
if reflect.DeepEqual(aps[i].IssuersRaw, aps[j].IssuersRaw) &&
bytes.Equal(aps[i].StorageRaw, aps[j].StorageRaw) &&
aps[i].MustStaple == aps[j].MustStaple &&
aps[i].KeyType == aps[j].KeyType &&
aps[i].OnDemand == aps[j].OnDemand &&
aps[i].RenewalWindowRatio == aps[j].RenewalWindowRatio {
if len(aps[i].Subjects) == 0 && len(aps[j].Subjects) > 0 {
aps = append(aps[:j], aps[j+1:]...)
} else if len(aps[i].Subjects) > 0 && len(aps[j].Subjects) == 0 {
aps = append(aps[:i], aps[i+1:]...)
if len(aps[i].Subjects) > 0 && len(aps[j].Subjects) == 0 {
// later policy (at j) has no subjects ("catch-all"), so we can
// remove the identical-but-more-specific policy that comes first
// AS LONG AS it is not shadowed by another policy before it; e.g.
// if policy i is for example.com, policy i+1 is '*.com', and policy
// j is catch-all, we cannot remove policy i because that would
// cause example.com to be served by the less specific policy for
// '*.com', which might be different (yes we've seen this happen)
if automationPolicyShadows(i, aps) >= j {
aps = append(aps[:i], aps[i+1:]...)
i--
continue outer
}
} else {
aps[i].Subjects = append(aps[i].Subjects, aps[j].Subjects...)
// avoid repeated subjects
for _, subj := range aps[j].Subjects {
if !sliceContains(aps[i].Subjects, subj) {
aps[i].Subjects = append(aps[i].Subjects, subj)
}
}
aps = append(aps[:j], aps[j+1:]...)
j--
}
i--
break
}
}
}
// ensure any catch-all policies go last
sort.SliceStable(aps, func(i, j int) bool {
return len(aps[i].Subjects) > len(aps[j].Subjects)
})
return aps
}
// automationPolicyIsSubset returns true if a's subjects are a subset
// of b's subjects.
func automationPolicyIsSubset(a, b *caddytls.AutomationPolicy) bool {
if len(b.Subjects) == 0 {
return true
}
if len(a.Subjects) == 0 {
return false
}
for _, aSubj := range a.Subjects {
var inSuperset bool
for _, bSubj := range b.Subjects {
if certmagic.MatchWildcard(aSubj, bSubj) {
inSuperset = true
break
}
}
if !inSuperset {
return false
}
}
return true
}
// automationPolicyShadows returns the index of a policy that aps[i] shadows;
// in other words, for all policies after position i, if that policy covers
// the same subjects but is less specific, that policy's position is returned,
// or -1 if no shadowing is found. For example, if policy i is for
// "foo.example.com" and policy i+2 is for "*.example.com", then i+2 will be
// returned, since that policy is shadowed by i, which is in front.
func automationPolicyShadows(i int, aps []*caddytls.AutomationPolicy) int {
for j := i + 1; j < len(aps); j++ {
if automationPolicyIsSubset(aps[i], aps[j]) {
return j
}
}
return -1
}
// subjectQualifiesForPublicCert is like certmagic.SubjectQualifiesForPublicCert() except
// that this allows domains with multiple wildcard levels like '*.*.example.com' to qualify
// if the automation policy has OnDemand enabled (i.e. this function is more lenient).
func subjectQualifiesForPublicCert(ap *caddytls.AutomationPolicy, subj string) bool {
return !certmagic.SubjectIsIP(subj) &&
!certmagic.SubjectIsInternal(subj) &&
(strings.Count(subj, "*.") < 2 || ap.OnDemand)
}
func automationPolicyHasAllPublicNames(ap *caddytls.AutomationPolicy) bool {
for _, subj := range ap.Subjects {
if !subjectQualifiesForPublicCert(ap, subj) {
return false
}
}
return true
}
+56
View File
@@ -0,0 +1,56 @@
package httpcaddyfile
import (
"testing"
"github.com/caddyserver/caddy/v2/modules/caddytls"
)
func TestAutomationPolicyIsSubset(t *testing.T) {
for i, test := range []struct {
a, b []string
expect bool
}{
{
a: []string{"example.com"},
b: []string{},
expect: true,
},
{
a: []string{},
b: []string{"example.com"},
expect: false,
},
{
a: []string{"foo.example.com"},
b: []string{"*.example.com"},
expect: true,
},
{
a: []string{"foo.example.com"},
b: []string{"foo.example.com"},
expect: true,
},
{
a: []string{"foo.example.com"},
b: []string{"example.com"},
expect: false,
},
{
a: []string{"example.com", "foo.example.com"},
b: []string{"*.com", "*.*.com"},
expect: true,
},
{
a: []string{"example.com", "foo.example.com"},
b: []string{"*.com"},
expect: false,
},
} {
apA := &caddytls.AutomationPolicy{Subjects: test.a}
apB := &caddytls.AutomationPolicy{Subjects: test.b}
if actual := automationPolicyIsSubset(apA, apB); actual != test.expect {
t.Errorf("Test %d: Expected %t but got %t (A: %v B: %v)", i, test.expect, actual, test.a, test.b)
}
}
}
+204
View File
@@ -0,0 +1,204 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyconfig
import (
"crypto/tls"
"crypto/x509"
"fmt"
"io"
"net/http"
"os"
"time"
"github.com/caddyserver/caddy/v2"
)
func init() {
caddy.RegisterModule(HTTPLoader{})
}
// HTTPLoader can load Caddy configs over HTTP(S). It can adapt the config
// based on the Content-Type header of the HTTP response.
type HTTPLoader struct {
// The method for the request. Default: GET
Method string `json:"method,omitempty"`
// The URL of the request.
URL string `json:"url,omitempty"`
// HTTP headers to add to the request.
Headers http.Header `json:"header,omitempty"`
// Maximum time allowed for a complete connection and request.
Timeout caddy.Duration `json:"timeout,omitempty"`
TLS *struct {
// Present this instance's managed remote identity credentials to the server.
UseServerIdentity bool `json:"use_server_identity,omitempty"`
// PEM-encoded client certificate filename to present to the server.
ClientCertificateFile string `json:"client_certificate_file,omitempty"`
// PEM-encoded key to use with the client certificate.
ClientCertificateKeyFile string `json:"client_certificate_key_file,omitempty"`
// List of PEM-encoded CA certificate files to add to the same trust
// store as RootCAPool (or root_ca_pool in the JSON).
RootCAPEMFiles []string `json:"root_ca_pem_files,omitempty"`
} `json:"tls,omitempty"`
}
// CaddyModule returns the Caddy module information.
func (HTTPLoader) CaddyModule() caddy.ModuleInfo {
return caddy.ModuleInfo{
ID: "caddy.config_loaders.http",
New: func() caddy.Module { return new(HTTPLoader) },
}
}
// LoadConfig loads a Caddy config.
func (hl HTTPLoader) LoadConfig(ctx caddy.Context) ([]byte, error) {
repl := caddy.NewReplacer()
client, err := hl.makeClient(ctx)
if err != nil {
return nil, err
}
method := repl.ReplaceAll(hl.Method, "")
if method == "" {
method = http.MethodGet
}
url := repl.ReplaceAll(hl.URL, "")
req, err := http.NewRequest(method, url, nil)
if err != nil {
return nil, err
}
for key, vals := range hl.Headers {
for _, val := range vals {
req.Header.Add(repl.ReplaceAll(key, ""), repl.ReplaceKnown(val, ""))
}
}
resp, err := doHttpCallWithRetries(ctx, client, req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode >= 400 {
return nil, fmt.Errorf("server responded with HTTP %d", resp.StatusCode)
}
body, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
result, warnings, err := adaptByContentType(resp.Header.Get("Content-Type"), body)
if err != nil {
return nil, err
}
for _, warn := range warnings {
ctx.Logger().Warn(warn.String())
}
return result, nil
}
func attemptHttpCall(client *http.Client, request *http.Request) (*http.Response, error) {
resp, err := client.Do(request)
if err != nil {
return nil, fmt.Errorf("problem calling http loader url: %v", err)
} else if resp.StatusCode < 200 || resp.StatusCode > 499 {
return nil, fmt.Errorf("bad response status code from http loader url: %v", resp.StatusCode)
}
return resp, nil
}
func doHttpCallWithRetries(ctx caddy.Context, client *http.Client, request *http.Request) (*http.Response, error) {
var resp *http.Response
var err error
const maxAttempts = 10
// attempt up to 10 times
for i := 0; i < maxAttempts; i++ {
resp, err = attemptHttpCall(client, request)
if err != nil && i < maxAttempts-1 {
// wait 500ms before reattempting, or until context is done
select {
case <-time.After(time.Millisecond * 500):
case <-ctx.Done():
return resp, ctx.Err()
}
}
}
return resp, err
}
func (hl HTTPLoader) makeClient(ctx caddy.Context) (*http.Client, error) {
client := &http.Client{
Timeout: time.Duration(hl.Timeout),
}
if hl.TLS != nil {
var tlsConfig *tls.Config
// client authentication
if hl.TLS.UseServerIdentity {
certs, err := ctx.IdentityCredentials(ctx.Logger())
if err != nil {
return nil, fmt.Errorf("getting server identity credentials: %v", err)
}
if tlsConfig == nil {
tlsConfig = new(tls.Config)
}
tlsConfig.Certificates = certs
} else if hl.TLS.ClientCertificateFile != "" && hl.TLS.ClientCertificateKeyFile != "" {
cert, err := tls.LoadX509KeyPair(hl.TLS.ClientCertificateFile, hl.TLS.ClientCertificateKeyFile)
if err != nil {
return nil, err
}
if tlsConfig == nil {
tlsConfig = new(tls.Config)
}
tlsConfig.Certificates = []tls.Certificate{cert}
}
// trusted server certs
if len(hl.TLS.RootCAPEMFiles) > 0 {
rootPool := x509.NewCertPool()
for _, pemFile := range hl.TLS.RootCAPEMFiles {
pemData, err := os.ReadFile(pemFile)
if err != nil {
return nil, fmt.Errorf("failed reading ca cert: %v", err)
}
rootPool.AppendCertsFromPEM(pemData)
}
if tlsConfig == nil {
tlsConfig = new(tls.Config)
}
tlsConfig.RootCAs = rootPool
}
client.Transport = &http.Transport{TLSClientConfig: tlsConfig}
}
return client, nil
}
var _ caddy.ConfigLoader = (*HTTPLoader)(nil)
+100 -39
View File
@@ -58,6 +58,10 @@ func (al adminLoad) Routes() []caddy.AdminRoute {
Pattern: "/load",
Handler: caddy.AdminHandlerFunc(al.handleLoad),
},
{
Pattern: "/adapt",
Handler: caddy.AdminHandlerFunc(al.handleAdapt),
},
}
}
@@ -69,8 +73,8 @@ func (al adminLoad) Routes() []caddy.AdminRoute {
func (adminLoad) handleLoad(w http.ResponseWriter, r *http.Request) error {
if r.Method != http.MethodPost {
return caddy.APIError{
Code: http.StatusMethodNotAllowed,
Err: fmt.Errorf("method not allowed"),
HTTPStatus: http.StatusMethodNotAllowed,
Err: fmt.Errorf("method not allowed"),
}
}
@@ -81,8 +85,8 @@ func (adminLoad) handleLoad(w http.ResponseWriter, r *http.Request) error {
_, err := io.Copy(buf, r.Body)
if err != nil {
return caddy.APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("reading request body: %v", err),
HTTPStatus: http.StatusBadRequest,
Err: fmt.Errorf("reading request body: %v", err),
}
}
body := buf.Bytes()
@@ -90,45 +94,21 @@ func (adminLoad) handleLoad(w http.ResponseWriter, r *http.Request) error {
// if the config is formatted other than Caddy's native
// JSON, we need to adapt it before loading it
if ctHeader := r.Header.Get("Content-Type"); ctHeader != "" {
ct, _, err := mime.ParseMediaType(ctHeader)
result, warnings, err := adaptByContentType(ctHeader, body)
if err != nil {
return caddy.APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("invalid Content-Type: %v", err),
HTTPStatus: http.StatusBadRequest,
Err: err,
}
}
if !strings.HasSuffix(ct, "/json") {
slashIdx := strings.Index(ct, "/")
if slashIdx < 0 {
return caddy.APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("malformed Content-Type"),
}
}
adapterName := ct[slashIdx+1:]
cfgAdapter := GetAdapter(adapterName)
if cfgAdapter == nil {
return caddy.APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("unrecognized config adapter '%s'", adapterName),
}
}
result, warnings, err := cfgAdapter.Adapt(body, nil)
if len(warnings) > 0 {
respBody, err := json.Marshal(warnings)
if err != nil {
return caddy.APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("adapting config using %s adapter: %v", adapterName, err),
}
caddy.Log().Named("admin.api.load").Error(err.Error())
}
if len(warnings) > 0 {
respBody, err := json.Marshal(warnings)
if err != nil {
caddy.Log().Named("admin.api.load").Error(err.Error())
}
_, _ = w.Write(respBody)
}
body = result
_, _ = w.Write(respBody)
}
body = result
}
forceReload := r.Header.Get("Cache-Control") == "must-revalidate"
@@ -136,8 +116,8 @@ func (adminLoad) handleLoad(w http.ResponseWriter, r *http.Request) error {
err = caddy.Load(body, forceReload)
if err != nil {
return caddy.APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("loading config: %v", err),
HTTPStatus: http.StatusBadRequest,
Err: fmt.Errorf("loading config: %v", err),
}
}
@@ -146,8 +126,89 @@ func (adminLoad) handleLoad(w http.ResponseWriter, r *http.Request) error {
return nil
}
// handleAdapt adapts the given Caddy config to JSON and responds with the result.
func (adminLoad) handleAdapt(w http.ResponseWriter, r *http.Request) error {
if r.Method != http.MethodPost {
return caddy.APIError{
HTTPStatus: http.StatusMethodNotAllowed,
Err: fmt.Errorf("method not allowed"),
}
}
buf := bufPool.Get().(*bytes.Buffer)
buf.Reset()
defer bufPool.Put(buf)
_, err := io.Copy(buf, r.Body)
if err != nil {
return caddy.APIError{
HTTPStatus: http.StatusBadRequest,
Err: fmt.Errorf("reading request body: %v", err),
}
}
result, warnings, err := adaptByContentType(r.Header.Get("Content-Type"), buf.Bytes())
if err != nil {
return caddy.APIError{
HTTPStatus: http.StatusBadRequest,
Err: err,
}
}
out := struct {
Warnings []Warning `json:"warnings,omitempty"`
Result json.RawMessage `json:"result"`
}{
Warnings: warnings,
Result: result,
}
w.Header().Set("Content-Type", "application/json")
return json.NewEncoder(w).Encode(out)
}
// adaptByContentType adapts body to Caddy JSON using the adapter specified by contentType.
// If contentType is empty or ends with "/json", the input will be returned, as a no-op.
func adaptByContentType(contentType string, body []byte) ([]byte, []Warning, error) {
// assume JSON as the default
if contentType == "" {
return body, nil, nil
}
ct, _, err := mime.ParseMediaType(contentType)
if err != nil {
return nil, nil, caddy.APIError{
HTTPStatus: http.StatusBadRequest,
Err: fmt.Errorf("invalid Content-Type: %v", err),
}
}
// if already JSON, no need to adapt
if strings.HasSuffix(ct, "/json") {
return body, nil, nil
}
// adapter name should be suffix of MIME type
_, adapterName, slashFound := strings.Cut(ct, "/")
if !slashFound {
return nil, nil, fmt.Errorf("malformed Content-Type")
}
cfgAdapter := GetAdapter(adapterName)
if cfgAdapter == nil {
return nil, nil, fmt.Errorf("unrecognized config adapter '%s'", adapterName)
}
result, warnings, err := cfgAdapter.Adapt(body, nil)
if err != nil {
return nil, nil, fmt.Errorf("adapting config using %s adapter: %v", adapterName, err)
}
return result, warnings, nil
}
var bufPool = sync.Pool{
New: func() interface{} {
New: func() any {
return new(bytes.Buffer)
},
}
+20
View File
@@ -0,0 +1,20 @@
-----BEGIN CERTIFICATE-----
MIIDSzCCAjOgAwIBAgIUfIRObjWNUA4jxQ/0x8BOCvE2Vw4wDQYJKoZIhvcNAQEL
BQAwFjEUMBIGA1UEAwwLRWFzeS1SU0EgQ0EwHhcNMTkwODI4MTYyNTU5WhcNMjkw
ODI1MTYyNTU5WjAWMRQwEgYDVQQDDAtFYXN5LVJTQSBDQTCCASIwDQYJKoZIhvcN
AQEBBQADggEPADCCAQoCggEBAK5m5elxhQfMp/3aVJ4JnpN9PUSz6LlP6LePAPFU
7gqohVVFVtDkChJAG3FNkNQNlieVTja/bgH9IcC6oKbROwdY1h0MvNV8AHHigvl0
3WuJD8g2ReVFXXwsnrPmKXCFzQyMI6TYk3m2gYrXsZOU1GLnfMRC3KAMRgE2F45t
wOs9hqG169YJ6mM2eQjzjCHWI6S2/iUYvYxRkCOlYUbLsMD/AhgAf1plzg6LPqNx
tdlwxZnA0ytgkmhK67HtzJu0+ovUCsMv0RwcMhsEo9T8nyFAGt9XLZ63X5WpBCTU
ApaAUhnG0XnerjmUWb6eUWw4zev54sEfY5F3x002iQaW6cECAwEAAaOBkDCBjTAd
BgNVHQ4EFgQU4CBUbZsS2GaNIkGRz/cBsD5ivjswUQYDVR0jBEowSIAU4CBUbZsS
2GaNIkGRz/cBsD5ivjuhGqQYMBYxFDASBgNVBAMMC0Vhc3ktUlNBIENBghR8hE5u
NY1QDiPFD/THwE4K8TZXDjAMBgNVHRMEBTADAQH/MAsGA1UdDwQEAwIBBjANBgkq
hkiG9w0BAQsFAAOCAQEAKB3V4HIzoiO/Ch6WMj9bLJ2FGbpkMrcb/Eq01hT5zcfK
D66lVS1MlK+cRL446Z2b2KDP1oFyVs+qmrmtdwrWgD+nfe2sBmmIHo9m9KygMkEO
fG3MghGTEcS+0cTKEcoHYWYyOqQh6jnedXY8Cdm4GM1hAc9MiL3/sqV8YCVSLNnk
oNysmr06/rZ0MCUZPGUtRmfd0heWhrfzAKw2HLgX+RAmpOE2MZqWcjvqKGyaRiaZ
ks4nJkP6521aC2Lgp0HhCz1j8/uQ5ldoDszCnu/iro0NAsNtudTMD+YoLQxLqdle
Ih6CW+illc2VdXwj7mn6J04yns9jfE2jRjW/yTLFuQ==
-----END CERTIFICATE-----
+109 -48
View File
@@ -7,13 +7,14 @@ import (
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"io"
"log"
"net"
"net/http"
"net/http/cookiejar"
"os"
"path"
"reflect"
"regexp"
"runtime"
"strings"
@@ -42,7 +43,7 @@ type Defaults struct {
// Default testing values
var Default = Defaults{
AdminPort: 2019,
AdminPort: 2999, // different from what a real server also running on a developer's machine might be
Certifcates: []string{"/caddy.localhost.crt", "/caddy.localhost.key"},
TestRequestTimeout: 5 * time.Second,
LoadRequestTimeout: 5 * time.Second,
@@ -55,8 +56,9 @@ var (
// Tester represents an instance of a test client.
type Tester struct {
Client *http.Client
t *testing.T
Client *http.Client
configLoaded bool
t *testing.T
}
// NewTester will create a new testing client with an attached cookie jar
@@ -73,7 +75,8 @@ func NewTester(t *testing.T) *Tester {
Jar: jar,
Timeout: Default.TestRequestTimeout,
},
t: t,
configLoaded: false,
t: t,
}
}
@@ -96,6 +99,10 @@ func (tc *Tester) InitServer(rawConfig string, configType string) {
tc.t.Logf("failed to load config: %s", err)
tc.t.Fail()
}
if err := tc.ensureConfigRunning(rawConfig, configType); err != nil {
tc.t.Logf("failed ensuring config is running: %s", err)
tc.t.Fail()
}
}
// InitServer this will configure the server with a configurion of a specific
@@ -114,17 +121,18 @@ func (tc *Tester) initServer(rawConfig string, configType string) error {
}
tc.t.Cleanup(func() {
if tc.t.Failed() {
if tc.t.Failed() && tc.configLoaded {
res, err := http.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
if err != nil {
tc.t.Log("unable to read the current config")
return
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
body, _ := io.ReadAll(res.Body)
var out bytes.Buffer
json.Indent(&out, body, "", " ")
_ = json.Indent(&out, body, "", " ")
tc.t.Logf("----------- failed with config -----------\n%s", out.String())
}
})
@@ -154,7 +162,7 @@ func (tc *Tester) initServer(rawConfig string, configType string) error {
timeElapsed(start, "caddytest: config load time")
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
body, err := io.ReadAll(res.Body)
if err != nil {
tc.t.Errorf("unable to read response. %s", err)
return err
@@ -164,23 +172,61 @@ func (tc *Tester) initServer(rawConfig string, configType string) error {
return configLoadError{Response: string(body)}
}
tc.configLoaded = true
return nil
}
var hasValidated bool
var arePrerequisitesValid bool
func validateTestPrerequisites() error {
if hasValidated {
if !arePrerequisitesValid {
return errors.New("caddy integration prerequisites failed. see first error")
func (tc *Tester) ensureConfigRunning(rawConfig string, configType string) error {
expectedBytes := []byte(prependCaddyFilePath(rawConfig))
if configType != "json" {
adapter := caddyconfig.GetAdapter(configType)
if adapter == nil {
return fmt.Errorf("adapter of config type is missing: %s", configType)
}
return nil
expectedBytes, _, _ = adapter.Adapt([]byte(rawConfig), nil)
}
hasValidated = true
arePrerequisitesValid = false
var expected any
err := json.Unmarshal(expectedBytes, &expected)
if err != nil {
return err
}
client := &http.Client{
Timeout: Default.LoadRequestTimeout,
}
fetchConfig := func(client *http.Client) any {
resp, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
if err != nil {
return nil
}
defer resp.Body.Close()
actualBytes, err := io.ReadAll(resp.Body)
if err != nil {
return nil
}
var actual any
err = json.Unmarshal(actualBytes, &actual)
if err != nil {
return nil
}
return actual
}
for retries := 10; retries > 0; retries-- {
if reflect.DeepEqual(expected, fetchConfig(client)) {
return nil
}
time.Sleep(10 * time.Millisecond)
}
tc.t.Errorf("POSTed configuration isn't active")
return errors.New("EnsureConfigRunning: POSTed configuration isn't active")
}
// validateTestPrerequisites ensures the certificates are available in the
// designated path and Caddy sub-process is running.
func validateTestPrerequisites() error {
// check certificates are found
for _, certName := range Default.Certifcates {
@@ -191,25 +237,19 @@ func validateTestPrerequisites() error {
if isCaddyAdminRunning() != nil {
// start inprocess caddy server
os.Args = []string{"caddy", "run"}
os.Args = []string{"caddy", "run", "--config", "./test.init.config", "--adapter", "caddyfile"}
go func() {
caddycmd.Main()
}()
// wait for caddy to start
retries := 4
for ; retries > 0 && isCaddyAdminRunning() != nil; retries-- {
// wait for caddy to start serving the initial config
for retries := 10; retries > 0 && isCaddyAdminRunning() != nil; retries-- {
time.Sleep(10 * time.Millisecond)
}
}
// assert that caddy is running
if err := isCaddyAdminRunning(); err != nil {
return err
}
arePrerequisitesValid = true
return nil
// one more time to return the error
return isCaddyAdminRunning()
}
func isCaddyAdminRunning() error {
@@ -217,10 +257,11 @@ func isCaddyAdminRunning() error {
client := &http.Client{
Timeout: Default.LoadRequestTimeout,
}
_, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
resp, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
if err != nil {
return errors.New("caddy integration test caddy server not running. Expected to be listening on localhost:2019")
return fmt.Errorf("caddy integration test caddy server not running. Expected to be listening on localhost:%d", Default.AdminPort)
}
resp.Body.Close()
return nil
}
@@ -268,7 +309,7 @@ func CreateTestingTransport() *http.Transport {
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 5 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, //nolint:gosec
}
}
@@ -309,35 +350,46 @@ func (tc *Tester) AssertRedirect(requestURI string, expectedToLocation string, e
if err != nil {
tc.t.Errorf("requesting \"%s\" expected location: \"%s\" but got error: %s", requestURI, expectedToLocation, err)
}
if expectedToLocation != loc.String() {
tc.t.Errorf("requesting \"%s\" expected location: \"%s\" but got \"%s\"", requestURI, expectedToLocation, loc.String())
if loc == nil && expectedToLocation != "" {
tc.t.Errorf("requesting \"%s\" expected a Location header, but didn't get one", requestURI)
}
if loc != nil {
if expectedToLocation != loc.String() {
tc.t.Errorf("requesting \"%s\" expected location: \"%s\" but got \"%s\"", requestURI, expectedToLocation, loc.String())
}
}
return resp
}
// AssertAdapt adapts a config and then tests it against an expected result
func AssertAdapt(t *testing.T, rawConfig string, adapterName string, expectedResponse string) {
// CompareAdapt adapts a config and then compares it against an expected result
func CompareAdapt(t *testing.T, filename, rawConfig string, adapterName string, expectedResponse string) bool {
cfgAdapter := caddyconfig.GetAdapter(adapterName)
if cfgAdapter == nil {
t.Errorf("unrecognized config adapter '%s'", adapterName)
return
t.Logf("unrecognized config adapter '%s'", adapterName)
return false
}
options := make(map[string]interface{})
options["pretty"] = "true"
options := make(map[string]any)
result, warnings, err := cfgAdapter.Adapt([]byte(rawConfig), options)
if err != nil {
t.Errorf("adapting config using %s adapter: %v", adapterName, err)
return
t.Logf("adapting config using %s adapter: %v", adapterName, err)
return false
}
// prettify results to keep tests human-manageable
var prettyBuf bytes.Buffer
err = json.Indent(&prettyBuf, result, "", "\t")
if err != nil {
return false
}
result = prettyBuf.Bytes()
if len(warnings) > 0 {
for _, w := range warnings {
t.Logf("warning: directive: %s : %s", w.Directive, w.Message)
t.Logf("warning: %s:%d: %s: %s", filename, w.Line, w.Directive, w.Message)
}
}
@@ -365,6 +417,15 @@ func AssertAdapt(t *testing.T, rawConfig string, adapterName string, expectedRes
fmt.Printf(" + %s\n", d.Payload)
}
}
return false
}
return true
}
// AssertAdapt adapts a config and then tests it against an expected result
func AssertAdapt(t *testing.T, rawConfig string, adapterName string, expectedResponse string) {
ok := CompareAdapt(t, "Caddyfile", rawConfig, adapterName, expectedResponse)
if !ok {
t.Fail()
}
}
@@ -410,14 +471,14 @@ func (tc *Tester) AssertResponse(req *http.Request, expectedStatusCode int, expe
resp := tc.AssertResponseCode(req, expectedStatusCode)
defer resp.Body.Close()
bytes, err := ioutil.ReadAll(resp.Body)
bytes, err := io.ReadAll(resp.Body)
if err != nil {
tc.t.Fatalf("unable to read the response body %s", err)
}
body := string(bytes)
if !strings.Contains(body, expectedBody) {
if body != expectedBody {
tc.t.Errorf("requesting \"%s\" expected response body \"%s\" but got \"%s\"", req.RequestURI, expectedBody, body)
}
+145
View File
@@ -0,0 +1,145 @@
package integration
import (
"net/http"
"testing"
"github.com/caddyserver/caddy/v2/caddytest"
)
func TestAutoHTTPtoHTTPSRedirectsImplicitPort(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
admin localhost:2999
skip_install_trust
http_port 9080
https_port 9443
}
localhost
respond "Yahaha! You found me!"
`, "caddyfile")
tester.AssertRedirect("http://localhost:9080/", "https://localhost/", http.StatusPermanentRedirect)
}
func TestAutoHTTPtoHTTPSRedirectsExplicitPortSameAsHTTPSPort(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port 9080
https_port 9443
}
localhost:9443
respond "Yahaha! You found me!"
`, "caddyfile")
tester.AssertRedirect("http://localhost:9080/", "https://localhost/", http.StatusPermanentRedirect)
}
func TestAutoHTTPtoHTTPSRedirectsExplicitPortDifferentFromHTTPSPort(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port 9080
https_port 9443
}
localhost:1234
respond "Yahaha! You found me!"
`, "caddyfile")
tester.AssertRedirect("http://localhost:9080/", "https://localhost:1234/", http.StatusPermanentRedirect)
}
func TestAutoHTTPRedirectsWithHTTPListenerFirstInAddresses(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
"admin": {
"listen": "localhost:2999"
},
"apps": {
"http": {
"http_port": 9080,
"https_port": 9443,
"servers": {
"ingress_server": {
"listen": [
":9080",
":9443"
],
"routes": [
{
"match": [
{
"host": ["localhost"]
}
]
}
]
}
}
},
"pki": {
"certificate_authorities": {
"local": {
"install_trust": false
}
}
}
}
}
`, "json")
tester.AssertRedirect("http://localhost:9080/", "https://localhost/", http.StatusPermanentRedirect)
}
func TestAutoHTTPRedirectsInsertedBeforeUserDefinedCatchAll(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port 9080
https_port 9443
local_certs
}
http://:9080 {
respond "Foo"
}
http://baz.localhost:9080 {
respond "Baz"
}
bar.localhost {
respond "Bar"
}
`, "caddyfile")
tester.AssertRedirect("http://bar.localhost:9080/", "https://bar.localhost/", http.StatusPermanentRedirect)
tester.AssertGetResponse("http://foo.localhost:9080/", 200, "Foo")
tester.AssertGetResponse("http://baz.localhost:9080/", 200, "Baz")
}
func TestAutoHTTPRedirectsInsertedBeforeUserDefinedCatchAllWithNoExplicitHTTPSite(t *testing.T) {
tester := caddytest.NewTester(t)
tester.InitServer(`
{
skip_install_trust
admin localhost:2999
http_port 9080
https_port 9443
local_certs
}
http://:9080 {
respond "Foo"
}
bar.localhost {
respond "Bar"
}
`, "caddyfile")
tester.AssertRedirect("http://bar.localhost:9080/", "https://bar.localhost/", http.StatusPermanentRedirect)
tester.AssertGetResponse("http://foo.localhost:9080/", 200, "Foo")
tester.AssertGetResponse("http://baz.localhost:9080/", 200, "Foo")
}
@@ -0,0 +1,34 @@
{
auto_https disable_redirects
}
localhost
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"localhost"
]
}
],
"terminal": true
}
],
"automatic_https": {
"disable_redirects": true
}
}
}
}
}
}
@@ -0,0 +1,34 @@
{
auto_https ignore_loaded_certs
}
localhost
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"localhost"
]
}
],
"terminal": true
}
],
"automatic_https": {
"ignore_loaded_certificates": true
}
}
}
}
}
}
@@ -0,0 +1,37 @@
{
auto_https off
}
localhost
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"localhost"
]
}
],
"terminal": true
}
],
"tls_connection_policies": [
{}
],
"automatic_https": {
"disable": true
}
}
}
}
}
}
@@ -0,0 +1,29 @@
example.com {
bind tcp6/[::]
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
"tcp6/[::]:443"
],
"routes": [
{
"match": [
{
"host": [
"example.com"
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,85 @@
:80
# All the options
encode gzip zstd {
minimum_length 256
match {
status 2xx 4xx 500
header Content-Type text/*
header Content-Type application/json*
header Content-Type application/javascript*
header Content-Type application/xhtml+xml*
header Content-Type application/atom+xml*
header Content-Type application/rss+xml*
header Content-Type image/svg+xml*
}
}
# Long way with a block for each encoding
encode {
zstd
gzip 5
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"handle": [
{
"encodings": {
"gzip": {},
"zstd": {}
},
"handler": "encode",
"match": {
"headers": {
"Content-Type": [
"text/*",
"application/json*",
"application/javascript*",
"application/xhtml+xml*",
"application/atom+xml*",
"application/rss+xml*",
"image/svg+xml*"
]
},
"status_code": [
2,
4,
500
]
},
"minimum_length": 256,
"prefer": [
"gzip",
"zstd"
]
},
{
"encodings": {
"gzip": {
"level": 5
},
"zstd": {}
},
"handler": "encode",
"prefer": [
"zstd",
"gzip"
]
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,138 @@
example.com {
root * /srv
# Trigger errors for certain paths
error /private* "Unauthorized" 403
error /hidden* "Not found" 404
# Handle the error by serving an HTML page
handle_errors {
rewrite * /{http.error.status_code}.html
file_server
}
file_server
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"example.com"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "vars",
"root": "/srv"
}
]
},
{
"handle": [
{
"error": "Unauthorized",
"handler": "error",
"status_code": 403
}
],
"match": [
{
"path": [
"/private*"
]
}
]
},
{
"handle": [
{
"error": "Not found",
"handler": "error",
"status_code": 404
}
],
"match": [
{
"path": [
"/hidden*"
]
}
]
},
{
"handle": [
{
"handler": "file_server",
"hide": [
"./Caddyfile"
]
}
]
}
]
}
],
"terminal": true
}
],
"errors": {
"routes": [
{
"match": [
{
"host": [
"example.com"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"group": "group0",
"handle": [
{
"handler": "rewrite",
"uri": "/{http.error.status_code}.html"
}
]
},
{
"handle": [
{
"handler": "file_server",
"hide": [
"./Caddyfile"
]
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
}
@@ -0,0 +1,114 @@
example.com
@a expression {http.error.status_code} == 400
abort @a
@b expression {http.error.status_code} == "401"
abort @b
@c expression {http.error.status_code} == `402`
abort @c
@d expression "{http.error.status_code} == 403"
abort @d
@e expression `{http.error.status_code} == 404`
abort @e
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"example.com"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"abort": true,
"handler": "static_response"
}
],
"match": [
{
"expression": "{http.error.status_code} == 400"
}
]
},
{
"handle": [
{
"abort": true,
"handler": "static_response"
}
],
"match": [
{
"expression": "{http.error.status_code} == \"401\""
}
]
},
{
"handle": [
{
"abort": true,
"handler": "static_response"
}
],
"match": [
{
"expression": "{http.error.status_code} == `402`"
}
]
},
{
"handle": [
{
"abort": true,
"handler": "static_response"
}
],
"match": [
{
"expression": "{http.error.status_code} == 403"
}
]
},
{
"handle": [
{
"abort": true,
"handler": "static_response"
}
],
"match": [
{
"expression": "{http.error.status_code} == 404"
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,32 @@
:80
file_server {
disable_canonical_uris
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"handle": [
{
"canonical_uris": false,
"handler": "file_server",
"hide": [
"./Caddyfile"
]
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,32 @@
:80
file_server {
pass_thru
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"handle": [
{
"handler": "file_server",
"hide": [
"./Caddyfile"
],
"pass_thru": true
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,41 @@
:80
file_server {
precompressed zstd br gzip
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"handle": [
{
"handler": "file_server",
"hide": [
"./Caddyfile"
],
"precompressed": {
"br": {},
"gzip": {},
"zstd": {}
},
"precompressed_order": [
"zstd",
"br",
"gzip"
]
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,112 @@
localhost
root * /srv
handle /nope* {
file_server {
status 403
}
}
handle /custom-status* {
file_server {
status {env.CUSTOM_STATUS}
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"localhost"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "vars",
"root": "/srv"
}
]
},
{
"group": "group2",
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "file_server",
"hide": [
"./Caddyfile"
],
"status_code": "{env.CUSTOM_STATUS}"
}
]
}
]
}
],
"match": [
{
"path": [
"/custom-status*"
]
}
]
},
{
"group": "group2",
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "file_server",
"hide": [
"./Caddyfile"
],
"status_code": 403
}
]
}
]
}
],
"match": [
{
"path": [
"/nope*"
]
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,111 @@
app.example.com {
forward_auth authelia:9091 {
uri /api/verify?rd=https://authelia.example.com
copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
}
reverse_proxy backend:8080
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"app.example.com"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handle_response": [
{
"match": {
"status_code": [
2
]
},
"routes": [
{
"handle": [
{
"handler": "headers",
"request": {
"set": {
"Remote-Email": [
"{http.reverse_proxy.header.Remote-Email}"
],
"Remote-Groups": [
"{http.reverse_proxy.header.Remote-Groups}"
],
"Remote-Name": [
"{http.reverse_proxy.header.Remote-Name}"
],
"Remote-User": [
"{http.reverse_proxy.header.Remote-User}"
]
}
}
}
]
}
]
}
],
"handler": "reverse_proxy",
"headers": {
"request": {
"set": {
"X-Forwarded-Method": [
"{http.request.method}"
],
"X-Forwarded-Uri": [
"{http.request.uri}"
]
}
}
},
"rewrite": {
"method": "GET",
"uri": "/api/verify?rd=https://authelia.example.com"
},
"upstreams": [
{
"dial": "authelia:9091"
}
]
},
{
"handler": "reverse_proxy",
"upstreams": [
{
"dial": "backend:8080"
}
]
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,90 @@
:8881
forward_auth localhost:9000 {
uri /auth
copy_headers A>1 B C>3 {
D
E>5
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":8881"
],
"routes": [
{
"handle": [
{
"handle_response": [
{
"match": {
"status_code": [
2
]
},
"routes": [
{
"handle": [
{
"handler": "headers",
"request": {
"set": {
"1": [
"{http.reverse_proxy.header.A}"
],
"3": [
"{http.reverse_proxy.header.C}"
],
"5": [
"{http.reverse_proxy.header.E}"
],
"B": [
"{http.reverse_proxy.header.B}"
],
"D": [
"{http.reverse_proxy.header.D}"
]
}
}
}
]
}
]
}
],
"handler": "reverse_proxy",
"headers": {
"request": {
"set": {
"X-Forwarded-Method": [
"{http.request.method}"
],
"X-Forwarded-Uri": [
"{http.request.uri}"
]
}
}
},
"rewrite": {
"method": "GET",
"uri": "/auth"
},
"upstreams": [
{
"dial": "localhost:9000"
}
]
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,82 @@
{
debug
http_port 8080
https_port 8443
grace_period 5s
shutdown_delay 10s
default_sni localhost
order root first
storage file_system {
root /data
}
acme_ca https://example.com
acme_ca_root /path/to/ca.crt
ocsp_stapling off
email test@example.com
admin off
on_demand_tls {
ask https://example.com
interval 30s
burst 20
}
local_certs
key_type ed25519
}
:80
----------
{
"admin": {
"disabled": true
},
"logging": {
"logs": {
"default": {
"level": "DEBUG"
}
}
},
"storage": {
"module": "file_system",
"root": "/data"
},
"apps": {
"http": {
"http_port": 8080,
"https_port": 8443,
"grace_period": 5000000000,
"shutdown_delay": 10000000000,
"servers": {
"srv0": {
"listen": [
":80"
]
}
}
},
"tls": {
"automation": {
"policies": [
{
"issuers": [
{
"module": "internal"
}
],
"key_type": "ed25519",
"disable_ocsp_stapling": true
}
],
"on_demand": {
"rate_limit": {
"interval": 30000000000,
"burst": 20
},
"ask": "https://example.com"
}
},
"disable_ocsp_stapling": true
}
}
}
@@ -0,0 +1,93 @@
{
debug
http_port 8080
https_port 8443
default_sni localhost
order root first
storage file_system {
root /data
}
acme_ca https://example.com
acme_eab {
key_id 4K2scIVbBpNd-78scadB2g
mac_key abcdefghijklmnopqrstuvwx-abcdefghijklnopqrstuvwxyz12ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefgh
}
acme_ca_root /path/to/ca.crt
email test@example.com
admin off
on_demand_tls {
ask https://example.com
interval 30s
burst 20
}
storage_clean_interval 7d
renew_interval 1d
ocsp_interval 2d
key_type ed25519
}
:80
----------
{
"admin": {
"disabled": true
},
"logging": {
"logs": {
"default": {
"level": "DEBUG"
}
}
},
"storage": {
"module": "file_system",
"root": "/data"
},
"apps": {
"http": {
"http_port": 8080,
"https_port": 8443,
"servers": {
"srv0": {
"listen": [
":80"
]
}
}
},
"tls": {
"automation": {
"policies": [
{
"issuers": [
{
"ca": "https://example.com",
"email": "test@example.com",
"external_account": {
"key_id": "4K2scIVbBpNd-78scadB2g",
"mac_key": "abcdefghijklmnopqrstuvwx-abcdefghijklnopqrstuvwxyz12ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefgh"
},
"module": "acme",
"trusted_roots_pem_files": [
"/path/to/ca.crt"
]
}
],
"key_type": "ed25519"
}
],
"on_demand": {
"rate_limit": {
"interval": 30000000000,
"burst": 20
},
"ask": "https://example.com"
},
"ocsp_interval": 172800000000000,
"renew_interval": 86400000000000,
"storage_clean_interval": 604800000000000
}
}
}
}
@@ -0,0 +1,83 @@
{
debug
http_port 8080
https_port 8443
default_sni localhost
order root first
storage file_system {
root /data
}
acme_ca https://example.com
acme_ca_root /path/to/ca.crt
email test@example.com
admin {
origins localhost:2019 [::1]:2019 127.0.0.1:2019 192.168.10.128
}
on_demand_tls {
ask https://example.com
interval 30s
burst 20
}
local_certs
key_type ed25519
}
:80
----------
{
"admin": {
"listen": "localhost:2019",
"origins": [
"localhost:2019",
"[::1]:2019",
"127.0.0.1:2019",
"192.168.10.128"
]
},
"logging": {
"logs": {
"default": {
"level": "DEBUG"
}
}
},
"storage": {
"module": "file_system",
"root": "/data"
},
"apps": {
"http": {
"http_port": 8080,
"https_port": 8443,
"servers": {
"srv0": {
"listen": [
":80"
]
}
}
},
"tls": {
"automation": {
"policies": [
{
"issuers": [
{
"module": "internal"
}
],
"key_type": "ed25519"
}
],
"on_demand": {
"rate_limit": {
"interval": 30000000000,
"burst": 20
},
"ask": "https://example.com"
}
}
}
}
}
@@ -0,0 +1,45 @@
{
debug
}
:8881 {
log {
format console
}
}
----------
{
"logging": {
"logs": {
"default": {
"level": "DEBUG",
"exclude": [
"http.log.access.log0"
]
},
"log0": {
"encoder": {
"format": "console"
},
"level": "DEBUG",
"include": [
"http.log.access.log0"
]
}
}
},
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":8881"
],
"logs": {
"default_logger_name": "log0"
}
}
}
}
}
}
@@ -0,0 +1,54 @@
{
default_bind tcp4/0.0.0.0 tcp6/[::]
}
example.com {
}
example.org:12345 {
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
"tcp4/0.0.0.0:12345",
"tcp6/[::]:12345"
],
"routes": [
{
"match": [
{
"host": [
"example.org"
]
}
],
"terminal": true
}
]
},
"srv1": {
"listen": [
"tcp4/0.0.0.0:443",
"tcp6/[::]:443"
],
"routes": [
{
"match": [
{
"host": [
"example.com"
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,77 @@
{
log {
output file caddy.log
include some-log-source
exclude admin.api admin2.api
}
log custom-logger {
output file caddy.log
level WARN
include custom-log-source
}
}
:8884 {
log {
format json
output file access.log
}
}
----------
{
"logging": {
"logs": {
"custom-logger": {
"writer": {
"filename": "caddy.log",
"output": "file"
},
"level": "WARN",
"include": [
"custom-log-source"
]
},
"default": {
"writer": {
"filename": "caddy.log",
"output": "file"
},
"include": [
"some-log-source"
],
"exclude": [
"admin.api",
"admin2.api",
"custom-log-source",
"http.log.access.log0"
]
},
"log0": {
"writer": {
"filename": "access.log",
"output": "file"
},
"encoder": {
"format": "json"
},
"include": [
"http.log.access.log0"
]
}
}
},
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":8884"
],
"logs": {
"default_logger_name": "log0"
}
}
}
}
}
}
@@ -0,0 +1,18 @@
{
log {
output file foo.log
}
}
----------
{
"logging": {
"logs": {
"default": {
"writer": {
"filename": "foo.log",
"output": "file"
}
}
}
}
}
@@ -0,0 +1,35 @@
{
log custom-logger {
format filter {
wrap console
fields {
request>remote_ip ip_mask {
ipv4 24
ipv6 32
}
}
}
}
}
----------
{
"logging": {
"logs": {
"custom-logger": {
"encoder": {
"fields": {
"request\u003eremote_ip": {
"filter": "ip_mask",
"ipv4_cidr": 24,
"ipv6_cidr": 32
}
},
"format": "filter",
"wrap": {
"format": "console"
}
}
}
}
}
}
@@ -0,0 +1,26 @@
{
log first {
output file foo.log
}
log second {
format json
}
}
----------
{
"logging": {
"logs": {
"first": {
"writer": {
"filename": "foo.log",
"output": "file"
}
},
"second": {
"encoder": {
"format": "json"
}
}
}
}
}
@@ -0,0 +1,56 @@
{
preferred_chains smallest
}
example.com
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"example.com"
]
}
],
"terminal": true
}
]
}
}
},
"tls": {
"automation": {
"policies": [
{
"subjects": [
"example.com"
],
"issuers": [
{
"module": "acme",
"preferred_chains": {
"smallest": true
}
},
{
"module": "zerossl",
"preferred_chains": {
"smallest": true
}
}
]
}
]
}
}
}
}
@@ -0,0 +1,168 @@
{
skip_install_trust
pki {
ca {
name "Local"
root_cn "Custom Local Root Name"
intermediate_cn "Custom Local Intermediate Name"
root {
cert /path/to/cert.pem
key /path/to/key.pem
format pem_file
}
intermediate {
cert /path/to/cert.pem
key /path/to/key.pem
format pem_file
}
}
ca foo {
name "Foo"
root_cn "Custom Foo Root Name"
intermediate_cn "Custom Foo Intermediate Name"
}
}
}
a.example.com {
tls internal
}
acme.example.com {
acme_server {
ca foo
}
}
acme-bar.example.com {
acme_server {
ca bar
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"acme-bar.example.com"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"ca": "bar",
"handler": "acme_server"
}
]
}
]
}
],
"terminal": true
},
{
"match": [
{
"host": [
"acme.example.com"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"ca": "foo",
"handler": "acme_server"
}
]
}
]
}
],
"terminal": true
},
{
"match": [
{
"host": [
"a.example.com"
]
}
],
"terminal": true
}
]
}
}
},
"pki": {
"certificate_authorities": {
"bar": {
"install_trust": false
},
"foo": {
"name": "Foo",
"root_common_name": "Custom Foo Root Name",
"intermediate_common_name": "Custom Foo Intermediate Name",
"install_trust": false
},
"local": {
"name": "Local",
"root_common_name": "Custom Local Root Name",
"intermediate_common_name": "Custom Local Intermediate Name",
"install_trust": false,
"root": {
"certificate": "/path/to/cert.pem",
"private_key": "/path/to/key.pem",
"format": "pem_file"
},
"intermediate": {
"certificate": "/path/to/cert.pem",
"private_key": "/path/to/key.pem",
"format": "pem_file"
}
}
}
},
"tls": {
"automation": {
"policies": [
{
"subjects": [
"acme-bar.example.com",
"acme.example.com"
]
},
{
"subjects": [
"a.example.com"
],
"issuers": [
{
"module": "internal"
}
]
}
]
}
}
}
}
@@ -0,0 +1,82 @@
{
servers {
timeouts {
idle 90s
}
strict_sni_host insecure_off
}
servers :80 {
timeouts {
idle 60s
}
}
servers :443 {
timeouts {
idle 30s
}
strict_sni_host
}
}
foo.com {
}
http://bar.com {
}
:8080 {
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"idle_timeout": 30000000000,
"routes": [
{
"match": [
{
"host": [
"foo.com"
]
}
],
"terminal": true
}
],
"strict_sni_host": true
},
"srv1": {
"listen": [
":80"
],
"idle_timeout": 60000000000,
"routes": [
{
"match": [
{
"host": [
"bar.com"
]
}
],
"terminal": true
}
]
},
"srv2": {
"listen": [
":8080"
],
"idle_timeout": 90000000000,
"strict_sni_host": false
}
}
}
}
}
@@ -0,0 +1,71 @@
{
servers {
listener_wrappers {
http_redirect
tls
}
timeouts {
read_body 30s
read_header 30s
write 30s
idle 30s
}
max_header_size 100MB
log_credentials
protocols h1 h2 h2c h3
strict_sni_host
}
}
foo.com {
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"listener_wrappers": [
{
"wrapper": "http_redirect"
},
{
"wrapper": "tls"
}
],
"read_timeout": 30000000000,
"read_header_timeout": 30000000000,
"write_timeout": 30000000000,
"idle_timeout": 30000000000,
"max_header_bytes": 100000000,
"routes": [
{
"match": [
{
"host": [
"foo.com"
]
}
],
"terminal": true
}
],
"strict_sni_host": true,
"logs": {
"should_log_credentials": true
},
"protocols": [
"h1",
"h2",
"h2c",
"h3"
]
}
}
}
}
}
@@ -0,0 +1,52 @@
:80
handle_path /api/v1/* {
respond "API v1"
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"match": [
{
"path": [
"/api/v1/*"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "rewrite",
"strip_path_prefix": "/api/v1"
}
]
},
{
"handle": [
{
"body": "API v1",
"handler": "static_response"
}
]
}
]
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,105 @@
:80 {
handle /api/* {
respond "api"
}
handle_path /static/* {
respond "static"
}
handle {
respond "handle"
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"group": "group3",
"match": [
{
"path": [
"/static/*"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"handler": "rewrite",
"strip_path_prefix": "/static"
}
]
},
{
"handle": [
{
"body": "static",
"handler": "static_response"
}
]
}
]
}
]
},
{
"group": "group3",
"match": [
{
"path": [
"/api/*"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "api",
"handler": "static_response"
}
]
}
]
}
]
},
{
"group": "group3",
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "handle",
"handler": "static_response"
}
]
}
]
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,147 @@
:80 {
header Denis "Ritchie"
header +Edsger "Dijkstra"
header ?John "von Neumann"
header -Wolfram
header {
Grace: "Hopper" # some users habitually suffix field names with a colon
+Ray "Solomonoff"
?Tim "Berners-Lee"
defer
}
@images path /images/*
header @images {
Cache-Control "public, max-age=3600, stale-while-revalidate=86400"
}
header {
+Link "Foo"
+Link "Bar"
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"match": [
{
"path": [
"/images/*"
]
}
],
"handle": [
{
"handler": "headers",
"response": {
"set": {
"Cache-Control": [
"public, max-age=3600, stale-while-revalidate=86400"
]
}
}
}
]
},
{
"handle": [
{
"handler": "headers",
"response": {
"set": {
"Denis": [
"Ritchie"
]
}
}
},
{
"handler": "headers",
"response": {
"add": {
"Edsger": [
"Dijkstra"
]
}
}
},
{
"handler": "headers",
"response": {
"require": {
"headers": {
"John": null
}
},
"set": {
"John": [
"von Neumann"
]
}
}
},
{
"handler": "headers",
"response": {
"deferred": true,
"delete": [
"Wolfram"
]
}
},
{
"handler": "headers",
"response": {
"add": {
"Ray": [
"Solomonoff"
]
},
"deferred": true,
"set": {
"Grace": [
"Hopper"
]
}
}
},
{
"handler": "headers",
"response": {
"require": {
"headers": {
"Tim": null
}
},
"set": {
"Tim": [
"Berners-Lee"
]
}
}
},
{
"handler": "headers",
"response": {
"add": {
"Link": [
"Foo",
"Bar"
]
}
}
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,45 @@
# https://github.com/caddyserver/caddy/issues/3977
http://* {
respond "Hello, world!"
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"match": [
{
"host": [
"*"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "Hello, world!",
"handler": "static_response"
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,37 @@
:80 {
respond /version 200 {
body "hello from localhost"
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"match": [
{
"path": [
"/version"
]
}
],
"handle": [
{
"body": "hello from localhost",
"handler": "static_response",
"status_code": 200
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,54 @@
http://a.caddy.localhost {
respond /version 200 {
body "hello from localhost"
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"match": [
{
"host": [
"a.caddy.localhost"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "hello from localhost",
"handler": "static_response",
"status_code": 200
}
],
"match": [
{
"path": [
"/version"
]
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,28 @@
# Issue #4113
:80, http://example.com {
respond "foo"
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"handle": [
{
"body": "foo",
"handler": "static_response"
}
]
}
]
}
}
}
}
}
@@ -0,0 +1,54 @@
localhost:80 {
respond /version 200 {
body "hello from localhost"
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":80"
],
"routes": [
{
"match": [
{
"host": [
"localhost"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "hello from localhost",
"handler": "static_response",
"status_code": 200
}
],
"match": [
{
"path": [
"/version"
]
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,59 @@
http://a.caddy.localhost:81 {
respond /version 200 {
body "hello from localhost"
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":81"
],
"routes": [
{
"match": [
{
"host": [
"a.caddy.localhost"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "hello from localhost",
"handler": "static_response",
"status_code": 200
}
],
"match": [
{
"path": [
"/version"
]
}
]
}
]
}
],
"terminal": true
}
],
"automatic_https": {
"skip": [
"a.caddy.localhost"
]
}
}
}
}
}
}
@@ -0,0 +1,54 @@
a.caddy.localhost {
respond /version 200 {
body "hello from localhost"
}
}
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"a.caddy.localhost"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "hello from localhost",
"handler": "static_response",
"status_code": 200
}
],
"match": [
{
"path": [
"/version"
]
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,49 @@
example.com
import testdata/import_respond.txt Groot Rocket
import testdata/import_respond.txt you "the confused man"
----------
{
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"example.com"
]
}
],
"handle": [
{
"handler": "subroute",
"routes": [
{
"handle": [
{
"body": "'I am Groot', hears Rocket",
"handler": "static_response"
},
{
"body": "'I am you', hears the confused man",
"handler": "static_response"
}
]
}
]
}
],
"terminal": true
}
]
}
}
}
}
}
@@ -0,0 +1,83 @@
(logging) {
log {
output file /var/log/caddy/{args.0}.access.log
}
}
a.example.com {
import logging a.example.com
}
b.example.com {
import logging b.example.com
}
----------
{
"logging": {
"logs": {
"default": {
"exclude": [
"http.log.access.log0",
"http.log.access.log1"
]
},
"log0": {
"writer": {
"filename": "/var/log/caddy/a.example.com.access.log",
"output": "file"
},
"include": [
"http.log.access.log0"
]
},
"log1": {
"writer": {
"filename": "/var/log/caddy/b.example.com.access.log",
"output": "file"
},
"include": [
"http.log.access.log1"
]
}
}
},
"apps": {
"http": {
"servers": {
"srv0": {
"listen": [
":443"
],
"routes": [
{
"match": [
{
"host": [
"a.example.com"
]
}
],
"terminal": true
},
{
"match": [
{
"host": [
"b.example.com"
]
}
],
"terminal": true
}
],
"logs": {
"logger_names": {
"a.example.com": "log0",
"b.example.com": "log1"
}
}
}
}
}
}
}

Some files were not shown because too many files have changed in this diff Show More