Compare commits

..

489 Commits

Author SHA1 Message Date
Matthew Holt 7a4548c582 Some hotfixes for beta 16 2020-03-13 19:14:49 -06:00
Matthew Holt 6cbd93736f Minor tweaks 2020-03-13 13:04:10 -06:00
Mark Sargent c447236357 caddyhttp: Fix default SNI for default conn policy (#3141)
* add integration tests

* removed SNI test

* remove integration test condition

* minor edit

* fix sni when using static certificates

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-03-13 11:32:53 -06:00
Matt Holt 5a19db5dc2 v2: Implement 'pki' app powered by Smallstep for localhost certificates (#3125)
* pki: Initial commit of PKI app (WIP) (see #2502 and #3021)

* pki: Ability to use root/intermediates, and sign with root

* pki: Fix benign misnamings left over from copy+paste

* pki: Only install root if not already trusted

* Make HTTPS port the default; all names use auto-HTTPS; bug fixes

* Fix build - what happened to our CI tests??

* Fix go.mod
2020-03-13 11:06:08 -06:00
Bill Glover cfe85a9fe6 Fix #3130: Crash at fuzzing target replacer (#3133)
* Fix #3130: Crash at fuzzing target replacer

* Add additional test case based on fuzzer feedback
2020-03-11 16:12:00 -06:00
Francis Lavoie 90f1f7bce7 httpcaddyfile: error for wrong arg count of admin opt (#3126) (#3131) 2020-03-10 08:25:26 -06:00
Matt Holt 2762f8f058 caddyhttp: New algorithm for auto HTTP->HTTPS redirects (fix #3127) (#3128)
It's still not perfect but I think it should be more correct for
slightly more complex configs. Might still fall apart for complex
configs that use on-demand TLS or at a large scale (workarounds are
to just implement your own redirects, very easy to do anyway).
2020-03-09 15:18:19 -06:00
Matthew Holt 99d34f1c1d cmd: Use loadConfig() for validate as run, start, and reload do 2020-03-09 00:09:15 -06:00
Bill Glover 36a6c7daf0 Rework Replacer loop to handle escaped braces (#3121)
Fixes #3116

* Rework Replacer loop to ignore escaped braces

* Add benchmark tests for replacer

* Optimise handling of escaped braces

* Handle escaped closing braces

* Remove additional check for closing brace

This commit removes the additional check for input in which the closing
brace appears before the opening brace. This check has been removed for
performance reasons as it is deemed an unlikely edge case.

* Check for escaped closing braces in placeholder name
2020-03-08 15:36:59 -06:00
evtr ca6e54bbb8 caddytls: customizable client auth modes (#2913)
* ability to specify that client cert must be present in SSL

* changed the clientauthtype to string and make room for the values supported by go as in caddy1

* renamed the config parameter according to review comments and added documentation on allowed values

* missed a reference

* Minor cleanup; docs enhancements

Co-authored-by: Matthew Holt <mholt@users.noreply.github.com>
2020-03-08 09:48:25 -06:00
Mohammed Al Sahaf fb5168d3b4 http_ntlm: fix panic due to unintialized embedded field (#3120) 2020-03-07 17:58:44 -07:00
Matthew Holt 217419f6d9 tls: Couple of quick fixes for 4d18587192 2020-03-07 11:47:55 -07:00
Matthew Holt 4d18587192 tls: Auto-migrate cert assets to new path (details in #3124) 2020-03-07 10:42:50 -07:00
Matthew Holt b216d285df Merge branch 'certmagic-refactor' into v2 2020-03-06 23:26:13 -07:00
Matthew Holt b8cba62643 Refactor for CertMagic v0.10; prepare for PKI app
This is a breaking change primarily in two areas:
 - Storage paths for certificates have changed
 - Slight changes to JSON config parameters

Huge improvements in this commit, to be detailed more in
the release notes.

The upcoming PKI app will be powered by Smallstep libraries.
2020-03-06 23:15:25 -07:00
Matt Holt 3f5d27cd5d ci: Optimize published artifacts (#3118)
Build the published executables with CGO disabled, stripped, and with `-trimpath` for more reproducible build
2020-03-04 13:19:25 -07:00
Mark Sargent 26fb8b3efd httpcaddyfile: remove certificate tags from global state (#3111)
* remove the certificate tag tracking from global state

* refactored helper state, added log counter

* moved state initialisation close to where it is used.

* added helper state comment
2020-03-04 09:58:49 -07:00
Marten Seemann e6c6210772 update quic-go to v0.15.1 (#3109) 2020-03-02 07:13:49 -07:00
Marten Seemann 1324da2241 go.mod: update quic-go to v0.15.0 (supporting QUIC draft-27) (#3107) 2020-03-01 12:34:57 -07:00
Vaibhav 71e81d262b fmt: Add support for block nesting. (#3105)
Previously the formatter did not include support for
blocks inside other blocks. Hence the formatter could
not indent some files properly. This fixes it.

Fixes #3104

Signed-off-by: Vaibhav <vrongmeal@gmail.com>
2020-02-29 13:23:08 -07:00
Vaibhav 5fe69ac4ab cmd: Add caddy fmt command. (#3090)
This takes the config file as input and formats it.
Prints the result to stdout. Can write changes to
file if `--write` flag is passed.

Fixes #3020

Signed-off-by: Vaibhav <vrongmeal@gmail.com>
2020-02-29 10:12:16 -07:00
Mohammed Al Sahaf e717028f83 ci: publish build artifacts (#3103)
* ci: publish build artifacts (per-commit Caddy binaries)

* ci: include OS name in artifact name of *nix binaries so they don't overwrite each other
2020-02-29 20:09:50 +03:00
Matthew Holt a60da8e7ab Simplify the logic in the previous commit 2020-02-28 13:49:51 -07:00
Matthew Holt 00e99df209 httpcaddyfile: Treat no matchers as 0-len path matchers (fix #3100)
+ a couple other minor changes from linter
2020-02-28 13:38:12 -07:00
Matthew Holt c83d40ccd4 reverse_proxy, php_fastcgi: Fix upstream parsing regression (fix #3101) 2020-02-28 08:57:59 -07:00
Matthew Holt e4ec08e977 Couple of minor docs tweaks 2020-02-27 21:08:21 -07:00
Matthew Holt 03ab55b51a httpcaddyfile: Allow "admin off" option 2020-02-27 21:04:28 -07:00
Matthew Holt cef6e098bb Refactor ExtractMatcherSet() 2020-02-27 21:04:28 -07:00
Matthew Holt 260982b2df reverse_proxy: Allow use of URL to specify scheme
This makes it more convenient to configure quick proxies that use HTTPS
but also introduces a lot of logical complexity. We have to do a lot of
verification for consistency and errors.

Path and query string is not supported (i.e. no rewriting).

Scheme and port can be inferred from each other if HTTP(S)/80/443.
If omitted, defaults to HTTP.

Any explicit transport config must be consistent with the upstream
schemes, and the upstream schemes must all match too.

But, this change allows a config that used to require this:

    reverse_proxy example.com:443 {
        transport http {
            tls
        }
    }

to be reduced to this:

    reverse_proxy https://example.com

which is really nice syntactic sugar (and is reminiscent of Caddy 1).
2020-02-27 21:04:28 -07:00
Matthew Holt 0130b699df cmd/reverse_proxy: Add --change-host-header flag
"Transparent mode" is the default, just like the actual handler.
2020-02-27 21:04:28 -07:00
Success Go ca5c679880 Fix typos (#3087)
* Fix typo

* Fix typo, thanks for Spell Checker under VS Code
2020-02-27 19:30:48 -07:00
Matthew Holt e2d41ee761 Revert "reverse_proxy: Add 'transparent' Caddyfile subdirective (closes #2873)"
This reverts commit 86b785e51c.
2020-02-27 11:08:56 -07:00
Matthew Holt 86b785e51c reverse_proxy: Add 'transparent' Caddyfile subdirective (closes #2873) 2020-02-27 10:20:13 -07:00
Success Go f6ae092507 It might be HTTP->HTTPS in the comment (#3086) 2020-02-27 00:50:36 -05:00
Success Go a2a41a5bdf Fix spelling error (#3085) 2020-02-27 00:22:40 -05:00
Mohammed Al Sahaf 6fb98ba188 ci: improve CI flow (#3083)
* ci: update golangci-lint
* ci: build Caddy to catch build error
* ci: remove GO111MODULE env var
* ci: update MacOS image
2020-02-27 03:51:54 +03:00
Zaq? Wiedmann 063ed1e7f9 caddyfile: expand environment variables within caddy files (#3082)
Small expansion to the work done in https://github.com/caddyserver/caddy/pull/2963 which simply calls `os.ExpandEnv` so env vars like `{$URL}` where `$URL=$SCHEME://$HOST:$PORT` (contrived) get the expanded $SCHEME, $HOST, and $PORT variables included
2020-02-26 16:06:34 -07:00
Mark Sargent 2de0acc11f Initial implementation of global default SNI option (#3047)
* add global default sni

* fixed grammar

* httpcaddyfile: Reduce some duplicated code

* Um, re-commit already-committed commit, I guess? (sigh)

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-02-26 16:01:47 -07:00
Matt Holt 5d97522d18 v2: 'log' directive for Caddyfile, and debug mode (#3052)
* httpcaddyfile: Begin implementing log directive, and debug mode

For now, debug mode just sets the log level for all logs to DEBUG
(unless a level is specified explicitly).

* httpcaddyfile: Finish 'log' directive

Also rename StringEncoder -> SingleFieldEncoder

* Fix minor bug in replacer (when vals are empty)
2020-02-25 22:00:33 -07:00
Matthew Holt f6b9cb7122 httpcaddyfile: Matchers can now be embedded into a nested scope
This is useful in 'handle' and 'route' directives, for instance, if you
want to keep your matcher definitions by the directives that use them.
2020-02-25 21:56:43 -07:00
Matthew Holt 78760c0ddc go.mod: Bump to Go 1.14 2020-02-25 19:24:13 -07:00
Cameron Moore b0a491aec8 Expose TLS placeholders (#2982)
* caddytls: Add CipherSuiteName and ProtocolName functions

The cipher_suites.go file is derived from a commit to the Go master
branch that's slated for Go 1.14.  Once Go 1.14 is released, this file
can be removed.

* caddyhttp: Use commonLogEmptyValue in common_log replacer

* caddyhttp: Add TLS placeholders

* caddytls: update unsupportedProtocols

Don't export unsupportedProtocols and update its godoc to mention that
it's used for logging only.

* caddyhttp: simplify getRegTLSReplacement signature

getRegTLSReplacement should receive a string instead of a pointer.

* caddyhttp: Remove http.request.tls.client.cert replacer

The previous behavior of printing the raw certificate bytes was ported
from Caddy 1, but the usefulness of that approach is suspect.  Remove
the client cert replacer from v2 until a use case is presented.

* caddyhttp: Use tls.CipherSuiteName from Go 1.14

Remove ported version of CipherSuiteName in the process.
2020-02-25 19:22:50 -07:00
Success Go 45b171ff3a Make comment more readable about caddy ModuleID's Name() method. (#3080) 2020-02-25 09:11:29 -07:00
Success Go 623a1c588e Fix typo in cmdStart comment 2020-02-25 02:33:33 -05:00
Matthew Holt 7cca291d62 reverse_proxy: Health checks: Don't cross the streams
Fixes https://caddy.community/t/v2-health-checks-are-going-to-the-wrong-upstream/7084?u=matt

... I think
2020-02-23 14:31:05 -07:00
Robin Lambertz e3591009dc caddyhttp: Add handler for unhandled errors in errorChain (#3063)
* Add handler for unhandled errors in errorChain

Currently, when an error chain is defined, the default error handler is
bypassed entirely - even if the error chain doesn't handle every error.
This results in pages returning a blank 200 OK page.

For instance, it's possible for an error chain to match on the error
status code and only handle a certain subtype of errors (like 403s). In
this case, we'd want any other errors to still go through the default
handler and return an empty page with the status code.

This PR changes the "suffix handler" passed to errorChain.Compile to
set the status code of the response to the error status code.

Fixes #3053

* Move the errorHandlerChain middleware to variable

* Style fix
2020-02-20 15:00:30 -07:00
Gilbert Gilb's 30c14084ab caddyhttp: Fixes for header and header_regexp directives (#3061)
* Fix crash when specifying "*" to header directive.

Fixes #3060

* Look Host header in header and header_regexp.

Also, if more than one header is provided, header_regexp now looks for
extra headers values to reflect the behavior from header.

Fixes #3059

* Fix parsing of named header_regexp in Caddyfile.

See #3059
2020-02-20 10:55:47 -07:00
Matthew Holt 99f91c4c6f httpcaddyfile: tls: Load repeated cert files only once, with one tag
See end of issue #3004. Loading the same certificate file multiple times
with different tags will result in it being de-duplicated in the in-
memory cache, because of course they all have the same bytes. This
meant that any certs of the same filename loaded with different tags
would be overwritten by the next certificate of the same filename, and
any conn policies looking for the tags of the previous ones would never
find them, causing connections to fail.

So, now we remember cert filenames and their tags, instead of loading
them multiple times and overwriting previous ones.

A user crafting their own JSON might make this error too... maybe we
won't see it happen. But if it does, one possibility is, when loading
a duplicate cert, instead of discarding it completely, merge the tag
list into the one that's already stored in the cache, then discard.
2020-02-20 10:18:29 -07:00
Matthew Holt 0005e3acdc httpcaddyfile: Combine repeated cert loaders (fix #3004)
Also only append 1 catch-all TLS connection policy to a server, even if
multiple site blocks contribute to that server.
2020-02-20 00:15:11 -07:00
Matthew Holt 0b09b070e5 httpcaddyfile: Properly add all cert loaders across sites (fixes #3056) 2020-02-18 11:13:51 -07:00
Matthew Holt 7f9cfcc0f2 http: Close HTTP/3 servers and listeners; upstream bug irreproducible
See https://github.com/lucas-clemente/quic-go/issues/2103
and https://github.com/caddyserver/caddy/pull/2727
2020-02-18 10:39:34 -07:00
Matthew Holt 87a742c1e5 tls: Fix panic loading automation management modules (fix #3004)
When AutomationPolicy was turned into a pointer, we continued passing
a double pointer to LoadModule, oops.
2020-02-18 09:54:14 -07:00
Robin Lambertz 57c6f22684 basicauth: default hash to bcrypt (#3050)
The documentation specifies that the hash algorithm defaults to bcrypt.
However, the implementation returns an error in provision if no hash is
provided.

Fix this inconsistency by *actually* defaulting to bcrypt.
2020-02-17 12:19:59 -07:00
Marten Seemann dd103a6787 go.mod: update quic-go to v0.14.4 (#3048) 2020-02-17 08:54:03 -07:00
Matthew Holt 23cc26d585 httpcaddyfile: 'handle_errors' directive
Not sure I love the name of the directive; might change it later.
2020-02-16 22:24:20 -07:00
Matthew Holt bc2e406572 httpcaddyfile: Refactor global options parsing; prevent duplicate keys 2020-02-16 15:28:27 -07:00
Matthew Holt bf776e7de7 http: Remove redundant test file
Forgot to delete this when I moved its test into a different file
2020-02-16 15:27:53 -07:00
Matthew Holt f42b138fb1 tls: Avoid duplication AutomationPolicies for large quantities of names
This should greatly reduce memory usage at scale. Part of an overall
effort between Caddy 2 and CertMagic to optimize for large numbers of
names.
2020-02-14 11:14:52 -07:00
Matthew Holt 2cc5d2227d Minor tweaks to docs/comments 2020-02-14 11:01:09 -07:00
Matthew Holt 15bf9c196c caddyfile: Refactor; NewFromNextSegment(); fix repeated matchers
Now multiple instances of the same matcher can be used within a named
matcher without overwriting previous ones.
2020-02-14 11:01:09 -07:00
Mark Sargent eb80165583 tls: Add acme_ca_root and tls/ca_root to caddyfile (#3040) 2020-02-12 13:07:25 -07:00
Matthew Holt 17d938fc54 httpcaddyfile: Add support for DNS challenge solvers
Configuration via the Caddyfile requires use of env variables, but
an upstream issue is currently blocking that:
https://github.com/go-acme/lego/issues/1054

Providers will need to be retrofitted upstream in order to support env
var configuration.
2020-02-08 18:43:35 -07:00
Jeremy Lin 98bbc54fdc browse: allow filter init via filter query param (#3027)
This allows creating links that display only a subset of files in a directory.
2020-02-08 12:36:37 -07:00
Mohammed Al Sahaf 9bdd6caa0b v2: Implement RegExp Vars Matcher (#2997)
* implement regexp var matcher

* use subtests pattern for tests

* be more consistent with naming: MatchVarRE -> MatchVarsRE, var_regexp -> vars_regexp
2020-02-08 12:26:31 -07:00
Matthew Holt f7f6e371ef tls: Slight adjustment to how DNS provider modules are loaded
We don't load the provider directly, because the lego provider types
aren't designed for JSON configuration and they are not implemented
as Caddy modules (there are some setup steps which a Provision call
would need to do, but they do not have Provision methods, they have
their own constructor functions that we have to wrap).

Instead of loading the challenge providers directly, the modules are
simple wrappers over the challenge providers, to facilitate the JSON
config structure and to provide a consistent experience. This also lets
us swap out the underlying challenge providers transparently if needed;
it acts as a layer of abstraction.
2020-02-07 21:59:25 -07:00
Matthew Holt b8cf4d5897 Fix typo in readme 2020-02-07 11:26:48 -07:00
Matthew Holt 04ec3c5f05 Update readme
The list of improvements and FAQ were moved to the wiki for now. They
still need to be updated.
2020-02-07 10:59:09 -07:00
Matthew Holt 8b28c36d48 Remove Starlark, for now
This is temporary as we prepare for a stable v2 release. We don't want
to make promises we don't know we can keep, and the Starlark integration
deserves much more focused attention which resources and funding do not
currently permit. When the project is financially stable, I will be able
to revisit this properly and add flexible, robust Starlark scripting
support to Caddy 2.
2020-02-06 18:46:52 -07:00
Matthew Holt 4a07a5d41e caddyfile: tls: Ensure there is always a catch-all conn policy (#3005)
If user provides their own certs or makes any hostname-specific TLS
connection policy, it means that no TLS connection would be served for
any other hostnames, even though you'd expect that TLS is enabled for
them, too. So now we append a catch-all conn policy if none exist, which
allows all ClientHellos to be matched and served.

We also fix the consolidation of automation policies, which previously
gobbled up automation policies without hosts in favor of automation
policies with hosts. Instead of a host-specific policy eating up an
identical catch-all policy, the catch-all policy eats up the identical
host-specific policy, ensuring that the policy is applied to all hosts
which need it.

See also:
https://caddy.community/t/v2-automatic-https-certificate-errors/6847/9?u=matt
2020-02-06 13:00:41 -07:00
Matthew Holt b81ae38686 caddyfile: tls: Tag manual certificates (#2588)
This ensure that if there are multiple certs that match a particular
ServerName or other parameter, then specifically the one the user
provided in the Caddyfile will be used.
2020-02-06 12:55:26 -07:00
Matthew Holt 5c7ca7d96e http: Split 2-phase auto-HTTPS into 3 phases
This is necessary to avoid a race for sockets. Both the HTTP servers and
CertMagic solvers will try to bind the HTTP/HTTPS ports, but we need to
make sure that our HTTP servers bind first. This is kind of a new thing
now that management is async in Caddy 2.

Also update to CertMagic 0.9.2, which fixes some async use cases at
scale.
2020-02-05 17:34:28 -07:00
Francis Lavoie ec56c25708 caddyhttp: Fix orig_uri placeholder docs (#3002)
Fixes #3001
2020-02-04 15:49:38 -07:00
Matthew Holt c0f827e0bd httpcaddyfile: Add {remote} shorthand placeholders
Also sort the list
2020-02-04 13:31:22 -07:00
Matthew Holt 490cd02f82 httpcaddyfile: Make root directive mutually exclusive
See https://caddy.community/t/caddyfile-and-v2/6766/22?u=matt
2020-02-04 13:04:34 -07:00
Matthew Holt 9639fe7d28 header: caddyfile: Defer header operations for deletions or manually
See https://caddy.community/t/caddy-server-that-returns-only-ip-address-as-text/6928/6?u=matt

In most cases, we will want to apply header operations immediately,
rather than waiting until the response is written. The exceptions are
generally going to be if we are deleting a header field or if a field is
to be overwritten. We now automatically defer header ops if deleting a
header field, and allow the user to manually enable deferred mode with
the defer subdirective.
2020-02-04 11:05:32 -07:00
Matthew Holt 3592e59399 cmd: adapt: Make --config flag optional when Caddyfile exists 2020-02-04 10:48:02 -07:00
Mohammed Al Sahaf f74fed3f54 v2: only compare TLS protocol versions if both are set (#3005) 2020-02-03 09:25:32 -07:00
Matthew Holt 8b2ad61220 httpcaddyfile: Skip hosts from auto-https when http:// scheme (fix #2998) 2020-01-23 13:17:16 -07:00
Matthew Holt 6614d1c495 cmd: Emit error if reload cannot find a config to load 2020-01-22 10:04:58 -07:00
Matthew Holt c6bddbfbe2 http: Fix vars matcher 2020-01-22 09:43:42 -07:00
Matthew Holt 0742530d3d rewrite: Prepend "/" if missing from strip path prefix
Paths always begin with a slash, and omitting the leading slash could be
convenient to avoid confusion with a path matcher in the Caddyfile. I do
not think there would be any harm to implicitly add the leading slash.
2020-01-22 09:36:05 -07:00
Matthew Holt 6b6cd934d0 reverseproxy: Fix casing of RootCAPEMFiles 2020-01-22 09:35:03 -07:00
Matthew Holt 5b878d5bd3 reverseproxy: Accept integer values for flush_interval (fix #2996) 2020-01-22 09:34:16 -07:00
Matthew Holt 2105d59936 httpcaddyfile: Rename 'headers' directive to 'header' 2020-01-22 09:33:53 -07:00
Matthew Holt 9a1370c2c8 cmd: Make --config flag optional for reload command
In case it is using the default Caddyfile
2020-01-22 09:33:22 -07:00
Matthew Holt d810637a9f httpcaddyfile: Update directive docs; put root after rewrite 2020-01-22 09:32:38 -07:00
Matthew Holt 5d3ccf1eb7 httpcaddyfile: Get rid of 'tls off' parameter; probably not useful 2020-01-22 09:29:50 -07:00
Matthew Holt aad9f90cad httpcaddyfile: Fix address parsing; don't infer port at parse-time
Before, listener ports could be wrong because ParseAddress doesn't know
about the user-configured HTTP/HTTPS ports, instead hard-coding port 80
or 443, which could be wrong if the user changed them to something else.
Now we defer port and scheme validation/inference to a later part of
building the output JSON.
2020-01-19 11:51:17 -07:00
Zaq? Wiedmann 07ef4b0c7d Merge pull request #2980 from moorereason/bugfix-ciphersuite-logging
v2: http: Fix ciphersuite logging
2020-01-18 19:37:50 -08:00
Mohammed Al Sahaf 2bfaf8e896 reverse_proxy: CB docs; rename type -> factor (#2986)
* v2: add documentation for circuit breaker config and "random selection" load balancing policy

* v2: rename circuit breaker config inline key from `type` to `breaker` to avoid json key clash between the `circuit_breaker` type and the `type` field of the generic circuit breaker Config struct used by circuit breaking implementations

* v2: restore the circuit breaker inline key to `type` and rename the name circuit breaker config field from `Type` to `Factor`
2020-01-18 18:42:56 -07:00
Matthew Holt 372540f0ee httpcaddyfile: Move redir before rewrite
Using rewrite is like saying, "I accept this request, but I just need
to act on it as if it came in differently."

Whereas redir implies more of, "I reject this request, send it to me
differently, then I will process it."

Makes sense for it to come before rewrites. This can always be changed
using the 'order' global option if needed.
2020-01-17 11:38:49 -07:00
Matthew Holt 793a405810 caddyhttp: Improve docs, and Caddyfile for respond directive 2020-01-17 10:57:57 -07:00
Matthew Holt 85ff0e3604 cmd: version: Add module replace to output 2020-01-17 09:50:23 -07:00
Matthew Holt e51e56a494 httpcaddyfile: Fix nested blocks; add handle directive; refactor
The fix that was initially put forth in #2971 was good, but only for
up to one layer of nesting. The real problem was that we forgot to
increment nesting when already inside a block if we saw another open
curly brace that opens another block (dispenser.go L157-158).

The new 'handle' directive allows HTTP Caddyfiles to be designed more
like nginx location blocks if the user prefers. Inside a handle block,
directives are still ordered just like they are outside of them, but
handler blocks at a given level of nesting are mutually exclusive.

This work benefitted from some refactoring and cleanup.
2020-01-16 17:08:52 -07:00
Cameron Moore 35174a8ba8 http: Fix ciphersuite logging 2020-01-16 15:44:49 -06:00
Matthew Holt 21643a007a httpcaddyfile: Replace 'handler_order' option with 'order'
This allows individual directives to be ordered relative to others,
where order matters (for example HTTP handlers). Will primarily be
useful when developing new directives, so you don't have to modify the
Caddy source code. Can also be useful if you prefer that redir comes
before rewrite, for example. Note that these are global options. The
route directive can be used to give a specific order to a specific group
of HTTP handler directives.
2020-01-16 12:09:54 -07:00
Matthew Holt 2466ed1484 httpcaddyfile: Group try_files routes together (#2891)
This ensures that only the first matching route is used.
2020-01-16 11:29:20 -07:00
Matthew Holt a66f461201 caddyfile: Sort site subroutes by key specificity, and make exclusive
In the v1 Caddyfile, only the first matching site definition would be
used, so setting these `Terminal: true` ensures that only the first
matching one is used in v2, too.

We also have to sort by key specificity... Caddy 1 had a special data
structure for selecting the most specific site definition, but we don't
have that structure in v2, so we need to sort by length (of host and
path, separately). For blocks where more than one key is present, we
choose the longest host and path (independently, need not be from same
key) by which to sort.
2020-01-15 13:51:12 -07:00
Matthew Holt 07ad4655db rewrite: Make URI modifications more transactional (#2891)
Before, modifying the path might have affected how a new query string
was built if the query string relied on the path. Now, we build each
component in isolation and only change the URI on the request later.

Also, prevent trailing & in query string.
2020-01-15 11:44:21 -07:00
Matthew Holt 271b5af148 http: Refactor automatic HTTPS (fixes #2972)
This splits automatic HTTPS into two phases. The first provisions the
route matchers and uses them to build the domain set and configure
auto HTTP->HTTPS redirects. This happens before the rest of the
provisioning does.

The second phase takes place at the beginning of the app start. It
attaches pointers to the tls app to each server, and begins certificate
management for the domains that were found in the first phase.
2020-01-13 16:16:20 -07:00
Matthew Holt 99e2b56519 cmd: adapt: Set config filename so it can be hidden (fixes #2974) 2020-01-12 18:20:19 -07:00
Matthew Holt 64f0173948 http: Fix subroutes, ensure that next handlers can still be called 2020-01-12 13:39:32 -07:00
Matthew Holt fe5a531c58 http: Fix empty responses
Sigh... this is what I get for writing code when I'm tired and sick.

See https://github.com/caddyserver/caddy/commit/8be1f0ea668492000cdefbd937e0359bdc24bfc1#r36764627
2020-01-12 13:34:55 -07:00
Matthew Holt 8c0c1a7b88 cmd: Assume Caddyfile if name starts with Caddyfile
And doesn't have .json extension -- in case someone names their
JSON config something like Caddyfile.json, which is unconventional.
2020-01-11 13:48:29 -07:00
Matthew Holt 25dea2903e http: A little more polish on rewrite handler and try_files directive 2020-01-11 13:47:42 -07:00
Matthew Holt d876de61e5 rewrite: Fix query string logic 2020-01-11 11:40:03 -07:00
Matthew Holt 8be1f0ea66 http: Ensure primary routes always get compiled (fix #2972)
Including servers for HTTP->HTTPS redirects which do not get provisioned
like the rest.
2020-01-11 00:33:47 -07:00
Matthew Holt 2eda21ec6d http: Remove {...query_string} placeholder, in favor of {...query}
I am not sure if the query_string one is necessary or useful yet. We
can always add it later if needed.
2020-01-10 17:02:11 -07:00
Matthew Holt d418e319ab rewrite: Rename parameters; implement custom query string parser
Our new parser also preserves original parameter order, rather than
re-encoding using the std lib (which sorts).

The renamed parameters are a breaking change but they're new enough
that I don't think anyone is using them.
2020-01-10 17:00:57 -07:00
Matthew Holt ba514f9660 cmd: Add build-info command; update CertMagic 2020-01-10 11:53:07 -07:00
Zaq? Wiedmann 3dcc34d341 caddyfile: advance cursor for claimed token in NewFromNextTokens() (#2971)
When we append a token to the new dispenser, we need to consume it in the parent, too; otherwise it gets scanned twice, which in this case messed up the nesting count which got decremented once too many times.
2020-01-09 20:48:15 -07:00
Mark Sargent 871abf1053 caddyfile: fix replacing variables on imported files (#2970)
* fix replacing variables on imported files

* refactored replaceEnvVars to ensure it is always called

* Use byte slices for easier use

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-01-09 19:34:22 -07:00
Matthew Holt 29315847a8 caddyfile: Use of vars no longer requires nesting in subroutes
This is because of our sequential handling logic which was recently
merged; if vars is the first handler in the chain, it will be run before
the next route's matchers are executed, so there's no need to nest the
handlers anymore.
2020-01-09 16:56:20 -07:00
Matthew Holt 994b9033e9 http: Don't use a Host matcher for HTTP->HTTPS redirects
In case on-demand TLS is enabled, in that case we don't know the only
names that have automatic HTTPS.

See https://caddy.community/t/v2-http-to-https-redirects-fail-for-on-demand-ssl-certs/6742?u=matt
2020-01-09 14:39:49 -07:00
Matthew Holt 590480513a Update docs for couple of Caddyfile directives 2020-01-09 14:38:59 -07:00
Matt Holt 7527c01705 v2: Implement Caddyfile enhancements (breaking changes) (#2960)
* http: path matcher: exact match by default; substring matches (#2959)

This is a breaking change.

* caddyfile: Change "matcher" directive to "@matcher" syntax (#2959)

* cmd: Assume caddyfile adapter for config files named Caddyfile

* Sub-sort handlers by path matcher length (#2959)

Caddyfile-generated subroutes have handlers, which are sorted first by
directive order (this is unchanged), but within directives we now sort
by specificity of path matcher in descending order (longest path first,
assuming that longest path is most specific).

This only applies if there is only one matcher set, and the path
matcher in that set has only one path in it. Path matchers with two or
more paths are not sorted like this; and routes with more than one
matcher set are not sorted like this either, since specificity is
difficult or impossible to infer correctly.

This is a special case, but definitely a very common one, as a lot of
routing decisions are based on paths.

* caddyfile: New 'route' directive for appearance-order handling (#2959)

* caddyfile: Make rewrite directives mutually exclusive (#2959)

This applies only to rewrites in the top-level subroute created by the
HTTP caddyfile.
2020-01-09 14:00:32 -07:00
Matthew Holt 8aef859a55 caddyfile: Less strict URL parsing; allows placeholders
See https://caddy.community/t/caddy-v2-reusable-snippets/6744/11?u=matt
2020-01-09 12:35:53 -07:00
Matt Holt a5ebec0041 http: Change routes to sequential matcher evaluation (#2967)
Previously, all matchers in a route would be evaluated before any
handlers were executed, and a composite route of the matching routes
would be created. This made rewrites especially tricky, since the only
way to defer later matchers' evaluation was to wrap them in a subroute,
or to invoke a "rehandle" which often caused bugs.

Instead, this new sequential design evaluates each route's matchers then
its handlers in lock-step; matcher-handlers-matcher-handlers...

If the first matching route consists of a rewrite, then the second route
will be evaluated against the rewritten request, rather than the original
one, and so on.

This should do away with any need for rehandling.

I've also taken this opportunity to avoid adding new values to the
request context in the handler chain, as this creates a copy of the
Request struct, which may possibly lead to bugs like it has in the past
(see PR #1542, PR #1481, and maybe issue #2463). We now add all the
expected context values in the top-level handler at the server, then
any new values can be added to the variable table via the VarsCtxKey
context key, or just the GetVar/SetVar functions. In particular, we are
using this facility to convey dial information in the reverse proxy.

Had to be careful in one place as the middleware compilation logic has
changed, and moved a bit. We no longer compile a middleware chain per-
request; instead, we can compile it at provision-time, and defer only the
evaluation of matchers to request-time, which should slightly improve
performance. Doing this, however, we take advantage of multiple function
closures, and we also changed the use of HandlerFunc (function pointer)
to Handler (interface)... this led to a situation where, if we aren't
careful, allows one request routed a certain way to permanently change
the "next" handler for all/most other requests! We avoid this by making
a copy of the interface value (which is a lightweight pointer copy) and
using exclusively that within our wrapped handlers. This way, the
original stack frame is preserved in a "read-only" fashion. The comments
in the code describe this phenomenon.

This may very well be a breaking change for some configurations, however
I do not expect it to impact many people. I will make it clear in the
release notes that this change has occurred.
2020-01-09 10:00:13 -07:00
Mark Sargent 7c419d5349 caddyfile: Preprocess env vars in {$THIS} format (#2963)
* transform a caddyfile with environment variables

* support adapt time and runtime variables in the caddyfile

* caddyfile: Pre-process environment variables before parsing

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
2020-01-09 09:40:16 -07:00
Matthew Holt 3828a3aaac go.mod: Update lego, tidy up 2020-01-08 18:40:17 -07:00
Matthew Holt 8bae8f5f5a http: Always set status code via response recorder
Fixes panic if no upstream handler wrote anything to the response
2020-01-08 18:37:41 -07:00
Zaq? Wiedmann 21f1f95e7b reverse_proxy: Add tls_trusted_ca_certs to Caddyfile (#2936)
Allows specifying ca certs with by filename in
`reverse_proxy.transport`.

Example
```
reverse_proxy /api api:443 {
    transport http {
        tls
        tls_trusted_ca_certs certs/rootCA.pem
    }
}
```
2020-01-07 12:07:42 -07:00
Matthew Holt 78e98c40d3 basicauth: Accept placeholders; move base64 decoding to provision
See https://caddy.community/t/v2-basicauth-bug/6738?u=matt
2020-01-07 08:50:18 -07:00
Matthew Holt 5c99267dd8 A few miscellaneous, minor fixes 2020-01-06 08:10:20 -07:00
Matthew Holt a6df4cdbbc logging: Add doc about which fields can't be filtered 2020-01-03 15:28:05 -07:00
Mohammed Al Sahaf dff78d82ce v2: housekeeping: address minor lint complaints (#2957)
* v2: housekeeping: update tools

* v2: housekeeping: adhere to US locale in spelling

* v2: housekeeping: simplify code
2020-01-03 11:33:22 -07:00
Matthew Holt 8c7c2e4af2 logging: Little fix for filtering object fields 2020-01-01 10:26:37 -07:00
Matthew Holt 3d9f8eac08 Couple of minor fixes, update readme 2019-12-31 22:51:55 -07:00
Matthew Holt 06ea0a5295 Tune AppConfigDir and docs for Storage module 2019-12-31 18:31:43 -07:00
Matthew Holt 788462bd4c file-server command: Use safer defaults; http: improve host matcher docs 2019-12-31 16:57:54 -07:00
Matthew Holt 5a0603ed72 Config auto-save; run --resume flag; update environ output (close #2903)
Config auto-saving is on by default and can be disabled. The --environ
flag (or environ subcommand) now print more useful information from
Caddy and the runtime, including some nifty paths.
2019-12-31 16:56:19 -07:00
Matthew Holt 984d384d14 Change storage paths to follow OS conventions; migrate folder (#2955) 2019-12-31 16:47:35 -07:00
Matthew Holt fdabac51a8 Improve docs, especially w.r.t. placeholders and template actions 2019-12-29 13:16:34 -07:00
Matthew Holt 95d944613b Export Replacer and use concrete type instead of interface
The interface was only making things difficult; a concrete pointer is
probably best.
2019-12-29 13:12:52 -07:00
Matthew Holt 2b33d9a5e5 http: Enable TLS for servers listening only on HTTPS port
It seems silly to have to add a single, empty TLS connection policy to
a server to enable TLS when it's only listening on the HTTPS port. We
now do this for the user as part of automatic HTTPS (thus, it can be
disabled / overridden).

See https://caddy.community/t/v2-catch-all-server-with-automatic-tls/6692/2?u=matt
2019-12-28 23:56:08 -07:00
Matthew Holt 5c8b502964 fastcgi: Set SERVER_SOFTWARE, _NAME, and _PORT properly (fixes #2952) 2019-12-28 16:35:29 -07:00
Matthew Holt 82bebfab8a templates: Change functions, add front matter support, better markdown 2019-12-23 12:56:41 -07:00
Matthew Holt be3849c267 Remove markdown module 2019-12-23 12:55:52 -07:00
Matthew Holt 16ee985c22 admin: Only write most CORS headers in OPTIONS requests 2019-12-23 12:46:01 -07:00
Matthew Holt 95ed603de7 Improve godocs all around
These will be used in the new automated documentation system
2019-12-23 12:45:35 -07:00
Matthew Holt cbb405f6aa cmd: Eliminate unintended use of cgo
This means the stop command can only use the API to stop the instance;
no more signaling, unless we find a cgo-free way of doing it.
2019-12-23 12:41:05 -07:00
Matthew Holt 724c728678 rewrite: Attempt query string fix (#2891) 2019-12-17 16:30:26 -07:00
Matthew Holt 21408212da http: query and query_string placeholders should use RawQuery, probably 2019-12-17 16:29:37 -07:00
Matthew Holt fe516575db core: Add ReplaceFunc method to Replacer to allow dynamic replacements 2019-12-17 16:29:09 -07:00
Matthew Holt 080a62d5c5 Update go.mod; use CertMagic v0.9.0 2019-12-17 10:59:35 -07:00
Matthew Holt dae4913fe3 http: Patch path matcher to ignore dots and spaces (#2917)
(Try saying "patch path match" ten times fast)
2019-12-17 10:14:04 -07:00
Matthew Holt 6455efa5d3 admin: POST /... expands and appends all array elements
Makes it easy to append many items to an array in one command
2019-12-17 10:11:45 -07:00
Matthew Holt 5ab17a3a37 admin: /stop endpoint gracefully shuts down; fixes caddy stop command 2019-12-16 13:46:39 -07:00
Abdelmalek Ihdene c3bcd967bd logging: Implement net writer (#2884)
* Implement UDP writer

* Implement Net Writer

* Utilize Caddy's address parsing functions

* A couple little fixes (see #2884)
2019-12-15 12:58:01 -07:00
Matthew Holt 6ea121ddf8 tls: Ensure conn policy is created when providing certs in Caddyfile
Fixes #2929
2019-12-13 16:32:27 -07:00
Matthew Holt 8005b7ab73 Couple of quick fixes 2019-12-13 15:36:00 -07:00
Matthew Holt b1a456cfe3 rewrite: strip_prefix, strip_suffix, and uri_replace dirs (closes #2906) 2019-12-12 15:46:13 -07:00
Matthew Holt 5e9d81b507 try_files, rewrite: allow query string in try_files (fix #2891)
Also some minor cleanup/improvements discovered along the way
2019-12-12 15:27:09 -07:00
Matthew Holt 09a8517065 rewrite: query string enh.; substring replace; add tests (see #2891) 2019-12-12 14:32:35 -07:00
Matthew Holt 87b6cf470b Minor improvements; comments and shorter placeholders & module IDs 2019-12-12 14:31:20 -07:00
Matthew Holt f935458e3e cmd: Fix validate command when JSON contains "@id" fields
Also, don't run admin server when validating...
2019-12-12 14:30:22 -07:00
Matt Holt 2e0615270d fuzz: Remove Caddyfile adapter from fuzz corpus (#2925)
The Caddyfile adapter does not need to be fuzzed, as all it really does
is invoke the Caddyfile parser, which is already fuzzed
2019-12-10 15:00:31 -07:00
Matthew Holt fab5e4372a core: Add godoc examples for LoadModule 2019-12-10 14:06:35 -07:00
Matt Holt 3c90e370a4 v2: Module documentation; refactor LoadModule(); new caddy struct tags (#2924)
This commit goes a long way toward making automated documentation of
Caddy config and Caddy modules possible. It's a broad, sweeping change,
but mostly internal. It allows us to automatically generate docs for all
Caddy modules (including future third-party ones) and make them viewable
on a web page; it also doubles as godoc comments.

As such, this commit makes significant progress in migrating the docs
from our temporary wiki page toward our new website which is still under
construction.

With this change, all host modules will use ctx.LoadModule() and pass in
both the struct pointer and the field name as a string. This allows the
reflect package to read the struct tag from that field so that it can
get the necessary information like the module namespace and the inline
key.

This has the nice side-effect of unifying the code and documentation. It
also simplifies module loading, and handles several variations on field
types for raw module fields (i.e. variations on json.RawMessage, such as
arrays and maps).

I also renamed ModuleInfo.Name -> ModuleInfo.ID, to make it clear that
the ID is the "full name" which includes both the module namespace and
the name. This clarity is helpful when describing module hierarchy.

As of this change, Caddy modules are no longer an experimental design.
I think the architecture is good enough to go forward.
2019-12-10 13:36:46 -07:00
Marten Seemann a8533e5630 update quic-go to v0.14.1 (#2918) 2019-12-07 10:29:03 -07:00
Matthew Holt b07f6958ac Use "IsUnixNetwork" function instead of repeating the logic 2019-12-06 12:00:04 -07:00
Matthew Holt 33a318d173 Don't append port to unix sockets
See https://caddy.community/t/caddy-v2-php-fpm-502-error/6571?u=matt
2019-12-06 11:45:50 -07:00
lu4p 68adfdc559 Fix misspellings (#2908) 2019-12-04 16:28:13 -07:00
Marten Seemann a841688cc0 update quic-go to v0.14.0 (#2916) 2019-12-03 20:49:01 -07:00
Matthew Holt 52ae5f70d2 Merge branch 'v2' of ssh://github.com/caddyserver/caddy into v2 2019-11-30 17:53:38 -07:00
Matthew Holt 44f23a67bb http: Don't listen 1 port beyond port range 2019-11-30 17:53:25 -07:00
Mark Sargent 8b7d6a9ee8 v2: fixes query matcher parsing (#2901)
* fixes query matcher parsing

* return correct argument error when parsing query matcher
2019-11-29 13:05:22 -07:00
Matthew Holt 7c7ef8d40e http: Shorten regexp matcher placeholders; allow "=/" for simple matcher 2019-11-29 11:23:49 -07:00
Matthew Holt 14d3fd7d03 http: path matcher supports exact matching with = prefix 2019-11-28 21:11:45 -07:00
Matthew Holt 512b004332 http: header matcher supports fast prefix and suffix matching (#2888) 2019-11-27 11:52:31 -07:00
Matthew Holt db4293cb5f reverse_proxy: Add flush_interval to caddyfile syntax (#1460)
Also add godoc for Caddyfile syntax for file_server
2019-11-27 11:51:32 -07:00
Matthew Holt 6e10586303 admin: Preserve "@id" fields through partial changes (fixes #2902) 2019-11-27 11:49:49 -07:00
Matthew Holt 8de1a76227 reverse_proxy: Fix invalid argument to Intn in RandomChoice selection 2019-11-18 14:22:55 -07:00
Matthew Holt 9fe54e1c60 file_server: Use HTTPS port when a qualifying domain is specified
Also little comment cleanups
2019-11-16 10:44:45 -07:00
Matthew Holt b43e986a52 file_server: Optional pass_thru mode
If enabled, will call the next handler in the chain instead of returning
a 404.
2019-11-15 17:32:13 -07:00
Matthew Holt 1228dd7d93 reverse_proxy: Allow buffering of client requests
This is a bad idea, but some backends apparently require it. See
discussion in #176.
2019-11-15 17:15:33 -07:00
Matthew Holt af26a03da1 http: Only enable access logs if configured 2019-11-15 17:01:07 -07:00
Matthew Holt 8025ad9107 cmd: Disable admin endpoint for file-server and reverse-proxy commands
This makes it easier to use multiple instances on the same machine
2019-11-15 15:52:19 -07:00
Matthew Holt 6cdb2392d7 cmd: Improve stop command by trying API before signaling process
This allows graceful shutdown on all platforms
2019-11-15 15:45:18 -07:00
Matthew Holt 0ca109db4a Minor cleanups 2019-11-15 12:47:38 -07:00
Matthew Holt 0fc97211ab http: Make path matcher case-insensitive
Adds tests for both the path matcher and host matcher for case
insensitivity.

If case sensitivity is required for the path, a regexp matcher can
be used instead.

This is the v2 equivalent fix of PR #2882.
2019-11-15 12:47:06 -07:00
Matthew Holt ad90b273db core: Add tests to Replacer; fix panic (fixes #2852) 2019-11-11 19:29:31 -07:00
Mohammed Al Sahaf 93bc1b72e3 core: Use port ranges to avoid OOM with bad inputs (#2859)
* fix OOM issue caught by fuzzing

* use ParsedAddress as the struct name for the result of ParseNetworkAddress

* simplify code using the ParsedAddress type

* minor cleanups
2019-11-11 15:33:38 -07:00
Matthew Holt a19da07b72 http: Add response headers to access logs 2019-11-11 14:02:01 -07:00
Matthew Holt 16782d9988 http: Use permanent redirects for HTTP->HTTPS 2019-11-11 14:01:42 -07:00
Sarat Chandra dfdddcfacb logging: Support placeholders in level and filename (#2872)
* Add support for placeholders in Config

Fixes #2870

* Replace placeholders only in logging config.

Placeholders in log level and filename incase of file output are replaced.

* Add Provision to filewriter module for replacing placeholders
2019-11-11 11:04:41 -07:00
Marten Seemann 7ff02f37b6 go.mod: update quic-go to v0.13.1 (#2871) 2019-11-09 08:10:43 -07:00
Matthew Holt e4a2add73f cmd: Print errors to stderr 2019-11-08 09:59:49 -07:00
Matthew Holt 95615f5377 reverse_proxy: Fix NTLM auth detection
D'oh. Got mixed up in a refactoring.
2019-11-06 00:16:16 -07:00
Matthew Holt 8e515289cb reverse_proxy: Add support for NTLM 2019-11-05 16:29:10 -07:00
Matthew Holt 6e95477224 http: Eliminate allocation in cloneURL; add RemoteAddr to origRequest 2019-11-05 16:28:33 -07:00
Matthew Holt 97d918df3e reverse_proxy: Make HTTP versions configurable, don't set NextProtos 2019-11-05 16:27:51 -07:00
Matthew Holt f5c6a8553c Prepare for beta 9 tag 2019-11-04 13:43:39 -07:00
Matthew Holt 263ffbfaec caddyfile: Fix bug with Delete
It now will delete the current token even if it is the last one
2019-11-04 13:25:37 -07:00
Matthew Holt bf363f061d reverse_proxy: Add UnmarshalCaddyfile for random_choose selection policy
Also allow caddy.Duration to be given integer values which are treated
like regular time.Duration values (nanoseconds).

Fixes #2856
2019-11-04 12:54:46 -07:00
Matthew Holt 7129f6c1c0 admin: Remove /unload endpoint (is same as DELETE /config/) 2019-11-04 12:53:14 -07:00
Matthew Holt cb25dd72ab reverse_proxy: Add port to upstream address if only implied in scheme 2019-11-04 12:18:42 -07:00
Matthew Holt d55fa68902 http: Only log handler errors >= 500
Errors in the 4xx range are client errors, and they don't need to be
entered into the server's error logs. 4xx errors are still recorded in
the access logs at the error level.
2019-11-04 12:18:01 -07:00
Matthew Holt b1f41d0ff1 logging: Default logger should use wall time with milliseconds
This format is easier for humans to read and is still very precise.
2019-11-04 12:14:22 -07:00
Matthew Holt 6011ce120a cmd: Move module imports into standard packages
This makes it easier to make "standard" caddy builds, since you'll only
need to add a single import to get all of Caddy's standard modules.

There is a package for all of Caddy's standard modules (modules/standard)
and a package for the HTTP app's standard modules only
(modules/caddyhttp/standard).

We still need to decide which of these, if not all of them, should be
kept in the standard build. Those which aren't should be moved out of
this repo. See #2780.
2019-11-04 12:13:21 -07:00
Matthew Holt 27e288ab19 core: Synchronize calls to SetDeadline within fakeCloseListener
First evidenced in #2658, listener deadlines would sometimes be set
after clearing them, resulting in endless i/o timeout errors, which
leave all requests hanging. This bug is fixed by synchronizing the
calls to SetDeadline: when Close() is called, the deadline is first
set to a time in the past, and the lock is released only after the
deadline is set, so when the other servers break out of their Accept()
calls, they will clear the deadline *after* it was set. Before, the
clearing could sometimes come before the set, which meant that it was
left in a timeout state indefinitely.

This may not yet be a perfect solution -- ideally, the setting and
clearing of the deadline would happen exactly once per underlying
listener, not once per fakeCloseListener, but in rigorous testing with
these changes (comprising tens of thousands of config reloads), I was
able to verify that no race condition is manifest.
2019-11-04 12:10:03 -07:00
Matthew Holt 35f70c98fa core: Major refactor of admin endpoint and config handling
Fixed several bugs and made other improvements. All config changes are
now mediated by the global config state manager. It used to be that
initial configs given at startup weren't tracked, so you could start
caddy with --config caddy.json and then do a GET /config/ and it would
return null. That is fixed, along with several other general flow/API
enhancements, with more to come.
2019-11-04 12:05:20 -07:00
Matthew Holt fb06c041c4 http: Ensure server loggers are not nil (fixes #2849) 2019-10-31 11:45:18 -06:00
Matthew Holt 8ef0a0b4f8 reverse_proxy: Fix panic for some CLI flag values (closes #2848) 2019-10-31 11:34:54 -06:00
Matthew Holt 8d3c64932e http: Avoid panic if handler errors lack underlying error value
Fixes #2845
2019-10-30 21:41:52 -06:00
Mohammed Al Sahaf 0dd9243478 Re-remove admin fuzz target from azure-pipelines.yml (#2846)
Fixing a git-oopsie on my behalf
2019-10-31 01:49:18 +03:00
Andreas Schneider 432b94239d admin listener as opt-in for initial config (#2834)
* Always cleanup admin endpoint first

* Error out if no config has been set (#2833)

* Ignore explicitly missing admin config (#2833)

* Separate config loading from admin initialization (#2833)

* Add admin option to specify admin listener address (#2833)

* Use zap for reporting admin endpoint status
2019-10-30 15:12:42 -06:00
Mohammed Al Sahaf 4611537f06 Add missing fuzzer (#2844)
* fuzz: add missing fuzzer by fixing .gitignore adding a negation for caddyfile/ directory

* ci: print fuzzing type for debuggability and traceability

* README: update the Fuzzit badge to point to the correct Caddy server Github organization
2019-10-30 23:57:22 +03:00
Matthew Holt 76c22c7b38 auth: Clean up basicauth 2019-10-30 13:56:27 -06:00
Matthew Holt c7da6175bc fuzz: Remove admin fuzzer
Not really necessary; underlying work is done by json.Unmarshal which
is part of the Go standard lib. Also, it called Run, which potentially
tries to get certificates; we should not let that happen.
2019-10-30 12:19:59 -06:00
Matthew Holt 11a2733dc2 ci: Change fuzz type from regression to local-regression
As per recommendation from Fuzzit devs
2019-10-30 11:50:19 -06:00
Matthew Holt 1be121cec7 fuzz: Don't call Load() in HTTP caddyfile adapter fuzz tests
Doing so has a tendency to request certificates...
2019-10-30 11:48:21 -06:00
Matthew Holt dccba71276 reverse_proxy: Structured logs 2019-10-29 16:02:58 -06:00
Mohammed Al Sahaf be36aade9a ci: Update fuzzer target name (#2841)
Update the fuzzer target name for the address parser so it better matches the func name
2019-10-29 13:20:34 -06:00
Matthew Holt ba0000678d Remove unused fields from HandlerError 2019-10-29 11:59:08 -06:00
Matthew Holt c4c45f8e01 logging: Tweak defaults (enable logging by default, color level enc.) 2019-10-29 11:58:29 -06:00
Matthew Holt 54e458b756 proxy: Forgot to commit import 2019-10-29 10:22:49 -06:00
Matthew Holt d803561212 caddyhttp: Fix nil pointer dereference 2019-10-29 00:08:06 -06:00
Matthew Holt 813fff0584 proxy: Enable HTTP/2 on transport to backend 2019-10-29 00:07:45 -06:00
Matthew Holt d2e7baed8d Plug in distributed STEK module 2019-10-29 00:06:04 -06:00
Matthew Holt d6dad04e96 cache: Make peer addresses configurable 2019-10-28 15:09:12 -06:00
Matthew Holt 442fd748f6 caddyhttp: Minor cleanup and fix nil pointer deref in caddyfile adapter 2019-10-28 15:08:45 -06:00
Matt Holt b00dfd3965 v2: Logging! (#2831)
* logging: Initial implementation

* logging: More encoder formats, better defaults

* logging: Fix repetition bug with FilterEncoder; add more presets

* logging: DiscardWriter; delete or no-op logs that discard their output

* logging: Add http.handlers.log module; enhance Replacer methods

The Replacer interface has new methods to customize how to handle empty
or unrecognized placeholders. Closes #2815.

* logging: Overhaul HTTP logging, fix bugs, improve filtering, etc.

* logging: General cleanup, begin transitioning to using new loggers

* Fixes after merge conflict
2019-10-28 14:39:37 -06:00
Mohammed Al Sahaf 6c533558a3 fuzz-ci: fix & enhance fuzzing process (#2835)
* fuzz-ci: fix the authentication call for fuzzit by using the --api-key flag rather than the `auth` command

* Allow fuzzing on schedules as well as non-fork PRs

Closes #2710
2019-10-28 20:45:55 +03:00
Mohammed Al Sahaf 2fbe2ff40b fuzz: introduce continuous fuzzing for Caddy (#2723)
* fuzz: lay down the foundation for continuous fuzzing

* improve the fuzzers and add some

* fuzz: add Fuzzit badge to README & enable fuzzers submission in CI

* v2-fuzz: do away with the submodule approach for fuzzers

* fuzz: enable fuzzit
2019-10-25 18:52:16 -06:00
Matthew Holt faf67b1067 tls: Make the on-demand rate limiter actually work
This required a custom rate limiter implementation in CertMagic
2019-10-21 12:03:51 -06:00
Matthew Holt 208f2ff93c rewrite: Options to strip prefix/suffix and issue redirects
Fixes #2011
2019-10-19 19:22:29 -06:00
Mohammed Al Sahaf 19e834cf36 v2 ci: speed up some of powershell's processes (#2818)
* v2: speed up some of powershell's processes

* v2-ci: downloading latest Go on Windows isn't slow anymore, so update the log message accordingly

* v2: CI: use 7z on Windows instead of Expand-Archive
2019-10-17 14:58:22 -06:00
Matthew Holt bce2edd22d tls: Asynchronous cert management at startup (uses CertMagic v0.8.0) 2019-10-16 15:20:27 -06:00
Matthew Holt a458544d9f Minor enhancements/fixes to rewrite directive and template virt req's 2019-10-16 15:18:02 -06:00
Matt Holt 2f91b44587 v2: Make tests work on Windows (#2782)
* file_server: Make tests work on Windows

* caddyfile: Fix escaping when character is not escapable

We only escape certain characters depending on inside or outside of
quotes (mainly newlines and quotes). We don't want everyone to have to
escape Windows file paths like C:\\Windows\\... but we can't drop the
\ either if it's just C:\Windows\...
2019-10-15 16:05:53 -06:00
Mohammed Al Sahaf e3726588b4 v2: Project-and-CI-wide linter config (#2812)
* v2: split golangci-lint configuration into its own file to allow code editors to take advantage of it

* v2: simplify code

* v2: set the correct lint output formatting

* v2: invert the logic of linter's configuration of output formatting to allow the editor  convenience over CI-specific customization. Customize the output format in CI by passing the flag.

* v2: remove irrelevant golangci-lint config
2019-10-15 15:37:46 -06:00
Matthew Holt abf5ab340e caddyhttp: Improve ResponseRecorder to buffer headers 2019-10-15 14:07:10 -06:00
Matthew Holt acf7dea68f caddyhttp: host labels placeholders endianness from right->left
https://caddy.community/t/labeln-placeholder-endian-issue/5366

(I thought we had this before but it must have gotten lost somewhere)
2019-10-14 12:09:43 -06:00
Pascal bc738991b6 caddyhttp: Support placeholders in MatchHost (#2810)
* Replace global placeholders in host matcher

* caddyhttp: Fix panic on MatchHost tests
2019-10-14 11:29:36 -06:00
yzongyue fcd8869f51 reverse_proxy: optimize MaxIdleConnsPerHost default (#2809) 2019-10-11 23:57:11 -06:00
Matthew Holt 1e31be8de0 reverse_proxy: Allow dynamic backends (closes #990 and #1539)
This PR enables the use of placeholders in an upstream's Dial address.

A Dial address must represent precisely one socket after replacements.

See also #998 and #1639.
2019-10-11 14:25:39 -06:00
Matthew Holt 4aa3af4b78 go.mod: Use latest certmagic which uses lego v3.1.0 2019-10-11 10:48:06 -06:00
Matthew Holt 8715a28320 reverse_proxy: Customize SNI value in upstream request (closes #2483) 2019-10-10 17:17:06 -06:00
Matthew Holt 715e6ddf51 go.mod: Update dependencies 2019-10-10 15:47:26 -06:00
Matthew Holt 9c0bf311f9 Miscellaneous cleanups / comments 2019-10-10 15:38:30 -06:00
Matthew Holt 5300949e0d caddyhttp: Make responseRecorder capable of counting body size 2019-10-10 15:36:28 -06:00
Matthew Holt 411152016e Remove unused/placeholder log handler 2019-10-10 15:35:33 -06:00
Matthew Holt 5c7640a8d9 cmd: Plug in the http.handlers.authentication module 2019-10-10 15:05:33 -06:00
Matthew Holt f8366c2f09 http: authentication module; hash-password cmd; http_basic provider
This implements HTTP basicauth into Caddy 2. The basic auth module will
not work with passwords that are not securely hashed, so a subcommand
hash-password was added to make it convenient to produce those hashes.

Also included is Caddyfile support.

Closes #2747.
2019-10-10 14:37:27 -06:00
Pascal fe36d26b63 caddyhttp: Add RemoteAddr placeholders (#2801)
* Ignore build artifacts

* Add RemoteAddr placeholders
2019-10-10 13:37:08 -06:00
Matt Holt b38365ff3b Merge pull request #2799 from caddyserver/v2-enterprise-merge
v2: Merge enterprise code into open source v2 branch
2019-10-10 11:27:45 -06:00
Matthew Holt 26cc883708 http: Add Starlark handler
This migrates a feature that was previously reserved for enterprise
users, according to #2786.

The Starlark integration needs to be updated since this was made before
some significant changes in the v2 code base. When functional, it makes
it possible to have very dynamic HTTP handlers. This will be a long-term
ongoing project.

Credit to Danny Navarro
2019-10-10 11:02:16 -06:00
Matthew Holt 93943a6ac2 readme: Remove mentions of Caddy Enterprise (as per #2786) 2019-10-09 20:30:21 -06:00
Matthew Holt 85ce15a5ad tls: Add custom certificate selection policy
This migrates a feature that was previously reserved for enterprise
users, according to https://github.com/caddyserver/caddy/issues/2786.

Custom certificate selection policies allow advanced control over which
cert is selected when multiple qualify to satisfy a TLS handshake.
2019-10-09 19:41:45 -06:00
Matthew Holt dedcfd4e3d tls: Add distributed_stek module
This migrates a feature that was previously reserved for enterprise
users, according to https://github.com/caddyserver/caddy/issues/2786.

TLS session ticket keys are sensitive, so they should be rotated on a
regular basis. Only Caddy does this by default. However, a cluster of
servers that rotate keys without synchronization will lose the benefits
of having sessions in the first place if the client is routed to a
different backend. This module coordinates STEK rotation in a fleet so
the same keys are used, and rotated, across the whole cluster. No other
server does this, but Twitter wrote about how they hacked together a
solution a few years ago:
https://blog.twitter.com/engineering/en_us/a/2013/forward-secrecy-at-twitter.html
2019-10-09 19:38:26 -06:00
Matthew Holt 20fe9cf024 tls: Add pem_loader module
This migrates a feature that was previously reserved for enterprise
users, according to https://github.com/caddyserver/caddy/issues/2786.

The PEM loader allows you to embed PEM files (certificates and keys)
directly into your config, rather than requiring them to be stored on
potentially insecure storage, which adds attack vectors. This is useful
in automated settings where sensitive key material is stored only in
memory.

Note that if the config is persisted to disk, that added benefit may go
away, but there will still be the benefit of having lesser dependence on
external files.
2019-10-09 19:34:14 -06:00
Matthew Holt bcbe1c220d reverse_proxy: Add local circuit breaker
This migrates a feature that was previously reserved for enterprise
users, according to https://github.com/caddyserver/caddy/issues/2786.

The local circuit breaker is a simple metrics counter that can cause
the reverse proxy to consider a backend unhealthy before it actually
goes offline, by measuring recent latencies over a sliding window.

Credit to Danny Navarro
2019-10-09 19:28:07 -06:00
Matthew Holt a53b27c62e http: Add work-in-progress cache handler module
This migrates a feature that was previously reserved for enterprise
users, according to https://github.com/caddyserver/caddy/issues/2786.

The cache HTTP handler will be a high-performing, distributed cache
layer for HTTP requests. Right now, the implementation is a very basic
proof-of-concept, and further development is required.
2019-10-09 19:22:46 -06:00
Matthew Holt 03306e646e admin: /config and /id endpoints
This integrates a feature that was previously reserved for enterprise
users, according to https://github.com/caddyserver/caddy/issues/2786.

The /config and /id endpoints make granular config changes possible as
well as the exporting of the current configuration.

The /load endpoint has been modified to wrap the /config handler so that
the currently-running config can always be available for export. The
difference is that /load allows configs of varying formats and converts
them using config adapters. The adapted config is then processed with
/config as JSON. The /config and /id endpoints accept only JSON.
2019-10-09 19:10:00 -06:00
yzongyue 53dd600b4d cmd: Built-in commands all use RegisterCommand (#2794) 2019-10-08 20:12:15 -06:00
Matthew Holt ce1205239a cmd/main: Plug in json5 and jsonc config adapters 2019-10-06 20:48:31 -06:00
Matthew Holt bc3e44c1a6 cmd: adapt: Default --adapter value is "caddyfile" 2019-10-06 20:48:09 -06:00
Matthew Holt 8c55167f71 rewrite: Return parse error if too many Caddyfile args (fixes #2791) 2019-10-06 20:46:10 -06:00
Matthew Holt be7abda7d4 reverse_proxy: Implement retry_match; by default only retry GET requests
See https://caddy.community/t/http-proxy-and-non-get-retries/6304
2019-10-05 16:22:05 -06:00
Matthew Holt 6fd28b81dc caddyhttp: Define MatcherSets and RawMatcherSets types 2019-10-05 16:20:07 -06:00
Matthew Holt 65c060f56e file_server: Set default address to :2015 if --listen not specified 2019-10-04 17:30:51 -06:00
Matthew Holt 44cb804b9e reverse_proxy: Configurable request headers on active health checks
See https://caddy.community/t/health-check-user-agent/6309
2019-10-04 17:21:38 -06:00
Matthew Holt c11e3bffd6 Add file-server and reverse-proxy subcommands 2019-10-03 16:00:41 -06:00
Matthew Holt f29a9eee0d caddytls: nil check on storageClean fields on Stop 2019-10-02 23:39:32 -06:00
Matthew Holt 370b78c5c7 Update CLI docs in README 2019-10-01 20:45:31 -06:00
Mohammed Al Sahaf 1ecb216001 v2: introduce CI (#2768)
* v2: introduce CI for v2 branch

* v2-ci: split test report generation from test pass to preserve exit code

* v2-ci: spilt lint results from unit test results

* v2-ci: fix testRunTitle name

* v2-ci: break up the steps for more accurate status indicators

* v2-ci: break steps into different jobs

* v2-ci: revert back to single-job pattern

* v2-ci: reflect the true result by coercing SucceededWithIssues into Failed in the last step

* v2-ci: don't fail the build on lint errors
2019-10-01 16:47:29 -06:00
Matthew Holt 94f98c0733 go.mod: Use latest certmagic 2019-10-01 11:25:52 -06:00
Matthew Holt 2c3657bb8a cmd: CLI improvements; add --validate to adapt command 2019-10-01 11:02:13 -06:00
Matthew Holt 5b36424cf0 cmd: Add validate subcommand; list-modules --versions; some renaming
Renames --config-adapter flag to --adapter, adapt-config command to
adapt, --print-env flag to --environ, and --input flag to --config.
2019-09-30 23:43:39 -06:00
aca 0006df6026 cmd: Refactor subcommands, add help, make them pluggable
* cli: Change command structure, add help subcommand (#328)

* cli: improve subcommand structure

- make help command as normal subcommand
- add flag usage message for each command

* cmd: Refactor subcommands and command line help; make commands pluggable
2019-09-30 21:23:58 -06:00
Matthew Holt c95db3551d caddytls: Ensure automation field is not nil when appending (fix #2779) 2019-09-30 11:53:21 -06:00
Matthew Holt 8eb2c37251 Clean up provisioned modules on error; refactor Run(); add Validate()
Modules that return an error during provisioning should still be cleaned
up so that they don't leak any resources they may have allocated before
the error occurred. Cleanup should be able to run even if Provision does
not complete fully.
2019-09-30 09:16:01 -06:00
Matthew Holt 1e66226217 httpcaddyfile: Add acme_ca and email global options
Also add ability to access options from individual unmarshalers through
the Helper values
2019-09-30 09:11:30 -06:00
Matthew Holt 7b4aa108c7 caddyhttp: 'not' matcher: Support Caddyfile unmarshaling 2019-09-30 09:09:57 -06:00
Matthew Holt 8b11ed347b Add license header to filestorage.go 2019-09-30 09:08:04 -06:00
Matthew Holt b249b45d10 tls: Change struct fields to pointers, add nil checks; rate.Burst update
Making them pointers makes for cleaner JSON when adapting configs, if
the struct is empty now it will be omitted entirely.

The x/time/rate package was updated to support changing the burst, so
we've incorporated that here and removed a TODO.
2019-09-30 09:07:43 -06:00
Matthew Holt c12bf4054c caddyfile: Fix lexer behavior with regards to escaped newlines
Newlines (\n) can be escaped outside of quoted areas and the newline
will be treated as whitespace but not as an actual line break. Escaping
newlines inside a quoted area is not necessary, and because quotes
trigger literal interpretation of the contents, the escaping backslash
will be parsed as a literal backslash, and the newline will not be
escaped.

Caveat: When a newline is escaped, tokens after it until an unescaped
newline will appear to the parser be on the same line as the initial
token after the last unescaped newline. This may technically lead to
some false line numbers if errors are given, but escaped newlines are
counted so that the next token after an unescaped newline is correct.

See #2766
2019-09-28 21:18:36 -06:00
Matthew Holt 735d6ce405 httpcaddyfile: Fix missing module name of storage adapter 2019-09-26 17:06:15 -07:00
Matthew Holt 7b33c8db31 tls: Make cert and OCSP check intervals configurable
This enables use of ACME CAs that issue shorter-lived certs
2019-09-24 17:04:03 -07:00
Matt Holt 11696793bd tls/acme: Ability to customize trusted roots for ACME servers (#2756)
Closes #2702
2019-09-24 15:46:39 -07:00
Matthew Holt 3e8bff594a go.mod: Update certmagic to v0.7.3 2019-09-20 13:17:17 -06:00
Matthew Holt 2f684e42d5 reverse_proxy/headers: Expose header replacement ability in Caddyfile
Adds header_up and header_down subdirectives to reverse_proxy
2019-09-20 13:13:49 -06:00
Matthew Holt ba29f9d41d httpcaddyfile: Global storage configuration (closes #2758) 2019-09-19 12:42:36 -06:00
Matthew Holt 40e05e5a01 http: Improve auto HTTP->HTTPS redirects, fix edge cases
See https://caddy.community/t/v2-issues-with-multiple-server-blocks-in-caddyfile-style-config/6206/13?u=matt

Also print pid when using `caddy start`
2019-09-18 18:01:32 -06:00
Matthew Holt 39d61cad2d httpcaddyfile: Fix nil pointer dereference 2019-09-18 10:51:49 -06:00
Matthew Holt bc9f944837 host matcher: Strip [ ] from IPv6 addresses 2019-09-18 09:45:21 -06:00
Matthew Holt 4c289fc6ad Allow domain fronting with TLS client auth if explicitly configured 2019-09-17 23:13:21 -06:00
Matthew Holt 19f36667f7 tls: Clean up expired OCSP staples and certificates 2019-09-17 16:00:15 -06:00
Matt Holt 484cee1ac1 fastcgi: Implement / redirect for index.php with php_fastcgi directive (#2754)
* fastcgi: Implement / redirect for index.php with php_fastcgi directive

See #2752 and https://caddy.community/t/v2-redirect-path-to-path-index-php-with-assets/6196?u=matt

* caddyhttp: MatchNegate implements json.Marshaler

* fastcgi: Add /index.php element to try_files matcher

* fastcgi: Make /index.php redirect permanent
2019-09-17 15:16:17 -06:00
Matthew Holt d030bfdae0 httpcaddyfile: static_response -> respond; minor cleanups 2019-09-16 11:04:18 -06:00
Matthew Holt db4c73dd58 reverse_proxy: Close idle connections on module unload 2019-09-14 18:10:29 -06:00
Matthew Holt f15f0d5839 Eliminate some TODOs 2019-09-14 18:05:45 -06:00
Matthew Holt e73b117332 reverse_proxy: Ability to mutate headers; set upstream placeholders 2019-09-14 13:25:26 -06:00
Matthew Holt 2fd22139c6 headers: Ability to mutate request headers including http.Request.Host
Also a few bug fixes
2019-09-14 13:22:48 -06:00
Mohammed Al Sahaf 5c9ebe3af1 Use keybase fork of mitchellh/go-ps for bug fixes (#2750) 2019-09-13 23:40:29 -06:00
Matthew Holt 2ab2d5bf9e Forgot to commit caddyfile.go changes in last commit 2019-09-13 23:38:52 -06:00
Matthew Holt c09e86fddc headers: Add ability to replace substrings in header fields
This will probably be useful so the proxy can rewrite header values.
2019-09-13 16:24:51 -06:00
Matthew Holt 46aaf02371 encode: Fix bug where default status code was being written
for small responses.

See https://caddy.community/t/v2-permanent-redirect-prompt/6190?u=matt
2019-09-13 16:00:03 -06:00
Matthew Holt 3b80c505fb Update v2 readme in prep for beta1 2019-09-13 12:50:06 -06:00
Matthew Holt 1d1e194229 Hard-code 'main' module name until bug upstream in Go modules is fixed
See https://github.com/golang/go/issues/29228
2019-09-13 12:43:28 -06:00
Matthew Holt 839507e24e http: Consider wildcards when evaluating automatic HTTPS 2019-09-13 11:46:58 -06:00
Matthew Holt 833d67446f admin: Allow listening on unix socket (closes #2749) 2019-09-13 11:24:07 -06:00
Matthew Holt d0c1756fc5 httpcaddyfile: Fix tls certificate loader module names (#2748) 2019-09-13 09:45:10 -06:00
Matthew Holt ed40a5dcab tls: Do away with SetDefaults which did nothing useful
CertMagic uses the same defaults for us
2019-09-12 17:31:54 -06:00
Matthew Holt 7799554baa go.mod: Use lego v3 and CertMagic 0.7.0 2019-09-12 17:31:10 -06:00
Matthew Holt 2cb01d43cf tls: Remove support for TLS 1.0 and TLS 1.1 2019-09-11 22:26:06 -06:00
Matthew Holt 758269124e reverseproxy: Fix host and port on requests; fix Caddyfile parser 2019-09-11 18:53:44 -06:00
Matthew Holt b4dce74e59 tls: Use Let's Encrypt production endpoint
We're done testing this in staging
2019-09-11 18:52:07 -06:00
Matthew Holt fe389fcbd7 http: Set Alt-Svc header if experimental HTTP3 server is enabled 2019-09-11 18:49:21 -06:00
Matthew Holt 005a11cf4b headers: New 'request_header' directive; handle Host header specially
Before this change, only response headers could be manipulated with the
Caddyfile's 'header' directive.

Also handle the request Host header specially, since the Go standard
library treats it separately from the other header fields...
2019-09-11 18:48:37 -06:00
Matthew Holt 194df652eb reverseproxy: Add 'tls' option to enable HTTPS with HTTP transport 2019-09-11 18:46:32 -06:00
Matthew Holt 53bbdf1766 httpcaddyfile: Add 'experimental_http3' option 2019-09-11 17:16:21 -06:00
Matthew Holt e48d83452e httpcaddyfile: Switch order; reverse_proxy comes before php_fastcgi 2019-09-11 12:02:35 -06:00
Matthew Holt 2459c292a4 caddyfile: Improve Dispenser.NextBlock() to support nesting 2019-09-10 19:21:52 -06:00
Matthew Holt 0cf592fa2e New 'php_fastcgi' directive for convenient PHP+FastCGI reverse proxy 2019-09-10 14:16:41 -06:00
Matthew Holt d9136fb0a0 rewrite: Caddyfile directive should always invoke a rehandle
This is unless each route's matcher is dynamically executed after
previous handlers...
2019-09-10 14:13:52 -06:00
Matthew Holt c32b7e8865 fastcgi: Make EnvVars a map instead of a slice 2019-09-10 14:12:51 -06:00
Matthew Holt 1ce10b453f Require Go 1.13; use Go 1.13's default support for TLS 1.3 2019-09-10 13:11:27 -06:00
Matt Holt 0c8ad52be1 Experimental IETF-standard HTTP/3 support (known issue exists) (#2727)
* Begin WIP integration of HTTP/3 support

* http3: Set actual Handler, make fakeClosePacketConn type for UDP sockets

Also use latest quic-go for ALPN fix

* Manually keep track of and close HTTP/3 listeners

* Update quic-go after working through some http3 bugs

* Fix go mod

* Make http3 optional for now
2019-09-10 08:03:37 -06:00
Matthew Holt d67d8cf5a8 Fix build (sigh) 2019-09-10 07:15:36 -06:00
Matt Holt 44b7ce9850 Merge pull request #2737 from caddyserver/fastcgi (reverse proxy!)
v2: Refactor reverse proxy and add FastCGI support
2019-09-09 21:46:21 -06:00
Matthew Holt b4f4fcd437 Migrate some selection policy tests over to v2 2019-09-09 21:44:58 -06:00
Matthew Holt 50e62d06bc reverse_proxy: Caddyfile integration (and fix blocks in Dispenser) 2019-09-09 12:23:27 -06:00
Matthew Holt 9169cd43d4 Log when auto HTTPS or auto HTTP->HTTPS redirects are disabled 2019-09-09 08:25:48 -06:00
Matthew Holt e12c62e60b file_server: Enforce URL canonicalization (closes #2741) 2019-09-09 08:21:45 -06:00
Ingo Gottwald 3e9e7555ef Fix build (#2740)
Build was broken with commit 50961ec.
2019-09-07 14:25:04 -06:00
Matthew Holt f6126acf37 Header matchers: allow matching presence of header with empty list 2019-09-06 14:25:16 -06:00
Matthew Holt 97ace2a39e File matcher enforces trailing-slash convention to match dirs/files 2019-09-06 13:32:02 -06:00
Matthew Holt 4bd9496525 Fix Schrodinger's file existence check in file matcher
See: https://stackoverflow.com/a/12518877/1048862

For example, trying to check the existence of "/www/index.php/index.php"
fails but not with an os.IsNotExist()-type error. So we have to assume
that a file that cannot be successfully stat'ed at all does not exist.
2019-09-06 12:57:12 -06:00
Matthew Holt 14f9662f9c Various fixes/tweaks to HTTP placeholder variables and file matching
- Rename http.var.* -> http.vars.* to be more consistent
- Prefixing a path matcher with * now invokes simple suffix matching
- Handlers and matchers that need a root path default to {http.vars.root}
- Clean replacer output on the file matcher's file selection suffix
2019-09-06 12:36:45 -06:00
Matthew Holt 21d7b662e7 fastcgi: Use request context as base, not a new one 2019-09-06 12:02:11 -06:00
Matthew Holt 3ba9e143a2 cli: Fix run and start when no config file is available 2019-09-05 14:59:19 -06:00
Matthew Holt d2e46c2be0 fastcgi: Set default root path; add interface guards 2019-09-05 13:42:20 -06:00
Matthew Holt 80b54f3b9d Add original URI to request context; implement into fastcgi env 2019-09-05 13:36:42 -06:00
Matthew Holt 0830fbad03 Reconcile upstream dial addresses and request host/URL information
My goodness that was complicated

Blessed be request.Context

Sort of
2019-09-05 13:14:39 -06:00
Matthew Holt a60d54dbfd reverse_proxy: Ignore context.Canceled errors
These happen when downstream clients cancel the request, but that's not
our problem nor a failure in our end
2019-09-03 19:10:09 -06:00
Matthew Holt acb8f0e0c2 Integrate circuit breaker modules with reverse proxy 2019-09-03 19:06:54 -06:00
Matthew Holt 652460e03e Some cleanup and godoc 2019-09-03 16:56:09 -06:00
Matthew Holt 4a1e1649bc reverse_proxy: Implement remaining TLS config for proxy to backend 2019-09-03 15:26:09 -06:00
Matthew Holt ccfb12347b reverse_proxy: Implement active health checks 2019-09-03 12:10:11 -06:00
Alexandre Stein 50961ecc77 Initial implementation of TLS client authentication (#2731)
* Add support for client TLS authentication

Signed-off-by: Alexandre Stein <alexandre_stein@interlab-net.com>

* make and use client authentication struct

* force StrictSNIHost if TLSConnPolicies is not empty

* Implement leafs verification

* Fixes issue when using multiple verification

* applies the comments from maintainers

* Apply comment

* Refactor/cleanup initial TLS client auth implementation
2019-09-03 09:35:36 -06:00
Matthew Holt 026df7c5cb reverse_proxy: WIP refactor and support for FastCGI 2019-09-02 22:01:02 -06:00
Matthew Holt 8e821b5039 caddyconfig: Add JSON5 and JSON-C adapters (closes #2735) 2019-09-02 12:21:41 -06:00
Matthew Holt 9d8bff28c2 oops, also update the Caddyfile's {query} var to use query_string 2019-08-27 14:41:57 -06:00
Matthew Holt d242f10eda Add query_string to HTTP replacer and use it for try_files 2019-08-27 14:38:24 -06:00
Ariel Núñez 2dc4fcc62b Fix caddyconfig import in admin.go (#2725) 2019-08-23 10:57:51 -06:00
Matthew Holt afd154119a admin: Support config adapters at /load endpoint
Based on Content-Type
2019-08-22 14:52:39 -06:00
Matthew Holt e34ff21a71 caddyfile: Allow handler order to be customized 2019-08-22 14:26:33 -06:00
Matthew Holt af25f0254e caddyfile: Support global config block; allow non-empty blocks w/ 0 keys 2019-08-22 13:38:37 -06:00
Mohammed Al Sahaf a0fd2b6c0a Fix SIV where /v2 was missing from caddyfile adapter work (#2721) 2019-08-22 12:26:48 -06:00
Matthew Holt c0da7d487a file_server: Automatically hide all involved Caddyfiles 2019-08-21 15:50:02 -06:00
Matthew Holt 8420a2f250 Clean up Dispenser and filename handling a bit 2019-08-21 15:23:00 -06:00
Matthew Holt 59910923d1 Update readme for v2 caddyfile and config adapters 2019-08-21 12:31:58 -06:00
Matt Holt 0544f0266a Merge pull request #2699 from caddyserver/cfadapter
v2: Implement config adapters and WIP Caddyfile adapter
2019-08-21 11:28:03 -06:00
Matthew Holt b2aa679c33 Fix snippet nesting bug 2019-08-21 11:26:48 -06:00
Matthew Holt fa334c4bdf Implement some shorthand placeholders for Caddyfile 2019-08-21 11:03:50 -06:00
Matthew Holt d73b650c26 Update go.mod 2019-08-21 10:47:09 -06:00
Matthew Holt c9980fd367 Refactor Caddyfile adapter and module registration
Use piles from which to draw config values.

Module values can return their name, so now we can do two-way mapping
from value to name and name to value; whereas before we could only map
name to value. This was problematic with the Caddyfile adapter since
it receives values and needs to know the name to put in the config.
2019-08-21 10:46:35 -06:00
Albert Shirima 42f75a4ca9 Fixing a compilation error (#2712)
./caddy.go:230:12: cannot use *dep (type debug.Module) as type *debug.Module in return argument
./caddy.go:233:12: cannot use bi.Main (type debug.Module) as type *debug.Module in return argument
2019-08-17 19:14:55 -06:00
Matthew Holt c4159ef76d Fix module-related errors 2019-08-09 12:19:56 -06:00
Matthew Holt ab885f07b8 Implement config adapters and beginning of Caddyfile adapter
Along with several other changes, such as renaming caddyhttp.ServerRoute
to caddyhttp.Route, exporting some types that were not exported before,
and tweaking the caddytls TLS values to be more consistent.

Notably, we also now disable automatic cert management for names which
already have a cert (manually) loaded into the cache. These names no
longer need to be specified in the "skip_certificates" field of the
automatic HTTPS config, because they will be skipped automatically.
2019-08-09 12:05:47 -06:00
Dominik Braun 4950ce485f Part 1: Optimize using compiler's inliner (#2687)
* optimized functions for inlining

* added note regarding ResponseWriterWrapper

* optimzed browseWrite* methods for FileServer

* created benchmarks for comparison

* creating browseListing instance in each function

* created benchmarks for openResponseWriter

* removed benchmarks of old implementations

* implemented sync.Pool for byte buffers

* using global sync.Pool for writing JSON/HTML
2019-08-07 23:59:02 -06:00
Dreamacro c8b0a97b1c Add missing imports (#2688) 2019-07-24 01:28:33 -06:00
Johannes Hörmann 95a447de9c Tests for replacer (#2675)
* Tests for Replacer: Replacer.Set and Replacer.Delete

* update replacer test to new implementation

* fix replacer: counted position wrong if placeholder was found

* fix replacer: found placeholder again, if it was a non-existing one

* test with spaces between the placeholders as this could have a different behaviour

* Tests for Replacer.Map

* Tests for Replacer.Set: check also for something like {l{test1}
This should be replaced as {lTEST1REPLACEMENT

* fix replacer: fix multiple occurrence of phOpen sign

* Tests for Replacer: rewrite Set and ReplaceAll tests to use implementation not interface

* Tests for Replacer: rewrite Delete test to use implementation not interface

* Tests for Replacer: rewrite Map tests to use implementation not interface

* Tests for Replacer: add test for NewReplacer

* Tests for Replacer: add test for default replacements

* Tests for Replacer: fixed and refactored tests

* Tests for Replacer: moved default replacement tests to New-test
as new should return a replace with provider which defines global replacements
2019-07-21 09:57:34 -06:00
Toby Allen d98f2faef9 Add /stop endpoint to admin (#2671)
* Add stop command to admin.  Exit after stop.

* Return error on incorrect http Method and provide better logging.

* reuse stopAndCleanup function for all graceful stops
2019-07-20 10:48:46 -06:00
Toby Allen b855e66170 Force quit on Windows with taskkill /f (#2670)
* Force quit /f on windows, also check for processname '.exe' on windows.

* Remove unneeded spaces

* fix tabs

* go fmt tabs

* Return consistent appname which always includes .exe

* Change func name
2019-07-20 10:44:54 -06:00
Matthew Holt 0d3f99e85a cmd: Add print-env flag to run command 2019-07-18 10:58:31 -06:00
Matthew Holt 28df6cedfe tls: Use IANA-standard cipher suite names 2019-07-18 09:52:43 -06:00
Matthew Holt dd6aa91d72 Fix DNS provider module unmarshaling (closes #2676) 2019-07-18 09:15:23 -06:00
Matt Holt b44a22a9d4 Performance improvements to Replacer implementation (placeholders) (#2674)
Closes #2673
2019-07-16 12:27:11 -06:00
Matthew Holt bdf92ee84e Minor tweaks 2019-07-15 17:33:47 -06:00
Matthew Holt f217181293 mod: Use blackfriday's standard v2 module import path 2019-07-15 17:33:08 -06:00
Matthew Holt ccb5d19c25 Get module name at runtime, and tidy up modules 2019-07-12 10:15:27 -06:00
Matthew Holt b780f0f49b Standardize exit codes and improve shutdown handling; update gitignore 2019-07-12 10:07:11 -06:00
Matthew Holt 2141626269 Fix readme example for updated handler structure 2019-07-12 08:53:02 -06:00
Matthew Holt 63674ba081 Rename handler modules to use http.handlers namespace 2019-07-11 22:03:12 -06:00
Matthew Holt 9722dbe18a Fix rehandling bug 2019-07-11 22:02:47 -06:00
Matthew Holt 4698352b20 Merge branch 'v2-handlers' into v2
# Conflicts:
#	modules/caddyhttp/caddyhttp.go
#	modules/caddyhttp/fileserver/staticfiles.go
#	modules/caddyhttp/routes.go
#	modules/caddyhttp/server.go
#	modules/caddyhttp/staticresp.go
#	modules/caddyhttp/staticresp_test.go
2019-07-11 17:07:52 -06:00
Matthew Holt eb8625f774 Add error & subroute handlers; weakString; other minor handler changes 2019-07-11 17:02:57 -06:00
Matt Holt 9343403358 Flatten HTTP handler config (#2662) (#2663)
Differentiating middleware and responders has one benefit, namely that
it's clear which module provides the response, but even then it's not
a great advantage. Linear handler config makes a little more sense,
giving greater flexibility and simplifying the core a bit, even though
it's slightly awkward that handlers which are responders may not use
the 'next' handler that is passed in at all.
2019-07-11 15:32:34 -06:00
Matthew Holt 4a3a418156 Flatten HTTP handler config (#2662)
Differentiating middleware and responders has one benefit, namely that
it's clear which module provides the response, but even then it's not
a great advantage. Linear handler config makes a little more sense,
giving greater flexibility and simplifying the core a bit, even though
it's slightly awkward that handlers which are responders may not use
the 'next' handler that is passed in at all.
2019-07-09 12:58:39 -06:00
Matthew Holt 6dfba5fda8 Add path components to HTTP replacer 2019-07-08 16:46:55 -06:00
Matthew Holt d25008d2c8 Move listen address functions into caddy package; fix unix bug 2019-07-08 16:46:38 -06:00
Matthew Holt 4eb5fc541b Better error handling in CLI commands 2019-07-07 16:39:21 -06:00
Matthew Holt 42acdad9e5 Fix error handling with Validate when loading modules (fixes #2658)
The return statement was improperly nested in context.go
2019-07-07 14:12:22 -06:00
Matthew Holt 84f9f7cd60 Little cleanups 2019-07-05 13:59:30 -06:00
Matthew Holt 79216d356c acmemanager: Use storage module key "module" instead of "system" 2019-07-05 09:59:46 -06:00
Matthew Holt 9429c843c8 cmd: New reload command 2019-07-05 09:59:13 -06:00
Matthew Holt 6bcba91fbe Lowercase env var names in replacer 2019-07-03 15:42:21 -06:00
Matthew Holt ab101d75d0 Update readme docs 2019-07-03 14:50:59 -06:00
Matthew Holt 7512ea1a64 Change storage module key from "system" to "module" 2019-07-03 10:40:25 -06:00
Matthew Holt 902ec37062 Minor improvements to readme 2019-07-02 21:00:49 -06:00
Matthew Holt bed05f2450 Fix links in readme 2019-07-02 16:18:35 -06:00
Matthew Holt fdd871e177 go.mod: Append /v2 to module name; update all import paths
See https://github.com/golang/go/wiki/Modules#semantic-import-versioning
2019-07-02 12:37:06 -06:00
Matthew Holt 94c28a2574 Fix README typo, sigh... 2019-07-02 12:29:38 -06:00
Matthew Holt 42386a7272 Add menu and list of improvements to readme 2019-07-02 12:13:09 -06:00
Matthew Holt 5e858a15f7 Add a proper readme 2019-07-01 18:08:56 -06:00
Matthew Holt 533d1afb4b tls: Enable TLS 1.3 by default; set sane defaults on tls.Config structs 2019-07-01 11:47:46 -06:00
Matthew Holt 9f8d3611eb encode: Add "Vary" response header 2019-06-30 23:38:36 -06:00
Matthew Holt 3177ee8010 Add license 2019-06-30 16:07:58 -06:00
Matthew Holt 7a7c5f00c0 Add authors file 2019-06-30 16:06:24 -06:00
Matthew Holt fee0b38b48 Fix encoder name bug; remove unused field in encode middleware struct 2019-06-29 16:57:55 -06:00
Matthew Holt d5ae3a4966 httpserver: Set default Server header 2019-06-28 19:28:47 -06:00
Matthew Holt 31ab737bf2 Refactor code related to getting current version
And set version in CertMagic for User-Agent purposes
2019-06-28 19:28:28 -06:00
Matthew Holt a4bdf249db Caddy 2 gets a CLI! And admin endpoint is now configurable via JSON 2019-06-28 15:39:41 -06:00
Matthew Holt 006dc1792f Use html/template for escaping by default
Allow HTML only with a few specific functions
2019-06-27 13:30:41 -06:00
Matthew Holt a63cb3e3fd Implement etag; fix related bugs in encode and templates middlewares 2019-06-27 13:09:10 -06:00
Matthew Holt 2b22d2e6ea Optionally enforce strict TLS SNI + HTTP Host matching, & misc. cleanup
We should look into a way to enable this by default when TLS client auth
is configured for a server
2019-06-26 16:03:29 -06:00
Matthew Holt a524bcfe78 Enable skipping just certificate management for some auto HTTPS names 2019-06-26 10:57:18 -06:00
Matthew Holt 91b03dccb0 Refactor automatic HTTPS configuration; ability to skip certain names 2019-06-26 10:49:32 -06:00
Matthew Holt 6000855c82 Fix panics by disallowing explicitly-defined null modules in config 2019-06-26 10:45:34 -06:00
Matthew Holt 38677aaa58 caddytls: Support tags for manually-loaded certificates 2019-06-24 12:16:10 -06:00
Matthew Holt d49f762f6d Various bug fixes and minor improvements
- Fix static responder so it doesn't replace its own headers config,
  and instead replaces the actual response header values
- caddyhttp.ResponseRecorder type optionally buffers response
- Add interface guards to ensure regexp matchers get provisioned
- Use default HTTP port if one is not explicitly set
- Encode middleware writes status code 200 if not written upstream
- Templates and markdown only try to execute on text responses
- Static file server sets Content-Type based on file extension only
  (this whole thing -- MIME sniffing, etc -- needs more configurability)
2019-06-21 14:36:26 -06:00
Matthew Holt 81a9e125b5 Oops 2019-06-21 08:52:15 -06:00
Matthew Holt 70c788ce0c Minor cleanups/improvements 2019-06-21 08:08:26 -06:00
Matthew Holt 1c443beb9c caddyhttp: ResponseRecorder type for middlewares to buffer responses
Unfortunately, templates and markdown require buffering the full
response before it can be processed and written to the client
2019-06-20 21:49:45 -06:00
Matthew Holt 269b1e9aa3 tls: Improve (and fix) on-demand configuration 2019-06-20 20:36:29 -06:00
Matthew Holt 6d0350d04e caddyhttp: Fix host matching when host has a port 2019-06-20 20:24:46 -06:00
Matthew Holt 15647bdfb7 templates: Remove context functions implemented by sprig 2019-06-18 15:43:51 -06:00
Matthew Holt 2663dd176d Refactor templates execution; add sprig functions 2019-06-18 15:17:48 -06:00
Matthew Holt 6706c9225a Implement templates handler; various minor cleanups and bug fixes 2019-06-18 11:13:12 -06:00
Matthew Holt 5137859e47 Rename caddy2 -> caddy
Removes the version from the package name
2019-06-14 11:58:28 -06:00
Matthew Holt b8e7453fef Implement brotli encoder; improve validation of other encoders 2019-06-13 11:20:43 -06:00
Matthew Holt f93dab755b Update go modules 2019-06-13 10:55:25 -06:00
Matthew Holt 0c8763a728 Add simple tests for static responder 2019-06-11 17:46:11 -06:00
Matt Holt f5b4f268dc Implement encode middleware (#2)
* Implement encode middleware

* Add missing break; and add missing JSON struct field tag
2019-06-10 10:21:25 -06:00
Matthew Holt ef5f29cfb2 Do not allow Go standard lib to sniff Content-Type header 2019-06-07 19:59:25 -06:00
Matt Holt 8947ae0cc1 Merge pull request #1 from caddyserver/fix/goroutine-leak-healthchecker
fix goroutine leak in healthcheckers
2019-06-07 17:24:10 -06:00
dev 878ae0002a fix goroutine leak in healthcheckers 2019-06-07 15:52:10 -04:00
dev 37da91cfe7 fix module import paths and add cors to admin endpoints
fix go module refs and add cors to admin endpoints
2019-06-07 11:40:25 -04:00
Matthew Holt b79f86f256 Fix bugs related to auto HTTPS and alternate port configurations 2019-06-04 22:43:21 -06:00
Matthew Holt 613aecb898 Change import paths to GitHub package names 2019-06-04 13:52:37 -06:00
Matthew Holt 39db06d9c4 Implement IP/CIDR matcher and Not (negated) matcher 2019-06-04 13:42:54 -06:00
Matthew Holt f064889a4f Customize admin endpoint address with -listen flag
This is a temporary holdover for development purposes
2019-06-03 15:35:14 -06:00
Matthew Holt 3439933235 Implement session ticket keys; default STEK module with rotation 2019-05-29 23:11:46 -06:00
Matthew Holt 1b6b422c63 Add cleanup callbacks to context 2019-05-29 23:10:12 -06:00
Matthew Holt 2265db9028 Fix bug unmarshaling custom duration values 2019-05-29 23:09:51 -06:00
Matthew Holt bf54615efc ResponseMatcher for conditional logic of response headers 2019-05-28 18:53:08 -06:00
Matthew Holt da6a8cfc86 Minor cleanups 2019-05-28 18:52:21 -06:00
Matthew Holt 9cd6f35e9d Separate out certificate selection 2019-05-27 11:31:47 -06:00
Matthew Holt 210d0cf7f1 Implement custom cert selection policies; optimize matching for SNI 2019-05-24 13:18:45 -06:00
Matthew Holt 5a4a1421de Fix error handling and matching catch-all routes 2019-05-23 14:42:14 -06:00
Matthew Holt 34a25dd558 Add very simple markdown middleware for now 2019-05-23 14:41:43 -06:00
Matthew Holt 9e576c76e7 Add request_body middleware and some limits to HTTP servers 2019-05-23 13:16:34 -06:00
Matthew Holt c24a3e389f Change admin listener to :1234 for now; output message when listening 2019-05-22 19:10:29 -06:00
Matthew Holt f976451d19 Disallow unknown fields (strict unmarshal) when loading modules
This makes it faster and easier to detect broken configurations, but
is a slight performance hit on config loads since we have to re-encode
the decoded struct back into JSON without the module name's key
2019-05-22 14:32:12 -06:00
Matthew Holt 869fbac632 Don't use auto HTTPS for servers with only HTTP port listeners 2019-05-22 14:14:26 -06:00
Matthew Holt 284fb3a98c Allow multiple matcher sets in routes (OR'ed together)
Also export MatchRegexp in case other matcher modules find it useful.
Add comments to the exported matchers.
2019-05-22 13:13:39 -06:00
Matthew Holt bc00d840e8 Export types and fields necessary to build configs (for config adapters)
Also flag most fields with 'omitempty' for JSON marshaling
2019-05-22 12:32:36 -06:00
Matthew Holt be9b6e7b57 Honor the configured CA value 2019-05-21 14:22:33 -06:00
Matthew Holt 2fd98cb040 Module.New() does not need to return an error 2019-05-21 14:22:21 -06:00
Matthew Holt 67d32e6779 Fix up matchers tests and take care of TODO in rewrite 2019-05-21 13:10:14 -06:00
Matthew Holt 9d54f655aa Take care of remaining TODOs in the browse responder 2019-05-21 13:03:52 -06:00
Matthew Holt 65195a726d Implement rewrite middleware; fix middleware stack bugs 2019-05-20 23:48:43 -06:00
Matthew Holt b84cb05848 Fix deferred header ops 2019-05-20 22:00:54 -06:00
Matthew Holt a969872850 Default error handler; rename StaticFiles -> FileServer 2019-05-20 21:21:33 -06:00
Matthew Holt aaacab1bc3 Sanitize paths in static file server; some cleanup
Also remove AutomaticHTTPSError for now
2019-05-20 17:15:38 -06:00
Matthew Holt d22f64e6d4 Implement headers middleware 2019-05-20 15:46:52 -06:00
Matthew Holt 22995e5655 Implement most of browse; fix a couple obvious bugs; some cleanup 2019-05-20 15:46:52 -06:00
dev 043eb1d9e5 move internal packages to pkg folder and update reverse proxy
* set automatic https error type for cert-magic failures
* add state to onload and unload methods
* update reverse proxy to use Provision() and Cleanup()
2019-05-20 14:48:26 -04:00
Matthew Holt fec7fa8bfd Implement most of static file server; refactor and improve Replacer 2019-05-20 10:59:20 -06:00
Matthew Holt 1a20fe330e Improve godoc for contexts 2019-05-17 08:48:12 -06:00
Matthew Holt 1f0c061ce3 Architectural shift to using context for config and module state 2019-05-16 16:05:38 -06:00
Matthew Holt ff5b4639d5 Some minor updates, and get rid of OnLoad/OnUnload 2019-05-16 11:46:17 -06:00
Matthew Holt f9d93ead4e Rename and export some types, other minor changes 2019-05-14 14:14:05 -06:00
Matthew Holt 8ae0d6a509 caddyhttp: Implement better HTTP matchers including regexp; add tests 2019-05-10 21:07:02 -06:00
Matthew Holt 48b5a80320 Remove (unimplemented) enterprise TLS matchers 2019-05-07 11:58:58 -06:00
Matthew Holt ad3d408067 Add some tests and fix vet warning 2019-05-07 10:15:46 -06:00
Matthew Holt e40bbecb16 Rough implementation of auto HTTP->HTTPS redirects
Also added GracePeriod for server shutdowns
2019-05-07 09:56:18 -06:00
dev 8eba582efe Add go module files 2019-05-06 17:26:05 -04:00
Matthew Holt fbea3374e9 Add missing run.go (oops) 2019-05-06 12:43:04 -06:00
Matthew Holt 2eb3593327 Begin implementing HTTP replacer and static responder 2019-05-04 13:21:20 -06:00
Matthew Holt 1136e2cfee Add reverse proxy 2019-05-04 10:49:50 -06:00
Matthew Holt 5859cd8dad Instantiate apps that are needed but not explicitly configured 2019-04-29 09:22:00 -06:00
Matthew Holt 43961b542b General cleanup and more godocs 2019-04-26 12:35:39 -06:00
Matthew Holt 2d056fbe66 Initial commit of Storage, TLS, and automatic HTTPS implementations 2019-04-25 13:54:48 -06:00
Matthew Holt 545f28008e Begin implementing error handling and re-handling 2019-04-11 20:42:55 -06:00
dev d42529348f Updated proxy module import 2019-04-08 16:25:27 -04:00
dev 27ecc7f384 Protocol and Caddyscript matchers
* Added matcher to determine what protocol the request is being made by
  - grpc, tls, http
* Added ability to run caddyscript in a matcher to evaluate the http request
* Added TLS field to caddyscript request time
* Added a library to manipulate and compare a new caddyscript time type
* Library for regex in starlark
2019-04-08 09:58:11 -04:00
Matthew Holt 402f423693 Implement "global" state for modules, OnLoad and OnUnload callbacks
Tested for memory leaks and performance. Obviously the added locking and
global state is not awesome, but the alternative is a little uglier IMO:
we'd have to make some sort of "liaison" value which stores the state,
then pass it around to every module, and so LoadModule becomes a lot
less accessible, and each module would need to maintain a reference to
it... nope, just ugly. I think this is the cleaner solution: just make
sure only one Start() happens at a time, and keep global things global.

Very simple log middleware is an example.

Might need to reorder the operations in Start() and handle errors
differently, etc. Otherwise, I'm mostly happy with this solution...
2019-04-08 00:00:14 -06:00
Matthew Holt 3eae6d43b6 Add Validator interface
Modules can now verify their own configurations
2019-04-03 11:41:36 -06:00
Matthew Holt 59a5d0db28 Close listeners which are no longer used 2019-04-02 15:31:02 -06:00
Matt Holt f976aa7443 Merged in deadlines (pull request #1)
Cleanly fake-close listeners

* WIP debugging listener deadlines

* Fix listener deadlines
2019-04-02 20:58:24 +00:00
Matthew Holt 6621406fa8 Very basic middleware and route matching functionality 2019-03-31 20:41:29 -06:00
Matthew Holt 27ff6aeccb Fix goroutine leak in Run
D'oh, the servers' Shutdown() would never be called because they were
never added to the list of servers.

Thanks Danny for finding this.
2019-03-27 12:36:30 -06:00
Matthew Holt a8dc73b4d9 Performance testing Load function 2019-03-26 19:42:52 -06:00
Matthew Holt 86e2d1b0a4 Rudimentary start of HTTP servers 2019-03-26 15:45:51 -06:00
Matthew Holt 859b5d7ea3 Initial commit 2019-03-26 12:00:54 -06:00
221 changed files with 35162 additions and 6535 deletions
+16 -10
View File
@@ -1,12 +1,18 @@
.DS_Store
Thumbs.db
_gitignore/
Vagrantfile
.vagrant/
dist/
error.log
access.log
/*.conf
*.log
Caddyfile
!caddyfile/
# artifacts from pprof tooling
*.prof
*.test
# build artifacts
cmd/caddy/caddy
cmd/caddy/caddy.exe
# mac specific
.DS_Store
# go modules
vendor
+49
View File
@@ -0,0 +1,49 @@
linters-settings:
errcheck:
ignore: fmt:.*,io/ioutil:^Read.*,github.com/caddyserver/caddy/v2/caddyconfig:RegisterAdapter,github.com/caddyserver/caddy/v2:RegisterModule
ignoretests: true
misspell:
locale: US
linters:
enable:
- bodyclose
- errcheck
- gofmt
- goimports
- gosec
- ineffassign
- misspell
run:
# default concurrency is a available CPU number.
# concurrency: 4 # explicitly omit this value to fully utilize available resources.
deadline: 5m
issues-exit-code: 1
tests: false
# output configuration options
output:
format: 'colored-line-number'
print-issued-lines: true
print-linter-name: true
issues:
exclude-rules:
# we aren't calling unknown URL
- text: "G107" # G107: Url provided to HTTP request as taint input
linters:
- gosec
# as a web server that's expected to handle any template, this is totally in the hands of the user.
- text: "G203" # G203: Use of unescaped data in HTML templates
linters:
- gosec
# we're shelling out to known commands, not relying on user-defined input.
- text: "G204" # G204: Audit use of command execution
linters:
- gosec
# the choice of weakrand is deliberate, hence the named import "weakrand"
- path: modules/caddyhttp/reverseproxy/selectionpolicies.go
text: "G404" # G404: Insecure random number source (rand)
linters:
- gosec
-2
View File
@@ -1,2 +0,0 @@
language: go
script: go test ./...
+10
View File
@@ -0,0 +1,10 @@
# This is the official list of Caddy Authors for copyright purposes.
# Authors may be either individual people or legal entities.
#
# Not all individual contributors are authors. For the full list of
# contributors, refer to the project's page on GitHub or the repo's
# commit history.
Matthew Holt <Matthew.Holt@gmail.com>
Light Code Labs <sales@lightcodelabs.com>
Ardan Labs <info@ardanlabs.com>
-11
View File
@@ -1,11 +0,0 @@
## Contributing to Caddy
**[Join us on Slack](https://gophers.slack.com/messages/caddy/)** to chat with other Caddy developers! ([Request an invite](http://bit.ly/go-slack-signup), then join the #caddy channel.)
This project gladly accepts contributions. Interested users are encouraged to get involved by [opening issues](https://github.com/mholt/caddy/issues) with their ideas, questions, and bug reports. Bug reports should contain clear instructions to reproduce the problem and state expected behavior.
For small tweaks and bug fixes, feel free to submit [pull requests](https://github.com/mholt/caddy/pulls) at any time. For new features or to change existing behavior, please open an issue first to discuss it and claim it. This prevents overlapping efforts and also keeps the project in-line with its goals.
If you've found a vulnerability that is serious, please email me: Matthew dot Holt at Gmail. If it's not a big deal, a pull request will probably be faster.
Thanks for your help!
+3 -2
View File
@@ -1,3 +1,4 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
@@ -178,7 +179,7 @@
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "{}"
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
@@ -186,7 +187,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright {yyyy} {name of copyright owner}
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
+109 -96
View File
@@ -1,134 +1,147 @@
[![Caddy](https://caddyserver.com/resources/images/caddy-boxed.png)](https://caddyserver.com)
Caddy 2
=======
[![Documentation](https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square)](https://godoc.org/github.com/mholt/caddy) [![Build Status](https://img.shields.io/travis/mholt/caddy.svg?style=flat-square)](https://travis-ci.org/mholt/caddy)
This is the development branch for Caddy 2.
Caddy is a lightweight, general-purpose web server for Windows, Mac, Linux, BSD, and [Android](https://github.com/mholt/caddy/wiki/Running-Caddy-on-Android). It is a capable alternative to other popular web servers that is easy to use.
**Caddy 2 is production-ready, but there may be breaking changes before the stable 2.0 release.** Please test it and deploy it as much as you are able, and submit your feedback!
The most notable features are HTTP/2, Virtual Hosts, TLS + SNI, and easy configuration with a [Caddyfile](https://caddyserver.com/docs/caddyfile). Usually, you have one Caddyfile per site. Most directives for the Caddyfile invoke a layer of middleware which can be [used in your own Go programs](https://github.com/mholt/caddy/wiki/Using-Caddy-Middleware-in-Your-Own-Programs).
**Caddy 2 is the web server of the Go community.** We are looking for maintainers to represent the community! Please become involved with issues, PRs, [our forum](https://caddy.community), sharing on social media, etc.
[Download](https://github.com/mholt/caddy/releases) · [User Guide](https://caddyserver.com/docs)
---
<p align="center">
<a href="https://caddyserver.com"><img src="https://user-images.githubusercontent.com/1128849/36338535-05fb646a-136f-11e8-987b-e6901e717d5a.png" alt="Caddy" width="450"></a>
</p>
<h3 align="center">Every site on HTTPS</h3>
<p align="center">Caddy is an extensible server platform that uses TLS by default.</p>
<p align="center">
<a href="https://dev.azure.com/mholt-dev/Caddy/_build/latest?definitionId=5&branchName=v2"><img src="https://dev.azure.com/mholt-dev/Caddy/_apis/build/status/Multiplatform%20Tests?branchName=v2"></a>
<a href="https://pkg.go.dev/github.com/caddyserver/caddy/v2"><img src="https://img.shields.io/badge/godoc-reference-blue.svg"></a>
<a href="https://app.fuzzit.dev/orgs/caddyserver-gh/dashboard"><img src="https://app.fuzzit.dev/badge?org_id=caddyserver-gh"></a>
<br>
<a href="https://twitter.com/caddyserver" title="@caddyserver on Twitter"><img src="https://img.shields.io/badge/twitter-@caddyserver-55acee.svg" alt="@caddyserver on Twitter"></a>
<a href="https://caddy.community" title="Caddy Forum"><img src="https://img.shields.io/badge/community-forum-ff69b4.svg" alt="Caddy Forum"></a>
<a href="https://sourcegraph.com/github.com/caddyserver/caddy?badge" title="Caddy on Sourcegraph"><img src="https://sourcegraph.com/github.com/caddyserver/caddy/-/badge.svg" alt="Caddy on Sourcegraph"></a>
</p>
<p align="center">
<a href="https://github.com/caddyserver/caddy/releases">Download</a> ·
<a href="https://caddyserver.com/docs/">Documentation</a> ·
<a href="https://caddy.community">Community</a>
</p>
### Menu
- [Getting Caddy](#getting-caddy)
- [Running from Source](#running-from-source)
- [Quick Start](#quick-start)
- [Contributing](#contributing)
- [About the Project](#about-the-project)
- [Build from source](#build-from-source)
- [Building with plugins](#building-with-plugins-and/or-version-information)
- [Getting started](#getting-started)
- [Overview](#overview)
- [Full documentation](#full-documentation)
- [Getting help](#getting-help)
- [About](#about)
<p align="center">
<b>Powered by</b>
<br>
<a href="https://github.com/caddyserver/certmagic"><img src="https://user-images.githubusercontent.com/1128849/49704830-49d37200-fbd5-11e8-8385-767e0cd033c3.png" alt="CertMagic" width="250"></a>
</p>
## Build from source
_**Note:** These steps [will not embed proper version information](https://github.com/golang/go/issues/29228). For that, please follow the instructions below for building with plugins (you do not have to add any plugins)._
## Getting Caddy
Requirements:
Caddy binaries have no dependencies and are available for nearly every platform.
- [Go 1.14 or newer](https://golang.org/dl/)
- Do NOT disable [Go modules](https://github.com/golang/go/wiki/Modules) (`export GO111MODULE=auto`)
[Latest release](https://github.com/mholt/caddy/releases/latest)
Download the `v2` source code:
## Running from Source
Note: You will need **[Go 1.4](https://golang.org/dl)** or newer
1. `$ go get github.com/mholt/caddy`
2. `cd` into your website's directory
3. Run `caddy` (assumes `$GOPATH/bin` is in your `$PATH`)
If you're tinkering, you can also use `go run main.go`.
By default, Caddy serves the current directory at [localhost:2015](http://localhost:2015). You can place a Caddyfile to configure Caddy for serving your site.
Caddy accepts some flags from the command line. Run `caddy -h` to view the help for flags. You can also pipe a Caddyfile into the caddy command.
#### Docker Container
Caddy is [available as a Docker container](https://registry.hub.docker.com/u/darron/caddy/).
#### 3rd-party libraries
Although Caddy's binaries are completely static, Caddy relies on some excellent libraries that really make the project possible.
- [bradfitz/http2](https://github.com/bradfitz/http2) for HTTP/2 support
- [russross/blackfriday](https://github.com/russross/blackfriday) for Markdown rendering
- [dustin/go-humanize](https://github.com/dustin/go-humanize) for pleasant times and sizes
- [flynn/go-shlex](https://github.com/flynn/go-shlex) to parse shell commands properly
This list may not be comprehensive, but [godoc.org](https://godoc.org/github.com/mholt/caddy) will list all packages that any given package imports.
## Quick Start
The website has [full documentation](https://caddyserver.com/docs) but this will get you started in about 30 seconds:
Place a file named "Caddyfile" with your site. Paste this into it and save:
```
localhost
gzip
browse
ext .html
websocket /echo cat
log ../access.log
header /api Access-Control-Allow-Origin *
```bash
$ git clone -b v2 "https://github.com/caddyserver/caddy.git"
```
Run `caddy` from that directory, and it will automatically use that Caddyfile to configure itself.
Build:
That simple file enables compression, allows directory browsing (for folders without an index file), serves clean URLs, hosts an echo server for WebSocket connections at /echo, logs accesses to access.log, and adds the coveted `Access-Control-Allow-Origin: *` header for all responses from some API.
Wow! Caddy can do a lot with just a few lines.
#### Defining multiple sites
You can run multiple sites from the same Caddyfile, too:
```
http://mysite.com,
http://www.mysite.com {
redir https://mysite.com
}
https://mysite.com {
tls mysite.crt mysite.key
# ...
}
```bash
$ cd caddy/cmd/caddy/
$ go build
```
Note that the secure host will automatically be served with HTTP/2 if the client supports it.
That will put a `caddy(.exe)` binary into the current directory.
For more documentation, please view [the website](https://caddyserver.com/docs). You may also be interested in the [developer guide](https://github.com/mholt/caddy/wiki) on this project's GitHub wiki.
If you encounter any Go-module-related errors, try clearing your Go module cache (`$GOPATH/pkg/mod`) or Go package cache (`$GOPATH/pkg`) and read [the Go wiki page about modules for help](https://github.com/golang/go/wiki/Modules). If you have issues with Go modules, please consult the Go community for help. But if there is an actual error in Caddy, please report it to us.
### Building with plugins and/or version information
Caddy is extensible with plugins. Plugins are added at compile-time, so all Caddy binaries are static (self-contained) and portable.
Instructions for doing this are also given in comments in [cmd/caddy/main.go](https://github.com/caddyserver/caddy/blob/v2/cmd/caddy/main.go) which you can copy and use as a template.
1. Create a new folder: `mkdir caddy`
2. Change into it: `cd caddy`
3. Copy [Caddy's main.go](https://github.com/caddyserver/caddy/blob/v2/cmd/caddy/main.go) into the empty folder. Add imports for any plugins you want to include.
4. Run: `go mod init caddy`
5. Run: `go get github.com/caddyserver/caddy/v2@TAG` replacing `TAG` with the latest v2 tag. (Won't be necessary after stable 2.0 release.)
6. Run: `go build`
Congrats, you now have a custom Caddy build with proper version information!
## Quick start
The [Caddy website](https://caddyserver.com/docs/) has documentation that includes tutorials, quick-start guides, reference, and more.
**We recommend that all users do our [Getting Started](https://caddyserver.com/docs/getting-started) guide to become familiar with using Caddy.**
## Contributing
**[Join us on Slack](https://gophers.slack.com/messages/caddy/)** to chat with other Caddy developers! ([Request an invite](http://bit.ly/go-slack-signup), then join the #caddy channel.)
This project gladly accepts contributions. Interested users are encouraged to get involved by opening issues with their ideas, questions, and bug reports. Bug reports should contain clear instructions to reproduce the problem and state expected behavior.
For small tweaks and bug fixes, feel free to submit pull requests at any time. For new features or to change existing behavior, please open an issue first to discuss it and claim it. This prevents overlapping efforts and also keeps the project in-line with its goals.
Thanks for making Caddy -- and the Web -- better!
If you've only got a few minutes, [the website has several quick-start tutorials](https://caddyserver.com/docs/quick-starts) to choose from! However, after finishing a quick-start tutorial, please read more documentation to understand how the software works. 🙂
## Overview
## About the project
Caddy is most often used as an HTTPS server, but it is suitable for any long-running Go program. First and foremost, it is a platform to run Go applications. Caddy "apps" are just Go programs that are implemented as Caddy modules. Two apps -- `tls` and `http` -- ship standard with Caddy.
Caddy was born out of the need for a "batteries-included" web server that runs anywhere and doesn't have to take its configuration with it. Caddy took inspiration from nginx, lighttpd, Websocketd, and Vagrant, and provides a pleasant mixture of features from each of them.
Caddy apps instantly benefit from [automated documentation](https://caddyserver.com/docs/json/), graceful on-line [config changes via API](https://caddyserver.com/docs/api), and unification with other Caddy apps.
Although [JSON](https://caddyserver.com/docs/json/) is Caddy's native config language, Caddy can accept input from [config adapters](https://caddyserver.com/docs/config-adapters) which can essentially convert any config format of your choice into JSON: Caddyfile, JSON 5, YAML, TOML, NGINX config, and more.
The primary way to configure Caddy is through [its API](https://caddyserver.com/docs/api), but if you prefer config files, the [command-line interface](https://caddyserver.com/docs/command-line) supports those too.
Caddy exposes an unprecedented level of control compared to any web server in existence. In Caddy, you are usually setting the actual values of the initialized types in memory that power everything from your HTTP handlers and TLS handshakes to your storage medium. Caddy is also ridiculously extensible, with a powerful plugin system that makes vast improvements over other web servers.
To wield the power of this design, you need to know how the config document is structured. Please see the [our documentation site](https://caddyserver.com/docs/) for details about [Caddy's config structure](https://caddyserver.com/docs/json/).
Nearly all of Caddy's configuration is contained in a single config document, rather than being scattered across CLI flags and env variables and a configuration file as with other web servers. This makes managing your server config more straightforward and reduces hidden variables/factors.
*Twitter: [@mholt6](https://twitter.com/mholt6)*
## Full documentation
Our website has complete documentation:
**https://caddyserver.com/docs/**
The docs are also open source. You can contribute to them here: https://github.com/caddyserver/website
## Getting help
- We **strongly recommend** that all professionals or companies using Caddy get a support contract through [Ardan Labs](https://www.ardanlabs.com/my/contact-us?dd=caddy) before help is needed.
- Individuals can exchange help for free on our community forum at https://caddy.community. Remember that people give help out of their spare time and good will. The best way to get help is to give it first!
Please use our [issue tracker](/caddyserver/caddy/issues) only for bug reports and feature requests, i.e. actionable development items (support questions will usually be referred to the forums).
## About
**The name "Caddy" is trademarked.** The name of the software is "Caddy", not "Caddy Server" or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the Caddy web server". Caddy is a registered trademark of Light Code Labs, LLC.
- _Project on Twitter: [@caddyserver](https://twitter.com/caddyserver)_
- _Author on Twitter: [@mholt6](https://twitter.com/mholt6)_
+875
View File
@@ -0,0 +1,875 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"bytes"
"context"
"encoding/json"
"expvar"
"fmt"
"io"
"mime"
"net/http"
"net/http/pprof"
"net/url"
"os"
"path"
"regexp"
"strconv"
"strings"
"sync"
"time"
"github.com/caddyserver/caddy/v2/caddyconfig"
"go.uber.org/zap"
)
// TODO: is there a way to make the admin endpoint so that it can be plugged into the HTTP app? see issue #2833
// AdminConfig configures Caddy's API endpoint, which is used
// to manage Caddy while it is running.
type AdminConfig struct {
// If true, the admin endpoint will be completely disabled.
// Note that this makes any runtime changes to the config
// impossible, since the interface to do so is through the
// admin endpoint.
Disabled bool `json:"disabled,omitempty"`
// The address to which the admin endpoint's listener should
// bind itself. Can be any single network address that can be
// parsed by Caddy. Default: localhost:2019
Listen string `json:"listen,omitempty"`
// If true, CORS headers will be emitted, and requests to the
// API will be rejected if their `Host` and `Origin` headers
// do not match the expected value(s). Use `origins` to
// customize which origins/hosts are allowed.If `origins` is
// not set, the listen address is the only value allowed by
// default.
EnforceOrigin bool `json:"enforce_origin,omitempty"`
// The list of allowed origins for API requests. Only used if
// `enforce_origin` is true. If not set, the listener address
// will be the default value. If set but empty, no origins will
// be allowed.
Origins []string `json:"origins,omitempty"`
// Options related to configuration management.
Config *ConfigSettings `json:"config,omitempty"`
}
// ConfigSettings configures the, uh, configuration... and
// management thereof.
type ConfigSettings struct {
// Whether to keep a copy of the active config on disk. Default is true.
Persist *bool `json:"persist,omitempty"`
}
// listenAddr extracts a singular listen address from ac.Listen,
// returning the network and the address of the listener.
func (admin AdminConfig) listenAddr() (string, string, error) {
input := admin.Listen
if input == "" {
input = DefaultAdminListen
}
listenAddr, err := ParseNetworkAddress(input)
if err != nil {
return "", "", fmt.Errorf("parsing admin listener address: %v", err)
}
if listenAddr.PortRangeSize() != 1 {
return "", "", fmt.Errorf("admin endpoint must have exactly one address; cannot listen on %v", listenAddr)
}
return listenAddr.Network, listenAddr.JoinHostPort(0), nil
}
// newAdminHandler reads admin's config and returns an http.Handler suitable
// for use in an admin endpoint server, which will be listening on listenAddr.
func (admin AdminConfig) newAdminHandler(listenAddr string) adminHandler {
muxWrap := adminHandler{
enforceOrigin: admin.EnforceOrigin,
allowedOrigins: admin.allowedOrigins(listenAddr),
mux: http.NewServeMux(),
}
// addRoute just calls muxWrap.mux.Handle after
// wrapping the handler with error handling
addRoute := func(pattern string, h AdminHandler) {
wrapper := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
err := h.ServeHTTP(w, r)
muxWrap.handleError(w, r, err)
})
muxWrap.mux.Handle(pattern, wrapper)
}
// register standard config control endpoints
addRoute("/load", AdminHandlerFunc(handleLoad))
addRoute("/"+rawConfigKey+"/", AdminHandlerFunc(handleConfig))
addRoute("/id/", AdminHandlerFunc(handleConfigID))
addRoute("/stop", AdminHandlerFunc(handleStop))
// register debugging endpoints
muxWrap.mux.HandleFunc("/debug/pprof/", pprof.Index)
muxWrap.mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
muxWrap.mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
muxWrap.mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
muxWrap.mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
muxWrap.mux.Handle("/debug/vars", expvar.Handler())
// register third-party module endpoints
for _, m := range GetModules("admin.api") {
router := m.New().(AdminRouter)
for _, route := range router.Routes() {
addRoute(route.Pattern, route.Handler)
}
}
return muxWrap
}
// allowedOrigins returns a list of origins that are allowed.
// If admin.Origins is nil (null), the provided listen address
// will be used as the default origin. If admin.Origins is
// empty, no origins will be allowed, effectively bricking the
// endpoint, but whatever.
func (admin AdminConfig) allowedOrigins(listen string) []string {
uniqueOrigins := make(map[string]struct{})
for _, o := range admin.Origins {
uniqueOrigins[o] = struct{}{}
}
if admin.Origins == nil {
uniqueOrigins[listen] = struct{}{}
}
var allowed []string
for origin := range uniqueOrigins {
allowed = append(allowed, origin)
}
return allowed
}
// replaceAdmin replaces the running admin server according
// to the relevant configuration in cfg. If no configuration
// for the admin endpoint exists in cfg, a default one is
// used, so that there is always an admin server (unless it
// is explicitly configured to be disabled).
func replaceAdmin(cfg *Config) error {
// always be sure to close down the old admin endpoint
// as gracefully as possible, even if the new one is
// disabled -- careful to use reference to the current
// (old) admin endpoint since it will be different
// when the function returns
oldAdminServer := adminServer
defer func() {
// do the shutdown asynchronously so that any
// current API request gets a response; this
// goroutine may last a few seconds
if oldAdminServer != nil {
go func(oldAdminServer *http.Server) {
err := stopAdminServer(oldAdminServer)
if err != nil {
Log().Named("admin").Error("stopping current admin endpoint", zap.Error(err))
}
}(oldAdminServer)
}
}()
// always get a valid admin config
adminConfig := DefaultAdminConfig
if cfg != nil && cfg.Admin != nil {
adminConfig = cfg.Admin
}
// if new admin endpoint is to be disabled, we're done
if adminConfig.Disabled {
Log().Named("admin").Warn("admin endpoint disabled")
return nil
}
// extract a singular listener address
netw, addr, err := adminConfig.listenAddr()
if err != nil {
return err
}
handler := adminConfig.newAdminHandler(addr)
ln, err := Listen(netw, addr)
if err != nil {
return err
}
adminServer = &http.Server{
Handler: handler,
ReadTimeout: 10 * time.Second,
ReadHeaderTimeout: 5 * time.Second,
IdleTimeout: 60 * time.Second,
MaxHeaderBytes: 1024 * 64,
}
go adminServer.Serve(ln)
Log().Named("admin").Info(
"admin endpoint started",
zap.String("address", addr),
zap.Bool("enforce_origin", adminConfig.EnforceOrigin),
zap.Strings("origins", handler.allowedOrigins),
)
return nil
}
func stopAdminServer(srv *http.Server) error {
if srv == nil {
return fmt.Errorf("no admin server")
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
err := srv.Shutdown(ctx)
if err != nil {
return fmt.Errorf("shutting down admin server: %v", err)
}
Log().Named("admin").Info("stopped previous server")
return nil
}
// AdminRouter is a type which can return routes for the admin API.
type AdminRouter interface {
Routes() []AdminRoute
}
// AdminRoute represents a route for the admin endpoint.
type AdminRoute struct {
Pattern string
Handler AdminHandler
}
type adminHandler struct {
enforceOrigin bool
allowedOrigins []string
mux *http.ServeMux
}
// ServeHTTP is the external entry point for API requests.
// It will only be called once per request.
func (h adminHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
Log().Named("admin.api").Info("received request",
zap.String("method", r.Method),
zap.String("uri", r.RequestURI),
zap.String("remote_addr", r.RemoteAddr),
zap.Reflect("headers", r.Header),
)
h.serveHTTP(w, r)
}
// serveHTTP is the internal entry point for API requests. It may
// be called more than once per request, for example if a request
// is rewritten (i.e. internal redirect).
func (h adminHandler) serveHTTP(w http.ResponseWriter, r *http.Request) {
if h.enforceOrigin {
// DNS rebinding mitigation
err := h.checkHost(r)
if err != nil {
h.handleError(w, r, err)
return
}
// cross-site mitigation
origin, err := h.checkOrigin(r)
if err != nil {
h.handleError(w, r, err)
return
}
if r.Method == http.MethodOptions {
w.Header().Set("Access-Control-Allow-Methods", "OPTIONS, GET, POST, PUT, PATCH, DELETE")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Content-Length, Cache-Control")
w.Header().Set("Access-Control-Allow-Credentials", "true")
}
w.Header().Set("Access-Control-Allow-Origin", origin)
}
// TODO: authentication & authorization, if configured
h.mux.ServeHTTP(w, r)
}
func (h adminHandler) handleError(w http.ResponseWriter, r *http.Request, err error) {
if err == nil {
return
}
if err == ErrInternalRedir {
h.serveHTTP(w, r)
return
}
apiErr, ok := err.(APIError)
if !ok {
apiErr = APIError{
Code: http.StatusInternalServerError,
Err: err,
}
}
if apiErr.Code == 0 {
apiErr.Code = http.StatusInternalServerError
}
if apiErr.Message == "" && apiErr.Err != nil {
apiErr.Message = apiErr.Err.Error()
}
Log().Named("admin.api").Error("request error",
zap.Error(err),
zap.Int("status_code", apiErr.Code),
)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(apiErr.Code)
json.NewEncoder(w).Encode(apiErr)
}
// checkHost returns a handler that wraps next such that
// it will only be called if the request's Host header matches
// a trustworthy/expected value. This helps to mitigate DNS
// rebinding attacks.
func (h adminHandler) checkHost(r *http.Request) error {
var allowed bool
for _, allowedHost := range h.allowedOrigins {
if r.Host == allowedHost {
allowed = true
break
}
}
if !allowed {
return APIError{
Code: http.StatusForbidden,
Err: fmt.Errorf("host not allowed: %s", r.Host),
}
}
return nil
}
// checkOrigin ensures that the Origin header, if
// set, matches the intended target; prevents arbitrary
// sites from issuing requests to our listener. It
// returns the origin that was obtained from r.
func (h adminHandler) checkOrigin(r *http.Request) (string, error) {
origin := h.getOriginHost(r)
if origin == "" {
return origin, APIError{
Code: http.StatusForbidden,
Err: fmt.Errorf("missing required Origin header"),
}
}
if !h.originAllowed(origin) {
return origin, APIError{
Code: http.StatusForbidden,
Err: fmt.Errorf("client is not allowed to access from origin %s", origin),
}
}
return origin, nil
}
func (h adminHandler) getOriginHost(r *http.Request) string {
origin := r.Header.Get("Origin")
if origin == "" {
origin = r.Header.Get("Referer")
}
originURL, err := url.Parse(origin)
if err == nil && originURL.Host != "" {
origin = originURL.Host
}
return origin
}
func (h adminHandler) originAllowed(origin string) bool {
for _, allowedOrigin := range h.allowedOrigins {
originCopy := origin
if !strings.Contains(allowedOrigin, "://") {
// no scheme specified, so allow both
originCopy = strings.TrimPrefix(originCopy, "http://")
originCopy = strings.TrimPrefix(originCopy, "https://")
}
if originCopy == allowedOrigin {
return true
}
}
return false
}
func handleLoad(w http.ResponseWriter, r *http.Request) error {
if r.Method != http.MethodPost {
return APIError{
Code: http.StatusMethodNotAllowed,
Err: fmt.Errorf("method not allowed"),
}
}
buf := bufPool.Get().(*bytes.Buffer)
buf.Reset()
defer bufPool.Put(buf)
_, err := io.Copy(buf, r.Body)
if err != nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("reading request body: %v", err),
}
}
body := buf.Bytes()
// if the config is formatted other than Caddy's native
// JSON, we need to adapt it before loading it
if ctHeader := r.Header.Get("Content-Type"); ctHeader != "" {
ct, _, err := mime.ParseMediaType(ctHeader)
if err != nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("invalid Content-Type: %v", err),
}
}
if !strings.HasSuffix(ct, "/json") {
slashIdx := strings.Index(ct, "/")
if slashIdx < 0 {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("malformed Content-Type"),
}
}
adapterName := ct[slashIdx+1:]
cfgAdapter := caddyconfig.GetAdapter(adapterName)
if cfgAdapter == nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("unrecognized config adapter '%s'", adapterName),
}
}
result, warnings, err := cfgAdapter.Adapt(body, nil)
if err != nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("adapting config using %s adapter: %v", adapterName, err),
}
}
if len(warnings) > 0 {
respBody, err := json.Marshal(warnings)
if err != nil {
Log().Named("admin.api.load").Error(err.Error())
}
w.Write(respBody)
}
body = result
}
}
forceReload := r.Header.Get("Cache-Control") == "must-revalidate"
err = Load(body, forceReload)
if err != nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("loading config: %v", err),
}
}
Log().Named("admin.api").Info("load complete")
return nil
}
func handleConfig(w http.ResponseWriter, r *http.Request) error {
switch r.Method {
case http.MethodGet:
w.Header().Set("Content-Type", "application/json")
err := readConfig(r.URL.Path, w)
if err != nil {
return APIError{Code: http.StatusBadRequest, Err: err}
}
return nil
case http.MethodPost,
http.MethodPut,
http.MethodPatch,
http.MethodDelete:
// DELETE does not use a body, but the others do
var body []byte
if r.Method != http.MethodDelete {
if ct := r.Header.Get("Content-Type"); !strings.Contains(ct, "/json") {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("unacceptable content-type: %v; 'application/json' required", ct),
}
}
buf := bufPool.Get().(*bytes.Buffer)
buf.Reset()
defer bufPool.Put(buf)
_, err := io.Copy(buf, r.Body)
if err != nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("reading request body: %v", err),
}
}
body = buf.Bytes()
}
forceReload := r.Header.Get("Cache-Control") == "must-revalidate"
err := changeConfig(r.Method, r.URL.Path, body, forceReload)
if err != nil {
return err
}
default:
return APIError{
Code: http.StatusMethodNotAllowed,
Err: fmt.Errorf("method %s not allowed", r.Method),
}
}
return nil
}
func handleConfigID(w http.ResponseWriter, r *http.Request) error {
idPath := r.URL.Path
parts := strings.Split(idPath, "/")
if len(parts) < 3 || parts[2] == "" {
return fmt.Errorf("request path is missing object ID")
}
if parts[0] != "" || parts[1] != "id" {
return fmt.Errorf("malformed object path")
}
id := parts[2]
// map the ID to the expanded path
currentCfgMu.RLock()
expanded, ok := rawCfgIndex[id]
defer currentCfgMu.RUnlock()
if !ok {
return fmt.Errorf("unknown object ID '%s'", id)
}
// piece the full URL path back together
parts = append([]string{expanded}, parts[3:]...)
r.URL.Path = path.Join(parts...)
return ErrInternalRedir
}
func handleStop(w http.ResponseWriter, r *http.Request) error {
err := handleUnload(w, r)
if err != nil {
Log().Named("admin.api").Error("unload error", zap.Error(err))
}
go func() {
err := stopAdminServer(adminServer)
var exitCode int
if err != nil {
exitCode = ExitCodeFailedQuit
Log().Named("admin.api").Error("failed to stop admin server gracefully", zap.Error(err))
}
Log().Named("admin.api").Info("stopping now, bye!! 👋")
os.Exit(exitCode)
}()
return nil
}
// handleUnload stops the current configuration that is running.
// Note that doing this can also be accomplished with DELETE /config/
// but we leave this function because handleStop uses it.
func handleUnload(w http.ResponseWriter, r *http.Request) error {
if r.Method != http.MethodPost {
return APIError{
Code: http.StatusMethodNotAllowed,
Err: fmt.Errorf("method not allowed"),
}
}
currentCfgMu.RLock()
hasCfg := currentCfg != nil
currentCfgMu.RUnlock()
if !hasCfg {
Log().Named("admin.api").Info("nothing to unload")
return nil
}
Log().Named("admin.api").Info("unloading")
if err := stopAndCleanup(); err != nil {
Log().Named("admin.api").Error("error unloading", zap.Error(err))
} else {
Log().Named("admin.api").Info("unloading completed")
}
return nil
}
// unsyncedConfigAccess traverses into the current config and performs
// the operation at path according to method, using body and out as
// needed. This is a low-level, unsynchronized function; most callers
// will want to use changeConfig or readConfig instead. This requires a
// read or write lock on currentCfgMu, depending on method (GET needs
// only a read lock; all others need a write lock).
func unsyncedConfigAccess(method, path string, body []byte, out io.Writer) error {
var err error
var val interface{}
// if there is a request body, decode it into the
// variable that will be set in the config according
// to method and path
if len(body) > 0 {
err = json.Unmarshal(body, &val)
if err != nil {
return fmt.Errorf("decoding request body: %v", err)
}
}
enc := json.NewEncoder(out)
cleanPath := strings.Trim(path, "/")
if cleanPath == "" {
return fmt.Errorf("no traversable path")
}
parts := strings.Split(cleanPath, "/")
if len(parts) == 0 {
return fmt.Errorf("path missing")
}
// A path that ends with "..." implies:
// 1) the part before it is an array
// 2) the payload is an array
// and means that the user wants to expand the elements
// in the payload array and append each one into the
// destination array, like so:
// array = append(array, elems...)
// This special case is handled below.
ellipses := parts[len(parts)-1] == "..."
if ellipses {
parts = parts[:len(parts)-1]
}
var ptr interface{} = rawCfg
traverseLoop:
for i, part := range parts {
switch v := ptr.(type) {
case map[string]interface{}:
// if the next part enters a slice, and the slice is our destination,
// handle it specially (because appending to the slice copies the slice
// header, which does not replace the original one like we want)
if arr, ok := v[part].([]interface{}); ok && i == len(parts)-2 {
var idx int
if method != http.MethodPost {
idxStr := parts[len(parts)-1]
idx, err = strconv.Atoi(idxStr)
if err != nil {
return fmt.Errorf("[%s] invalid array index '%s': %v",
path, idxStr, err)
}
if idx < 0 || idx >= len(arr) {
return fmt.Errorf("[%s] array index out of bounds: %s", path, idxStr)
}
}
switch method {
case http.MethodGet:
err = enc.Encode(arr[idx])
if err != nil {
return fmt.Errorf("encoding config: %v", err)
}
case http.MethodPost:
if ellipses {
valArray, ok := val.([]interface{})
if !ok {
return fmt.Errorf("final element is not an array")
}
v[part] = append(arr, valArray...)
} else {
v[part] = append(arr, val)
}
case http.MethodPut:
// avoid creation of new slice and a second copy (see
// https://github.com/golang/go/wiki/SliceTricks#insert)
arr = append(arr, nil)
copy(arr[idx+1:], arr[idx:])
arr[idx] = val
v[part] = arr
case http.MethodPatch:
arr[idx] = val
case http.MethodDelete:
v[part] = append(arr[:idx], arr[idx+1:]...)
default:
return fmt.Errorf("unrecognized method %s", method)
}
break traverseLoop
}
if i == len(parts)-1 {
switch method {
case http.MethodGet:
err = enc.Encode(v[part])
if err != nil {
return fmt.Errorf("encoding config: %v", err)
}
case http.MethodPost:
// if the part is an existing list, POST appends to
// it, otherwise it just sets or creates the value
if arr, ok := v[part].([]interface{}); ok {
if ellipses {
valArray, ok := val.([]interface{})
if !ok {
return fmt.Errorf("final element is not an array")
}
v[part] = append(arr, valArray...)
} else {
v[part] = append(arr, val)
}
} else {
v[part] = val
}
case http.MethodPut:
if _, ok := v[part]; ok {
return fmt.Errorf("[%s] key already exists: %s", path, part)
}
v[part] = val
case http.MethodPatch:
if _, ok := v[part]; !ok {
return fmt.Errorf("[%s] key does not exist: %s", path, part)
}
v[part] = val
case http.MethodDelete:
delete(v, part)
default:
return fmt.Errorf("unrecognized method %s", method)
}
} else {
// if we are "PUTting" a new resource, the key(s) in its path
// might not exist yet; that's OK but we need to make them as
// we go, while we still have a pointer from the level above
if v[part] == nil && method == http.MethodPut {
v[part] = make(map[string]interface{})
}
ptr = v[part]
}
case []interface{}:
partInt, err := strconv.Atoi(part)
if err != nil {
return fmt.Errorf("[/%s] invalid array index '%s': %v",
strings.Join(parts[:i+1], "/"), part, err)
}
if partInt < 0 || partInt >= len(v) {
return fmt.Errorf("[/%s] array index out of bounds: %s",
strings.Join(parts[:i+1], "/"), part)
}
ptr = v[partInt]
default:
return fmt.Errorf("invalid traversal path at: %s", strings.Join(parts[:i+1], "/"))
}
}
return nil
}
// RemoveMetaFields removes meta fields like "@id" from a JSON message
// by using a simple regular expression. (An alternate way to do this
// would be to delete them from the raw, map[string]interface{}
// representation as they are indexed, then iterate the index we made
// and add them back after encoding as JSON, but this is simpler.)
func RemoveMetaFields(rawJSON []byte) []byte {
return idRegexp.ReplaceAllFunc(rawJSON, func(in []byte) []byte {
// matches with a comma on both sides (when "@id" property is
// not the first or last in the object) need to keep exactly
// one comma for correct JSON syntax
comma := []byte{','}
if bytes.HasPrefix(in, comma) && bytes.HasSuffix(in, comma) {
return comma
}
return []byte{}
})
}
// AdminHandler is like http.Handler except ServeHTTP may return an error.
//
// If any handler encounters an error, it should be returned for proper
// handling.
type AdminHandler interface {
ServeHTTP(http.ResponseWriter, *http.Request) error
}
// AdminHandlerFunc is a convenience type like http.HandlerFunc.
type AdminHandlerFunc func(http.ResponseWriter, *http.Request) error
// ServeHTTP implements the Handler interface.
func (f AdminHandlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) error {
return f(w, r)
}
// APIError is a structured error that every API
// handler should return for consistency in logging
// and client responses. If Message is unset, then
// Err.Error() will be serialized in its place.
type APIError struct {
Code int `json:"-"`
Err error `json:"-"`
Message string `json:"error"`
}
func (e APIError) Error() string {
if e.Err != nil {
return e.Err.Error()
}
return e.Message
}
var (
// DefaultAdminListen is the address for the admin
// listener, if none is specified at startup.
DefaultAdminListen = "localhost:2019"
// ErrInternalRedir indicates an internal redirect
// and is useful when admin API handlers rewrite
// the request; in that case, authentication and
// authorization needs to happen again for the
// rewritten request.
ErrInternalRedir = fmt.Errorf("internal redirect; re-authorization required")
// DefaultAdminConfig is the default configuration
// for the administration endpoint.
DefaultAdminConfig = &AdminConfig{
Listen: DefaultAdminListen,
}
)
// idRegexp is used to match ID fields and their associated values
// in the config. It also matches adjacent commas so that syntax
// can be preserved no matter where in the object the field appears.
// It supports string and most numeric values.
var idRegexp = regexp.MustCompile(`(?m),?\s*"` + idKey + `":\s?(-?[0-9]+(\.[0-9]+)?|(?U)".*")\s*,?`)
const (
rawConfigKey = "config"
idKey = "@id"
)
var bufPool = sync.Pool{
New: func() interface{} {
return new(bytes.Buffer)
},
}
var adminServer *http.Server
+132
View File
@@ -0,0 +1,132 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"encoding/json"
"reflect"
"testing"
)
func TestUnsyncedConfigAccess(t *testing.T) {
// each test is performed in sequence, so
// each change builds on the previous ones;
// the config is not reset between tests
for i, tc := range []struct {
method string
path string // rawConfigKey will be prepended
payload string
expect string // JSON representation of what the whole config is expected to be after the request
shouldErr bool
}{
{
method: "POST",
path: "",
payload: `{"foo": "bar", "list": ["a", "b", "c"]}`, // starting value
expect: `{"foo": "bar", "list": ["a", "b", "c"]}`,
},
{
method: "POST",
path: "/foo",
payload: `"jet"`,
expect: `{"foo": "jet", "list": ["a", "b", "c"]}`,
},
{
method: "POST",
path: "/bar",
payload: `{"aa": "bb", "qq": "zz"}`,
expect: `{"foo": "jet", "bar": {"aa": "bb", "qq": "zz"}, "list": ["a", "b", "c"]}`,
},
{
method: "DELETE",
path: "/bar/qq",
expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c"]}`,
},
{
method: "POST",
path: "/list",
payload: `"e"`,
expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "e"]}`,
},
{
method: "PUT",
path: "/list/3",
payload: `"d"`,
expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "d", "e"]}`,
},
{
method: "DELETE",
path: "/list/3",
expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "e"]}`,
},
{
method: "PATCH",
path: "/list/3",
payload: `"d"`,
expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "d"]}`,
},
{
method: "POST",
path: "/list/...",
payload: `["e", "f", "g"]`,
expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "d", "e", "f", "g"]}`,
},
} {
err := unsyncedConfigAccess(tc.method, rawConfigKey+tc.path, []byte(tc.payload), nil)
if tc.shouldErr && err == nil {
t.Fatalf("Test %d: Expected error return value, but got: %v", i, err)
}
if !tc.shouldErr && err != nil {
t.Fatalf("Test %d: Should not have had error return value, but got: %v", i, err)
}
// decode the expected config so we can do a convenient DeepEqual
var expectedDecoded interface{}
err = json.Unmarshal([]byte(tc.expect), &expectedDecoded)
if err != nil {
t.Fatalf("Test %d: Unmarshaling expected config: %v", i, err)
}
// make sure the resulting config is as we expect it
if !reflect.DeepEqual(rawCfg[rawConfigKey], expectedDecoded) {
t.Fatalf("Test %d:\nExpected:\n\t%#v\nActual:\n\t%#v",
i, expectedDecoded, rawCfg[rawConfigKey])
}
}
}
func BenchmarkLoad(b *testing.B) {
for i := 0; i < b.N; i++ {
cfg := []byte(`{
"apps": {
"http": {
"servers": {
"myserver": {
"listen": ["tcp/localhost:8080-8084"],
"read_timeout": "30s"
},
"yourserver": {
"listen": ["127.0.0.1:5000"],
"read_header_timeout": "15s"
}
}
}
}
}
`)
Load(cfg, true)
}
}
+263
View File
@@ -0,0 +1,263 @@
# Mutilated beyond recognition from the example at:
# https://docs.microsoft.com/azure/devops/pipelines/languages/go
trigger:
- v2
schedules:
- cron: "0 0 * * *"
displayName: Daily midnight fuzzing
branches:
include:
- v2
always: true
variables:
GOROOT: $(gorootDir)/go
GOPATH: $(system.defaultWorkingDirectory)/gopath
GOBIN: $(GOPATH)/bin
modulePath: '$(GOPATH)/src/github.com/$(build.repository.name)'
jobs:
- job: crossPlatformTest
displayName: "Cross-Platform Tests"
strategy:
matrix:
linux:
imageName: ubuntu-16.04
gorootDir: /usr/local
mac:
imageName: macos-10.14
gorootDir: /usr/local
windows:
imageName: windows-2019
gorootDir: C:\
pool:
vmImage: $(imageName)
steps:
- bash: |
latestGo=$(curl "https://golang.org/VERSION?m=text")
echo "##vso[task.setvariable variable=LATEST_GO]$latestGo"
echo "Latest Go version: $latestGo"
displayName: "Get latest Go version"
- bash: |
sudo rm -f $(which go)
echo '##vso[task.prependpath]$(GOBIN)'
echo '##vso[task.prependpath]$(GOROOT)/bin'
mkdir -p '$(modulePath)'
shopt -s extglob
shopt -s dotglob
mv !(gopath) '$(modulePath)'
displayName: Remove old Go, set GOBIN/GOROOT, and move project into GOPATH
# Install Go (this varies by platform)
- bash: |
wget "https://dl.google.com/go/$(LATEST_GO).linux-amd64.tar.gz"
sudo tar -C $(gorootDir) -xzf "$(LATEST_GO).linux-amd64.tar.gz"
condition: eq( variables['Agent.OS'], 'Linux' )
displayName: Install Go on Linux
- bash: |
wget "https://dl.google.com/go/$(LATEST_GO).darwin-amd64.tar.gz"
sudo tar -C $(gorootDir) -xzf "$(LATEST_GO).darwin-amd64.tar.gz"
condition: eq( variables['Agent.OS'], 'Darwin' )
displayName: Install Go on macOS
# The low performance is partly due to PowerShell's attempt to update the progress bar. Disabling it speeds up the process.
# Reference: https://github.com/PowerShell/PowerShell/issues/2138
- powershell: |
$ProgressPreference = 'SilentlyContinue'
Write-Host "Downloading Go..."
(New-Object System.Net.WebClient).DownloadFile("https://dl.google.com/go/$(LATEST_GO).windows-amd64.zip", "$(LATEST_GO).windows-amd64.zip")
Write-Host "Extracting Go... (I'm slow too)"
7z x "$(LATEST_GO).windows-amd64.zip" -o"$(gorootDir)"
condition: eq( variables['Agent.OS'], 'Windows_NT' )
displayName: Install Go on Windows
- bash: curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.23.6
displayName: Install golangci-lint
- script: |
go get github.com/axw/gocov/gocov
go get github.com/AlekSi/gocov-xml
go get -u github.com/jstemmer/go-junit-report
displayName: Install test and coverage analysis tools
- bash: |
printf "Using go at: $(which go)\n"
printf "Go version: $(go version)\n"
printf "\n\nGo environment:\n\n"
go env
printf "\n\nSystem environment:\n\n"
env
displayName: Print Go version and environment
- script: |
go get -v -t -d ./...
mkdir test-results
workingDirectory: '$(modulePath)'
displayName: Get dependencies
- bash: CGO_ENABLED=0 go build -trimpath -a -ldflags="-w -s" -v
workingDirectory: '$(modulePath)/cmd/caddy'
displayName: Build Caddy
- task: PublishBuildArtifacts@1
condition: eq( variables['Agent.OS'], 'Windows_NT' )
inputs:
pathtoPublish: '$(modulePath)/cmd/caddy/caddy.exe'
artifactName: caddy_v2.exe
- task: PublishBuildArtifacts@1
condition: ne( variables['Agent.OS'], 'Windows_NT' )
inputs:
pathtoPublish: '$(modulePath)/cmd/caddy/caddy'
artifactName: 'caddy_v2_$(Agent.OS)'
# its behavior is governed by .golangci.yml
- script: |
(golangci-lint run --out-format junit-xml) > test-results/lint-result.xml
exit 0
workingDirectory: '$(modulePath)'
continueOnError: true
displayName: Run lint check
- script: |
(go test -v -coverprofile=cover-profile.out -race ./... 2>&1) > test-results/test-result.out
workingDirectory: '$(modulePath)'
continueOnError: true
displayName: Run tests
- script: |
set -e
cmd/caddy/caddy start
go test -v -count=1 ./caddytest/...
cmd/caddy/caddy stop
workingDirectory: '$(modulePath)'
continueOnError: false
displayName: Run Integration tests
- script: |
mkdir coverage
gocov convert cover-profile.out > coverage/coverage.json
# Because Windows doesn't work with input redirection like *nix, but output redirection works.
(cat ./coverage/coverage.json | gocov-xml) > coverage/coverage.xml
workingDirectory: '$(modulePath)'
displayName: Prepare coverage reports
- script: |
(cat ./test-results/test-result.out | go-junit-report) > test-results/test-result.xml
workingDirectory: '$(modulePath)'
displayName: Prepare test report
- task: PublishCodeCoverageResults@1
displayName: Publish test coverage report
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: $(modulePath)/coverage/coverage.xml
- task: PublishTestResults@2
displayName: Publish unit test
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: $(modulePath)/test-results/test-result.xml
testRunTitle: $(agent.OS) Unit Test
mergeTestResults: false
- task: PublishTestResults@2
displayName: Publish lint results
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: $(modulePath)/test-results/lint-result.xml
testRunTitle: $(agent.OS) Lint
mergeTestResults: false
- bash: |
exit 1
condition: eq(variables['Agent.JobStatus'], 'SucceededWithIssues')
displayName: Coerce correct build result
- job: fuzzing
displayName: 'Fuzzing'
# Only run this job on schedules or PRs for non-forks.
condition: or(eq(variables['System.PullRequest.IsFork'], 'False'), eq(variables['Build.Reason'], 'Schedule') )
strategy:
matrix:
linux:
imageName: ubuntu-16.04
gorootDir: /usr/local
pool:
vmImage: $(imageName)
steps:
- bash: |
latestGo=$(curl "https://golang.org/VERSION?m=text")
echo "##vso[task.setvariable variable=LATEST_GO]$latestGo"
echo "Latest Go version: $latestGo"
displayName: "Get latest Go version"
- bash: |
sudo rm -f $(which go)
echo '##vso[task.prependpath]$(GOBIN)'
echo '##vso[task.prependpath]$(GOROOT)/bin'
mkdir -p '$(modulePath)'
shopt -s extglob
shopt -s dotglob
mv !(gopath) '$(modulePath)'
displayName: Remove old Go, set GOBIN/GOROOT, and move project into GOPATH
- bash: |
wget "https://dl.google.com/go/$(LATEST_GO).linux-amd64.tar.gz"
sudo tar -C $(gorootDir) -xzf "$(LATEST_GO).linux-amd64.tar.gz"
condition: eq( variables['Agent.OS'], 'Linux' )
displayName: Install Go on Linux
- bash: |
# Install Clang-7.0 because other versions seem to be missing the file libclang_rt.fuzzer-x86_64.a
sudo add-apt-repository "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-7 main"
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
sudo apt update && sudo apt install -y clang-7 lldb-7 lld-7
go get -v github.com/dvyukov/go-fuzz/go-fuzz github.com/dvyukov/go-fuzz/go-fuzz-build
wget -q -O fuzzit https://github.com/fuzzitdev/fuzzit/releases/download/v2.4.77/fuzzit_Linux_x86_64
chmod a+x fuzzit
mv fuzzit $(GOBIN)
displayName: Download go-fuzz tools and the Fuzzit CLI, and move Fuzzit CLI to GOBIN
condition: and(eq(variables['System.PullRequest.IsFork'], 'False') , eq( variables['Agent.OS'], 'Linux' ))
- bash: |
declare -A fuzzers_funcs=(\
["./caddyconfig/httpcaddyfile/addresses_fuzz.go"]="FuzzParseAddress" \
["./caddyconfig/caddyfile/parse_fuzz.go"]="FuzzParseCaddyfile" \
["./listeners_fuzz.go"]="FuzzParseNetworkAddress" \
["./replacer_fuzz.go"]="FuzzReplacer" \
)
declare -A fuzzers_targets=(\
["./caddyconfig/httpcaddyfile/addresses_fuzz.go"]="parse-address" \
["./caddyconfig/caddyfile/parse_fuzz.go"]="parse-caddyfile" \
["./listeners_fuzz.go"]="parse-network-address" \
["./replacer_fuzz.go"]="replacer" \
)
fuzz_type="local-regression"
if [[ $(Build.Reason) == "Schedule" ]]; then
fuzz_type="fuzzing"
fi
echo "Fuzzing type: $fuzz_type"
for f in $(find . -name \*_fuzz.go); do
FUZZER_DIRECTORY=$(dirname $f)
echo "go-fuzz-build func ${fuzzers_funcs[$f]} residing in $f"
go-fuzz-build -func "${fuzzers_funcs[$f]}" -libfuzzer -o "$FUZZER_DIRECTORY/${fuzzers_targets[$f]}.a" $FUZZER_DIRECTORY
echo "Generating fuzzer binary of func ${fuzzers_funcs[$f]} which resides in $f"
clang-7 -fsanitize=fuzzer "$FUZZER_DIRECTORY/${fuzzers_targets[$f]}.a" -o "$FUZZER_DIRECTORY/${fuzzers_targets[$f]}"
fuzzit create job caddyserver/${fuzzers_targets[$f]} $FUZZER_DIRECTORY/${fuzzers_targets[$f]} --api-key ${FUZZIT_API_KEY} --type "${fuzz_type}" --branch "${SYSTEM_PULLREQUEST_SOURCEBRANCH}" --revision "${BUILD_SOURCEVERSION}"
echo "Completed $f"
done
env:
FUZZIT_API_KEY: $(FUZZIT_API_KEY)
workingDirectory: '$(modulePath)'
displayName: Generate fuzzers & submit them to Fuzzit
+570
View File
@@ -0,0 +1,570 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"os"
"path"
"path/filepath"
"runtime/debug"
"strconv"
"strings"
"sync"
"time"
"github.com/caddyserver/certmagic"
"go.uber.org/zap"
)
// Config is the top (or beginning) of the Caddy configuration structure.
// Caddy config is expressed natively as a JSON document. If you prefer
// not to work with JSON directly, there are [many config adapters](/docs/config-adapters)
// available that can convert various inputs into Caddy JSON.
//
// Many parts of this config are extensible through the use of Caddy modules.
// Fields which have a json.RawMessage type and which appear as dots (•••) in
// the online docs can be fulfilled by modules in a certain module
// namespace. The docs show which modules can be used in a given place.
//
// Whenever a module is used, its name must be given either inline as part of
// the module, or as the key to the module's value. The docs will make it clear
// which to use.
//
// Generally, all config settings are optional, as it is Caddy convention to
// have good, documented default values. If a parameter is required, the docs
// should say so.
//
// Go programs which are directly building a Config struct value should take
// care to populate the JSON-encodable fields of the struct (i.e. the fields
// with `json` struct tags) if employing the module lifecycle (e.g. Provision
// method calls).
type Config struct {
Admin *AdminConfig `json:"admin,omitempty"`
Logging *Logging `json:"logging,omitempty"`
// StorageRaw is a storage module that defines how/where Caddy
// stores assets (such as TLS certificates). The default storage
// module is `caddy.storage.file_system` (the local file system),
// and the default path
// [depends on the OS and environment](/docs/conventions#data-directory).
StorageRaw json.RawMessage `json:"storage,omitempty" caddy:"namespace=caddy.storage inline_key=module"`
// AppsRaw are the apps that Caddy will load and run. The
// app module name is the key, and the app's config is the
// associated value.
AppsRaw ModuleMap `json:"apps,omitempty" caddy:"namespace="`
apps map[string]App
storage certmagic.Storage
cancelFunc context.CancelFunc
}
// App is a thing that Caddy runs.
type App interface {
Start() error
Stop() error
}
// Run runs the given config, replacing any existing config.
func Run(cfg *Config) error {
cfgJSON, err := json.Marshal(cfg)
if err != nil {
return err
}
return Load(cfgJSON, true)
}
// Load loads the given config JSON and runs it only
// if it is different from the current config or
// forceReload is true.
func Load(cfgJSON []byte, forceReload bool) error {
return changeConfig(http.MethodPost, "/"+rawConfigKey, cfgJSON, forceReload)
}
// changeConfig changes the current config (rawCfg) according to the
// method, traversed via the given path, and uses the given input as
// the new value (if applicable; i.e. "DELETE" doesn't have an input).
// If the resulting config is the same as the previous, no reload will
// occur unless forceReload is true. This function is safe for
// concurrent use.
func changeConfig(method, path string, input []byte, forceReload bool) error {
switch method {
case http.MethodGet,
http.MethodHead,
http.MethodOptions,
http.MethodConnect,
http.MethodTrace:
return fmt.Errorf("method not allowed")
}
currentCfgMu.Lock()
defer currentCfgMu.Unlock()
err := unsyncedConfigAccess(method, path, input, nil)
if err != nil {
return err
}
// the mutation is complete, so encode the entire config as JSON
newCfg, err := json.Marshal(rawCfg[rawConfigKey])
if err != nil {
return APIError{
Code: http.StatusBadRequest,
Err: fmt.Errorf("encoding new config: %v", err),
}
}
// if nothing changed, no need to do a whole reload unless the client forces it
if !forceReload && bytes.Equal(rawCfgJSON, newCfg) {
Log().Named("admin.api").Info("config is unchanged")
return nil
}
// find any IDs in this config and index them
idx := make(map[string]string)
err = indexConfigObjects(rawCfg[rawConfigKey], "/"+rawConfigKey, idx)
if err != nil {
return APIError{
Code: http.StatusInternalServerError,
Err: fmt.Errorf("indexing config: %v", err),
}
}
// load this new config; if it fails, we need to revert to
// our old representation of caddy's actual config
err = unsyncedDecodeAndRun(newCfg)
if err != nil {
if len(rawCfgJSON) > 0 {
// restore old config state to keep it consistent
// with what caddy is still running; we need to
// unmarshal it again because it's likely that
// pointers deep in our rawCfg map were modified
var oldCfg interface{}
err2 := json.Unmarshal(rawCfgJSON, &oldCfg)
if err2 != nil {
err = fmt.Errorf("%v; additionally, restoring old config: %v", err, err2)
}
rawCfg[rawConfigKey] = oldCfg
}
return fmt.Errorf("loading new config: %v", err)
}
// success, so update our stored copy of the encoded
// config to keep it consistent with what caddy is now
// running (storing an encoded copy is not strictly
// necessary, but avoids an extra json.Marshal for
// each config change)
rawCfgJSON = newCfg
rawCfgIndex = idx
return nil
}
// readConfig traverses the current config to path
// and writes its JSON encoding to out.
func readConfig(path string, out io.Writer) error {
currentCfgMu.RLock()
defer currentCfgMu.RUnlock()
return unsyncedConfigAccess(http.MethodGet, path, nil, out)
}
// indexConfigObjects recursively searches ptr for object fields named
// "@id" and maps that ID value to the full configPath in the index.
// This function is NOT safe for concurrent access; obtain a write lock
// on currentCfgMu.
func indexConfigObjects(ptr interface{}, configPath string, index map[string]string) error {
switch val := ptr.(type) {
case map[string]interface{}:
for k, v := range val {
if k == idKey {
switch idVal := v.(type) {
case string:
index[idVal] = configPath
case float64: // all JSON numbers decode as float64
index[fmt.Sprintf("%v", idVal)] = configPath
default:
return fmt.Errorf("%s: %s field must be a string or number", configPath, idKey)
}
continue
}
// traverse this object property recursively
err := indexConfigObjects(val[k], path.Join(configPath, k), index)
if err != nil {
return err
}
}
case []interface{}:
// traverse each element of the array recursively
for i := range val {
err := indexConfigObjects(val[i], path.Join(configPath, strconv.Itoa(i)), index)
if err != nil {
return err
}
}
}
return nil
}
// unsyncedDecodeAndRun removes any meta fields (like @id tags)
// from cfgJSON, decodes the result into a *Config, and runs
// it as the new config, replacing any other current config.
// It does NOT update the raw config state, as this is a
// lower-level function; most callers will want to use Load
// instead. A write lock on currentCfgMu is required!
func unsyncedDecodeAndRun(cfgJSON []byte) error {
// remove any @id fields from the JSON, which would cause
// loading to break since the field wouldn't be recognized
strippedCfgJSON := RemoveMetaFields(cfgJSON)
var newCfg *Config
err := strictUnmarshalJSON(strippedCfgJSON, &newCfg)
if err != nil {
return err
}
// run the new config and start all its apps
err = run(newCfg, true)
if err != nil {
return err
}
// swap old config with the new one
oldCfg := currentCfg
currentCfg = newCfg
// Stop, Cleanup each old app
unsyncedStop(oldCfg)
// autosave a non-nil config, if not disabled
if newCfg != nil &&
(newCfg.Admin == nil ||
newCfg.Admin.Config == nil ||
newCfg.Admin.Config.Persist == nil ||
*newCfg.Admin.Config.Persist) {
dir := filepath.Dir(ConfigAutosavePath)
err := os.MkdirAll(dir, 0700)
if err != nil {
Log().Error("unable to create folder for config autosave",
zap.String("dir", dir),
zap.Error(err))
} else {
err := ioutil.WriteFile(ConfigAutosavePath, cfgJSON, 0600)
if err == nil {
Log().Info("autosaved config", zap.String("file", ConfigAutosavePath))
} else {
Log().Error("unable to autosave config",
zap.String("file", ConfigAutosavePath),
zap.Error(err))
}
}
}
return nil
}
// run runs newCfg and starts all its apps if
// start is true. If any errors happen, cleanup
// is performed if any modules were provisioned;
// apps that were started already will be stopped,
// so this function should not leak resources if
// an error is returned. However, if no error is
// returned and start == false, you should cancel
// the config if you are not going to start it,
// so that each provisioned module will be
// cleaned up.
//
// This is a low-level function; most callers
// will want to use Run instead, which also
// updates the config's raw state.
func run(newCfg *Config, start bool) error {
// because we will need to roll back any state
// modifications if this function errors, we
// keep a single error value and scope all
// sub-operations to their own functions to
// ensure this error value does not get
// overridden or missed when it should have
// been set by a short assignment
var err error
// start the admin endpoint (and stop any prior one)
if start {
err = replaceAdmin(newCfg)
if err != nil {
return fmt.Errorf("starting caddy administration endpoint: %v", err)
}
}
if newCfg == nil {
return nil
}
// prepare the new config for use
newCfg.apps = make(map[string]App)
// create a context within which to load
// modules - essentially our new config's
// execution environment; be sure that
// cleanup occurs when we return if there
// was an error; if no error, it will get
// cleaned up on next config cycle
ctx, cancel := NewContext(Context{Context: context.Background(), cfg: newCfg})
defer func() {
if err != nil {
// if there were any errors during startup,
// we should cancel the new context we created
// since the associated config won't be used;
// this will cause all modules that were newly
// provisioned to clean themselves up
cancel()
// also undo any other state changes we made
if currentCfg != nil {
certmagic.Default.Storage = currentCfg.storage
}
}
}()
newCfg.cancelFunc = cancel // clean up later
// set up logging before anything bad happens
if newCfg.Logging == nil {
newCfg.Logging = new(Logging)
}
err = newCfg.Logging.openLogs(ctx)
if err != nil {
return err
}
// set up global storage and make it CertMagic's default storage, too
err = func() error {
if newCfg.StorageRaw != nil {
val, err := ctx.LoadModule(newCfg, "StorageRaw")
if err != nil {
return fmt.Errorf("loading storage module: %v", err)
}
stor, err := val.(StorageConverter).CertMagicStorage()
if err != nil {
return fmt.Errorf("creating storage value: %v", err)
}
newCfg.storage = stor
}
if newCfg.storage == nil {
newCfg.storage = &certmagic.FileStorage{Path: AppDataDir()}
}
certmagic.Default.Storage = newCfg.storage
return nil
}()
if err != nil {
return err
}
// Load and Provision each app and their submodules
err = func() error {
for appName := range newCfg.AppsRaw {
if _, err := ctx.App(appName); err != nil {
return err
}
}
return nil
}()
if err != nil {
return err
}
if !start {
return nil
}
// Start
return func() error {
var started []string
for name, a := range newCfg.apps {
err := a.Start()
if err != nil {
// an app failed to start, so we need to stop
// all other apps that were already started
for _, otherAppName := range started {
err2 := newCfg.apps[otherAppName].Stop()
if err2 != nil {
err = fmt.Errorf("%v; additionally, aborting app %s: %v",
err, otherAppName, err2)
}
}
return fmt.Errorf("%s app module: start: %v", name, err)
}
started = append(started, name)
}
return nil
}()
}
// Stop stops running the current configuration.
// It is the antithesis of Run(). This function
// will log any errors that occur during the
// stopping of individual apps and continue to
// stop the others. Stop should only be called
// if not replacing with a new config.
func Stop() error {
currentCfgMu.Lock()
defer currentCfgMu.Unlock()
unsyncedStop(currentCfg)
currentCfg = nil
rawCfgJSON = nil
rawCfgIndex = nil
rawCfg[rawConfigKey] = nil
return nil
}
// unsyncedStop stops cfg from running, but has
// no locking around cfg. It is a no-op if cfg is
// nil. If any app returns an error when stopping,
// it is logged and the function continues stopping
// the next app. This function assumes all apps in
// cfg were successfully started first.
func unsyncedStop(cfg *Config) {
if cfg == nil {
return
}
// stop each app
for name, a := range cfg.apps {
err := a.Stop()
if err != nil {
log.Printf("[ERROR] stop %s: %v", name, err)
}
}
// clean up all modules
cfg.cancelFunc()
}
// stopAndCleanup calls stop and cleans up anything
// else that is expedient. This should only be used
// when stopping and not replacing with a new config.
func stopAndCleanup() error {
if err := Stop(); err != nil {
return err
}
certmagic.CleanUpOwnLocks()
return nil
}
// Validate loads, provisions, and validates
// cfg, but does not start running it.
func Validate(cfg *Config) error {
err := run(cfg, false)
if err == nil {
cfg.cancelFunc() // call Cleanup on all modules
}
return err
}
// Duration can be an integer or a string. An integer is
// interpreted as nanoseconds. If a string, it is a Go
// time.Duration value such as `300ms`, `1.5h`, or `2h45m`;
// valid units are `ns`, `us`/`µs`, `ms`, `s`, `m`, and `h`.
type Duration time.Duration
// UnmarshalJSON satisfies json.Unmarshaler.
func (d *Duration) UnmarshalJSON(b []byte) error {
if len(b) == 0 {
return io.EOF
}
var dur time.Duration
var err error
if b[0] == byte('"') && b[len(b)-1] == byte('"') {
dur, err = time.ParseDuration(strings.Trim(string(b), `"`))
} else {
err = json.Unmarshal(b, &dur)
}
*d = Duration(dur)
return err
}
// GoModule returns the build info of this Caddy
// build from debug.BuildInfo (requires Go modules).
// If no version information is available, a non-nil
// value will still be returned, but with an
// unknown version.
func GoModule() *debug.Module {
var mod debug.Module
return goModule(&mod)
}
// goModule holds the actual implementation of GoModule.
// Allocating debug.Module in GoModule() and passing a
// reference to goModule enables mid-stack inlining.
func goModule(mod *debug.Module) *debug.Module {
mod.Version = "unknown"
bi, ok := debug.ReadBuildInfo()
if ok {
mod.Path = bi.Main.Path
// The recommended way to build Caddy involves
// creating a separate main module, which
// TODO: track related Go issue: https://github.com/golang/go/issues/29228
// once that issue is fixed, we should just be able to use bi.Main... hopefully.
for _, dep := range bi.Deps {
if dep.Path == ImportPath {
return dep
}
}
return &bi.Main
}
return mod
}
// CtxKey is a value type for use with context.WithValue.
type CtxKey string
// This group of variables pertains to the current configuration.
var (
// currentCfgMu protects everything in this var block.
currentCfgMu sync.RWMutex
// currentCfg is the currently-running configuration.
currentCfg *Config
// rawCfg is the current, generic-decoded configuration;
// we initialize it as a map with one field ("config")
// to maintain parity with the API endpoint and to avoid
// the special case of having to access/mutate the variable
// directly without traversing into it.
rawCfg = map[string]interface{}{
rawConfigKey: nil,
}
// rawCfgJSON is the JSON-encoded form of rawCfg. Keeping
// this around avoids an extra Marshal call during changes.
rawCfgJSON []byte
// rawCfgIndex is the map of user-assigned ID to expanded
// path, for converting /id/ paths to /config/ paths.
rawCfgIndex map[string]string
)
// ImportPath is the package import path for Caddy core.
const ImportPath = "github.com/caddyserver/caddy/v2"
+86
View File
@@ -0,0 +1,86 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"encoding/json"
"fmt"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
)
// Adapter adapts Caddyfile to Caddy JSON.
type Adapter struct {
ServerType ServerType
}
// Adapt converts the Caddyfile config in body to Caddy JSON.
func (a Adapter) Adapt(body []byte, options map[string]interface{}) ([]byte, []caddyconfig.Warning, error) {
if a.ServerType == nil {
return nil, nil, fmt.Errorf("no server type")
}
if options == nil {
options = make(map[string]interface{})
}
filename, _ := options["filename"].(string)
if filename == "" {
filename = "Caddyfile"
}
serverBlocks, err := Parse(filename, body)
if err != nil {
return nil, nil, err
}
cfg, warnings, err := a.ServerType.Setup(serverBlocks, options)
if err != nil {
return nil, warnings, err
}
marshalFunc := json.Marshal
if options["pretty"] == "true" {
marshalFunc = caddyconfig.JSONIndent
}
result, err := marshalFunc(cfg)
return result, warnings, err
}
// Unmarshaler is a type that can unmarshal
// Caddyfile tokens to set itself up for a
// JSON encoding. The goal of an unmarshaler
// is not to set itself up for actual use,
// but to set itself up for being marshaled
// into JSON. Caddyfile-unmarshaled values
// will not be used directly; they will be
// encoded as JSON and then used from that.
type Unmarshaler interface {
UnmarshalCaddyfile(d *Dispenser) error
}
// ServerType is a type that can evaluate a Caddyfile and set up a caddy config.
type ServerType interface {
// Setup takes the server blocks which
// contain tokens, as well as options
// (e.g. CLI flags) and creates a Caddy
// config, along with any warnings or
// an error.
Setup([]ServerBlock, map[string]interface{}) (*caddy.Config, []caddyconfig.Warning, error)
}
// Interface guard
var _ caddyconfig.Adapter = (*Adapter)(nil)
+384
View File
@@ -0,0 +1,384 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"errors"
"fmt"
"strings"
)
// Dispenser is a type that dispenses tokens, similarly to a lexer,
// except that it can do so with some notion of structure. An empty
// Dispenser is invalid; call NewDispenser to make a proper instance.
type Dispenser struct {
tokens []Token
cursor int
nesting int
}
// NewDispenser returns a Dispenser filled with the given tokens.
func NewDispenser(tokens []Token) *Dispenser {
return &Dispenser{
tokens: tokens,
cursor: -1,
}
}
// Next loads the next token. Returns true if a token
// was loaded; false otherwise. If false, all tokens
// have been consumed.
func (d *Dispenser) Next() bool {
if d.cursor < len(d.tokens)-1 {
d.cursor++
return true
}
return false
}
// Prev moves to the previous token. It does the inverse
// of Next(), except this function may decrement the cursor
// to -1 so that the next call to Next() points to the
// first token; this allows dispensing to "start over". This
// method returns true if the cursor ends up pointing to a
// valid token.
func (d *Dispenser) Prev() bool {
if d.cursor > -1 {
d.cursor--
return d.cursor > -1
}
return false
}
// NextArg loads the next token if it is on the same
// line and if it is not a block opening (open curly
// brace). Returns true if an argument token was
// loaded; false otherwise. If false, all tokens on
// the line have been consumed except for potentially
// a block opening. It handles imported tokens
// correctly.
func (d *Dispenser) NextArg() bool {
if !d.nextOnSameLine() {
return false
}
if d.Val() == "{" {
// roll back; a block opening is not an argument
d.cursor--
return false
}
return true
}
// nextOnSameLine advances the cursor if the next
// token is on the same line of the same file.
func (d *Dispenser) nextOnSameLine() bool {
if d.cursor < 0 {
d.cursor++
return true
}
if d.cursor >= len(d.tokens) {
return false
}
if d.cursor < len(d.tokens)-1 &&
d.tokens[d.cursor].File == d.tokens[d.cursor+1].File &&
d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) == d.tokens[d.cursor+1].Line {
d.cursor++
return true
}
return false
}
// NextLine loads the next token only if it is not on the same
// line as the current token, and returns true if a token was
// loaded; false otherwise. If false, there is not another token
// or it is on the same line. It handles imported tokens correctly.
func (d *Dispenser) NextLine() bool {
if d.cursor < 0 {
d.cursor++
return true
}
if d.cursor >= len(d.tokens) {
return false
}
if d.cursor < len(d.tokens)-1 &&
(d.tokens[d.cursor].File != d.tokens[d.cursor+1].File ||
d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) < d.tokens[d.cursor+1].Line) {
d.cursor++
return true
}
return false
}
// NextBlock can be used as the condition of a for loop
// to load the next token as long as it opens a block or
// is already in a block nested more than initialNestingLevel.
// In other words, a loop over NextBlock() will iterate
// all tokens in the block assuming the next token is an
// open curly brace, until the matching closing brace.
// The open and closing brace tokens for the outer-most
// block will be consumed internally and omitted from
// the iteration.
//
// Proper use of this method looks like this:
//
// for nesting := d.Nesting(); d.NextBlock(nesting); {
// }
//
// However, in simple cases where it is known that the
// Dispenser is new and has not already traversed state
// by a loop over NextBlock(), this will do:
//
// for d.NextBlock(0) {
// }
//
// As with other token parsing logic, a loop over
// NextBlock() should be contained within a loop over
// Next(), as it is usually prudent to skip the initial
// token.
func (d *Dispenser) NextBlock(initialNestingLevel int) bool {
if d.nesting > initialNestingLevel {
if !d.Next() {
return false // should be EOF error
}
if d.Val() == "}" && !d.nextOnSameLine() {
d.nesting--
} else if d.Val() == "{" && !d.nextOnSameLine() {
d.nesting++
}
return d.nesting > initialNestingLevel
}
if !d.nextOnSameLine() { // block must open on same line
return false
}
if d.Val() != "{" {
d.cursor-- // roll back if not opening brace
return false
}
d.Next() // consume open curly brace
if d.Val() == "}" {
return false // open and then closed right away
}
d.nesting++
return true
}
// Nesting returns the current nesting level. Necessary
// if using NextBlock()
func (d *Dispenser) Nesting() int {
return d.nesting
}
// Val gets the text of the current token. If there is no token
// loaded, it returns empty string.
func (d *Dispenser) Val() string {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return ""
}
return d.tokens[d.cursor].Text
}
// Line gets the line number of the current token.
// If there is no token loaded, it returns 0.
func (d *Dispenser) Line() int {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return 0
}
return d.tokens[d.cursor].Line
}
// File gets the filename where the current token originated.
func (d *Dispenser) File() string {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return ""
}
return d.tokens[d.cursor].File
}
// Args is a convenience function that loads the next arguments
// (tokens on the same line) into an arbitrary number of strings
// pointed to in targets. If there are not enough argument tokens
// available to fill targets, false is returned and the remaining
// targets are left unchanged. If all the targets are filled,
// then true is returned.
func (d *Dispenser) Args(targets ...*string) bool {
for i := 0; i < len(targets); i++ {
if !d.NextArg() {
return false
}
*targets[i] = d.Val()
}
return true
}
// AllArgs is like Args, but if there are more argument tokens
// available than there are targets, false is returned. The
// number of available argument tokens must match the number of
// targets exactly to return true.
func (d *Dispenser) AllArgs(targets ...*string) bool {
if !d.Args(targets...) {
return false
}
if d.NextArg() {
d.Prev()
return false
}
return true
}
// RemainingArgs loads any more arguments (tokens on the same line)
// into a slice and returns them. Open curly brace tokens also indicate
// the end of arguments, and the curly brace is not included in
// the return value nor is it loaded.
func (d *Dispenser) RemainingArgs() []string {
var args []string
for d.NextArg() {
args = append(args, d.Val())
}
return args
}
// NewFromNextSegment returns a new dispenser with a copy of
// the tokens from the current token until the end of the
// "directive" whether that be to the end of the line or
// the end of a block that starts at the end of the line;
// in other words, until the end of the segment.
func (d *Dispenser) NewFromNextSegment() *Dispenser {
return NewDispenser(d.NextSegment())
}
// NextSegment returns a copy of the tokens from the current
// token until the end of the line or block that starts at
// the end of the line.
func (d *Dispenser) NextSegment() Segment {
tkns := Segment{d.Token()}
for d.NextArg() {
tkns = append(tkns, d.Token())
}
var openedBlock bool
for nesting := d.Nesting(); d.NextBlock(nesting); {
if !openedBlock {
// because NextBlock() consumes the initial open
// curly brace, we rewind here to append it, since
// our case is special in that we want the new
// dispenser to have all the tokens including
// surrounding curly braces
d.Prev()
tkns = append(tkns, d.Token())
d.Next()
openedBlock = true
}
tkns = append(tkns, d.Token())
}
if openedBlock {
// include closing brace
tkns = append(tkns, d.Token())
// do not consume the closing curly brace; the
// next iteration of the enclosing loop will
// call Next() and consume it
}
return tkns
}
// Token returns the current token.
func (d *Dispenser) Token() Token {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return Token{}
}
return d.tokens[d.cursor]
}
// Reset sets d's cursor to the beginning, as
// if this was a new and unused dispenser.
func (d *Dispenser) Reset() {
d.cursor = -1
d.nesting = 0
}
// ArgErr returns an argument error, meaning that another
// argument was expected but not found. In other words,
// a line break or open curly brace was encountered instead of
// an argument.
func (d *Dispenser) ArgErr() error {
if d.Val() == "{" {
return d.Err("Unexpected token '{', expecting argument")
}
return d.Errf("Wrong argument count or unexpected line ending after '%s'", d.Val())
}
// SyntaxErr creates a generic syntax error which explains what was
// found and what was expected.
func (d *Dispenser) SyntaxErr(expected string) error {
msg := fmt.Sprintf("%s:%d - Syntax error: Unexpected token '%s', expecting '%s'", d.File(), d.Line(), d.Val(), expected)
return errors.New(msg)
}
// EOFErr returns an error indicating that the dispenser reached
// the end of the input when searching for the next token.
func (d *Dispenser) EOFErr() error {
return d.Errf("Unexpected EOF")
}
// Err generates a custom parse-time error with a message of msg.
func (d *Dispenser) Err(msg string) error {
msg = fmt.Sprintf("%s:%d - Error during parsing: %s", d.File(), d.Line(), msg)
return errors.New(msg)
}
// Errf is like Err, but for formatted error messages
func (d *Dispenser) Errf(format string, args ...interface{}) error {
return d.Err(fmt.Sprintf(format, args...))
}
// Delete deletes the current token and returns the updated slice
// of tokens. The cursor is not advanced to the next token.
// Because deletion modifies the underlying slice, this method
// should only be called if you have access to the original slice
// of tokens and/or are using the slice of tokens outside this
// Dispenser instance. If you do not re-assign the slice with the
// return value of this method, inconsistencies in the token
// array will become apparent (or worse, hide from you like they
// did me for 3 and a half freaking hours late one night).
func (d *Dispenser) Delete() []Token {
if d.cursor >= 0 && d.cursor <= len(d.tokens)-1 {
d.tokens = append(d.tokens[:d.cursor], d.tokens[d.cursor+1:]...)
d.cursor--
}
return d.tokens
}
// numLineBreaks counts how many line breaks are in the token
// value given by the token index tknIdx. It returns 0 if the
// token does not exist or there are no line breaks.
func (d *Dispenser) numLineBreaks(tknIdx int) int {
if tknIdx < 0 || tknIdx >= len(d.tokens) {
return 0
}
return strings.Count(d.tokens[tknIdx].Text, "\n")
}
// isNewLine determines whether the current token is on a different
// line (higher line number) than the previous token. It handles imported
// tokens correctly. If there isn't a previous token, it returns true.
func (d *Dispenser) isNewLine() bool {
if d.cursor < 1 {
return true
}
if d.cursor > len(d.tokens)-1 {
return false
}
return d.tokens[d.cursor-1].File != d.tokens[d.cursor].File ||
d.tokens[d.cursor-1].Line+d.numLineBreaks(d.cursor-1) < d.tokens[d.cursor].Line
}
+37 -14
View File
@@ -1,6 +1,22 @@
package parse
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"io"
"log"
"reflect"
"strings"
"testing"
@@ -11,7 +27,7 @@ func TestDispenser_Val_Next(t *testing.T) {
dir1 arg1
dir2 arg2 arg3
dir3`
d := NewDispenser("Testfile", strings.NewReader(input))
d := newTestDispenser(input)
if val := d.Val(); val != "" {
t.Fatalf("Val(): Should return empty string when no token loaded; got '%s'", val)
@@ -49,7 +65,7 @@ func TestDispenser_NextArg(t *testing.T) {
input := `dir1 arg1
dir2 arg2 arg3
dir3`
d := NewDispenser("Testfile", strings.NewReader(input))
d := newTestDispenser(input)
assertNext := func(shouldLoad bool, expectedVal string, expectedCursor int) {
if d.Next() != shouldLoad {
@@ -64,7 +80,7 @@ func TestDispenser_NextArg(t *testing.T) {
}
assertNextArg := func(expectedVal string, loadAnother bool, expectedCursor int) {
if d.NextArg() != true {
if !d.NextArg() {
t.Error("NextArg(): Should load next argument but got false instead")
}
if d.cursor != expectedCursor {
@@ -74,7 +90,7 @@ func TestDispenser_NextArg(t *testing.T) {
t.Errorf("Val(): Expected '%s' but got '%s'", expectedVal, val)
}
if !loadAnother {
if d.NextArg() != false {
if d.NextArg() {
t.Fatalf("NextArg(): Should NOT load another argument, but got true instead (val: '%s')", d.Val())
}
if d.cursor != expectedCursor {
@@ -96,7 +112,7 @@ func TestDispenser_NextLine(t *testing.T) {
input := `host:port
dir1 arg1
dir2 arg2 arg3`
d := NewDispenser("Testfile", strings.NewReader(input))
d := newTestDispenser(input)
assertNextLine := func(shouldLoad bool, expectedVal string, expectedCursor int) {
if d.NextLine() != shouldLoad {
@@ -129,10 +145,10 @@ func TestDispenser_NextBlock(t *testing.T) {
}
foobar2 {
}`
d := NewDispenser("Testfile", strings.NewReader(input))
d := newTestDispenser(input)
assertNextBlock := func(shouldLoad bool, expectedCursor, expectedNesting int) {
if loaded := d.NextBlock(); loaded != shouldLoad {
if loaded := d.NextBlock(0); loaded != shouldLoad {
t.Errorf("NextBlock(): Should return %v but got %v", shouldLoad, loaded)
}
if d.cursor != expectedCursor {
@@ -149,9 +165,8 @@ func TestDispenser_NextBlock(t *testing.T) {
assertNextBlock(true, 3, 1)
assertNextBlock(true, 4, 1)
assertNextBlock(false, 5, 0)
d.Next() // foobar2
assertNextBlock(true, 8, 1)
assertNextBlock(false, 8, 0)
d.Next() // foobar2
assertNextBlock(false, 8, 0) // empty block is as if it didn't exist
}
func TestDispenser_Args(t *testing.T) {
@@ -160,7 +175,7 @@ func TestDispenser_Args(t *testing.T) {
dir2 arg4 arg5
dir3 arg6 arg7
dir4`
d := NewDispenser("Testfile", strings.NewReader(input))
d := newTestDispenser(input)
d.Next() // dir1
@@ -227,7 +242,7 @@ func TestDispenser_RemainingArgs(t *testing.T) {
dir2 arg4 arg5
dir3 arg6 { arg7
dir4`
d := NewDispenser("Testfile", strings.NewReader(input))
d := newTestDispenser(input)
d.Next() // dir1
@@ -264,7 +279,7 @@ func TestDispenser_ArgErr_Err(t *testing.T) {
input := `dir1 {
}
dir2 arg1 arg2`
d := NewDispenser("Testfile", strings.NewReader(input))
d := newTestDispenser(input)
d.cursor = 1 // {
@@ -291,3 +306,11 @@ func TestDispenser_ArgErr_Err(t *testing.T) {
t.Errorf("Expected error message with custom message in it ('foobar'); got '%v'", err)
}
}
func newTestDispenser(input string) *Dispenser {
tokens, err := allTokens("Testfile", []byte(input))
if err != nil && err != io.EOF {
log.Fatalf("getting all tokens from input: %v", err)
}
return NewDispenser(tokens)
}
+140
View File
@@ -0,0 +1,140 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"bytes"
"io"
"unicode"
)
// Format formats a Caddyfile to conventional standards.
func Format(body []byte) []byte {
reader := bytes.NewReader(body)
result := new(bytes.Buffer)
var (
commented,
quoted,
escaped,
environ,
lineBegin bool
firstIteration = true
indentation = 0
prev,
curr,
next rune
err error
)
for {
prev = curr
curr = next
if curr < 0 {
break
}
next, _, err = reader.ReadRune()
if err != nil {
if err == io.EOF {
next = -1
} else {
panic(err)
}
}
if firstIteration {
firstIteration = false
lineBegin = true
continue
}
if quoted {
if escaped {
escaped = false
} else {
if curr == '\\' {
escaped = true
}
if curr == '"' {
quoted = false
}
}
if curr == '\n' {
quoted = false
}
} else if commented {
if curr == '\n' {
commented = false
}
} else {
if curr == '"' {
quoted = true
}
if curr == '#' {
commented = true
}
if curr == '}' {
if environ {
environ = false
} else if indentation > 0 {
indentation--
}
}
if curr == '{' {
if unicode.IsSpace(next) {
indentation++
if !unicode.IsSpace(prev) {
result.WriteRune(' ')
}
} else {
environ = true
}
}
if lineBegin {
if curr == ' ' || curr == '\t' {
continue
} else {
lineBegin = false
if indentation > 0 {
for tabs := indentation; tabs > 0; tabs-- {
result.WriteRune('\t')
}
}
}
} else {
if prev == '{' &&
(curr == ' ' || curr == '\t') &&
(next != '\n' && next != '\r') {
curr = '\n'
}
}
}
if curr == '\n' {
lineBegin = true
}
result.WriteRune(curr)
}
return result.Bytes()
}
+195
View File
@@ -0,0 +1,195 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"testing"
)
func TestFormatBasicIndentation(t *testing.T) {
input := []byte(`
a
b
c {
d
}
e { f
}
g {
h {
i
}
}
j { k {
l
}
}
m {
n { o
}
}
`)
expected := []byte(`
a
b
c {
d
}
e {
f
}
g {
h {
i
}
}
j {
k {
l
}
}
m {
n {
o
}
}
`)
testFormat(t, input, expected)
}
func TestFormatBasicSpacing(t *testing.T) {
input := []byte(`
a{
b
}
c{ d
}
`)
expected := []byte(`
a {
b
}
c {
d
}
`)
testFormat(t, input, expected)
}
func TestFormatEnvironmentVariable(t *testing.T) {
input := []byte(`
{$A}
b {
{$C}
}
d { {$E}
}
`)
expected := []byte(`
{$A}
b {
{$C}
}
d {
{$E}
}
`)
testFormat(t, input, expected)
}
func TestFormatComments(t *testing.T) {
input := []byte(`
# a "\n"
# b {
c
}
d {
e # f
# g
}
h { # i
}
`)
expected := []byte(`
# a "\n"
# b {
c
}
d {
e # f
# g
}
h {
# i
}
`)
testFormat(t, input, expected)
}
func TestFormatQuotesAndEscapes(t *testing.T) {
input := []byte(`
"a \"b\" #c
d
e {
"f"
}
g { "h"
}
`)
expected := []byte(`
"a \"b\" #c
d
e {
"f"
}
g {
"h"
}
`)
testFormat(t, input, expected)
}
func testFormat(t *testing.T, input, expected []byte) {
output := Format(input)
if string(output) != string(expected) {
t.Errorf("Expected:\n%s\ngot:\n%s", string(output), string(expected))
}
}
+163
View File
@@ -0,0 +1,163 @@
// Copyright 2015 Light Code Labs, LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"bufio"
"io"
"unicode"
)
type (
// lexer is a utility which can get values, token by
// token, from a Reader. A token is a word, and tokens
// are separated by whitespace. A word can be enclosed
// in quotes if it contains whitespace.
lexer struct {
reader *bufio.Reader
token Token
line int
skippedLines int
}
// Token represents a single parsable unit.
Token struct {
File string
Line int
Text string
}
)
// load prepares the lexer to scan an input for tokens.
// It discards any leading byte order mark.
func (l *lexer) load(input io.Reader) error {
l.reader = bufio.NewReader(input)
l.line = 1
// discard byte order mark, if present
firstCh, _, err := l.reader.ReadRune()
if err != nil {
return err
}
if firstCh != 0xFEFF {
err := l.reader.UnreadRune()
if err != nil {
return err
}
}
return nil
}
// next loads the next token into the lexer.
// A token is delimited by whitespace, unless
// the token starts with a quotes character (")
// in which case the token goes until the closing
// quotes (the enclosing quotes are not included).
// Inside quoted strings, quotes may be escaped
// with a preceding \ character. No other chars
// may be escaped. The rest of the line is skipped
// if a "#" character is read in. Returns true if
// a token was loaded; false otherwise.
func (l *lexer) next() bool {
var val []rune
var comment, quoted, escaped bool
makeToken := func() bool {
l.token.Text = string(val)
return true
}
for {
ch, _, err := l.reader.ReadRune()
if err != nil {
if len(val) > 0 {
return makeToken()
}
if err == io.EOF {
return false
}
panic(err)
}
if !escaped && ch == '\\' {
escaped = true
continue
}
if quoted {
if escaped {
// all is literal in quoted area,
// so only escape quotes
if ch != '"' {
val = append(val, '\\')
}
escaped = false
} else {
if ch == '"' {
return makeToken()
}
}
if ch == '\n' {
l.line += 1 + l.skippedLines
l.skippedLines = 0
}
val = append(val, ch)
continue
}
if unicode.IsSpace(ch) {
if ch == '\r' {
continue
}
if ch == '\n' {
if escaped {
l.skippedLines++
escaped = false
} else {
l.line += 1 + l.skippedLines
l.skippedLines = 0
}
comment = false
}
if len(val) > 0 {
return makeToken()
}
continue
}
if ch == '#' {
comment = true
}
if comment {
continue
}
if len(val) == 0 {
l.token = Token{Line: l.line}
if ch == '"' {
quoted = true
continue
}
}
if escaped {
val = append(val, '\\')
escaped = false
}
val = append(val, ch)
}
}
+238
View File
@@ -0,0 +1,238 @@
// Copyright 2015 Light Code Labs, LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"log"
"strings"
"testing"
)
type lexerTestCase struct {
input string
expected []Token
}
func TestLexer(t *testing.T) {
testCases := []lexerTestCase{
{
input: `host:123`,
expected: []Token{
{Line: 1, Text: "host:123"},
},
},
{
input: `host:123
directive`,
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 3, Text: "directive"},
},
},
{
input: `host:123 {
directive
}`,
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 1, Text: "{"},
{Line: 2, Text: "directive"},
{Line: 3, Text: "}"},
},
},
{
input: `host:123 { directive }`,
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 1, Text: "{"},
{Line: 1, Text: "directive"},
{Line: 1, Text: "}"},
},
},
{
input: `host:123 {
#comment
directive
# comment
foobar # another comment
}`,
expected: []Token{
{Line: 1, Text: "host:123"},
{Line: 1, Text: "{"},
{Line: 3, Text: "directive"},
{Line: 5, Text: "foobar"},
{Line: 6, Text: "}"},
},
},
{
input: `a "quoted value" b
foobar`,
expected: []Token{
{Line: 1, Text: "a"},
{Line: 1, Text: "quoted value"},
{Line: 1, Text: "b"},
{Line: 2, Text: "foobar"},
},
},
{
input: `A "quoted \"value\" inside" B`,
expected: []Token{
{Line: 1, Text: "A"},
{Line: 1, Text: `quoted "value" inside`},
{Line: 1, Text: "B"},
},
},
{
input: "An escaped \"newline\\\ninside\" quotes",
expected: []Token{
{Line: 1, Text: "An"},
{Line: 1, Text: "escaped"},
{Line: 1, Text: "newline\\\ninside"},
{Line: 2, Text: "quotes"},
},
},
{
input: "An escaped newline\\\noutside quotes",
expected: []Token{
{Line: 1, Text: "An"},
{Line: 1, Text: "escaped"},
{Line: 1, Text: "newline"},
{Line: 1, Text: "outside"},
{Line: 1, Text: "quotes"},
},
},
{
input: "line1\\\nescaped\nline2\nline3",
expected: []Token{
{Line: 1, Text: "line1"},
{Line: 1, Text: "escaped"},
{Line: 3, Text: "line2"},
{Line: 4, Text: "line3"},
},
},
{
input: "line1\\\nescaped1\\\nescaped2\nline4\nline5",
expected: []Token{
{Line: 1, Text: "line1"},
{Line: 1, Text: "escaped1"},
{Line: 1, Text: "escaped2"},
{Line: 4, Text: "line4"},
{Line: 5, Text: "line5"},
},
},
{
input: `"unescapable\ in quotes"`,
expected: []Token{
{Line: 1, Text: `unescapable\ in quotes`},
},
},
{
input: `"don't\escape"`,
expected: []Token{
{Line: 1, Text: `don't\escape`},
},
},
{
input: `"don't\\escape"`,
expected: []Token{
{Line: 1, Text: `don't\\escape`},
},
},
{
input: `un\escapable`,
expected: []Token{
{Line: 1, Text: `un\escapable`},
},
},
{
input: `A "quoted value with line
break inside" {
foobar
}`,
expected: []Token{
{Line: 1, Text: "A"},
{Line: 1, Text: "quoted value with line\n\t\t\t\t\tbreak inside"},
{Line: 2, Text: "{"},
{Line: 3, Text: "foobar"},
{Line: 4, Text: "}"},
},
},
{
input: `"C:\php\php-cgi.exe"`,
expected: []Token{
{Line: 1, Text: `C:\php\php-cgi.exe`},
},
},
{
input: `empty "" string`,
expected: []Token{
{Line: 1, Text: `empty`},
{Line: 1, Text: ``},
{Line: 1, Text: `string`},
},
},
{
input: "skip those\r\nCR characters",
expected: []Token{
{Line: 1, Text: "skip"},
{Line: 1, Text: "those"},
{Line: 2, Text: "CR"},
{Line: 2, Text: "characters"},
},
},
{
input: "\xEF\xBB\xBF:8080", // test with leading byte order mark
expected: []Token{
{Line: 1, Text: ":8080"},
},
},
}
for i, testCase := range testCases {
actual := tokenize(testCase.input)
lexerCompare(t, i, testCase.expected, actual)
}
}
func tokenize(input string) (tokens []Token) {
l := lexer{}
if err := l.load(strings.NewReader(input)); err != nil {
log.Printf("[ERROR] load failed: %v", err)
}
for l.next() {
tokens = append(tokens, l.token)
}
return
}
func lexerCompare(t *testing.T, n int, expected, actual []Token) {
if len(expected) != len(actual) {
t.Errorf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
}
for i := 0; i < len(actual) && i < len(expected); i++ {
if actual[i].Line != expected[i].Line {
t.Errorf("Test case %d token %d ('%s'): expected line %d but was line %d",
n, i, expected[i].Text, expected[i].Line, actual[i].Line)
break
}
if actual[i].Text != expected[i].Text {
t.Errorf("Test case %d token %d: expected text '%s' but was '%s'",
n, i, expected[i].Text, actual[i].Text)
break
}
}
}
+536
View File
@@ -0,0 +1,536 @@
// Copyright 2015 Light Code Labs, LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"bytes"
"io/ioutil"
"log"
"os"
"path/filepath"
"strings"
)
// Parse parses the input just enough to group tokens, in
// order, by server block. No further parsing is performed.
// Server blocks are returned in the order in which they appear.
// Directives that do not appear in validDirectives will cause
// an error. If you do not want to check for valid directives,
// pass in nil instead.
//
// Environment variables in {$ENVIRONMENT_VARIABLE} notation
// will be replaced before parsing begins.
func Parse(filename string, input []byte) ([]ServerBlock, error) {
tokens, err := allTokens(filename, input)
if err != nil {
return nil, err
}
p := parser{Dispenser: NewDispenser(tokens)}
return p.parseAll()
}
// replaceEnvVars replaces all occurrences of environment variables.
func replaceEnvVars(input []byte) ([]byte, error) {
var offset int
for {
begin := bytes.Index(input[offset:], spanOpen)
if begin < 0 {
break
}
begin += offset // make beginning relative to input, not offset
end := bytes.Index(input[begin+len(spanOpen):], spanClose)
if end < 0 {
break
}
end += begin + len(spanOpen) // make end relative to input, not begin
// get the name; if there is no name, skip it
envVarName := input[begin+len(spanOpen) : end]
if len(envVarName) == 0 {
offset = end + len(spanClose)
continue
}
// get the value of the environment variable
envVarValue := []byte(os.ExpandEnv(os.Getenv(string(envVarName))))
// splice in the value
input = append(input[:begin],
append(envVarValue, input[end+len(spanClose):]...)...)
// continue at the end of the replacement
offset = begin + len(envVarValue)
}
return input, nil
}
// allTokens lexes the entire input, but does not parse it.
// It returns all the tokens from the input, unstructured
// and in order.
func allTokens(filename string, input []byte) ([]Token, error) {
input, err := replaceEnvVars(input)
if err != nil {
return nil, err
}
l := new(lexer)
err = l.load(bytes.NewReader(input))
if err != nil {
return nil, err
}
var tokens []Token
for l.next() {
l.token.File = filename
tokens = append(tokens, l.token)
}
return tokens, nil
}
type parser struct {
*Dispenser
block ServerBlock // current server block being parsed
eof bool // if we encounter a valid EOF in a hard place
definedSnippets map[string][]Token
nesting int
}
func (p *parser) parseAll() ([]ServerBlock, error) {
var blocks []ServerBlock
for p.Next() {
err := p.parseOne()
if err != nil {
return blocks, err
}
if len(p.block.Keys) > 0 || len(p.block.Segments) > 0 {
blocks = append(blocks, p.block)
}
if p.nesting > 0 {
return blocks, p.EOFErr()
}
}
return blocks, nil
}
func (p *parser) parseOne() error {
p.block = ServerBlock{}
return p.begin()
}
func (p *parser) begin() error {
if len(p.tokens) == 0 {
return nil
}
err := p.addresses()
if err != nil {
return err
}
if p.eof {
// this happens if the Caddyfile consists of only
// a line of addresses and nothing else
return nil
}
if ok, name := p.isSnippet(); ok {
if p.definedSnippets == nil {
p.definedSnippets = map[string][]Token{}
}
if _, found := p.definedSnippets[name]; found {
return p.Errf("redeclaration of previously declared snippet %s", name)
}
// consume all tokens til matched close brace
tokens, err := p.snippetTokens()
if err != nil {
return err
}
p.definedSnippets[name] = tokens
// empty block keys so we don't save this block as a real server.
p.block.Keys = nil
return nil
}
return p.blockContents()
}
func (p *parser) addresses() error {
var expectingAnother bool
for {
tkn := p.Val()
// special case: import directive replaces tokens during parse-time
if tkn == "import" && p.isNewLine() {
err := p.doImport()
if err != nil {
return err
}
continue
}
// Open brace definitely indicates end of addresses
if tkn == "{" {
if expectingAnother {
return p.Errf("Expected another address but had '%s' - check for extra comma", tkn)
}
break
}
if tkn != "" { // empty token possible if user typed ""
// Trailing comma indicates another address will follow, which
// may possibly be on the next line
if tkn[len(tkn)-1] == ',' {
tkn = tkn[:len(tkn)-1]
expectingAnother = true
} else {
expectingAnother = false // but we may still see another one on this line
}
p.block.Keys = append(p.block.Keys, tkn)
}
// Advance token and possibly break out of loop or return error
hasNext := p.Next()
if expectingAnother && !hasNext {
return p.EOFErr()
}
if !hasNext {
p.eof = true
break // EOF
}
if !expectingAnother && p.isNewLine() {
break
}
}
return nil
}
func (p *parser) blockContents() error {
errOpenCurlyBrace := p.openCurlyBrace()
if errOpenCurlyBrace != nil {
// single-server configs don't need curly braces
p.cursor--
}
err := p.directives()
if err != nil {
return err
}
// only look for close curly brace if there was an opening
if errOpenCurlyBrace == nil {
err = p.closeCurlyBrace()
if err != nil {
return err
}
}
return nil
}
// directives parses through all the lines for directives
// and it expects the next token to be the first
// directive. It goes until EOF or closing curly brace
// which ends the server block.
func (p *parser) directives() error {
for p.Next() {
// end of server block
if p.Val() == "}" {
// p.nesting has already been decremented
break
}
// special case: import directive replaces tokens during parse-time
if p.Val() == "import" {
err := p.doImport()
if err != nil {
return err
}
p.cursor-- // cursor is advanced when we continue, so roll back one more
continue
}
// normal case: parse a directive as a new segment
// (a "segment" is a line which starts with a directive
// and which ends at the end of the line or at the end of
// the block that is opened at the end of the line)
if err := p.directive(); err != nil {
return err
}
}
return nil
}
// doImport swaps out the import directive and its argument
// (a total of 2 tokens) with the tokens in the specified file
// or globbing pattern. When the function returns, the cursor
// is on the token before where the import directive was. In
// other words, call Next() to access the first token that was
// imported.
func (p *parser) doImport() error {
// syntax checks
if !p.NextArg() {
return p.ArgErr()
}
importPattern := p.Val()
if importPattern == "" {
return p.Err("Import requires a non-empty filepath")
}
if p.NextArg() {
return p.Err("Import takes only one argument (glob pattern or file)")
}
// splice out the import directive and its argument (2 tokens total)
tokensBefore := p.tokens[:p.cursor-1]
tokensAfter := p.tokens[p.cursor+1:]
var importedTokens []Token
// first check snippets. That is a simple, non-recursive replacement
if p.definedSnippets != nil && p.definedSnippets[importPattern] != nil {
importedTokens = p.definedSnippets[importPattern]
} else {
// make path relative to the file of the _token_ being processed rather
// than current working directory (issue #867) and then use glob to get
// list of matching filenames
absFile, err := filepath.Abs(p.Dispenser.File())
if err != nil {
return p.Errf("Failed to get absolute path of file: %s: %v", p.Dispenser.File(), err)
}
var matches []string
var globPattern string
if !filepath.IsAbs(importPattern) {
globPattern = filepath.Join(filepath.Dir(absFile), importPattern)
} else {
globPattern = importPattern
}
if strings.Count(globPattern, "*") > 1 || strings.Count(globPattern, "?") > 1 ||
(strings.Contains(globPattern, "[") && strings.Contains(globPattern, "]")) {
// See issue #2096 - a pattern with many glob expansions can hang for too long
return p.Errf("Glob pattern may only contain one wildcard (*), but has others: %s", globPattern)
}
matches, err = filepath.Glob(globPattern)
if err != nil {
return p.Errf("Failed to use import pattern %s: %v", importPattern, err)
}
if len(matches) == 0 {
if strings.ContainsAny(globPattern, "*?[]") {
log.Printf("[WARNING] No files matching import glob pattern: %s", importPattern)
} else {
return p.Errf("File to import not found: %s", importPattern)
}
}
// collect all the imported tokens
for _, importFile := range matches {
newTokens, err := p.doSingleImport(importFile)
if err != nil {
return err
}
importedTokens = append(importedTokens, newTokens...)
}
}
// splice the imported tokens in the place of the import statement
// and rewind cursor so Next() will land on first imported token
p.tokens = append(tokensBefore, append(importedTokens, tokensAfter...)...)
p.cursor--
return nil
}
// doSingleImport lexes the individual file at importFile and returns
// its tokens or an error, if any.
func (p *parser) doSingleImport(importFile string) ([]Token, error) {
file, err := os.Open(importFile)
if err != nil {
return nil, p.Errf("Could not import %s: %v", importFile, err)
}
defer file.Close()
if info, err := file.Stat(); err != nil {
return nil, p.Errf("Could not import %s: %v", importFile, err)
} else if info.IsDir() {
return nil, p.Errf("Could not import %s: is a directory", importFile)
}
input, err := ioutil.ReadAll(file)
if err != nil {
return nil, p.Errf("Could not read imported file %s: %v", importFile, err)
}
importedTokens, err := allTokens(importFile, input)
if err != nil {
return nil, p.Errf("Could not read tokens while importing %s: %v", importFile, err)
}
// Tack the file path onto these tokens so errors show the imported file's name
// (we use full, absolute path to avoid bugs: issue #1892)
filename, err := filepath.Abs(importFile)
if err != nil {
return nil, p.Errf("Failed to get absolute path of file: %s: %v", importFile, err)
}
for i := 0; i < len(importedTokens); i++ {
importedTokens[i].File = filename
}
return importedTokens, nil
}
// directive collects tokens until the directive's scope
// closes (either end of line or end of curly brace block).
// It expects the currently-loaded token to be a directive
// (or } that ends a server block). The collected tokens
// are loaded into the current server block for later use
// by directive setup functions.
func (p *parser) directive() error {
// a segment is a list of tokens associated with this directive
var segment Segment
// the directive itself is appended as a relevant token
segment = append(segment, p.Token())
for p.Next() {
if p.Val() == "{" {
p.nesting++
} else if p.isNewLine() && p.nesting == 0 {
p.cursor-- // read too far
break
} else if p.Val() == "}" && p.nesting > 0 {
p.nesting--
} else if p.Val() == "}" && p.nesting == 0 {
return p.Err("Unexpected '}' because no matching opening brace")
} else if p.Val() == "import" && p.isNewLine() {
if err := p.doImport(); err != nil {
return err
}
p.cursor-- // cursor is advanced when we continue, so roll back one more
continue
}
segment = append(segment, p.Token())
}
p.block.Segments = append(p.block.Segments, segment)
if p.nesting > 0 {
return p.EOFErr()
}
return nil
}
// openCurlyBrace expects the current token to be an
// opening curly brace. This acts like an assertion
// because it returns an error if the token is not
// a opening curly brace. It does NOT advance the token.
func (p *parser) openCurlyBrace() error {
if p.Val() != "{" {
return p.SyntaxErr("{")
}
return nil
}
// closeCurlyBrace expects the current token to be
// a closing curly brace. This acts like an assertion
// because it returns an error if the token is not
// a closing curly brace. It does NOT advance the token.
func (p *parser) closeCurlyBrace() error {
if p.Val() != "}" {
return p.SyntaxErr("}")
}
return nil
}
func (p *parser) isSnippet() (bool, string) {
keys := p.block.Keys
// A snippet block is a single key with parens. Nothing else qualifies.
if len(keys) == 1 && strings.HasPrefix(keys[0], "(") && strings.HasSuffix(keys[0], ")") {
return true, strings.TrimSuffix(keys[0][1:], ")")
}
return false, ""
}
// read and store everything in a block for later replay.
func (p *parser) snippetTokens() ([]Token, error) {
// snippet must have curlies.
err := p.openCurlyBrace()
if err != nil {
return nil, err
}
nesting := 1 // count our own nesting in snippets
tokens := []Token{}
for p.Next() {
if p.Val() == "}" {
nesting--
if nesting == 0 {
break
}
}
if p.Val() == "{" {
nesting++
}
tokens = append(tokens, p.tokens[p.cursor])
}
// make sure we're matched up
if nesting != 0 {
return nil, p.SyntaxErr("}")
}
return tokens, nil
}
// ServerBlock associates any number of keys from the
// head of the server block with tokens, which are
// grouped by segments.
type ServerBlock struct {
Keys []string
Segments []Segment
}
// DispenseDirective returns a dispenser that contains
// all the tokens in the server block.
func (sb ServerBlock) DispenseDirective(dir string) *Dispenser {
var tokens []Token
for _, seg := range sb.Segments {
if len(seg) > 0 && seg[0].Text == dir {
tokens = append(tokens, seg...)
}
}
return NewDispenser(tokens)
}
// Segment is a list of tokens which begins with a directive
// and ends at the end of the directive (either at the end of
// the line, or at the end of a block it opens).
type Segment []Token
// Directive returns the directive name for the segment.
// The directive name is the text of the first token.
func (s Segment) Directive() string {
if len(s) > 0 {
return s[0].Text
}
return ""
}
// spanOpen and spanClose are used to bound spans that
// contain the name of an environment variable.
var spanOpen, spanClose = []byte{'{', '$'}, []byte{'}'}
+36
View File
@@ -0,0 +1,36 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +build gofuzz
// +build gofuzz_libfuzzer
package caddyfile
import (
"bytes"
)
func FuzzParseCaddyfile(data []byte) (score int) {
sb, err := Parse("Caddyfile", bytes.NewReader(data))
if err != nil {
// if both an error is received and some ServerBlocks,
// then the parse was able to parse partially. Mark this
// result as interesting to push the fuzzer further through the parser.
if sb != nil && len(sb) > 0 {
return 1
}
return 0
}
return 1
}
+674
View File
@@ -0,0 +1,674 @@
// Copyright 2015 Light Code Labs, LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyfile
import (
"bytes"
"io/ioutil"
"os"
"path/filepath"
"testing"
)
func TestAllTokens(t *testing.T) {
input := []byte("a b c\nd e")
expected := []string{"a", "b", "c", "d", "e"}
tokens, err := allTokens("TestAllTokens", input)
if err != nil {
t.Fatalf("Expected no error, got %v", err)
}
if len(tokens) != len(expected) {
t.Fatalf("Expected %d tokens, got %d", len(expected), len(tokens))
}
for i, val := range expected {
if tokens[i].Text != val {
t.Errorf("Token %d should be '%s' but was '%s'", i, val, tokens[i].Text)
}
}
}
func TestParseOneAndImport(t *testing.T) {
testParseOne := func(input string) (ServerBlock, error) {
p := testParser(input)
p.Next() // parseOne doesn't call Next() to start, so we must
err := p.parseOne()
return p.block, err
}
for i, test := range []struct {
input string
shouldErr bool
keys []string
numTokens []int // number of tokens to expect in each segment
}{
{`localhost`, false, []string{
"localhost",
}, []int{}},
{`localhost
dir1`, false, []string{
"localhost",
}, []int{1}},
{`localhost:1234
dir1 foo bar`, false, []string{
"localhost:1234",
}, []int{3},
},
{`localhost {
dir1
}`, false, []string{
"localhost",
}, []int{1}},
{`localhost:1234 {
dir1 foo bar
dir2
}`, false, []string{
"localhost:1234",
}, []int{3, 1}},
{`http://localhost https://localhost
dir1 foo bar`, false, []string{
"http://localhost",
"https://localhost",
}, []int{3}},
{`http://localhost https://localhost {
dir1 foo bar
}`, false, []string{
"http://localhost",
"https://localhost",
}, []int{3}},
{`http://localhost, https://localhost {
dir1 foo bar
}`, false, []string{
"http://localhost",
"https://localhost",
}, []int{3}},
{`http://localhost, {
}`, true, []string{
"http://localhost",
}, []int{}},
{`host1:80, http://host2.com
dir1 foo bar
dir2 baz`, false, []string{
"host1:80",
"http://host2.com",
}, []int{3, 2}},
{`http://host1.com,
http://host2.com,
https://host3.com`, false, []string{
"http://host1.com",
"http://host2.com",
"https://host3.com",
}, []int{}},
{`http://host1.com:1234, https://host2.com
dir1 foo {
bar baz
}
dir2`, false, []string{
"http://host1.com:1234",
"https://host2.com",
}, []int{6, 1}},
{`127.0.0.1
dir1 {
bar baz
}
dir2 {
foo bar
}`, false, []string{
"127.0.0.1",
}, []int{5, 5}},
{`localhost
dir1 {
foo`, true, []string{
"localhost",
}, []int{3}},
{`localhost
dir1 {
}`, false, []string{
"localhost",
}, []int{3}},
{`localhost
dir1 {
} }`, true, []string{
"localhost",
}, []int{}},
{`localhost
dir1 {
nested {
foo
}
}
dir2 foo bar`, false, []string{
"localhost",
}, []int{7, 3}},
{``, false, []string{}, []int{}},
{`localhost
dir1 arg1
import testdata/import_test1.txt`, false, []string{
"localhost",
}, []int{2, 3, 1}},
{`import testdata/import_test2.txt`, false, []string{
"host1",
}, []int{1, 2}},
{`import testdata/import_test1.txt testdata/import_test2.txt`, true, []string{}, []int{}},
{`import testdata/not_found.txt`, true, []string{}, []int{}},
{`""`, false, []string{}, []int{}},
{``, false, []string{}, []int{}},
// test cases found by fuzzing!
{`import }{$"`, true, []string{}, []int{}},
{`import /*/*.txt`, true, []string{}, []int{}},
{`import /???/?*?o`, true, []string{}, []int{}},
{`import /??`, true, []string{}, []int{}},
{`import /[a-z]`, true, []string{}, []int{}},
{`import {$}`, true, []string{}, []int{}},
{`import {%}`, true, []string{}, []int{}},
{`import {$$}`, true, []string{}, []int{}},
{`import {%%}`, true, []string{}, []int{}},
} {
result, err := testParseOne(test.input)
if test.shouldErr && err == nil {
t.Errorf("Test %d: Expected an error, but didn't get one", i)
}
if !test.shouldErr && err != nil {
t.Errorf("Test %d: Expected no error, but got: %v", i, err)
}
if len(result.Keys) != len(test.keys) {
t.Errorf("Test %d: Expected %d keys, got %d",
i, len(test.keys), len(result.Keys))
continue
}
for j, addr := range result.Keys {
if addr != test.keys[j] {
t.Errorf("Test %d, key %d: Expected '%s', but was '%s'",
i, j, test.keys[j], addr)
}
}
if len(result.Segments) != len(test.numTokens) {
t.Errorf("Test %d: Expected %d segments, had %d",
i, len(test.numTokens), len(result.Segments))
continue
}
for j, seg := range result.Segments {
if len(seg) != test.numTokens[j] {
t.Errorf("Test %d, segment %d: Expected %d tokens, counted %d",
i, j, test.numTokens[j], len(seg))
continue
}
}
}
}
func TestRecursiveImport(t *testing.T) {
testParseOne := func(input string) (ServerBlock, error) {
p := testParser(input)
p.Next() // parseOne doesn't call Next() to start, so we must
err := p.parseOne()
return p.block, err
}
isExpected := func(got ServerBlock) bool {
if len(got.Keys) != 1 || got.Keys[0] != "localhost" {
t.Errorf("got keys unexpected: expect localhost, got %v", got.Keys)
return false
}
if len(got.Segments) != 2 {
t.Errorf("got wrong number of segments: expect 2, got %d", len(got.Segments))
return false
}
if len(got.Segments[0]) != 1 || len(got.Segments[1]) != 2 {
t.Errorf("got unexpected tokens: %v", got.Segments)
return false
}
return true
}
recursiveFile1, err := filepath.Abs("testdata/recursive_import_test1")
if err != nil {
t.Fatal(err)
}
recursiveFile2, err := filepath.Abs("testdata/recursive_import_test2")
if err != nil {
t.Fatal(err)
}
// test relative recursive import
err = ioutil.WriteFile(recursiveFile1, []byte(
`localhost
dir1
import recursive_import_test2`), 0644)
if err != nil {
t.Fatal(err)
}
defer os.Remove(recursiveFile1)
err = ioutil.WriteFile(recursiveFile2, []byte("dir2 1"), 0644)
if err != nil {
t.Fatal(err)
}
defer os.Remove(recursiveFile2)
// import absolute path
result, err := testParseOne("import " + recursiveFile1)
if err != nil {
t.Fatal(err)
}
if !isExpected(result) {
t.Error("absolute+relative import failed")
}
// import relative path
result, err = testParseOne("import testdata/recursive_import_test1")
if err != nil {
t.Fatal(err)
}
if !isExpected(result) {
t.Error("relative+relative import failed")
}
// test absolute recursive import
err = ioutil.WriteFile(recursiveFile1, []byte(
`localhost
dir1
import `+recursiveFile2), 0644)
if err != nil {
t.Fatal(err)
}
// import absolute path
result, err = testParseOne("import " + recursiveFile1)
if err != nil {
t.Fatal(err)
}
if !isExpected(result) {
t.Error("absolute+absolute import failed")
}
// import relative path
result, err = testParseOne("import testdata/recursive_import_test1")
if err != nil {
t.Fatal(err)
}
if !isExpected(result) {
t.Error("relative+absolute import failed")
}
}
func TestDirectiveImport(t *testing.T) {
testParseOne := func(input string) (ServerBlock, error) {
p := testParser(input)
p.Next() // parseOne doesn't call Next() to start, so we must
err := p.parseOne()
return p.block, err
}
isExpected := func(got ServerBlock) bool {
if len(got.Keys) != 1 || got.Keys[0] != "localhost" {
t.Errorf("got keys unexpected: expect localhost, got %v", got.Keys)
return false
}
if len(got.Segments) != 2 {
t.Errorf("got wrong number of segments: expect 2, got %d", len(got.Segments))
return false
}
if len(got.Segments[0]) != 1 || len(got.Segments[1]) != 8 {
t.Errorf("got unexpected tokens: %v", got.Segments)
return false
}
return true
}
directiveFile, err := filepath.Abs("testdata/directive_import_test")
if err != nil {
t.Fatal(err)
}
err = ioutil.WriteFile(directiveFile, []byte(`prop1 1
prop2 2`), 0644)
if err != nil {
t.Fatal(err)
}
defer os.Remove(directiveFile)
// import from existing file
result, err := testParseOne(`localhost
dir1
proxy {
import testdata/directive_import_test
transparent
}`)
if err != nil {
t.Fatal(err)
}
if !isExpected(result) {
t.Error("directive import failed")
}
// import from nonexistent file
_, err = testParseOne(`localhost
dir1
proxy {
import testdata/nonexistent_file
transparent
}`)
if err == nil {
t.Fatal("expected error when importing a nonexistent file")
}
}
func TestParseAll(t *testing.T) {
for i, test := range []struct {
input string
shouldErr bool
keys [][]string // keys per server block, in order
}{
{`localhost`, false, [][]string{
{"localhost"},
}},
{`localhost:1234`, false, [][]string{
{"localhost:1234"},
}},
{`localhost:1234 {
}
localhost:2015 {
}`, false, [][]string{
{"localhost:1234"},
{"localhost:2015"},
}},
{`localhost:1234, http://host2`, false, [][]string{
{"localhost:1234", "http://host2"},
}},
{`localhost:1234, http://host2,`, true, [][]string{}},
{`http://host1.com, http://host2.com {
}
https://host3.com, https://host4.com {
}`, false, [][]string{
{"http://host1.com", "http://host2.com"},
{"https://host3.com", "https://host4.com"},
}},
{`import testdata/import_glob*.txt`, false, [][]string{
{"glob0.host0"},
{"glob0.host1"},
{"glob1.host0"},
{"glob2.host0"},
}},
{`import notfound/*`, false, [][]string{}}, // glob needn't error with no matches
{`import notfound/file.conf`, true, [][]string{}}, // but a specific file should
} {
p := testParser(test.input)
blocks, err := p.parseAll()
if test.shouldErr && err == nil {
t.Errorf("Test %d: Expected an error, but didn't get one", i)
}
if !test.shouldErr && err != nil {
t.Errorf("Test %d: Expected no error, but got: %v", i, err)
}
if len(blocks) != len(test.keys) {
t.Errorf("Test %d: Expected %d server blocks, got %d",
i, len(test.keys), len(blocks))
continue
}
for j, block := range blocks {
if len(block.Keys) != len(test.keys[j]) {
t.Errorf("Test %d: Expected %d keys in block %d, got %d",
i, len(test.keys[j]), j, len(block.Keys))
continue
}
for k, addr := range block.Keys {
if addr != test.keys[j][k] {
t.Errorf("Test %d, block %d, key %d: Expected '%s', but got '%s'",
i, j, k, test.keys[j][k], addr)
}
}
}
}
}
func TestEnvironmentReplacement(t *testing.T) {
os.Setenv("FOOBAR", "foobar")
for i, test := range []struct {
input string
expect string
}{
{
input: "",
expect: "",
},
{
input: "foo",
expect: "foo",
},
{
input: "{$NOT_SET}",
expect: "",
},
{
input: "foo{$NOT_SET}bar",
expect: "foobar",
},
{
input: "{$FOOBAR}",
expect: "foobar",
},
{
input: "foo {$FOOBAR} bar",
expect: "foo foobar bar",
},
{
input: "foo{$FOOBAR}bar",
expect: "foofoobarbar",
},
{
input: "foo\n{$FOOBAR}\nbar",
expect: "foo\nfoobar\nbar",
},
{
input: "{$FOOBAR} {$FOOBAR}",
expect: "foobar foobar",
},
{
input: "{$FOOBAR}{$FOOBAR}",
expect: "foobarfoobar",
},
{
input: "{$FOOBAR",
expect: "{$FOOBAR",
},
{
input: "{$LONGER_NAME $FOOBAR}",
expect: "",
},
{
input: "{$}",
expect: "{$}",
},
{
input: "{$$}",
expect: "",
},
{
input: "{$",
expect: "{$",
},
{
input: "}{$",
expect: "}{$",
},
} {
actual, err := replaceEnvVars([]byte(test.input))
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(actual, []byte(test.expect)) {
t.Errorf("Test %d: Expected: '%s' but got '%s'", i, test.expect, actual)
}
}
}
func TestSnippets(t *testing.T) {
p := testParser(`
(common) {
gzip foo
errors stderr
}
http://example.com {
import common
}
`)
blocks, err := p.parseAll()
if err != nil {
t.Fatal(err)
}
for _, b := range blocks {
t.Log(b.Keys)
t.Log(b.Segments)
}
if len(blocks) != 1 {
t.Fatalf("Expect exactly one server block. Got %d.", len(blocks))
}
if actual, expected := blocks[0].Keys[0], "http://example.com"; expected != actual {
t.Errorf("Expected server name to be '%s' but was '%s'", expected, actual)
}
if len(blocks[0].Segments) != 2 {
t.Fatalf("Server block should have tokens from import, got: %+v", blocks[0])
}
if actual, expected := blocks[0].Segments[0][0].Text, "gzip"; expected != actual {
t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
}
if actual, expected := blocks[0].Segments[1][1].Text, "stderr"; expected != actual {
t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
}
}
func writeStringToTempFileOrDie(t *testing.T, str string) (pathToFile string) {
file, err := ioutil.TempFile("", t.Name())
if err != nil {
panic(err) // get a stack trace so we know where this was called from.
}
if _, err := file.WriteString(str); err != nil {
panic(err)
}
if err := file.Close(); err != nil {
panic(err)
}
return file.Name()
}
func TestImportedFilesIgnoreNonDirectiveImportTokens(t *testing.T) {
fileName := writeStringToTempFileOrDie(t, `
http://example.com {
# This isn't an import directive, it's just an arg with value 'import'
basicauth / import password
}
`)
// Parse the root file that imports the other one.
p := testParser(`import ` + fileName)
blocks, err := p.parseAll()
if err != nil {
t.Fatal(err)
}
for _, b := range blocks {
t.Log(b.Keys)
t.Log(b.Segments)
}
auth := blocks[0].Segments[0]
line := auth[0].Text + " " + auth[1].Text + " " + auth[2].Text + " " + auth[3].Text
if line != "basicauth / import password" {
// Previously, it would be changed to:
// basicauth / import /path/to/test/dir/password
// referencing a file that (probably) doesn't exist and changing the
// password!
t.Errorf("Expected basicauth tokens to be 'basicauth / import password' but got %#q", line)
}
}
func TestSnippetAcrossMultipleFiles(t *testing.T) {
// Make the derived Caddyfile that expects (common) to be defined.
fileName := writeStringToTempFileOrDie(t, `
http://example.com {
import common
}
`)
// Parse the root file that defines (common) and then imports the other one.
p := testParser(`
(common) {
gzip foo
}
import ` + fileName + `
`)
blocks, err := p.parseAll()
if err != nil {
t.Fatal(err)
}
for _, b := range blocks {
t.Log(b.Keys)
t.Log(b.Segments)
}
if len(blocks) != 1 {
t.Fatalf("Expect exactly one server block. Got %d.", len(blocks))
}
if actual, expected := blocks[0].Keys[0], "http://example.com"; expected != actual {
t.Errorf("Expected server name to be '%s' but was '%s'", expected, actual)
}
if len(blocks[0].Segments) != 1 {
t.Fatalf("Server block should have tokens from import")
}
if actual, expected := blocks[0].Segments[0][0].Text, "gzip"; expected != actual {
t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
}
}
func testParser(input string) parser {
return parser{Dispenser: newTestDispenser(input)}
}
+6
View File
@@ -0,0 +1,6 @@
glob0.host0 {
dir2 arg1
}
glob0.host1 {
}
+4
View File
@@ -0,0 +1,4 @@
glob1.host0 {
dir1
dir2 arg1
}
+3
View File
@@ -0,0 +1,3 @@
glob2.host0 {
dir2 arg1
}
+2
View File
@@ -0,0 +1,2 @@
dir2 arg1 arg2
dir3
+4
View File
@@ -0,0 +1,4 @@
host1 {
dir1
dir2 arg1
}
+117
View File
@@ -0,0 +1,117 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddyconfig
import (
"encoding/json"
"fmt"
)
// Adapter is a type which can adapt a configuration to Caddy JSON.
// It returns the results and any warnings, or an error.
type Adapter interface {
Adapt(body []byte, options map[string]interface{}) ([]byte, []Warning, error)
}
// Warning represents a warning or notice related to conversion.
type Warning struct {
File string `json:"file,omitempty"`
Line int `json:"line,omitempty"`
Directive string `json:"directive,omitempty"`
Message string `json:"message,omitempty"`
}
// JSON encodes val as JSON, returning it as a json.RawMessage. Any
// marshaling errors (which are highly unlikely with correct code)
// are converted to warnings. This is convenient when filling config
// structs that require a json.RawMessage, without having to worry
// about errors.
func JSON(val interface{}, warnings *[]Warning) json.RawMessage {
b, err := json.Marshal(val)
if err != nil {
if warnings != nil {
*warnings = append(*warnings, Warning{Message: err.Error()})
}
return nil
}
return b
}
// JSONModuleObject is like JSON, except it marshals val into a JSON object
// and then adds a key to that object named fieldName with the value fieldVal.
// This is useful for JSON-encoding module values where the module name has to
// be described within the object by a certain key; for example,
// "responder": "file_server" for a file server HTTP responder. The val must
// encode into a map[string]interface{} (i.e. it must be a struct or map),
// and any errors are converted into warnings, so this can be conveniently
// used when filling a struct. For correct code, there should be no errors.
func JSONModuleObject(val interface{}, fieldName, fieldVal string, warnings *[]Warning) json.RawMessage {
// encode to a JSON object first
enc, err := json.Marshal(val)
if err != nil {
if warnings != nil {
*warnings = append(*warnings, Warning{Message: err.Error()})
}
return nil
}
// then decode the object
var tmp map[string]interface{}
err = json.Unmarshal(enc, &tmp)
if err != nil {
if warnings != nil {
*warnings = append(*warnings, Warning{Message: err.Error()})
}
return nil
}
// so we can easily add the module's field with its appointed value
tmp[fieldName] = fieldVal
// then re-marshal as JSON
result, err := json.Marshal(tmp)
if err != nil {
if warnings != nil {
*warnings = append(*warnings, Warning{Message: err.Error()})
}
return nil
}
return result
}
// JSONIndent is used to JSON-marshal the final resulting Caddy
// configuration in a consistent, human-readable way.
func JSONIndent(val interface{}) ([]byte, error) {
return json.MarshalIndent(val, "", "\t")
}
// RegisterAdapter registers a config adapter with the given name.
// This should usually be done at init-time.
func RegisterAdapter(name string, adapter Adapter) error {
if _, ok := configAdapters[name]; ok {
return fmt.Errorf("%s: already registered", name)
}
configAdapters[name] = adapter
return nil
}
// GetAdapter returns the adapter with the given name,
// or nil if one with that name is not registered.
func GetAdapter(name string) Adapter {
return configAdapters[name]
}
var configAdapters = make(map[string]Adapter)
+349
View File
@@ -0,0 +1,349 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package httpcaddyfile
import (
"fmt"
"net"
"reflect"
"strconv"
"strings"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/caddyserver/certmagic"
)
// mapAddressToServerBlocks returns a map of listener address to list of server
// blocks that will be served on that address. To do this, each server block is
// expanded so that each one is considered individually, although keys of a
// server block that share the same address stay grouped together so the config
// isn't repeated unnecessarily. For example, this Caddyfile:
//
// example.com {
// bind 127.0.0.1
// }
// www.example.com, example.net/path, localhost:9999 {
// bind 127.0.0.1 1.2.3.4
// }
//
// has two server blocks to start with. But expressed in this Caddyfile are
// actually 4 listener addresses: 127.0.0.1:443, 1.2.3.4:443, 127.0.0.1:9999,
// and 127.0.0.1:9999. This is because the bind directive is applied to each
// key of its server block (specifying the host part), and each key may have
// a different port. And we definitely need to be sure that a site which is
// bound to be served on a specific interface is not served on others just
// because that is more convenient: it would be a potential security risk
// if the difference between interfaces means private vs. public.
//
// So what this function does for the example above is iterate each server
// block, and for each server block, iterate its keys. For the first, it
// finds one key (example.com) and determines its listener address
// (127.0.0.1:443 - because of 'bind' and automatic HTTPS). It then adds
// the listener address to the map value returned by this function, with
// the first server block as one of its associations.
//
// It then iterates each key on the second server block and associates them
// with one or more listener addresses. Indeed, each key in this block has
// two listener addresses because of the 'bind' directive. Once we know
// which addresses serve which keys, we can create a new server block for
// each address containing the contents of the server block and only those
// specific keys of the server block which use that address.
//
// It is possible and even likely that some keys in the returned map have
// the exact same list of server blocks (i.e. they are identical). This
// happens when multiple hosts are declared with a 'bind' directive and
// the resulting listener addresses are not shared by any other server
// block (or the other server blocks are exactly identical in their token
// contents). This happens with our example above because 1.2.3.4:443
// and 1.2.3.4:9999 are used exclusively with the second server block. This
// repetition may be undesirable, so call consolidateAddrMappings() to map
// multiple addresses to the same lists of server blocks (a many:many mapping).
// (Doing this is essentially a map-reduce technique.)
func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBlock,
options map[string]interface{}) (map[string][]serverBlock, error) {
sbmap := make(map[string][]serverBlock)
for i, sblock := range originalServerBlocks {
// within a server block, we need to map all the listener addresses
// implied by the server block to the keys of the server block which
// will be served by them; this has the effect of treating each
// key of a server block as its own, but without having to repeat its
// contents in cases where multiple keys really can be served together
addrToKeys := make(map[string][]string)
for j, key := range sblock.block.Keys {
// a key can have multiple listener addresses if there are multiple
// arguments to the 'bind' directive (although they will all have
// the same port, since the port is defined by the key or is implicit
// through automatic HTTPS)
addrs, err := st.listenerAddrsForServerBlockKey(sblock, key, options)
if err != nil {
return nil, fmt.Errorf("server block %d, key %d (%s): determining listener address: %v", i, j, key, err)
}
// associate this key with each listener address it is served on
for _, addr := range addrs {
addrToKeys[addr] = append(addrToKeys[addr], key)
}
}
// now that we know which addresses serve which keys of this
// server block, we iterate that mapping and create a list of
// new server blocks for each address where the keys of the
// server block are only the ones which use the address; but
// the contents (tokens) are of course the same
for addr, keys := range addrToKeys {
sbmap[addr] = append(sbmap[addr], serverBlock{
block: caddyfile.ServerBlock{
Keys: keys,
Segments: sblock.block.Segments,
},
pile: sblock.pile,
})
}
}
return sbmap, nil
}
// consolidateAddrMappings eliminates repetition of identical server blocks in a mapping of
// single listener addresses to lists of server blocks. Since multiple addresses may serve
// identical sites (server block contents), this function turns a 1:many mapping into a
// many:many mapping. Server block contents (tokens) must be exactly identical so that
// reflect.DeepEqual returns true in order for the addresses to be combined. Identical
// entries are deleted from the addrToServerBlocks map. Essentially, each pairing (each
// association from multiple addresses to multiple server blocks; i.e. each element of
// the returned slice) becomes a server definition in the output JSON.
func (st *ServerType) consolidateAddrMappings(addrToServerBlocks map[string][]serverBlock) []sbAddrAssociation {
var sbaddrs []sbAddrAssociation
for addr, sblocks := range addrToServerBlocks {
// we start with knowing that at least this address
// maps to these server blocks
a := sbAddrAssociation{
addresses: []string{addr},
serverBlocks: sblocks,
}
// now find other addresses that map to identical
// server blocks and add them to our list of
// addresses, while removing them from the map
for otherAddr, otherSblocks := range addrToServerBlocks {
if addr == otherAddr {
continue
}
if reflect.DeepEqual(sblocks, otherSblocks) {
a.addresses = append(a.addresses, otherAddr)
delete(addrToServerBlocks, otherAddr)
}
}
sbaddrs = append(sbaddrs, a)
}
return sbaddrs
}
func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key string,
options map[string]interface{}) ([]string, error) {
addr, err := ParseAddress(key)
if err != nil {
return nil, fmt.Errorf("parsing key: %v", err)
}
addr = addr.Normalize()
// figure out the HTTP and HTTPS ports; either
// use defaults, or override with user config
httpPort, httpsPort := strconv.Itoa(certmagic.HTTPPort), strconv.Itoa(certmagic.HTTPSPort)
if hport, ok := options["http_port"]; ok {
httpPort = strconv.Itoa(hport.(int))
}
if hsport, ok := options["https_port"]; ok {
httpsPort = strconv.Itoa(hsport.(int))
}
// default port is the HTTPS port
lnPort := httpsPort
if addr.Port != "" {
// port explicitly defined
lnPort = addr.Port
} else if addr.Scheme == "http" {
// port inferred from scheme
lnPort = httpPort
}
// error if scheme and port combination violate convention
if (addr.Scheme == "http" && lnPort == httpsPort) || (addr.Scheme == "https" && lnPort == httpPort) {
return nil, fmt.Errorf("[%s] scheme and port violate convention", key)
}
// the bind directive specifies hosts, but is optional
var lnHosts []string
for _, cfgVal := range sblock.pile["bind"] {
lnHosts = append(lnHosts, cfgVal.Value.([]string)...)
}
if len(lnHosts) == 0 {
lnHosts = []string{""}
}
// use a map to prevent duplication
listeners := make(map[string]struct{})
for _, host := range lnHosts {
listeners[net.JoinHostPort(host, lnPort)] = struct{}{}
}
// now turn map into list
var listenersList []string
for lnStr := range listeners {
listenersList = append(listenersList, lnStr)
}
return listenersList, nil
}
// Address represents a site address. It contains
// the original input value, and the component
// parts of an address. The component parts may be
// updated to the correct values as setup proceeds,
// but the original value should never be changed.
//
// The Host field must be in a normalized form.
type Address struct {
Original, Scheme, Host, Port, Path string
}
// ParseAddress parses an address string into a structured format with separate
// scheme, host, port, and path portions, as well as the original input string.
func ParseAddress(str string) (Address, error) {
const maxLen = 4096
if len(str) > maxLen {
str = str[:maxLen]
}
remaining := strings.TrimSpace(str)
a := Address{Original: remaining}
// extract scheme
splitScheme := strings.SplitN(remaining, "://", 2)
switch len(splitScheme) {
case 0:
return a, nil
case 1:
remaining = splitScheme[0]
case 2:
a.Scheme = splitScheme[0]
remaining = splitScheme[1]
}
// extract host and port
hostSplit := strings.SplitN(remaining, "/", 2)
if len(hostSplit) > 0 {
host, port, err := net.SplitHostPort(hostSplit[0])
if err != nil {
host, port, err = net.SplitHostPort(hostSplit[0] + ":")
if err != nil {
host = hostSplit[0]
}
}
a.Host = host
a.Port = port
}
if len(hostSplit) == 2 {
// all that remains is the path
a.Path = "/" + hostSplit[1]
}
// make sure port is valid
if a.Port != "" {
if portNum, err := strconv.Atoi(a.Port); err != nil {
return Address{}, fmt.Errorf("invalid port '%s': %v", a.Port, err)
} else if portNum < 0 || portNum > 65535 {
return Address{}, fmt.Errorf("port %d is out of range", portNum)
}
}
return a, nil
}
// TODO: which of the methods on Address are even used?
// String returns a human-readable form of a. It will
// be a cleaned-up and filled-out URL string.
func (a Address) String() string {
if a.Host == "" && a.Port == "" {
return ""
}
scheme := a.Scheme
if scheme == "" {
if a.Port == strconv.Itoa(certmagic.HTTPSPort) {
scheme = "https"
} else {
scheme = "http"
}
}
s := scheme
if s != "" {
s += "://"
}
if a.Port != "" &&
((scheme == "https" && a.Port != strconv.Itoa(caddyhttp.DefaultHTTPSPort)) ||
(scheme == "http" && a.Port != strconv.Itoa(caddyhttp.DefaultHTTPPort))) {
s += net.JoinHostPort(a.Host, a.Port)
} else {
s += a.Host
}
if a.Path != "" {
s += a.Path
}
return s
}
// Normalize returns a normalized version of a.
func (a Address) Normalize() Address {
path := a.Path
// ensure host is normalized if it's an IP address
host := a.Host
if ip := net.ParseIP(host); ip != nil {
host = ip.String()
}
return Address{
Original: a.Original,
Scheme: strings.ToLower(a.Scheme),
Host: strings.ToLower(host),
Port: a.Port,
Path: path,
}
}
// Key returns a string form of a, much like String() does, but this
// method doesn't add anything default that wasn't in the original.
func (a Address) Key() string {
res := ""
if a.Scheme != "" {
res += a.Scheme + "://"
}
if a.Host != "" {
res += a.Host
}
// insert port only if the original has its own explicit port
if a.Port != "" &&
len(a.Original) >= len(res) &&
strings.HasPrefix(a.Original[len(res):], ":"+a.Port) {
res += ":" + a.Port
}
if a.Path != "" {
res += a.Path
}
return res
}
@@ -0,0 +1,29 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +build gofuzz
// +build gofuzz_libfuzzer
package httpcaddyfile
func FuzzParseAddress(data []byte) int {
addr, err := ParseAddress(string(data))
if err != nil {
if addr == (Address{}) {
return 1
}
return 0
}
return 1
}
+163
View File
@@ -0,0 +1,163 @@
package httpcaddyfile
import (
"testing"
)
func TestParseAddress(t *testing.T) {
for i, test := range []struct {
input string
scheme, host, port, path string
shouldErr bool
}{
{``, "", "", "", "", false},
{`localhost`, "", "localhost", "", "", false},
{`localhost:1234`, "", "localhost", "1234", "", false},
{`localhost:`, "", "localhost", "", "", false},
{`0.0.0.0`, "", "0.0.0.0", "", "", false},
{`127.0.0.1:1234`, "", "127.0.0.1", "1234", "", false},
{`:1234`, "", "", "1234", "", false},
{`[::1]`, "", "::1", "", "", false},
{`[::1]:1234`, "", "::1", "1234", "", false},
{`:`, "", "", "", "", false},
{`:http`, "", "", "", "", true},
{`:https`, "", "", "", "", true},
{`localhost:http`, "", "", "", "", true}, // using service name in port is verboten, as of Go 1.12.8
{`localhost:https`, "", "", "", "", true},
{`http://localhost:https`, "", "", "", "", true}, // conflict
{`http://localhost:http`, "", "", "", "", true}, // repeated scheme
{`host:https/path`, "", "", "", "", true},
{`http://localhost:443`, "http", "localhost", "443", "", false}, // NOTE: not conventional
{`https://localhost:80`, "https", "localhost", "80", "", false}, // NOTE: not conventional
{`http://localhost`, "http", "localhost", "", "", false},
{`https://localhost`, "https", "localhost", "", "", false},
{`http://{env.APP_DOMAIN}`, "http", "{env.APP_DOMAIN}", "", "", false},
{`{env.APP_DOMAIN}:80`, "", "{env.APP_DOMAIN}", "80", "", false},
{`{env.APP_DOMAIN}/path`, "", "{env.APP_DOMAIN}", "", "/path", false},
{`example.com/{env.APP_PATH}`, "", "example.com", "", "/{env.APP_PATH}", false},
{`http://127.0.0.1`, "http", "127.0.0.1", "", "", false},
{`https://127.0.0.1`, "https", "127.0.0.1", "", "", false},
{`http://[::1]`, "http", "::1", "", "", false},
{`http://localhost:1234`, "http", "localhost", "1234", "", false},
{`https://127.0.0.1:1234`, "https", "127.0.0.1", "1234", "", false},
{`http://[::1]:1234`, "http", "::1", "1234", "", false},
{``, "", "", "", "", false},
{`::1`, "", "::1", "", "", false},
{`localhost::`, "", "localhost::", "", "", false},
{`#$%@`, "", "#$%@", "", "", false}, // don't want to presume what the hostname could be
{`host/path`, "", "host", "", "/path", false},
{`http://host/`, "http", "host", "", "/", false},
{`//asdf`, "", "", "", "//asdf", false},
{`:1234/asdf`, "", "", "1234", "/asdf", false},
{`http://host/path`, "http", "host", "", "/path", false},
{`https://host:443/path/foo`, "https", "host", "443", "/path/foo", false},
{`host:80/path`, "", "host", "80", "/path", false},
{`/path`, "", "", "", "/path", false},
} {
actual, err := ParseAddress(test.input)
if err != nil && !test.shouldErr {
t.Errorf("Test %d (%s): Expected no error, but had error: %v", i, test.input, err)
}
if err == nil && test.shouldErr {
t.Errorf("Test %d (%s): Expected error, but had none (%#v)", i, test.input, actual)
}
if !test.shouldErr && actual.Original != test.input {
t.Errorf("Test %d (%s): Expected original '%s', got '%s'", i, test.input, test.input, actual.Original)
}
if actual.Scheme != test.scheme {
t.Errorf("Test %d (%s): Expected scheme '%s', got '%s'", i, test.input, test.scheme, actual.Scheme)
}
if actual.Host != test.host {
t.Errorf("Test %d (%s): Expected host '%s', got '%s'", i, test.input, test.host, actual.Host)
}
if actual.Port != test.port {
t.Errorf("Test %d (%s): Expected port '%s', got '%s'", i, test.input, test.port, actual.Port)
}
if actual.Path != test.path {
t.Errorf("Test %d (%s): Expected path '%s', got '%s'", i, test.input, test.path, actual.Path)
}
}
}
func TestAddressString(t *testing.T) {
for i, test := range []struct {
addr Address
expected string
}{
{Address{Scheme: "http", Host: "host", Port: "1234", Path: "/path"}, "http://host:1234/path"},
{Address{Scheme: "", Host: "host", Port: "", Path: ""}, "http://host"},
{Address{Scheme: "", Host: "host", Port: "80", Path: ""}, "http://host"},
{Address{Scheme: "", Host: "host", Port: "443", Path: ""}, "https://host"},
{Address{Scheme: "https", Host: "host", Port: "443", Path: ""}, "https://host"},
{Address{Scheme: "https", Host: "host", Port: "", Path: ""}, "https://host"},
{Address{Scheme: "", Host: "host", Port: "80", Path: "/path"}, "http://host/path"},
{Address{Scheme: "http", Host: "", Port: "1234", Path: ""}, "http://:1234"},
{Address{Scheme: "", Host: "", Port: "", Path: ""}, ""},
} {
actual := test.addr.String()
if actual != test.expected {
t.Errorf("Test %d: expected '%s' but got '%s'", i, test.expected, actual)
}
}
}
func TestKeyNormalization(t *testing.T) {
testCases := []struct {
input string
expect string
}{
{
input: "http://host:1234/path",
expect: "http://host:1234/path",
},
{
input: "HTTP://A/ABCDEF",
expect: "http://a/ABCDEF",
},
{
input: "A/ABCDEF",
expect: "a/ABCDEF",
},
{
input: "A:2015/Path",
expect: "a:2015/Path",
},
{
input: ":80",
expect: ":80",
},
{
input: ":443",
expect: ":443",
},
{
input: ":1234",
expect: ":1234",
},
{
input: "",
expect: "",
},
{
input: ":",
expect: "",
},
{
input: "[::]",
expect: "::",
},
}
for i, tc := range testCases {
addr, err := ParseAddress(tc.input)
if err != nil {
t.Errorf("Test %d: Parsing address '%s': %v", i, tc.input, err)
continue
}
if actual := addr.Normalize().Key(); actual != tc.expect {
t.Errorf("Test %d: Normalized key for address '%s' was '%s' but expected '%s'", i, tc.input, actual, tc.expect)
}
}
}
+579
View File
@@ -0,0 +1,579 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package httpcaddyfile
import (
"encoding/json"
"fmt"
"html"
"net/http"
"reflect"
"strings"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
"github.com/caddyserver/caddy/v2/modules/caddytls"
"go.uber.org/zap/zapcore"
)
func init() {
RegisterDirective("bind", parseBind)
RegisterDirective("root", parseRoot) // TODO: isn't this a handler directive?
RegisterDirective("tls", parseTLS)
RegisterHandlerDirective("redir", parseRedir)
RegisterHandlerDirective("respond", parseRespond)
RegisterHandlerDirective("route", parseRoute)
RegisterHandlerDirective("handle", parseHandle)
RegisterDirective("handle_errors", parseHandleErrors)
RegisterDirective("log", parseLog)
}
// parseBind parses the bind directive. Syntax:
//
// bind <addresses...>
//
func parseBind(h Helper) ([]ConfigValue, error) {
var lnHosts []string
for h.Next() {
lnHosts = append(lnHosts, h.RemainingArgs()...)
}
return h.NewBindAddresses(lnHosts), nil
}
// parseRoot parses the root directive. Syntax:
//
// root [<matcher>] <path>
//
func parseRoot(h Helper) ([]ConfigValue, error) {
if !h.Next() {
return nil, h.ArgErr()
}
matcherSet, ok, err := h.MatcherToken()
if err != nil {
return nil, err
}
if !ok {
// no matcher token; oops
h.Dispenser.Prev()
}
if !h.NextArg() {
return nil, h.ArgErr()
}
root := h.Val()
if h.NextArg() {
return nil, h.ArgErr()
}
varsHandler := caddyhttp.VarsMiddleware{"root": root}
route := caddyhttp.Route{
HandlersRaw: []json.RawMessage{
caddyconfig.JSONModuleObject(varsHandler, "handler", "vars", nil),
},
}
if matcherSet != nil {
route.MatcherSetsRaw = []caddy.ModuleMap{matcherSet}
}
return []ConfigValue{{Class: "route", Value: route}}, nil
}
// parseTLS parses the tls directive. Syntax:
//
// tls [<email>|internal]|[<cert_file> <key_file>] {
// protocols <min> [<max>]
// ciphers <cipher_suites...>
// curves <curves...>
// alpn <values...>
// load <paths...>
// ca <acme_ca_endpoint>
// dns <provider_name>
// }
//
func parseTLS(h Helper) ([]ConfigValue, error) {
var cp *caddytls.ConnectionPolicy
var fileLoader caddytls.FileLoader
var folderLoader caddytls.FolderLoader
var acmeIssuer *caddytls.ACMEIssuer
var internalIssuer *caddytls.InternalIssuer
for h.Next() {
// file certificate loader
firstLine := h.RemainingArgs()
switch len(firstLine) {
case 0:
case 1:
if firstLine[0] == "internal" {
internalIssuer = new(caddytls.InternalIssuer)
} else if !strings.Contains(firstLine[0], "@") {
return nil, h.Err("single argument must either be 'internal' or an email address")
} else {
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
acmeIssuer.Email = firstLine[0]
}
case 2:
certFilename := firstLine[0]
keyFilename := firstLine[1]
// tag this certificate so if multiple certs match, specifically
// this one that the user has provided will be used, see #2588:
// https://github.com/caddyserver/caddy/issues/2588 ... but we
// must be careful about how we do this; being careless will
// lead to failed handshakes
//
// we need to remember which cert files we've seen, since we
// must load each cert only once; otherwise, they each get a
// different tag... since a cert loaded twice has the same
// bytes, it will overwrite the first one in the cache, and
// only the last cert (and its tag) will survive, so a any conn
// policy that is looking for any tag but the last one to be
// loaded won't find it, and TLS handshakes will fail (see end)
// of issue #3004)
//
// tlsCertTags maps certificate filenames to their tag.
// This is used to remember which tag is used for each
// certificate files, since we need to avoid loading
// the same certificate files more than once, overwriting
// previous tags
tlsCertTags, ok := h.State["tlsCertTags"].(map[string]string)
if !ok {
tlsCertTags = make(map[string]string)
h.State["tlsCertTags"] = tlsCertTags
}
tag, ok := tlsCertTags[certFilename]
if !ok {
// haven't seen this cert file yet, let's give it a tag
// and add a loader for it
tag = fmt.Sprintf("cert%d", len(tlsCertTags))
fileLoader = append(fileLoader, caddytls.CertKeyFilePair{
Certificate: certFilename,
Key: keyFilename,
Tags: []string{tag},
})
// remember this for next time we see this cert file
tlsCertTags[certFilename] = tag
}
certSelector := caddytls.CustomCertSelectionPolicy{Tag: tag}
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
cp.CertSelection = caddyconfig.JSONModuleObject(certSelector, "policy", "custom", h.warnings)
default:
return nil, h.ArgErr()
}
var hasBlock bool
for h.NextBlock(0) {
hasBlock = true
switch h.Val() {
// connection policy
case "protocols":
args := h.RemainingArgs()
if len(args) == 0 {
return nil, h.SyntaxErr("one or two protocols")
}
if len(args) > 0 {
if _, ok := caddytls.SupportedProtocols[args[0]]; !ok {
return nil, h.Errf("Wrong protocol name or protocol not supported: '%s'", args[0])
}
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
cp.ProtocolMin = args[0]
}
if len(args) > 1 {
if _, ok := caddytls.SupportedProtocols[args[1]]; !ok {
return nil, h.Errf("Wrong protocol name or protocol not supported: '%s'", args[1])
}
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
cp.ProtocolMax = args[1]
}
case "ciphers":
for h.NextArg() {
if _, ok := caddytls.SupportedCipherSuites[h.Val()]; !ok {
return nil, h.Errf("Wrong cipher suite name or cipher suite not supported: '%s'", h.Val())
}
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
cp.CipherSuites = append(cp.CipherSuites, h.Val())
}
case "curves":
for h.NextArg() {
if _, ok := caddytls.SupportedCurves[h.Val()]; !ok {
return nil, h.Errf("Wrong curve name or curve not supported: '%s'", h.Val())
}
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
cp.Curves = append(cp.Curves, h.Val())
}
case "alpn":
args := h.RemainingArgs()
if len(args) == 0 {
return nil, h.ArgErr()
}
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
cp.ALPN = args
// certificate folder loader
case "load":
folderLoader = append(folderLoader, h.RemainingArgs()...)
// automation policy
case "ca":
arg := h.RemainingArgs()
if len(arg) != 1 {
return nil, h.ArgErr()
}
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
acmeIssuer.CA = arg[0]
// DNS provider for ACME DNS challenge
case "dns":
if !h.Next() {
return nil, h.ArgErr()
}
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
provName := h.Val()
if acmeIssuer.Challenges == nil {
acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
}
dnsProvModule, err := caddy.GetModule("tls.dns." + provName)
if err != nil {
return nil, h.Errf("getting DNS provider module named '%s': %v", provName, err)
}
acmeIssuer.Challenges.DNSRaw = caddyconfig.JSONModuleObject(dnsProvModule.New(), "provider", provName, h.warnings)
case "ca_root":
arg := h.RemainingArgs()
if len(arg) != 1 {
return nil, h.ArgErr()
}
if acmeIssuer == nil {
acmeIssuer = new(caddytls.ACMEIssuer)
}
acmeIssuer.TrustedRootsPEMFiles = append(acmeIssuer.TrustedRootsPEMFiles, arg[0])
default:
return nil, h.Errf("unknown subdirective: %s", h.Val())
}
}
// a naked tls directive is not allowed
if len(firstLine) == 0 && !hasBlock {
return nil, h.ArgErr()
}
}
// begin building the final config values
var configVals []ConfigValue
// certificate loaders
if len(fileLoader) > 0 {
configVals = append(configVals, ConfigValue{
Class: "tls.certificate_loader",
Value: fileLoader,
})
// ensure server uses HTTPS by setting non-nil conn policy
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
}
if len(folderLoader) > 0 {
configVals = append(configVals, ConfigValue{
Class: "tls.certificate_loader",
Value: folderLoader,
})
// ensure server uses HTTPS by setting non-nil conn policy
if cp == nil {
cp = new(caddytls.ConnectionPolicy)
}
}
// connection policy
if cp != nil {
configVals = append(configVals, ConfigValue{
Class: "tls.connection_policy",
Value: cp,
})
}
// automation policy
if acmeIssuer != nil && internalIssuer != nil {
// the logic to support this would be complex
return nil, h.Err("cannot use both ACME and internal issuers in same server block")
}
if acmeIssuer != nil {
// fill in global defaults, if configured
if email := h.Option("email"); email != nil && acmeIssuer.Email == "" {
acmeIssuer.Email = email.(string)
}
if acmeCA := h.Option("acme_ca"); acmeCA != nil && acmeIssuer.CA == "" {
acmeIssuer.CA = acmeCA.(string)
}
if caPemFile := h.Option("acme_ca_root"); caPemFile != nil {
acmeIssuer.TrustedRootsPEMFiles = append(acmeIssuer.TrustedRootsPEMFiles, caPemFile.(string))
}
configVals = append(configVals, ConfigValue{
Class: "tls.cert_issuer",
Value: acmeIssuer,
})
} else if internalIssuer != nil {
configVals = append(configVals, ConfigValue{
Class: "tls.cert_issuer",
Value: internalIssuer,
})
}
return configVals, nil
}
// parseRedir parses the redir directive. Syntax:
//
// redir [<matcher>] <to> [<code>]
//
func parseRedir(h Helper) (caddyhttp.MiddlewareHandler, error) {
if !h.Next() {
return nil, h.ArgErr()
}
if !h.NextArg() {
return nil, h.ArgErr()
}
to := h.Val()
var code string
if h.NextArg() {
code = h.Val()
}
if code == "permanent" {
code = "301"
}
if code == "temporary" || code == "" {
code = "302"
}
var body string
if code == "html" {
// Script tag comes first since that will better imitate a redirect in the browser's
// history, but the meta tag is a fallback for most non-JS clients.
const metaRedir = `<!DOCTYPE html>
<html>
<head>
<title>Redirecting...</title>
<script>window.location.replace("%s");</script>
<meta http-equiv="refresh" content="0; URL='%s'">
</head>
<body>Redirecting to <a href="%s">%s</a>...</body>
</html>
`
safeTo := html.EscapeString(to)
body = fmt.Sprintf(metaRedir, safeTo, safeTo, safeTo, safeTo)
}
return caddyhttp.StaticResponse{
StatusCode: caddyhttp.WeakString(code),
Headers: http.Header{"Location": []string{to}},
Body: body,
}, nil
}
// parseRespond parses the respond directive.
func parseRespond(h Helper) (caddyhttp.MiddlewareHandler, error) {
sr := new(caddyhttp.StaticResponse)
err := sr.UnmarshalCaddyfile(h.Dispenser)
if err != nil {
return nil, err
}
return sr, nil
}
// parseRoute parses the route directive.
func parseRoute(h Helper) (caddyhttp.MiddlewareHandler, error) {
sr := new(caddyhttp.Subroute)
for h.Next() {
for nesting := h.Nesting(); h.NextBlock(nesting); {
dir := h.Val()
dirFunc, ok := registeredDirectives[dir]
if !ok {
return nil, h.Errf("unrecognized directive: %s", dir)
}
subHelper := h
subHelper.Dispenser = h.NewFromNextSegment()
results, err := dirFunc(subHelper)
if err != nil {
return nil, h.Errf("parsing caddyfile tokens for '%s': %v", dir, err)
}
for _, result := range results {
handler, ok := result.Value.(caddyhttp.Route)
if !ok {
return nil, h.Errf("%s directive returned something other than an HTTP route: %#v (only handler directives can be used in routes)", dir, result.Value)
}
sr.Routes = append(sr.Routes, handler)
}
}
}
return sr, nil
}
func parseHandle(h Helper) (caddyhttp.MiddlewareHandler, error) {
return parseSegmentAsSubroute(h)
}
func parseHandleErrors(h Helper) ([]ConfigValue, error) {
subroute, err := parseSegmentAsSubroute(h)
if err != nil {
return nil, err
}
return []ConfigValue{
{
Class: "error_route",
Value: subroute,
},
}, nil
}
// parseLog parses the log directive. Syntax:
//
// log {
// output <writer_module> ...
// format <encoder_module> ...
// level <level>
// }
//
func parseLog(h Helper) ([]ConfigValue, error) {
var configValues []ConfigValue
for h.Next() {
cl := new(caddy.CustomLog)
for h.NextBlock(0) {
switch h.Val() {
case "output":
if !h.NextArg() {
return nil, h.ArgErr()
}
moduleName := h.Val()
// can't use the usual caddyfile.Unmarshaler flow with the
// standard writers because they are in the caddy package
// (because they are the default) and implementing that
// interface there would unfortunately create circular import
var wo caddy.WriterOpener
switch moduleName {
case "stdout":
wo = caddy.StdoutWriter{}
case "stderr":
wo = caddy.StderrWriter{}
case "discard":
wo = caddy.DiscardWriter{}
default:
mod, err := caddy.GetModule("caddy.logging.writers." + moduleName)
if err != nil {
return nil, h.Errf("getting log writer module named '%s': %v", moduleName, err)
}
unm, ok := mod.New().(caddyfile.Unmarshaler)
if !ok {
return nil, h.Errf("log writer module '%s' is not a Caddyfile unmarshaler", mod)
}
err = unm.UnmarshalCaddyfile(h.NewFromNextSegment())
if err != nil {
return nil, err
}
wo, ok = unm.(caddy.WriterOpener)
if !ok {
return nil, h.Errf("module %s is not a WriterOpener", mod)
}
}
cl.WriterRaw = caddyconfig.JSONModuleObject(wo, "output", moduleName, h.warnings)
case "format":
if !h.NextArg() {
return nil, h.ArgErr()
}
moduleName := h.Val()
mod, err := caddy.GetModule("caddy.logging.encoders." + moduleName)
if err != nil {
return nil, h.Errf("getting log encoder module named '%s': %v", moduleName, err)
}
unm, ok := mod.New().(caddyfile.Unmarshaler)
if !ok {
return nil, h.Errf("log encoder module '%s' is not a Caddyfile unmarshaler", mod)
}
err = unm.UnmarshalCaddyfile(h.NewFromNextSegment())
if err != nil {
return nil, err
}
enc, ok := unm.(zapcore.Encoder)
if !ok {
return nil, h.Errf("module %s is not a zapcore.Encoder", mod)
}
cl.EncoderRaw = caddyconfig.JSONModuleObject(enc, "format", moduleName, h.warnings)
case "level":
if !h.NextArg() {
return nil, h.ArgErr()
}
cl.Level = h.Val()
if h.NextArg() {
return nil, h.ArgErr()
}
default:
return nil, h.Errf("unrecognized subdirective: %s", h.Val())
}
}
var val namedCustomLog
if !reflect.DeepEqual(cl, new(caddy.CustomLog)) {
logCounter, ok := h.State["logCounter"].(int)
if !ok {
logCounter = 0
}
cl.Include = []string{"http.log.access"}
val.name = fmt.Sprintf("log%d", logCounter)
val.log = cl
logCounter++
h.State["logCounter"] = logCounter
}
configValues = append(configValues, ConfigValue{
Class: "custom_log",
Value: val,
})
}
return configValues, nil
}
+402
View File
@@ -0,0 +1,402 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package httpcaddyfile
import (
"encoding/json"
"sort"
"strings"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/caddy/v2/modules/caddyhttp"
)
// directiveOrder specifies the order
// to apply directives in HTTP routes.
var directiveOrder = []string{
"redir",
"rewrite",
"root",
"strip_prefix",
"strip_suffix",
"uri_replace",
"try_files",
// middleware handlers that typically wrap responses
"basicauth",
"header",
"request_header",
"encode",
"templates",
"handle",
"route",
// handlers that typically respond to requests
"respond",
"reverse_proxy",
"php_fastcgi",
"file_server",
}
// directiveIsOrdered returns true if dir is
// a known, ordered (sorted) directive.
func directiveIsOrdered(dir string) bool {
for _, d := range directiveOrder {
if d == dir {
return true
}
}
return false
}
// RegisterDirective registers a unique directive dir with an
// associated unmarshaling (setup) function. When directive dir
// is encountered in a Caddyfile, setupFunc will be called to
// unmarshal its tokens.
func RegisterDirective(dir string, setupFunc UnmarshalFunc) {
if _, ok := registeredDirectives[dir]; ok {
panic("directive " + dir + " already registered")
}
registeredDirectives[dir] = setupFunc
}
// RegisterHandlerDirective is like RegisterDirective, but for
// directives which specifically output only an HTTP handler.
// Directives registered with this function will always have
// an optional matcher token as the first argument.
func RegisterHandlerDirective(dir string, setupFunc UnmarshalHandlerFunc) {
RegisterDirective(dir, func(h Helper) ([]ConfigValue, error) {
if !h.Next() {
return nil, h.ArgErr()
}
matcherSet, ok, err := h.MatcherToken()
if err != nil {
return nil, err
}
if ok {
// strip matcher token; we don't need to
// use the return value here because a
// new dispenser should have been made
// solely for this directive's tokens,
// with no other uses of same slice
h.Dispenser.Delete()
}
h.Dispenser.Reset() // pretend this lookahead never happened
val, err := setupFunc(h)
if err != nil {
return nil, err
}
return h.NewRoute(matcherSet, val), nil
})
}
// Helper is a type which helps setup a value from
// Caddyfile tokens.
type Helper struct {
*caddyfile.Dispenser
// State stores intermediate variables during caddyfile adaptation.
State map[string]interface{}
options map[string]interface{}
warnings *[]caddyconfig.Warning
matcherDefs map[string]caddy.ModuleMap
parentBlock caddyfile.ServerBlock
groupCounter counter
}
// Option gets the option keyed by name.
func (h Helper) Option(name string) interface{} {
return h.options[name]
}
// Caddyfiles returns the list of config files from
// which tokens in the current server block were loaded.
func (h Helper) Caddyfiles() []string {
// first obtain set of names of files involved
// in this server block, without duplicates
files := make(map[string]struct{})
for _, segment := range h.parentBlock.Segments {
for _, token := range segment {
files[token.File] = struct{}{}
}
}
// then convert the set into a slice
filesSlice := make([]string, 0, len(files))
for file := range files {
filesSlice = append(filesSlice, file)
}
return filesSlice
}
// JSON converts val into JSON. Any errors are added to warnings.
func (h Helper) JSON(val interface{}) json.RawMessage {
return caddyconfig.JSON(val, h.warnings)
}
// MatcherToken assumes the next argument token is (possibly) a matcher,
// and if so, returns the matcher set along with a true value. If the next
// token is not a matcher, nil and false is returned. Note that a true
// value may be returned with a nil matcher set if it is a catch-all.
func (h Helper) MatcherToken() (caddy.ModuleMap, bool, error) {
if !h.NextArg() {
return nil, false, nil
}
return matcherSetFromMatcherToken(h.Dispenser.Token(), h.matcherDefs, h.warnings)
}
// ExtractMatcherSet is like MatcherToken, except this is a higher-level
// method that returns the matcher set described by the matcher token,
// or nil if there is none, and deletes the matcher token from the
// dispenser and resets it as if this look-ahead never happened. Useful
// when wrapping a route (one or more handlers) in a user-defined matcher.
func (h Helper) ExtractMatcherSet() (caddy.ModuleMap, error) {
matcherSet, hasMatcher, err := h.MatcherToken()
if err != nil {
return nil, err
}
if hasMatcher {
h.Dispenser.Delete() // strip matcher token
}
h.Dispenser.Reset() // pretend this lookahead never happened
return matcherSet, nil
}
// NewRoute returns config values relevant to creating a new HTTP route.
func (h Helper) NewRoute(matcherSet caddy.ModuleMap,
handler caddyhttp.MiddlewareHandler) []ConfigValue {
mod, err := caddy.GetModule(caddy.GetModuleID(handler))
if err != nil {
*h.warnings = append(*h.warnings, caddyconfig.Warning{
File: h.File(),
Line: h.Line(),
Message: err.Error(),
})
return nil
}
var matcherSetsRaw []caddy.ModuleMap
if matcherSet != nil {
matcherSetsRaw = append(matcherSetsRaw, matcherSet)
}
return []ConfigValue{
{
Class: "route",
Value: caddyhttp.Route{
MatcherSetsRaw: matcherSetsRaw,
HandlersRaw: []json.RawMessage{caddyconfig.JSONModuleObject(handler, "handler", mod.ID.Name(), h.warnings)},
},
},
}
}
// GroupRoutes adds the routes (caddyhttp.Route type) in vals to the
// same group, if there is more than one route in vals.
func (h Helper) GroupRoutes(vals []ConfigValue) {
// ensure there's at least two routes; group of one is pointless
var count int
for _, v := range vals {
if _, ok := v.Value.(caddyhttp.Route); ok {
count++
if count > 1 {
break
}
}
}
if count < 2 {
return
}
// now that we know the group will have some effect, do it
groupName := h.groupCounter.nextGroup()
for i := range vals {
if route, ok := vals[i].Value.(caddyhttp.Route); ok {
route.Group = groupName
vals[i].Value = route
}
}
}
// NewBindAddresses returns config values relevant to adding
// listener bind addresses to the config.
func (h Helper) NewBindAddresses(addrs []string) []ConfigValue {
return []ConfigValue{{Class: "bind", Value: addrs}}
}
// ConfigValue represents a value to be added to the final
// configuration, or a value to be consulted when building
// the final configuration.
type ConfigValue struct {
// The kind of value this is. As the config is
// being built, the adapter will look in the
// "pile" for values belonging to a certain
// class when it is setting up a certain part
// of the config. The associated value will be
// type-asserted and placed accordingly.
Class string
// The value to be used when building the config.
// Generally its type is associated with the
// name of the Class.
Value interface{}
directive string
}
func sortRoutes(routes []ConfigValue) {
dirPositions := make(map[string]int)
for i, dir := range directiveOrder {
dirPositions[dir] = i
}
// while we are sorting, we will need to decode a route's path matcher
// in order to sub-sort by path length; we can amortize this operation
// for efficiency by storing the decoded matchers in a slice
decodedMatchers := make([]caddyhttp.MatchPath, len(routes))
sort.SliceStable(routes, func(i, j int) bool {
iDir, jDir := routes[i].directive, routes[j].directive
if iDir == jDir {
// directives are the same; sub-sort by path matcher length
// if there's only one matcher set and one path (common case)
iRoute, ok := routes[i].Value.(caddyhttp.Route)
if !ok {
return false
}
jRoute, ok := routes[j].Value.(caddyhttp.Route)
if !ok {
return false
}
// use already-decoded matcher, or decode if it's the first time seeing it
iPM, jPM := decodedMatchers[i], decodedMatchers[j]
if iPM == nil && len(iRoute.MatcherSetsRaw) == 1 {
var pathMatcher caddyhttp.MatchPath
_ = json.Unmarshal(iRoute.MatcherSetsRaw[0]["path"], &pathMatcher)
decodedMatchers[i] = pathMatcher
iPM = pathMatcher
}
if jPM == nil && len(jRoute.MatcherSetsRaw) == 1 {
var pathMatcher caddyhttp.MatchPath
_ = json.Unmarshal(jRoute.MatcherSetsRaw[0]["path"], &pathMatcher)
decodedMatchers[j] = pathMatcher
jPM = pathMatcher
}
// sort by longer path (more specific) first; missing
// path matchers are treated as zero-length paths
var iPathLen, jPathLen int
if iPM != nil {
iPathLen = len(iPM[0])
}
if jPM != nil {
jPathLen = len(jPM[0])
}
return iPathLen > jPathLen
}
return dirPositions[iDir] < dirPositions[jDir]
})
}
// parseSegmentAsSubroute parses the segment such that its subdirectives
// are themselves treated as directives, from which a subroute is built
// and returned.
func parseSegmentAsSubroute(h Helper) (caddyhttp.MiddlewareHandler, error) {
var allResults []ConfigValue
for h.Next() {
// slice the linear list of tokens into top-level segments
var segments []caddyfile.Segment
for nesting := h.Nesting(); h.NextBlock(nesting); {
segments = append(segments, h.NextSegment())
}
// copy existing matcher definitions so we can augment
// new ones that are defined only in this scope
matcherDefs := make(map[string]caddy.ModuleMap, len(h.matcherDefs))
for key, val := range h.matcherDefs {
matcherDefs[key] = val
}
// find and extract any embedded matcher definitions in this scope
for i, seg := range segments {
if strings.HasPrefix(seg.Directive(), matcherPrefix) {
err := parseMatcherDefinitions(caddyfile.NewDispenser(seg), matcherDefs)
if err != nil {
return nil, err
}
segments = append(segments[:i], segments[i+1:]...)
}
}
// with matchers ready to go, evaluate each directive's segment
for _, seg := range segments {
dir := seg.Directive()
dirFunc, ok := registeredDirectives[dir]
if !ok {
return nil, h.Errf("unrecognized directive: %s", dir)
}
subHelper := h
subHelper.Dispenser = caddyfile.NewDispenser(seg)
subHelper.matcherDefs = matcherDefs
results, err := dirFunc(subHelper)
if err != nil {
return nil, h.Errf("parsing caddyfile tokens for '%s': %v", dir, err)
}
for _, result := range results {
result.directive = dir
allResults = append(allResults, result)
}
}
}
return buildSubroute(allResults, h.groupCounter)
}
// serverBlock pairs a Caddyfile server block
// with a "pile" of config values, keyed by class
// name.
type serverBlock struct {
block caddyfile.ServerBlock
pile map[string][]ConfigValue // config values obtained from directives
}
type (
// UnmarshalFunc is a function which can unmarshal Caddyfile
// tokens into zero or more config values using a Helper type.
// These are passed in a call to RegisterDirective.
UnmarshalFunc func(h Helper) ([]ConfigValue, error)
// UnmarshalHandlerFunc is like UnmarshalFunc, except the
// output of the unmarshaling is an HTTP handler. This
// function does not need to deal with HTTP request matching
// which is abstracted away. Since writing HTTP handlers
// with Caddyfile support is very common, this is a more
// convenient way to add a handler to the chain since a lot
// of the details common to HTTP handlers are taken care of
// for you. These are passed to a call to
// RegisterHandlerDirective.
UnmarshalHandlerFunc func(h Helper) (caddyhttp.MiddlewareHandler, error)
)
var registeredDirectives = make(map[string]UnmarshalFunc)
File diff suppressed because it is too large Load Diff
+149
View File
@@ -0,0 +1,149 @@
package httpcaddyfile
import (
"testing"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
)
func TestServerType(t *testing.T) {
for i, tc := range []struct {
input string
expectWarn bool
expectError bool
}{
{
input: `http://localhost
@debug {
query showdebug=1
}
`,
expectWarn: false,
expectError: false,
},
{
input: `http://localhost
@debug {
query bad format
}
`,
expectWarn: false,
expectError: true,
},
} {
adapter := caddyfile.Adapter{
ServerType: ServerType{},
}
_, warnings, err := adapter.Adapt([]byte(tc.input), nil)
if len(warnings) > 0 != tc.expectWarn {
t.Errorf("Test %d warning expectation failed Expected: %v, got %v", i, tc.expectWarn, warnings)
continue
}
if err != nil != tc.expectError {
t.Errorf("Test %d error expectation failed Expected: %v, got %s", i, tc.expectError, err)
continue
}
}
}
func TestSpecificity(t *testing.T) {
for i, tc := range []struct {
input string
expect int
}{
{"", 0},
{"*", 0},
{"*.*", 1},
{"{placeholder}", 0},
{"/{placeholder}", 1},
{"foo", 3},
{"example.com", 11},
{"a.example.com", 13},
{"*.example.com", 12},
{"/foo", 4},
{"/foo*", 4},
{"{placeholder}.example.com", 12},
{"{placeholder.example.com", 24},
{"}.", 2},
{"}{", 2},
{"{}", 0},
{"{{{}}", 1},
} {
actual := specificity(tc.input)
if actual != tc.expect {
t.Errorf("Test %d (%s): Expected %d but got %d", i, tc.input, tc.expect, actual)
}
}
}
func TestGlobalOptions(t *testing.T) {
for i, tc := range []struct {
input string
expectWarn bool
expectError bool
}{
{
input: `
{
email test@example.com
}
:80
`,
expectWarn: false,
expectError: false,
},
{
input: `
{
admin off
}
:80
`,
expectWarn: false,
expectError: false,
},
{
input: `
{
admin 127.0.0.1:2020
}
:80
`,
expectWarn: false,
expectError: false,
},
{
input: `
{
admin {
disabled false
}
}
:80
`,
expectWarn: false,
expectError: true,
},
} {
adapter := caddyfile.Adapter{
ServerType: ServerType{},
}
_, warnings, err := adapter.Adapt([]byte(tc.input), nil)
if len(warnings) > 0 != tc.expectWarn {
t.Errorf("Test %d warning expectation failed Expected: %v, got %v", i, tc.expectWarn, warnings)
continue
}
if err != nil != tc.expectError {
t.Errorf("Test %d error expectation failed Expected: %v, got %s", i, tc.expectError, err)
continue
}
}
}
+189
View File
@@ -0,0 +1,189 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package httpcaddyfile
import (
"fmt"
"strconv"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
)
func parseOptHTTPPort(d *caddyfile.Dispenser) (int, error) {
var httpPort int
for d.Next() {
var httpPortStr string
if !d.AllArgs(&httpPortStr) {
return 0, d.ArgErr()
}
var err error
httpPort, err = strconv.Atoi(httpPortStr)
if err != nil {
return 0, d.Errf("converting port '%s' to integer value: %v", httpPortStr, err)
}
}
return httpPort, nil
}
func parseOptHTTPSPort(d *caddyfile.Dispenser) (int, error) {
var httpsPort int
for d.Next() {
var httpsPortStr string
if !d.AllArgs(&httpsPortStr) {
return 0, d.ArgErr()
}
var err error
httpsPort, err = strconv.Atoi(httpsPortStr)
if err != nil {
return 0, d.Errf("converting port '%s' to integer value: %v", httpsPortStr, err)
}
}
return httpsPort, nil
}
func parseOptExperimentalHTTP3(d *caddyfile.Dispenser) (bool, error) {
return true, nil
}
func parseOptOrder(d *caddyfile.Dispenser) ([]string, error) {
newOrder := directiveOrder
for d.Next() {
// get directive name
if !d.Next() {
return nil, d.ArgErr()
}
dirName := d.Val()
if _, ok := registeredDirectives[dirName]; !ok {
return nil, fmt.Errorf("%s is not a registered directive", dirName)
}
// get positional token
if !d.Next() {
return nil, d.ArgErr()
}
pos := d.Val()
// if directive exists, first remove it
for i, d := range newOrder {
if d == dirName {
newOrder = append(newOrder[:i], newOrder[i+1:]...)
break
}
}
// act on the positional
switch pos {
case "first":
newOrder = append([]string{dirName}, newOrder...)
if d.NextArg() {
return nil, d.ArgErr()
}
directiveOrder = newOrder
return newOrder, nil
case "last":
newOrder = append(newOrder, dirName)
if d.NextArg() {
return nil, d.ArgErr()
}
directiveOrder = newOrder
return newOrder, nil
case "before":
case "after":
default:
return nil, fmt.Errorf("unknown positional '%s'", pos)
}
// get name of other directive
if !d.NextArg() {
return nil, d.ArgErr()
}
otherDir := d.Val()
if d.NextArg() {
return nil, d.ArgErr()
}
// insert directive into proper position
for i, d := range newOrder {
if d == otherDir {
if pos == "before" {
newOrder = append(newOrder[:i], append([]string{dirName}, newOrder[i:]...)...)
} else if pos == "after" {
newOrder = append(newOrder[:i+1], append([]string{dirName}, newOrder[i+1:]...)...)
}
break
}
}
}
directiveOrder = newOrder
return newOrder, nil
}
func parseOptStorage(d *caddyfile.Dispenser) (caddy.StorageConverter, error) {
if !d.Next() {
return nil, d.ArgErr()
}
args := d.RemainingArgs()
if len(args) != 1 {
return nil, d.ArgErr()
}
modName := args[0]
mod, err := caddy.GetModule("caddy.storage." + modName)
if err != nil {
return nil, fmt.Errorf("getting storage module '%s': %v", modName, err)
}
unm, ok := mod.New().(caddyfile.Unmarshaler)
if !ok {
return nil, fmt.Errorf("storage module '%s' is not a Caddyfile unmarshaler", mod.ID)
}
err = unm.UnmarshalCaddyfile(d.NewFromNextSegment())
if err != nil {
return nil, err
}
storage, ok := unm.(caddy.StorageConverter)
if !ok {
return nil, fmt.Errorf("module %s is not a StorageConverter", mod.ID)
}
return storage, nil
}
func parseOptSingleString(d *caddyfile.Dispenser) (string, error) {
d.Next() // consume parameter name
if !d.Next() {
return "", d.ArgErr()
}
val := d.Val()
if d.Next() {
return "", d.ArgErr()
}
return val, nil
}
func parseOptAdmin(d *caddyfile.Dispenser) (string, error) {
if d.Next() {
var listenAddress string
if !d.AllArgs(&listenAddress) {
return "", d.ArgErr()
}
if listenAddress == "" {
listenAddress = caddy.DefaultAdminListen
}
return listenAddress, nil
}
return "", nil
}
+43
View File
@@ -0,0 +1,43 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package json5adapter
import (
"encoding/json"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/ilibs/json5"
)
func init() {
caddyconfig.RegisterAdapter("json5", Adapter{})
}
// Adapter adapts JSON5 to Caddy JSON.
type Adapter struct{}
// Adapt converts the JSON5 config in body to Caddy JSON.
func (a Adapter) Adapt(body []byte, options map[string]interface{}) (result []byte, warnings []caddyconfig.Warning, err error) {
var decoded interface{}
err = json5.Unmarshal(body, &decoded)
if err != nil {
return
}
result, err = json.Marshal(decoded)
return
}
// Interface guard
var _ caddyconfig.Adapter = (*Adapter)(nil)
+49
View File
@@ -0,0 +1,49 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package jsoncadapter
import (
"encoding/json"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/muhammadmuzzammil1998/jsonc"
)
func init() {
caddyconfig.RegisterAdapter("jsonc", Adapter{})
}
// Adapter adapts JSON-C to Caddy JSON.
type Adapter struct{}
// Adapt converts the JSON-C config in body to Caddy JSON.
func (a Adapter) Adapt(body []byte, options map[string]interface{}) (result []byte, warnings []caddyconfig.Warning, err error) {
result = jsonc.ToJSON(body)
// any errors in the JSON will be
// reported during config load, but
// we can at least warn here that
// it is not valid JSON
if !json.Valid(result) {
warnings = append(warnings, caddyconfig.Warning{
Message: "Resulting JSON is invalid.",
})
}
return
}
// Interface guard
var _ caddyconfig.Adapter = (*Adapter)(nil)
+23
View File
@@ -0,0 +1,23 @@
-----BEGIN CERTIFICATE-----
MIID5zCCAs8CFFmAAFKV79uhzxc5qXbUw3oBNsYXMA0GCSqGSIb3DQEBCwUAMIGv
MQswCQYDVQQGEwJVUzELMAkGA1UECAwCTlkxGzAZBgNVBAoMEkxvY2FsIERldmVs
b3BlbWVudDEbMBkGA1UEBwwSTG9jYWwgRGV2ZWxvcGVtZW50MRowGAYDVQQDDBEq
LmNhZGR5LmxvY2FsaG9zdDEbMBkGA1UECwwSTG9jYWwgRGV2ZWxvcGVtZW50MSAw
HgYJKoZIhvcNAQkBFhFhZG1pbkBjYWRkeS5sb2NhbDAeFw0yMDAzMDIwODAxMTZa
Fw0zMDAyMjgwODAxMTZaMIGvMQswCQYDVQQGEwJVUzELMAkGA1UECAwCTlkxGzAZ
BgNVBAoMEkxvY2FsIERldmVsb3BlbWVudDEbMBkGA1UEBwwSTG9jYWwgRGV2ZWxv
cGVtZW50MRowGAYDVQQDDBEqLmNhZGR5LmxvY2FsaG9zdDEbMBkGA1UECwwSTG9j
YWwgRGV2ZWxvcGVtZW50MSAwHgYJKoZIhvcNAQkBFhFhZG1pbkBjYWRkeS5sb2Nh
bDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAJngfeirQkWaU8ihgIC5
SKpRQX/3koRjljDK/oCbhLs+wg592kIwVv06l7+mn7NSaNBloabjuA1GqyLRsNLL
ptrv0HvXa5qLx28+icsb2Ny3dJnQaj9w9PwjxQ1qZqEJfWRH1D8Vz9AmB+QSV/Gu
8e8alGFewlYZVfH1kbxoTT6QorF37TeA3bh1fgKFtzsGYKswcaZNdDBBHzLunCKZ
HU6U6L45hm+yLADj3mmDLafUeiVOt6MRLLoSD1eLRVSXGrNo+brJ87zkZntI9+W1
JxOBoXtZCwka7k2DlAtLihsrmBZA2ZC9yVeu/SQy3qb3iCNnTFTCyAnWeTCr6Tcq
6w8CAwEAATANBgkqhkiG9w0BAQsFAAOCAQEAOWfXqpAmD4C3wGiMeZAeaaS4hDAR
+JmN+avPDA6F6Bq7DB4NJuIwVUlaDL2s07w5VJJtW52aZVKoBlgHR5yG/XUli6J7
YUJRmdQJvHUSu26cmKvyoOaTrEYbmvtGICWtZc8uTlMf9wQZbJA4KyxTgEQJDXsZ
B2XFe+wVdhAgEpobYDROi+l/p8TL5z3U24LpwVTcJy5sEZVv7Wfs886IyxU8ORt8
VZNcDiH6V53OIGeiufIhia/mPe6jbLntfGZfIFxtCcow4IA/lTy1ned7K5fmvNNb
ZilxOQUk+wVK8genjdrZVAnAxsYLHJIb5yf9O7rr6fWciVMF3a0k5uNK1w==
-----END CERTIFICATE-----
+27
View File
@@ -0,0 +1,27 @@
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAmeB96KtCRZpTyKGAgLlIqlFBf/eShGOWMMr+gJuEuz7CDn3a
QjBW/TqXv6afs1Jo0GWhpuO4DUarItGw0sum2u/Qe9drmovHbz6JyxvY3Ld0mdBq
P3D0/CPFDWpmoQl9ZEfUPxXP0CYH5BJX8a7x7xqUYV7CVhlV8fWRvGhNPpCisXft
N4DduHV+AoW3OwZgqzBxpk10MEEfMu6cIpkdTpTovjmGb7IsAOPeaYMtp9R6JU63
oxEsuhIPV4tFVJcas2j5usnzvORme0j35bUnE4Ghe1kLCRruTYOUC0uKGyuYFkDZ
kL3JV679JDLepveII2dMVMLICdZ5MKvpNyrrDwIDAQABAoIBAFcPK01zb6hfm12c
+k5aBiHOnUdgc/YRPg1XHEz5MEycQkDetZjTLrRQ7UBSbnKPgpu9lIsOtbhVLkgh
6XAqJroiCou2oruqr+hhsqZGmBiwdvj7cNF6ADGTr05az7v22YneFdinZ481pStF
sZocx+bm2+KHMV5zMSwXKyA0xtdJLxs2yklniDBxSZRppgppq1pDPprP5DkgKPfe
3ekUmbQd5bHmivhW8ItbJLuf82XSsMBZ9ZhKiKIlWlbKAgiSV3SqnUQb5fi7l8hG
yYZxbuCUIGFwKmEpUBBt/nyxrOlMiNtDh9JhrPmijTV3slq70pCLwLL/Ai2aeear
EVA5VhkCgYEAyAmxfPqc2P7BsDAp67/sA7OEPso9qM4WyuWiVdlX2gb9TLNLYbPX
Kk/UmpAIVzpoTAGY5Zp3wkvdD/ou8uUQsE8ioNn4S1a4G9XURH1wVhcEbUiAKI1S
QVBH9B/Pj3eIp5OTKwob0Wj7DNdxoH7ed/Eok0EaTWzOA8pCWADKv/MCgYEAxOzY
YsX7Nl+eyZr2+9unKyeAK/D1DCT/o99UUAHx72/xaBVP/06cfzpvKBNcF9iYc+fq
R1yIUIrDRoSmYKBq+Kb3+nOg1nrqih/NBTokbTiI4Q+/30OQt0Al1e7y9iNKqV8H
jYZItzluGNrWKedZbATwBwbVCY2jnNl6RMDnS3UCgYBxj3cwQUHLuoyQjjcuO80r
qLzZvIxWiXDNDKIk5HcIMlGYOmz/8U2kGp/SgxQJGQJeq8V2C0QTjGfaCyieAcaA
oNxCvptDgd6RBsoze5bLeNOtiqwe2WOp6n5+q5R0mOJ+Z7vzghCayGNFPgWmnH+F
TeW/+wSIkc0+v5L8TK7NWwKBgBrlWlyLO9deUfqpHqihhICBYaEexOlGuF+yZfqT
eW7BdFBJ8OYm33sFCR+JHV/oZlIWT8o1Wizd9vPPtEWoQ1P4wg/D8Si6GwSIeWEI
YudD/HX4x7T/rmlI6qIAg9CYW18sqoRq3c2gm2fro6qPfYgiWIItLbWjUcBfd7Ki
QjTtAoGARKdRv3jMWL84rlEx1nBRgL3pe9Dt+Uxzde2xT3ZeF+5Hp9NfU01qE6M6
1I6H64smqpetlsXmCEVKwBemP3pJa6avLKgIYiQvHAD/v4rs9mqgy1RTqtYyGNhR
1A/6dKkbiZ6wzePLLPasXVZxSKEviXf5gJooqumQVSVhCswyCZ0=
-----END RSA PRIVATE KEY-----
+272
View File
@@ -0,0 +1,272 @@
package caddytest
import (
"bytes"
"context"
"crypto/tls"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"log"
"net"
"net/http"
"os"
"path"
"regexp"
"runtime"
"strings"
"testing"
"time"
)
// Defaults store any configuration required to make the tests run
type Defaults struct {
// Port we expect caddy to listening on
AdminPort int
// Certificates we expect to be loaded before attempting to run the tests
Certifcates []string
}
// Default testing values
var Default = Defaults{
AdminPort: 2019,
Certifcates: []string{"/caddy.localhost.crt", "/caddy.localhost.key"},
}
var (
matchKey = regexp.MustCompile(`(/[\w\d\.]+\.key)`)
matchCert = regexp.MustCompile(`(/[\w\d\.]+\.crt)`)
)
type configLoadError struct {
Response string
}
func (e configLoadError) Error() string { return e.Response }
// InitServer this will configure the server with a configurion of a specific
// type. The configType must be either "json" or the adapter type.
func InitServer(t *testing.T, rawConfig string, configType string) {
if err := initServer(t, rawConfig, configType); errors.Is(err, &configLoadError{}) {
t.Logf("failed to load config: %s", err)
t.Fail()
}
}
// InitServer this will configure the server with a configurion of a specific
// type. The configType must be either "json" or the adapter type.
func initServer(t *testing.T, rawConfig string, configType string) error {
err := validateTestPrerequisites()
if err != nil {
t.Skipf("skipping tests as failed integration prerequisites. %s", err)
return nil
}
t.Cleanup(func() {
if t.Failed() {
res, err := http.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
if err != nil {
t.Log("unable to read the current config")
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
var out bytes.Buffer
json.Indent(&out, body, "", " ")
t.Logf("----------- failed with config -----------\n%s", out.String())
}
})
rawConfig = prependCaddyFilePath(rawConfig)
client := &http.Client{
Timeout: time.Second * 2,
}
req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/load", Default.AdminPort), strings.NewReader(rawConfig))
if err != nil {
t.Errorf("failed to create request. %s", err)
return err
}
if configType == "json" {
req.Header.Add("Content-Type", "application/json")
} else {
req.Header.Add("Content-Type", "text/"+configType)
}
res, err := client.Do(req)
if err != nil {
t.Errorf("unable to contact caddy server. %s", err)
return err
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
t.Errorf("unable to read response. %s", err)
return err
}
if res.StatusCode != 200 {
return configLoadError{Response: string(body)}
}
return nil
}
var hasValidated bool
var arePrerequisitesValid bool
func validateTestPrerequisites() error {
if hasValidated {
if !arePrerequisitesValid {
return errors.New("caddy integration prerequisites failed. see first error")
}
return nil
}
hasValidated = true
arePrerequisitesValid = false
// check certificates are found
for _, certName := range Default.Certifcates {
if _, err := os.Stat(getIntegrationDir() + certName); os.IsNotExist(err) {
return fmt.Errorf("caddy integration test certificates (%s) not found", certName)
}
}
// assert that caddy is running
client := &http.Client{
Timeout: time.Second * 2,
}
_, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
if err != nil {
return errors.New("caddy integration test caddy server not running. Expected to be listening on localhost:2019")
}
arePrerequisitesValid = true
return nil
}
func getIntegrationDir() string {
_, filename, _, ok := runtime.Caller(1)
if !ok {
panic("unable to determine the current file path")
}
return path.Dir(filename)
}
// use the convention to replace /[certificatename].[crt|key] with the full path
// this helps reduce the noise in test configurations and also allow this
// to run in any path
func prependCaddyFilePath(rawConfig string) string {
r := matchKey.ReplaceAllString(rawConfig, getIntegrationDir()+"$1")
r = matchCert.ReplaceAllString(r, getIntegrationDir()+"$1")
return r
}
// creates a testing transport that forces call dialing connections to happen locally
func createTestingTransport() *http.Transport {
dialer := net.Dialer{
Timeout: 5 * time.Second,
KeepAlive: 5 * time.Second,
DualStack: true,
}
dialContext := func(ctx context.Context, network, addr string) (net.Conn, error) {
parts := strings.Split(addr, ":")
destAddr := fmt.Sprintf("127.0.0.1:%s", parts[1])
log.Printf("caddytest: redirecting the dialer from %s to %s", addr, destAddr)
return dialer.DialContext(ctx, network, destAddr)
}
return &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: dialContext,
ForceAttemptHTTP2: true,
MaxIdleConns: 100,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 5 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
}
// AssertLoadError will load a config and expect an error
func AssertLoadError(t *testing.T, rawConfig string, configType string, expectedError string) {
err := initServer(t, rawConfig, configType)
if !strings.Contains(err.Error(), expectedError) {
t.Errorf("expected error \"%s\" but got \"%s\"", expectedError, err.Error())
}
}
// AssertGetResponse request a URI and assert the status code and the body contains a string
func AssertGetResponse(t *testing.T, requestURI string, statusCode int, expectedBody string) (*http.Response, string) {
resp, body := AssertGetResponseBody(t, requestURI, statusCode)
if !strings.Contains(body, expectedBody) {
t.Errorf("expected response body \"%s\" but got \"%s\"", expectedBody, body)
}
return resp, string(body)
}
// AssertGetResponseBody request a URI and assert the status code matches
func AssertGetResponseBody(t *testing.T, requestURI string, expectedStatusCode int) (*http.Response, string) {
client := &http.Client{
Transport: createTestingTransport(),
}
resp, err := client.Get(requestURI)
if err != nil {
t.Errorf("failed to call server %s", err)
return nil, ""
}
defer resp.Body.Close()
if expectedStatusCode != resp.StatusCode {
t.Errorf("expected status code: %d but got %d", expectedStatusCode, resp.StatusCode)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
t.Errorf("unable to read the response body %s", err)
return nil, ""
}
return resp, string(body)
}
// AssertRedirect makes a request and asserts the redirection happens
func AssertRedirect(t *testing.T, requestURI string, expectedToLocation string, expectedStatusCode int) *http.Response {
redirectPolicyFunc := func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
client := &http.Client{
CheckRedirect: redirectPolicyFunc,
Transport: createTestingTransport(),
}
resp, err := client.Get(requestURI)
if err != nil {
t.Errorf("failed to call server %s", err)
return nil
}
if expectedStatusCode != resp.StatusCode {
t.Errorf("expected status code: %d but got %d", expectedStatusCode, resp.StatusCode)
}
loc, err := resp.Location()
if expectedToLocation != loc.String() {
t.Errorf("expected location: \"%s\" but got \"%s\"", expectedToLocation, loc.String())
}
return resp
}
+33
View File
@@ -0,0 +1,33 @@
package caddytest
import (
"strings"
"testing"
)
func TestReplaceCertificatePaths(t *testing.T) {
rawConfig := `a.caddy.localhost:9443 {
tls /caddy.localhost.crt /caddy.localhost.key {
}
redir / https://b.caddy.localhost:9443/version 301
respond /version 200 {
body "hello from a.caddy.localhost"
}
}`
r := prependCaddyFilePath(rawConfig)
if !strings.Contains(r, getIntegrationDir()+"/caddy.localhost.crt") {
t.Error("expected the /caddy.localhost.crt to be expanded to include the full path")
}
if !strings.Contains(r, getIntegrationDir()+"/caddy.localhost.key") {
t.Error("expected the /caddy.localhost.crt to be expanded to include the full path")
}
if !strings.Contains(r, "https://b.caddy.localhost:9443/version") {
t.Error("expected redirect uri to be unchanged")
}
}
+91
View File
@@ -0,0 +1,91 @@
package integration
import (
"testing"
"github.com/caddyserver/caddy/v2/caddytest"
)
func TestRespond(t *testing.T) {
// arrange
caddytest.InitServer(t, `
{
http_port 9080
https_port 9443
}
localhost:9080 {
respond /version 200 {
body "hello from a.caddy.localhost"
}
}
`, "caddyfile")
// act and assert
caddytest.AssertGetResponse(t, "http://localhost:9080/version", 200, "hello from a.caddy.localhost")
}
func TestRedirect(t *testing.T) {
// arrange
caddytest.InitServer(t, `
{
http_port 9080
https_port 9443
}
localhost:9080 {
redir / http://localhost:9080/hello 301
respond /hello 200 {
body "hello from b"
}
}
`, "caddyfile")
// act and assert
caddytest.AssertRedirect(t, "http://localhost:9080/", "http://localhost:9080/hello", 301)
// follow redirect
caddytest.AssertGetResponse(t, "http://localhost:9080/", 200, "hello from b")
}
func TestDuplicateHosts(t *testing.T) {
// act and assert
caddytest.AssertLoadError(t,
`
localhost:9080 {
}
localhost:9080 {
}
`,
"caddyfile",
"duplicate site address not allowed")
}
func TestDefaultSNI(t *testing.T) {
// arrange
caddytest.InitServer(t, `
{
http_port 9080
https_port 9443
default_sni *.caddy.localhost
}
127.0.0.1:9443 {
tls /caddy.localhost.crt /caddy.localhost.key {
}
respond /version 200 {
body "hello from a.caddy.localhost"
}
}
`, "caddyfile")
// act and assert
caddytest.AssertGetResponse(t, "https://127.0.0.1:9443/version", 200, "hello from a.caddy.localhost")
}
+38
View File
@@ -0,0 +1,38 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package main is the entry point of the Caddy application.
// Most of Caddy's functionality is provided through modules,
// which can be plugged in by adding their import below.
//
// There is no need to modify the Caddy source code to customize your
// builds. You can easily build a custom Caddy with these simple steps:
//
// 1. Copy this file (main.go) into a new folder
// 2. Edit the imports below to include the modules you want plugged in
// 3. Run `go mod init caddy`
// 4. Run `go install` or `go build` - you now have a custom binary!
//
package main
import (
caddycmd "github.com/caddyserver/caddy/v2/cmd"
// plug in Caddy modules here
_ "github.com/caddyserver/caddy/v2/modules/standard"
)
func main() {
caddycmd.Main()
}
+627
View File
@@ -0,0 +1,627 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddycmd
import (
"bytes"
"crypto/rand"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net"
"net/http"
"os"
"os/exec"
"reflect"
"runtime"
"runtime/debug"
"sort"
"strings"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
"github.com/caddyserver/certmagic"
"go.uber.org/zap"
)
func cmdStart(fl Flags) (int, error) {
startCmdConfigFlag := fl.String("config")
startCmdConfigAdapterFlag := fl.String("adapter")
// open a listener to which the child process will connect when
// it is ready to confirm that it has successfully started
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("opening listener for success confirmation: %v", err)
}
defer ln.Close()
// craft the command with a pingback address and with a
// pipe for its stdin, so we can tell it our confirmation
// code that we expect so that some random port scan at
// the most unfortunate time won't fool us into thinking
// the child succeeded (i.e. the alternative is to just
// wait for any connection on our listener, but better to
// ensure it's the process we're expecting - we can be
// sure by giving it some random bytes and having it echo
// them back to us)
cmd := exec.Command(os.Args[0], "run", "--pingback", ln.Addr().String())
if startCmdConfigFlag != "" {
cmd.Args = append(cmd.Args, "--config", startCmdConfigFlag)
}
if startCmdConfigAdapterFlag != "" {
cmd.Args = append(cmd.Args, "--adapter", startCmdConfigAdapterFlag)
}
stdinpipe, err := cmd.StdinPipe()
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("creating stdin pipe: %v", err)
}
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
// generate the random bytes we'll send to the child process
expect := make([]byte, 32)
_, err = rand.Read(expect)
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("generating random confirmation bytes: %v", err)
}
// begin writing the confirmation bytes to the child's
// stdin; use a goroutine since the child hasn't been
// started yet, and writing synchronously would result
// in a deadlock
go func() {
stdinpipe.Write(expect)
stdinpipe.Close()
}()
// start the process
err = cmd.Start()
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("starting caddy process: %v", err)
}
// there are two ways we know we're done: either
// the process will connect to our listener, or
// it will exit with an error
success, exit := make(chan struct{}), make(chan error)
// in one goroutine, we await the success of the child process
go func() {
for {
conn, err := ln.Accept()
if err != nil {
if !strings.Contains(err.Error(), "use of closed network connection") {
log.Println(err)
}
break
}
err = handlePingbackConn(conn, expect)
if err == nil {
close(success)
break
}
log.Println(err)
}
}()
// in another goroutine, we await the failure of the child process
go func() {
err := cmd.Wait() // don't send on this line! Wait blocks, but send starts before it unblocks
exit <- err // sending on separate line ensures select won't trigger until after Wait unblocks
}()
// when one of the goroutines unblocks, we're done and can exit
select {
case <-success:
fmt.Printf("Successfully started Caddy (pid=%d) - Caddy is running in the background\n", cmd.Process.Pid)
case err := <-exit:
return caddy.ExitCodeFailedStartup,
fmt.Errorf("caddy process exited with error: %v", err)
}
return caddy.ExitCodeSuccess, nil
}
func cmdRun(fl Flags) (int, error) {
runCmdConfigFlag := fl.String("config")
runCmdConfigAdapterFlag := fl.String("adapter")
runCmdResumeFlag := fl.Bool("resume")
runCmdPrintEnvFlag := fl.Bool("environ")
runCmdPingbackFlag := fl.String("pingback")
// if we are supposed to print the environment, do that first
if runCmdPrintEnvFlag {
printEnvironment()
}
// TODO: This is TEMPORARY, until the RCs
moveStorage()
// load the config, depending on flags
var config []byte
var err error
if runCmdResumeFlag {
config, err = ioutil.ReadFile(caddy.ConfigAutosavePath)
if os.IsNotExist(err) {
// not a bad error; just can't resume if autosave file doesn't exist
caddy.Log().Info("no autosave file exists", zap.String("autosave_file", caddy.ConfigAutosavePath))
runCmdResumeFlag = false
} else if err != nil {
return caddy.ExitCodeFailedStartup, err
} else {
caddy.Log().Info("resuming from last configuration", zap.String("autosave_file", caddy.ConfigAutosavePath))
}
}
// we don't use 'else' here since this value might have been changed in 'if' block; i.e. not mutually exclusive
if !runCmdResumeFlag {
config, _, err = loadConfig(runCmdConfigFlag, runCmdConfigAdapterFlag)
if err != nil {
return caddy.ExitCodeFailedStartup, err
}
}
// set a fitting User-Agent for ACME requests
goModule := caddy.GoModule()
cleanModVersion := strings.TrimPrefix(goModule.Version, "v")
certmagic.UserAgent = "Caddy/" + cleanModVersion
// run the initial config
err = caddy.Load(config, true)
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("loading initial config: %v", err)
}
caddy.Log().Info("serving initial configuration")
// if we are to report to another process the successful start
// of the server, do so now by echoing back contents of stdin
if runCmdPingbackFlag != "" {
confirmationBytes, err := ioutil.ReadAll(os.Stdin)
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("reading confirmation bytes from stdin: %v", err)
}
conn, err := net.Dial("tcp", runCmdPingbackFlag)
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("dialing confirmation address: %v", err)
}
defer conn.Close()
_, err = conn.Write(confirmationBytes)
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("writing confirmation bytes to %s: %v", runCmdPingbackFlag, err)
}
}
// warn if the environment does not provide enough information about the disk
hasXDG := os.Getenv("XDG_DATA_HOME") != "" &&
os.Getenv("XDG_CONFIG_HOME") != "" &&
os.Getenv("XDG_CACHE_HOME") != ""
switch runtime.GOOS {
case "windows":
if os.Getenv("HOME") == "" && os.Getenv("USERPROFILE") == "" && !hasXDG {
caddy.Log().Warn("neither HOME nor USERPROFILE environment variables are set - please fix; some assets might be stored in ./caddy")
}
case "plan9":
if os.Getenv("home") == "" && !hasXDG {
caddy.Log().Warn("$home environment variable is empty - please fix; some assets might be stored in ./caddy")
}
default:
if os.Getenv("HOME") == "" && !hasXDG {
caddy.Log().Warn("$HOME environment variable is empty - please fix; some assets might be stored in ./caddy")
}
}
select {}
}
func cmdStop(fl Flags) (int, error) {
stopCmdAddrFlag := fl.String("address")
adminAddr := caddy.DefaultAdminListen
if stopCmdAddrFlag != "" {
adminAddr = stopCmdAddrFlag
}
stopEndpoint := fmt.Sprintf("http://%s/stop", adminAddr)
req, err := http.NewRequest(http.MethodPost, stopEndpoint, nil)
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("making request: %v", err)
}
req.Header.Set("Origin", adminAddr)
err = apiRequest(req)
if err != nil {
caddy.Log().Warn("failed using API to stop instance",
zap.String("endpoint", stopEndpoint),
zap.Error(err),
)
return caddy.ExitCodeFailedStartup, err
}
return caddy.ExitCodeSuccess, nil
}
func cmdReload(fl Flags) (int, error) {
reloadCmdConfigFlag := fl.String("config")
reloadCmdConfigAdapterFlag := fl.String("adapter")
reloadCmdAddrFlag := fl.String("address")
// get the config in caddy's native format
config, hasConfig, err := loadConfig(reloadCmdConfigFlag, reloadCmdConfigAdapterFlag)
if err != nil {
return caddy.ExitCodeFailedStartup, err
}
if !hasConfig {
return caddy.ExitCodeFailedStartup, fmt.Errorf("no config file to load")
}
// get the address of the admin listener and craft endpoint URL
adminAddr := reloadCmdAddrFlag
if adminAddr == "" && len(config) > 0 {
var tmpStruct struct {
Admin caddy.AdminConfig `json:"admin"`
}
err = json.Unmarshal(config, &tmpStruct)
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("unmarshaling admin listener address from config: %v", err)
}
adminAddr = tmpStruct.Admin.Listen
}
if adminAddr == "" {
adminAddr = caddy.DefaultAdminListen
}
loadEndpoint := fmt.Sprintf("http://%s/load", adminAddr)
// prepare the request to update the configuration
req, err := http.NewRequest(http.MethodPost, loadEndpoint, bytes.NewReader(config))
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("making request: %v", err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Origin", adminAddr)
err = apiRequest(req)
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("sending configuration to instance: %v", err)
}
return caddy.ExitCodeSuccess, nil
}
func cmdVersion(_ Flags) (int, error) {
goModule := caddy.GoModule()
fmt.Print(goModule.Version)
if goModule.Sum != "" {
// a build with a known version will also have a checksum
fmt.Printf(" %s", goModule.Sum)
}
if goModule.Replace != nil {
fmt.Printf(" => %s", goModule.Replace.Path)
if goModule.Replace.Version != "" {
fmt.Printf(" %s", goModule.Replace.Version)
}
}
fmt.Println()
return caddy.ExitCodeSuccess, nil
}
func cmdBuildInfo(fl Flags) (int, error) {
bi, ok := debug.ReadBuildInfo()
if !ok {
return caddy.ExitCodeFailedStartup, fmt.Errorf("no build information")
}
fmt.Printf("path: %s\n", bi.Path)
fmt.Printf("main: %s %s %s\n", bi.Main.Path, bi.Main.Version, bi.Main.Sum)
fmt.Println("dependencies:")
for _, goMod := range bi.Deps {
fmt.Printf("%s %s %s", goMod.Path, goMod.Version, goMod.Sum)
if goMod.Replace != nil {
fmt.Printf(" => %s %s %s", goMod.Replace.Path, goMod.Replace.Version, goMod.Replace.Sum)
}
fmt.Println()
}
return caddy.ExitCodeSuccess, nil
}
func cmdListModules(fl Flags) (int, error) {
versions := fl.Bool("versions")
bi, ok := debug.ReadBuildInfo()
if !ok || !versions {
// if there's no build information,
// just print out the modules
for _, m := range caddy.Modules() {
fmt.Println(m)
}
return caddy.ExitCodeSuccess, nil
}
for _, modID := range caddy.Modules() {
modInfo, err := caddy.GetModule(modID)
if err != nil {
// that's weird
fmt.Println(modID)
continue
}
// to get the Caddy plugin's version info, we need to know
// the package that the Caddy module's value comes from; we
// can use reflection but we need a non-pointer value (I'm
// not sure why), and since New() should return a pointer
// value, we need to dereference it first
iface := interface{}(modInfo.New())
if rv := reflect.ValueOf(iface); rv.Kind() == reflect.Ptr {
iface = reflect.New(reflect.TypeOf(iface).Elem()).Elem().Interface()
}
modPkgPath := reflect.TypeOf(iface).PkgPath()
// now we find the Go module that the Caddy module's package
// belongs to; we assume the Caddy module package path will
// be prefixed by its Go module path, and we will choose the
// longest matching prefix in case there are nested modules
var matched *debug.Module
for _, dep := range bi.Deps {
if strings.HasPrefix(modPkgPath, dep.Path) {
if matched == nil || len(dep.Path) > len(matched.Path) {
matched = dep
}
}
}
// if we could find no matching module, just print out
// the module ID instead
if matched == nil {
fmt.Println(modID)
continue
}
fmt.Printf("%s %s\n", modID, matched.Version)
}
return caddy.ExitCodeSuccess, nil
}
func cmdEnviron(_ Flags) (int, error) {
printEnvironment()
return caddy.ExitCodeSuccess, nil
}
func cmdAdaptConfig(fl Flags) (int, error) {
adaptCmdInputFlag := fl.String("config")
adaptCmdAdapterFlag := fl.String("adapter")
adaptCmdPrettyFlag := fl.Bool("pretty")
adaptCmdValidateFlag := fl.Bool("validate")
// if no input file was specified, try a default
// Caddyfile if the Caddyfile adapter is plugged in
if adaptCmdInputFlag == "" && caddyconfig.GetAdapter("caddyfile") != nil {
_, err := os.Stat("Caddyfile")
if err == nil {
// default Caddyfile exists
adaptCmdInputFlag = "Caddyfile"
caddy.Log().Info("using adjacent Caddyfile")
} else if !os.IsNotExist(err) {
// default Caddyfile exists, but error accessing it
return caddy.ExitCodeFailedStartup, fmt.Errorf("accessing default Caddyfile: %v", err)
}
}
if adaptCmdInputFlag == "" {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("input file required when there is no Caddyfile in current directory (use --config flag)")
}
if adaptCmdAdapterFlag == "" {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("adapter name is required (use --adapt flag or leave unspecified for default)")
}
cfgAdapter := caddyconfig.GetAdapter(adaptCmdAdapterFlag)
if cfgAdapter == nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("unrecognized config adapter: %s", adaptCmdAdapterFlag)
}
input, err := ioutil.ReadFile(adaptCmdInputFlag)
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("reading input file: %v", err)
}
opts := make(map[string]interface{})
if adaptCmdPrettyFlag {
opts["pretty"] = "true"
}
opts["filename"] = adaptCmdInputFlag
adaptedConfig, warnings, err := cfgAdapter.Adapt(input, opts)
if err != nil {
return caddy.ExitCodeFailedStartup, err
}
// print warnings to stderr
for _, warn := range warnings {
msg := warn.Message
if warn.Directive != "" {
msg = fmt.Sprintf("%s: %s", warn.Directive, warn.Message)
}
fmt.Fprintf(os.Stderr, "[WARNING][%s] %s:%d: %s\n", adaptCmdAdapterFlag, warn.File, warn.Line, msg)
}
// print result to stdout
fmt.Println(string(adaptedConfig))
// validate output if requested
if adaptCmdValidateFlag {
var cfg *caddy.Config
err = json.Unmarshal(adaptedConfig, &cfg)
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("decoding config: %v", err)
}
err = caddy.Validate(cfg)
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("validation: %v", err)
}
}
return caddy.ExitCodeSuccess, nil
}
func cmdValidateConfig(fl Flags) (int, error) {
validateCmdConfigFlag := fl.String("config")
validateCmdAdapterFlag := fl.String("adapter")
input, _, err := loadConfig(validateCmdConfigFlag, validateCmdAdapterFlag)
if err != nil {
return caddy.ExitCodeFailedStartup, err
}
input = caddy.RemoveMetaFields(input)
var cfg *caddy.Config
err = json.Unmarshal(input, &cfg)
if err != nil {
return caddy.ExitCodeFailedStartup, fmt.Errorf("decoding config: %v", err)
}
err = caddy.Validate(cfg)
if err != nil {
return caddy.ExitCodeFailedStartup, err
}
fmt.Println("Valid configuration")
return caddy.ExitCodeSuccess, nil
}
func cmdFormatConfig(fl Flags) (int, error) {
// Default path of file is Caddyfile
formatCmdConfigFile := fl.Arg(0)
if formatCmdConfigFile == "" {
formatCmdConfigFile = "Caddyfile"
}
formatCmdWriteFlag := fl.Bool("write")
input, err := ioutil.ReadFile(formatCmdConfigFile)
if err != nil {
return caddy.ExitCodeFailedStartup,
fmt.Errorf("reading input file: %v", err)
}
output := caddyfile.Format(input)
if formatCmdWriteFlag {
err = ioutil.WriteFile(formatCmdConfigFile, output, 0644)
if err != nil {
return caddy.ExitCodeFailedStartup, nil
}
} else {
fmt.Print(string(output))
}
return caddy.ExitCodeSuccess, nil
}
func cmdHelp(fl Flags) (int, error) {
const fullDocs = `Full documentation is available at:
https://caddyserver.com/docs/command-line`
args := fl.Args()
if len(args) == 0 {
s := `Caddy is an extensible server platform.
usage:
caddy <command> [<args...>]
commands:
`
keys := make([]string, 0, len(commands))
for k := range commands {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
cmd := commands[k]
short := strings.TrimSuffix(cmd.Short, ".")
s += fmt.Sprintf(" %-15s %s\n", cmd.Name, short)
}
s += "\nUse 'caddy help <command>' for more information about a command.\n"
s += "\n" + fullDocs + "\n"
fmt.Print(s)
return caddy.ExitCodeSuccess, nil
} else if len(args) > 1 {
return caddy.ExitCodeFailedStartup, fmt.Errorf("can only give help with one command")
}
subcommand, ok := commands[args[0]]
if !ok {
return caddy.ExitCodeFailedStartup, fmt.Errorf("unknown command: %s", args[0])
}
helpText := strings.TrimSpace(subcommand.Long)
if helpText == "" {
helpText = subcommand.Short
if !strings.HasSuffix(helpText, ".") {
helpText += "."
}
}
result := fmt.Sprintf("%s\n\nusage:\n caddy %s %s\n",
helpText,
subcommand.Name,
strings.TrimSpace(subcommand.Usage),
)
if help := flagHelp(subcommand.Flags); help != "" {
result += fmt.Sprintf("\nflags:\n%s", help)
}
result += "\n" + fullDocs + "\n"
fmt.Print(result)
return caddy.ExitCodeSuccess, nil
}
func apiRequest(req *http.Request) error {
resp, err := http.DefaultClient.Do(req)
if err != nil {
return fmt.Errorf("performing request: %v", err)
}
defer resp.Body.Close()
// if it didn't work, let the user know
if resp.StatusCode >= 400 {
respBody, err := ioutil.ReadAll(io.LimitReader(resp.Body, 1024*10))
if err != nil {
return fmt.Errorf("HTTP %d: reading error message: %v", resp.StatusCode, err)
}
return fmt.Errorf("caddy responded with error: HTTP %d: %s", resp.StatusCode, respBody)
}
return nil
}
+298
View File
@@ -0,0 +1,298 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddycmd
import (
"flag"
"regexp"
)
// Command represents a subcommand. Name, Func,
// and Short are required.
type Command struct {
// The name of the subcommand. Must conform to the
// format described by the RegisterCommand() godoc.
// Required.
Name string
// Run is a function that executes a subcommand using
// the parsed flags. It returns an exit code and any
// associated error.
// Required.
Func CommandFunc
// Usage is a brief message describing the syntax of
// the subcommand's flags and args. Use [] to indicate
// optional parameters and <> to enclose literal values
// intended to be replaced by the user. Do not prefix
// the string with "caddy" or the name of the command
// since these will be prepended for you; only include
// the actual parameters for this command.
Usage string
// Short is a one-line message explaining what the
// command does. Should not end with punctuation.
// Required.
Short string
// Long is the full help text shown to the user.
// Will be trimmed of whitespace on both ends before
// being printed.
Long string
// Flags is the flagset for command.
Flags *flag.FlagSet
}
// CommandFunc is a command's function. It runs the
// command and returns the proper exit code along with
// any error that occurred.
type CommandFunc func(Flags) (int, error)
var commands = make(map[string]Command)
func init() {
RegisterCommand(Command{
Name: "help",
Func: cmdHelp,
Usage: "<command>",
Short: "Shows help for a Caddy subcommand",
})
RegisterCommand(Command{
Name: "start",
Func: cmdStart,
Usage: "[--config <path> [[--adapter <name>]]",
Short: "Starts the Caddy process in the background and then returns",
Long: `
Starts the Caddy process, optionally bootstrapped with an initial config file.
This command unblocks after the server starts running or fails to run.
On Windows, the spawned child process will remain attached to the terminal, so
closing the window will forcefully stop Caddy; to avoid forgetting this, try
using 'caddy run' instead to keep it in the foreground.`,
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("start", flag.ExitOnError)
fs.String("config", "", "Configuration file")
fs.String("adapter", "", "Name of config adapter to apply")
return fs
}(),
})
RegisterCommand(Command{
Name: "run",
Func: cmdRun,
Usage: "[--config <path> [--adapter <name>]] [--environ]",
Short: `Starts the Caddy process and blocks indefinitely`,
Long: `
Starts the Caddy process, optionally bootstrapped with an initial config file,
and blocks indefinitely until the server is stopped; i.e. runs Caddy in
"daemon" mode (foreground).
If a config file is specified, it will be applied immediately after the process
is running. If the config file is not in Caddy's native JSON format, you can
specify an adapter with --adapter to adapt the given config file to
Caddy's native format. The config adapter must be a registered module. Any
warnings will be printed to the log, but beware that any adaptation without
errors will immediately be used. If you want to review the results of the
adaptation first, use the 'adapt' subcommand.
As a special case, if the current working directory has a file called
"Caddyfile" and the caddyfile config adapter is plugged in (default), then
that file will be loaded and used to configure Caddy, even without any command
line flags.
If --environ is specified, the environment as seen by the Caddy process will
be printed before starting. This is the same as the environ command but does
not quit after printing, and can be useful for troubleshooting.
The --resume flag will override the --config flag if there is a config auto-
save file. It is not an error if --resume is used and no autosave file exists.`,
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("run", flag.ExitOnError)
fs.String("config", "", "Configuration file")
fs.String("adapter", "", "Name of config adapter to apply")
fs.Bool("resume", false, "Use saved config, if any (and prefer over --config file)")
fs.Bool("environ", false, "Print environment")
fs.String("pingback", "", "Echo confirmation bytes to this address on success")
return fs
}(),
})
RegisterCommand(Command{
Name: "stop",
Func: cmdStop,
Short: "Gracefully stops a started Caddy process",
Long: `
Stops the background Caddy process as gracefully as possible.
It requires that the admin API is enabled and accessible, since it will
use the API's /stop endpoint. The address of this request can be
customized using the --address flag if it is not the default.`,
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("stop", flag.ExitOnError)
fs.String("address", "", "The address to use to reach the admin API endpoint, if not the default")
return fs
}(),
})
RegisterCommand(Command{
Name: "reload",
Func: cmdReload,
Usage: "--config <path> [--adapter <name>] [--address <interface>]",
Short: "Changes the config of the running Caddy instance",
Long: `
Gives the running Caddy instance a new configuration. This has the same effect
as POSTing a document to the /load API endpoint, but is convenient for simple
workflows revolving around config files.
Since the admin endpoint is configurable, the endpoint configuration is loaded
from the --address flag if specified; otherwise it is loaded from the given
config file; otherwise the default is assumed.`,
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("reload", flag.ExitOnError)
fs.String("config", "", "Configuration file (required)")
fs.String("adapter", "", "Name of config adapter to apply")
fs.String("address", "", "Address of the administration listener, if different from config")
return fs
}(),
})
RegisterCommand(Command{
Name: "version",
Func: cmdVersion,
Short: "Prints the version",
})
RegisterCommand(Command{
Name: "list-modules",
Func: cmdListModules,
Usage: "[--versions]",
Short: "Lists the installed Caddy modules",
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("list-modules", flag.ExitOnError)
fs.Bool("versions", false, "Print version information")
return fs
}(),
})
RegisterCommand(Command{
Name: "build-info",
Func: cmdBuildInfo,
Short: "Prints information about this build",
})
RegisterCommand(Command{
Name: "environ",
Func: cmdEnviron,
Short: "Prints the environment",
})
RegisterCommand(Command{
Name: "adapt",
Func: cmdAdaptConfig,
Usage: "--config <path> [--adapter <name>] [--pretty] [--validate]",
Short: "Adapts a configuration to Caddy's native JSON",
Long: `
Adapts a configuration to Caddy's native JSON format and writes the
output to stdout, along with any warnings to stderr.
If --pretty is specified, the output will be formatted with indentation
for human readability.
If --validate is used, the adapted config will be checked for validity.
If the config is invalid, an error will be printed to stderr and a non-
zero exit status will be returned.`,
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("adapt", flag.ExitOnError)
fs.String("config", "", "Configuration file to adapt (required)")
fs.String("adapter", "caddyfile", "Name of config adapter")
fs.Bool("pretty", false, "Format the output for human readability")
fs.Bool("validate", false, "Validate the output")
return fs
}(),
})
RegisterCommand(Command{
Name: "validate",
Func: cmdValidateConfig,
Usage: "--config <path> [--adapter <name>]",
Short: "Tests whether a configuration file is valid",
Long: `
Loads and provisions the provided config, but does not start running it.
This reveals any errors with the configuration through the loading and
provisioning stages.`,
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("load", flag.ExitOnError)
fs.String("config", "", "Input configuration file")
fs.String("adapter", "", "Name of config adapter")
return fs
}(),
})
RegisterCommand(Command{
Name: "fmt",
Func: cmdFormatConfig,
Usage: "[--write] [<path>]",
Short: "Formats a Caddyfile",
Long: `
Formats the Caddyfile by adding proper indentation and spaces to improve
human readability. It prints the result to stdout.
If --write is specified, the output will be written to the config file
directly instead of printing it.`,
Flags: func() *flag.FlagSet {
fs := flag.NewFlagSet("format", flag.ExitOnError)
fs.Bool("write", false, "Over-write the output to specified file")
return fs
}(),
})
}
// RegisterCommand registers the command cmd.
// cmd.Name must be unique and conform to the
// following format:
//
// - lowercase
// - alphanumeric and hyphen characters only
// - cannot start or end with a hyphen
// - hyphen cannot be adjacent to another hyphen
//
// This function panics if the name is already registered,
// if the name does not meet the described format, or if
// any of the fields are missing from cmd.
//
// This function should be used in init().
func RegisterCommand(cmd Command) {
if cmd.Name == "" {
panic("command name is required")
}
if cmd.Func == nil {
panic("command function missing")
}
if cmd.Short == "" {
panic("command short string is required")
}
if _, exists := commands[cmd.Name]; exists {
panic("command already registered: " + cmd.Name)
}
if !commandNameRegex.MatchString(cmd.Name) {
panic("invalid command name")
}
commands[cmd.Name] = cmd
}
var commandNameRegex = regexp.MustCompile(`^[a-z0-9]$|^([a-z0-9]+-?[a-z0-9]*)+[a-z0-9]$`)
+331
View File
@@ -0,0 +1,331 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddycmd
import (
"bytes"
"flag"
"fmt"
"io"
"io/ioutil"
"net"
"os"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
"github.com/caddyserver/caddy/v2"
"github.com/caddyserver/caddy/v2/caddyconfig"
"go.uber.org/zap"
)
// Main implements the main function of the caddy command.
// Call this if Caddy is to be the main() if your program.
func Main() {
caddy.TrapSignals()
switch len(os.Args) {
case 0:
fmt.Printf("[FATAL] no arguments provided by OS; args[0] must be command\n")
os.Exit(caddy.ExitCodeFailedStartup)
case 1:
os.Args = append(os.Args, "help")
}
subcommandName := os.Args[1]
subcommand, ok := commands[subcommandName]
if !ok {
if strings.HasPrefix(os.Args[1], "-") {
// user probably forgot to type the subcommand
fmt.Println("[ERROR] first argument must be a subcommand; see 'caddy help'")
} else {
fmt.Printf("[ERROR] '%s' is not a recognized subcommand; see 'caddy help'\n", os.Args[1])
}
os.Exit(caddy.ExitCodeFailedStartup)
}
fs := subcommand.Flags
if fs == nil {
fs = flag.NewFlagSet(subcommand.Name, flag.ExitOnError)
}
err := fs.Parse(os.Args[2:])
if err != nil {
fmt.Println(err)
os.Exit(caddy.ExitCodeFailedStartup)
}
exitCode, err := subcommand.Func(Flags{fs})
if err != nil {
fmt.Fprintf(os.Stderr, "%s: %v\n", subcommand.Name, err)
}
os.Exit(exitCode)
}
// handlePingbackConn reads from conn and ensures it matches
// the bytes in expect, or returns an error if it doesn't.
func handlePingbackConn(conn net.Conn, expect []byte) error {
defer conn.Close()
confirmationBytes, err := ioutil.ReadAll(io.LimitReader(conn, 32))
if err != nil {
return err
}
if !bytes.Equal(confirmationBytes, expect) {
return fmt.Errorf("wrong confirmation: %x", confirmationBytes)
}
return nil
}
// loadConfig loads the config from configFile and adapts it
// using adapterName. If adapterName is specified, configFile
// must be also. If no configFile is specified, it tries
// loading a default config file. The lack of a config file is
// not treated as an error, but false will be returned if
// there is no config available. It prints any warnings to stderr,
// and returns the resulting JSON config bytes along with
// whether a config file was loaded or not.
func loadConfig(configFile, adapterName string) ([]byte, bool, error) {
// specifying an adapter without a config file is ambiguous
if adapterName != "" && configFile == "" {
return nil, false, fmt.Errorf("cannot adapt config without config file (use --config)")
}
// load initial config and adapter
var config []byte
var cfgAdapter caddyconfig.Adapter
var err error
if configFile != "" {
config, err = ioutil.ReadFile(configFile)
if err != nil {
return nil, false, fmt.Errorf("reading config file: %v", err)
}
caddy.Log().Info("using provided configuration",
zap.String("config_file", configFile),
zap.String("config_adapter", adapterName))
} else if adapterName == "" {
// as a special case when no config file or adapter
// is specified, see if the Caddyfile adapter is
// plugged in, and if so, try using a default Caddyfile
cfgAdapter = caddyconfig.GetAdapter("caddyfile")
if cfgAdapter != nil {
config, err = ioutil.ReadFile("Caddyfile")
if os.IsNotExist(err) {
// okay, no default Caddyfile; pretend like this never happened
cfgAdapter = nil
} else if err != nil {
// default Caddyfile exists, but error reading it
return nil, false, fmt.Errorf("reading default Caddyfile: %v", err)
} else {
// success reading default Caddyfile
configFile = "Caddyfile"
caddy.Log().Info("using adjacent Caddyfile")
}
}
}
// as a special case, if a config file called "Caddyfile" was
// specified, and no adapter is specified, assume caddyfile adapter
// for convenience
if strings.HasPrefix(filepath.Base(configFile), "Caddyfile") &&
filepath.Ext(configFile) != ".json" &&
adapterName == "" {
adapterName = "caddyfile"
}
// load config adapter
if adapterName != "" {
cfgAdapter = caddyconfig.GetAdapter(adapterName)
if cfgAdapter == nil {
return nil, false, fmt.Errorf("unrecognized config adapter: %s", adapterName)
}
}
// adapt config
if cfgAdapter != nil {
adaptedConfig, warnings, err := cfgAdapter.Adapt(config, map[string]interface{}{
"filename": configFile,
})
if err != nil {
return nil, false, fmt.Errorf("adapting config using %s: %v", adapterName, err)
}
for _, warn := range warnings {
msg := warn.Message
if warn.Directive != "" {
msg = fmt.Sprintf("%s: %s", warn.Directive, warn.Message)
}
fmt.Printf("[WARNING][%s] %s:%d: %s\n", adapterName, warn.File, warn.Line, msg)
}
config = adaptedConfig
}
return config, configFile != "", nil
}
// Flags wraps a FlagSet so that typed values
// from flags can be easily retrieved.
type Flags struct {
*flag.FlagSet
}
// String returns the string representation of the
// flag given by name. It panics if the flag is not
// in the flag set.
func (f Flags) String(name string) string {
return f.FlagSet.Lookup(name).Value.String()
}
// Bool returns the boolean representation of the
// flag given by name. It returns false if the flag
// is not a boolean type. It panics if the flag is
// not in the flag set.
func (f Flags) Bool(name string) bool {
val, _ := strconv.ParseBool(f.String(name))
return val
}
// Int returns the integer representation of the
// flag given by name. It returns 0 if the flag
// is not an integer type. It panics if the flag is
// not in the flag set.
func (f Flags) Int(name string) int {
val, _ := strconv.ParseInt(f.String(name), 0, strconv.IntSize)
return int(val)
}
// Float64 returns the float64 representation of the
// flag given by name. It returns false if the flag
// is not a float63 type. It panics if the flag is
// not in the flag set.
func (f Flags) Float64(name string) float64 {
val, _ := strconv.ParseFloat(f.String(name), 64)
return val
}
// Duration returns the duration representation of the
// flag given by name. It returns false if the flag
// is not a duration type. It panics if the flag is
// not in the flag set.
func (f Flags) Duration(name string) time.Duration {
val, _ := time.ParseDuration(f.String(name))
return val
}
// flagHelp returns the help text for fs.
func flagHelp(fs *flag.FlagSet) string {
if fs == nil {
return ""
}
// temporarily redirect output
out := fs.Output()
defer fs.SetOutput(out)
buf := new(bytes.Buffer)
fs.SetOutput(buf)
fs.PrintDefaults()
return buf.String()
}
func printEnvironment() {
fmt.Printf("caddy.HomeDir=%s\n", caddy.HomeDir())
fmt.Printf("caddy.AppDataDir=%s\n", caddy.AppDataDir())
fmt.Printf("caddy.AppConfigDir=%s\n", caddy.AppConfigDir())
fmt.Printf("caddy.ConfigAutosavePath=%s\n", caddy.ConfigAutosavePath)
fmt.Printf("runtime.GOOS=%s\n", runtime.GOOS)
fmt.Printf("runtime.GOARCH=%s\n", runtime.GOARCH)
fmt.Printf("runtime.Compiler=%s\n", runtime.Compiler)
fmt.Printf("runtime.NumCPU=%d\n", runtime.NumCPU())
fmt.Printf("runtime.GOMAXPROCS=%d\n", runtime.GOMAXPROCS(0))
fmt.Printf("runtime.Version=%s\n", runtime.Version())
cwd, err := os.Getwd()
if err != nil {
cwd = fmt.Sprintf("<error: %v>", err)
}
fmt.Printf("os.Getwd=%s\n\n", cwd)
for _, v := range os.Environ() {
fmt.Println(v)
}
}
// moveStorage moves the old default dataDir to the new default dataDir.
// TODO: This is TEMPORARY until the release candidates.
func moveStorage() {
// get the home directory (the old way)
oldHome := os.Getenv("HOME")
if oldHome == "" && runtime.GOOS == "windows" {
drive := os.Getenv("HOMEDRIVE")
path := os.Getenv("HOMEPATH")
oldHome = drive + path
if drive == "" || path == "" {
oldHome = os.Getenv("USERPROFILE")
}
}
if oldHome == "" {
oldHome = "."
}
oldDataDir := filepath.Join(oldHome, ".local", "share", "caddy")
// nothing to do if old data dir doesn't exist
_, err := os.Stat(oldDataDir)
if os.IsNotExist(err) {
return
}
// nothing to do if the new data dir is the same as the old one
newDataDir := caddy.AppDataDir()
if oldDataDir == newDataDir {
return
}
logger := caddy.Log().Named("automigrate").With(
zap.String("old_dir", oldDataDir),
zap.String("new_dir", newDataDir))
logger.Info("beginning one-time data directory migration",
zap.String("details", "https://github.com/caddyserver/caddy/issues/2955"))
// if new data directory exists, avoid auto-migration as a conservative safety measure
_, err = os.Stat(newDataDir)
if !os.IsNotExist(err) {
logger.Error("new data directory already exists; skipping auto-migration as conservative safety measure",
zap.Error(err),
zap.String("instructions", "https://github.com/caddyserver/caddy/issues/2955#issuecomment-570000333"))
return
}
// construct the new data directory's parent folder
err = os.MkdirAll(filepath.Dir(newDataDir), 0700)
if err != nil {
logger.Error("unable to make new datadirectory - follow link for instructions",
zap.String("instructions", "https://github.com/caddyserver/caddy/issues/2955#issuecomment-570000333"),
zap.Error(err))
return
}
// folder structure is same, so just try to rename (move) it;
// this fails if the new path is on a separate device
err = os.Rename(oldDataDir, newDataDir)
if err != nil {
logger.Error("new data directory already exists; skipping auto-migration as conservative safety measure - follow link for instructions",
zap.String("instructions", "https://github.com/caddyserver/caddy/issues/2955#issuecomment-570000333"),
zap.Error(err))
}
logger.Info("successfully completed one-time migration of data directory",
zap.String("details", "https://github.com/caddyserver/caddy/issues/2955"))
}
+37
View File
@@ -0,0 +1,37 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +build !windows
package caddycmd
import (
"fmt"
"os"
"path/filepath"
"syscall"
)
func gracefullyStopProcess(pid int) error {
fmt.Print("Graceful stop... ")
err := syscall.Kill(pid, syscall.SIGINT)
if err != nil {
return fmt.Errorf("kill: %v", err)
}
return nil
}
func getProcessName() string {
return filepath.Base(os.Args[0])
}
+44
View File
@@ -0,0 +1,44 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddycmd
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strconv"
)
func gracefullyStopProcess(pid int) error {
fmt.Print("Forceful stop... ")
// process on windows will not stop unless forced with /f
cmd := exec.Command("taskkill", "/pid", strconv.Itoa(pid), "/f")
if err := cmd.Run(); err != nil {
return fmt.Errorf("taskkill: %v", err)
}
return nil
}
// On Windows the app name passed in os.Args[0] will match how
// caddy was started eg will match caddy or caddy.exe.
// So return appname with .exe for consistency
func getProcessName() string {
base := filepath.Base(os.Args[0])
if filepath.Ext(base) == "" {
return base + ".exe"
}
return base
}
-115
View File
@@ -1,115 +0,0 @@
package config
import (
"io"
"log"
"github.com/mholt/caddy/config/parse"
"github.com/mholt/caddy/config/setup"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/server"
)
const (
DefaultHost = "0.0.0.0"
DefaultPort = "2015"
DefaultRoot = "."
// The default configuration file to load if none is specified
DefaultConfigFile = "Caddyfile"
)
func Load(filename string, input io.Reader) ([]server.Config, error) {
var configs []server.Config
// turn off timestamp for parsing
flags := log.Flags()
log.SetFlags(0)
serverBlocks, err := parse.ServerBlocks(filename, input)
if err != nil {
return configs, err
}
// Each server block represents a single server/address.
// Iterate each server block and make a config for each one,
// executing the directives that were parsed.
for _, sb := range serverBlocks {
config := server.Config{
Host: sb.Host,
Port: sb.Port,
Root: Root,
Middleware: make(map[string][]middleware.Middleware),
ConfigFile: filename,
AppName: AppName,
AppVersion: AppVersion,
}
// It is crucial that directives are executed in the proper order.
for _, dir := range directiveOrder {
// Execute directive if it is in the server block
if tokens, ok := sb.Tokens[dir.name]; ok {
// Each setup function gets a controller, which is the
// server config and the dispenser containing only
// this directive's tokens.
controller := &setup.Controller{
Config: &config,
Dispenser: parse.NewDispenserTokens(filename, tokens),
}
midware, err := dir.setup(controller)
if err != nil {
return configs, err
}
if midware != nil {
// TODO: For now, we only support the default path scope /
config.Middleware["/"] = append(config.Middleware["/"], midware)
}
}
}
if config.Port == "" {
config.Port = Port
}
configs = append(configs, config)
}
// restore logging settings
log.SetFlags(flags)
return configs, nil
}
// validDirective returns true if d is a valid
// directive; false otherwise.
func validDirective(d string) bool {
for _, dir := range directiveOrder {
if dir.name == d {
return true
}
}
return false
}
// Default makes a default configuration which
// is empty except for root, host, and port,
// which are essentials for serving the cwd.
func Default() server.Config {
return server.Config{
Root: Root,
Host: Host,
Port: Port,
}
}
// These three defaults are configurable through the command line
var (
Root = DefaultRoot
Host = DefaultHost
Port = DefaultPort
)
// The application should set these so that various middlewares
// can access the proper information for their own needs.
var AppName, AppVersion string
-79
View File
@@ -1,79 +0,0 @@
package config
import (
"github.com/mholt/caddy/config/parse"
"github.com/mholt/caddy/config/setup"
"github.com/mholt/caddy/middleware"
)
func init() {
// The parse package must know which directives
// are valid, but it must not import the setup
// or config package. To solve this problem, we
// fill up this map in our init function here.
// The parse package does not need to know the
// ordering of the directives.
for _, dir := range directiveOrder {
parse.ValidDirectives[dir.name] = struct{}{}
}
}
// Directives are registered in the order they should be
// executed. Middleware (directives that inject a handler)
// are executed in the order A-B-C-*-C-B-A, assuming
// they all call the Next handler in the chain.
//
// Ordering is VERY important. Every middleware will
// feel the effects of all other middleware below
// (after) them during a request, but they must not
// care what middleware above them are doing.
//
// For example, log needs to know the status code and
// exactly how many bytes were written to the client,
// which every other middleware can affect, so it gets
// registered first. The errors middleware does not
// care if gzip or log modifies its response, so it
// gets registered below them. Gzip, on the other hand,
// DOES care what errors does to the response since it
// must compress every output to the client, even error
// pages, so it must be registered before the errors
// middleware and any others that would write to the
// response.
var directiveOrder = []directive{
// Essential directives that initialize vital configuration settings
{"root", setup.Root},
{"tls", setup.TLS},
{"bind", setup.BindHost},
// Other directives that don't create HTTP handlers
{"startup", setup.Startup},
{"shutdown", setup.Shutdown},
{"git", setup.Git},
// Directives that inject handlers (middleware)
{"log", setup.Log},
{"gzip", setup.Gzip},
{"errors", setup.Errors},
{"header", setup.Headers},
{"rewrite", setup.Rewrite},
{"redir", setup.Redir},
{"ext", setup.Ext},
{"basicauth", setup.BasicAuth},
{"proxy", setup.Proxy},
{"fastcgi", setup.FastCGI},
{"websocket", setup.WebSocket},
{"markdown", setup.Markdown},
{"templates", setup.Templates},
{"browse", setup.Browse},
}
// directive ties together a directive name with its setup function.
type directive struct {
name string
setup setupFunc
}
// A setup function takes a setup controller. Its return values may
// both be nil. If middleware is not nil, it will be chained into
// the HTTP handlers in the order specified in this package.
type setupFunc func(c *setup.Controller) (middleware.Middleware, error)
-213
View File
@@ -1,213 +0,0 @@
package parse
import (
"errors"
"fmt"
"io"
"strings"
)
// Dispenser is a type that dispenses tokens, similarly to a lexer,
// except that it can do so with some notion of structure and has
// some really convenient methods.
type Dispenser struct {
filename string
tokens []token
cursor int
nesting int
}
// NewDispenser returns a Dispenser, ready to use for parsing the given input.
func NewDispenser(filename string, input io.Reader) Dispenser {
return Dispenser{
filename: filename,
tokens: allTokens(input),
cursor: -1,
}
}
// NewDispenserTokens returns a Dispenser filled with the given tokens.
func NewDispenserTokens(filename string, tokens []token) Dispenser {
return Dispenser{
filename: filename,
tokens: tokens,
cursor: -1,
}
}
// Next loads the next token. Returns true if a token
// was loaded; false otherwise. If false, all tokens
// have already been consumed.
func (d *Dispenser) Next() bool {
if d.cursor < len(d.tokens)-1 {
d.cursor++
return true
}
return false
}
// NextArg loads the next token if it is on the same
// line. Returns true if a token was loaded; false
// otherwise. If false, all tokens on the line have
// been consumed.
func (d *Dispenser) NextArg() bool {
if d.cursor < 0 {
d.cursor++
return true
}
if d.cursor >= len(d.tokens) {
return false
}
if d.cursor < len(d.tokens)-1 &&
(d.tokens[d.cursor].line+d.numLineBreaks(d.cursor) == d.tokens[d.cursor+1].line) {
d.cursor++
return true
}
return false
}
// NextLine loads the next token only if it is not on the same
// line as the current token, and returns true if a token was
// loaded; false otherwise. If false, there is not another token
// or it is on the same line.
func (d *Dispenser) NextLine() bool {
if d.cursor < 0 {
d.cursor++
return true
}
if d.cursor >= len(d.tokens) {
return false
}
if d.cursor < len(d.tokens)-1 &&
d.tokens[d.cursor].line+d.numLineBreaks(d.cursor) < d.tokens[d.cursor+1].line {
d.cursor++
return true
}
return false
}
// NextBlock can be used as the condition of a for loop
// to load the next token as long as it opens a block or
// is already in a block. It returns true if a token was
// loaded, or false when the block's closing curly brace
// was loaded and thus the block ended. Nested blocks are
// not supported.
func (d *Dispenser) NextBlock() bool {
if d.nesting > 0 {
d.Next()
if d.Val() == "}" {
d.nesting--
return false
}
return true
}
if !d.NextArg() { // block must open on same line
return false
}
if d.Val() != "{" {
d.cursor-- // roll back if not opening brace
return false
}
d.Next()
d.nesting++
return true
}
// Val gets the text of the current token. If there is no token
// loaded, it returns empty string.
func (d *Dispenser) Val() string {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return ""
}
return d.tokens[d.cursor].text
}
// Line gets the line number of the current token. If there is no token
// loaded, it returns 0.
func (d *Dispenser) Line() int {
if d.cursor < 0 || d.cursor >= len(d.tokens) {
return 0
}
return d.tokens[d.cursor].line
}
// Args is a convenience function that loads the next arguments
// (tokens on the same line) into an arbitrary number of strings
// pointed to in targets. If there are fewer tokens available
// than string pointers, the remaining strings will not be changed
// and false will be returned. If there were enough tokens available
// to fill the arguments, then true will be returned.
func (d *Dispenser) Args(targets ...*string) bool {
enough := true
for i := 0; i < len(targets); i++ {
if !d.NextArg() {
enough = false
break
}
*targets[i] = d.Val()
}
return enough
}
// RemainingArgs loads any more arguments (tokens on the same line)
// into a slice and returns them. Open curly brace tokens also indicate
// the end of arguments, and the curly brace is not included in
// the return value nor is it loaded.
func (d *Dispenser) RemainingArgs() []string {
var args []string
for d.NextArg() {
if d.Val() == "{" {
d.cursor--
break
}
args = append(args, d.Val())
}
return args
}
// ArgErr returns an argument error, meaning that another
// argument was expected but not found. In other words,
// a line break or open curly brace was encountered instead of
// an argument.
func (d *Dispenser) ArgErr() error {
if d.Val() == "{" {
return d.Err("Unexpected token '{', expecting argument")
}
return d.Errf("Wrong argument count or unexpected line ending after '%s'", d.Val())
}
// SyntaxErr creates a generic syntax error which explains what was
// found and what was expected.
func (d *Dispenser) SyntaxErr(expected string) error {
msg := fmt.Sprintf("%s:%d - Syntax error: Unexpected token '%s', expecting '%s'", d.filename, d.Line(), d.Val(), expected)
return errors.New(msg)
}
// EofErr returns an EOF error, meaning that end of input
// was found when another token was expected.
func (d *Dispenser) EofErr() error {
return d.Errf("Unexpected EOF")
}
// Err generates a custom parse error with a message of msg.
func (d *Dispenser) Err(msg string) error {
msg = fmt.Sprintf("%s:%d - Parse error: %s", d.filename, d.Line(), msg)
return errors.New(msg)
}
// Errf is like Err, but for formatted error messages
func (d *Dispenser) Errf(format string, args ...interface{}) error {
return d.Err(fmt.Sprintf(format, args...))
}
// numLineBreaks counts how many line breaks are in the token
// value given by the token index tknIdx. It returns 0 if the
// token does not exist or there are no line breaks.
func (d *Dispenser) numLineBreaks(tknIdx int) int {
if tknIdx < 0 || tknIdx >= len(d.tokens) {
return 0
}
return strings.Count(d.tokens[tknIdx].text, "\n")
}
-114
View File
@@ -1,114 +0,0 @@
package parse
import (
"bufio"
"io"
"unicode"
)
type (
// lexer is a utility which can get values, token by
// token, from a Reader. A token is a word, and tokens
// are separated by whitespace. A word can be enclosed
// in quotes if it contains whitespace.
lexer struct {
reader *bufio.Reader
token token
line int
}
// token represents a single parsable unit.
token struct {
line int
text string
}
)
// load prepares the lexer to scan an input for tokens.
func (l *lexer) load(input io.Reader) error {
l.reader = bufio.NewReader(input)
l.line = 1
return nil
}
// next loads the next token into the lexer.
// A token is delimited by whitespace, unless
// the token starts with a quotes character (")
// in which case the token goes until the closing
// quotes (the enclosing quotes are not included).
// The rest of the line is skipped if a "#"
// character is read in. Returns true if a token
// was loaded; false otherwise.
func (l *lexer) next() bool {
var val []rune
var comment, quoted, escaped bool
makeToken := func() bool {
l.token.text = string(val)
return true
}
for {
ch, _, err := l.reader.ReadRune()
if err != nil {
if len(val) > 0 {
return makeToken()
}
if err == io.EOF {
return false
} else {
panic(err)
}
}
if quoted {
if !escaped {
if ch == '\\' {
escaped = true
continue
} else if ch == '"' {
quoted = false
return makeToken()
}
}
if ch == '\n' {
l.line++
}
val = append(val, ch)
escaped = false
continue
}
if unicode.IsSpace(ch) {
if ch == '\r' {
continue
}
if ch == '\n' {
l.line++
comment = false
}
if len(val) > 0 {
return makeToken()
}
continue
}
if ch == '#' {
comment = true
}
if comment {
continue
}
if len(val) == 0 {
l.token = token{line: l.line}
if ch == '"' {
quoted = true
continue
}
}
val = append(val, ch)
}
}
-139
View File
@@ -1,139 +0,0 @@
package parse
import (
"strings"
"testing"
)
type lexerTestCase struct {
input string
expected []token
}
func TestLexer(t *testing.T) {
testCases := []lexerTestCase{
{
input: `host:123`,
expected: []token{
{line: 1, text: "host:123"},
},
},
{
input: `host:123
directive`,
expected: []token{
{line: 1, text: "host:123"},
{line: 3, text: "directive"},
},
},
{
input: `host:123 {
directive
}`,
expected: []token{
{line: 1, text: "host:123"},
{line: 1, text: "{"},
{line: 2, text: "directive"},
{line: 3, text: "}"},
},
},
{
input: `host:123 { directive }`,
expected: []token{
{line: 1, text: "host:123"},
{line: 1, text: "{"},
{line: 1, text: "directive"},
{line: 1, text: "}"},
},
},
{
input: `host:123 {
#comment
directive
# comment
foobar # another comment
}`,
expected: []token{
{line: 1, text: "host:123"},
{line: 1, text: "{"},
{line: 3, text: "directive"},
{line: 5, text: "foobar"},
{line: 6, text: "}"},
},
},
{
input: `a "quoted value" b
foobar`,
expected: []token{
{line: 1, text: "a"},
{line: 1, text: "quoted value"},
{line: 1, text: "b"},
{line: 2, text: "foobar"},
},
},
{
input: `A "quoted \"value\" inside" B`,
expected: []token{
{line: 1, text: "A"},
{line: 1, text: `quoted "value" inside`},
{line: 1, text: "B"},
},
},
{
input: `A "quoted value with line
break inside" {
foobar
}`,
expected: []token{
{line: 1, text: "A"},
{line: 1, text: "quoted value with line\n\t\t\t\t\tbreak inside"},
{line: 2, text: "{"},
{line: 3, text: "foobar"},
{line: 4, text: "}"},
},
},
{
input: "skip those\r\nCR characters",
expected: []token{
{line: 1, text: "skip"},
{line: 1, text: "those"},
{line: 2, text: "CR"},
{line: 2, text: "characters"},
},
},
}
for i, testCase := range testCases {
actual := tokenize(testCase.input)
lexerCompare(t, i, testCase.expected, actual)
}
}
func tokenize(input string) (tokens []token) {
l := lexer{}
l.load(strings.NewReader(input))
for l.next() {
tokens = append(tokens, l.token)
}
return
}
func lexerCompare(t *testing.T, n int, expected, actual []token) {
if len(expected) != len(actual) {
t.Errorf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
}
for i := 0; i < len(actual) && i < len(expected); i++ {
if actual[i].line != expected[i].line {
t.Errorf("Test case %d token %d ('%s'): expected line %d but was line %d",
n, i, expected[i].text, expected[i].line, actual[i].line)
break
}
if actual[i].text != expected[i].text {
t.Errorf("Test case %d token %d: expected text '%s' but was '%s'",
n, i, expected[i].text, actual[i].text)
break
}
}
}
-27
View File
@@ -1,27 +0,0 @@
// Package parse provides facilities for parsing configuration files.
package parse
import "io"
// ServerBlocks parses the input just enough to organize tokens,
// in order, by server block. No further parsing is performed.
// Server blocks are returned in the order in which they appear.
func ServerBlocks(filename string, input io.Reader) ([]serverBlock, error) {
p := parser{Dispenser: NewDispenser(filename, input)}
blocks, err := p.parseAll()
return blocks, err
}
// allTokens lexes the entire input, but does not parse it.
// It returns all the tokens from the input, unstructured
// and in order.
func allTokens(input io.Reader) (tokens []token) {
l := new(lexer)
l.load(input)
for l.next() {
tokens = append(tokens, l.token)
}
return
}
var ValidDirectives = make(map[string]struct{})
-300
View File
@@ -1,300 +0,0 @@
package parse
import (
"net"
"os"
"strings"
)
type parser struct {
Dispenser
block multiServerBlock // current server block being parsed
eof bool // if we encounter a valid EOF in a hard place
}
func (p *parser) parseAll() ([]serverBlock, error) {
var blocks []serverBlock
for p.Next() {
err := p.parseOne()
if err != nil {
return blocks, err
}
// explode the multiServerBlock into multiple serverBlocks
for _, addr := range p.block.addresses {
blocks = append(blocks, serverBlock{
Host: addr.host,
Port: addr.port,
Tokens: p.block.tokens,
})
}
}
return blocks, nil
}
func (p *parser) parseOne() error {
p.block = multiServerBlock{tokens: make(map[string][]token)}
err := p.begin()
if err != nil {
return err
}
return nil
}
func (p *parser) begin() error {
err := p.addresses()
if err != nil {
return err
}
if p.eof {
// this happens if the Caddyfile consists of only
// a line of addresses and nothing else
return nil
}
err = p.blockContents()
if err != nil {
return err
}
return nil
}
func (p *parser) addresses() error {
var expectingAnother bool
for {
tkn, startLine := p.Val(), p.Line()
// Open brace definitely indicates end of addresses
if tkn == "{" {
if expectingAnother {
return p.Errf("Expected another address but had '%s' - check for extra comma", tkn)
}
break
}
// Trailing comma indicates another address will follow, which
// may possibly be on the next line
if tkn[len(tkn)-1] == ',' {
tkn = tkn[:len(tkn)-1]
expectingAnother = true
} else {
expectingAnother = false // but we may still see another one on this line
}
// Parse and save this address
host, port, err := standardAddress(tkn)
if err != nil {
return err
}
p.block.addresses = append(p.block.addresses, address{host, port})
// Advance token and possibly break out of loop or return error
hasNext := p.Next()
if expectingAnother && !hasNext {
return p.EofErr()
}
if !expectingAnother && p.Line() > startLine {
break
}
if !hasNext {
p.eof = true
break // EOF
}
}
return nil
}
func (p *parser) blockContents() error {
errOpenCurlyBrace := p.openCurlyBrace()
if errOpenCurlyBrace != nil {
// single-server configs don't need curly braces
p.cursor--
}
err := p.directives()
if err != nil {
return err
}
// Only look for close curly brace if there was an opening
if errOpenCurlyBrace == nil {
err = p.closeCurlyBrace()
if err != nil {
return err
}
}
return nil
}
// directives parses through all the lines for directives
// and it expects the next token to be the first
// directive. It goes until EOF or closing curly brace
// which ends the server block.
func (p *parser) directives() error {
for p.Next() {
// end of server block
if p.Val() == "}" {
break
}
// special case: import directive replaces tokens during parse-time
if p.Val() == "import" {
err := p.doImport()
if err != nil {
return err
}
continue
}
// normal case: parse a directive on this line
if err := p.directive(); err != nil {
return err
}
}
return nil
}
// doImport swaps out the import directive and its argument
// (a total of 2 tokens) with the tokens in the file specified.
// When the function returns, the cursor is on the token before
// where the import directive was. In other words, call Next()
// to access the first token that was imported.
func (p *parser) doImport() error {
if !p.NextArg() {
return p.ArgErr()
}
importFile := p.Val()
if p.NextArg() {
return p.Err("Import allows only one file to import")
}
file, err := os.Open(importFile)
if err != nil {
return p.Errf("Could not import %s - %v", importFile, err)
}
defer file.Close()
importedTokens := allTokens(file)
// Splice out the import directive and its argument (2 tokens total)
// and insert the imported tokens.
tokensBefore := p.tokens[:p.cursor-1]
tokensAfter := p.tokens[p.cursor+1:]
p.tokens = append(tokensBefore, append(importedTokens, tokensAfter...)...)
p.cursor -= 2
return nil
}
// directive collects tokens until the directive's scope
// closes (either end of line or end of curly brace block).
// It expects the currently-loaded token to be a directive
// (or } that ends a server block). The collected tokens
// are loaded into the current server block for later use
// by directive setup functions.
func (p *parser) directive() error {
dir := p.Val()
line := p.Line()
nesting := 0
if _, ok := ValidDirectives[dir]; !ok {
return p.Errf("Unknown directive '%s'", dir)
}
// The directive itself is appended as a relevant token
p.block.tokens[dir] = append(p.block.tokens[dir], p.tokens[p.cursor])
for p.Next() {
if p.Val() == "{" {
nesting++
} else if p.Line()+p.numLineBreaks(p.cursor) > line && nesting == 0 {
p.cursor-- // read too far
break
} else if p.Val() == "}" && nesting > 0 {
nesting--
} else if p.Val() == "}" && nesting == 0 {
return p.Err("Unexpected '}' because no matching opening brace")
}
p.block.tokens[dir] = append(p.block.tokens[dir], p.tokens[p.cursor])
}
if nesting > 0 {
return p.EofErr()
}
return nil
}
// openCurlyBrace expects the current token to be an
// opening curly brace. This acts like an assertion
// because it returns an error if the token is not
// a opening curly brace. It does not advance the token.
func (p *parser) openCurlyBrace() error {
if p.Val() != "{" {
return p.SyntaxErr("{")
}
return nil
}
// closeCurlyBrace expects the current token to be
// a closing curly brace. This acts like an assertion
// because it returns an error if the token is not
// a closing curly brace. It does not advance the token.
func (p *parser) closeCurlyBrace() error {
if p.Val() != "}" {
return p.SyntaxErr("}")
}
return nil
}
// standardAddress turns the accepted host and port patterns
// into a format accepted by net.Dial.
func standardAddress(str string) (host, port string, err error) {
var schemePort string
if strings.HasPrefix(str, "https://") {
schemePort = "https"
str = str[8:]
} else if strings.HasPrefix(str, "http://") {
schemePort = "http"
str = str[7:]
} else if !strings.Contains(str, ":") {
str += ":" // + Port
}
host, port, err = net.SplitHostPort(str)
if err != nil && schemePort != "" {
host = str
port = schemePort // assume port from scheme
err = nil
}
return
}
type (
// serverBlock stores tokens by directive name for a
// single host:port (address)
serverBlock struct {
Host, Port string
Tokens map[string][]token // directive name to tokens (including directive)
}
// multiServerBlock is the same as serverBlock but for
// multiple addresses that share the same tokens
multiServerBlock struct {
addresses []address
tokens map[string][]token
}
address struct {
host, port string
}
)
-53
View File
@@ -1,53 +0,0 @@
package setup
import (
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/basicauth"
)
// BasicAuth configures a new BasicAuth middleware instance.
func BasicAuth(c *Controller) (middleware.Middleware, error) {
rules, err := basicAuthParse(c)
if err != nil {
return nil, err
}
basic := basicauth.BasicAuth{Rules: rules}
return func(next middleware.Handler) middleware.Handler {
basic.Next = next
return basic
}, nil
}
func basicAuthParse(c *Controller) ([]basicauth.Rule, error) {
var rules []basicauth.Rule
for c.Next() {
var rule basicauth.Rule
args := c.RemainingArgs()
switch len(args) {
case 2:
rule.Username = args[0]
rule.Password = args[1]
for c.NextBlock() {
rule.Resources = append(rule.Resources, c.Val())
if c.NextArg() {
return rules, c.Errf("Expecting only one resource per line (extra '%s')", c.Val())
}
}
case 3:
rule.Resources = append(rule.Resources, args[0])
rule.Username = args[1]
rule.Password = args[2]
default:
return rules, c.ArgErr()
}
rules = append(rules, rule)
}
return rules, nil
}
-12
View File
@@ -1,12 +0,0 @@
package setup
import "github.com/mholt/caddy/middleware"
func BindHost(c *Controller) (middleware.Middleware, error) {
for c.Next() {
if !c.Args(&c.BindHost) {
return nil, c.ArgErr()
}
}
return nil, nil
}
-222
View File
@@ -1,222 +0,0 @@
package setup
import (
"fmt"
"html/template"
"io/ioutil"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/browse"
)
// Browse configures a new Browse middleware instance.
func Browse(c *Controller) (middleware.Middleware, error) {
configs, err := browseParse(c)
if err != nil {
return nil, err
}
browse := browse.Browse{
Root: c.Root,
Configs: configs,
}
return func(next middleware.Handler) middleware.Handler {
browse.Next = next
return browse
}, nil
}
func browseParse(c *Controller) ([]browse.Config, error) {
var configs []browse.Config
appendCfg := func(bc browse.Config) error {
for _, c := range configs {
if c.PathScope == bc.PathScope {
return fmt.Errorf("Duplicate browsing config for %s", c.PathScope)
}
}
configs = append(configs, bc)
return nil
}
for c.Next() {
var bc browse.Config
// First argument is directory to allow browsing; default is site root
if c.NextArg() {
bc.PathScope = c.Val()
} else {
bc.PathScope = "/"
}
// Second argument would be the template file to use
var tplText string
if c.NextArg() {
tplBytes, err := ioutil.ReadFile(c.Val())
if err != nil {
return configs, err
}
tplText = string(tplBytes)
} else {
tplText = defaultTemplate
}
// Build the template
tpl, err := template.New("listing").Parse(tplText)
if err != nil {
return configs, err
}
bc.Template = tpl
// Save configuration
err = appendCfg(bc)
if err != nil {
return configs, err
}
}
return configs, nil
}
// The default template to use when serving up directory listings
const defaultTemplate = `<!DOCTYPE html>
<html>
<head>
<title>{{.Name}}</title>
<meta charset="utf-8">
<style>
* { padding: 0; margin: 0; }
body {
padding: 1% 2%;
font: 16px Arial;
}
header {
font-size: 45px;
padding: 25px;
}
header a {
text-decoration: none;
color: inherit;
}
header .up {
display: inline-block;
height: 50px;
width: 50px;
text-align: center;
margin-right: 20px;
}
header a.up:hover {
background: #000;
color: #FFF;
}
h1 {
font-size: 30px;
display: inline;
}
table {
border: 0;
border-collapse: collapse;
max-width: 750px;
margin: 0 auto;
}
th,
td {
padding: 4px 20px;
vertical-align: middle;
line-height: 1.5em; /* emoji are kind of odd heights */
}
th {
text-align: left;
}
@media (max-width: 700px) {
.hideable {
display: none;
}
body {
padding: 0;
}
header,
header h1 {
font-size: 16px;
}
header {
position: fixed;
top: 0;
width: 100%;
background: #333;
color: #FFF;
padding: 15px;
text-align: center;
}
header .up {
height: auto;
width: auto;
display: none;
}
header a.up {
display: inline-block;
position: absolute;
left: 0;
top: 0;
width: 40px;
height: 48px;
font-size: 35px;
}
header h1 {
font-weight: normal;
}
main {
margin-top: 70px;
}
}
</style>
</head>
<body>
<header>
{{if .CanGoUp}}
<a href=".." class="up" title="Up one level">&#11025;</a>
{{else}}
<div class="up">&nbsp;</div>
{{end}}
<h1>{{.Path}}</h1>
</header>
<main>
<table>
<tr>
<th>Name</th>
<th>Size</th>
<th class="hideable">Modified</th>
</tr>
{{range .Items}}
<tr>
<td>
{{if .IsDir}}&#128194;{{else}}&#128196;{{end}}
<a href="{{.URL}}">{{.Name}}</a>
</td>
<td>{{.HumanSize}}</td>
<td class="hideable">{{.HumanModTime "01/02/2006 3:04:05 PM -0700"}}</td>
</tr>
{{end}}
</table>
</main>
</body>
</html>`
-11
View File
@@ -1,11 +0,0 @@
package setup
import (
"github.com/mholt/caddy/config/parse"
"github.com/mholt/caddy/server"
)
type Controller struct {
*server.Config
parse.Dispenser
}
-17
View File
@@ -1,17 +0,0 @@
package setup
import (
"strings"
"github.com/mholt/caddy/config/parse"
"github.com/mholt/caddy/server"
)
// newTestController creates a new *Controller for
// the input specified, with a filename of "Testfile"
func newTestController(input string) *Controller {
return &Controller{
Config: &server.Config{},
Dispenser: parse.NewDispenser("Testfile", strings.NewReader(input)),
}
}
-103
View File
@@ -1,103 +0,0 @@
package setup
import (
"log"
"os"
"path"
"strconv"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/errors"
)
// Errors configures a new gzip middleware instance.
func Errors(c *Controller) (middleware.Middleware, error) {
handler, err := errorsParse(c)
if err != nil {
return nil, err
}
// Open the log file for writing when the server starts
c.Startup = append(c.Startup, func() error {
var err error
var file *os.File
if handler.LogFile == "stdout" {
file = os.Stdout
} else if handler.LogFile == "stderr" {
file = os.Stderr
} else if handler.LogFile != "" {
file, err = os.OpenFile(handler.LogFile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
if err != nil {
return err
}
}
handler.Log = log.New(file, "", 0)
return nil
})
return func(next middleware.Handler) middleware.Handler {
handler.Next = next
return handler
}, nil
}
func errorsParse(c *Controller) (*errors.ErrorHandler, error) {
// Very important that we make a pointer because the Startup
// function that opens the log file must have access to the
// same instance of the handler, not a copy.
handler := &errors.ErrorHandler{ErrorPages: make(map[int]string)}
optionalBlock := func() (bool, error) {
var hadBlock bool
for c.NextBlock() {
hadBlock = true
what := c.Val()
if !c.NextArg() {
return hadBlock, c.ArgErr()
}
where := c.Val()
if what == "log" {
handler.LogFile = where
} else {
// Error page; ensure it exists
where = path.Join(c.Root, where)
f, err := os.Open(where)
if err != nil {
return hadBlock, c.Err("Unable to open error page '" + where + "': " + err.Error())
}
f.Close()
whatInt, err := strconv.Atoi(what)
if err != nil {
return hadBlock, c.Err("Expecting a numeric status code, got '" + what + "'")
}
handler.ErrorPages[whatInt] = where
}
}
return hadBlock, nil
}
for c.Next() {
// Configuration may be in a block
hadBlock, err := optionalBlock()
if err != nil {
return handler, err
}
// Otherwise, the only argument would be an error log file name
if !hadBlock {
if c.NextArg() {
handler.LogFile = c.Val()
} else {
handler.LogFile = errors.DefaultLogFilename
}
}
}
return handler, nil
}
-54
View File
@@ -1,54 +0,0 @@
package setup
import (
"os"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/extensions"
)
// Ext configures a new instance of 'extensions' middleware for clean URLs.
func Ext(c *Controller) (middleware.Middleware, error) {
root := c.Root
exts, err := extParse(c)
if err != nil {
return nil, err
}
return func(next middleware.Handler) middleware.Handler {
return extensions.Ext{
Next: next,
Extensions: exts,
Root: root,
}
}, nil
}
// extParse sets up an instance of extension middleware
// from a middleware controller and returns a list of extensions.
func extParse(c *Controller) ([]string, error) {
var exts []string
for c.Next() {
// At least one extension is required
if !c.NextArg() {
return exts, c.ArgErr()
}
exts = append(exts, c.Val())
// Tack on any other extensions that may have been listed
exts = append(exts, c.RemainingArgs()...)
}
return exts, nil
}
// resourceExists returns true if the file specified at
// root + path exists; false otherwise.
func resourceExists(root, path string) bool {
_, err := os.Stat(root + path)
// technically we should use os.IsNotExist(err)
// but we don't handle any other kinds of errors anyway
return err == nil
}
-110
View File
@@ -1,110 +0,0 @@
package setup
import (
"errors"
"net/http"
"path/filepath"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/fastcgi"
)
// FastCGI configures a new FastCGI middleware instance.
func FastCGI(c *Controller) (middleware.Middleware, error) {
absRoot, err := filepath.Abs(c.Root)
if err != nil {
return nil, err
}
rules, err := fastcgiParse(c)
if err != nil {
return nil, err
}
return func(next middleware.Handler) middleware.Handler {
return fastcgi.Handler{
Next: next,
Rules: rules,
Root: c.Root,
AbsRoot: absRoot,
FileSys: http.Dir(c.Root),
SoftwareName: c.AppName,
SoftwareVersion: c.AppVersion,
ServerName: c.Host,
ServerPort: c.Port,
}
}, nil
}
func fastcgiParse(c *Controller) ([]fastcgi.Rule, error) {
var rules []fastcgi.Rule
for c.Next() {
var rule fastcgi.Rule
args := c.RemainingArgs()
switch len(args) {
case 0:
return rules, c.ArgErr()
case 1:
rule.Path = "/"
rule.Address = args[0]
case 2:
rule.Path = args[0]
rule.Address = args[1]
case 3:
rule.Path = args[0]
rule.Address = args[1]
err := fastcgiPreset(args[2], &rule)
if err != nil {
return rules, c.Err("Invalid fastcgi rule preset '" + args[2] + "'")
}
}
for c.NextBlock() {
switch c.Val() {
case "ext":
if !c.NextArg() {
return rules, c.ArgErr()
}
rule.Ext = c.Val()
case "split":
if !c.NextArg() {
return rules, c.ArgErr()
}
rule.SplitPath = c.Val()
case "index":
args := c.RemainingArgs()
if len(args) == 0 {
return rules, c.ArgErr()
}
rule.IndexFiles = args
case "env":
envArgs := c.RemainingArgs()
if len(envArgs) < 2 {
return rules, c.ArgErr()
}
rule.EnvVars = append(rule.EnvVars, [2]string{envArgs[0], envArgs[1]})
}
}
rules = append(rules, rule)
}
return rules, nil
}
// fastcgiPreset configures rule according to name. It returns an error if
// name is not a recognized preset name.
func fastcgiPreset(name string, rule *fastcgi.Rule) error {
switch name {
case "php":
rule.Ext = ".php"
rule.SplitPath = ".php"
rule.IndexFiles = []string{"index.php"}
default:
return errors.New(name + " is not a valid preset name")
}
return nil
}
-171
View File
@@ -1,171 +0,0 @@
package setup
import (
"fmt"
"log"
"net/url"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/git"
)
// Git configures a new Git service routine.
func Git(c *Controller) (middleware.Middleware, error) {
repo, err := gitParse(c)
if err != nil {
return nil, err
}
c.Startup = append(c.Startup, func() error {
// Startup functions are blocking; start
// service routine in background
go func() {
for {
time.Sleep(repo.Interval)
err := repo.Pull()
if err != nil {
if git.Logger == nil {
log.Println(err)
} else {
git.Logger.Println(err)
}
}
}
}()
// Do a pull right away to return error
return repo.Pull()
})
return nil, err
}
func gitParse(c *Controller) (*git.Repo, error) {
repo := &git.Repo{Branch: "master", Interval: git.DefaultInterval, Path: c.Root}
for c.Next() {
args := c.RemainingArgs()
switch len(args) {
case 2:
repo.Path = filepath.Clean(c.Root + string(filepath.Separator) + args[1])
fallthrough
case 1:
repo.Url = args[0]
}
for c.NextBlock() {
switch c.Val() {
case "repo":
if !c.NextArg() {
return nil, c.ArgErr()
}
repo.Url = c.Val()
case "path":
if !c.NextArg() {
return nil, c.ArgErr()
}
repo.Path = filepath.Clean(c.Root + string(filepath.Separator) + c.Val())
case "branch":
if !c.NextArg() {
return nil, c.ArgErr()
}
repo.Branch = c.Val()
case "key":
if !c.NextArg() {
return nil, c.ArgErr()
}
repo.KeyPath = c.Val()
case "interval":
if !c.NextArg() {
return nil, c.ArgErr()
}
t, _ := strconv.Atoi(c.Val())
if t > 0 {
repo.Interval = time.Duration(t) * time.Second
}
case "then":
thenArgs := c.RemainingArgs()
if len(thenArgs) == 0 {
return nil, c.ArgErr()
}
repo.Then = strings.Join(thenArgs, " ")
}
}
}
// if repo is not specified, return error
if repo.Url == "" {
return nil, c.ArgErr()
}
// if private key is not specified, convert repository url to https
// to avoid ssh authentication
// else validate git url
// Note: private key support not yet available on Windows
var err error
if repo.KeyPath == "" {
repo.Url, repo.Host, err = sanitizeHttp(repo.Url)
} else {
repo.Url, repo.Host, err = sanitizeGit(repo.Url)
// TODO add Windows support for private repos
if runtime.GOOS == "windows" {
return nil, fmt.Errorf("Private repository not yet supported on Windows")
}
}
if err != nil {
return nil, err
}
// validate git availability in PATH
if err = git.InitGit(); err != nil {
return nil, err
}
return repo, repo.Prepare()
}
// sanitizeHttp cleans up repository url and converts to https format
// if currently in ssh format.
// Returns sanitized url, hostName (e.g. github.com, bitbucket.com)
// and possible error
func sanitizeHttp(repoUrl string) (string, string, error) {
url, err := url.Parse(repoUrl)
if err != nil {
return "", "", err
}
if url.Host == "" && strings.HasPrefix(url.Path, "git@") {
url.Path = url.Path[len("git@"):]
i := strings.Index(url.Path, ":")
if i < 0 {
return "", "", fmt.Errorf("Invalid git url %s", repoUrl)
}
url.Host = url.Path[:i]
url.Path = "/" + url.Path[i+1:]
}
repoUrl = "https://" + url.Host + url.Path
return repoUrl, url.Host, nil
}
// sanitizeGit cleans up repository url and validate ssh format.
// Returns sanitized url, hostName (e.g. github.com, bitbucket.com)
// and possible error
func sanitizeGit(repoUrl string) (string, string, error) {
repoUrl = strings.TrimSpace(repoUrl)
if !strings.HasPrefix(repoUrl, "git@") || strings.Index(repoUrl, ":") < len("git@a:") {
return "", "", fmt.Errorf("Invalid git url %s", repoUrl)
}
hostUrl := repoUrl[len("git@"):]
i := strings.Index(hostUrl, ":")
host := hostUrl[:i]
return repoUrl, host, nil
}
-13
View File
@@ -1,13 +0,0 @@
package setup
import (
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/gzip"
)
// Gzip configures a new gzip middleware instance.
func Gzip(c *Controller) (middleware.Middleware, error) {
return func(next middleware.Handler) middleware.Handler {
return gzip.Gzip{Next: next}
}, nil
}
-84
View File
@@ -1,84 +0,0 @@
package setup
import (
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/headers"
)
// Headers configures a new Headers middleware instance.
func Headers(c *Controller) (middleware.Middleware, error) {
rules, err := headersParse(c)
if err != nil {
return nil, err
}
return func(next middleware.Handler) middleware.Handler {
return headers.Headers{Next: next, Rules: rules}
}, nil
}
func headersParse(c *Controller) ([]headers.Rule, error) {
var rules []headers.Rule
for c.NextLine() {
var head headers.Rule
var isNewPattern bool
if !c.NextArg() {
return rules, c.ArgErr()
}
pattern := c.Val()
// See if we already have a definition for this URL pattern...
for _, h := range rules {
if h.Url == pattern {
head = h
break
}
}
// ...otherwise, this is a new pattern
if head.Url == "" {
head.Url = pattern
isNewPattern = true
}
for c.NextBlock() {
// A block of headers was opened...
h := headers.Header{Name: c.Val()}
if c.NextArg() {
h.Value = c.Val()
}
head.Headers = append(head.Headers, h)
}
if c.NextArg() {
// ... or single header was defined as an argument instead.
h := headers.Header{Name: c.Val()}
h.Value = c.Val()
if c.NextArg() {
h.Value = c.Val()
}
head.Headers = append(head.Headers, h)
}
if isNewPattern {
rules = append(rules, head)
} else {
for i := 0; i < len(rules); i++ {
if rules[i].Url == pattern {
rules[i] = head
break
}
}
}
}
return rules, nil
}
-90
View File
@@ -1,90 +0,0 @@
package setup
import (
"log"
"os"
"github.com/mholt/caddy/middleware"
caddylog "github.com/mholt/caddy/middleware/log"
)
func Log(c *Controller) (middleware.Middleware, error) {
rules, err := logParse(c)
if err != nil {
return nil, err
}
// Open the log files for writing when the server starts
c.Startup = append(c.Startup, func() error {
for i := 0; i < len(rules); i++ {
var err error
var file *os.File
if rules[i].OutputFile == "stdout" {
file = os.Stdout
} else if rules[i].OutputFile == "stderr" {
file = os.Stderr
} else {
file, err = os.OpenFile(rules[i].OutputFile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
if err != nil {
return err
}
}
rules[i].Log = log.New(file, "", 0)
}
return nil
})
return func(next middleware.Handler) middleware.Handler {
return caddylog.Logger{Next: next, Rules: rules}
}, nil
}
func logParse(c *Controller) ([]caddylog.LogRule, error) {
var rules []caddylog.LogRule
for c.Next() {
args := c.RemainingArgs()
if len(args) == 0 {
// Nothing specified; use defaults
rules = append(rules, caddylog.LogRule{
PathScope: "/",
OutputFile: caddylog.DefaultLogFilename,
Format: caddylog.DefaultLogFormat,
})
} else if len(args) == 1 {
// Only an output file specified
rules = append(rules, caddylog.LogRule{
PathScope: "/",
OutputFile: args[0],
Format: caddylog.DefaultLogFormat,
})
} else {
// Path scope, output file, and maybe a format specified
format := caddylog.DefaultLogFormat
if len(args) > 2 {
switch args[2] {
case "{common}":
format = caddylog.CommonLogFormat
case "{combined}":
format = caddylog.CombinedLogFormat
default:
format = args[2]
}
}
rules = append(rules, caddylog.LogRule{
PathScope: args[0],
OutputFile: args[1],
Format: format,
})
}
}
return rules, nil
}
-78
View File
@@ -1,78 +0,0 @@
package setup
import (
"net/http"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/markdown"
"github.com/russross/blackfriday"
)
// Markdown configures a new Markdown middleware instance.
func Markdown(c *Controller) (middleware.Middleware, error) {
mdconfigs, err := markdownParse(c)
if err != nil {
return nil, err
}
md := markdown.Markdown{
Root: c.Root,
FileSys: http.Dir(c.Root),
Configs: mdconfigs,
IndexFiles: []string{"index.md"},
}
return func(next middleware.Handler) middleware.Handler {
md.Next = next
return md
}, nil
}
func markdownParse(c *Controller) ([]markdown.Config, error) {
var mdconfigs []markdown.Config
for c.Next() {
md := markdown.Config{
Renderer: blackfriday.HtmlRenderer(0, "", ""),
}
// Get the path scope
if !c.NextArg() || c.Val() == "{" {
return mdconfigs, c.ArgErr()
}
md.PathScope = c.Val()
// Load any other configuration parameters
for c.NextBlock() {
switch c.Val() {
case "ext":
exts := c.RemainingArgs()
if len(exts) == 0 {
return mdconfigs, c.ArgErr()
}
md.Extensions = append(md.Extensions, exts...)
case "css":
if !c.NextArg() {
return mdconfigs, c.ArgErr()
}
md.Styles = append(md.Styles, c.Val())
case "js":
if !c.NextArg() {
return mdconfigs, c.ArgErr()
}
md.Scripts = append(md.Scripts, c.Val())
default:
return mdconfigs, c.Err("Expected valid markdown configuration property")
}
}
// If no extensions were specified, assume .md
if len(md.Extensions) == 0 {
md.Extensions = []string{".md"}
}
mdconfigs = append(mdconfigs, md)
}
return mdconfigs, nil
}
-17
View File
@@ -1,17 +0,0 @@
package setup
import (
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/proxy"
)
// Proxy configures a new Proxy middleware instance.
func Proxy(c *Controller) (middleware.Middleware, error) {
if upstreams, err := proxy.NewStaticUpstreams(c.Dispenser); err == nil {
return func(next middleware.Handler) middleware.Handler {
return proxy.Proxy{Next: next, Upstreams: upstreams}
}, nil
} else {
return nil, err
}
}
-77
View File
@@ -1,77 +0,0 @@
package setup
import (
"net/http"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/redirect"
)
// Redir configures a new Redirect middleware instance.
func Redir(c *Controller) (middleware.Middleware, error) {
rules, err := redirParse(c)
if err != nil {
return nil, err
}
return func(next middleware.Handler) middleware.Handler {
return redirect.Redirect{Next: next, Rules: rules}
}, nil
}
func redirParse(c *Controller) ([]redirect.Rule, error) {
var redirects []redirect.Rule
for c.Next() {
var rule redirect.Rule
args := c.RemainingArgs()
switch len(args) {
case 1:
// To specified
rule.From = "/"
rule.To = args[0]
rule.Code = http.StatusMovedPermanently
case 2:
// To and Code specified
rule.From = "/"
rule.To = args[0]
if code, ok := httpRedirs[args[1]]; !ok {
return redirects, c.Err("Invalid redirect code '" + args[1] + "'")
} else {
rule.Code = code
}
case 3:
// From, To, and Code specified
rule.From = args[0]
rule.To = args[1]
if code, ok := httpRedirs[args[2]]; !ok {
return redirects, c.Err("Invalid redirect code '" + args[2] + "'")
} else {
rule.Code = code
}
default:
return redirects, c.ArgErr()
}
if rule.From == rule.To {
return redirects, c.Err("Redirect rule cannot allow From and To arguments to be the same.")
}
redirects = append(redirects, rule)
}
return redirects, nil
}
// httpRedirs is a list of supported HTTP redirect codes.
var httpRedirs = map[string]int{
"300": 300,
"301": 301,
"302": 302,
"303": 303,
"304": 304,
"305": 305,
"307": 307,
"308": 308,
}
-40
View File
@@ -1,40 +0,0 @@
package setup
import (
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/rewrite"
)
// Rewrite configures a new Rewrite middleware instance.
func Rewrite(c *Controller) (middleware.Middleware, error) {
rewrites, err := rewriteParse(c)
if err != nil {
return nil, err
}
return func(next middleware.Handler) middleware.Handler {
return rewrite.Rewrite{Next: next, Rules: rewrites}
}, nil
}
func rewriteParse(c *Controller) ([]rewrite.Rule, error) {
var rewrites []rewrite.Rule
for c.Next() {
var rule rewrite.Rule
if !c.NextArg() {
return rewrites, c.ArgErr()
}
rule.From = c.Val()
if !c.NextArg() {
return rewrites, c.ArgErr()
}
rule.To = c.Val()
rewrites = append(rewrites, rule)
}
return rewrites, nil
}
-31
View File
@@ -1,31 +0,0 @@
package setup
import (
"log"
"os"
"github.com/mholt/caddy/middleware"
)
func Root(c *Controller) (middleware.Middleware, error) {
for c.Next() {
if !c.NextArg() {
return nil, c.ArgErr()
}
c.Root = c.Val()
}
// Check if root path exists
_, err := os.Stat(c.Root)
if err != nil {
if os.IsNotExist(err) {
// Allow this, because the folder might appear later.
// But make sure the user knows!
log.Printf("Warning: Root path does not exist: %s", c.Root)
} else {
return nil, c.Errf("Unable to access root path '%s': %v", c.Root, err)
}
}
return nil, nil
}
-58
View File
@@ -1,58 +0,0 @@
package setup
import (
"os"
"os/exec"
"strings"
"github.com/mholt/caddy/middleware"
)
func Startup(c *Controller) (middleware.Middleware, error) {
return nil, registerCallback(c, &c.Startup)
}
func Shutdown(c *Controller) (middleware.Middleware, error) {
return nil, registerCallback(c, &c.Shutdown)
}
// registerCallback registers a callback function to execute by
// using c to parse the line. It appends the callback function
// to the list of callback functions passed in by reference.
func registerCallback(c *Controller, list *[]func() error) error {
for c.Next() {
args := c.RemainingArgs()
if len(args) == 0 {
return c.ArgErr()
}
nonblock := false
if len(args) > 1 && args[len(args)-1] == "&" {
// Run command in background; non-blocking
nonblock = true
args = args[:len(args)-1]
}
command, args, err := middleware.SplitCommandAndArgs(strings.Join(args, " "))
if err != nil {
return c.Err(err.Error())
}
fn := func() error {
cmd := exec.Command(command, args...)
cmd.Stdin = os.Stdin
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
if nonblock {
return cmd.Start()
} else {
return cmd.Run()
}
}
*list = append(*list, fn)
}
return nil
}
-61
View File
@@ -1,61 +0,0 @@
package setup
import (
"net/http"
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/templates"
)
// Templates configures a new Templates middleware instance.
func Templates(c *Controller) (middleware.Middleware, error) {
rules, err := templatesParse(c)
if err != nil {
return nil, err
}
tmpls := templates.Templates{
Rules: rules,
Root: c.Root,
FileSys: http.Dir(c.Root),
}
return func(next middleware.Handler) middleware.Handler {
tmpls.Next = next
return tmpls
}, nil
}
func templatesParse(c *Controller) ([]templates.Rule, error) {
var rules []templates.Rule
for c.Next() {
var rule templates.Rule
if c.NextArg() {
// First argument would be the path
rule.Path = c.Val()
// Any remaining arguments are extensions
rule.Extensions = c.RemainingArgs()
if len(rule.Extensions) == 0 {
rule.Extensions = defaultExtensions
}
} else {
rule.Path = defaultPath
rule.Extensions = defaultExtensions
}
for _, ext := range rule.Extensions {
rule.IndexFiles = append(rule.IndexFiles, "index"+ext)
}
rules = append(rules, rule)
}
return rules, nil
}
const defaultPath = "/"
var defaultExtensions = []string{".html", ".htm", ".tmpl", ".tpl", ".txt"}
-31
View File
@@ -1,31 +0,0 @@
package setup
import (
"github.com/mholt/caddy/middleware"
"log"
)
func TLS(c *Controller) (middleware.Middleware, error) {
c.TLS.Enabled = true
if c.Port == "http" {
c.TLS.Enabled = false
log.Printf("Warning: TLS was disabled on host http://%s."+
" Make sure you are specifying https://%s in your config (if you haven't already)."+
" If you meant to serve tls on port 80,"+
" specify port 80 in your config (http://%s:80).", c.Host, c.Host, c.Host)
}
for c.Next() {
if !c.NextArg() {
return nil, c.ArgErr()
}
c.TLS.Certificate = c.Val()
if !c.NextArg() {
return nil, c.ArgErr()
}
c.TLS.Key = c.Val()
}
return nil, nil
}
-77
View File
@@ -1,77 +0,0 @@
package setup
import (
"github.com/mholt/caddy/middleware"
"github.com/mholt/caddy/middleware/websockets"
)
// WebSocket configures a new WebSockets middleware instance.
func WebSocket(c *Controller) (middleware.Middleware, error) {
var websocks []websockets.Config
var respawn bool
optionalBlock := func() (hadBlock bool, err error) {
for c.NextBlock() {
hadBlock = true
if c.Val() == "respawn" {
respawn = true
} else {
return true, c.Err("Expected websocket configuration parameter in block")
}
}
return
}
for c.Next() {
var val, path, command string
// Path or command; not sure which yet
if !c.NextArg() {
return nil, c.ArgErr()
}
val = c.Val()
// Extra configuration may be in a block
hadBlock, err := optionalBlock()
if err != nil {
return nil, err
}
if !hadBlock {
// The next argument on this line will be the command or an open curly brace
if c.NextArg() {
path = val
command = c.Val()
} else {
path = "/"
command = val
}
// Okay, check again for optional block
hadBlock, err = optionalBlock()
if err != nil {
return nil, err
}
}
// Split command into the actual command and its arguments
cmd, args, err := middleware.SplitCommandAndArgs(command)
if err != nil {
return nil, err
}
websocks = append(websocks, websockets.Config{
Path: path,
Command: cmd,
Arguments: args,
Respawn: respawn, // TODO: This isn't used currently
})
}
websockets.GatewayInterface = c.AppName + "-CGI/1.1"
websockets.ServerSoftware = c.AppName + "/" + c.AppVersion
return func(next middleware.Handler) middleware.Handler {
return websockets.WebSockets{Next: next, Sockets: websocks}
}, nil
}
+407
View File
@@ -0,0 +1,407 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"context"
"encoding/json"
"fmt"
"log"
"reflect"
"github.com/caddyserver/certmagic"
"go.uber.org/zap"
)
// Context is a type which defines the lifetime of modules that
// are loaded and provides access to the parent configuration
// that spawned the modules which are loaded. It should be used
// with care and wrapped with derivation functions from the
// standard context package only if you don't need the Caddy
// specific features. These contexts are canceled when the
// lifetime of the modules loaded from it is over.
//
// Use NewContext() to get a valid value (but most modules will
// not actually need to do this).
type Context struct {
context.Context
moduleInstances map[string][]interface{}
cfg *Config
cleanupFuncs []func()
}
// NewContext provides a new context derived from the given
// context ctx. Normally, you will not need to call this
// function unless you are loading modules which have a
// different lifespan than the ones for the context the
// module was provisioned with. Be sure to call the cancel
// func when the context is to be cleaned up so that
// modules which are loaded will be properly unloaded.
// See standard library context package's documentation.
func NewContext(ctx Context) (Context, context.CancelFunc) {
newCtx := Context{moduleInstances: make(map[string][]interface{}), cfg: ctx.cfg}
c, cancel := context.WithCancel(ctx.Context)
wrappedCancel := func() {
cancel()
for _, f := range ctx.cleanupFuncs {
f()
}
for modName, modInstances := range newCtx.moduleInstances {
for _, inst := range modInstances {
if cu, ok := inst.(CleanerUpper); ok {
err := cu.Cleanup()
if err != nil {
log.Printf("[ERROR] %s (%p): cleanup: %v", modName, inst, err)
}
}
}
}
}
newCtx.Context = c
return newCtx, wrappedCancel
}
// OnCancel executes f when ctx is canceled.
func (ctx *Context) OnCancel(f func()) {
ctx.cleanupFuncs = append(ctx.cleanupFuncs, f)
}
// LoadModule loads the Caddy module(s) from the specified field of the parent struct
// pointer and returns the loaded module(s). The struct pointer and its field name as
// a string are necessary so that reflection can be used to read the struct tag on the
// field to get the module namespace and inline module name key (if specified).
//
// The field can be any one of the supported raw module types: json.RawMessage,
// []json.RawMessage, map[string]json.RawMessage, or []map[string]json.RawMessage.
// ModuleMap may be used in place of map[string]json.RawMessage. The return value's
// underlying type mirrors the input field's type:
//
// json.RawMessage => interface{}
// []json.RawMessage => []interface{}
// map[string]json.RawMessage => map[string]interface{}
// []map[string]json.RawMessage => []map[string]interface{}
//
// The field must have a "caddy" struct tag in this format:
//
// caddy:"key1=val1 key2=val2"
//
// To load modules, a "namespace" key is required. For example, to load modules
// in the "http.handlers" namespace, you'd put: `namespace=http.handlers` in the
// Caddy struct tag.
//
// The module name must also be available. If the field type is a map or slice of maps,
// then key is assumed to be the module name if an "inline_key" is NOT specified in the
// caddy struct tag. In this case, the module name does NOT need to be specified in-line
// with the module itself.
//
// If not a map, or if inline_key is non-empty, then the module name must be embedded
// into the values, which must be objects; then there must be a key in those objects
// where its associated value is the module name. This is called the "inline key",
// meaning the key containing the module's name that is defined inline with the module
// itself. You must specify the inline key in a struct tag, along with the namespace:
//
// caddy:"namespace=http.handlers inline_key=handler"
//
// This will look for a key/value pair like `"handler": "..."` in the json.RawMessage
// in order to know the module name.
//
// To make use of the loaded module(s) (the return value), you will probably want
// to type-assert each interface{} value(s) to the types that are useful to you
// and store them on the same struct. Storing them on the same struct makes for
// easy garbage collection when your host module is no longer needed.
//
// Loaded modules have already been provisioned and validated. Upon returning
// successfully, this method clears the json.RawMessage(s) in the field since
// the raw JSON is no longer needed, and this allows the GC to free up memory.
func (ctx Context) LoadModule(structPointer interface{}, fieldName string) (interface{}, error) {
val := reflect.ValueOf(structPointer).Elem().FieldByName(fieldName)
typ := val.Type()
field, ok := reflect.TypeOf(structPointer).Elem().FieldByName(fieldName)
if !ok {
panic(fmt.Sprintf("field %s does not exist in %#v", fieldName, structPointer))
}
opts, err := ParseStructTag(field.Tag.Get("caddy"))
if err != nil {
panic(fmt.Sprintf("malformed tag on field %s: %v", fieldName, err))
}
moduleNamespace, ok := opts["namespace"]
if !ok {
panic(fmt.Sprintf("missing 'namespace' key in struct tag on field %s", fieldName))
}
inlineModuleKey := opts["inline_key"]
var result interface{}
switch val.Kind() {
case reflect.Slice:
if isJSONRawMessage(typ) {
// val is `json.RawMessage` ([]uint8 under the hood)
if inlineModuleKey == "" {
panic("unable to determine module name without inline_key when type is not a ModuleMap")
}
val, err := ctx.loadModuleInline(inlineModuleKey, moduleNamespace, val.Interface().(json.RawMessage))
if err != nil {
return nil, err
}
result = val
} else if isJSONRawMessage(typ.Elem()) {
// val is `[]json.RawMessage`
if inlineModuleKey == "" {
panic("unable to determine module name without inline_key because type is not a ModuleMap")
}
var all []interface{}
for i := 0; i < val.Len(); i++ {
val, err := ctx.loadModuleInline(inlineModuleKey, moduleNamespace, val.Index(i).Interface().(json.RawMessage))
if err != nil {
return nil, fmt.Errorf("position %d: %v", i, err)
}
all = append(all, val)
}
result = all
} else if isModuleMapType(typ.Elem()) {
// val is `[]map[string]json.RawMessage`
var all []map[string]interface{}
for i := 0; i < val.Len(); i++ {
thisSet, err := ctx.loadModulesFromSomeMap(moduleNamespace, inlineModuleKey, val.Index(i))
if err != nil {
return nil, err
}
all = append(all, thisSet)
}
result = all
}
case reflect.Map:
// val is a ModuleMap or some other kind of map
result, err = ctx.loadModulesFromSomeMap(moduleNamespace, inlineModuleKey, val)
if err != nil {
return nil, err
}
default:
return nil, fmt.Errorf("unrecognized type for module: %s", typ)
}
// we're done with the raw bytes; allow GC to deallocate
val.Set(reflect.Zero(typ))
return result, nil
}
// loadModulesFromSomeMap loads modules from val, which must be a type of map[string]interface{}.
// Depending on inlineModuleKey, it will be interpreted as either a ModuleMap (key is the module
// name) or as a regular map (key is not the module name, and module name is defined inline).
func (ctx Context) loadModulesFromSomeMap(namespace, inlineModuleKey string, val reflect.Value) (map[string]interface{}, error) {
// if no inline_key is specified, then val must be a ModuleMap,
// where the key is the module name
if inlineModuleKey == "" {
if !isModuleMapType(val.Type()) {
panic(fmt.Sprintf("expected ModuleMap because inline_key is empty; but we do not recognize this type: %s", val.Type()))
}
return ctx.loadModuleMap(namespace, val)
}
// otherwise, val is a map with modules, but the module name is
// inline with each value (the key means something else)
return ctx.loadModulesFromRegularMap(namespace, inlineModuleKey, val)
}
// loadModulesFromRegularMap loads modules from val, where val is a map[string]json.RawMessage.
// Map keys are NOT interpreted as module names, so module names are still expected to appear
// inline with the objects.
func (ctx Context) loadModulesFromRegularMap(namespace, inlineModuleKey string, val reflect.Value) (map[string]interface{}, error) {
mods := make(map[string]interface{})
iter := val.MapRange()
for iter.Next() {
k := iter.Key()
v := iter.Value()
mod, err := ctx.loadModuleInline(inlineModuleKey, namespace, v.Interface().(json.RawMessage))
if err != nil {
return nil, fmt.Errorf("key %s: %v", k, err)
}
mods[k.String()] = mod
}
return mods, nil
}
// loadModuleMap loads modules from a ModuleMap, i.e. map[string]interface{}, where the key is the
// module name. With a module map, module names do not need to be defined inline with their values.
func (ctx Context) loadModuleMap(namespace string, val reflect.Value) (map[string]interface{}, error) {
all := make(map[string]interface{})
iter := val.MapRange()
for iter.Next() {
k := iter.Key().Interface().(string)
v := iter.Value().Interface().(json.RawMessage)
moduleName := namespace + "." + k
if namespace == "" {
moduleName = k
}
val, err := ctx.LoadModuleByID(moduleName, v)
if err != nil {
return nil, fmt.Errorf("module name '%s': %v", k, err)
}
all[k] = val
}
return all, nil
}
// LoadModuleByID decodes rawMsg into a new instance of mod and
// returns the value. If mod.New is nil, an error is returned.
// If the module implements Validator or Provisioner interfaces,
// those methods are invoked to ensure the module is fully
// configured and valid before being used.
//
// This is a lower-level method and will usually not be called
// directly by most modules. However, this method is useful when
// dynamically loading/unloading modules in their own context,
// like from embedded scripts, etc.
func (ctx Context) LoadModuleByID(id string, rawMsg json.RawMessage) (interface{}, error) {
modulesMu.RLock()
mod, ok := modules[id]
modulesMu.RUnlock()
if !ok {
return nil, fmt.Errorf("unknown module: %s", id)
}
if mod.New == nil {
return nil, fmt.Errorf("module '%s' has no constructor", mod.ID)
}
val := mod.New().(interface{})
// value must be a pointer for unmarshaling into concrete type, even if
// the module's concrete type is a slice or map; New() *should* return
// a pointer, otherwise unmarshaling errors or panics will occur
if rv := reflect.ValueOf(val); rv.Kind() != reflect.Ptr {
log.Printf("[WARNING] ModuleInfo.New() for module '%s' did not return a pointer,"+
" so we are using reflection to make a pointer instead; please fix this by"+
" using new(Type) or &Type notation in your module's New() function.", id)
val = reflect.New(rv.Type()).Elem().Addr().Interface().(Module)
}
// fill in its config only if there is a config to fill in
if len(rawMsg) > 0 {
err := strictUnmarshalJSON(rawMsg, &val)
if err != nil {
return nil, fmt.Errorf("decoding module config: %s: %v", mod, err)
}
}
if val == nil {
// returned module values are almost always type-asserted
// before being used, so a nil value would panic; and there
// is no good reason to explicitly declare null modules in
// a config; it might be because the user is trying to achieve
// a result the developer isn't expecting, which is a smell
return nil, fmt.Errorf("module value cannot be null")
}
if prov, ok := val.(Provisioner); ok {
err := prov.Provision(ctx)
if err != nil {
// incomplete provisioning could have left state
// dangling, so make sure it gets cleaned up
if cleanerUpper, ok := val.(CleanerUpper); ok {
err2 := cleanerUpper.Cleanup()
if err2 != nil {
err = fmt.Errorf("%v; additionally, cleanup: %v", err, err2)
}
}
return nil, fmt.Errorf("provision %s: %v", mod, err)
}
}
if validator, ok := val.(Validator); ok {
err := validator.Validate()
if err != nil {
// since the module was already provisioned, make sure we clean up
if cleanerUpper, ok := val.(CleanerUpper); ok {
err2 := cleanerUpper.Cleanup()
if err2 != nil {
err = fmt.Errorf("%v; additionally, cleanup: %v", err, err2)
}
}
return nil, fmt.Errorf("%s: invalid configuration: %v", mod, err)
}
}
ctx.moduleInstances[id] = append(ctx.moduleInstances[id], val)
return val, nil
}
// loadModuleInline loads a module from a JSON raw message which decodes to
// a map[string]interface{}, where one of the object keys is moduleNameKey
// and the corresponding value is the module name (as a string) which can
// be found in the given scope. In other words, the module name is declared
// in-line with the module itself.
//
// This allows modules to be decoded into their concrete types and used when
// their names cannot be the unique key in a map, such as when there are
// multiple instances in the map or it appears in an array (where there are
// no custom keys). In other words, the key containing the module name is
// treated special/separate from all the other keys in the object.
func (ctx Context) loadModuleInline(moduleNameKey, moduleScope string, raw json.RawMessage) (interface{}, error) {
moduleName, raw, err := getModuleNameInline(moduleNameKey, raw)
if err != nil {
return nil, err
}
val, err := ctx.LoadModuleByID(moduleScope+"."+moduleName, raw)
if err != nil {
return nil, fmt.Errorf("loading module '%s': %v", moduleName, err)
}
return val, nil
}
// App returns the configured app named name. If no app with
// that name is currently configured, a new empty one will be
// instantiated. (The app module must still be registered.)
func (ctx Context) App(name string) (interface{}, error) {
if app, ok := ctx.cfg.apps[name]; ok {
return app, nil
}
appRaw := ctx.cfg.AppsRaw[name]
modVal, err := ctx.LoadModuleByID(name, appRaw)
if err != nil {
return nil, fmt.Errorf("loading %s app module: %v", name, err)
}
if appRaw != nil {
ctx.cfg.AppsRaw[name] = nil // allow GC to deallocate
}
ctx.cfg.apps[name] = modVal.(App)
return modVal, nil
}
// Storage returns the configured Caddy storage implementation.
func (ctx Context) Storage() certmagic.Storage {
return ctx.cfg.storage
}
// Logger returns a logger that can be used by mod.
func (ctx Context) Logger(mod Module) *zap.Logger {
return ctx.cfg.Logging.Logger(mod)
}
+118
View File
@@ -0,0 +1,118 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"encoding/json"
"io"
)
func ExampleContext_LoadModule() {
// this whole first part is just setting up for the example;
// note the struct tags - very important; we specify inline_key
// because that is the only way to know the module name
var ctx Context
myStruct := &struct {
// This godoc comment will appear in module documentation.
GuestModuleRaw json.RawMessage `json:"guest_module,omitempty" caddy:"namespace=example inline_key=name"`
// this is where the decoded module will be stored; in this
// example, we pretend we need an io.Writer but it can be
// any interface type that is useful to you
guestModule io.Writer
}{
GuestModuleRaw: json.RawMessage(`{"name":"module_name","foo":"bar"}`),
}
// if a guest module is provided, we can load it easily
if myStruct.GuestModuleRaw != nil {
mod, err := ctx.LoadModule(myStruct, "GuestModuleRaw")
if err != nil {
// you'd want to actually handle the error here
// return fmt.Errorf("loading guest module: %v", err)
}
// mod contains the loaded and provisioned module,
// it is now ready for us to use
myStruct.guestModule = mod.(io.Writer)
}
// use myStruct.guestModule from now on
}
func ExampleContext_LoadModule_array() {
// this whole first part is just setting up for the example;
// note the struct tags - very important; we specify inline_key
// because that is the only way to know the module name
var ctx Context
myStruct := &struct {
// This godoc comment will appear in module documentation.
GuestModulesRaw []json.RawMessage `json:"guest_modules,omitempty" caddy:"namespace=example inline_key=name"`
// this is where the decoded module will be stored; in this
// example, we pretend we need an io.Writer but it can be
// any interface type that is useful to you
guestModules []io.Writer
}{
GuestModulesRaw: []json.RawMessage{
json.RawMessage(`{"name":"module1_name","foo":"bar1"}`),
json.RawMessage(`{"name":"module2_name","foo":"bar2"}`),
},
}
// since our input is []json.RawMessage, the output will be []interface{}
mods, err := ctx.LoadModule(myStruct, "GuestModulesRaw")
if err != nil {
// you'd want to actually handle the error here
// return fmt.Errorf("loading guest modules: %v", err)
}
for _, mod := range mods.([]interface{}) {
myStruct.guestModules = append(myStruct.guestModules, mod.(io.Writer))
}
// use myStruct.guestModules from now on
}
func ExampleContext_LoadModule_map() {
// this whole first part is just setting up for the example;
// note the struct tags - very important; we don't specify
// inline_key because the map key is the module name
var ctx Context
myStruct := &struct {
// This godoc comment will appear in module documentation.
GuestModulesRaw ModuleMap `json:"guest_modules,omitempty" caddy:"namespace=example"`
// this is where the decoded module will be stored; in this
// example, we pretend we need an io.Writer but it can be
// any interface type that is useful to you
guestModules map[string]io.Writer
}{
GuestModulesRaw: ModuleMap{
"module1_name": json.RawMessage(`{"foo":"bar1"}`),
"module2_name": json.RawMessage(`{"foo":"bar2"}`),
},
}
// since our input is map[string]json.RawMessage, the output will be map[string]interface{}
mods, err := ctx.LoadModule(myStruct, "GuestModulesRaw")
if err != nil {
// you'd want to actually handle the error here
// return fmt.Errorf("loading guest modules: %v", err)
}
for modName, mod := range mods.(map[string]interface{}) {
myStruct.guestModules[modName] = mod.(io.Writer)
}
// use myStruct.guestModules from now on
}
+35
View File
@@ -0,0 +1,35 @@
module github.com/caddyserver/caddy/v2
go 1.14
require (
github.com/Masterminds/sprig/v3 v3.0.2
github.com/alecthomas/chroma v0.7.2-0.20200305040604-4f3623dce67a
github.com/andybalholm/brotli v1.0.0
github.com/caddyserver/certmagic v0.10.2
github.com/dustin/go-humanize v1.0.1-0.20200219035652-afde56e7acac
github.com/go-acme/lego/v3 v3.4.0
github.com/golang/groupcache v0.0.0-20200121045136-8c9f03a8e57e
github.com/ilibs/json5 v1.0.1
github.com/jsternberg/zap-logfmt v1.2.0
github.com/klauspost/compress v1.10.3
github.com/klauspost/cpuid v1.2.3
github.com/lucas-clemente/quic-go v0.15.2
github.com/manifoldco/promptui v0.7.0 // indirect
github.com/miekg/dns v1.1.28 // indirect
github.com/muhammadmuzzammil1998/jsonc v0.0.0-20200303171503-1e787b591db7
github.com/naoina/go-stringutil v0.1.0 // indirect
github.com/naoina/toml v0.1.1
github.com/smallstep/certificates v0.14.0-rc.5
github.com/smallstep/cli v0.14.0-rc.3
github.com/smallstep/truststore v0.9.4
github.com/vulcand/oxy v1.0.0
github.com/yuin/goldmark v1.1.25
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691
go.uber.org/zap v1.14.0
golang.org/x/crypto v0.0.0-20200311171314-f7b00557c8c4
golang.org/x/net v0.0.0-20200301022130-244492dfa37a
gopkg.in/natefinch/lumberjack.v2 v2.0.0
gopkg.in/square/go-jose.v2 v2.4.1 // indirect
gopkg.in/yaml.v2 v2.2.8
)
+1076
View File
File diff suppressed because it is too large Load Diff
+394
View File
@@ -0,0 +1,394 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"fmt"
"log"
"net"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
)
// Listen returns a listener suitable for use in a Caddy module.
// Always be sure to close listeners when you are done with them.
func Listen(network, addr string) (net.Listener, error) {
lnKey := network + "/" + addr
listenersMu.Lock()
defer listenersMu.Unlock()
// if listener already exists, increment usage counter, then return listener
if lnGlobal, ok := listeners[lnKey]; ok {
atomic.AddInt32(&lnGlobal.usage, 1)
return &fakeCloseListener{
usage: &lnGlobal.usage,
deadline: &lnGlobal.deadline,
deadlineMu: &lnGlobal.deadlineMu,
key: lnKey,
Listener: lnGlobal.ln,
}, nil
}
// or, create new one and save it
ln, err := net.Listen(network, addr)
if err != nil {
return nil, err
}
// make sure to start its usage counter at 1
lnGlobal := &globalListener{usage: 1, ln: ln}
listeners[lnKey] = lnGlobal
return &fakeCloseListener{
usage: &lnGlobal.usage,
deadline: &lnGlobal.deadline,
deadlineMu: &lnGlobal.deadlineMu,
key: lnKey,
Listener: ln,
}, nil
}
// ListenPacket returns a net.PacketConn suitable for use in a Caddy module.
// Always be sure to close the PacketConn when you are done.
func ListenPacket(network, addr string) (net.PacketConn, error) {
lnKey := network + "/" + addr
listenersMu.Lock()
defer listenersMu.Unlock()
// if listener already exists, increment usage counter, then return listener
if lnGlobal, ok := listeners[lnKey]; ok {
atomic.AddInt32(&lnGlobal.usage, 1)
log.Printf("[DEBUG] %s: Usage counter should not go above 2 or maybe 3, is now: %d", lnKey, atomic.LoadInt32(&lnGlobal.usage)) // TODO: remove
return &fakeClosePacketConn{usage: &lnGlobal.usage, key: lnKey, PacketConn: lnGlobal.pc}, nil
}
// or, create new one and save it
pc, err := net.ListenPacket(network, addr)
if err != nil {
return nil, err
}
// make sure to start its usage counter at 1
lnGlobal := &globalListener{usage: 1, pc: pc}
listeners[lnKey] = lnGlobal
return &fakeClosePacketConn{usage: &lnGlobal.usage, key: lnKey, PacketConn: pc}, nil
}
// fakeCloseListener's Close() method is a no-op. This allows
// stopping servers that are using the listener without giving
// up the socket; thus, servers become hot-swappable while the
// listener remains running. Listeners should be re-wrapped in
// a new fakeCloseListener each time the listener is reused.
// Other than the 'closed' field (which pertains to this value
// only), the other fields in this struct should be pointers to
// the associated globalListener's struct fields (except 'key'
// which is there for read-only purposes, so it can be a copy).
type fakeCloseListener struct {
closed int32 // accessed atomically; belongs to this struct only
usage *int32 // accessed atomically; global
deadline *bool // protected by deadlineMu; global
deadlineMu *sync.Mutex // global
key string // global, but read-only, so can be copy
net.Listener // global
}
// Accept accepts connections until Close() is called.
func (fcl *fakeCloseListener) Accept() (net.Conn, error) {
// if the listener is already "closed", return error
if atomic.LoadInt32(&fcl.closed) == 1 {
return nil, fcl.fakeClosedErr()
}
// wrap underlying accept
conn, err := fcl.Listener.Accept()
if err == nil {
return conn, nil
}
// accept returned with error
// TODO: This may be better as a condition variable so the deadline is cleared only once?
fcl.deadlineMu.Lock()
if *fcl.deadline {
switch ln := fcl.Listener.(type) {
case *net.TCPListener:
ln.SetDeadline(time.Time{})
case *net.UnixListener:
ln.SetDeadline(time.Time{})
}
*fcl.deadline = false
}
fcl.deadlineMu.Unlock()
if atomic.LoadInt32(&fcl.closed) == 1 {
// if we canceled the Accept() by setting a deadline
// on the listener, we need to make sure any callers of
// Accept() think the listener was actually closed;
// if we return the timeout error instead, callers might
// simply retry, leaking goroutines for longer
if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
return nil, fcl.fakeClosedErr()
}
}
return nil, err
}
// Close stops accepting new connections without
// closing the underlying listener, unless no one
// else is using it.
func (fcl *fakeCloseListener) Close() error {
if atomic.CompareAndSwapInt32(&fcl.closed, 0, 1) {
// unfortunately, there is no way to cancel any
// currently-blocking calls to Accept() that are
// awaiting connections since we're not actually
// closing the listener; so we cheat by setting
// a deadline in the past, which forces it to
// time out; note that this only works for
// certain types of listeners...
fcl.deadlineMu.Lock()
if !*fcl.deadline {
switch ln := fcl.Listener.(type) {
case *net.TCPListener:
ln.SetDeadline(time.Now().Add(-1 * time.Minute))
case *net.UnixListener:
ln.SetDeadline(time.Now().Add(-1 * time.Minute))
}
*fcl.deadline = true
}
fcl.deadlineMu.Unlock()
// since we're no longer using this listener,
// decrement the usage counter and, if no one
// else is using it, close underlying listener
if atomic.AddInt32(fcl.usage, -1) == 0 {
listenersMu.Lock()
delete(listeners, fcl.key)
listenersMu.Unlock()
err := fcl.Listener.Close()
if err != nil {
return err
}
}
}
return nil
}
func (fcl *fakeCloseListener) fakeClosedErr() error {
return &net.OpError{
Op: "accept",
Net: fcl.Listener.Addr().Network(),
Addr: fcl.Listener.Addr(),
Err: errFakeClosed,
}
}
type fakeClosePacketConn struct {
closed int32 // accessed atomically
usage *int32 // accessed atomically
key string
net.PacketConn
}
func (fcpc *fakeClosePacketConn) Close() error {
log.Println("[DEBUG] Fake-closing underlying packet conn") // TODO: remove this
if atomic.CompareAndSwapInt32(&fcpc.closed, 0, 1) {
// since we're no longer using this listener,
// decrement the usage counter and, if no one
// else is using it, close underlying listener
if atomic.AddInt32(fcpc.usage, -1) == 0 {
listenersMu.Lock()
delete(listeners, fcpc.key)
listenersMu.Unlock()
err := fcpc.PacketConn.Close()
if err != nil {
return err
}
}
}
return nil
}
// ErrFakeClosed is the underlying error value returned by
// fakeCloseListener.Accept() after Close() has been called,
// indicating that it is pretending to be closed so that the
// server using it can terminate, while the underlying
// socket is actually left open.
var errFakeClosed = fmt.Errorf("listener 'closed' 😉")
// globalListener keeps global state for a listener
// that may be shared by multiple servers. In other
// words, values in this struct exist only once and
// all other uses of these values point to the ones
// in this struct. In particular, the usage count
// (how many callers are using the listener), the
// actual listener, and synchronization of the
// listener's deadline changes are singular, global
// values that must not be copied.
type globalListener struct {
usage int32 // accessed atomically
deadline bool
deadlineMu sync.Mutex
ln net.Listener
pc net.PacketConn
}
// ParsedAddress contains the individual components
// for a parsed network address of the form accepted
// by ParseNetworkAddress(). Network should be a
// network value accepted by Go's net package. Port
// ranges are given by [StartPort, EndPort].
type ParsedAddress struct {
Network string
Host string
StartPort uint
EndPort uint
}
// IsUnixNetwork returns true if pa.Network is
// unix, unixgram, or unixpacket.
func (pa ParsedAddress) IsUnixNetwork() bool {
return isUnixNetwork(pa.Network)
}
// JoinHostPort is like net.JoinHostPort, but where the port
// is StartPort + offset.
func (pa ParsedAddress) JoinHostPort(offset uint) string {
if pa.IsUnixNetwork() {
return pa.Host
}
return net.JoinHostPort(pa.Host, strconv.Itoa(int(pa.StartPort+offset)))
}
// PortRangeSize returns how many ports are in
// pa's port range. Port ranges are inclusive,
// so the size is the difference of start and
// end ports plus one.
func (pa ParsedAddress) PortRangeSize() uint {
return (pa.EndPort - pa.StartPort) + 1
}
// String reconstructs the address string to the form expected
// by ParseNetworkAddress().
func (pa ParsedAddress) String() string {
port := strconv.FormatUint(uint64(pa.StartPort), 10)
if pa.StartPort != pa.EndPort {
port += "-" + strconv.FormatUint(uint64(pa.EndPort), 10)
}
return JoinNetworkAddress(pa.Network, pa.Host, port)
}
func isUnixNetwork(netw string) bool {
return netw == "unix" || netw == "unixgram" || netw == "unixpacket"
}
// ParseNetworkAddress parses addr into its individual
// components. The input string is expected to be of
// the form "network/host:port-range" where any part is
// optional. The default network, if unspecified, is tcp.
// Port ranges are inclusive.
//
// Network addresses are distinct from URLs and do not
// use URL syntax.
func ParseNetworkAddress(addr string) (ParsedAddress, error) {
var host, port string
network, host, port, err := SplitNetworkAddress(addr)
if network == "" {
network = "tcp"
}
if err != nil {
return ParsedAddress{}, err
}
if isUnixNetwork(network) {
return ParsedAddress{
Network: network,
Host: host,
}, nil
}
ports := strings.SplitN(port, "-", 2)
if len(ports) == 1 {
ports = append(ports, ports[0])
}
var start, end uint64
start, err = strconv.ParseUint(ports[0], 10, 16)
if err != nil {
return ParsedAddress{}, fmt.Errorf("invalid start port: %v", err)
}
end, err = strconv.ParseUint(ports[1], 10, 16)
if err != nil {
return ParsedAddress{}, fmt.Errorf("invalid end port: %v", err)
}
if end < start {
return ParsedAddress{}, fmt.Errorf("end port must not be less than start port")
}
if (end - start) > maxPortSpan {
return ParsedAddress{}, fmt.Errorf("port range exceeds %d ports", maxPortSpan)
}
return ParsedAddress{
Network: network,
Host: host,
StartPort: uint(start),
EndPort: uint(end),
}, nil
}
// SplitNetworkAddress splits a into its network, host, and port components.
// Note that port may be a port range (:X-Y), or omitted for unix sockets.
func SplitNetworkAddress(a string) (network, host, port string, err error) {
if idx := strings.Index(a, "/"); idx >= 0 {
network = strings.ToLower(strings.TrimSpace(a[:idx]))
a = a[idx+1:]
}
if isUnixNetwork(network) {
host = a
return
}
host, port, err = net.SplitHostPort(a)
return
}
// JoinNetworkAddress combines network, host, and port into a single
// address string of the form accepted by ParseNetworkAddress(). For
// unix sockets, the network should be "unix" (or "unixgram" or
// "unixpacket") and the path to the socket should be given as the
// host parameter.
func JoinNetworkAddress(network, host, port string) string {
var a string
if network != "" {
a = network + "/"
}
if host != "" && port == "" {
a += host
} else if port != "" {
a += net.JoinHostPort(host, port)
}
return a
}
var (
listeners = make(map[string]*globalListener)
listenersMu sync.Mutex
)
const maxPortSpan = 65535
+26
View File
@@ -0,0 +1,26 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// +build gofuzz
// +build gofuzz_libfuzzer
package caddy
func FuzzParseNetworkAddress(data []byte) int {
_, err := ParseNetworkAddress(string(data))
if err != nil {
return 0
}
return 1
}
+301
View File
@@ -0,0 +1,301 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"reflect"
"testing"
)
func TestSplitNetworkAddress(t *testing.T) {
for i, tc := range []struct {
input string
expectNetwork string
expectHost string
expectPort string
expectErr bool
}{
{
input: "",
expectErr: true,
},
{
input: "foo",
expectErr: true,
},
{
input: "foo:1234",
expectHost: "foo",
expectPort: "1234",
},
{
input: "foo:1234-5678",
expectHost: "foo",
expectPort: "1234-5678",
},
{
input: "udp/foo:1234",
expectNetwork: "udp",
expectHost: "foo",
expectPort: "1234",
},
{
input: "tcp6/foo:1234-5678",
expectNetwork: "tcp6",
expectHost: "foo",
expectPort: "1234-5678",
},
{
input: "udp/",
expectNetwork: "udp",
expectErr: true,
},
{
input: "unix//foo/bar",
expectNetwork: "unix",
expectHost: "/foo/bar",
},
{
input: "unixgram//foo/bar",
expectNetwork: "unixgram",
expectHost: "/foo/bar",
},
{
input: "unixpacket//foo/bar",
expectNetwork: "unixpacket",
expectHost: "/foo/bar",
},
} {
actualNetwork, actualHost, actualPort, err := SplitNetworkAddress(tc.input)
if tc.expectErr && err == nil {
t.Errorf("Test %d: Expected error but got: %v", i, err)
}
if !tc.expectErr && err != nil {
t.Errorf("Test %d: Expected no error but got: %v", i, err)
}
if actualNetwork != tc.expectNetwork {
t.Errorf("Test %d: Expected network '%s' but got '%s'", i, tc.expectNetwork, actualNetwork)
}
if actualHost != tc.expectHost {
t.Errorf("Test %d: Expected host '%s' but got '%s'", i, tc.expectHost, actualHost)
}
if actualPort != tc.expectPort {
t.Errorf("Test %d: Expected port '%s' but got '%s'", i, tc.expectPort, actualPort)
}
}
}
func TestJoinNetworkAddress(t *testing.T) {
for i, tc := range []struct {
network, host, port string
expect string
}{
{
network: "", host: "", port: "",
expect: "",
},
{
network: "tcp", host: "", port: "",
expect: "tcp/",
},
{
network: "", host: "foo", port: "",
expect: "foo",
},
{
network: "", host: "", port: "1234",
expect: ":1234",
},
{
network: "", host: "", port: "1234-5678",
expect: ":1234-5678",
},
{
network: "", host: "foo", port: "1234",
expect: "foo:1234",
},
{
network: "udp", host: "foo", port: "1234",
expect: "udp/foo:1234",
},
{
network: "udp", host: "", port: "1234",
expect: "udp/:1234",
},
{
network: "unix", host: "/foo/bar", port: "",
expect: "unix//foo/bar",
},
{
network: "", host: "::1", port: "1234",
expect: "[::1]:1234",
},
} {
actual := JoinNetworkAddress(tc.network, tc.host, tc.port)
if actual != tc.expect {
t.Errorf("Test %d: Expected '%s' but got '%s'", i, tc.expect, actual)
}
}
}
func TestParseNetworkAddress(t *testing.T) {
for i, tc := range []struct {
input string
expectAddr ParsedAddress
expectErr bool
}{
{
input: "",
expectErr: true,
},
{
input: ":",
expectErr: true,
},
{
input: ":1234",
expectAddr: ParsedAddress{
Network: "tcp",
Host: "",
StartPort: 1234,
EndPort: 1234,
},
},
{
input: "tcp/:1234",
expectAddr: ParsedAddress{
Network: "tcp",
Host: "",
StartPort: 1234,
EndPort: 1234,
},
},
{
input: "tcp6/:1234",
expectAddr: ParsedAddress{
Network: "tcp6",
Host: "",
StartPort: 1234,
EndPort: 1234,
},
},
{
input: "tcp4/localhost:1234",
expectAddr: ParsedAddress{
Network: "tcp4",
Host: "localhost",
StartPort: 1234,
EndPort: 1234,
},
},
{
input: "unix//foo/bar",
expectAddr: ParsedAddress{
Network: "unix",
Host: "/foo/bar",
},
},
{
input: "localhost:1234-1234",
expectAddr: ParsedAddress{
Network: "tcp",
Host: "localhost",
StartPort: 1234,
EndPort: 1234,
},
},
{
input: "localhost:2-1",
expectErr: true,
},
{
input: "localhost:0",
expectAddr: ParsedAddress{
Network: "tcp",
Host: "localhost",
StartPort: 0,
EndPort: 0,
},
},
{
input: "localhost:1-999999999999",
expectErr: true,
},
} {
actualAddr, err := ParseNetworkAddress(tc.input)
if tc.expectErr && err == nil {
t.Errorf("Test %d: Expected error but got: %v", i, err)
}
if !tc.expectErr && err != nil {
t.Errorf("Test %d: Expected no error but got: %v", i, err)
}
if actualAddr.Network != tc.expectAddr.Network {
t.Errorf("Test %d: Expected network '%v' but got '%v'", i, tc.expectAddr, actualAddr)
}
if !reflect.DeepEqual(tc.expectAddr, actualAddr) {
t.Errorf("Test %d: Expected addresses %v but got %v", i, tc.expectAddr, actualAddr)
}
}
}
func TestJoinHostPort(t *testing.T) {
for i, tc := range []struct {
pa ParsedAddress
offset uint
expect string
}{
{
pa: ParsedAddress{
Network: "tcp",
Host: "localhost",
StartPort: 1234,
EndPort: 1234,
},
expect: "localhost:1234",
},
{
pa: ParsedAddress{
Network: "tcp",
Host: "localhost",
StartPort: 1234,
EndPort: 1235,
},
expect: "localhost:1234",
},
{
pa: ParsedAddress{
Network: "tcp",
Host: "localhost",
StartPort: 1234,
EndPort: 1235,
},
offset: 1,
expect: "localhost:1235",
},
{
pa: ParsedAddress{
Network: "unix",
Host: "/run/php/php7.3-fpm.sock",
},
expect: "/run/php/php7.3-fpm.sock",
},
} {
actual := tc.pa.JoinHostPort(tc.offset)
if actual != tc.expect {
t.Errorf("Test %d: Expected '%s' but got '%s'", i, tc.expect, actual)
}
}
}
+689
View File
@@ -0,0 +1,689 @@
// Copyright 2015 Matthew Holt and The Caddy Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package caddy
import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"strings"
"sync"
"time"
"go.uber.org/zap"
"go.uber.org/zap/zapcore"
)
func init() {
RegisterModule(StdoutWriter{})
RegisterModule(StderrWriter{})
RegisterModule(DiscardWriter{})
}
// Logging facilitates logging within Caddy.
//
// By default, all logs at INFO level and higher are written to
// standard error ("stderr" writer) in a human-readable format
// ("console" encoder). The default log is called "default" and
// you can customize it. You can also define additional logs.
//
// All defined logs accept all log entries by default, but you
// can filter by level and module/logger names. A logger's name
// is the same as the module's name, but a module may append to
// logger names for more specificity. For example, you can
// filter logs emitted only by HTTP handlers using the name
// "http.handlers", because all HTTP handler module names have
// that prefix.
//
// Caddy logs (except the sink) are mostly zero-allocation, so
// they are very high-performing in terms of memory and CPU time.
// Enabling sampling can further increase throughput on extremely
// high-load servers.
type Logging struct {
// Sink is the destination for all unstructured logs emitted
// from Go's standard library logger. These logs are common
// in dependencies that are not designed specifically for use
// in Caddy. Because it is global and unstructured, the sink
// lacks most advanced features and customizations.
Sink *StandardLibLog `json:"sink,omitempty"`
// Logs are your logs, keyed by an arbitrary name of your
// choosing. The default log can be customized by defining
// a log called "default". You can further define other logs
// and filter what kinds of entries they accept.
Logs map[string]*CustomLog `json:"logs,omitempty"`
// a list of all keys for open writers; all writers
// that are opened to provision this logging config
// must have their keys added to this list so they
// can be closed when cleaning up
writerKeys []string
}
// openLogs sets up the config and opens all the configured writers.
// It closes its logs when ctx is canceled, so it should clean up
// after itself.
func (logging *Logging) openLogs(ctx Context) error {
// make sure to deallocate resources when context is done
ctx.OnCancel(func() {
err := logging.closeLogs()
if err != nil {
Log().Error("closing logs", zap.Error(err))
}
})
// set up the "sink" log first (std lib's default global logger)
if logging.Sink != nil {
err := logging.Sink.provision(ctx, logging)
if err != nil {
return fmt.Errorf("setting up sink log: %v", err)
}
}
// as a special case, set up the default structured Caddy log next
if err := logging.setupNewDefault(ctx); err != nil {
return err
}
// then set up any other custom logs
for name, l := range logging.Logs {
// the default log is already set up
if name == "default" {
continue
}
err := l.provision(ctx, logging)
if err != nil {
return fmt.Errorf("setting up custom log '%s': %v", name, err)
}
// Any other logs that use the discard writer can be deleted
// entirely. This avoids encoding and processing of each
// log entry that would just be thrown away anyway. Notably,
// we do not reach this point for the default log, which MUST
// exist, otherwise core log emissions would panic because
// they use the Log() function directly which expects a non-nil
// logger. Even if we keep logs with a discard writer, they
// have a nop core, and keeping them at all seems unnecessary.
if _, ok := l.writerOpener.(*DiscardWriter); ok {
delete(logging.Logs, name)
continue
}
}
return nil
}
func (logging *Logging) setupNewDefault(ctx Context) error {
if logging.Logs == nil {
logging.Logs = make(map[string]*CustomLog)
}
// extract the user-defined default log, if any
newDefault := new(defaultCustomLog)
if userDefault, ok := logging.Logs["default"]; ok {
newDefault.CustomLog = userDefault
} else {
// if none, make one with our own default settings
var err error
newDefault, err = newDefaultProductionLog()
if err != nil {
return fmt.Errorf("setting up default Caddy log: %v", err)
}
logging.Logs["default"] = newDefault.CustomLog
}
// set up this new log
err := newDefault.CustomLog.provision(ctx, logging)
if err != nil {
return fmt.Errorf("setting up default log: %v", err)
}
newDefault.logger = zap.New(newDefault.CustomLog.core)
// redirect the default caddy logs
defaultLoggerMu.Lock()
oldDefault := defaultLogger
defaultLogger = newDefault
defaultLoggerMu.Unlock()
// if the new writer is different, indicate it in the logs for convenience
var newDefaultLogWriterKey, currentDefaultLogWriterKey string
var newDefaultLogWriterStr, currentDefaultLogWriterStr string
if newDefault.writerOpener != nil {
newDefaultLogWriterKey = newDefault.writerOpener.WriterKey()
newDefaultLogWriterStr = newDefault.writerOpener.String()
}
if oldDefault.writerOpener != nil {
currentDefaultLogWriterKey = oldDefault.writerOpener.WriterKey()
currentDefaultLogWriterStr = oldDefault.writerOpener.String()
}
if newDefaultLogWriterKey != currentDefaultLogWriterKey {
oldDefault.logger.Info("redirected default logger",
zap.String("from", currentDefaultLogWriterStr),
zap.String("to", newDefaultLogWriterStr),
)
}
return nil
}
// closeLogs cleans up resources allocated during openLogs.
// A successful call to openLogs calls this automatically
// when the context is canceled.
func (logging *Logging) closeLogs() error {
for _, key := range logging.writerKeys {
_, err := writers.Delete(key)
if err != nil {
log.Printf("[ERROR] Closing log writer %v: %v", key, err)
}
}
return nil
}
// Logger returns a logger that is ready for the module to use.
func (logging *Logging) Logger(mod Module) *zap.Logger {
modID := string(mod.CaddyModule().ID)
var cores []zapcore.Core
if logging != nil {
for _, l := range logging.Logs {
if l.matchesModule(modID) {
if len(l.Include) == 0 && len(l.Exclude) == 0 {
cores = append(cores, l.core)
continue
}
cores = append(cores, &filteringCore{Core: l.core, cl: l})
}
}
}
multiCore := zapcore.NewTee(cores...)
return zap.New(multiCore).Named(string(modID))
}
// openWriter opens a writer using opener, and returns true if
// the writer is new, or false if the writer already exists.
func (logging *Logging) openWriter(opener WriterOpener) (io.WriteCloser, bool, error) {
key := opener.WriterKey()
writer, loaded, err := writers.LoadOrNew(key, func() (Destructor, error) {
w, err := opener.OpenWriter()
return writerDestructor{w}, err
})
if err != nil {
return nil, false, err
}
logging.writerKeys = append(logging.writerKeys, key)
return writer.(io.WriteCloser), !loaded, nil
}
// WriterOpener is a module that can open a log writer.
// It can return a human-readable string representation
// of itself so that operators can understand where
// the logs are going.
type WriterOpener interface {
fmt.Stringer
// WriterKey is a string that uniquely identifies this
// writer configuration. It is not shown to humans.
WriterKey() string
// OpenWriter opens a log for writing. The writer
// should be safe for concurrent use but need not
// be synchronous.
OpenWriter() (io.WriteCloser, error)
}
type writerDestructor struct {
io.WriteCloser
}
func (wdest writerDestructor) Destruct() error {
return wdest.Close()
}
// StandardLibLog configures the default Go standard library
// global logger in the log package. This is necessary because
// module dependencies which are not built specifically for
// Caddy will use the standard logger. This is also known as
// the "sink" logger.
type StandardLibLog struct {
// The module that writes out log entries for the sink.
WriterRaw json.RawMessage `json:"writer,omitempty" caddy:"namespace=caddy.logging.writers inline_key=output"`
writer io.WriteCloser
}
func (sll *StandardLibLog) provision(ctx Context, logging *Logging) error {
if sll.WriterRaw != nil {
mod, err := ctx.LoadModule(sll, "WriterRaw")
if err != nil {
return fmt.Errorf("loading sink log writer module: %v", err)
}
wo := mod.(WriterOpener)
var isNew bool
sll.writer, isNew, err = logging.openWriter(wo)
if err != nil {
return fmt.Errorf("opening sink log writer %#v: %v", mod, err)
}
if isNew {
log.Printf("[INFO] Redirecting sink to: %s", wo)
log.SetOutput(sll.writer)
log.Printf("[INFO] Redirected sink to here (%s)", wo)
}
}
return nil
}
// CustomLog represents a custom logger configuration.
//
// By default, a log will emit all log entries. Some entries
// will be skipped if sampling is enabled. Further, the Include
// and Exclude parameters define which loggers (by name) are
// allowed or rejected from emitting in this log. If both Include
// and Exclude are populated, their values must be mutually
// exclusive, and longer namespaces have priority. If neither
// are populated, all logs are emitted.
type CustomLog struct {
// The writer defines where log entries are emitted.
WriterRaw json.RawMessage `json:"writer,omitempty" caddy:"namespace=caddy.logging.writers inline_key=output"`
// The encoder is how the log entries are formatted or encoded.
EncoderRaw json.RawMessage `json:"encoder,omitempty" caddy:"namespace=caddy.logging.encoders inline_key=format"`
// Level is the minimum level to emit, and is inclusive.
// Possible levels: DEBUG, INFO, WARN, ERROR, PANIC, and FATAL
Level string `json:"level,omitempty"`
// Sampling configures log entry sampling. If enabled,
// only some log entries will be emitted. This is useful
// for improving performance on extremely high-pressure
// servers.
Sampling *LogSampling `json:"sampling,omitempty"`
// Include defines the names of loggers to emit in this
// log. For example, to include only logs emitted by the
// admin API, you would include "admin.api".
Include []string `json:"include,omitempty"`
// Exclude defines the names of loggers that should be
// skipped by this log. For example, to exclude only
// HTTP access logs, you would exclude "http.log.access".
Exclude []string `json:"exclude,omitempty"`
writerOpener WriterOpener
writer io.WriteCloser
encoder zapcore.Encoder
levelEnabler zapcore.LevelEnabler
core zapcore.Core
}
func (cl *CustomLog) provision(ctx Context, logging *Logging) error {
// Replace placeholder for log level
repl := NewReplacer()
level, err := repl.ReplaceOrErr(cl.Level, true, true)
if err != nil {
return fmt.Errorf("invalid log level: %v", err)
}
level = strings.ToLower(level)
// set up the log level
switch level {
case "debug":
cl.levelEnabler = zapcore.DebugLevel
case "", "info":
cl.levelEnabler = zapcore.InfoLevel
case "warn":
cl.levelEnabler = zapcore.WarnLevel
case "error":
cl.levelEnabler = zapcore.ErrorLevel
case "panic":
cl.levelEnabler = zapcore.PanicLevel
case "fatal":
cl.levelEnabler = zapcore.FatalLevel
default:
return fmt.Errorf("unrecognized log level: %s", cl.Level)
}
// If both Include and Exclude lists are populated, then each item must
// be a superspace or subspace of an item in the other list, because
// populating both lists means that any given item is either a rule
// or an exception to another rule. But if the item is not a super-
// or sub-space of any item in the other list, it is neither a rule
// nor an exception, and is a contradiction. Ensure, too, that the
// sets do not intersect, which is also a contradiction.
if len(cl.Include) > 0 && len(cl.Exclude) > 0 {
// prevent intersections
for _, allow := range cl.Include {
for _, deny := range cl.Exclude {
if allow == deny {
return fmt.Errorf("include and exclude must not intersect, but found %s in both lists", allow)
}
}
}
// ensure namespaces are nested
outer:
for _, allow := range cl.Include {
for _, deny := range cl.Exclude {
if strings.HasPrefix(allow+".", deny+".") ||
strings.HasPrefix(deny+".", allow+".") {
continue outer
}
}
return fmt.Errorf("when both include and exclude are populated, each element must be a superspace or subspace of one in the other list; check '%s' in include", allow)
}
}
if cl.EncoderRaw != nil {
mod, err := ctx.LoadModule(cl, "EncoderRaw")
if err != nil {
return fmt.Errorf("loading log encoder module: %v", err)
}
cl.encoder = mod.(zapcore.Encoder)
}
if cl.encoder == nil {
cl.encoder = newDefaultProductionLogEncoder()
}
if cl.WriterRaw != nil {
mod, err := ctx.LoadModule(cl, "WriterRaw")
if err != nil {
return fmt.Errorf("loading log writer module: %v", err)
}
cl.writerOpener = mod.(WriterOpener)
}
if cl.writerOpener == nil {
cl.writerOpener = StderrWriter{}
}
cl.writer, _, err = logging.openWriter(cl.writerOpener)
if err != nil {
return fmt.Errorf("opening log writer using %#v: %v", cl.writerOpener, err)
}
cl.buildCore()
return nil
}
func (cl *CustomLog) buildCore() {
// logs which only discard their output don't need
// to perform encoding or any other processing steps
// at all, so just shorcut to a nop core instead
if _, ok := cl.writerOpener.(*DiscardWriter); ok {
cl.core = zapcore.NewNopCore()
return
}
c := zapcore.NewCore(
cl.encoder,
zapcore.AddSync(cl.writer),
cl.levelEnabler,
)
if cl.Sampling != nil {
if cl.Sampling.Interval == 0 {
cl.Sampling.Interval = 1 * time.Second
}
if cl.Sampling.First == 0 {
cl.Sampling.First = 100
}
if cl.Sampling.Thereafter == 0 {
cl.Sampling.Thereafter = 100
}
c = zapcore.NewSampler(c, cl.Sampling.Interval,
cl.Sampling.First, cl.Sampling.Thereafter)
}
cl.core = c
}
func (cl *CustomLog) matchesModule(moduleID string) bool {
return cl.loggerAllowed(string(moduleID), true)
}
// loggerAllowed returns true if name is allowed to emit
// to cl. isModule should be true if name is the name of
// a module and you want to see if ANY of that module's
// logs would be permitted.
func (cl *CustomLog) loggerAllowed(name string, isModule bool) bool {
// accept all loggers by default
if len(cl.Include) == 0 && len(cl.Exclude) == 0 {
return true
}
// append a dot so that partial names don't match
// (i.e. we don't want "foo.b" to match "foo.bar"); we
// will also have to append a dot when we do HasPrefix
// below to compensate for when when namespaces are equal
if name != "" && name != "*" && name != "." {
name += "."
}
var longestAccept, longestReject int
if len(cl.Include) > 0 {
for _, namespace := range cl.Include {
var hasPrefix bool
if isModule {
hasPrefix = strings.HasPrefix(namespace+".", name)
} else {
hasPrefix = strings.HasPrefix(name, namespace+".")
}
if hasPrefix && len(namespace) > longestAccept {
longestAccept = len(namespace)
}
}
// the include list was populated, meaning that
// a match in this list is absolutely required
// if we are to accept the entry
if longestAccept == 0 {
return false
}
}
if len(cl.Exclude) > 0 {
for _, namespace := range cl.Exclude {
// * == all logs emitted by modules
// . == all logs emitted by core
if (namespace == "*" && name != ".") ||
(namespace == "." && name == ".") {
return false
}
if strings.HasPrefix(name, namespace+".") &&
len(namespace) > longestReject {
longestReject = len(namespace)
}
}
// the reject list is populated, so we have to
// reject this entry if its match is better
// than the best from the accept list
if longestReject > longestAccept {
return false
}
}
return (longestAccept > longestReject) ||
(len(cl.Include) == 0 && longestReject == 0)
}
// filteringCore filters log entries based on logger name,
// according to the rules of a CustomLog.
type filteringCore struct {
zapcore.Core
cl *CustomLog
}
// With properly wraps With.
func (fc *filteringCore) With(fields []zapcore.Field) zapcore.Core {
return &filteringCore{
Core: fc.Core.With(fields),
cl: fc.cl,
}
}
// Check only allows the log entry if its logger name
// is allowed from the include/exclude rules of fc.cl.
func (fc *filteringCore) Check(e zapcore.Entry, ce *zapcore.CheckedEntry) *zapcore.CheckedEntry {
if fc.cl.loggerAllowed(e.LoggerName, false) {
return fc.Core.Check(e, ce)
}
return ce
}
// LogSampling configures log entry sampling.
type LogSampling struct {
// The window over which to conduct sampling.
Interval time.Duration `json:"interval,omitempty"`
// Log this many entries within a given level and
// message for each interval.
First int `json:"first,omitempty"`
// If more entries with the same level and message
// are seen during the same interval, keep one in
// this many entries until the end of the interval.
Thereafter int `json:"thereafter,omitempty"`
}
type (
// StdoutWriter writes logs to standard out.
StdoutWriter struct{}
// StderrWriter writes logs to standard error.
StderrWriter struct{}
// DiscardWriter discards all writes.
DiscardWriter struct{}
)
// CaddyModule returns the Caddy module information.
func (StdoutWriter) CaddyModule() ModuleInfo {
return ModuleInfo{
ID: "caddy.logging.writers.stdout",
New: func() Module { return new(StdoutWriter) },
}
}
// CaddyModule returns the Caddy module information.
func (StderrWriter) CaddyModule() ModuleInfo {
return ModuleInfo{
ID: "caddy.logging.writers.stderr",
New: func() Module { return new(StderrWriter) },
}
}
// CaddyModule returns the Caddy module information.
func (DiscardWriter) CaddyModule() ModuleInfo {
return ModuleInfo{
ID: "caddy.logging.writers.discard",
New: func() Module { return new(DiscardWriter) },
}
}
func (StdoutWriter) String() string { return "stdout" }
func (StderrWriter) String() string { return "stderr" }
func (DiscardWriter) String() string { return "discard" }
// WriterKey returns a unique key representing stdout.
func (StdoutWriter) WriterKey() string { return "std:out" }
// WriterKey returns a unique key representing stderr.
func (StderrWriter) WriterKey() string { return "std:err" }
// WriterKey returns a unique key representing discard.
func (DiscardWriter) WriterKey() string { return "discard" }
// OpenWriter returns os.Stdout that can't be closed.
func (StdoutWriter) OpenWriter() (io.WriteCloser, error) {
return notClosable{os.Stdout}, nil
}
// OpenWriter returns os.Stderr that can't be closed.
func (StderrWriter) OpenWriter() (io.WriteCloser, error) {
return notClosable{os.Stderr}, nil
}
// OpenWriter returns ioutil.Discard that can't be closed.
func (DiscardWriter) OpenWriter() (io.WriteCloser, error) {
return notClosable{ioutil.Discard}, nil
}
// notClosable is an io.WriteCloser that can't be closed.
type notClosable struct{ io.Writer }
func (fc notClosable) Close() error { return nil }
type defaultCustomLog struct {
*CustomLog
logger *zap.Logger
}
// newDefaultProductionLog configures a custom log that is
// intended for use by default if no other log is specified
// in a config. It writes to stderr, uses the console encoder,
// and enables INFO-level logs and higher.
func newDefaultProductionLog() (*defaultCustomLog, error) {
cl := new(CustomLog)
cl.writerOpener = StderrWriter{}
var err error
cl.writer, err = cl.writerOpener.OpenWriter()
if err != nil {
return nil, err
}
cl.encoder = newDefaultProductionLogEncoder()
cl.levelEnabler = zapcore.InfoLevel
cl.buildCore()
return &defaultCustomLog{
CustomLog: cl,
logger: zap.New(cl.core),
}, nil
}
func newDefaultProductionLogEncoder() zapcore.Encoder {
encCfg := zap.NewProductionEncoderConfig()
encCfg.EncodeLevel = zapcore.CapitalColorLevelEncoder
encCfg.EncodeTime = func(ts time.Time, encoder zapcore.PrimitiveArrayEncoder) {
encoder.AppendString(ts.UTC().Format("2006/01/02 15:04:05.000"))
}
return zapcore.NewConsoleEncoder(encCfg)
}
// Log returns the current default logger.
func Log() *zap.Logger {
defaultLoggerMu.RLock()
defer defaultLoggerMu.RUnlock()
return defaultLogger.logger
}
var (
defaultLogger, _ = newDefaultProductionLog()
defaultLoggerMu sync.RWMutex
)
var writers = NewUsagePool()
// Interface guards
var (
_ io.WriteCloser = (*notClosable)(nil)
_ WriterOpener = (*StdoutWriter)(nil)
_ WriterOpener = (*StderrWriter)(nil)
)
-208
View File
@@ -1,208 +0,0 @@
package main
import (
"bytes"
"errors"
"flag"
"fmt"
"io/ioutil"
"log"
"net"
"os"
"path"
"runtime"
"strconv"
"strings"
"sync"
"github.com/mholt/caddy/config"
"github.com/mholt/caddy/server"
)
var (
conf string
http2 bool // TODO: temporary flag until http2 is standard
quiet bool
cpu string
)
func init() {
flag.StringVar(&conf, "conf", "", "Configuration file to use (default="+config.DefaultConfigFile+")")
flag.BoolVar(&http2, "http2", true, "Enable HTTP/2 support") // TODO: temporary flag until http2 merged into std lib
flag.BoolVar(&quiet, "quiet", false, "Quiet mode (no initialization output)")
flag.StringVar(&cpu, "cpu", "100%", "CPU cap")
flag.StringVar(&config.Root, "root", config.DefaultRoot, "Root path to default site")
flag.StringVar(&config.Host, "host", config.DefaultHost, "Default host")
flag.StringVar(&config.Port, "port", config.DefaultPort, "Default port")
config.AppName = "Caddy"
config.AppVersion = "0.6.0"
}
func main() {
flag.Parse()
var wg sync.WaitGroup
// Set CPU cap
err := setCPU(cpu)
if err != nil {
log.Fatal(err)
}
// Load config from file
allConfigs, err := loadConfigs()
if err != nil {
log.Fatal(err)
}
// Group by address (virtual hosts)
addresses, err := arrangeBindings(allConfigs)
if err != nil {
log.Fatal(err)
}
// Start each server with its one or more configurations
for addr, configs := range addresses {
s, err := server.New(addr, configs, configs[0].TLS.Enabled)
if err != nil {
log.Fatal(err)
}
s.HTTP2 = http2 // TODO: This setting is temporary
wg.Add(1)
go func(s *server.Server) {
defer wg.Done()
err := s.Serve()
if err != nil {
log.Fatal(err) // kill whole process to avoid a half-alive zombie server
}
}(s)
if !quiet {
for _, config := range configs {
fmt.Println(config.Address())
}
}
}
wg.Wait()
}
// loadConfigs loads configuration from a file or stdin (piped).
// Configuration is obtained from one of three sources, tried
// in this order: 1. -conf flag, 2. stdin, 3. Caddyfile.
// If none of those are available, a default configuration is
// loaded.
func loadConfigs() ([]server.Config, error) {
// -conf flag
if conf != "" {
file, err := os.Open(conf)
if err != nil {
return []server.Config{}, err
}
defer file.Close()
return config.Load(path.Base(conf), file)
}
// stdin
fi, err := os.Stdin.Stat()
if err == nil && fi.Mode()&os.ModeCharDevice == 0 {
// Note that a non-nil error is not a problem. Windows
// will not create a stdin if there is no pipe, which
// produces an error when calling Stat(). But Unix will
// make one either way, which is why we also check that
// bitmask.
confBody, err := ioutil.ReadAll(os.Stdin)
if err != nil {
log.Fatal(err)
}
if len(confBody) > 0 {
return config.Load("stdin", bytes.NewReader(confBody))
}
}
// Caddyfile
file, err := os.Open(config.DefaultConfigFile)
if err != nil {
if os.IsNotExist(err) {
return []server.Config{config.Default()}, nil
}
return []server.Config{}, err
}
defer file.Close()
return config.Load(config.DefaultConfigFile, file)
}
// arrangeBindings groups configurations by their bind address. For example,
// a server that should listen on localhost and another on 127.0.0.1 will
// be grouped into the same address: 127.0.0.1. It will return an error
// if the address lookup fails or if a TLS listener is configured on the
// same address as a plaintext HTTP listener.
func arrangeBindings(allConfigs []server.Config) (map[string][]server.Config, error) {
addresses := make(map[string][]server.Config)
// Group configs by bind address
for _, conf := range allConfigs {
addr, err := net.ResolveTCPAddr("tcp", conf.Address())
if err != nil {
return addresses, errors.New("Could not serve " + conf.Address() + " - " + err.Error())
}
addresses[addr.String()] = append(addresses[addr.String()], conf)
}
// Don't allow HTTP and HTTPS to be served on the same address
for _, configs := range addresses {
isTLS := configs[0].TLS.Enabled
for _, config := range configs {
if config.TLS.Enabled != isTLS {
thisConfigProto, otherConfigProto := "HTTP", "HTTP"
if config.TLS.Enabled {
thisConfigProto = "HTTPS"
}
if configs[0].TLS.Enabled {
otherConfigProto = "HTTPS"
}
return addresses, fmt.Errorf("Configuration error: Cannot multiplex %s (%s) and %s (%s) on same address",
configs[0].Address(), otherConfigProto, config.Address(), thisConfigProto)
}
}
}
return addresses, nil
}
// setCPU parses string cpu and sets GOMAXPROCS
// according to its value. It accepts either
// a number (e.g. 3) or a percent (e.g. 50%).
func setCPU(cpu string) error {
var numCPU int
availCPU := runtime.NumCPU()
if strings.HasSuffix(cpu, "%") {
// Percent
var percent float32
pctStr := cpu[:len(cpu)-1]
pctInt, err := strconv.Atoi(pctStr)
if err != nil || pctInt < 1 || pctInt > 100 {
return errors.New("Invalid CPU value: percentage must be between 1-100")
}
percent = float32(pctInt) / 100
numCPU = int(float32(availCPU) * percent)
} else {
// Number
num, err := strconv.Atoi(cpu)
if err != nil || num < 1 {
return errors.New("Invalid CPU value: provide a number or percent greater than 0")
}
numCPU = num
}
if numCPU > availCPU {
numCPU = availCPU
}
runtime.GOMAXPROCS(numCPU)
return nil
}
-53
View File
@@ -1,53 +0,0 @@
// Package basicauth implements HTTP Basic Authentication.
package basicauth
import (
"net/http"
"github.com/mholt/caddy/middleware"
)
// BasicAuth is middleware to protect resources with a username and password.
// Note that HTTP Basic Authentication is not secure by itself and should
// not be used to protect important assets without HTTPS. Even then, the
// security of HTTP Basic Auth is disputed. Use discretion when deciding
// what to protect with BasicAuth.
type BasicAuth struct {
Next middleware.Handler
Rules []Rule
}
// ServeHTTP implements the middleware.Handler interface.
func (a BasicAuth) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
for _, rule := range a.Rules {
for _, res := range rule.Resources {
if !middleware.Path(r.URL.Path).Matches(res) {
continue
}
// Path matches; parse auth header
username, password, ok := r.BasicAuth()
// Check credentials
if !ok || username != rule.Username || password != rule.Password {
w.Header().Set("WWW-Authenticate", "Basic")
return http.StatusUnauthorized, nil
}
// "It's an older code, sir, but it checks out. I was about to clear them."
return a.Next.ServeHTTP(w, r)
}
}
// Pass-thru when no paths match
return a.Next.ServeHTTP(w, r)
}
// Rule represents a BasicAuth rule. A username and password
// combination protect the associated resources, which are
// file or directory paths.
type Rule struct {
Username string
Password string
Resources []string
}
-178
View File
@@ -1,178 +0,0 @@
// Package browse provides middleware for listing files in a directory
// when directory path is requested instead of a specific file.
package browse
import (
"bytes"
"html/template"
"net/http"
"net/url"
"os"
"path"
"strings"
"time"
"github.com/dustin/go-humanize"
"github.com/mholt/caddy/middleware"
)
// Browse is an http.Handler that can show a file listing when
// directories in the given paths are specified.
type Browse struct {
Next middleware.Handler
Root string
Configs []Config
}
// Config is a configuration for browsing in a particular path.
type Config struct {
PathScope string
Template *template.Template
}
// A Listing is used to fill out a template.
type Listing struct {
// The name of the directory (the last element of the path)
Name string
// The full path of the request
Path string
// Whether the parent directory is browsable
CanGoUp bool
// The items (files and folders) in the path
Items []FileInfo
}
// FileInfo is the info about a particular file or directory
type FileInfo struct {
IsDir bool
Name string
Size int64
URL string
ModTime time.Time
Mode os.FileMode
}
func (fi FileInfo) HumanSize() string {
return humanize.Bytes(uint64(fi.Size))
}
func (fi FileInfo) HumanModTime(format string) string {
return fi.ModTime.Format(format)
}
var IndexPages = []string{
"index.html",
"index.htm",
"default.html",
"default.htm",
}
// ServeHTTP implements the middleware.Handler interface.
func (b Browse) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
filename := b.Root + r.URL.Path
info, err := os.Stat(filename)
if err != nil {
return b.Next.ServeHTTP(w, r)
}
if !info.IsDir() {
return b.Next.ServeHTTP(w, r)
}
// See if there's a browse configuration to match the path
for _, bc := range b.Configs {
if !middleware.Path(r.URL.Path).Matches(bc.PathScope) {
continue
}
// Browsing navigation gets messed up if browsing a directory
// that doesn't end in "/" (which it should, anyway)
if r.URL.Path[len(r.URL.Path)-1] != '/' {
http.Redirect(w, r, r.URL.Path+"/", http.StatusTemporaryRedirect)
return 0, nil
}
// Load directory contents
file, err := os.Open(b.Root + r.URL.Path)
if err != nil {
return http.StatusForbidden, err
}
defer file.Close()
files, err := file.Readdir(-1)
if err != nil {
return http.StatusForbidden, err
}
// Assemble listing of directory contents
var fileinfos []FileInfo
var abort bool // we bail early if we find an index file
for _, f := range files {
name := f.Name()
// Directory is not browseable if it contains index file
for _, indexName := range IndexPages {
if name == indexName {
abort = true
break
}
}
if abort {
break
}
if f.IsDir() {
name += "/"
}
url := url.URL{Path: name}
fileinfos = append(fileinfos, FileInfo{
IsDir: f.IsDir(),
Name: f.Name(),
Size: f.Size(),
URL: url.String(),
ModTime: f.ModTime(),
Mode: f.Mode(),
})
}
if abort {
// this dir has an index file, so not browsable
continue
}
// Determine if user can browse up another folder
var canGoUp bool
curPath := strings.TrimSuffix(r.URL.Path, "/")
for _, other := range b.Configs {
if strings.HasPrefix(path.Dir(curPath), other.PathScope) {
canGoUp = true
break
}
}
listing := Listing{
Name: path.Base(r.URL.Path),
Path: r.URL.Path,
CanGoUp: canGoUp,
Items: fileinfos,
}
var buf bytes.Buffer
err = bc.Template.Execute(&buf, listing)
if err != nil {
return http.StatusInternalServerError, err
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
buf.WriteTo(w)
return http.StatusOK, nil
}
// Didn't qualify; pass-thru
return b.Next.ServeHTTP(w, r)
}
-27
View File
@@ -1,27 +0,0 @@
package middleware
import (
"errors"
"github.com/flynn/go-shlex"
)
// SplitCommandAndArgs takes a command string and parses it
// shell-style into the command and its separate arguments.
func SplitCommandAndArgs(command string) (cmd string, args []string, err error) {
parts, err := shlex.Split(command)
if err != nil {
err = errors.New("Error parsing command: " + err.Error())
return
} else if len(parts) == 0 {
err = errors.New("No command contained in '" + command + "'")
return
}
cmd = parts[0]
if len(parts) > 1 {
args = parts[1:]
}
return
}
-114
View File
@@ -1,114 +0,0 @@
// Package errors implements an HTTP error handling middleware.
package errors
import (
"fmt"
"io"
"log"
"net/http"
"os"
"runtime"
"strings"
"github.com/mholt/caddy/middleware"
)
// ErrorHandler handles HTTP errors (or errors from other middleware).
type ErrorHandler struct {
Next middleware.Handler
ErrorPages map[int]string // map of status code to filename
LogFile string
Log *log.Logger
}
func (h ErrorHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
defer h.recovery(w, r)
status, err := h.Next.ServeHTTP(w, r)
if err != nil {
h.Log.Printf("[ERROR %d %s] %v", status, r.URL.Path, err)
}
if status >= 400 {
h.errorPage(w, status)
return 0, err // status < 400 signals that a response has been written
}
return status, err
}
// errorPage serves a static error page to w according to the status
// code. If there is an error serving the error page, a plaintext error
// message is written instead, and the extra error is logged.
func (h ErrorHandler) errorPage(w http.ResponseWriter, code int) {
defaultBody := fmt.Sprintf("%d %s", code, http.StatusText(code))
// See if an error page for this status code was specified
if pagePath, ok := h.ErrorPages[code]; ok {
// Try to open it
errorPage, err := os.Open(pagePath)
if err != nil {
// An error handling an error... <insert grumpy cat here>
h.Log.Printf("HTTP %d could not load error page %s: %v", code, pagePath, err)
http.Error(w, defaultBody, code)
return
}
defer errorPage.Close()
// Copy the page body into the response
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(code)
_, err = io.Copy(w, errorPage)
if err != nil {
// Epic fail... sigh.
h.Log.Printf("HTTP %d could not respond with %s: %v", code, pagePath, err)
http.Error(w, defaultBody, code)
}
return
}
// Default error response
http.Error(w, defaultBody, code)
}
func (h ErrorHandler) recovery(w http.ResponseWriter, r *http.Request) {
rec := recover()
if rec == nil {
return
}
// Obtain source of panic
// From: https://gist.github.com/swdunlop/9629168
var name, file string // function name, file name
var line int
var pc [16]uintptr
n := runtime.Callers(3, pc[:])
for _, pc := range pc[:n] {
fn := runtime.FuncForPC(pc)
if fn == nil {
continue
}
file, line = fn.FileLine(pc)
name = fn.Name()
if !strings.HasPrefix(name, "runtime.") {
break
}
}
// Trim file path
delim := "/caddy/"
pkgPathPos := strings.Index(file, delim)
if pkgPathPos > -1 && len(file) > pkgPathPos+len(delim) {
file = file[pkgPathPos+len(delim):]
}
// Currently we don't use the function name, as file:line is more conventional
h.Log.Printf("[PANIC %s] %s:%d - %v", r.URL.String(), file, line, rec)
h.errorPage(w, http.StatusInternalServerError)
}
const DefaultLogFilename = "error.log"
-52
View File
@@ -1,52 +0,0 @@
// Package extension is middleware for clean URLs.
//
// The root path of the site is passed in as well as possible extensions
// to try internally for paths requested that don't match an existing
// resource. The first path+ext combination that matches a valid file
// will be used.
package extensions
import (
"net/http"
"os"
"path"
"strings"
"github.com/mholt/caddy/middleware"
)
// Ext can assume an extension from clean URLs.
// It tries extensions in the order listed in Extensions.
type Ext struct {
// Next handler in the chain
Next middleware.Handler
// Path to ther root of the site
Root string
// List of extensions to try
Extensions []string
}
// ServeHTTP implements the middleware.Handler interface.
func (e Ext) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
urlpath := strings.TrimSuffix(r.URL.Path, "/")
if path.Ext(urlpath) == "" {
for _, ext := range e.Extensions {
if resourceExists(e.Root, urlpath+ext) {
r.URL.Path = urlpath + ext
break
}
}
}
return e.Next.ServeHTTP(w, r)
}
// resourceExists returns true if the file specified at
// root + path exists; false otherwise.
func resourceExists(root, path string) bool {
_, err := os.Stat(root + path)
// technically we should use os.IsNotExist(err)
// but we don't handle any other kinds of errors anyway
return err == nil
}
-228
View File
@@ -1,228 +0,0 @@
// Package fastcgi has middleware that acts as a FastCGI client. Requests
// that get forwarded to FastCGI stop the middleware execution chain.
// The most common use for this package is to serve PHP websites via php-fpm.
package fastcgi
import (
"io"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/mholt/caddy/middleware"
)
// Handler is a middleware type that can handle requests as a FastCGI client.
type Handler struct {
Next middleware.Handler
Rules []Rule
Root string
AbsRoot string // same as root, but absolute path
FileSys http.FileSystem
// These are sent to CGI scripts in env variables
SoftwareName string
SoftwareVersion string
ServerName string
ServerPort string
}
// ServeHTTP satisfies the middleware.Handler interface.
func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
for _, rule := range h.Rules {
// First requirement: Base path must match
if !middleware.Path(r.URL.Path).Matches(rule.Path) {
continue
}
// In addition to matching the path, a request must meet some
// other criteria before being proxied as FastCGI. For example,
// we probably want to exclude static assets (CSS, JS, images...)
// but we also want to be flexible for the script we proxy to.
fpath := r.URL.Path
if idx, ok := middleware.IndexFile(h.FileSys, fpath, rule.IndexFiles); ok {
fpath = idx
}
// These criteria work well in this order for PHP sites
if fpath[len(fpath)-1] == '/' || strings.HasSuffix(fpath, rule.Ext) || !h.exists(fpath) {
// Create environment for CGI script
env, err := h.buildEnv(r, rule, fpath)
if err != nil {
return http.StatusInternalServerError, err
}
// Connect to FastCGI gateway
fcgi, err := Dial("tcp", rule.Address)
if err != nil {
return http.StatusBadGateway, err
}
var resp *http.Response
contentLength, _ := strconv.Atoi(r.Header.Get("Content-Length"))
switch r.Method {
case "HEAD":
resp, err = fcgi.Head(env)
case "GET":
resp, err = fcgi.Get(env)
case "OPTIONS":
resp, err = fcgi.Options(env)
case "POST":
resp, err = fcgi.Post(env, r.Header.Get("Content-Type"), r.Body, contentLength)
case "PUT":
resp, err = fcgi.Put(env, r.Header.Get("Content-Type"), r.Body, contentLength)
case "PATCH":
resp, err = fcgi.Patch(env, r.Header.Get("Content-Type"), r.Body, contentLength)
case "DELETE":
resp, err = fcgi.Delete(env, r.Header.Get("Content-Type"), r.Body, contentLength)
default:
return http.StatusMethodNotAllowed, nil
}
defer resp.Body.Close()
if err != nil && err != io.EOF {
return http.StatusBadGateway, err
}
// Write the response header
for key, vals := range resp.Header {
for _, val := range vals {
w.Header().Add(key, val)
}
}
w.WriteHeader(resp.StatusCode)
// Write the response body
// TODO: If this has an error, the response will already be
// partly written. We should copy out of resp.Body into a buffer
// first, then write it to the response...
_, err = io.Copy(w, resp.Body)
if err != nil {
return http.StatusBadGateway, err
}
return 0, nil
}
}
return h.Next.ServeHTTP(w, r)
}
func (h Handler) exists(path string) bool {
if _, err := os.Stat(h.Root + path); err == nil {
return true
}
return false
}
// buildEnv returns a set of CGI environment variables for the request.
func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]string, error) {
var env map[string]string
// Get absolute path of requested resource
absPath := filepath.Join(h.AbsRoot, fpath)
// Separate remote IP and port; more lenient than net.SplitHostPort
var ip, port string
if idx := strings.Index(r.RemoteAddr, ":"); idx > -1 {
ip = r.RemoteAddr[:idx]
port = r.RemoteAddr[idx+1:]
} else {
ip = r.RemoteAddr
}
// Split path in preparation for env variables
splitPos := strings.Index(fpath, rule.SplitPath)
var docURI, scriptName, scriptFilename, pathInfo string
if splitPos == -1 {
// Request doesn't have the extension, so assume index file in root
docURI = "/" + rule.IndexFiles[0]
scriptName = "/" + rule.IndexFiles[0]
scriptFilename = filepath.Join(h.AbsRoot, rule.IndexFiles[0])
pathInfo = fpath
} else {
// Request has the extension; path was split successfully
docURI = fpath[:splitPos+len(rule.SplitPath)]
pathInfo = fpath[splitPos+len(rule.SplitPath):]
scriptName = fpath
scriptFilename = absPath
}
// Some variables are unused but cleared explicitly to prevent
// the parent environment from interfering.
env = map[string]string{
// Variables defined in CGI 1.1 spec
"AUTH_TYPE": "", // Not used
"CONTENT_LENGTH": r.Header.Get("Content-Length"),
"CONTENT_TYPE": r.Header.Get("Content-Type"),
"GATEWAY_INTERFACE": "CGI/1.1",
"PATH_INFO": pathInfo,
"PATH_TRANSLATED": filepath.Join(h.AbsRoot, pathInfo), // Info: http://www.oreilly.com/openbook/cgi/ch02_04.html
"QUERY_STRING": r.URL.RawQuery,
"REMOTE_ADDR": ip,
"REMOTE_HOST": ip, // For speed, remote host lookups disabled
"REMOTE_PORT": port,
"REMOTE_IDENT": "", // Not used
"REMOTE_USER": "", // Not used
"REQUEST_METHOD": r.Method,
"SERVER_NAME": h.ServerName,
"SERVER_PORT": h.ServerPort,
"SERVER_PROTOCOL": r.Proto,
"SERVER_SOFTWARE": h.SoftwareName + "/" + h.SoftwareVersion,
// Other variables
"DOCUMENT_ROOT": h.AbsRoot,
"DOCUMENT_URI": docURI,
"HTTP_HOST": r.Host, // added here, since not always part of headers
"REQUEST_URI": r.URL.RequestURI(),
"SCRIPT_FILENAME": scriptFilename,
"SCRIPT_NAME": scriptName,
}
// Add env variables from config
for _, envVar := range rule.EnvVars {
env[envVar[0]] = envVar[1]
}
// Add all HTTP headers to env variables
for field, val := range r.Header {
header := strings.ToUpper(field)
header = headerNameReplacer.Replace(header)
env["HTTP_"+header] = strings.Join(val, ", ")
}
return env, nil
}
// Rule represents a FastCGI handling rule.
type Rule struct {
// The base path to match. Required.
Path string
// The address of the FastCGI server. Required.
Address string
// Always process files with this extension with fastcgi.
Ext string
// The path in the URL will be split into two, with the first piece ending
// with the value of SplitPath. The first piece will be assumed as the
// actual resource (CGI script) name, and the second piece will be set to
// PATH_INFO for the CGI script to use.
SplitPath string
// If the URL ends with '/' (which indicates a directory), these index
// files will be tried instead.
IndexFiles []string
// Environment Variables
EnvVars [][2]string
}
var headerNameReplacer = strings.NewReplacer(" ", "_", "-", "_")
-79
View File
@@ -1,79 +0,0 @@
<?php
ini_set("display_errors",1);
echo "resp: start\n";//.print_r($GLOBALS,1)."\n".print_r($_SERVER,1)."\n";
//echo print_r($_SERVER,1)."\n";
$length = 0;
$stat = "PASSED";
$ret = "[";
if (count($_POST) || count($_FILES)) {
foreach($_POST as $key => $val) {
$md5 = md5($val);
if ($key != $md5) {
$stat = "FAILED";
echo "server:err ".$md5." != ".$key."\n";
}
$length += strlen($key) + strlen($val);
$ret .= $key."(".strlen($key).") ";
}
$ret .= "] [";
foreach ($_FILES as $k0 => $val) {
$error = $val["error"];
if ($error == UPLOAD_ERR_OK) {
$tmp_name = $val["tmp_name"];
$name = $val["name"];
$datafile = "/tmp/test.go";
move_uploaded_file($tmp_name, $datafile);
$md5 = md5_file($datafile);
if ($k0 != $md5) {
$stat = "FAILED";
echo "server:err ".$md5." != ".$key."\n";
}
$length += strlen($k0) + filesize($datafile);
unlink($datafile);
$ret .= $k0."(".strlen($k0).") ";
}
else{
$stat = "FAILED";
echo "server:file err ".file_upload_error_message($error)."\n";
}
}
$ret .= "]";
echo "server:got data length " .$length."\n";
}
echo "-{$stat}-POST(".count($_POST).") FILE(".count($_FILES).")\n";
function file_upload_error_message($error_code) {
switch ($error_code) {
case UPLOAD_ERR_INI_SIZE:
return 'The uploaded file exceeds the upload_max_filesize directive in php.ini';
case UPLOAD_ERR_FORM_SIZE:
return 'The uploaded file exceeds the MAX_FILE_SIZE directive that was specified in the HTML form';
case UPLOAD_ERR_PARTIAL:
return 'The uploaded file was only partially uploaded';
case UPLOAD_ERR_NO_FILE:
return 'No file was uploaded';
case UPLOAD_ERR_NO_TMP_DIR:
return 'Missing a temporary folder';
case UPLOAD_ERR_CANT_WRITE:
return 'Failed to write file to disk';
case UPLOAD_ERR_EXTENSION:
return 'File upload stopped by extension';
default:
return 'Unknown upload error';
}
}
-516
View File
@@ -1,516 +0,0 @@
// Forked Jan. 2015 from http://bitbucket.org/PinIdea/fcgi_client
// (which is forked from https://code.google.com/p/go-fastcgi-client/)
// This fork contains several fixes and improvements by Matt Holt and
// other contributors to this project.
// Copyright 2012 Junqing Tan <ivan@mysqlab.net> and The Go Authors
// Use of this source code is governed by a BSD-style
// Part of source code is from Go fcgi package
package fastcgi
import (
"bufio"
"bytes"
"encoding/binary"
"errors"
"io"
"io/ioutil"
"mime/multipart"
"net"
"net/http"
"net/http/httputil"
"net/textproto"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
)
const FCGI_LISTENSOCK_FILENO uint8 = 0
const FCGI_HEADER_LEN uint8 = 8
const VERSION_1 uint8 = 1
const FCGI_NULL_REQUEST_ID uint8 = 0
const FCGI_KEEP_CONN uint8 = 1
const doubleCRLF = "\r\n\r\n"
const (
FCGI_BEGIN_REQUEST uint8 = iota + 1
FCGI_ABORT_REQUEST
FCGI_END_REQUEST
FCGI_PARAMS
FCGI_STDIN
FCGI_STDOUT
FCGI_STDERR
FCGI_DATA
FCGI_GET_VALUES
FCGI_GET_VALUES_RESULT
FCGI_UNKNOWN_TYPE
FCGI_MAXTYPE = FCGI_UNKNOWN_TYPE
)
const (
FCGI_RESPONDER uint8 = iota + 1
FCGI_AUTHORIZER
FCGI_FILTER
)
const (
FCGI_REQUEST_COMPLETE uint8 = iota
FCGI_CANT_MPX_CONN
FCGI_OVERLOADED
FCGI_UNKNOWN_ROLE
)
const (
FCGI_MAX_CONNS string = "MAX_CONNS"
FCGI_MAX_REQS string = "MAX_REQS"
FCGI_MPXS_CONNS string = "MPXS_CONNS"
)
const (
maxWrite = 65500 // 65530 may work, but for compatibility
maxPad = 255
)
type header struct {
Version uint8
Type uint8
Id uint16
ContentLength uint16
PaddingLength uint8
Reserved uint8
}
// for padding so we don't have to allocate all the time
// not synchronized because we don't care what the contents are
var pad [maxPad]byte
func (h *header) init(recType uint8, reqId uint16, contentLength int) {
h.Version = 1
h.Type = recType
h.Id = reqId
h.ContentLength = uint16(contentLength)
h.PaddingLength = uint8(-contentLength & 7)
}
type record struct {
h header
rbuf []byte
}
func (rec *record) read(r io.Reader) (buf []byte, err error) {
if err = binary.Read(r, binary.BigEndian, &rec.h); err != nil {
return
}
if rec.h.Version != 1 {
err = errors.New("fcgi: invalid header version")
return
}
if rec.h.Type == FCGI_END_REQUEST {
err = io.EOF
return
}
n := int(rec.h.ContentLength) + int(rec.h.PaddingLength)
if len(rec.rbuf) < n {
rec.rbuf = make([]byte, n)
}
if n, err = io.ReadFull(r, rec.rbuf[:n]); err != nil {
return
}
buf = rec.rbuf[:int(rec.h.ContentLength)]
return
}
type FCGIClient struct {
mutex sync.Mutex
rwc io.ReadWriteCloser
h header
buf bytes.Buffer
keepAlive bool
reqId uint16
}
// Connects to the fcgi responder at the specified network address.
// See func net.Dial for a description of the network and address parameters.
func Dial(network, address string) (fcgi *FCGIClient, err error) {
var conn net.Conn
conn, err = net.Dial(network, address)
if err != nil {
return
}
fcgi = &FCGIClient{
rwc: conn,
keepAlive: false,
reqId: 1,
}
return
}
// Close fcgi connnection
func (this *FCGIClient) Close() {
this.rwc.Close()
}
func (this *FCGIClient) writeRecord(recType uint8, content []byte) (err error) {
this.mutex.Lock()
defer this.mutex.Unlock()
this.buf.Reset()
this.h.init(recType, this.reqId, len(content))
if err := binary.Write(&this.buf, binary.BigEndian, this.h); err != nil {
return err
}
if _, err := this.buf.Write(content); err != nil {
return err
}
if _, err := this.buf.Write(pad[:this.h.PaddingLength]); err != nil {
return err
}
_, err = this.rwc.Write(this.buf.Bytes())
return err
}
func (this *FCGIClient) writeBeginRequest(role uint16, flags uint8) error {
b := [8]byte{byte(role >> 8), byte(role), flags}
return this.writeRecord(FCGI_BEGIN_REQUEST, b[:])
}
func (this *FCGIClient) writeEndRequest(appStatus int, protocolStatus uint8) error {
b := make([]byte, 8)
binary.BigEndian.PutUint32(b, uint32(appStatus))
b[4] = protocolStatus
return this.writeRecord(FCGI_END_REQUEST, b)
}
func (this *FCGIClient) writePairs(recType uint8, pairs map[string]string) error {
w := newWriter(this, recType)
b := make([]byte, 8)
nn := 0
for k, v := range pairs {
m := 8 + len(k) + len(v)
if m > maxWrite {
// param data size exceed 65535 bytes"
vl := maxWrite - 8 - len(k)
v = v[:vl]
}
n := encodeSize(b, uint32(len(k)))
n += encodeSize(b[n:], uint32(len(v)))
m = n + len(k) + len(v)
if (nn + m) > maxWrite {
w.Flush()
nn = 0
}
nn += m
if _, err := w.Write(b[:n]); err != nil {
return err
}
if _, err := w.WriteString(k); err != nil {
return err
}
if _, err := w.WriteString(v); err != nil {
return err
}
}
w.Close()
return nil
}
func readSize(s []byte) (uint32, int) {
if len(s) == 0 {
return 0, 0
}
size, n := uint32(s[0]), 1
if size&(1<<7) != 0 {
if len(s) < 4 {
return 0, 0
}
n = 4
size = binary.BigEndian.Uint32(s)
size &^= 1 << 31
}
return size, n
}
func readString(s []byte, size uint32) string {
if size > uint32(len(s)) {
return ""
}
return string(s[:size])
}
func encodeSize(b []byte, size uint32) int {
if size > 127 {
size |= 1 << 31
binary.BigEndian.PutUint32(b, size)
return 4
}
b[0] = byte(size)
return 1
}
// bufWriter encapsulates bufio.Writer but also closes the underlying stream when
// Closed.
type bufWriter struct {
closer io.Closer
*bufio.Writer
}
func (w *bufWriter) Close() error {
if err := w.Writer.Flush(); err != nil {
w.closer.Close()
return err
}
return w.closer.Close()
}
func newWriter(c *FCGIClient, recType uint8) *bufWriter {
s := &streamWriter{c: c, recType: recType}
w := bufio.NewWriterSize(s, maxWrite)
return &bufWriter{s, w}
}
// streamWriter abstracts out the separation of a stream into discrete records.
// It only writes maxWrite bytes at a time.
type streamWriter struct {
c *FCGIClient
recType uint8
}
func (w *streamWriter) Write(p []byte) (int, error) {
nn := 0
for len(p) > 0 {
n := len(p)
if n > maxWrite {
n = maxWrite
}
if err := w.c.writeRecord(w.recType, p[:n]); err != nil {
return nn, err
}
nn += n
p = p[n:]
}
return nn, nil
}
func (w *streamWriter) Close() error {
// send empty record to close the stream
return w.c.writeRecord(w.recType, nil)
}
type streamReader struct {
c *FCGIClient
buf []byte
}
func (w *streamReader) Read(p []byte) (n int, err error) {
if len(p) > 0 {
if len(w.buf) == 0 {
rec := &record{}
w.buf, err = rec.read(w.c.rwc)
if err != nil {
return
}
}
n = len(p)
if n > len(w.buf) {
n = len(w.buf)
}
copy(p, w.buf[:n])
w.buf = w.buf[n:]
}
return
}
// Do made the request and returns a io.Reader that translates the data read
// from fcgi responder out of fcgi packet before returning it.
func (this *FCGIClient) Do(p map[string]string, req io.Reader) (r io.Reader, err error) {
err = this.writeBeginRequest(uint16(FCGI_RESPONDER), 0)
if err != nil {
return
}
err = this.writePairs(FCGI_PARAMS, p)
if err != nil {
return
}
body := newWriter(this, FCGI_STDIN)
if req != nil {
io.Copy(body, req)
}
body.Close()
r = &streamReader{c: this}
return
}
// Request returns a HTTP Response with Header and Body
// from fcgi responder
func (this *FCGIClient) Request(p map[string]string, req io.Reader) (resp *http.Response, err error) {
r, err := this.Do(p, req)
if err != nil {
return
}
rb := bufio.NewReader(r)
tp := textproto.NewReader(rb)
resp = new(http.Response)
// Parse the response headers.
mimeHeader, err := tp.ReadMIMEHeader()
if err != nil && err != io.EOF {
return
}
resp.Header = http.Header(mimeHeader)
if resp.Header.Get("Status") != "" {
statusParts := strings.SplitN(resp.Header.Get("Status"), " ", 2)
resp.StatusCode, err = strconv.Atoi(statusParts[0])
if err != nil {
return
}
resp.Status = statusParts[1]
} else {
resp.StatusCode = http.StatusOK
}
// TODO: fixTransferEncoding ?
resp.TransferEncoding = resp.Header["Transfer-Encoding"]
resp.ContentLength, _ = strconv.ParseInt(resp.Header.Get("Content-Length"), 10, 64)
if chunked(resp.TransferEncoding) {
resp.Body = ioutil.NopCloser(httputil.NewChunkedReader(rb))
} else {
resp.Body = ioutil.NopCloser(rb)
}
return
}
// Get issues a GET request to the fcgi responder.
func (this *FCGIClient) Get(p map[string]string) (resp *http.Response, err error) {
p["REQUEST_METHOD"] = "GET"
p["CONTENT_LENGTH"] = "0"
return this.Request(p, nil)
}
// Head issues a HEAD request to the fcgi responder.
func (this *FCGIClient) Head(p map[string]string) (resp *http.Response, err error) {
p["REQUEST_METHOD"] = "HEAD"
p["CONTENT_LENGTH"] = "0"
return this.Request(p, nil)
}
// Options issues an OPTIONS request to the fcgi responder.
func (this *FCGIClient) Options(p map[string]string) (resp *http.Response, err error) {
p["REQUEST_METHOD"] = "OPTIONS"
p["CONTENT_LENGTH"] = "0"
return this.Request(p, nil)
}
// Post issues a POST request to the fcgi responder. with request body
// in the format that bodyType specified
func (this *FCGIClient) Post(p map[string]string, bodyType string, body io.Reader, l int) (resp *http.Response, err error) {
if len(p["REQUEST_METHOD"]) == 0 || p["REQUEST_METHOD"] == "GET" {
p["REQUEST_METHOD"] = "POST"
}
p["CONTENT_LENGTH"] = strconv.Itoa(l)
if len(bodyType) > 0 {
p["CONTENT_TYPE"] = bodyType
} else {
p["CONTENT_TYPE"] = "application/x-www-form-urlencoded"
}
return this.Request(p, body)
}
// Put issues a PUT request to the fcgi responder.
func (this *FCGIClient) Put(p map[string]string, bodyType string, body io.Reader, l int) (resp *http.Response, err error) {
p["REQUEST_METHOD"] = "PUT"
return this.Post(p, bodyType, body, l)
}
// Patch issues a PATCH request to the fcgi responder.
func (this *FCGIClient) Patch(p map[string]string, bodyType string, body io.Reader, l int) (resp *http.Response, err error) {
p["REQUEST_METHOD"] = "PATCH"
return this.Post(p, bodyType, body, l)
}
// Delete issues a DELETE request to the fcgi responder.
func (this *FCGIClient) Delete(p map[string]string, bodyType string, body io.Reader, l int) (resp *http.Response, err error) {
p["REQUEST_METHOD"] = "DELETE"
return this.Post(p, bodyType, body, l)
}
// PostForm issues a POST to the fcgi responder, with form
// as a string key to a list values (url.Values)
func (this *FCGIClient) PostForm(p map[string]string, data url.Values) (resp *http.Response, err error) {
body := bytes.NewReader([]byte(data.Encode()))
return this.Post(p, "application/x-www-form-urlencoded", body, body.Len())
}
// PostFile issues a POST to the fcgi responder in multipart(RFC 2046) standard,
// with form as a string key to a list values (url.Values),
// and/or with file as a string key to a list file path.
func (this *FCGIClient) PostFile(p map[string]string, data url.Values, file map[string]string) (resp *http.Response, err error) {
buf := &bytes.Buffer{}
writer := multipart.NewWriter(buf)
bodyType := writer.FormDataContentType()
for key, val := range data {
for _, v0 := range val {
err = writer.WriteField(key, v0)
if err != nil {
return
}
}
}
for key, val := range file {
fd, e := os.Open(val)
if e != nil {
return nil, e
}
defer fd.Close()
part, e := writer.CreateFormFile(key, filepath.Base(val))
if e != nil {
return nil, e
}
_, err = io.Copy(part, fd)
}
err = writer.Close()
if err != nil {
return
}
return this.Post(p, bodyType, buf, buf.Len())
}
// Checks whether chunked is part of the encodings stack
func chunked(te []string) bool { return len(te) > 0 && te[0] == "chunked" }
-71
View File
@@ -1,71 +0,0 @@
// Package git is the middleware that pull sites from git repo
//
// Caddyfile Syntax :
// git repo path {
// repo
// path
// branch
// key
// interval
// then command args
// }
// repo - git repository
// compulsory. Both ssh (e.g. git@github.com:user/project.git)
// and https(e.g. https://github.com/user/project) are supported.
// Can be specified in either config block or top level
//
// path - directory to pull into, relative to site root
// optional. Defaults to site root.
//
// branch - git branch or tag
// optional. Defaults to master
//
// key - path to private ssh key
// optional. Required for private repositories. e.g. /home/user/.ssh/id_rsa
//
// interval- interval between git pulls in seconds
// optional. Defaults to 3600 (1 Hour).
//
// then - command to execute after successful pull
// optional. If set, will execute only when there are new changes.
//
// Examples :
//
// public repo pulled into site root
// git github.com/user/myproject
//
// public repo pulled into <root>/mysite
// git https://github.com/user/myproject mysite
//
// private repo pulled into <root>/mysite with tag v1.0 and interval of 1 day.
// git {
// repo git@github.com:user/myproject
// branch v1.0
// path mysite
// key /home/user/.ssh/id_rsa
// interval 86400 # 1 day
// }
//
// Caddyfile with private git repo and php support via fastcgi.
// path defaults to /var/www/html/myphpsite as specified in root config.
//
// 0.0.0.0:8080
//
// git {
// repo git@github.com:user/myphpsite
// key /home/user/.ssh/id_rsa
// interval 86400 # 1 day
// }
//
// fastcgi / 127.0.0.1:9000 php
//
// root /var/www/html/myphpsite
//
// A pull is first attempted after initialization. Afterwards, a pull is attempted
// after request to server and if time taken since last successful pull is higher than interval.
//
// After the first successful pull (should be during initialization except an error occurs),
// subsequent pulls are done in background and do not impact request time.
//
// Note: private repositories are currently only supported and tested on Linux and OSX
package git
-339
View File
@@ -1,339 +0,0 @@
package git
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"strings"
"sync"
"time"
"github.com/mholt/caddy/middleware"
)
// DefaultInterval is the minimum interval to delay before
// requesting another git pull
const DefaultInterval time.Duration = time.Hour * 1
// Number of retries if git pull fails
const numRetries = 3
// gitBinary holds the absolute path to git executable
var gitBinary string
// initMutex prevents parallel attempt to validate
// git availability in PATH
var initMutex sync.Mutex = sync.Mutex{}
// Logger is used to log errors; if nil, the default log.Logger is used.
var Logger *log.Logger
// logger is an helper function to retrieve the available logger
func logger() *log.Logger {
if Logger == nil {
Logger = log.New(os.Stderr, "", log.LstdFlags)
}
return Logger
}
// Repo is the structure that holds required information
// of a git repository.
type Repo struct {
Url string // Repository URL
Path string // Directory to pull to
Host string // Git domain host e.g. github.com
Branch string // Git branch
KeyPath string // Path to private ssh key
Interval time.Duration // Interval between pulls
Then string // Command to execute after successful git pull
pulled bool // true if there was a successful pull
lastPull time.Time // time of the last successful pull
lastCommit string // hash for the most recent commit
sync.Mutex
}
// Pull attempts a git clone.
// It retries at most numRetries times if error occurs
func (r *Repo) Pull() error {
r.Lock()
defer r.Unlock()
// if it is less than interval since last pull, return
if time.Since(r.lastPull) <= r.Interval {
return nil
}
// keep last commit hash for comparison later
lastCommit := r.lastCommit
var err error
// Attempt to pull at most numRetries times
for i := 0; i < numRetries; i++ {
if err = r.pull(); err == nil {
break
}
logger().Println(err)
}
if err != nil {
return err
}
// check if there are new changes,
// then execute post pull command
if r.lastCommit == lastCommit {
logger().Println("No new changes.")
return nil
}
return r.postPullCommand()
}
// Pull performs git clone, or git pull if repository exists
func (r *Repo) pull() error {
params := []string{"clone", "-b", r.Branch, r.Url, r.Path}
if r.pulled {
params = []string{"pull", "origin", r.Branch}
}
// if key is specified, pull using ssh key
if r.KeyPath != "" {
return r.pullWithKey(params)
}
dir := ""
if r.pulled {
dir = r.Path
}
var err error
if err = runCmd(gitBinary, params, dir); err == nil {
r.pulled = true
r.lastPull = time.Now()
logger().Printf("%v pulled.\n", r.Url)
r.lastCommit, err = r.getMostRecentCommit()
}
return err
}
// pullWithKey is used for private repositories and requires an ssh key.
// Note: currently only limited to Linux and OSX.
func (r *Repo) pullWithKey(params []string) error {
var gitSsh, script *os.File
// ensure temporary files deleted after usage
defer func() {
if gitSsh != nil {
os.Remove(gitSsh.Name())
}
if script != nil {
os.Remove(script.Name())
}
}()
var err error
// write git.sh script to temp file
gitSsh, err = writeScriptFile(gitWrapperScript(gitBinary))
if err != nil {
return err
}
// write git clone bash script to file
script, err = writeScriptFile(bashScript(gitSsh.Name(), r, params))
if err != nil {
return err
}
dir := ""
if r.pulled {
dir = r.Path
}
if err = runCmd(script.Name(), nil, dir); err == nil {
r.pulled = true
r.lastPull = time.Now()
logger().Printf("%v pulled.\n", r.Url)
r.lastCommit, err = r.getMostRecentCommit()
}
return err
}
// Prepare prepares for a git pull
// and validates the configured directory
func (r *Repo) Prepare() error {
// check if directory exists or is empty
// if not, create directory
fs, err := ioutil.ReadDir(r.Path)
if err != nil || len(fs) == 0 {
return os.MkdirAll(r.Path, os.FileMode(0755))
}
// validate git repo
isGit := false
for _, f := range fs {
if f.IsDir() && f.Name() == ".git" {
isGit = true
break
}
}
if isGit {
// check if same repository
var repoUrl string
if repoUrl, err = r.getRepoUrl(); err == nil && repoUrl == r.Url {
r.pulled = true
return nil
}
if err != nil {
return fmt.Errorf("Cannot retrieve repo url for %v Error: %v", r.Path, err)
}
return fmt.Errorf("Another git repo '%v' exists at %v", repoUrl, r.Path)
}
return fmt.Errorf("Cannot git clone into %v, directory not empty.", r.Path)
}
// getMostRecentCommit gets the hash of the most recent commit to the
// repository. Useful for checking if changes occur.
func (r *Repo) getMostRecentCommit() (string, error) {
command := gitBinary + ` --no-pager log -n 1 --pretty=format:"%H"`
c, args, err := middleware.SplitCommandAndArgs(command)
if err != nil {
return "", err
}
return runCmdOutput(c, args, r.Path)
}
// getRepoUrl retrieves remote origin url for the git repository at path
func (r *Repo) getRepoUrl() (string, error) {
_, err := os.Stat(r.Path)
if err != nil {
return "", err
}
args := []string{"config", "--get", "remote.origin.url"}
return runCmdOutput(gitBinary, args, r.Path)
}
// postPullCommand executes r.Then.
// It is trigged after successful git pull
func (r *Repo) postPullCommand() error {
if r.Then == "" {
return nil
}
c, args, err := middleware.SplitCommandAndArgs(r.Then)
if err != nil {
return err
}
if err = runCmd(c, args, r.Path); err == nil {
logger().Printf("Command %v successful.\n", r.Then)
}
return err
}
// InitGit validates git installation and locates the git executable
// binary in PATH
func InitGit() error {
// prevent concurrent call
initMutex.Lock()
defer initMutex.Unlock()
// if validation has been done before and binary located in
// PATH, return.
if gitBinary != "" {
return nil
}
// locate git binary in path
var err error
gitBinary, err = exec.LookPath("git")
return err
}
// runCmd is a helper function to run commands.
// It runs command with args from directory at dir.
// The executed process outputs to os.Stderr
func runCmd(command string, args []string, dir string) error {
cmd := exec.Command(command, args...)
cmd.Stderr = os.Stderr
cmd.Stdout = os.Stderr
cmd.Dir = dir
if err := cmd.Start(); err != nil {
return err
}
return cmd.Wait()
}
// runCmdOutput is a helper function to run commands and return output.
// It runs command with args from directory at dir.
// If successful, returns output and nil error
func runCmdOutput(command string, args []string, dir string) (string, error) {
cmd := exec.Command(command, args...)
cmd.Dir = dir
var err error
if output, err := cmd.Output(); err == nil {
return string(bytes.TrimSpace(output)), nil
}
return "", err
}
// writeScriptFile writes content to a temporary file.
// It changes the temporary file mode to executable and
// closes it to prepare it for execution.
func writeScriptFile(content []byte) (file *os.File, err error) {
if file, err = ioutil.TempFile("", "caddy"); err != nil {
return nil, err
}
if _, err = file.Write(content); err != nil {
return nil, err
}
if err = file.Chmod(os.FileMode(0755)); err != nil {
return nil, err
}
return file, file.Close()
}
// gitWrapperScript forms content for git.sh script
var gitWrapperScript = func(gitBinary string) []byte {
return []byte(fmt.Sprintf(`#!/bin/bash
# The MIT License (MIT)
# Copyright (c) 2013 Alvin Abad
if [ $# -eq 0 ]; then
echo "Git wrapper script that can specify an ssh-key file
Usage:
git.sh -i ssh-key-file git-command
"
exit 1
fi
# remove temporary file on exit
trap 'rm -f /tmp/.git_ssh.$$' 0
if [ "$1" = "-i" ]; then
SSH_KEY=$2; shift; shift
echo "ssh -i $SSH_KEY \$@" > /tmp/.git_ssh.$$
chmod +x /tmp/.git_ssh.$$
export GIT_SSH=/tmp/.git_ssh.$$
fi
# in case the git command is repeated
[ "$1" = "git" ] && shift
# Run the git command
%v "$@"
`, gitBinary))
}
// bashScript forms content of bash script to clone or update a repo using ssh
var bashScript = func(gitShPath string, repo *Repo, params []string) []byte {
return []byte(fmt.Sprintf(`#!/bin/bash
mkdir -p ~/.ssh;
touch ~/.ssh/known_hosts;
ssh-keyscan -t rsa,dsa %v 2>&1 | sort -u - ~/.ssh/known_hosts > ~/.ssh/tmp_hosts;
cat ~/.ssh/tmp_hosts >> ~/.ssh/known_hosts;
%v -i %v %v;
`, repo.Host, gitShPath, repo.KeyPath, strings.Join(params, " ")))
}
-76
View File
@@ -1,76 +0,0 @@
// Package gzip provides a simple middleware layer that performs
// gzip compression on the response.
package gzip
import (
"compress/gzip"
"fmt"
"io"
"net/http"
"strings"
"github.com/mholt/caddy/middleware"
)
// Gzip is a middleware type which gzips HTTP responses. It is
// imperative that any handler which writes to a gzipped response
// specifies the Content-Type, otherwise some clients will assume
// application/x-gzip and try to download a file.
type Gzip struct {
Next middleware.Handler
}
// ServeHTTP serves a gzipped response if the client supports it.
func (g Gzip) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
return g.Next.ServeHTTP(w, r)
}
// Delete this header so gzipping isn't repeated later in the chain
r.Header.Del("Accept-Encoding")
w.Header().Set("Content-Encoding", "gzip")
gzipWriter := gzip.NewWriter(w)
defer gzipWriter.Close()
gz := gzipResponseWriter{Writer: gzipWriter, ResponseWriter: w}
// Any response in forward middleware will now be compressed
status, err := g.Next.ServeHTTP(gz, r)
// If there was an error that remained unhandled, we need
// to send something back before gzipWriter gets closed at
// the return of this method!
if status >= 400 {
gz.Header().Set("Content-Type", "text/plain") // very necessary
gz.WriteHeader(status)
fmt.Fprintf(gz, "%d %s", status, http.StatusText(status))
return 0, err
} else {
return status, err
}
}
// gzipResponeWriter wraps the underlying Write method
// with a gzip.Writer to compress the output.
type gzipResponseWriter struct {
io.Writer
http.ResponseWriter
}
// WriteHeader wraps the underlying WriteHeader method to prevent
// problems with conflicting headers from proxied backends. For
// example, a backend system that calculates Content-Length would
// be wrong because it doesn't know it's being gzipped.
func (w gzipResponseWriter) WriteHeader(code int) {
w.Header().Del("Content-Length")
w.ResponseWriter.WriteHeader(code)
}
// Write wraps the underlying Write method to do compression.
func (w gzipResponseWriter) Write(b []byte) (int, error) {
if w.Header().Get("Content-Type") == "" {
w.Header().Set("Content-Type", http.DetectContentType(b))
}
n, err := w.Writer.Write(b)
return n, err
}
-45
View File
@@ -1,45 +0,0 @@
// Package headers provides middleware that appends headers to
// requests based on a set of configuration rules that define
// which routes receive which headers.
package headers
import (
"net/http"
"github.com/mholt/caddy/middleware"
)
// Headers is middleware that adds headers to the responses
// for requests matching a certain path.
type Headers struct {
Next middleware.Handler
Rules []Rule
}
// ServeHTTP implements the middleware.Handler interface and serves requests,
// adding headers to the response according to the configured rules.
func (h Headers) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
for _, rule := range h.Rules {
if middleware.Path(r.URL.Path).Matches(rule.Url) {
for _, header := range rule.Headers {
w.Header().Set(header.Name, header.Value)
}
}
}
return h.Next.ServeHTTP(w, r)
}
type (
// Rule groups a slice of HTTP headers by a URL pattern.
// TODO: use http.Header type instead?
Rule struct {
Url string
Headers []Header
}
// Header represents a single HTTP header, simply a name and value.
Header struct {
Name string
Value string
}
)
-41
View File
@@ -1,41 +0,0 @@
// Package log implements basic but useful request (access) logging middleware.
package log
import (
"log"
"net/http"
"github.com/mholt/caddy/middleware"
)
type Logger struct {
Next middleware.Handler
Rules []LogRule
}
func (l Logger) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
for _, rule := range l.Rules {
if middleware.Path(r.URL.Path).Matches(rule.PathScope) {
responseRecorder := middleware.NewResponseRecorder(w)
status, err := l.Next.ServeHTTP(responseRecorder, r)
rep := middleware.NewReplacer(r, responseRecorder)
rule.Log.Println(rep.Replace(rule.Format))
return status, err
}
}
return l.Next.ServeHTTP(w, r)
}
type LogRule struct {
PathScope string
OutputFile string
Format string
Log *log.Logger
}
const (
DefaultLogFilename = "access.log"
CommonLogFormat = `{remote} ` + middleware.EmptyStringReplacer + ` [{when}] "{method} {uri} {proto}" {status} {size}`
CombinedLogFormat = CommonLogFormat + ` "{>Referer}" "{>User-Agent}"`
DefaultLogFormat = CommonLogFormat
)
-138
View File
@@ -1,138 +0,0 @@
// Package markdown is middleware to render markdown files as HTML
// on-the-fly.
package markdown
import (
"bytes"
"io/ioutil"
"net/http"
"os"
"path"
"strings"
"github.com/mholt/caddy/middleware"
"github.com/russross/blackfriday"
)
// Markdown implements a layer of middleware that serves
// markdown as HTML.
type Markdown struct {
// Server root
Root string
// Jail the requests to site root with a mock file system
FileSys http.FileSystem
// Next HTTP handler in the chain
Next middleware.Handler
// The list of markdown configurations
Configs []Config
// The list of index files to try
IndexFiles []string
}
// Config stores markdown middleware configurations.
type Config struct {
// Markdown renderer
Renderer blackfriday.Renderer
// Base path to match
PathScope string
// List of extensions to consider as markdown files
Extensions []string
// List of style sheets to load for each markdown file
Styles []string
// List of JavaScript files to load for each markdown file
Scripts []string
}
// ServeHTTP implements the http.Handler interface.
func (md Markdown) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
for _, m := range md.Configs {
if !middleware.Path(r.URL.Path).Matches(m.PathScope) {
continue
}
fpath := r.URL.Path
if idx, ok := middleware.IndexFile(md.FileSys, fpath, md.IndexFiles); ok {
fpath = idx
}
for _, ext := range m.Extensions {
if strings.HasSuffix(fpath, ext) {
f, err := md.FileSys.Open(fpath)
if err != nil {
if os.IsPermission(err) {
return http.StatusForbidden, err
}
return http.StatusNotFound, nil
}
body, err := ioutil.ReadAll(f)
if err != nil {
return http.StatusInternalServerError, err
}
content := blackfriday.Markdown(body, m.Renderer, 0)
var scripts, styles string
for _, style := range m.Styles {
styles += strings.Replace(cssTemplate, "{{url}}", style, 1) + "\r\n"
}
for _, script := range m.Scripts {
scripts += strings.Replace(jsTemplate, "{{url}}", script, 1) + "\r\n"
}
// Title is first line (length-limited), otherwise filename
title := path.Base(fpath)
newline := bytes.Index(body, []byte("\n"))
if newline > -1 {
firstline := body[:newline]
newTitle := strings.TrimSpace(string(firstline))
if len(newTitle) > 1 {
if len(newTitle) > 128 {
title = newTitle[:128]
} else {
title = newTitle
}
}
}
html := htmlTemplate
html = strings.Replace(html, "{{title}}", title, 1)
html = strings.Replace(html, "{{css}}", styles, 1)
html = strings.Replace(html, "{{js}}", scripts, 1)
html = strings.Replace(html, "{{body}}", string(content), 1)
w.Write([]byte(html))
return http.StatusOK, nil
}
}
}
// Didn't qualify to serve as markdown; pass-thru
return md.Next.ServeHTTP(w, r)
}
const (
htmlTemplate = `<!DOCTYPE html>
<html>
<head>
<title>{{title}}</title>
<meta charset="utf-8">
{{css}}
{{js}}
</head>
<body>
{{body}}
</body>
</html>`
cssTemplate = `<link rel="stylesheet" href="{{url}}">`
jsTemplate = `<script src="{{url}}"></script>`
)
-73
View File
@@ -1,73 +0,0 @@
// Package middleware provides some types and functions common among middleware.
package middleware
import (
"net/http"
"path/filepath"
)
type (
// Middleware is the middle layer which represents the traditional
// idea of middleware: it chains one Handler to the next by being
// passed the next Handler in the chain.
Middleware func(Handler) Handler
// Handler is like http.Handler except ServeHTTP returns a status code
// and an error. The status code is for the client's benefit; the error
// value is for the server's benefit. The status code will be sent to
// the client while the error value will be logged privately. Sometimes,
// an error status code (4xx or 5xx) may be returned with a nil error
// when there is no reason to log the error on the server.
//
// If a HandlerFunc returns an error (status >= 400), it should NOT
// write to the response. This philosophy makes middleware.Handler
// different from http.Handler: error handling should happen at the
// application layer or in dedicated error-handling middleware only
// rather than with an "every middleware for itself" paradigm.
//
// The application or error-handling middleware should incorporate logic
// to ensure that the client always gets a proper response according to
// the status code. For security reasons, it should probably not reveal
// the actual error message. (Instead it should be logged, for example.)
//
// Handlers which do write to the response should return a status value
// < 400 as a signal that a response has been written. In other words,
// only error-handling middleware or the application will write to the
// response for a status code >= 400. When ANY handler writes to the
// response, it should return a status code < 400 to signal others to
// NOT write to the response again, which would be erroneous.
Handler interface {
ServeHTTP(http.ResponseWriter, *http.Request) (int, error)
}
// HandlerFunc is a convenience type like http.HandlerFunc, except
// ServeHTTP returns a status code and an error. See Handler
// documentation for more information.
HandlerFunc func(http.ResponseWriter, *http.Request) (int, error)
)
// ServeHTTP implements the Handler interface.
func (f HandlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
return f(w, r)
}
// IndexFile looks for a file in /root/fpath/indexFile for each string
// in indexFiles. If an index file is found, it returns the root-relative
// path to the file and true. If no index file is found, empty string
// and false is returned. fpath must end in a forward slash '/'
// otherwise no index files will be tried (directory paths must end
// in a forward slash according to HTTP).
func IndexFile(root http.FileSystem, fpath string, indexFiles []string) (string, bool) {
if fpath[len(fpath)-1] != '/' || root == nil {
return "", false
}
for _, indexFile := range indexFiles {
fp := filepath.Join(fpath, indexFile)
f, err := root.Open(fp)
if err == nil {
f.Close()
return fp, true
}
}
return "", false
}

Some files were not shown because too many files have changed in this diff Show More