Commit Graph

256 Commits

Author SHA1 Message Date
Matthew Holt d1171af679 httpserver: Don't obtain certs for unmanaged configs (fixes #2387) 2018-12-11 19:37:08 -07:00
Matthew Holt 33f2b16a1b Merge branch 'certmagic' 2018-12-10 20:08:55 -07:00
Matthew Holt f03ad80701 Update tests after large refactor 2018-12-10 20:08:29 -07:00
Matthew Holt e0f1a02c37 Extract most of caddytls core code into external CertMagic package
All code relating to a caddytls.Config and setting it up from the
Caddyfile is still intact; only the certificate management-related
code was removed into a separate package.

I don't expect this to build in CI successfully; updating dependencies
and vendor is coming next.

I've also removed the ad-hoc, half-baked storage plugins that we need
to finish making first-class Caddy plugins (they were never documented
anyway). The new certmagic package has a much better storage interface,
and we can finally move toward making a new storage plugin type, but
it shouldn't be configurable in the Caddyfile, I think, since it doesn't
make sense for a Caddy instance to use more than one storage config...

We also have the option of eliminating DNS provider plugins and just
shipping all of lego's DNS providers by using a lego package (the
caddytls/setup.go file has a comment describing how) -- but it doubles
Caddy's binary size by 100% from about 19 MB to around 40 MB...!
2018-12-10 19:49:29 -07:00
Francis Lavoie 1533652b78 Fix s3browser plugin name
Thanks to @webprofusion-chrisc for spotting this: https://github.com/mholt/caddy/pull/2383#issuecomment-445432256
2018-12-08 03:55:34 -05:00
techknowlogick c7562e46a4 httpserver: Add s3browser directive (#2383)
Fixes https://github.com/techknowlogick/caddy-s3browser/issues/2
2018-12-07 19:17:36 -07:00
Matt Holt 09188981c4 tls: Add support for the tls-alpn-01 challenge (#2201)
* tls: Add support for the tls-alpn-01 challenge

Also updates lego/acme to latest on master.

TODO: This implementation of the tls-alpn challenge is not yet solvable
in a distributed Caddy cluster like the http challenge is.

* build: Allow building with the race detector

* tls: Support distributed solving of the TLS-ALPN-01 challenge

* Update vendor and add a todo in MITM checker
2018-12-05 17:33:23 -07:00
Matthew Holt b7091650f8 Revert "bind: support multiple values (#2128)"
This reverts commit 3a810c6502.
2018-11-27 15:57:38 -07:00
zhsj 3a810c6502 bind: support multiple values (#2128)
Signed-off-by: Shengjing Zhu <i@zhsj.me>
2018-11-26 18:27:58 -07:00
Kris Kwiatkowski d3e3fc533f templates: TLSVersion (#2323)
* new template action: TLS protocol version

* new template action: use caddytls.GetSupportedProtocolName

Avoids code duplication by reusing existing method to get TLS protocol
version used on connection. Also adds tests
2018-10-19 11:51:10 -06:00
Makeev Ivan 9edc16e4d6 Adding {when_unix_ms} requests placeholder (unix timestamp with a milliseconds precision) (#2260) 2018-08-28 11:08:55 +01:00
Henrique Dias a457b35750 httpserver: update minify ordering (#2273) 2018-08-22 09:19:37 -06:00
cmulk 16722e4d99 Update paths for filebrowser plugins 2018-08-21 10:30:42 -05:00
Jiri Tyr 9239f3cbcc Adding TLS client cert placeholders (#2217)
* Adding TLS client cert placeholders

* Use function to get the peer certificate

* Changing SHA1 to SHA256

* Use UTC instead of GMT

* Adding tests

* Adding getters for Protocol and Cipher
2018-07-28 09:26:24 +01:00
Henrique Dias 06b067b02c caddyhttp: correct import path for filemanager plugins (#2211) 2018-07-26 14:15:18 -06:00
emersion be84b74d01 httpserver: Register 'wkd' (Web Key Directory) directive (#2239) 2018-07-26 13:50:18 -06:00
Matthew Holt 9160789b42 telemetry: Make http_user_agent a normalized field
This way we store a short 8-byte hash of the UA instead of the full
string; exactly the same way we store TLS ClientHello info.
2018-05-10 08:57:25 -06:00
Matthew Holt fe03c1aefa telemetry: Fix MITM tests 2018-05-07 16:42:35 -06:00
Matthew Holt 078770a5a6 telemetry: Record TLS ClientHellos by hash of key of structured data
Also improve handling of disabled metrics, and record TLS ClientHello
in association with User-Agent
2018-05-07 16:09:39 -06:00
Matthew Holt b019501b8b Merge branch 'master' into telemetry
# Conflicts:
#	caddy/caddymain/run.go
#	caddyhttp/httpserver/plugin.go
#	caddytls/client.go
2018-04-20 00:03:57 -06:00
Matthew Holt 917a604094 httpserver: Ignore ErrServerClosed when closing server 2018-04-02 08:17:21 -06:00
Lucas Lorentz b33b24fc9e httpserver: Add 'supervisor' directive (#2061) 2018-03-31 17:31:35 -06:00
Matt Holt 4d9ee000c8 httpserver: Prevent TLS client authentication bypass in 3 ways (#2099)
- Introduce StrictHostMatching mode for sites that require clientauth
- Error if QUIC is enabled whilst TLS clientauth is configured
  (Our QUIC implementation does not yet support TLS clientauth, but
  maybe it will in the future - fixes #2095)
- Error if one but not all TLS configs for the same hostname have a
  different ClientAuth CA pool
2018-03-30 14:40:04 -06:00
Matthew Holt 2966db7b78 httpserver: Fix test that relies on external DNS lookup
Apparently Cloudflare just caused 1.1.1.1 to resolve, so we have to
change our test IP, hopefully this is better
2018-03-30 06:39:46 -06:00
Toby Allen 1896b420d8 log: 'except' subdirective to skip logging certain requests (#2028)
* proof of concept

* Initial implementation with debug code

* Tidy up debug code

* remove unneeded import

* removed extra line

* Move ShouldLog function to rule entry Logger type

* add tests for ShouldLog

* Added tests for log exceptions

* Fix logic

* fix govet fail for test

* Updates requested for code clarity

* Update requested for style

* log: Minor style tweaks to logic of log exceptions
2018-03-26 17:17:43 -06:00
Matt Holt 95514da91b Merge pull request #2072 from mholt/acmev2
tls: Use ACMEv2 and support automatic wildcard certificates
2018-03-25 22:09:03 -06:00
Matthew Holt 2ed1dd6afc Merge branch 'master' into acmev2
# Conflicts:
#	caddyhttp/httpserver/replacer.go
#	caddyhttp/httpserver/replacer_test.go
2018-03-25 21:56:11 -06:00
Denis a8dfa9f0b7 httpserver: CaseSensitivePath applied to paths in site keys (#2034)
* different cases in path make different keys

* Respect CaseSensitivePath variable when matching paths
2018-03-25 21:32:30 -06:00
Matthew Holt 52316952a5 Refactor diagnostics -> telemetry 2018-03-22 18:05:31 -06:00
Matthew Holt 4df8028bc3 diagnostics: Add/remove metrics 2018-03-21 17:01:14 -06:00
Matthew Fay f1eaae9b0d httpserver: Rework Replacer loop to ignore escaped braces (#2075)
* httpserver.Replacer: Rework loop to ignore escaped placeholder braces

* Fix typo and ineffectual assignment to ret

* Remove redundant idxOffset declaration, simplify escape check

* Add benchmark tests for new Replacer code
2018-03-18 20:42:43 -06:00
David Somers ca34a3e1aa httpserver: Placeholders for tls_protocol and tls_cipher (#2062)
Also add SSL_PROTOCOL and SSL_CIPHER env vars for fastcgi.

* Implement placeholders for ssl_protocol and ssl_cipher

* gofmt

* goimports

* Housekeeping and implement as {tls_protocol} and {tls_cipher}
2018-03-17 17:27:10 -06:00
Toby Allen 3ee6d30659 httpserver: Fix #2038 (query string being lost from URI) (#2039) 2018-03-17 17:17:42 -06:00
Matthew Holt 3afb1ae380 Merge branch 'master' into acmev2 2018-03-17 11:30:21 -06:00
Matthew Holt 37c852c382 tls: Add 'wildcard' subdirective to force wildcard certificate
Should only be used when many sites are defined in the Caddyfile, and
you would run up against Let's Encrypt rate limits without a wildcard.
2018-03-17 11:29:19 -06:00
Andrey Blinov 64c9f20919 httpserver: Add geoip directive (closes #1819) (#2066)
* Add Geoip plugin to httpserver/plugin.go

* Move GeoIP plugin higher
2018-03-15 07:30:25 -06:00
Matthew Holt d10d8c23c4 httpserver: Add a couple test cases for the Replacer on {labelN} 2018-03-14 22:11:13 -06:00
Matthew Holt aaec7e469c httpserver: Add {labelN} placeholders for parts of hostnames
For example, {label1} would match "sub" in "sub.example.com" or whatever
value is in the wildcard spot of "*.example.com". Useful for rewrite!
2018-03-14 21:57:25 -06:00
Toby Allen 3b66865da5 httpserver: Placeholder for response header fields (#2029)
* Allow Response Headers in logs

* Remove log line

* remove unneeded log import

* Check if rr is nil.  Added test to check

* merge if statements

* remove temp file
2018-02-18 14:21:06 -07:00
Matthew Holt 269a8b5fce Merge branch 'master' into diagnostics
# Conflicts:
#	plugins.go
#	vendor/manifest
2018-02-16 22:42:14 -07:00
Toby Allen faa5248d1f httpserver: Leave %2f encoded when trimming path in site address Fix #1927 (#2014)
* Trim path prefix using EscapedPath()

* clarify comments

* Added Tests for trimPathPrefix

* Ensure path with trailing slash is properly trimmed

* Updated tests to match prepatch behaviour

* Updated tests to match prepatch behaviour

* call parse on url rather than instance

* add additional tests

* return unmodified url if error.  Additional tests
2018-02-16 14:18:02 -07:00
Matthew Holt a03eba6fbc tls: In HTTP->HTTPS redirects, preserve redir port in some circumstances
Only strip the port from the Location URL value if the port is NOT the
HTTPSPort (before, we compared against DefaultHTTPSPort instead of
HTTPSPort). The HTTPSPort can be changed, but is done so for port
forwarding, since in reality you can't 'change' the standard HTTPS port,
you can only forward it.
2018-02-16 12:36:28 -07:00
Matthew Holt 8db80c4a88 tls: Fix HTTP->HTTPS redirects and HTTP challenge when using custom port 2018-02-16 12:05:34 -07:00
Matthew Holt be96cc0e65 httpserver: Raise error when adjusted site addresses clash at startup
See discussion on #2015 for how this situation was discovered. For a
Caddyfile like this:

	localhost {
		...
	}
	:2015 {
		...
	}

Running Caddy like this:

	caddy -host localhost

Produces two sites both defined as `localhost:2015` because the flag
changes the default host value to be `localhost`. This should be an
error since the sites are not distinct and it is confusing. It can also
cause issues with TLS handshakes loading the wrong cert, as the linked
discussion shows.
2018-02-15 00:04:31 -07:00
Matthew Holt f26447e2fb Merge branch 'master' into cert-cache
# Conflicts:
#	sigtrap_posix.go
2018-02-13 13:25:29 -07:00
Matthew Holt 6b3c2212a1 diagnostics: AppendUnique(), restructure sets, add metrics, fix bugs 2018-02-10 12:59:23 -07:00
Matthew Holt 388ff6bc0a diagnostics: Implemented collection functions and create first metrics
- Also implemented robust error handling and failovers
- Vendored klauspost/cpuid
2018-02-08 19:55:44 -07:00
Matthew Holt fc2ff9155c tls: Restructure and improve certificate management
- Expose the list of Caddy instances through caddy.Instances()

- Added arbitrary storage to caddy.Instance

- The cache of loaded certificates is no longer global; now scoped
  per-instance, meaning upon reload (like SIGUSR1) the old cert cache
  will be discarded entirely, whereas before, aggressively reloading
  config that added and removed lots of sites would cause unnecessary
  build-up in the cache over time.

- Key certificates in the cache by their SHA-256 hash instead of
  by their names. This means certificates will not be duplicated in
  memory (within each instance), making Caddy much more memory-efficient
  for large-scale deployments with thousands of sites sharing certs.

- Perform name-to-certificate lookups scoped per caddytls.Config instead
  of a single global lookup. This prevents certificates from stepping on
  each other when they overlap in their names.

- Do not allow TLS configurations keyed by the same hostname to be
  different; this now throws an error.

- Updated relevant tests, with a stark awareness that more tests are
  needed.

- Change the NewContext function signature to include an *Instance.

- Strongly recommend (basically require) use of caddytls.NewConfig()
  to create a new *caddytls.Config, to ensure pointers to the instance
  certificate cache are initialized properly.

- Update the TLS-SNI challenge solver (even though TLS-SNI is disabled
  currently on the CA side). Store temporary challenge cert in instance
  cache, but do so directly by the ACME challenge name, not the hash.
  Modified the getCertificate function to check the cache directly for
  a name match if one isn't found otherwise. This will allow any
  caddytls.Config to be able to help solve a TLS-SNI challenge, with one
  extra side-effect that might actually be kind of interesting (and
  useless): clients could send a certificate's hash as the SNI and
  Caddy would be able to serve that certificate for the handshake.

- Do not attempt to match a "default" (random) certificate when SNI
  is present but unrecognized; return no certificate so a TLS alert
  happens instead.

- Store an Instance in the list of instances even while the instance
  is still starting up (this allows access to the cert cache for
  performing renewals at startup, etc). Will be removed from list again
  if instance startup fails.

- Laid groundwork for ACMEv2 and Let's Encrypt wildcard support.

Server type plugins will need to be updated slightly to accommodate
minor adjustments to their API (like passing in an Instance). This
commit includes the changes for the HTTP server.

Certain Caddyfile configurations might error out with this change, if
they configured different TLS settings for the same hostname.

This change trades some complexity for other complexity, but ultimately
this new complexity is more correct and robust than earlier logic.

Fixes #1991
Fixes #1994
Fixes #1303
2018-02-04 00:58:27 -07:00
Tw 1ba5512015 ResponseBuffer: add missing header writing (#1997)
Signed-off-by: Tw <tw19881113@gmail.com>
2018-01-15 18:32:19 -07:00
Toby Allen c0efec52d9 Allow Masking of IP address in Logfile. (#1930)
* First working mask

* IP Mask working with defaults and empty

* add tests for ipmask

* Store Mask as setup, some tidying, cleaner flow

* Prevent mask from running when directive not present

* use custom replacement to store masked ip
2017-12-23 10:52:11 +00:00