Update readme and changes for version 0.10.9

Benchmarks with wrk showed no noticeable performance impact

.github/CONTRIBUTING.md

@@ -23,13 +23,13 @@ Other menu items:
 
 ### Contributing code
 
-You can have a direct impact on the project by helping with its code. To contribute code to Caddy, open a [pull request](https://github.com/mholt/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.**
+You can have a direct impact on the project by helping with its code. To contribute code to Caddy, open a [pull request](https://github.com/mholt/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with Caddy's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/mholt/caddy/-/search).
 
 We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :wink: If your change is on the right track, we can guide you to make it mergable.
 
 Here are some of the expectations we have of contributors:
 
-- If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, you can comment on the existing one to claim it.
+- If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, comment on the existing issue to claim it.
 
 - **Keep pull requests small.** Smaller PRs are more likely to be merged because they are easier to review! We might ask you to break up large PRs into smaller ones. [An example of what we DON'T do.](https://twitter.com/iamdevloper/status/397664295875805184)
 
@@ -37,11 +37,11 @@ Here are some of the expectations we have of contributors:
 
 - **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass.
 
-- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven to work better with benchmarks or profiling.
+- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks or profiling.
 
-- **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance.
+- **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance. Interactive rebase can do this, or a simpler way is `git reset --soft <diverging-commit>` then `git commit -s`.
 
-- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors can help maintain their change after it is merged.
+- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors help maintain their change after it is merged.
 
 - **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious.
 
@@ -126,7 +126,9 @@ Collabators have push rights to the repository. We grant this permission after o
 
 - **Do not merge pull requests until they have been approved by one or two other collaborators.** If a project owner approves the PR, it can be merged (as long as the conversation has finished too).
 
-- **Prefer squashed commits over a messy merge.** If there are many little commits, please squash the commits so we don't clutter the commit history.
+- **Prefer squashed commits over a messy merge.** If there are many little commits, please [squash the commits](https://stackoverflow.com/a/11732910/1048862) so we don't clutter the commit history.
+
+- **Don't accept new dependencies lightly.** Dependencies can make the world crash and burn, but they are sometimes necessary. Choose carefully. Extremely small dependencies (a few lines of code) can be inlined. The rest may not be needed. For those that are, Caddy vendors all dependencies with the help of [gvt](https://github.com/FiloSottile/gvt). All external dependencies must be vendored, and _Caddy must not export any types defined by those dependencies_. Check this diligently!
 
 - **Be extra careful in some areas of the code.** There are some critical areas in the Caddy code base that we review extra meticulously: the `caddy` and `caddytls` packages especially.
 

.github/ISSUE_TEMPLATE.md

@@ -1,4 +1,6 @@
-(Are you asking for help with using Caddy? Please use our forum instead: https://caddy.community. If you are filing a bug report, please take a few minutes to carefully answer the following questions. If your issue is not a bug report, you do not need to use this template. Thanks!)
+<!--
+Are you asking for help with using Caddy? Please use our forum instead: https://caddy.community. If you are filing a bug report, please take a few minutes to carefully answer the following questions. If your issue is not a bug report, you do not need to use this template. Thanks!)
+-->
 
 ### 1. What version of Caddy are you using (`caddy -version`)?
 
@@ -16,7 +18,7 @@
 
 ### 5. Please paste any relevant HTTP request(s) here.
 
-(paste curl command, or full HTTP request including headers and body, here)
+<!-- Paste curl command, or full HTTP request including headers and body, here. -->
 
 
 ### 6. What did you expect to see?
@@ -27,4 +29,4 @@
 
 ### 8. How can someone who is starting from scratch reproduce the bug as minimally as possible?
 
-(Please strip away any extra infrastructure such as containers, reverse proxies, upstream apps, caches, dependencies, etc, to prove this is a bug in Caddy and not an external misconfiguration. Your chances of getting this bug fixed go way up the easier it is to replicate. Thank you!)
+<!-- Please strip away any extra infrastructure such as containers, reverse proxies, upstream apps, caches, dependencies, etc, to prove this is a bug in Caddy and not an external misconfiguration. Your chances of getting this bug fixed go way up the easier it is to replicate. Thank you! -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,4 +1,6 @@
-(Thank you for contributing to Caddy! Please fill this out to help us make the most of your pull request.)
+<!--
+Thank you for contributing to Caddy! Please fill this out to help us make the most of your pull request.
+-->
 
 ### 1. What does this change do, exactly?
 

.travis.yml

@@ -1,7 +1,11 @@
 language: go
 
+addons:
+  hosts:
+    - quic.clemente.io
+
 go:
-  - 1.8
+  - 1.9
   - tip
 
 matrix:
@@ -20,14 +24,17 @@ install:
   - if [ "$TRAVIS_PULL_REQUEST" = "false" ]; then bash dist/gitcookie.sh; fi
   - go get -t ./...
   - go get github.com/golang/lint/golint
-  - go get github.com/gordonklaus/ineffassign
+  - go get github.com/FiloSottile/vendorcheck
+  # Install gometalinter and certain linters
+  - go get github.com/alecthomas/gometalinter
   - go get github.com/client9/misspell/cmd/misspell
+  - go get github.com/gordonklaus/ineffassign
+  - go get golang.org/x/tools/cmd/goimports
+  - go get github.com/tsenart/deadcode
 
 script:
-  - diff <(echo -n) <(gofmt -s -d .)
-  - ineffassign .
-  - misspell -error .
-  - go vet ./...
+  - gometalinter --disable-all -E vet -E gofmt -E misspell -E ineffassign -E goimports -E deadcode --tests --vendor ./...
+  - vendorcheck ./...
   - go test -race ./...
 
 after_script:

README.md

@@ -1,7 +1,7 @@
 <p align="center">
 	<a href="https://caddyserver.com"><img src="https://cloud.githubusercontent.com/assets/1128849/25305033/12916fce-2731-11e7-86ec-580d4d31cb16.png" alt="Caddy" width="400"></a>
 </p>
-<h3 align="center">Every Site on HTTPS <!-- Serve Confidently. --></h3>
+<h3 align="center">Every Site on HTTPS <!-- Serve Confidently --></h3>
 <p align="center">Caddy is a general-purpose HTTP/2 web server that serves HTTPS by default.</p>
 <p align="center">
 	<a href="https://travis-ci.org/mholt/caddy"><img src="https://img.shields.io/travis/mholt/caddy.svg?label=linux+build"></a>
@@ -57,7 +57,7 @@ Caddy binaries have no dependencies and are available for every platform. Get Ca
 customize your build in the browser
 - **[Latest release](https://github.com/mholt/caddy/releases/latest)** for
 pre-built, vanilla binaries
-- **go get** to build from source: `go get github.com/mholt/caddy/caddy` (requires Go 1.8 or newer)
+- **go get** to build from source: `go get github.com/mholt/caddy/caddy` (requires Go 1.8 or newer) - to build with proper version information (required when filing issues), `cd` to the `caddy` folder and use `go run build.go`.
 
 Then make sure the `caddy` binary is in your PATH.
 
@@ -123,12 +123,14 @@ The Caddy project does not officially maintain any system-specific integrations
 
 How you choose to run Caddy is up to you. Many users are satisfied with `nohup caddy &`. Others use `screen`. Users who need Caddy to come back up after reboots either do so in the script that caused the reboot, add a command to an init script, or configure a service with their OS.
 
+If you have questions or concerns about Caddy' underlying crypto implementations, consult Go's [crypto packages](https://golang.org/pkg/crypto), starting with their documentation, then issues, then the code itself; as Caddy uses mainly those libraries.
+
 
 ## Contributing
 
-**[Join our forum](https://caddy.community) where you can chat with other Caddy users and developers!**
+**[Join our forum](https://caddy.community) where you can chat with other Caddy users and developers!** To get familiar with the code base, try [Caddy code search on Sourcegraph](https://sourcegraph.com/github.com/mholt/caddy/-/search)!
 
-Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/CONTRIBUTING.md). If you want to write a plugin, check out the [developer wiki](https://github.com/mholt/caddy/wiki).
+Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/.github/CONTRIBUTING.md) for instructions. If you want to write a plugin, check out the [developer wiki](https://github.com/mholt/caddy/wiki).
 
 We use GitHub issues and pull requests only for discussing bug reports and the development of specific changes. We welcome all other topics on the [forum](https://caddy.community)!
 

appveyor.yml

@@ -1,5 +1,8 @@
 version: "{build}"
 
+hosts:
+  quic.clemente.io: 127.0.0.1
+
 os: Windows Server 2012 R2
 
 clone_folder: c:\gopath\src\github.com\mholt\caddy
@@ -9,21 +12,27 @@ environment:
 
 install:
   - rmdir c:\go /s /q
-  - appveyor DownloadFile https://storage.googleapis.com/golang/go1.8.windows-amd64.zip
-  - 7z x go1.8.windows-amd64.zip -y -oC:\ > NUL
+  - appveyor DownloadFile https://storage.googleapis.com/golang/go1.9.windows-amd64.zip
+  - 7z x go1.9.windows-amd64.zip -y -oC:\ > NUL
+  - set PATH=%GOPATH%\bin;%PATH%
   - go version
   - go env
   - go get -t ./...
   - go get github.com/golang/lint/golint
+  - go get github.com/FiloSottile/vendorcheck
+   # Install gometalinter and certain linters
+  - go get github.com/alecthomas/gometalinter
+  - go get github.com/client9/misspell/cmd/misspell
   - go get github.com/gordonklaus/ineffassign
-  - set PATH=%GOPATH%\bin;%PATH%
+  - go get golang.org/x/tools/cmd/goimports
+  - go get github.com/tsenart/deadcode
 
 build: off
 
 test_script:
-  - go vet ./...
+  - gometalinter --disable-all -E vet -E gofmt -E misspell -E ineffassign -E goimports -E deadcode --tests --vendor ./...
+  - vendorcheck ./...
   - go test -race ./...
-  - ineffassign .
 
 after_test:
   - golint ./...

caddy.go

@@ -17,6 +17,7 @@ package caddy
 
 import (
 	"bytes"
+	"encoding/gob"
 	"fmt"
 	"io"
 	"io/ioutil"
@@ -51,7 +52,7 @@ var (
 	// isUpgrade will be set to true if this process
 	// was started as part of an upgrade, where a parent
 	// Caddy process started this one.
-	isUpgrade bool
+	isUpgrade = os.Getenv("CADDY__UPGRADE") == "1"
 
 	// started will be set to true when the first
 	// instance is started; it never gets set to
@@ -360,6 +361,16 @@ type AfterStartup interface {
 // is returned. Consequently, this function never returns a nil
 // value as long as there are no errors.
 func LoadCaddyfile(serverType string) (Input, error) {
+	// If we are finishing an upgrade, we must obtain the Caddyfile
+	// from our parent process, regardless of configured loaders.
+	if IsUpgrade() {
+		err := gob.NewDecoder(os.Stdin).Decode(&loadedGob)
+		if err != nil {
+			return nil, err
+		}
+		return loadedGob.Caddyfile, nil
+	}
+
 	// Ask plugged-in loaders for a Caddyfile
 	cdyfile, err := loadCaddyfileInput(serverType)
 	if err != nil {
@@ -424,9 +435,16 @@ func (i *Instance) Caddyfile() Input {
 //
 // This function blocks until all the servers are listening.
 func Start(cdyfile Input) (*Instance, error) {
-	writePidFile()
 	inst := &Instance{serverType: cdyfile.ServerType(), wg: new(sync.WaitGroup)}
-	return inst, startWithListenerFds(cdyfile, inst, nil)
+	err := startWithListenerFds(cdyfile, inst, nil)
+	if err != nil {
+		return inst, err
+	}
+	signalSuccessToParent()
+	if pidErr := writePidFile(); pidErr != nil {
+		log.Printf("[ERROR] Could not write pidfile: %v", pidErr)
+	}
+	return inst, nil
 }
 
 func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]restartTriple) error {
@@ -445,7 +463,8 @@ func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]r
 	}
 
 	// run startup callbacks
-	if restartFds == nil {
+	if !IsUpgrade() && restartFds == nil {
+		// first startup means not a restart or upgrade
 		for _, firstStartupFunc := range inst.onFirstStartup {
 			err := firstStartupFunc()
 			if err != nil {
@@ -500,7 +519,6 @@ func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]r
 // callbacks will not be executed between directives, since the purpose
 // is only to check the input for valid syntax.
 func ValidateAndExecuteDirectives(cdyfile Input, inst *Instance, justValidate bool) error {
-
 	// If parsing only inst will be nil, create an instance for this function call only.
 	if justValidate {
 		inst = &Instance{serverType: cdyfile.ServerType(), wg: new(sync.WaitGroup)}
@@ -536,7 +554,6 @@ func ValidateAndExecuteDirectives(cdyfile Input, inst *Instance, justValidate bo
 	}
 
 	return nil
-
 }
 
 func executeDirectives(inst *Instance, filename string,
@@ -616,6 +633,30 @@ func startServers(serverList []Server, inst *Instance, restartFds map[string]res
 			err error
 		)
 
+		// if performing an upgrade, obtain listener file descriptors
+		// from parent process
+		if IsUpgrade() {
+			if gs, ok := s.(GracefulServer); ok {
+				addr := gs.Address()
+				if fdIndex, ok := loadedGob.ListenerFds["tcp"+addr]; ok {
+					file := os.NewFile(fdIndex, "")
+					ln, err = net.FileListener(file)
+					file.Close()
+					if err != nil {
+						return err
+					}
+				}
+				if fdIndex, ok := loadedGob.ListenerFds["udp"+addr]; ok {
+					file := os.NewFile(fdIndex, "")
+					pc, err = net.FilePacketConn(file)
+					file.Close()
+					if err != nil {
+						return err
+					}
+				}
+			}
+		}
+
 		// If this is a reload and s is a GracefulServer,
 		// reuse the listener for a graceful restart.
 		if gs, ok := s.(GracefulServer); ok && restartFds != nil {
@@ -795,25 +836,6 @@ func IsInternal(addr string) bool {
 	return false
 }
 
-// Upgrade re-launches the process, preserving the listeners
-// for a graceful restart. It does NOT load new configuration;
-// it only starts the process anew with a fresh binary.
-//
-// TODO: This is not yet implemented
-func Upgrade() error {
-	return fmt.Errorf("not implemented")
-	// TODO: have child process set isUpgrade = true
-}
-
-// IsUpgrade returns true if this process is part of an upgrade
-// where a parent caddy process spawned this one to upgrade
-// the binary.
-func IsUpgrade() bool {
-	mu.Lock()
-	defer mu.Unlock()
-	return isUpgrade
-}
-
 // Started returns true if at least one instance has been
 // started by this package. It never gets reset to false
 // once it is set to true.

caddy/build.bash

@@ -1,56 +0,0 @@
-#!/usr/bin/env bash
-#
-# Caddy build script. Automates proper versioning.
-#
-# Usage:
-#
-#     $ ./build.bash [output_filename] [git_repo]
-#
-# Outputs compiled program in current directory.
-# Default git repo is current directory.
-# Builds always take place from current directory.
-
-set -euo pipefail
-
-: ${output_filename:="${1:-}"}
-: ${output_filename:=""}
-
-: ${git_repo:="${2:-}"}
-: ${git_repo:="."}
-
-pkg=github.com/mholt/caddy/caddy/caddymain
-ldflags=()
-
-# Timestamp of build
-name="${pkg}.buildDate"
-value=$(date -u +"%a %b %d %H:%M:%S %Z %Y")
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Current tag, if HEAD is on a tag
-name="${pkg}.gitTag"
-set +e
-value="$(git -C "${git_repo}" describe --exact-match HEAD 2>/dev/null)"
-set -e
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Nearest tag on branch
-name="${pkg}.gitNearestTag"
-value="$(git -C "${git_repo}" describe --abbrev=0 --tags HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Commit SHA
-name="${pkg}.gitCommit"
-value="$(git -C "${git_repo}" rev-parse --short HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Summary of uncommitted changes
-name="${pkg}.gitShortStat"
-value="$(git -C "${git_repo}" diff-index --shortstat HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# List of modified files
-name="${pkg}.gitFilesModified"
-value="$(git -C "${git_repo}" diff-index --name-only HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-go build -ldflags "${ldflags[*]}" -o "${output_filename}"

caddy/build.go

@@ -0,0 +1,73 @@
+// +build dev
+
+// build.go automates proper versioning of caddy binaries.
+// Use it like:   go run build.go
+// You can customize the build with the -goos, -goarch, and
+// -goarm CLI options:   go run build.go -goos=windows
+//
+// To get proper version information, this program must be
+// run from the directory of this file, and the source code
+// must be a working git repository, since it needs to know
+// if the source is in a clean state.
+//
+// This program is NOT required to build Caddy from source
+// since it is go-gettable. (You can run plain `go build`
+// in this directory to get a binary.) However, issues filed
+// without version information will likely be closed.
+package main
+
+import (
+	"flag"
+	"fmt"
+	"log"
+	"os"
+	"os/exec"
+	"path/filepath"
+
+	"github.com/caddyserver/builds"
+)
+
+var goos, goarch, goarm string
+
+func init() {
+	flag.StringVar(&goos, "goos", "", "GOOS for which to build")
+	flag.StringVar(&goarch, "goarch", "", "GOARCH for which to build")
+	flag.StringVar(&goarm, "goarm", "", "GOARM for which to build")
+}
+
+func main() {
+	flag.Parse()
+
+	gopath := os.Getenv("GOPATH")
+
+	pwd, err := os.Getwd()
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	ldflags, err := builds.MakeLdFlags(filepath.Join(pwd, ".."))
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	args := []string{"build", "-ldflags", ldflags}
+	args = append(args, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
+	args = append(args, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
+	cmd := exec.Command("go", args...)
+	cmd.Stderr = os.Stderr
+	cmd.Stdout = os.Stdout
+	cmd.Env = os.Environ()
+	for _, env := range []string{
+		"CGO_ENABLED=0",
+		"GOOS=" + goos,
+		"GOARCH=" + goarch,
+		"GOARM=" + goarm,
+	} {
+		cmd.Env = append(cmd.Env, env)
+	}
+
+	err = cmd.Run()
+	if err != nil {
+		log.Fatal(err)
+	}
+}

caddy/caddymain/run.go

@@ -101,7 +101,7 @@ func Run() {
 	}
 
 	// Executes Startup events
-	caddy.EmitEvent(caddy.StartupEvent)
+	caddy.EmitEvent(caddy.StartupEvent, nil)
 
 	// Get Caddyfile input
 	caddyfileinput, err := caddy.LoadCaddyfile(serverType)
@@ -186,10 +186,14 @@ func setVersion() {
 	// A development build is one that's not at a tag or has uncommitted changes
 	devBuild = gitTag == "" || gitShortStat != ""
 
+	if buildDate != "" {
+		buildDate = " " + buildDate
+	}
+
 	// Only set the appVersion if -ldflags was used
 	if gitNearestTag != "" || gitTag != "" {
 		if devBuild && gitNearestTag != "" {
-			appVersion = fmt.Sprintf("%s (+%s %s)",
+			appVersion = fmt.Sprintf("%s (+%s%s)",
 				strings.TrimPrefix(gitNearestTag, "v"), gitCommit, buildDate)
 		} else if gitTag != "" {
 			appVersion = strings.TrimPrefix(gitTag, "v")

caddy_test.go

@@ -142,13 +142,13 @@ func TestListenerAddrEqual(t *testing.T) {
 		addr   string
 		expect bool
 	}{
-		{ln1, ":1234", false},
-		{ln1, "0.0.0.0:1234", false},
+		{ln1, ":" + ln2port, false},
+		{ln1, "0.0.0.0:" + ln2port, false},
 		{ln1, "0.0.0.0", false},
 		{ln1, ":" + ln1port, true},
 		{ln1, "0.0.0.0:" + ln1port, true},
 		{ln2, ":" + ln2port, false},
-		{ln2, "127.0.0.1:1234", false},
+		{ln2, "127.0.0.1:" + ln1port, false},
 		{ln2, "127.0.0.1", false},
 		{ln2, "127.0.0.1:" + ln2port, true},
 	} {

caddyfile/dispenser.go

@@ -210,9 +210,9 @@ func (d *Dispenser) EOFErr() error {
 	return d.Errf("Unexpected EOF")
 }
 
-// Err generates a custom parse error with a message of msg.
+// Err generates a custom parse-time error with a message of msg.
 func (d *Dispenser) Err(msg string) error {
-	msg = fmt.Sprintf("%s:%d - Parse error: %s", d.File(), d.Line(), msg)
+	msg = fmt.Sprintf("%s:%d - Error during parsing: %s", d.File(), d.Line(), msg)
 	return errors.New(msg)
 }
 

caddyhttp/basicauth/setup.go

@@ -91,8 +91,9 @@ func basicAuthParse(c *caddy.Controller) ([]Rule, error) {
 }
 
 func passwordMatcher(username, passw, siteRoot string) (PasswordMatcher, error) {
-	if !strings.HasPrefix(passw, "htpasswd=") {
+	htpasswdPrefix := "htpasswd="
+	if !strings.HasPrefix(passw, htpasswdPrefix) {
 		return PlainMatcher(passw), nil
 	}
-	return GetHtpasswdMatcher(passw[9:], username, siteRoot)
+	return GetHtpasswdMatcher(passw[len(htpasswdPrefix):], username, siteRoot)
 }

caddyhttp/browse/browse.go

@@ -112,12 +112,13 @@ func (l Listing) Breadcrumbs() []Crumb {
 
 // FileInfo is the info about a particular file or directory
 type FileInfo struct {
-	Name    string
-	Size    int64
-	URL     string
-	ModTime time.Time
-	Mode    os.FileMode
-	IsDir   bool
+	Name      string
+	Size      int64
+	URL       string
+	ModTime   time.Time
+	Mode      os.FileMode
+	IsDir     bool
+	IsSymlink bool
 }
 
 // HumanSize returns the size of the file as a human-readable string
@@ -244,7 +245,9 @@ func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string, config
 			}
 		}
 
-		if f.IsDir() {
+		isDir := f.IsDir() || isSymlinkTargetDir(f, urlPath, config)
+
+		if isDir {
 			name += "/"
 			dirCount++
 		} else {
@@ -258,12 +261,13 @@ func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string, config
 		url := url.URL{Path: "./" + name} // prepend with "./" to fix paths with ':' in the name
 
 		fileinfos = append(fileinfos, FileInfo{
-			IsDir:   f.IsDir(),
-			Name:    f.Name(),
-			Size:    f.Size(),
-			URL:     url.String(),
-			ModTime: f.ModTime().UTC(),
-			Mode:    f.Mode(),
+			IsDir:     isDir,
+			IsSymlink: isSymlink(f),
+			Name:      f.Name(),
+			Size:      f.Size(),
+			URL:       url.String(),
+			ModTime:   f.ModTime().UTC(),
+			Mode:      f.Mode(),
 		})
 	}
 
@@ -277,6 +281,32 @@ func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string, config
 	}, hasIndexFile
 }
 
+// isSymlink return true if f is a symbolic link
+func isSymlink(f os.FileInfo) bool {
+	return f.Mode()&os.ModeSymlink != 0
+}
+
+// isSymlinkTargetDir return true if f's symbolic link target
+// is a directory. Return false if not a symbolic link.
+func isSymlinkTargetDir(f os.FileInfo, urlPath string, config *Config) bool {
+	if !isSymlink(f) {
+		return false
+	}
+
+	// a bit strange, but we want Stat thru the jailed filesystem to be safe
+	target, err := config.Fs.Root.Open(path.Join(urlPath, f.Name()))
+	if err != nil {
+		return false
+	}
+	defer target.Close()
+	targetInfo, err := target.Stat()
+	if err != nil {
+		return false
+	}
+
+	return targetInfo.IsDir()
+}
+
 // ServeHTTP determines if the request is for this plugin, and if all prerequisites are met.
 // If so, control is handed over to ServeListing.
 func (b Browse) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {

caddyhttp/browse/browse_test.go

@@ -3,12 +3,16 @@ package browse
 import (
 	"context"
 	"encoding/json"
+	"fmt"
+	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
 	"os"
 	"path/filepath"
+	"runtime"
 	"sort"
+	"strings"
 	"testing"
 	"text/template"
 	"time"
@@ -17,6 +21,8 @@ import (
 	"github.com/mholt/caddy/caddyhttp/staticfiles"
 )
 
+const testDirPrefix = "caddy_browse_test"
+
 func TestSort(t *testing.T) {
 	// making up []fileInfo with bogus values;
 	// to be used to make up our "listing"
@@ -221,7 +227,7 @@ func TestBrowseTemplate(t *testing.T) {
 func TestBrowseJson(t *testing.T) {
 	b := Browse{
 		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
-			t.Fatalf("Next shouldn't be called")
+			t.Fatalf("Next shouldn't be called: %s", r.URL)
 			return 0, nil
 		}),
 		Configs: []Config{
@@ -453,3 +459,153 @@ func TestBrowseRedirect(t *testing.T) {
 		}
 	}
 }
+
+func TestDirSymlink(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		// Windows support for symlinks is limited, and we had a hard time getting
+		// all these tests to pass with the permissions of CI; so just skip them
+		fmt.Println("Skipping browse symlink tests on Windows...")
+		return
+	}
+
+	testCases := []struct {
+		source       string
+		target       string
+		pathScope    string
+		url          string
+		expectedName string
+		expectedURL  string
+	}{
+		// test case can expect a directory "dir" and a symlink to it called "symlink"
+
+		{"dir", "$TMP/rel_symlink_to_dir", "/", "/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/abs_symlink_to_dir", "/", "/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"../../dir", "$TMP/sub/dir/rel_symlink_to_dir", "/", "/sub/dir/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/sub/dir/abs_symlink_to_dir", "/", "/sub/dir/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"../../dir", "$TMP/with/scope/rel_symlink_to_dir", "/with/scope", "/with/scope/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/with/scope/abs_symlink_to_dir", "/with/scope", "/with/scope/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"../../../../dir", "$TMP/with/scope/sub/dir/rel_symlink_to_dir", "/with/scope", "/with/scope/sub/dir/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/with/scope/sub/dir/abs_symlink_to_dir", "/with/scope", "/with/scope/sub/dir/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"symlink", "$TMP/rel_symlink_to_symlink", "/", "/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/abs_symlink_to_symlink", "/", "/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+
+		{"../../symlink", "$TMP/sub/dir/rel_symlink_to_symlink", "/", "/sub/dir/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/sub/dir/abs_symlink_to_symlink", "/", "/sub/dir/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+
+		{"../../symlink", "$TMP/with/scope/rel_symlink_to_symlink", "/with/scope", "/with/scope/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/with/scope/abs_symlink_to_symlink", "/with/scope", "/with/scope/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+
+		{"../../../../symlink", "$TMP/with/scope/sub/dir/rel_symlink_to_symlink", "/with/scope", "/with/scope/sub/dir/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/with/scope/sub/dir/abs_symlink_to_symlink", "/with/scope", "/with/scope/sub/dir/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+	}
+
+	for i, tc := range testCases {
+		func() {
+			tmpdir, err := ioutil.TempDir("", testDirPrefix)
+			if err != nil {
+				t.Fatalf("failed to create test directory: %v", err)
+			}
+			defer os.RemoveAll(tmpdir)
+
+			if err := os.MkdirAll(filepath.Join(tmpdir, "dir"), 0755); err != nil {
+				t.Fatalf("failed to create test dir 'dir': %v", err)
+			}
+			if err := os.Symlink("dir", filepath.Join(tmpdir, "symlink")); err != nil {
+				t.Fatalf("failed to create test symlink 'symlink': %v", err)
+			}
+
+			sourceResolved := strings.Replace(tc.source, "$TMP", tmpdir, -1)
+			targetResolved := strings.Replace(tc.target, "$TMP", tmpdir, -1)
+
+			if err := os.MkdirAll(filepath.Dir(sourceResolved), 0755); err != nil {
+				t.Fatalf("failed to create source symlink dir: %v", err)
+			}
+			if err := os.MkdirAll(filepath.Dir(targetResolved), 0755); err != nil {
+				t.Fatalf("failed to create target symlink dir: %v", err)
+			}
+			if err := os.Symlink(sourceResolved, targetResolved); err != nil {
+				t.Fatalf("failed to create test symlink: %v", err)
+			}
+
+			b := Browse{
+				Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+					t.Fatalf("Test %d - Next shouldn't be called", i)
+					return 0, nil
+				}),
+				Configs: []Config{
+					{
+						PathScope: tc.pathScope,
+						Fs: staticfiles.FileServer{
+							Root: http.Dir(tmpdir),
+						},
+					},
+				},
+			}
+
+			req, err := http.NewRequest("GET", tc.url, nil)
+			req.Header.Add("Accept", "application/json")
+			if err != nil {
+				t.Fatalf("Test %d - could not create HTTP request: %v", i, err)
+			}
+
+			rec := httptest.NewRecorder()
+
+			returnCode, _ := b.ServeHTTP(rec, req)
+			if returnCode != http.StatusOK {
+				t.Fatalf("Test %d - wrong return code, expected %d, got %d",
+					i, http.StatusOK, returnCode)
+			}
+
+			type jsonEntry struct {
+				Name      string
+				IsDir     bool
+				IsSymlink bool
+				URL       string
+			}
+			var entries []jsonEntry
+			if err := json.Unmarshal(rec.Body.Bytes(), &entries); err != nil {
+				t.Fatalf("Test %d - failed to parse json: %v", i, err)
+			}
+
+			found := false
+			for _, e := range entries {
+				if e.Name != tc.expectedName {
+					continue
+				}
+				found = true
+				if !e.IsDir {
+					t.Errorf("Test %d - expected to be a dir, got %v", i, e.IsDir)
+				}
+				if !e.IsSymlink {
+					t.Errorf("Test %d - expected to be a symlink, got %v", i, e.IsSymlink)
+				}
+				if e.URL != tc.expectedURL {
+					t.Errorf("Test %d - wrong URL, expected %v, got %v", i, tc.expectedURL, e.URL)
+				}
+			}
+			if !found {
+				t.Errorf("Test %d - failed, could not find name %v", i, tc.expectedName)
+			}
+		}()
+	}
+}

caddyhttp/browse/setup.go

@@ -150,16 +150,22 @@ h1 {
 	white-space: nowrap;
 	overflow-x: hidden;
 	text-overflow: ellipsis;
+	color: #999;
 }
 
 h1 a {
-	color: inherit;
+	color: #000;
+	margin: 0 4px;
 }
 
 h1 a:hover {
 	text-decoration: underline;
 }
 
+h1 a:first-child {
+	margin: 0;
+}
+
 main {
 	display: block;
 }
@@ -216,11 +222,16 @@ th svg {
 }
 
 td {
+	white-space: nowrap;
 	font-size: 14px;
 }
 
 td:first-child {
-	width: 50%;
+	width: 100%;
+}
+
+td:nth-child(2) {
+	padding: 0 20px 0 20px;
 }
 
 th:last-child,
@@ -284,6 +295,18 @@ footer {
 		padding-right: 5%;
 		text-align: right;
 	}
+
+	h1 {
+		color: #000;
+	}
+
+	h1 a {
+		margin: 0;
+	}
+
+	#filter {
+		max-width: 100px;
+	}
 }
 </style>
 	</head>
@@ -291,45 +314,32 @@ footer {
 		<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" height="0" width="0" style="position: absolute;">
 			<defs>
 				<!-- Folder -->
-				<linearGradient id="f" y2="640" gradientUnits="userSpaceOnUse" x2="244.84" gradientTransform="matrix(.97319 0 0 1.0135 -.50695 -13.679)" y1="415.75" x1="244.84">
-					<stop stop-color="#b3ddfd" offset="0"/>
-					<stop stop-color="#69c" offset="1"/>
-				</linearGradient>
-				<linearGradient id="e" y2="571.06" gradientUnits="userSpaceOnUse" x2="238.03" gradientTransform="translate(0,2)" y1="346.05" x1="236.26">
-					<stop stop-color="#ace" offset="0"/>
-					<stop stop-color="#369" offset="1"/>
-				</linearGradient>
-				<g id="folder" transform="translate(-266.06 -193.36)">
-					<g transform="matrix(.066019 0 0 .066019 264.2 170.93)">
-						<g transform="matrix(1.4738 0 0 1.4738 -52.053 -166.93)">
-							<path fill="#69c" d="m98.424 343.78c-11.08 0-20 8.92-20 20v48.5 33.719 105.06c0 11.08 8.92 20 20 20h279.22c11.08 0 20-8.92 20-20v-138.78c0-11.08-8.92-20-20-20h-117.12c-7.5478-1.1844-9.7958-6.8483-10.375-11.312v-5.625-11.562c0-11.08-8.92-20-20-20h-131.72z"/>
-							<rect rx="12.885" ry="12.199" height="227.28" width="366.69" y="409.69" x="54.428" fill="#369"/>
-							<path fill="url(#e)" d="m98.424 345.78c-11.08 0-20 8.92-20 20v48.5 33.719 105.06c0 11.08 8.92 20 20 20h279.22c11.08 0 20-8.92 20-20v-138.78c0-11.08-8.92-20-20-20h-117.12c-7.5478-1.1844-9.7958-6.8483-10.375-11.312v-5.625-11.562c0-11.08-8.92-20-20-20h-131.72z"/>
-							<rect rx="12.885" ry="12.199" height="227.28" width="366.69" y="407.69" x="54.428" fill="url(#f)"/>
+				<g id="folder" fill-rule="nonzero" fill="none">
+					<path d="M285.22 37.55h-142.6L110.9 0H31.7C14.25 0 0 16.9 0 37.55v75.1h316.92V75.1c0-20.65-14.26-37.55-31.7-37.55z" fill="#FFA000"/>
+					<path d="M285.22 36H31.7C14.25 36 0 50.28 0 67.74v158.7c0 17.47 14.26 31.75 31.7 31.75H285.2c17.44 0 31.7-14.3 31.7-31.75V67.75c0-17.47-14.26-31.75-31.7-31.75z" fill="#FFCA28"/>
+				</g>
+				<g id="folder-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+					<g id="folder-shortcut-group" fill-rule="nonzero">
+						<g id="folder-shortcut-shape">
+							<path d="M285.224876,37.5486902 L142.612438,37.5486902 L110.920785,0 L31.6916529,0 C14.2612438,0 0,16.8969106 0,37.5486902 L0,112.646071 L316.916529,112.646071 L316.916529,75.0973805 C316.916529,54.4456008 302.655285,37.5486902 285.224876,37.5486902 Z" id="Shape" fill="#FFA000"></path>
+							<path d="M285.224876,36 L31.6916529,36 C14.2612438,36 0,50.2838568 0,67.7419039 L0,226.451424 C0,243.909471 14.2612438,258.193328 31.6916529,258.193328 L285.224876,258.193328 C302.655285,258.193328 316.916529,243.909471 316.916529,226.451424 L316.916529,67.7419039 C316.916529,50.2838568 302.655285,36 285.224876,36 Z" id="Shape" fill="#FFCA28"></path>
 						</g>
+						<path d="M126.154134,250.559184 C126.850974,251.883673 127.300549,253.006122 127.772602,254.106122 C128.469442,255.206122 128.919016,256.104082 129.638335,257.002041 C130.559962,258.326531 131.728855,259 133.100057,259 C134.493737,259 135.415364,258.55102 136.112204,257.67551 C136.809044,257.002041 137.258619,255.902041 137.258619,254.577551 C137.258619,253.904082 137.258619,252.804082 137.033832,251.457143 C136.786566,249.908163 136.561779,249.032653 136.561779,248.583673 C136.089726,242.814286 135.864939,237.920408 135.864939,233.273469 C135.864939,225.057143 136.786566,217.514286 138.180246,210.846939 C139.798713,204.202041 141.889234,198.634694 144.429328,193.763265 C147.216689,188.869388 150.678411,184.873469 154.836973,181.326531 C158.995535,177.779592 163.626149,174.883673 168.481552,172.661224 C173.336954,170.438776 179.113983,168.665306 185.587852,167.340816 C192.061722,166.218367 198.760378,165.342857 205.481514,164.669388 C212.18017,164.220408 219.598146,163.995918 228.162535,163.995918 L246.055591,163.995918 L246.055591,195.514286 C246.055591,197.736735 246.752431,199.510204 248.370899,201.059184 C250.214153,202.608163 252.079886,203.506122 254.372715,203.506122 C256.463236,203.506122 258.531277,202.608163 260.172223,201.059184 L326.102289,137.797959 C327.720757,136.24898 328.642384,134.47551 328.642384,132.253061 C328.642384,130.030612 327.720757,128.257143 326.102289,126.708163 L260.172223,63.4469388 C258.553756,61.8979592 256.463236,61 254.395194,61 C252.079886,61 250.236632,61.8979592 248.393377,63.4469388 C246.77491,64.9959184 246.07807,66.7693878 246.07807,68.9918367 L246.07807,100.510204 L228.162535,100.510204 C166.863084,100.510204 129.166282,117.167347 115.274437,150.459184 C110.666301,161.54898 108.350993,175.310204 108.350993,191.742857 C108.350993,205.279592 113.903236,223.912245 124.760454,247.438776 C125.00772,248.112245 125.457294,249.010204 126.154134,250.559184 Z" id="Shape" fill="#FFFFFF" transform="translate(218.496689, 160.000000) scale(-1, 1) translate(-218.496689, -160.000000) "></path>
 					</g>
 				</g>
 
 				<!-- File -->
-				<linearGradient id="a">
-					<stop stop-color="#cbcbcb" offset="0"/>
-					<stop stop-color="#f0f0f0" offset=".34923"/>
-					<stop stop-color="#e2e2e2" offset="1"/>
-				</linearGradient>
-				<linearGradient id="d" y2="686.15" xlink:href="#a" gradientUnits="userSpaceOnUse" y1="207.83" gradientTransform="matrix(.28346 0 0 .31053 -608.52 485.11)" x2="380.1" x1="749.25"/>
-				<linearGradient id="c" y2="287.74" xlink:href="#a" gradientUnits="userSpaceOnUse" y1="169.44" gradientTransform="matrix(.28342 0 0 .31057 -608.52 485.11)" x2="622.33" x1="741.64"/>
-				<linearGradient id="b" y2="418.54" gradientUnits="userSpaceOnUse" y1="236.13" gradientTransform="matrix(.29343 0 0 .29999 -608.52 485.11)" x2="330.88" x1="687.96">
-					<stop stop-color="#fff" offset="0"/>
-					<stop stop-color="#fff" stop-opacity="0" offset="1"/>
-				</linearGradient>
-				<g id="file" transform="translate(-278.15 -216.59)">
-					<g fill-rule="evenodd" transform="matrix(.19775 0 0 .19775 381.05 112.68)">
-						<path d="m-520.17 525.5v36.739 36.739 36.739 36.739h33.528 33.528 33.528 33.528v-36.739-36.739-36.739l-33.528-36.739h-33.528-33.528-33.528z" stroke-opacity=".36478" stroke-width=".42649" fill="#fff"/>
-						<g>
-							<path d="m-520.11 525.68v36.739 36.739 36.739 36.739h33.528 33.528 33.528 33.528v-36.739-36.739-36.739l-33.528-36.739h-33.528-33.528-33.528z" stroke-opacity=".36478" stroke="#000" stroke-width=".42649" fill="url(#d)"/>
-							<path d="m-386 562.42c-10.108-2.9925-23.206-2.5682-33.101-0.86253 1.7084-10.962 1.922-24.701-0.4271-35.877l33.528 36.739z" stroke-width=".95407pt" fill="url(#c)"/>
-							<path d="m-519.13 537-0.60402 134.7h131.68l0.0755-33.296c-2.9446 1.1325-32.692-40.998-70.141-39.186-37.483 1.8137-27.785-56.777-61.006-62.214z" stroke-width="1pt" fill="url(#b)"/>
+				<g id="file" stroke="#000" stroke-width="25" fill="#FFF" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round">
+					<path d="M13 24.12v274.76c0 6.16 5.87 11.12 13.17 11.12H239c7.3 0 13.17-4.96 13.17-11.12V136.15S132.6 13 128.37 13H26.17C18.87 13 13 17.96 13 24.12z"/>
+					<path d="M129.37 13L129 113.9c0 10.58 7.26 19.1 16.27 19.1H249L129.37 13z"/>
+				</g>
+				<g id="file-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+					<g id="file-shortcut-group" transform="translate(13.000000, 13.000000)">
+						<g id="file-shortcut-shape" stroke="#000000" stroke-width="25" fill="#FFFFFF" stroke-linecap="round" stroke-linejoin="round">
+							<path d="M0,11.1214886 L0,285.878477 C0,292.039924 5.87498876,296.999983 13.1728373,296.999983 L225.997983,296.999983 C233.295974,296.999983 239.17082,292.039942 239.17082,285.878477 L239.17082,123.145388 C239.17082,123.145388 119.58541,2.84217094e-14 115.369423,2.84217094e-14 L13.1728576,2.84217094e-14 C5.87500907,-1.71479982e-05 0,4.96022995 0,11.1214886 Z" id="rect1171"></path>
+							<path d="M116.37005,0 L116,100.904964 C116,111.483663 123.258008,120 132.273377,120 L236,120 L116.37005,0 L116.37005,0 Z" id="rect1794"></path>
 						</g>
+						<path d="M47.803141,294.093878 C48.4999811,295.177551 48.9495553,296.095918 49.4216083,296.995918 C50.1184484,297.895918 50.5680227,298.630612 51.2873415,299.365306 C52.2089688,300.44898 53.3778619,301 54.7490634,301 C56.1427436,301 57.0643709,300.632653 57.761211,299.916327 C58.4580511,299.365306 58.9076254,298.465306 58.9076254,297.381633 C58.9076254,296.830612 58.9076254,295.930612 58.6828382,294.828571 C58.4355724,293.561224 58.2107852,292.844898 58.2107852,292.477551 C57.7387323,287.757143 57.5139451,283.753061 57.5139451,279.95102 C57.5139451,273.228571 58.4355724,267.057143 59.8292526,261.602041 C61.44772,256.165306 63.5382403,251.610204 66.0783349,247.62449 C68.8656954,243.620408 72.3274172,240.35102 76.4859792,237.44898 C80.6445412,234.546939 85.2751561,232.177551 90.1305582,230.359184 C94.9859603,228.540816 100.76299,227.089796 107.236859,226.006122 C113.710728,225.087755 120.409385,224.371429 127.13052,223.820408 C133.829177,223.453061 141.247152,223.269388 149.811542,223.269388 L167.704598,223.269388 L167.704598,249.057143 C167.704598,250.87551 168.401438,252.326531 170.019905,253.593878 C171.86316,254.861224 173.728893,255.595918 176.021722,255.595918 C178.112242,255.595918 180.180284,254.861224 181.82123,253.593878 L247.751296,201.834694 C249.369763,200.567347 250.291391,199.116327 250.291391,197.297959 C250.291391,195.479592 249.369763,194.028571 247.751296,192.761224 L181.82123,141.002041 C180.202763,139.734694 178.112242,139 176.044201,139 C173.728893,139 171.885639,139.734694 170.042384,141.002041 C168.423917,142.269388 167.727077,143.720408 167.727077,145.538776 L167.727077,171.326531 L149.811542,171.326531 C88.5120908,171.326531 50.8152886,184.955102 36.9234437,212.193878 C32.3153075,221.267347 30,232.526531 30,245.971429 C30,257.046939 35.5522422,272.291837 46.4094607,291.540816 C46.6567266,292.091837 47.1063009,292.826531 47.803141,294.093878 Z" id="Shape-Copy" fill="#000000" fill-rule="nonzero" transform="translate(140.145695, 220.000000) scale(-1, 1) translate(-140.145695, -220.000000) "></path>
 					</g>
 				</g>
 
@@ -419,9 +429,9 @@ footer {
 						<td>
 							<a href="{{html .URL}}">
 								{{- if .IsDir}}
-								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 35.678803 28.527945"><use xlink:href="#folder"></use></svg>
+								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 317 259"><use xlink:href="#folder{{if .IsSymlink}}-shortcut{{end}}"></use></svg>
 								{{- else}}
-								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 26.604381 29.144726"><use xlink:href="#file"></use></svg>
+								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 265 323"><use xlink:href="#file{{if .IsSymlink}}-shortcut{{end}}"></use></svg>
 								{{- end}}
 								<span class="name">{{html .Name}}</span>
 							</a>
@@ -443,6 +453,8 @@ footer {
 		</footer>
 		<script>
 			var filterEl = document.getElementById('filter');
+			filterEl.focus();
+
 			function filter() {
 				var q = filterEl.value.trim().toLowerCase();
 				var elems = document.querySelectorAll('tr.file');

caddyhttp/caddyhttp.go

@@ -16,14 +16,15 @@ import (
 	_ "github.com/mholt/caddy/caddyhttp/header"
 	_ "github.com/mholt/caddy/caddyhttp/index"
 	_ "github.com/mholt/caddy/caddyhttp/internalsrv"
+	_ "github.com/mholt/caddy/caddyhttp/limits"
 	_ "github.com/mholt/caddy/caddyhttp/log"
 	_ "github.com/mholt/caddy/caddyhttp/markdown"
-	_ "github.com/mholt/caddy/caddyhttp/maxrequestbody"
 	_ "github.com/mholt/caddy/caddyhttp/mime"
 	_ "github.com/mholt/caddy/caddyhttp/pprof"
 	_ "github.com/mholt/caddy/caddyhttp/proxy"
 	_ "github.com/mholt/caddy/caddyhttp/push"
 	_ "github.com/mholt/caddy/caddyhttp/redirect"
+	_ "github.com/mholt/caddy/caddyhttp/requestid"
 	_ "github.com/mholt/caddy/caddyhttp/rewrite"
 	_ "github.com/mholt/caddy/caddyhttp/root"
 	_ "github.com/mholt/caddy/caddyhttp/status"

caddyhttp/caddyhttp_test.go

@@ -11,7 +11,7 @@ import (
 // ensure that the standard plugins are in fact plugged in
 // and registered properly; this is a quick/naive way to do it.
 func TestStandardPlugins(t *testing.T) {
-	numStandardPlugins := 31 // importing caddyhttp plugs in this many plugins
+	numStandardPlugins := 32 // importing caddyhttp plugs in this many plugins
 	s := caddy.DescribePlugins()
 	if got, want := strings.Count(s, "\n"), numStandardPlugins+5; got != want {
 		t.Errorf("Expected all standard plugins to be plugged in, got:\n%s", s)

caddyhttp/errors/setup.go

@@ -44,18 +44,20 @@ func errorsParse(c *caddy.Controller) (*ErrorHandler, error) {
 		for c.NextBlock() {
 
 			what := c.Val()
-			if !c.NextArg() {
-				return c.ArgErr()
-			}
-			where := c.Val()
+			where := c.RemainingArgs()
 
 			if httpserver.IsLogRollerSubdirective(what) {
 				var err error
-				err = httpserver.ParseRoller(handler.Log.Roller, what, where)
+				err = httpserver.ParseRoller(handler.Log.Roller, what, where...)
 				if err != nil {
 					return err
 				}
 			} else {
+				if len(where) != 1 {
+					return c.ArgErr()
+				}
+				where := where[0]
+
 				// Error page; ensure it exists
 				if !filepath.IsAbs(where) {
 					where = filepath.Join(cfg.Root, where)

caddyhttp/errors/setup_test.go

@@ -85,13 +85,19 @@ func TestErrorsParse(t *testing.T) {
 				Roller: httpserver.DefaultLogRoller(),
 			},
 		}},
-		{`errors errors.txt { rotate_size 2 rotate_age 10 rotate_keep 3 }`, false, ErrorHandler{
+		{`errors errors.txt {
+			rotate_size 2
+			rotate_age 10
+			rotate_keep 3
+			rotate_compress
+		}`, false, ErrorHandler{
 			ErrorPages: map[int]string{},
 			Log: &httpserver.Logger{
 				Output: "errors.txt", Roller: &httpserver.LogRoller{
 					MaxSize:    2,
 					MaxAge:     10,
 					MaxBackups: 3,
+					Compress:   true,
 					LocalTime:  true,
 				},
 			},
@@ -113,6 +119,7 @@ func TestErrorsParse(t *testing.T) {
 					MaxSize:    3,
 					MaxAge:     11,
 					MaxBackups: 5,
+					Compress:   false,
 					LocalTime:  true,
 				},
 			},
@@ -142,6 +149,12 @@ func TestErrorsParse(t *testing.T) {
 				},
 				Log: &httpserver.Logger{},
 			}},
+		{`errors errors.txt { rotate_size 2 rotate_age 10 rotate_keep 3 rotate_compress }`,
+			true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
+		{`errors errors.txt {
+			rotate_compress invalid
+		}`,
+			true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
 		// Next two test cases is the detection of duplicate status codes
 		{`errors {
 			503 503.html

caddyhttp/fastcgi/dialer.go

@@ -1,102 +0,0 @@
-package fastcgi
-
-import (
-	"errors"
-	"sync"
-	"sync/atomic"
-	"time"
-)
-
-type dialer interface {
-	Dial() (Client, error)
-	Close(Client) error
-}
-
-// basicDialer is a basic dialer that wraps default fcgi functions.
-type basicDialer struct {
-	network string
-	address string
-	timeout time.Duration
-}
-
-func (b basicDialer) Dial() (Client, error) {
-	return DialTimeout(b.network, b.address, b.timeout)
-}
-
-func (b basicDialer) Close(c Client) error { return c.Close() }
-
-// persistentDialer keeps a pool of fcgi connections.
-// connections are not closed after use, rather added back to the pool for reuse.
-type persistentDialer struct {
-	size    int
-	network string
-	address string
-	timeout time.Duration
-	pool    []Client
-	sync.Mutex
-}
-
-func (p *persistentDialer) Dial() (Client, error) {
-	p.Lock()
-	// connection is available, return first one.
-	if len(p.pool) > 0 {
-		client := p.pool[0]
-		p.pool = p.pool[1:]
-		p.Unlock()
-
-		return client, nil
-	}
-
-	p.Unlock()
-
-	// no connection available, create new one
-	return DialTimeout(p.network, p.address, p.timeout)
-}
-
-func (p *persistentDialer) Close(client Client) error {
-	p.Lock()
-	if len(p.pool) < p.size {
-		// pool is not full yet, add connection for reuse
-		p.pool = append(p.pool, client)
-		p.Unlock()
-
-		return nil
-	}
-
-	p.Unlock()
-
-	// otherwise, close the connection.
-	return client.Close()
-}
-
-type loadBalancingDialer struct {
-	current int64
-	dialers []dialer
-}
-
-func (m *loadBalancingDialer) Dial() (Client, error) {
-	nextDialerIndex := atomic.AddInt64(&m.current, 1) % int64(len(m.dialers))
-	currentDialer := m.dialers[nextDialerIndex]
-
-	client, err := currentDialer.Dial()
-
-	if err != nil {
-		return nil, err
-	}
-
-	return &dialerAwareClient{Client: client, dialer: currentDialer}, nil
-}
-
-func (m *loadBalancingDialer) Close(c Client) error {
-	// Close the client according to dialer behaviour
-	if da, ok := c.(*dialerAwareClient); ok {
-		return da.dialer.Close(c)
-	}
-
-	return errors.New("Cannot close client")
-}
-
-type dialerAwareClient struct {
-	Client
-	dialer dialer
-}

caddyhttp/fastcgi/dialer_test.go

@@ -1,126 +0,0 @@
-package fastcgi
-
-import (
-	"errors"
-	"testing"
-)
-
-func TestLoadbalancingDialer(t *testing.T) {
-	// given
-	runs := 100
-	mockDialer1 := new(mockDialer)
-	mockDialer2 := new(mockDialer)
-
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1, mockDialer2}}
-
-	// when
-	for i := 0; i < runs; i++ {
-		client, err := dialer.Dial()
-		dialer.Close(client)
-
-		if err != nil {
-			t.Errorf("Expected error to be nil")
-		}
-	}
-
-	// then
-	if mockDialer1.dialCalled != mockDialer2.dialCalled && mockDialer1.dialCalled != 50 {
-		t.Errorf("Expected dialer to call Dial() on multiple backend dialers %d times [actual: %d, %d]", 50, mockDialer1.dialCalled, mockDialer2.dialCalled)
-	}
-
-	if mockDialer1.closeCalled != mockDialer2.closeCalled && mockDialer1.closeCalled != 50 {
-		t.Errorf("Expected dialer to call Close() on multiple backend dialers %d times [actual: %d, %d]", 50, mockDialer1.closeCalled, mockDialer2.closeCalled)
-	}
-}
-
-func TestLoadBalancingDialerShouldReturnDialerAwareClient(t *testing.T) {
-	// given
-	mockDialer1 := new(mockDialer)
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1}}
-
-	// when
-	client, err := dialer.Dial()
-
-	// then
-	if err != nil {
-		t.Errorf("Expected error to be nil")
-	}
-
-	if awareClient, ok := client.(*dialerAwareClient); !ok {
-		t.Error("Expected dialer to wrap client")
-	} else {
-		if awareClient.dialer != mockDialer1 {
-			t.Error("Expected wrapped client to have reference to dialer")
-		}
-	}
-}
-
-func TestLoadBalancingDialerShouldUnderlyingReturnDialerError(t *testing.T) {
-	// given
-	mockDialer1 := new(errorReturningDialer)
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1}}
-
-	// when
-	_, err := dialer.Dial()
-
-	// then
-	if err.Error() != "Error during dial" {
-		t.Errorf("Expected 'Error during dial', got: '%s'", err.Error())
-	}
-}
-
-func TestLoadBalancingDialerShouldCloseClient(t *testing.T) {
-	// given
-	mockDialer1 := new(mockDialer)
-	mockDialer2 := new(mockDialer)
-
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1, mockDialer2}}
-	client, _ := dialer.Dial()
-
-	// when
-	err := dialer.Close(client)
-
-	// then
-	if err != nil {
-		t.Error("Expected error not to occur")
-	}
-
-	// load balancing starts from index 1
-	if mockDialer2.client != client {
-		t.Errorf("Expected Close() to be called on referenced dialer")
-	}
-}
-
-type mockDialer struct {
-	dialCalled  int
-	closeCalled int
-	client      Client
-}
-
-type mockClient struct {
-	Client
-}
-
-func (m *mockDialer) Dial() (Client, error) {
-	m.dialCalled++
-	return mockClient{Client: &FCGIClient{}}, nil
-}
-
-func (m *mockDialer) Close(c Client) error {
-	m.client = c
-	m.closeCalled++
-	return nil
-}
-
-type errorReturningDialer struct {
-	client Client
-}
-
-func (m *errorReturningDialer) Dial() (Client, error) {
-	return mockClient{Client: &FCGIClient{}}, errors.New("Error during dial")
-}
-
-func (m *errorReturningDialer) Close(c Client) error {
-	m.client = c
-	return errors.New("Error during close")
-}

caddyhttp/fastcgi/fastcgi.go

@@ -4,6 +4,7 @@
 package fastcgi
 
 import (
+	"context"
 	"errors"
 	"io"
 	"net"
@@ -14,8 +15,10 @@ import (
 	"path/filepath"
 	"strconv"
 	"strings"
+	"sync/atomic"
 	"time"
 
+	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
@@ -36,9 +39,25 @@ type Handler struct {
 // ServeHTTP satisfies the httpserver.Handler interface.
 func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	for _, rule := range h.Rules {
-
-		// First requirement: Base path must match and the path must be allowed.
-		if !httpserver.Path(r.URL.Path).Matches(rule.Path) || !rule.AllowedPath(r.URL.Path) {
+		// First requirement: Base path must match request path. If it doesn't,
+		// we check to make sure the leading slash is not missing, and if so,
+		// we check again with it prepended. This is in case people forget
+		// a leading slash when performing rewrites, and we don't want to expose
+		// the contents of the (likely PHP) script. See issue #1645.
+		hpath := httpserver.Path(r.URL.Path)
+		if !hpath.Matches(rule.Path) {
+			if strings.HasPrefix(string(hpath), "/") {
+				// this is a normal-looking path, and it doesn't match; try next rule
+				continue
+			}
+			hpath = httpserver.Path("/" + string(hpath)) // prepend leading slash
+			if !hpath.Matches(rule.Path) {
+				// even after fixing the request path, it still doesn't match; try next rule
+				continue
+			}
+		}
+		// The path must also be allowed (not ignored).
+		if !rule.AllowedPath(r.URL.Path) {
 			continue
 		}
 
@@ -74,16 +93,28 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 			}
 
 			// Connect to FastCGI gateway
-			fcgiBackend, err := rule.dialer.Dial()
+			network, address := parseAddress(rule.Address())
+
+			ctx := context.Background()
+			if rule.ConnectTimeout > 0 {
+				var cancel context.CancelFunc
+				ctx, cancel = context.WithTimeout(ctx, rule.ConnectTimeout)
+				defer cancel()
+			}
+
+			fcgiBackend, err := DialContext(ctx, network, address)
 			if err != nil {
-				if err, ok := err.(net.Error); ok && err.Timeout() {
-					return http.StatusGatewayTimeout, err
-				}
 				return http.StatusBadGateway, err
 			}
 			defer fcgiBackend.Close()
-			fcgiBackend.SetReadTimeout(rule.ReadTimeout)
-			fcgiBackend.SetSendTimeout(rule.SendTimeout)
+
+			// read/write timeouts
+			if err := fcgiBackend.SetReadTimeout(rule.ReadTimeout); err != nil {
+				return http.StatusInternalServerError, err
+			}
+			if err := fcgiBackend.SetSendTimeout(rule.SendTimeout); err != nil {
+				return http.StatusInternalServerError, err
+			}
 
 			var resp *http.Response
 
@@ -105,6 +136,10 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 				resp, err = fcgiBackend.Post(env, r.Method, r.Header.Get("Content-Type"), r.Body, contentLength)
 			}
 
+			if resp != nil && resp.Body != nil {
+				defer resp.Body.Close()
+			}
+
 			if err != nil {
 				if err, ok := err.(net.Error); ok && err.Timeout() {
 					return http.StatusGatewayTimeout, err
@@ -123,9 +158,9 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 			}
 
 			// Log any stderr output from upstream
-			if stderr := fcgiBackend.StdErr(); stderr.Len() != 0 {
+			if fcgiBackend.stderr.Len() != 0 {
 				// Remove trailing newline, error logger already does this.
-				err = LogError(strings.TrimSuffix(stderr.String(), "\n"))
+				err = LogError(strings.TrimSuffix(fcgiBackend.stderr.String(), "\n"))
 			}
 
 			// Normally we would return the status code if it is an error status (>= 400),
@@ -214,6 +249,11 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 	// Strip PATH_INFO from SCRIPT_NAME
 	scriptName = strings.TrimSuffix(scriptName, pathInfo)
 
+	// Add vhost path prefix to scriptName. Otherwise, some PHP software will
+	// have difficulty discovering its URL.
+	pathPrefix, _ := r.Context().Value(caddy.CtxKey("path_prefix")).(string)
+	scriptName = path.Join(pathPrefix, scriptName)
+
 	// Get the request URI from context. The context stores the original URI in case
 	// it was changed by a middleware such as rewrite. By default, we pass the
 	// original URI in as the value of REQUEST_URI (the user can overwrite this
@@ -287,8 +327,8 @@ type Rule struct {
 	// The base path to match. Required.
 	Path string
 
-	// The address of the FastCGI server. Required.
-	Address string
+	// upstream load balancer
+	balancer
 
 	// Always process files with this extension with fastcgi.
 	Ext string
@@ -313,14 +353,35 @@ type Rule struct {
 	// Ignored paths
 	IgnoredSubPaths []string
 
+	// The duration used to set a deadline when connecting to an upstream.
+	ConnectTimeout time.Duration
+
 	// The duration used to set a deadline when reading from the FastCGI server.
 	ReadTimeout time.Duration
 
 	// The duration used to set a deadline when sending to the FastCGI server.
 	SendTimeout time.Duration
+}
 
-	// FCGI dialer
-	dialer dialer
+// balancer is a fastcgi upstream load balancer.
+type balancer interface {
+	// Address picks an upstream address from the
+	// underlying load balancer.
+	Address() string
+}
+
+// roundRobin is a round robin balancer for fastcgi upstreams.
+type roundRobin struct {
+	// Known Go bug: https://golang.org/pkg/sync/atomic/#pkg-note-BUG
+	// must be first field for 64 bit alignment
+	// on x86 and arm.
+	index     int64
+	addresses []string
+}
+
+func (r *roundRobin) Address() string {
+	index := atomic.AddInt64(&r.index, 1) % int64(len(r.addresses))
+	return r.addresses[index]
 }
 
 // canSplit checks if path can split into two based on rule.SplitPath.

caddyhttp/fastcgi/fastcgi_test.go

@@ -12,6 +12,7 @@ import (
 	"testing"
 	"time"
 
+	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
@@ -29,16 +30,9 @@ func TestServeHTTP(t *testing.T) {
 		w.Write([]byte(body))
 	}))
 
-	network, address := parseAddress(listener.Addr().String())
 	handler := Handler{
-		Next: nil,
-		Rules: []Rule{
-			{
-				Path:    "/",
-				Address: listener.Addr().String(),
-				dialer:  basicDialer{network: network, address: address},
-			},
-		},
+		Next:  nil,
+		Rules: []Rule{{Path: "/", balancer: address(listener.Addr().String())}},
 	}
 	r, err := http.NewRequest("GET", "/", nil)
 	if err != nil {
@@ -62,122 +56,25 @@ func TestServeHTTP(t *testing.T) {
 	}
 }
 
-// connectionCounter in fact is a listener with an added counter to keep track
-// of the number of accepted connections.
-type connectionCounter struct {
-	net.Listener
-	sync.Mutex
-	counter int
-}
-
-func (l *connectionCounter) Accept() (net.Conn, error) {
-	l.Lock()
-	l.counter++
-	l.Unlock()
-	return l.Listener.Accept()
-}
-
-// TestPersistent ensures that persistent
-// as well as the non-persistent fastCGI servers
-// send the answers corresnponding to the correct request.
-// It also checks the number of tcp connections used.
-func TestPersistent(t *testing.T) {
-	numberOfRequests := 32
-
-	for _, poolsize := range []int{0, 1, 5, numberOfRequests} {
-		l, err := net.Listen("tcp", "127.0.0.1:0")
-		if err != nil {
-			t.Fatalf("Unable to create listener for test: %v", err)
-		}
-
-		listener := &connectionCounter{l, *new(sync.Mutex), 0}
-
-		// this fcgi server replies with the request URL
-		go fcgi.Serve(listener, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			body := "This answers a request to " + r.URL.Path
-			bodyLenStr := strconv.Itoa(len(body))
-
-			w.Header().Set("Content-Length", bodyLenStr)
-			w.Write([]byte(body))
-		}))
-
-		network, address := parseAddress(listener.Addr().String())
-		handler := Handler{
-			Next:  nil,
-			Rules: []Rule{{Path: "/", Address: listener.Addr().String(), dialer: &persistentDialer{size: poolsize, network: network, address: address}}},
-		}
-
-		var semaphore sync.WaitGroup
-		serialMutex := new(sync.Mutex)
-
-		serialCounter := 0
-		parallelCounter := 0
-		// make some serial followed by some
-		// parallel requests to challenge the handler
-		for _, serialize := range []bool{true, false, false, false} {
-			if serialize {
-				serialCounter++
-			} else {
-				parallelCounter++
-			}
-			semaphore.Add(numberOfRequests)
-
-			for i := 0; i < numberOfRequests; i++ {
-				go func(i int, serialize bool) {
-					defer semaphore.Done()
-					if serialize {
-						serialMutex.Lock()
-						defer serialMutex.Unlock()
-					}
-					r, err := http.NewRequest("GET", "/"+strconv.Itoa(i), nil)
-					if err != nil {
-						t.Errorf("Unable to create request: %v", err)
-					}
-					ctx := context.WithValue(r.Context(), httpserver.OriginalURLCtxKey, *r.URL)
-					r = r.WithContext(ctx)
-					w := httptest.NewRecorder()
-
-					status, err := handler.ServeHTTP(w, r)
-
-					if status != 0 {
-						t.Errorf("Handler(pool: %v) return status %v", poolsize, status)
-					}
-					if err != nil {
-						t.Errorf("Handler(pool: %v) Error: %v", poolsize, err)
-					}
-					want := "This answers a request to /" + strconv.Itoa(i)
-					if got := w.Body.String(); got != want {
-						t.Errorf("Expected response from handler(pool: %v) to be '%s', got: '%s'", poolsize, want, got)
-					}
-				}(i, serialize)
-			} //next request
-			semaphore.Wait()
-		} // next set of requests (serial/parallel)
-
-		listener.Close()
-		t.Logf("The pool: %v test used %v tcp connections to answer %v * %v serial and %v * %v parallel requests.", poolsize, listener.counter, serialCounter, numberOfRequests, parallelCounter, numberOfRequests)
-	} // next handler (persistent/non-persistent)
-}
-
 func TestRuleParseAddress(t *testing.T) {
 	getClientTestTable := []struct {
 		rule            *Rule
 		expectednetwork string
 		expectedaddress string
 	}{
-		{&Rule{Address: "tcp://172.17.0.1:9000"}, "tcp", "172.17.0.1:9000"},
-		{&Rule{Address: "fastcgi://localhost:9000"}, "tcp", "localhost:9000"},
-		{&Rule{Address: "172.17.0.15"}, "tcp", "172.17.0.15"},
-		{&Rule{Address: "/my/unix/socket"}, "unix", "/my/unix/socket"},
-		{&Rule{Address: "unix:/second/unix/socket"}, "unix", "/second/unix/socket"},
+		{&Rule{balancer: address("tcp://172.17.0.1:9000")}, "tcp", "172.17.0.1:9000"},
+		{&Rule{balancer: address("fastcgi://localhost:9000")}, "tcp", "localhost:9000"},
+		{&Rule{balancer: address("172.17.0.15")}, "tcp", "172.17.0.15"},
+		{&Rule{balancer: address("/my/unix/socket")}, "unix", "/my/unix/socket"},
+		{&Rule{balancer: address("unix:/second/unix/socket")}, "unix", "/second/unix/socket"},
 	}
 
 	for _, entry := range getClientTestTable {
-		if actualnetwork, _ := parseAddress(entry.rule.Address); actualnetwork != entry.expectednetwork {
-			t.Errorf("Unexpected network for address string %v. Got %v, expected %v", entry.rule.Address, actualnetwork, entry.expectednetwork)
+		if actualnetwork, _ := parseAddress(entry.rule.Address()); actualnetwork != entry.expectednetwork {
+			t.Errorf("Unexpected network for address string %v. Got %v, expected %v", entry.rule.Address(), actualnetwork, entry.expectednetwork)
 		}
-		if _, actualaddress := parseAddress(entry.rule.Address); actualaddress != entry.expectedaddress {
-			t.Errorf("Unexpected parsed address for address string %v. Got %v, expected %v", entry.rule.Address, actualaddress, entry.expectedaddress)
+		if _, actualaddress := parseAddress(entry.rule.Address()); actualaddress != entry.expectedaddress {
+			t.Errorf("Unexpected parsed address for address string %v. Got %v, expected %v", entry.rule.Address(), actualaddress, entry.expectedaddress)
 		}
 	}
 }
@@ -226,7 +123,11 @@ func TestBuildEnv(t *testing.T) {
 		}
 	}
 
-	rule := Rule{}
+	rule := Rule{
+		Ext:        ".php",
+		SplitPath:  ".php",
+		IndexFiles: []string{"index.php"},
+	}
 	url, err := url.Parse("http://localhost:2015/fgci_test.php?test=foobar")
 	if err != nil {
 		t.Error("Unexpected error:", err.Error())
@@ -260,6 +161,7 @@ func TestBuildEnv(t *testing.T) {
 			"QUERY_STRING":    "test=foobar",
 			"REQUEST_METHOD":  "GET",
 			"HTTP_HOST":       "localhost:2015",
+			"SCRIPT_NAME":     "/fgci_test.php",
 		}
 	}
 
@@ -310,6 +212,14 @@ func TestBuildEnv(t *testing.T) {
 	envExpected["CUSTOM_URI"] = "custom_uri/fgci_test.php?test=foobar"
 	envExpected["CUSTOM_QUERY"] = "custom=true&test=foobar"
 	testBuildEnv(r, rule, fpath, envExpected)
+
+	// 6. Test SCRIPT_NAME includes path prefix
+	r = newReq()
+	ctx := context.WithValue(r.Context(), caddy.CtxKey("path_prefix"), "/test")
+	r = r.WithContext(ctx)
+	envExpected = newEnv()
+	envExpected["SCRIPT_NAME"] = "/test/fgci_test.php"
+	testBuildEnv(r, rule, fpath, envExpected)
 }
 
 func TestReadTimeout(t *testing.T) {
@@ -332,14 +242,12 @@ func TestReadTimeout(t *testing.T) {
 		}
 		defer listener.Close()
 
-		network, address := parseAddress(listener.Addr().String())
 		handler := Handler{
 			Next: nil,
 			Rules: []Rule{
 				{
 					Path:        "/",
-					Address:     listener.Addr().String(),
-					dialer:      basicDialer{network: network, address: address},
+					balancer:    address(listener.Addr().String()),
 					ReadTimeout: test.readTimeout,
 				},
 			},
@@ -394,14 +302,12 @@ func TestSendTimeout(t *testing.T) {
 		}
 		defer listener.Close()
 
-		network, address := parseAddress(listener.Addr().String())
 		handler := Handler{
 			Next: nil,
 			Rules: []Rule{
 				{
 					Path:        "/",
-					Address:     listener.Addr().String(),
-					dialer:      basicDialer{network: network, address: address},
+					balancer:    address(listener.Addr().String()),
 					SendTimeout: test.sendTimeout,
 				},
 			},
@@ -434,3 +340,28 @@ func TestSendTimeout(t *testing.T) {
 		}
 	}
 }
+
+func TestBalancer(t *testing.T) {
+	tests := [][]string{
+		{"localhost", "host.local"},
+		{"localhost"},
+		{"localhost", "host.local", "example.com"},
+		{"localhost", "host.local", "example.com", "127.0.0.1"},
+	}
+	for i, test := range tests {
+		b := address(test...)
+		for _, host := range test {
+			a := b.Address()
+			if a != host {
+				t.Errorf("Test %d: expected %s, found %s", i, host, a)
+			}
+		}
+	}
+}
+
+func address(addresses ...string) balancer {
+	return &roundRobin{
+		addresses: addresses,
+		index:     -1,
+	}
+}

caddyhttp/fastcgi/fcgiclient.go

@@ -13,6 +13,7 @@ package fastcgi
 import (
 	"bufio"
 	"bytes"
+	"context"
 	"encoding/binary"
 	"errors"
 	"io"
@@ -107,18 +108,6 @@ const (
 	maxPad   = 255
 )
 
-// Client interface
-type Client interface {
-	Get(pair map[string]string) (response *http.Response, err error)
-	Head(pair map[string]string) (response *http.Response, err error)
-	Options(pairs map[string]string) (response *http.Response, err error)
-	Post(pairs map[string]string, method string, bodyType string, body io.Reader, contentLength int64) (response *http.Response, err error)
-	Close() error
-	StdErr() bytes.Buffer
-	SetReadTimeout(time.Duration) error
-	SetSendTimeout(time.Duration) error
-}
-
 type header struct {
 	Version       uint8
 	Type          uint8
@@ -150,7 +139,7 @@ func (rec *record) read(r io.Reader) (buf []byte, err error) {
 		return
 	}
 	if rec.h.Version != 1 {
-		err = errInvalidHeaderVersion
+		err = errors.New("fcgi: invalid header version")
 		return
 	}
 	if rec.h.Type == EndRequest {
@@ -173,7 +162,7 @@ func (rec *record) read(r io.Reader) (buf []byte, err error) {
 // interfacing external applications with Web servers.
 type FCGIClient struct {
 	mutex       sync.Mutex
-	conn        net.Conn
+	rwc         io.ReadWriteCloser
 	h           header
 	buf         bytes.Buffer
 	stderr      bytes.Buffer
@@ -183,53 +172,57 @@ type FCGIClient struct {
 	sendTimeout time.Duration
 }
 
-// DialTimeout connects to the fcgi responder at the specified network address, using default net.Dialer.
+// DialWithDialerContext connects to the fcgi responder at the specified network address, using custom net.Dialer
+// and a context.
 // See func net.Dial for a description of the network and address parameters.
-func DialTimeout(network string, address string, timeout time.Duration) (fcgi *FCGIClient, err error) {
-	conn, err := net.DialTimeout(network, address, timeout)
+func DialWithDialerContext(ctx context.Context, network, address string, dialer net.Dialer) (fcgi *FCGIClient, err error) {
+	var conn net.Conn
+	conn, err = dialer.DialContext(ctx, network, address)
 	if err != nil {
 		return
 	}
 
-	fcgi = &FCGIClient{conn: conn, keepAlive: false, reqID: 1}
+	fcgi = &FCGIClient{
+		rwc:       conn,
+		keepAlive: false,
+		reqID:     1,
+	}
 
-	return fcgi, nil
+	return
 }
 
-// Close closes fcgi connnection.
-func (c *FCGIClient) Close() error {
-	return c.conn.Close()
+// DialContext is like Dial but passes ctx to dialer.Dial.
+func DialContext(ctx context.Context, network, address string) (fcgi *FCGIClient, err error) {
+	return DialWithDialerContext(ctx, network, address, net.Dialer{})
 }
 
-func (c *FCGIClient) writeRecord(recType uint8, content []byte) error {
+// Dial connects to the fcgi responder at the specified network address, using default net.Dialer.
+// See func net.Dial for a description of the network and address parameters.
+func Dial(network, address string) (fcgi *FCGIClient, err error) {
+	return DialContext(context.Background(), network, address)
+}
+
+// Close closes fcgi connnection
+func (c *FCGIClient) Close() {
+	c.rwc.Close()
+}
+
+func (c *FCGIClient) writeRecord(recType uint8, content []byte) (err error) {
 	c.mutex.Lock()
 	defer c.mutex.Unlock()
 	c.buf.Reset()
 	c.h.init(recType, c.reqID, len(content))
-
 	if err := binary.Write(&c.buf, binary.BigEndian, c.h); err != nil {
 		return err
 	}
-
 	if _, err := c.buf.Write(content); err != nil {
 		return err
 	}
-
 	if _, err := c.buf.Write(pad[:c.h.PaddingLength]); err != nil {
 		return err
 	}
-
-	if c.sendTimeout != 0 {
-		if err := c.conn.SetWriteDeadline(time.Now().Add(c.sendTimeout)); err != nil {
-			return err
-		}
-	}
-
-	if _, err := c.conn.Write(c.buf.Bytes()); err != nil {
-		return err
-	}
-
-	return nil
+	_, err = c.rwc.Write(c.buf.Bytes())
+	return err
 }
 
 func (c *FCGIClient) writeBeginRequest(role uint16, flags uint8) error {
@@ -345,14 +338,13 @@ func (w *streamReader) Read(p []byte) (n int, err error) {
 
 	if len(p) > 0 {
 		if len(w.buf) == 0 {
+
 			// filter outputs for error log
 			for {
 				rec := &record{}
 				var buf []byte
-				buf, err = rec.read(w.c.conn)
-				if err == errInvalidHeaderVersion {
-					continue
-				} else if err != nil {
+				buf, err = rec.read(w.c.rwc)
+				if err != nil {
 					return
 				}
 				// standard error output
@@ -376,15 +368,10 @@ func (w *streamReader) Read(p []byte) (n int, err error) {
 	return
 }
 
-// StdErr returns stderr stream
-func (c *FCGIClient) StdErr() bytes.Buffer {
-	return c.stderr
-}
-
 // Do made the request and returns a io.Reader that translates the data read
 // from fcgi responder out of fcgi packet before returning it.
 func (c *FCGIClient) Do(p map[string]string, req io.Reader) (r io.Reader, err error) {
-	err = c.writeBeginRequest(uint16(Responder), FCGIKeepConn)
+	err = c.writeBeginRequest(uint16(Responder), 0)
 	if err != nil {
 		return
 	}
@@ -407,11 +394,11 @@ func (c *FCGIClient) Do(p map[string]string, req io.Reader) (r io.Reader, err er
 // clientCloser is a io.ReadCloser. It wraps a io.Reader with a Closer
 // that closes FCGIClient connection.
 type clientCloser struct {
-	f *FCGIClient
+	*FCGIClient
 	io.Reader
 }
 
-func (c clientCloser) Close() error { return c.f.Close() }
+func (f clientCloser) Close() error { return f.rwc.Close() }
 
 // Request returns a HTTP Response with Header and Body
 // from fcgi responder
@@ -425,12 +412,6 @@ func (c *FCGIClient) Request(p map[string]string, req io.Reader) (resp *http.Res
 	tp := textproto.NewReader(rb)
 	resp = new(http.Response)
 
-	if c.readTimeout != 0 {
-		if err = c.conn.SetReadDeadline(time.Now().Add(c.readTimeout)); err != nil {
-			return
-		}
-	}
-
 	// Parse the response headers.
 	mimeHeader, err := tp.ReadMIMEHeader()
 	if err != nil && err != io.EOF {
@@ -566,18 +547,20 @@ func (c *FCGIClient) PostFile(p map[string]string, data url.Values, file map[str
 // SetReadTimeout sets the read timeout for future calls that read from the
 // fcgi responder. A zero value for t means no timeout will be set.
 func (c *FCGIClient) SetReadTimeout(t time.Duration) error {
-	c.readTimeout = t
+	if conn, ok := c.rwc.(net.Conn); ok && t != 0 {
+		return conn.SetReadDeadline(time.Now().Add(t))
+	}
 	return nil
 }
 
 // SetSendTimeout sets the read timeout for future calls that send data to
 // the fcgi responder. A zero value for t means no timeout will be set.
 func (c *FCGIClient) SetSendTimeout(t time.Duration) error {
-	c.sendTimeout = t
+	if conn, ok := c.rwc.(net.Conn); ok && t != 0 {
+		return conn.SetWriteDeadline(time.Now().Add(t))
+	}
 	return nil
 }
 
 // Checks whether chunked is part of the encodings stack
 func chunked(te []string) bool { return len(te) > 0 && te[0] == "chunked" }
-
-var errInvalidHeaderVersion = errors.New("fcgi: invalid header version")

caddyhttp/fastcgi/fcgiclient_test.go

@@ -103,7 +103,7 @@ func (s FastCGIServer) ServeHTTP(resp http.ResponseWriter, req *http.Request) {
 }
 
 func sendFcgi(reqType int, fcgiParams map[string]string, data []byte, posts map[string]string, files map[string]string) (content []byte) {
-	fcgi, err := DialTimeout("tcp", ipPort, 0)
+	fcgi, err := Dial("tcp", ipPort)
 	if err != nil {
 		log.Println("err:", err)
 		return

caddyhttp/fastcgi/setup.go

@@ -4,8 +4,6 @@ import (
 	"errors"
 	"net/http"
 	"path/filepath"
-	"strconv"
-	"strings"
 	"time"
 
 	"github.com/mholt/caddy"
@@ -61,10 +59,8 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 		}
 
 		rule := Rule{
-			Root:        absRoot,
-			Path:        args[0],
-			ReadTimeout: 60 * time.Second,
-			SendTimeout: 60 * time.Second,
+			Root: absRoot,
+			Path: args[0],
 		}
 		upstreams := []string{args[1]}
 
@@ -75,10 +71,6 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 		}
 
 		var err error
-		var pool int
-		var connectTimeout = 60 * time.Second
-		var dialers []dialer
-		var poolSize = -1
 
 		for c.NextBlock() {
 			switch c.Val() {
@@ -126,24 +118,11 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 				}
 				rule.IgnoredSubPaths = ignoredPaths
 
-			case "pool":
-				if !c.NextArg() {
-					return rules, c.ArgErr()
-				}
-				pool, err = strconv.Atoi(c.Val())
-				if err != nil {
-					return rules, err
-				}
-				if pool >= 0 {
-					poolSize = pool
-				} else {
-					return rules, c.Errf("positive integer expected, found %d", pool)
-				}
 			case "connect_timeout":
 				if !c.NextArg() {
 					return rules, c.ArgErr()
 				}
-				connectTimeout, err = time.ParseDuration(c.Val())
+				rule.ConnectTimeout, err = time.ParseDuration(c.Val())
 				if err != nil {
 					return rules, err
 				}
@@ -168,29 +147,10 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 			}
 		}
 
-		for _, rawAddress := range upstreams {
-			network, address := parseAddress(rawAddress)
-			if poolSize >= 0 {
-				dialers = append(dialers, &persistentDialer{
-					size:    poolSize,
-					network: network,
-					address: address,
-					timeout: connectTimeout,
-				})
-			} else {
-				dialers = append(dialers, basicDialer{
-					network: network,
-					address: address,
-					timeout: connectTimeout,
-				})
-			}
-		}
+		rule.balancer = &roundRobin{addresses: upstreams, index: -1}
 
-		rule.dialer = &loadBalancingDialer{dialers: dialers}
-		rule.Address = strings.Join(upstreams, ",")
 		rules = append(rules, rule)
 	}
-
 	return rules, nil
 }
 

caddyhttp/fastcgi/setup_test.go

@@ -2,10 +2,7 @@ package fastcgi
 
 import (
 	"fmt"
-	"os"
-	"reflect"
 	"testing"
-	"time"
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -32,45 +29,13 @@ func TestSetup(t *testing.T) {
 	if myHandler.Rules[0].Path != "/" {
 		t.Errorf("Expected / as the Path")
 	}
-	if myHandler.Rules[0].Address != "127.0.0.1:9000" {
+	if myHandler.Rules[0].Address() != "127.0.0.1:9000" {
 		t.Errorf("Expected 127.0.0.1:9000 as the Address")
 	}
 
 }
 
-func (p *persistentDialer) Equals(q *persistentDialer) bool {
-	if p.size != q.size {
-		return false
-	}
-	if p.network != q.network {
-		return false
-	}
-	if p.address != q.address {
-		return false
-	}
-
-	if len(p.pool) != len(q.pool) {
-		return false
-	}
-	for i, client := range p.pool {
-		if client != q.pool[i] {
-			return false
-		}
-	}
-	// ignore mutex state
-	return true
-}
-
 func TestFastcgiParse(t *testing.T) {
-	rootPath, err := os.Getwd()
-	if err != nil {
-		t.Errorf("Can't determine current working directory; got '%v'", err)
-	}
-
-	defaultAddress := "127.0.0.1:9001"
-	network, address := parseAddress(defaultAddress)
-	t.Logf("Address '%v' was parsed to network '%v' and address '%v'", defaultAddress, network, address)
-
 	tests := []struct {
 		inputFastcgiConfig    string
 		shouldErr             bool
@@ -79,193 +44,34 @@ func TestFastcgiParse(t *testing.T) {
 
 		{`fastcgi /blog 127.0.0.1:9000 php`,
 			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000",
-				Ext:         ".php",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{"index.php"},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
+				Path:       "/blog",
+				balancer:   &roundRobin{addresses: []string{"127.0.0.1:9000"}},
+				Ext:        ".php",
+				SplitPath:  ".php",
+				IndexFiles: []string{"index.php"},
 			}}},
-		{`fastcgi /blog 127.0.0.1:9000 php {
-			root /tmp
-		}`,
-			false, []Rule{{
-				Root:        "/tmp",
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000",
-				Ext:         ".php",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{"index.php"},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi /blog 127.0.0.1:9000 php {
-			upstream 127.0.0.1:9001
-		}`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000,127.0.0.1:9001",
-				Ext:         ".php",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}, basicDialer{network: "tcp", address: "127.0.0.1:9001", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{"index.php"},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi /blog 127.0.0.1:9000 {
-			upstream 127.0.0.1:9001 
-		}`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000,127.0.0.1:9001",
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}, basicDialer{network: "tcp", address: "127.0.0.1:9001", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / ` + defaultAddress + ` {
+		{`fastcgi / 127.0.0.1:9001 {
 	              split .html
 	              }`,
 			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   ".html",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
+				Path:       "/",
+				balancer:   &roundRobin{addresses: []string{"127.0.0.1:9001"}},
+				Ext:        "",
+				SplitPath:  ".html",
+				IndexFiles: []string{},
 			}}},
-		{`fastcgi / ` + defaultAddress + ` {
+		{`fastcgi / 127.0.0.1:9001 {
 	              split .html
 	              except /admin /user
 	              }`,
 			false, []Rule{{
-				Root:            rootPath,
 				Path:            "/",
-				Address:         "127.0.0.1:9001",
+				balancer:        &roundRobin{addresses: []string{"127.0.0.1:9001"}},
 				Ext:             "",
 				SplitPath:       ".html",
-				dialer:          &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
 				IndexFiles:      []string{},
 				IgnoredSubPaths: []string{"/admin", "/user"},
-				ReadTimeout:     60 * time.Second,
-				SendTimeout:     60 * time.Second,
 			}}},
-		{`fastcgi / ` + defaultAddress + ` {
-	              pool 0
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{&persistentDialer{size: 0, network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / 127.0.0.1:8080  {
-			upstream 127.0.0.1:9000
-	              pool 5
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     "127.0.0.1:8080,127.0.0.1:9000",
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{&persistentDialer{size: 5, network: "tcp", address: "127.0.0.1:8080", timeout: 60 * time.Second}, &persistentDialer{size: 5, network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / ` + defaultAddress + ` {
-	              split .php
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / ` + defaultAddress + ` {
-	              connect_timeout 5s
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 5 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{
-			`fastcgi / ` + defaultAddress + ` { connect_timeout BADVALUE }`,
-			true,
-			[]Rule{},
-		},
-		{`fastcgi / ` + defaultAddress + ` {
-	              read_timeout 5s
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 5 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{
-			`fastcgi / ` + defaultAddress + ` { read_timeout BADVALUE }`,
-			true,
-			[]Rule{},
-		},
-		{`fastcgi / ` + defaultAddress + ` {
-	              send_timeout 5s
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 5 * time.Second,
-			}}},
-		{
-			`fastcgi / ` + defaultAddress + ` { send_timeout BADVALUE }`,
-			true,
-			[]Rule{},
-		},
-		{`fastcgi / {
-
-		              }`,
-			true, []Rule{},
-		},
 	}
 	for i, test := range tests {
 		actualFastcgiConfigs, err := fastcgiParse(caddy.NewTestController("http", test.inputFastcgiConfig))
@@ -281,19 +87,14 @@ func TestFastcgiParse(t *testing.T) {
 		}
 		for j, actualFastcgiConfig := range actualFastcgiConfigs {
 
-			if actualFastcgiConfig.Root != test.expectedFastcgiConfig[j].Root {
-				t.Errorf("Test %d expected %dth FastCGI Root to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].Root, actualFastcgiConfig.Root)
-			}
-
 			if actualFastcgiConfig.Path != test.expectedFastcgiConfig[j].Path {
 				t.Errorf("Test %d expected %dth FastCGI Path to be  %s  , but got %s",
 					i, j, test.expectedFastcgiConfig[j].Path, actualFastcgiConfig.Path)
 			}
 
-			if actualFastcgiConfig.Address != test.expectedFastcgiConfig[j].Address {
+			if actualFastcgiConfig.Address() != test.expectedFastcgiConfig[j].Address() {
 				t.Errorf("Test %d expected %dth FastCGI Address to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].Address, actualFastcgiConfig.Address)
+					i, j, test.expectedFastcgiConfig[j].Address(), actualFastcgiConfig.Address())
 			}
 
 			if actualFastcgiConfig.Ext != test.expectedFastcgiConfig[j].Ext {
@@ -306,16 +107,6 @@ func TestFastcgiParse(t *testing.T) {
 					i, j, test.expectedFastcgiConfig[j].SplitPath, actualFastcgiConfig.SplitPath)
 			}
 
-			if reflect.TypeOf(actualFastcgiConfig.dialer) != reflect.TypeOf(test.expectedFastcgiConfig[j].dialer) {
-				t.Errorf("Test %d expected %dth FastCGI dialer to be of type %T, but got %T",
-					i, j, test.expectedFastcgiConfig[j].dialer, actualFastcgiConfig.dialer)
-			} else {
-				if !areDialersEqual(actualFastcgiConfig.dialer, test.expectedFastcgiConfig[j].dialer, t) {
-					t.Errorf("Test %d expected %dth FastCGI dialer to be %v, but got %v",
-						i, j, test.expectedFastcgiConfig[j].dialer, actualFastcgiConfig.dialer)
-				}
-			}
-
 			if fmt.Sprint(actualFastcgiConfig.IndexFiles) != fmt.Sprint(test.expectedFastcgiConfig[j].IndexFiles) {
 				t.Errorf("Test %d expected %dth FastCGI IndexFiles to be  %s  , but got %s",
 					i, j, test.expectedFastcgiConfig[j].IndexFiles, actualFastcgiConfig.IndexFiles)
@@ -325,43 +116,7 @@ func TestFastcgiParse(t *testing.T) {
 				t.Errorf("Test %d expected %dth FastCGI IgnoredSubPaths to be  %s  , but got %s",
 					i, j, test.expectedFastcgiConfig[j].IgnoredSubPaths, actualFastcgiConfig.IgnoredSubPaths)
 			}
-
-			if fmt.Sprint(actualFastcgiConfig.ReadTimeout) != fmt.Sprint(test.expectedFastcgiConfig[j].ReadTimeout) {
-				t.Errorf("Test %d expected %dth FastCGI ReadTimeout to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].ReadTimeout, actualFastcgiConfig.ReadTimeout)
-			}
-
-			if fmt.Sprint(actualFastcgiConfig.SendTimeout) != fmt.Sprint(test.expectedFastcgiConfig[j].SendTimeout) {
-				t.Errorf("Test %d expected %dth FastCGI SendTimeout to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].SendTimeout, actualFastcgiConfig.SendTimeout)
-			}
 		}
 	}
-}
-
-func areDialersEqual(current, expected dialer, t *testing.T) bool {
-
-	switch actual := current.(type) {
-	case *loadBalancingDialer:
-		if expected, ok := expected.(*loadBalancingDialer); ok {
-			for i := 0; i < len(actual.dialers); i++ {
-				if !areDialersEqual(actual.dialers[i], expected.dialers[i], t) {
-					return false
-				}
-			}
-
-			return true
-		}
-	case basicDialer:
-		return current == expected
-	case *persistentDialer:
-		if expected, ok := expected.(*persistentDialer); ok {
-			return actual.Equals(expected)
-		}
-
-	default:
-		t.Errorf("Unknown dialer type %T", current)
-	}
-
-	return false
+
 }

caddyhttp/gzip/gzip.go

@@ -3,9 +3,7 @@
 package gzip
 
 import (
-	"bufio"
 	"io"
-	"net"
 	"net/http"
 	"strings"
 
@@ -58,7 +56,10 @@ outer:
 		// original form.
 		gzipWriter := getWriter(c.Level)
 		defer putWriter(c.Level, gzipWriter)
-		gz := &gzipResponseWriter{Writer: gzipWriter, ResponseWriter: w}
+		gz := &gzipResponseWriter{
+			Writer:                gzipWriter,
+			ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
+		}
 
 		var rw http.ResponseWriter
 		// if no response filter is used
@@ -92,7 +93,7 @@ outer:
 // with a gzip.Writer to compress the output.
 type gzipResponseWriter struct {
 	io.Writer
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 	statusCodeWritten bool
 }
 
@@ -104,7 +105,11 @@ func (w *gzipResponseWriter) WriteHeader(code int) {
 	w.Header().Del("Content-Length")
 	w.Header().Set("Content-Encoding", "gzip")
 	w.Header().Add("Vary", "Accept-Encoding")
-	w.ResponseWriter.WriteHeader(code)
+	originalEtag := w.Header().Get("ETag")
+	if originalEtag != "" && !strings.HasPrefix(originalEtag, "W/") {
+		w.Header().Set("ETag", "W/"+originalEtag)
+	}
+	w.ResponseWriterWrapper.WriteHeader(code)
 	w.statusCodeWritten = true
 }
 
@@ -120,44 +125,5 @@ func (w *gzipResponseWriter) Write(b []byte) (int, error) {
 	return n, err
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (w *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := w.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: w.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (w *gzipResponseWriter) Flush() {
-	if f, ok := w.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: w.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (w *gzipResponseWriter) CloseNotify() <-chan bool {
-	if cn, ok := w.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: w.ResponseWriter})
-}
-
-func (w *gzipResponseWriter) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := w.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return httpserver.NonFlusherError{Underlying: w.ResponseWriter}
-}
-
 // Interface guards
-var _ http.Pusher = (*gzipResponseWriter)(nil)
-var _ http.Flusher = (*gzipResponseWriter)(nil)
-var _ http.CloseNotifier = (*gzipResponseWriter)(nil)
-var _ http.Hijacker = (*gzipResponseWriter)(nil)
+var _ httpserver.HTTPInterfaces = (*gzipResponseWriter)(nil)

caddyhttp/gzip/gzip_test.go

@@ -38,6 +38,14 @@ func TestGzipHandler(t *testing.T) {
 			t.Error(err)
 		}
 		r.Header.Set("Accept-Encoding", "gzip")
+		w.Header().Set("ETag", `"2n9cd"`)
+		_, err = gz.ServeHTTP(w, r)
+		if err != nil {
+			t.Error(err)
+		}
+
+		// The second pass, test if the ETag is already weak
+		w.Header().Set("ETag", `W/"2n9cd"`)
 		_, err = gz.ServeHTTP(w, r)
 		if err != nil {
 			t.Error(err)
@@ -109,10 +117,14 @@ func nextFunc(shouldGzip bool) httpserver.Handler {
 
 		if shouldGzip {
 			if w.Header().Get("Content-Encoding") != "gzip" {
-				return 0, fmt.Errorf("Content-Encoding must be gzip, found %v", r.Header.Get("Content-Encoding"))
+				return 0, fmt.Errorf("Content-Encoding must be gzip, found %v", w.Header().Get("Content-Encoding"))
 			}
 			if w.Header().Get("Vary") != "Accept-Encoding" {
-				return 0, fmt.Errorf("Vary must be Accept-Encoding, found %v", r.Header.Get("Vary"))
+				return 0, fmt.Errorf("Vary must be Accept-Encoding, found %v", w.Header().Get("Vary"))
+			}
+			etag := w.Header().Get("ETag")
+			if etag != "" && etag != `W/"2n9cd"` {
+				return 0, fmt.Errorf("ETag must be converted to weak Etag, found %v", w.Header().Get("ETag"))
 			}
 			if _, ok := w.(*gzipResponseWriter); !ok {
 				return 0, fmt.Errorf("ResponseWriter should be gzipResponseWriter, found %T", w)

caddyhttp/gzip/responsefilter_test.go

@@ -33,7 +33,7 @@ func TestLengthFilter(t *testing.T) {
 		for j, filter := range filters {
 			r := httptest.NewRecorder()
 			r.Header().Set("Content-Length", fmt.Sprint(ts.length))
-			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), r, false})
+			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), &httpserver.ResponseWriterWrapper{ResponseWriter: r}, false})
 			if filter.ShouldCompress(wWriter) != ts.shouldCompress[j] {
 				t.Errorf("Test %v: Expected %v found %v", i, ts.shouldCompress[j], filter.ShouldCompress(r))
 			}

caddyhttp/gzip/setup.go

@@ -146,11 +146,7 @@ func initWriterPool() {
 
 	// add default writer pool
 	defaultWriterPoolIndex = i
-	writerPool[defaultWriterPoolIndex] = &sync.Pool{
-		New: func() interface{} {
-			return gzip.NewWriter(ioutil.Discard)
-		},
-	}
+	writerPool[defaultWriterPoolIndex] = newWriterPool(gzip.DefaultCompression)
 }
 
 func getWriter(level int) *gzip.Writer {

caddyhttp/header/header.go

@@ -4,8 +4,6 @@
 package header
 
 import (
-	"bufio"
-	"net"
 	"net/http"
 	"strings"
 
@@ -23,10 +21,16 @@ type Headers struct {
 // setting headers on the response according to the configured rules.
 func (h Headers) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	replacer := httpserver.NewReplacer(r, nil, "")
-	rww := &responseWriterWrapper{ResponseWriter: w}
+	rww := &responseWriterWrapper{
+		ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
+	}
 	for _, rule := range h.Rules {
 		if httpserver.Path(r.URL.Path).Matches(rule.Path) {
 			for name := range rule.Headers {
+				if name == "Caddy-Sponsors" || name == "-Caddy-Sponsors" {
+					// see EULA
+					continue
+				}
 
 				// One can either delete a header, add multiple values to a header, or simply
 				// set a header.
@@ -62,20 +66,20 @@ type headerOperation func(http.Header)
 // responseWriterWrapper wraps the real ResponseWriter.
 // It defers header operations until writeHeader
 type responseWriterWrapper struct {
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 	ops         []headerOperation
 	wroteHeader bool
 }
 
 func (rww *responseWriterWrapper) Header() http.Header {
-	return rww.ResponseWriter.Header()
+	return rww.ResponseWriterWrapper.Header()
 }
 
 func (rww *responseWriterWrapper) Write(d []byte) (int, error) {
 	if !rww.wroteHeader {
 		rww.WriteHeader(http.StatusOK)
 	}
-	return rww.ResponseWriter.Write(d)
+	return rww.ResponseWriterWrapper.Write(d)
 }
 
 func (rww *responseWriterWrapper) WriteHeader(status int) {
@@ -91,7 +95,7 @@ func (rww *responseWriterWrapper) WriteHeader(status int) {
 		op(h)
 	}
 
-	rww.ResponseWriter.WriteHeader(status)
+	rww.ResponseWriterWrapper.WriteHeader(status)
 }
 
 // delHeader deletes the existing header according to the key
@@ -106,45 +110,5 @@ func (rww *responseWriterWrapper) delHeader(key string) {
 	})
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (rww *responseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := rww.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: rww.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (rww *responseWriterWrapper) Flush() {
-	if f, ok := rww.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: rww.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-// It panics if the underlying ResponseWriter is not a CloseNotifier.
-func (rww *responseWriterWrapper) CloseNotify() <-chan bool {
-	if cn, ok := rww.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: rww.ResponseWriter})
-}
-
-func (rww *responseWriterWrapper) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := rww.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return httpserver.NonPusherError{Underlying: rww.ResponseWriter}
-}
-
 // Interface guards
-var _ http.Pusher = (*responseWriterWrapper)(nil)
-var _ http.Flusher = (*responseWriterWrapper)(nil)
-var _ http.CloseNotifier = (*responseWriterWrapper)(nil)
-var _ http.Hijacker = (*responseWriterWrapper)(nil)
+var _ httpserver.HTTPInterfaces = (*responseWriterWrapper)(nil)

caddyhttp/httpserver/condition.go

@@ -48,115 +48,95 @@ const (
 	isOp         = "is"
 	notOp        = "not"
 	hasOp        = "has"
-	notHasOp     = "not_has"
 	startsWithOp = "starts_with"
 	endsWithOp   = "ends_with"
 	matchOp      = "match"
-	notMatchOp   = "not_match"
 )
 
-func operatorError(operator string) error {
-	return fmt.Errorf("Invalid operator %v", operator)
+// ifCondition is a 'if' condition.
+type ifFunc func(a, b string) bool
+
+// ifCond is statement for a IfMatcher condition.
+type ifCond struct {
+	a   string
+	op  string
+	b   string
+	neg bool
+	rex *regexp.Regexp
+	f   ifFunc
 }
 
-// ifCondition is a 'if' condition.
-type ifCondition func(string, string) bool
+// newIfCond creates a new If condition.
+func newIfCond(a, op, b string) (ifCond, error) {
+	i := ifCond{a: a, op: op, b: b}
+	if strings.HasPrefix(op, "not_") {
+		i.neg = true
+		i.op = op[4:]
+	}
 
-var ifConditions = map[string]ifCondition{
-	isOp:         isFunc,
-	notOp:        notFunc,
-	hasOp:        hasFunc,
-	notHasOp:     notHasFunc,
-	startsWithOp: startsWithFunc,
-	endsWithOp:   endsWithFunc,
-	matchOp:      matchFunc,
-	notMatchOp:   notMatchFunc,
+	switch i.op {
+	case isOp:
+		// It checks for equality.
+		i.f = i.isFunc
+	case notOp:
+		// It checks for inequality.
+		i.f = i.notFunc
+	case hasOp:
+		// It checks if b is a substring of a.
+		i.f = strings.Contains
+	case startsWithOp:
+		// It checks if b is a prefix of a.
+		i.f = strings.HasPrefix
+	case endsWithOp:
+		// It checks if b is a suffix of a.
+		i.f = strings.HasSuffix
+	case matchOp:
+		// It does regexp matching of a against pattern in b and returns if they match.
+		var err error
+		if i.rex, err = regexp.Compile(i.b); err != nil {
+			return ifCond{}, fmt.Errorf("Invalid regular expression: '%s', %v", i.b, err)
+		}
+		i.f = i.matchFunc
+	default:
+		return ifCond{}, fmt.Errorf("Invalid operator %v", i.op)
+	}
+
+	return i, nil
 }
 
 // isFunc is condition for Is operator.
-// It checks for equality.
-func isFunc(a, b string) bool {
+func (i ifCond) isFunc(a, b string) bool {
 	return a == b
 }
 
 // notFunc is condition for Not operator.
-// It checks for inequality.
-func notFunc(a, b string) bool {
+func (i ifCond) notFunc(a, b string) bool {
 	return a != b
 }
 
-// hasFunc is condition for Has operator.
-// It checks if b is a substring of a.
-func hasFunc(a, b string) bool {
-	return strings.Contains(a, b)
-}
-
-// notHasFunc is condition for NotHas operator.
-// It checks if b is not a substring of a.
-func notHasFunc(a, b string) bool {
-	return !strings.Contains(a, b)
-}
-
-// startsWithFunc is condition for StartsWith operator.
-// It checks if b is a prefix of a.
-func startsWithFunc(a, b string) bool {
-	return strings.HasPrefix(a, b)
-}
-
-// endsWithFunc is condition for EndsWith operator.
-// It checks if b is a suffix of a.
-func endsWithFunc(a, b string) bool {
-	return strings.HasSuffix(a, b)
-}
-
 // matchFunc is condition for Match operator.
-// It does regexp matching of a against pattern in b
-// and returns if they match.
-func matchFunc(a, b string) bool {
-	matched, _ := regexp.MatchString(b, a)
-	return matched
-}
-
-// notMatchFunc is condition for NotMatch operator.
-// It does regexp matching of a against pattern in b
-// and returns if they do not match.
-func notMatchFunc(a, b string) bool {
-	matched, _ := regexp.MatchString(b, a)
-	return !matched
-}
-
-// ifCond is statement for a IfMatcher condition.
-type ifCond struct {
-	a  string
-	op string
-	b  string
-}
-
-// newIfCond creates a new If condition.
-func newIfCond(a, operator, b string) (ifCond, error) {
-	if _, ok := ifConditions[operator]; !ok {
-		return ifCond{}, operatorError(operator)
-	}
-	return ifCond{
-		a:  a,
-		op: operator,
-		b:  b,
-	}, nil
+func (i ifCond) matchFunc(a, b string) bool {
+	return i.rex.MatchString(a)
 }
 
 // True returns true if the condition is true and false otherwise.
 // If r is not nil, it replaces placeholders before comparison.
 func (i ifCond) True(r *http.Request) bool {
-	if c, ok := ifConditions[i.op]; ok {
+	if i.f != nil {
 		a, b := i.a, i.b
 		if r != nil {
 			replacer := NewReplacer(r, nil, "")
 			a = replacer.Replace(i.a)
-			b = replacer.Replace(i.b)
+			if i.op != matchOp {
+				b = replacer.Replace(i.b)
+			}
 		}
-		return c(a, b)
+		if i.neg {
+			return !i.f(a, b)
+		}
+		return i.f(a, b)
 	}
-	return false
+	return i.neg // false if not negated, true otherwise
 }
 
 // IfMatcher is a RequestMatcher for 'if' conditions.

caddyhttp/httpserver/condition_test.go

@@ -2,8 +2,8 @@ package httpserver
 
 import (
 	"context"
-	"fmt"
 	"net/http"
+	"regexp"
 	"strings"
 	"testing"
 
@@ -14,60 +14,72 @@ func TestConditions(t *testing.T) {
 	tests := []struct {
 		condition string
 		isTrue    bool
+		shouldErr bool
 	}{
-		{"a is b", false},
-		{"a is a", true},
-		{"a not b", true},
-		{"a not a", false},
-		{"a has a", true},
-		{"a has b", false},
-		{"ba has b", true},
-		{"bab has b", true},
-		{"bab has bb", false},
-		{"a not_has a", false},
-		{"a not_has b", true},
-		{"ba not_has b", false},
-		{"bab not_has b", false},
-		{"bab not_has bb", true},
-		{"bab starts_with bb", false},
-		{"bab starts_with ba", true},
-		{"bab starts_with bab", true},
-		{"bab ends_with bb", false},
-		{"bab ends_with bab", true},
-		{"bab ends_with ab", true},
-		{"a match *", false},
-		{"a match a", true},
-		{"a match .*", true},
-		{"a match a.*", true},
-		{"a match b.*", false},
-		{"ba match b.*", true},
-		{"ba match b[a-z]", true},
-		{"b0 match b[a-z]", false},
-		{"b0a match b[a-z]", false},
-		{"b0a match b[a-z]+", false},
-		{"b0a match b[a-z0-9]+", true},
-		{"a not_match *", true},
-		{"a not_match a", false},
-		{"a not_match .*", false},
-		{"a not_match a.*", false},
-		{"a not_match b.*", true},
-		{"ba not_match b.*", false},
-		{"ba not_match b[a-z]", false},
-		{"b0 not_match b[a-z]", true},
-		{"b0a not_match b[a-z]", true},
-		{"b0a not_match b[a-z]+", true},
-		{"b0a not_match b[a-z0-9]+", false},
+		{"a is b", false, false},
+		{"a is a", true, false},
+		{"a not b", true, false},
+		{"a not a", false, false},
+		{"a has a", true, false},
+		{"a has b", false, false},
+		{"ba has b", true, false},
+		{"bab has b", true, false},
+		{"bab has bb", false, false},
+		{"a not_has a", false, false},
+		{"a not_has b", true, false},
+		{"ba not_has b", false, false},
+		{"bab not_has b", false, false},
+		{"bab not_has bb", true, false},
+		{"bab starts_with bb", false, false},
+		{"bab starts_with ba", true, false},
+		{"bab starts_with bab", true, false},
+		{"bab not_starts_with bb", true, false},
+		{"bab not_starts_with ba", false, false},
+		{"bab not_starts_with bab", false, false},
+		{"bab ends_with bb", false, false},
+		{"bab ends_with bab", true, false},
+		{"bab ends_with ab", true, false},
+		{"bab not_ends_with bb", true, false},
+		{"bab not_ends_with ab", false, false},
+		{"bab not_ends_with bab", false, false},
+		{"a match *", false, true},
+		{"a match a", true, false},
+		{"a match .*", true, false},
+		{"a match a.*", true, false},
+		{"a match b.*", false, false},
+		{"ba match b.*", true, false},
+		{"ba match b[a-z]", true, false},
+		{"b0 match b[a-z]", false, false},
+		{"b0a match b[a-z]", false, false},
+		{"b0a match b[a-z]+", false, false},
+		{"b0a match b[a-z0-9]+", true, false},
+		{"bac match b[a-z]{2}", true, false},
+		{"a not_match *", false, true},
+		{"a not_match a", false, false},
+		{"a not_match .*", false, false},
+		{"a not_match a.*", false, false},
+		{"a not_match b.*", true, false},
+		{"ba not_match b.*", false, false},
+		{"ba not_match b[a-z]", false, false},
+		{"b0 not_match b[a-z]", true, false},
+		{"b0a not_match b[a-z]", true, false},
+		{"b0a not_match b[a-z]+", true, false},
+		{"b0a not_match b[a-z0-9]+", false, false},
+		{"bac not_match b[a-z]{2}", false, false},
 	}
 
 	for i, test := range tests {
 		str := strings.Fields(test.condition)
 		ifCond, err := newIfCond(str[0], str[1], str[2])
 		if err != nil {
-			t.Error(err)
+			if !test.shouldErr {
+				t.Error(err)
+			}
+			continue
 		}
 		isTrue := ifCond.True(nil)
 		if isTrue != test.isTrue {
-			t.Errorf("Test %d: expected %v found %v", i, test.isTrue, isTrue)
+			t.Errorf("Test %d: '%s' expected %v found %v", i, test.condition, test.isTrue, isTrue)
 		}
 	}
 
@@ -186,6 +198,7 @@ func TestIfMatcher(t *testing.T) {
 }
 
 func TestSetupIfMatcher(t *testing.T) {
+	rex_b, _ := regexp.Compile("b")
 	tests := []struct {
 		input     string
 		shouldErr bool
@@ -195,7 +208,7 @@ func TestSetupIfMatcher(t *testing.T) {
 			if	a match b
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "a", op: "match", b: "b"},
+				{a: "a", op: "match", b: "b", neg: false, rex: rex_b},
 			},
 		}},
 		{`test {
@@ -203,7 +216,7 @@ func TestSetupIfMatcher(t *testing.T) {
 			if_op or
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "a", op: "match", b: "b"},
+				{a: "a", op: "match", b: "b", neg: false, rex: rex_b},
 			},
 			isOr: true,
 		}},
@@ -221,26 +234,26 @@ func TestSetupIfMatcher(t *testing.T) {
 		},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "goal", op: "has", b: "go"},
-				{a: "cook", op: "not_has", b: "go"},
+				{a: "goal", op: "has", b: "go", neg: false},
+				{a: "cook", op: "has", b: "go", neg: true},
 			},
 		}},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 			if_op and
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "goal", op: "has", b: "go"},
-				{a: "cook", op: "not_has", b: "go"},
+				{a: "goal", op: "has", b: "go", neg: false},
+				{a: "cook", op: "has", b: "go", neg: true},
 			},
 		}},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 			if_op not
 		 }`, true, IfMatcher{},
 		},
@@ -249,6 +262,7 @@ func TestSetupIfMatcher(t *testing.T) {
 	for i, test := range tests {
 		c := caddy.NewTestController("http", test.input)
 		c.Next()
+
 		matcher, err := SetupIfMatcher(c)
 		if err == nil && test.shouldErr {
 			t.Errorf("Test %d didn't error, but it should have", i)
@@ -257,15 +271,60 @@ func TestSetupIfMatcher(t *testing.T) {
 		} else if err != nil && test.shouldErr {
 			continue
 		}
-		if _, ok := matcher.(IfMatcher); !ok {
+
+		test_if, ok := matcher.(IfMatcher)
+		if !ok {
 			t.Error("RequestMatcher should be of type IfMatcher")
 		}
+
 		if err != nil {
 			t.Errorf("Expected no error, but got: %v", err)
 		}
-		if fmt.Sprint(matcher) != fmt.Sprint(test.expected) {
-			t.Errorf("Test %v: Expected %v, found %v", i,
-				fmt.Sprint(test.expected), fmt.Sprint(matcher))
+
+		if len(test_if.ifs) != len(test.expected.ifs) {
+			t.Errorf("Test %d: Expected %d ifConditions, found %v", i,
+				len(test.expected.ifs), len(test_if.ifs))
+		}
+
+		for j, if_c := range test_if.ifs {
+			expected_c := test.expected.ifs[j]
+
+			if if_c.a != expected_c.a {
+				t.Errorf("Test %d, ifCond %d: Expected A=%s, got %s",
+					i, j, if_c.a, expected_c.a)
+			}
+
+			if if_c.op != expected_c.op {
+				t.Errorf("Test %d, ifCond %d: Expected Op=%s, got %s",
+					i, j, if_c.op, expected_c.op)
+			}
+
+			if if_c.b != expected_c.b {
+				t.Errorf("Test %d, ifCond %d: Expected B=%s, got %s",
+					i, j, if_c.b, expected_c.b)
+			}
+
+			if if_c.neg != expected_c.neg {
+				t.Errorf("Test %d, ifCond %d: Expected Neg=%v, got %v",
+					i, j, if_c.neg, expected_c.neg)
+			}
+
+			if expected_c.rex != nil && if_c.rex == nil {
+				t.Errorf("Test %d, ifCond %d: Expected Rex=%v, got <nil>",
+					i, j, expected_c.rex)
+			}
+
+			if expected_c.rex == nil && if_c.rex != nil {
+				t.Errorf("Test %d, ifCond %d: Expected Rex=<nil>, got %v",
+					i, j, if_c.rex)
+			}
+
+			if expected_c.rex != nil && if_c.rex != nil {
+				if if_c.rex.String() != expected_c.rex.String() {
+					t.Errorf("Test %d, ifCond %d: Expected Rex=%v, got %v",
+						i, j, if_c.rex, expected_c.rex)
+				}
+			}
 		}
 	}
 }

caddyhttp/httpserver/https.go

@@ -44,9 +44,14 @@ func activateHTTPS(cctx caddy.Context) error {
 	// renew all relevant certificates that need renewal. this is important
 	// to do right away so we guarantee that renewals aren't missed, and
 	// also the user can respond to any potential errors that occur.
-	err = caddytls.RenewManagedCertificates(true)
-	if err != nil {
-		return err
+	// (skip if upgrading, because the parent process is likely already listening
+	// on the ports we'd need to do ACME before we finish starting; parent process
+	// already running renewal ticker, so renewal won't be missed anyway.)
+	if !caddy.IsUpgrade() {
+		err = caddytls.RenewManagedCertificates(true)
+		if err != nil {
+			return err
+		}
 	}
 
 	if !caddy.Quiet && operatorPresent {

caddyhttp/httpserver/logger_test.go

@@ -12,7 +12,7 @@ import (
 	"sync"
 	"testing"
 
-	"gopkg.in/mcuadros/go-syslog.v2"
+	syslog "gopkg.in/mcuadros/go-syslog.v2"
 	"gopkg.in/mcuadros/go-syslog.v2/format"
 )
 

caddyhttp/httpserver/middleware.go

@@ -205,4 +205,7 @@ const (
 
 	// MitmCtxKey is the key for the result of MITM detection
 	MitmCtxKey caddy.CtxKey = "mitm"
+
+	// RequestIDCtxKey is the key for the U4 UUID value
+	RequestIDCtxKey caddy.CtxKey = "request_id"
 )

caddyhttp/httpserver/mitm.go

@@ -7,6 +7,7 @@ import (
 	"io"
 	"net"
 	"net/http"
+	"strconv"
 	"strings"
 	"sync"
 )
@@ -65,7 +66,16 @@ func (h *tlsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		mitm = !info.looksLikeChrome() && !info.looksLikeSafari()
 	} else if strings.Contains(ua, "Firefox") {
 		checked = true
-		mitm = !info.looksLikeFirefox()
+		if strings.Contains(ua, "Windows") {
+			ver := getVersion(ua, "Firefox")
+			if ver == 45.0 || ver == 52.0 {
+				mitm = !info.looksLikeTor()
+			} else {
+				mitm = !info.looksLikeFirefox()
+			}
+		} else {
+			mitm = !info.looksLikeFirefox()
+		}
 	} else if strings.Contains(ua, "Safari") {
 		checked = true
 		mitm = !info.looksLikeSafari()
@@ -87,6 +97,36 @@ func (h *tlsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	h.next.ServeHTTP(w, r)
 }
 
+// getVersion returns a (possibly simplified) representation of the version string
+// from a UserAgent string. It returns a float, so it can represent major and minor
+// versions; the rest of the version is just tacked on behind the decimal point.
+// The purpose of this is to stay simple while allowing for basic, fast comparisons.
+// If the version for softwareName is not found in ua, -1 is returned.
+func getVersion(ua, softwareName string) float64 {
+	search := softwareName + "/"
+	start := strings.Index(ua, search)
+	if start < 0 {
+		return -1
+	}
+	start += len(search)
+	end := strings.Index(ua[start:], " ")
+	if end < 0 {
+		end = len(ua)
+	} else {
+		end += start
+	}
+	strVer := strings.Replace(ua[start:end], "-", "", -1)
+	firstDot := strings.Index(strVer, ".")
+	if firstDot >= 0 {
+		strVer = strVer[:firstDot+1] + strings.Replace(strVer[firstDot+1:], ".", "", -1)
+	}
+	ver, err := strconv.ParseFloat(strVer, 64)
+	if err != nil {
+		return -1
+	}
+	return ver
+}
+
 // clientHelloConn reads the ClientHello
 // and stores it in the attached listener.
 type clientHelloConn struct {
@@ -323,27 +363,38 @@ func (info rawHelloInfo) looksLikeFirefox() bool {
 	// "To determine whether a Firefox session has been
 	// intercepted, we check for the presence and order
 	// of extensions, cipher suites, elliptic curves,
-	// EC point formats, and handshake compression methods."
+	// EC point formats, and handshake compression methods." (early 2016)
 
 	// We check for the presence and order of the extensions.
-	// Note: Sometimes padding (21) is present, sometimes not.
+	// Note: Sometimes 0x15 (21, padding) is present, sometimes not.
 	// Note: Firefox 51+ does not advertise 0x3374 (13172, NPN).
 	// Note: Firefox doesn't advertise 0x0 (0, SNI) when connecting to IP addresses.
-	requiredExtensionsOrder := []uint16{23, 65281, 10, 11, 35, 16, 5, 65283, 13}
+	// Note: Firefox 55+ doesn't appear to advertise 0xFF03 (65283, short headers). It used to be between 5 and 13.
+	// Note: Firefox on Fedora (or RedHat) doesn't include ECC suites because of patent liability.
+	requiredExtensionsOrder := []uint16{23, 65281, 10, 11, 35, 16, 5, 13}
 	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
 		return false
 	}
 
 	// We check for both presence of curves and their ordering.
-	expectedCurves := []tls.CurveID{29, 23, 24, 25}
-	if len(info.curves) != len(expectedCurves) {
+	requiredCurves := []tls.CurveID{29, 23, 24, 25}
+	if len(info.curves) < len(requiredCurves) {
 		return false
 	}
-	for i := range expectedCurves {
-		if info.curves[i] != expectedCurves[i] {
+	for i := range requiredCurves {
+		if info.curves[i] != requiredCurves[i] {
 			return false
 		}
 	}
+	if len(info.curves) > len(requiredCurves) {
+		// newer Firefox (55 Nightly?) may have additional curves at end of list
+		allowedCurves := []tls.CurveID{256, 257}
+		for i := range allowedCurves {
+			if info.curves[len(requiredCurves)+i] != allowedCurves[i] {
+				return false
+			}
+		}
+	}
 
 	if hasGreaseCiphers(info.cipherSuites) {
 		return false
@@ -353,6 +404,9 @@ func (info rawHelloInfo) looksLikeFirefox() bool {
 	// according to the paper, cipher suites may be not be added
 	// or reordered by the user, but they may be disabled.
 	expectedCipherSuiteOrder := []uint16{
+		TLS_AES_128_GCM_SHA256,                      // 0x1301
+		TLS_CHACHA20_POLY1305_SHA256,                // 0x1303
+		TLS_AES_256_GCM_SHA384,                      // 0x1302
 		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
 		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,   // 0xc02f
 		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,  // 0xcca9
@@ -379,7 +433,7 @@ func (info rawHelloInfo) looksLikeChrome() bool {
 	// to not support, but do not check for the inclusion of
 	// specific ciphers or extensions, nor do we validate their
 	// order. When appropriate, we check the presence and order
-	// of elliptic curves, compression methods, and EC point formats."
+	// of elliptic curves, compression methods, and EC point formats." (early 2016)
 
 	// Not in Chrome 56, but present in Safari 10 (Feb. 2017):
 	// TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
@@ -401,14 +455,14 @@ func (info rawHelloInfo) looksLikeChrome() bool {
 	// 0xc00a, 0xc014, 0xc009, 0x9c, 0x9d, 0x2f, 0x35, 0xa
 
 	chromeCipherExclusions := map[uint16]struct{}{
-		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:   {}, // 0xc024
-		TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256:   {}, // 0xc023
-		TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:     {}, // 0xc028
-		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256: {}, // 0xc027
-		TLS_RSA_WITH_AES_256_CBC_SHA256:           {}, // 0x3d
-		tls.TLS_RSA_WITH_AES_128_CBC_SHA256:       {}, // 0x3c
-		TLS_DHE_RSA_WITH_AES_128_CBC_SHA:          {}, // 0x33
-		TLS_DHE_RSA_WITH_AES_256_CBC_SHA:          {}, // 0x39
+		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:     {}, // 0xc024
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256: {}, // 0xc023
+		TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:       {}, // 0xc028
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256:   {}, // 0xc027
+		TLS_RSA_WITH_AES_256_CBC_SHA256:             {}, // 0x3d
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA256:         {}, // 0x3c
+		TLS_DHE_RSA_WITH_AES_128_CBC_SHA:            {}, // 0x33
+		TLS_DHE_RSA_WITH_AES_256_CBC_SHA:            {}, // 0x39
 	}
 	for _, ext := range info.cipherSuites {
 		if _, ok := chromeCipherExclusions[ext]; ok {
@@ -436,7 +490,7 @@ func (info rawHelloInfo) looksLikeEdge() bool {
 	// "SChannel connections can by uniquely identified because SChannel
 	// is the only TLS library we tested that includes the OCSP status
 	// request extension before the supported groups and EC point formats
-	// extensions."
+	// extensions." (early 2016)
 	//
 	// More specifically, the OCSP status request extension appears
 	// *directly* before the other two extensions, which occur in that
@@ -482,24 +536,28 @@ func (info rawHelloInfo) looksLikeSafari() bool {
 	// in the HTTP User-Agent header. We allow for any of the
 	// updates when validating handshakes, and we check for the
 	// presence and ordering of ciphers, extensions, elliptic
-	// curves, and compression methods."
+	// curves, and compression methods." (early 2016)
 
 	// Note that any C lib (e.g. curl) compiled on macOS
 	// will probably use Secure Transport which will also
 	// share the TLS handshake characteristics of Safari.
 
-	// Let's do the easy check first... should be sufficient in many cases.
-	if len(info.cipherSuites) < 1 {
-		return false
-	}
-	if info.cipherSuites[0] != scsvRenegotiation {
-		return false
-	}
-
 	// We check for the presence and order of the extensions.
 	requiredExtensionsOrder := []uint16{10, 11, 13, 13172, 16, 5, 18, 23}
 	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
-		return false
+		// Safari on iOS 11 (beta) uses different set/ordering of extensions
+		requiredExtensionsOrderiOS11 := []uint16{65281, 0, 23, 13, 5, 13172, 18, 16, 11, 10}
+		if !assertPresenceAndOrdering(requiredExtensionsOrderiOS11, info.extensions, true) {
+			return false
+		}
+	} else {
+		// For these versions of Safari, expect TLS_EMPTY_RENEGOTIATION_INFO_SCSV first.
+		if len(info.cipherSuites) < 1 {
+			return false
+		}
+		if info.cipherSuites[0] != scsvRenegotiation {
+			return false
+		}
 	}
 
 	if hasGreaseCiphers(info.cipherSuites) {
@@ -511,7 +569,7 @@ func (info rawHelloInfo) looksLikeSafari() bool {
 		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, // 0xc02c
 		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
 		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,     // 0xc024
-		TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,     // 0xc023
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, // 0xc023
 		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,    // 0xc00a
 		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,    // 0xc009
 		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,   // 0xc030
@@ -523,13 +581,77 @@ func (info rawHelloInfo) looksLikeSafari() bool {
 		tls.TLS_RSA_WITH_AES_256_GCM_SHA384,         // 0x9d
 		tls.TLS_RSA_WITH_AES_128_GCM_SHA256,         // 0x9c
 		TLS_RSA_WITH_AES_256_CBC_SHA256,             // 0x3d
-		TLS_RSA_WITH_AES_128_CBC_SHA256,             // 0x3c
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA256,         // 0x3c
 		tls.TLS_RSA_WITH_AES_256_CBC_SHA,            // 0x35
 		tls.TLS_RSA_WITH_AES_128_CBC_SHA,            // 0x2f
 	}
 	return assertPresenceAndOrdering(expectedCipherSuiteOrder, info.cipherSuites, true)
 }
 
+// looksLikeTor returns true if the info looks like a ClientHello from Tor browser
+// (based on Firefox).
+func (info rawHelloInfo) looksLikeTor() bool {
+	requiredExtensionsOrder := []uint16{10, 11, 16, 5, 13}
+	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
+		return false
+	}
+
+	// check for session tickets support; Tor doesn't support them to prevent tracking
+	for _, ext := range info.extensions {
+		if ext == 35 {
+			return false
+		}
+	}
+
+	// We check for both presence of curves and their ordering, including
+	// an optional curve at the beginning (for Tor based on Firefox 52)
+	infoCurves := info.curves
+	if len(info.curves) == 4 {
+		if info.curves[0] != 29 {
+			return false
+		}
+		infoCurves = info.curves[1:]
+	}
+	requiredCurves := []tls.CurveID{23, 24, 25}
+	if len(infoCurves) < len(requiredCurves) {
+		return false
+	}
+	for i := range requiredCurves {
+		if infoCurves[i] != requiredCurves[i] {
+			return false
+		}
+	}
+
+	if hasGreaseCiphers(info.cipherSuites) {
+		return false
+	}
+
+	// We check for order of cipher suites but not presence, since
+	// according to the paper, cipher suites may be not be added
+	// or reordered by the user, but they may be disabled.
+	expectedCipherSuiteOrder := []uint16{
+		TLS_AES_128_GCM_SHA256,                      // 0x1301
+		TLS_CHACHA20_POLY1305_SHA256,                // 0x1303
+		TLS_AES_256_GCM_SHA384,                      // 0x1302
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
+		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,   // 0xc02f
+		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,  // 0xcca9
+		tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,    // 0xcca8
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, // 0xc02c
+		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,   // 0xc030
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,    // 0xc00a
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,    // 0xc009
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,      // 0xc013
+		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,      // 0xc014
+		TLS_DHE_RSA_WITH_AES_128_CBC_SHA,            // 0x33
+		TLS_DHE_RSA_WITH_AES_256_CBC_SHA,            // 0x39
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA,            // 0x2f
+		tls.TLS_RSA_WITH_AES_256_CBC_SHA,            // 0x35
+		tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,           // 0xa
+	}
+	return assertPresenceAndOrdering(expectedCipherSuiteOrder, info.cipherSuites, false)
+}
+
 // assertPresenceAndOrdering will return true if candidateList contains
 // the items in requiredItems in the same order as requiredItems.
 //
@@ -610,11 +732,17 @@ const (
 	// cipher suites missing from the crypto/tls package,
 	// in no particular order here
 	TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 = 0xc024
-	TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 = 0xc023
 	TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384   = 0xc028
-	TLS_RSA_WITH_AES_128_CBC_SHA256         = 0x3c
 	TLS_RSA_WITH_AES_256_CBC_SHA256         = 0x3d
 	TLS_DHE_RSA_WITH_AES_128_CBC_SHA        = 0x33
 	TLS_DHE_RSA_WITH_AES_256_CBC_SHA        = 0x39
 	TLS_RSA_WITH_RC4_128_MD5                = 0x4
+
+	// new PSK ciphers introduced by TLS 1.3, not (yet) in crypto/tls
+	// https://tlswg.github.io/tls13-spec/#rfc.appendix.A.4)
+	TLS_AES_128_GCM_SHA256       = 0x1301
+	TLS_AES_256_GCM_SHA384       = 0x1302
+	TLS_CHACHA20_POLY1305_SHA256 = 0x1303
+	TLS_AES_128_CCM_SHA256       = 0x1304
+	TLS_AES_128_CCM_8_SHA256     = 0x1305
 )

caddyhttp/httpserver/mitm_test.go

@@ -108,20 +108,20 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010000c003031dae75222dae1433a5a283ddcde8ddabaefbf16d84f250eee6fdff48cdfff8a00000201a1ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a010000777a7a0000ff010001000000000e000c0000096c6f63616c686f73740017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a0008aaaa001d001700182a2a000100`,
 				interception: false,
 			},
-			// TODO: Chrome on iOS will use iOS' TLS stack for requests that load
-			// the web page (apparently required by the dev ToS) but will use its
-			// own TLS stack for everything else, it seems. Figure out a decent way
-			// to test this with a nice, unified corpus that allows for this variance.
-			// {
-			// 	// Chrome on iOS
-			// 	userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
-			// 	helloHex:  `010000de030358b062c509b21410a6496b5a82bfec74436cdecebe8ea1da29799939bbd3c17200002c00ffc02cc02bc024c023c00ac009c008c030c02fc028c027c014c013c012009d009c003d003c0035002f000a0100008900000014001200000f66696e6572706978656c732e636f6d000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
-			// },
-			// {
-			// 	// Chrome on iOS (requesting favicon)
-			// 	userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
-			// 	helloHex:  `010000c20303863eb64788e3b9638c261300318411cbdd8f09576d58eec1e744b6ce944f574f0000208a8acca9cca8cc14cc13c02bc02fc02cc030c013c014009c009d002f0035000a01000079baba0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e31000b00020100000a000a00083a3a001d001700184a4a000100`,
-			// },
+			{
+				// Chrome on iOS will use iOS' TLS stack for requests that load
+				// the web page (apparently required by the dev ToS) but will use its
+				// own TLS stack for everything else, it seems.
+
+				// Chrome on iOS
+				userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
+				helloHex:  `010000de030358b062c509b21410a6496b5a82bfec74436cdecebe8ea1da29799939bbd3c17200002c00ffc02cc02bc024c023c00ac009c008c030c02fc028c027c014c013c012009d009c003d003c0035002f000a0100008900000014001200000f66696e6572706978656c732e636f6d000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
+			},
+			{
+				// Chrome on iOS (requesting favicon)
+				userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
+				helloHex:  `010000c20303863eb64788e3b9638c261300318411cbdd8f09576d58eec1e744b6ce944f574f0000208a8acca9cca8cc14cc13c02bc02fc02cc030c013c014009c009d002f0035000a01000079baba0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e31000b00020100000a000a00083a3a001d001700184a4a000100`,
+			},
 			{
 				userAgent:    "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
 				helloHex:     `010000c603036f717a88212c3e9e41940f82c42acb3473e0e4a64e8f52d9af33d34e972e08a30000206a6ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a0100007d7a7a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00087a7a001d001700188a8a000100`,
@@ -132,6 +132,16 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010001fc030383141d213d1bf069171843489faf808028d282c9828e1ba87637c863833c730720a67e76e152f4b704523b72317ef4587e231f02e2395e0ecac6be9f28c35e6ce600208a8ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a010001931a1a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d00170000002300785e85429bf1764f33111cd3ad5d1c56d765976fd962b49dbecbb6f7865e2a8d8536ad854f1fa99a8bbbf998814fee54a63a0bf162869d2bba37e9778304e7c4140825718e191b574c6246a0611de6447bdd80417f83ff9d9b7124069a9f74b90394ecb89bec5f6a1a67c1b89e50b8674782f53dd51807651a000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00081a1a001d001700182a2a0001000015009a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000`,
 				interception: false,
 			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
+				helloHex:     `010000c203034166c97e2016046e0c88ad867c410d0aee470f4d9b4ec8fe41a751d2a6348e3100001c4a4ac02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a0100007dcaca0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00086a6a001d001700187a7a000100`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
+				helloHex:     `010000c203037741795e73cd5b4949f79a0dc9cccc8b006e4c0ec324f965c6fe9f0833909f0100001c7a7ac02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a0100007d7a7a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00084a4a001d001700185a5a000100`,
+				interception: false,
+			},
 		},
 		"Firefox": {
 			{
@@ -139,6 +149,28 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010000bd030375f9022fc3a6562467f3540d68013b2d0b961979de6129e944efe0b35531323500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a010000760000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000ff030000000d0020001e040305030603020308040805080604010501060102010402050206020202`,
 				interception: false,
 			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010001fc0303c99d54ae0628bbb9fea3833a4244c6a712cac9d7738f4930b8b9d8e2f6bd578220f7936cedb48907981c9292fb08ceee6f59bd6fddb3d4271ccd7c12380c5038ab001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a01000195001500af000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b000201000023007886da2d41843ff42131b856982c19a545837b70e604325423a817d925e9d95bd084737682cea6b804dfb7cbe336a3b27b8d520d57520c29cfe5f4f3d3236183b84b05c18f0ca30bf598111e390086fea00d9631f1f78527277eb7838b86e73c4e5d15b55d086b1a4a8aa29f12a55126c6274bcd499bbeb23a0010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010000b1030365d899820b999245d571c2f7d6b850f63ad931d3c68ceb9cf5a508421a871dc500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100006a0000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+			{
+				// this was a Nightly release at the time
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0",
+				helloHex:     `010001fc030331e380b7d12018e1202ef3327607203df5c5732b4fa5ab5abaf0b60034c2fb662070c836b9b89123e37f4f1074d152df438fa8ee8a0f89b036fd952f4fcc0b994f001c130113031302c02bc02fcca9cca8c02cc030c013c014002f0035000a0100019700000014001200000f63616464797365727665722e636f6d00170000ff01000100000a000e000c001d00170018001901000101000b0002010000230078c97e7716a041e2ea824571bef26a3dff2bf50a883cd15d904ab2d17deb514f6e0a079ee7c212c000178387ffafc2e530b6df6662f570aae134330f13c458a0eaad5a96a9696f572110918740b15db1143d19aaaa706942030b433a7e6150f62b443c0564e5b8f7ee9577bf3bf7faec8c67425b648ab54d880010000e000c02683208687474702f312e310005000501000000000028006b0069001d0020aee6e596155ee6f79f943e81ceabe0979d27fbbb8b9189ccb2ebc75226351f32001700410421875a44e510decac11ef1d7cfddd4dfe105d5cd3a2d42fba03ebde23e51e8ce65bda1b48be82d4848d1db2bfce68e94092e925a9ce0dbf5df35479558108489002b0009087f12030303020301000d0018001604030503060308040805080604010501060102030201002d000201010015002500000000000000000000000000000000000000000000000000000000000000000000000000`,
+				interception: false,
+			},
+			{
+				// Firefox on Fedora (RedHat) doesn't include ECC ciphers because of patent liabilities
+				userAgent:    "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010000b70303f5280b74d617d42e39fd77b78a2b537b1d7787ce4fcbcf3604c9fbcd677c6c5500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100007000000014001200000f66696e6572706978656c732e636f6d00170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
 		},
 		"Edge": {
 			{
@@ -153,6 +185,23 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010000d2030358a295b513c8140c6ff880f4a8a73cc830ed2dab2c4f2068eb365228d828732e00002600ffc02cc02bc024c023c00ac009c030c02fc028c027c014c013009d009c003d003c0035002f010000830000000e000c0000096c6f63616c686f7374000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
 				interception: false,
 			},
+			{
+				userAgent:    "Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.28 (KHTML, like Gecko) Version/11.0 Mobile/15A5318g Safari/604.1",
+				helloHex:     `010000e10303be294e11847ba01301e0bb6129f4a0d66344602141a8f0a1ab0750a1db145755000028c02cc02bc024c023cca9c00ac009c030c02fc028c027cca8c014c013009d009c003d003c0035002f01000090ff0100010000000014001200000f66696e6572706978656c732e636f6d00170000000d00140012040308040401050308050501080606010201000500050100000000337400000012000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e31000b00020100000a00080006001d00170018`,
+				interception: false,
+			},
+		},
+		"Tor": {
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0",
+				helloHex:     `010000a40303137f05d4151f2d9095aee4254416d9dce73d6a1d857e8097ea20d021c04a7a81000016c02bc02fc00ac009c013c01400330039002f0035000a0100006500000014001200000f66696e6572706978656c732e636f6dff01000100000a00080006001700180019000b00020100337400000010000b000908687474702f312e31000500050100000000000d001600140401050106010201040305030603020304020202`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0",
+				helloHex:     `010000b4030322e1f3aff4c37caba303c2ce53ba1689b3e70117a46f413d44f70a74cb6a496100001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100006d00000014001200000f66696e6572706978656c732e636f6d00170000ff01000100000a000a0008001d001700180019000b000201000010000b000908687474702f312e31000500050100000000ff030000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
 		},
 		"Other": { // these are either non-browser clients or intercepted client hellos
 			{
@@ -221,7 +270,7 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				interception: true,
 			},
 			{
-				// IE 11 on Windows 10, intercepted by Fortigate (same firewallas above)
+				// IE 11 on Windows 10, intercepted by Fortigate (same firewall as above)
 				userAgent:    "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko",
 				helloHex:     `010000e5030158ac634c5278d7b17421f23a64cc91d68c470c6b247322fe867ba035b373d05c000064003300320039003800160013c013c009c014c00ac012c008002f0035000a00150012003d003c00670040006b006ac011c0070096009a009900410084004500440088008700ba00be00bd00c000c400c3c03cc044c042c03dc045c04300090005000400ff01000058000a003600340000000100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019000b0002010000000014001200000f66696e6572706978656c732e636f6d`,
 				interception: true,
@@ -254,6 +303,7 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 			isFirefox := parsed.looksLikeFirefox()
 			isSafari := parsed.looksLikeSafari()
 			isEdge := parsed.looksLikeEdge()
+			isTor := parsed.looksLikeTor()
 
 			// we want each of the heuristic functions to be as
 			// exclusive but as low-maintenance as possible;
@@ -261,26 +311,17 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 			// should return false, with as little logic as possible,
 			// but with enough logic to force TLS proxies to do a
 			// good job preserving characterstics of the handshake.
-			var correct bool
-			switch client {
-			case "Chrome":
-				correct = isChrome && !isFirefox && !isSafari && !isEdge
-			case "Firefox":
-				correct = !isChrome && isFirefox && !isSafari && !isEdge
-			case "Safari":
-				correct = !isChrome && !isFirefox && isSafari && !isEdge
-			case "Edge":
-				correct = !isChrome && !isFirefox && !isSafari && isEdge
-			case "Other":
-				correct = !isChrome && !isFirefox && !isSafari && !isEdge
+			if (isChrome && (isFirefox || isSafari || isEdge || isTor)) ||
+				(isFirefox && (isChrome || isSafari || isEdge || isTor)) ||
+				(isSafari && (isChrome || isFirefox || isEdge || isTor)) ||
+				(isEdge && (isChrome || isFirefox || isSafari || isTor)) ||
+				(isTor && (isChrome || isFirefox || isSafari || isEdge)) {
+				t.Errorf("[%s] Test %d: Multiple fingerprinting functions matched: "+
+					"Chrome=%v Firefox=%v Safari=%v Edge=%v Tor=%v\n\tparsed hello dec: %+v\n\tparsed hello hex: %#x\n",
+					client, i, isChrome, isFirefox, isSafari, isEdge, isTor, parsed, parsed)
 			}
 
-			if !correct {
-				t.Errorf("[%s] Test %d: Chrome=%v, Firefox=%v, Safari=%v, Edge=%v; parsed hello: %+v",
-					client, i, isChrome, isFirefox, isSafari, isEdge, parsed)
-			}
-
-			// test the handler too
+			// test the handler and detection results
 			var got, checked bool
 			want := ch.interception
 			handler := &tlsHandler{
@@ -305,7 +346,54 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 			if got != want {
 				t.Errorf("[%s] Test %d: Expected MITM=%v but got %v (type assertion OK (checked)=%v)",
 					client, i, want, got, checked)
+				t.Errorf("[%s] Test %d: Looks like Chrome=%v Firefox=%v Safari=%v Edge=%v Tor=%v\n\tparsed hello dec: %+v\n\tparsed hello hex: %#x\n",
+					client, i, isChrome, isFirefox, isSafari, isEdge, isTor, parsed, parsed)
 			}
 		}
 	}
 }
+
+func TestGetVersion(t *testing.T) {
+	for i, test := range []struct {
+		UserAgent    string
+		SoftwareName string
+		Version      float64
+	}{
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0",
+			SoftwareName: "Firefox",
+			Version:      45.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0 more_stuff_here",
+			SoftwareName: "Firefox",
+			Version:      45.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Safari",
+			Version:      537.36,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Chrome",
+			Version:      51.0270479,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Mozilla",
+			Version:      5.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "curl",
+			Version:      -1,
+		},
+	} {
+		actual := getVersion(test.UserAgent, test.SoftwareName)
+		if actual != test.Version {
+			t.Errorf("Test [%d]: Expected version=%f, got version=%f for %s in '%s'",
+				i, test.Version, actual, test.SoftwareName, test.UserAgent)
+		}
+	}
+}

caddyhttp/httpserver/path.go

@@ -2,22 +2,46 @@ package httpserver
 
 import (
 	"net/http"
+	"path"
 	"strings"
 )
 
-// Path represents a URI path.
+// Path represents a URI path. It should usually be
+// set to the value of a request path.
 type Path string
 
-// Matches checks to see if other matches p.
+// Matches checks to see if base matches p. The correct
+// usage of this method sets p as the request path, and
+// base as a Caddyfile (user-defined) rule path.
 //
 // Path matching will probably not always be a direct
 // comparison; this method assures that paths can be
 // easily and consistently matched.
-func (p Path) Matches(other string) bool {
-	if CaseSensitivePath {
-		return strings.HasPrefix(string(p), other)
+//
+// Multiple slashes are collapsed/merged. See issue #1859.
+func (p Path) Matches(base string) bool {
+	if base == "/" || base == "" {
+		return true
 	}
-	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(other))
+
+	// sanitize the paths for comparison, very important
+	// (slightly lossy if the base path requires multiple
+	// consecutive forward slashes, since those will be merged)
+	pHasTrailingSlash := strings.HasSuffix(string(p), "/")
+	baseHasTrailingSlash := strings.HasSuffix(base, "/")
+	p = Path(path.Clean(string(p)))
+	base = path.Clean(base)
+	if pHasTrailingSlash {
+		p += "/"
+	}
+	if baseHasTrailingSlash {
+		base += "/"
+	}
+
+	if CaseSensitivePath {
+		return strings.HasPrefix(string(p), base)
+	}
+	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(base))
 }
 
 // PathMatcher is a Path RequestMatcher.

caddyhttp/httpserver/path_test.go

@@ -0,0 +1,132 @@
+package httpserver
+
+import "testing"
+
+func TestPathMatches(t *testing.T) {
+	for i, testcase := range []struct {
+		reqPath         Path
+		rulePath        string // or "base path" as in Caddyfile docs
+		shouldMatch     bool
+		caseInsensitive bool
+	}{
+		{
+			reqPath:     "/",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "/foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/bar",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/foo/",
+			rulePath:    "/foo/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/Foobar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+
+			reqPath:     "/FooBar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:         "/foobar",
+			rulePath:        "/FooBar",
+			shouldMatch:     true,
+			caseInsensitive: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "/", // a lone forward slash means to match all requests (see issue #1645) - many future test cases related to this issue
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "foobar.php",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/no/leading/slash",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			// see issue #1859
+			reqPath:     "//double-slash",
+			rulePath:    "/double-slash",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/double//slash",
+			rulePath:    "/double/slash",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "//more/double//slashes",
+			rulePath:    "/more/double/slashes",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/path/../traversal",
+			rulePath:    "/traversal",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/path/../traversal",
+			rulePath:    "/path",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/keep-slashes/http://something/foo/bar",
+			rulePath:    "/keep-slashes/http://something",
+			shouldMatch: true,
+		},
+	} {
+		CaseSensitivePath = !testcase.caseInsensitive
+		if got, want := testcase.reqPath.Matches(testcase.rulePath), testcase.shouldMatch; got != want {
+			t.Errorf("Test %d: For request path '%s' and base path '%s': expected %v, got %v",
+				i, testcase.reqPath, testcase.rulePath, want, got)
+		}
+	}
+}

caddyhttp/httpserver/plugin.go

@@ -436,13 +436,14 @@ var directives = []string{
 	"root",
 	"index",
 	"bind",
-	"maxrequestbody", // TODO: 'limits'
+	"limits",
 	"timeouts",
 	"tls",
 
 	// services/utilities, or other directives that don't necessarily inject handlers
 	"startup",
 	"shutdown",
+	"request_id",
 	"realip", // github.com/captncraig/caddy-realip
 	"git",    // github.com/abiosoft/caddy-git
 
@@ -452,22 +453,28 @@ var directives = []string{
 	// directives that add middleware to the stack
 	"locale", // github.com/simia-tech/caddy-locale
 	"log",
+	"cache", // github.com/nicolasazrak/caddy-cache
 	"rewrite",
 	"ext",
 	"gzip",
 	"header",
 	"errors",
-	"filter",    // github.com/echocat/caddy-filter
-	"minify",    // github.com/hacdias/caddy-minify
-	"ipfilter",  // github.com/pyed/ipfilter
-	"ratelimit", // github.com/xuqingfeng/caddy-rate-limit
-	"search",    // github.com/pedronasser/caddy-search
-	"expires",   // github.com/epicagency/caddy-expires
+	"authz",        // github.com/casbin/caddy-authz
+	"filter",       // github.com/echocat/caddy-filter
+	"minify",       // github.com/hacdias/caddy-minify
+	"ipfilter",     // github.com/pyed/ipfilter
+	"ratelimit",    // github.com/xuqingfeng/caddy-rate-limit
+	"search",       // github.com/pedronasser/caddy-search
+	"expires",      // github.com/epicagency/caddy-expires
+	"forwardproxy", // github.com/caddyserver/forwardproxy
 	"basicauth",
 	"redir",
 	"status",
-	"cors", // github.com/captncraig/cors/caddy
+	"cors",   // github.com/captncraig/cors/caddy
+	"nobots", // github.com/Xumeiquer/nobots
 	"mime",
+	"login",     // github.com/tarent/loginsrv/caddy
+	"reauth",    // github.com/freman/caddy-reauth
 	"jwt",       // github.com/BTBurke/caddy-jwt
 	"jsonp",     // github.com/pschlump/caddy-jsonp
 	"upload",    // blitznote.com/src/caddy.upload
@@ -476,18 +483,25 @@ var directives = []string{
 	"pprof",
 	"expvar",
 	"push",
+	"datadog",    // github.com/payintech/caddy-datadog
 	"prometheus", // github.com/miekg/caddy-prometheus
+	"templates",
 	"proxy",
 	"fastcgi",
 	"cgi", // github.com/jung-kurt/caddy-cgi
 	"websocket",
-	"filemanager", // github.com/hacdias/caddy-filemanager
+	"filemanager", // github.com/hacdias/filemanager/caddy/filemanager
+	"webdav",      // github.com/hacdias/caddy-webdav
 	"markdown",
-	"templates",
 	"browse",
-	"hugo",      // github.com/hacdias/caddy-hugo
+	"jekyll",    // github.com/hacdias/filemanager/caddy/jekyll
+	"hugo",      // github.com/hacdias/filemanager/caddy/hugo
 	"mailout",   // github.com/SchumacherFM/mailout
+	"awses",     // github.com/miquella/caddy-awses
 	"awslambda", // github.com/coopernurse/caddy-awslambda
+	"grpc",      // github.com/pieterlouw/caddy-grpc
+	"gopkg",     // github.com/zikes/gopkg
+	"restic",    // github.com/restic/caddy
 }
 
 const (

caddyhttp/httpserver/recorder.go

@@ -1,9 +1,10 @@
 package httpserver
 
 import (
-	"bufio"
-	"net"
+	"bytes"
+	"io"
 	"net/http"
+	"sync"
 	"time"
 )
 
@@ -20,24 +21,22 @@ import (
 //
 // Beware when accessing the Replacer value; it may be nil!
 type ResponseRecorder struct {
-	http.ResponseWriter
+	*ResponseWriterWrapper
 	Replacer Replacer
 	status   int
 	size     int
 	start    time.Time
 }
 
-// NewResponseRecorder makes and returns a new responseRecorder,
-// which captures the HTTP Status code from the ResponseWriter
-// and also the length of the response body written through it.
+// NewResponseRecorder makes and returns a new ResponseRecorder.
 // Because a status is not set unless WriteHeader is called
 // explicitly, this constructor initializes with a status code
 // of 200 to cover the default case.
 func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 	return &ResponseRecorder{
-		ResponseWriter: w,
-		status:         http.StatusOK,
-		start:          time.Now(),
+		ResponseWriterWrapper: &ResponseWriterWrapper{ResponseWriter: w},
+		status:                http.StatusOK,
+		start:                 time.Now(),
 	}
 }
 
@@ -45,68 +44,190 @@ func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 // underlying ResponseWriter's WriteHeader method.
 func (r *ResponseRecorder) WriteHeader(status int) {
 	r.status = status
-	r.ResponseWriter.WriteHeader(status)
+	r.ResponseWriterWrapper.WriteHeader(status)
 }
 
 // Write is a wrapper that records the size of the body
 // that gets written.
 func (r *ResponseRecorder) Write(buf []byte) (int, error) {
-	n, err := r.ResponseWriter.Write(buf)
+	n, err := r.ResponseWriterWrapper.Write(buf)
 	if err == nil {
 		r.size += n
 	}
 	return n, err
 }
 
-// Size is a Getter to size property
+// Size returns the size of the recorded response body.
 func (r *ResponseRecorder) Size() int {
 	return r.size
 }
 
-// Status is a Getter to status property
+// Status returns the recorded response status code.
 func (r *ResponseRecorder) Status() int {
 	return r.status
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (r *ResponseRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := r.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, NonHijackerError{Underlying: r.ResponseWriter}
+// ResponseBuffer is a type that conditionally buffers the
+// response in memory. It implements http.ResponseWriter so
+// that it can stream the response if it is not buffering.
+// Whether it buffers is decided by a func passed into the
+// constructor, NewResponseBuffer.
+//
+// This type implements http.ResponseWriter, so you can pass
+// this to the Next() middleware in the chain and record its
+// response. However, since the entire response body will be
+// buffered in memory, only use this when explicitly configured
+// and required for some specific reason. For example, the
+// text/template package only parses templates out of []byte
+// and not io.Reader, so the templates directive uses this
+// type to obtain the entire template text, but only on certain
+// requests that match the right Content-Type, etc.
+//
+// ResponseBuffer also implements io.ReaderFrom for performance
+// reasons. The standard lib's http.response type (unexported)
+// uses io.Copy to write the body. io.Copy makes an allocation
+// if the destination does not have a ReadFrom method (or if
+// the source does not have a WriteTo method, but that's
+// irrelevant here). Our ReadFrom is smart: if buffering, it
+// calls the buffer's ReadFrom, which makes no allocs because
+// it is already a buffer! If we're streaming the response
+// instead, ReadFrom uses io.CopyBuffer with a pooled buffer
+// that is managed within this package.
+type ResponseBuffer struct {
+	*ResponseWriterWrapper
+	Buffer       *bytes.Buffer
+	header       http.Header
+	status       int
+	shouldBuffer func(status int, header http.Header) bool
+	stream       bool
+	rw           http.ResponseWriter
 }
 
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or does nothing.
-func (r *ResponseRecorder) Flush() {
-	if f, ok := r.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(NonFlusherError{Underlying: r.ResponseWriter}) // should be recovered at the beginning of middleware stack
+// NewResponseBuffer returns a new ResponseBuffer that will
+// use buf to store the full body of the response if shouldBuffer
+// returns true. If shouldBuffer returns false, then the response
+// body will be streamed directly to rw.
+//
+// shouldBuffer will be passed the status code and header fields of
+// the response. With that information, the function should decide
+// whether to buffer the response in memory. For example: the templates
+// directive uses this to determine whether the response is the
+// right Content-Type (according to user config) for a template.
+//
+// For performance, the buf you pass in should probably be obtained
+// from a sync.Pool in order to reuse allocated space.
+func NewResponseBuffer(buf *bytes.Buffer, rw http.ResponseWriter,
+	shouldBuffer func(status int, header http.Header) bool) *ResponseBuffer {
+	rb := &ResponseBuffer{
+		Buffer:       buf,
+		header:       make(http.Header),
+		status:       http.StatusOK, // default status code
+		shouldBuffer: shouldBuffer,
+		rw:           rw,
+	}
+	rb.ResponseWriterWrapper = &ResponseWriterWrapper{ResponseWriter: rw}
+	return rb
+}
+
+// Header returns the response header map.
+func (rb *ResponseBuffer) Header() http.Header {
+	return rb.header
+}
+
+// WriteHeader calls shouldBuffer to decide whether the
+// upcoming body should be buffered, and then writes
+// the header to the response.
+func (rb *ResponseBuffer) WriteHeader(status int) {
+	rb.status = status
+	rb.stream = !rb.shouldBuffer(status, rb.header)
+	if rb.stream {
+		rb.CopyHeader()
+		rb.ResponseWriterWrapper.WriteHeader(status)
 	}
 }
 
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (r *ResponseRecorder) CloseNotify() <-chan bool {
-	if cn, ok := r.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
+// Write writes buf to rb.Buffer if buffering, otherwise
+// to the ResponseWriter directly if streaming.
+func (rb *ResponseBuffer) Write(buf []byte) (int, error) {
+	if rb.stream {
+		return rb.ResponseWriterWrapper.Write(buf)
 	}
-	panic(NonCloseNotifierError{Underlying: r.ResponseWriter})
+	return rb.Buffer.Write(buf)
 }
 
-// Push resource to client
-func (r *ResponseRecorder) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := r.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
+// Buffered returns whether rb has decided to buffer the response.
+func (rb *ResponseBuffer) Buffered() bool {
+	return !rb.stream
+}
 
-	return NonPusherError{Underlying: r.ResponseWriter}
+// CopyHeader copies the buffered header in rb to the ResponseWriter,
+// but it does not write the header out.
+func (rb *ResponseBuffer) CopyHeader() {
+	for field, val := range rb.header {
+		rb.ResponseWriterWrapper.Header()[field] = val
+	}
+}
+
+// ReadFrom avoids allocations when writing to the buffer (if buffering),
+// and reduces allocations when writing to the ResponseWriter directly
+// (if streaming).
+//
+// In local testing with the templates directive, req/sec were improved
+// from ~8,200 to ~9,600 on templated files by ensuring that this type
+// implements io.ReaderFrom.
+func (rb *ResponseBuffer) ReadFrom(src io.Reader) (int64, error) {
+	if rb.stream {
+		// first see if we can avoid any allocations at all
+		if wt, ok := src.(io.WriterTo); ok {
+			return wt.WriteTo(rb.ResponseWriterWrapper)
+		}
+		// if not, use a pooled copy buffer to reduce allocs
+		// (this improved req/sec from ~25,300 to ~27,000 on
+		// static files served directly with the fileserver,
+		// but results fluctuated a little on each run).
+		// a note of caution:
+		// https://go-review.googlesource.com/c/22134#message-ff351762308fe05f6b72a487d6842e3988916486
+		buf := respBufPool.Get().([]byte)
+		n, err := io.CopyBuffer(rb.ResponseWriterWrapper, src, buf)
+		respBufPool.Put(buf) // defer'ing this slowed down benchmarks a smidgin, I think
+		return n, err
+	}
+	return rb.Buffer.ReadFrom(src)
+}
+
+// StatusCodeWriter returns an http.ResponseWriter that always
+// writes the status code stored in rb from when a response
+// was buffered to it.
+func (rb *ResponseBuffer) StatusCodeWriter(w http.ResponseWriter) http.ResponseWriter {
+	return forcedStatusCodeWriter{w, rb}
+}
+
+// forcedStatusCodeWriter is used to force a status code when
+// writing the header. It uses the status code saved on rb.
+// This is useful if passing a http.ResponseWriter into
+// http.ServeContent because ServeContent hard-codes 2xx status
+// codes. If we buffered the response, we force that status code
+// instead.
+type forcedStatusCodeWriter struct {
+	http.ResponseWriter
+	rb *ResponseBuffer
+}
+
+func (fscw forcedStatusCodeWriter) WriteHeader(int) {
+	fscw.ResponseWriter.WriteHeader(fscw.rb.status)
+}
+
+// respBufPool is used for io.CopyBuffer when ResponseBuffer
+// is configured to stream a response.
+var respBufPool = &sync.Pool{
+	New: func() interface{} {
+		return make([]byte, 32*1024)
+	},
 }
 
 // Interface guards
-var _ http.Pusher = (*ResponseRecorder)(nil)
-var _ http.Flusher = (*ResponseRecorder)(nil)
-var _ http.CloseNotifier = (*ResponseRecorder)(nil)
-var _ http.Hijacker = (*ResponseRecorder)(nil)
+var (
+	_ HTTPInterfaces = (*ResponseRecorder)(nil)
+	_ HTTPInterfaces = (*ResponseBuffer)(nil)
+	_ io.ReaderFrom  = (*ResponseBuffer)(nil)
+)

caddyhttp/httpserver/replacer.go

@@ -243,6 +243,9 @@ func (r *replacer) getSubstitution(key string) string {
 	case "{path_escaped}":
 		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
 		return url.QueryEscape(u.Path)
+	case "{request_id}":
+		reqid, _ := r.request.Context().Value(RequestIDCtxKey).(string)
+		return reqid
 	case "{rewrite_path}":
 		return r.request.URL.Path
 	case "{rewrite_path_escaped}":
@@ -284,6 +287,8 @@ func (r *replacer) getSubstitution(key string) string {
 		return now().Format(timeFormat)
 	case "{when_iso}":
 		return now().UTC().Format(timeFormatISOUTC)
+	case "{when_unix}":
+		return strconv.FormatInt(now().Unix(), 10)
 	case "{file}":
 		_, file := path.Split(r.request.URL.Path)
 		return file
@@ -302,7 +307,7 @@ func (r *replacer) getSubstitution(key string) string {
 		}
 		_, err := ioutil.ReadAll(r.request.Body)
 		if err != nil {
-			if _, ok := err.(MaxBytesExceeded); ok {
+			if err == ErrMaxBytesExceeded {
 				return r.emptyValue
 			}
 		}
@@ -312,7 +317,6 @@ func (r *replacer) getSubstitution(key string) string {
 			if val {
 				return "likely"
 			}
-
 			return "unlikely"
 		}
 		return "unknown"

caddyhttp/httpserver/replacer_test.go

@@ -75,6 +75,7 @@ func TestReplace(t *testing.T) {
 		{"The response status is {status}.", "The response status is 200."},
 		{"{when}", "02/Jan/2006:15:04:05 +0000"},
 		{"{when_iso}", "2006-01-02T15:04:12Z"},
+		{"{when_unix}", "1136214252"},
 		{"The Custom header is {>Custom}.", "The Custom header is foobarbaz."},
 		{"The CustomAdd header is {>CustomAdd}.", "The CustomAdd header is caddy."},
 		{"The request is {request}.", "The request is POST /?foo=bar HTTP/1.1\\r\\nHost: localhost\\r\\n" +

caddyhttp/httpserver/responsewriterwrapper.go

@@ -0,0 +1,65 @@
+package httpserver
+
+import (
+	"bufio"
+	"net"
+	"net/http"
+)
+
+// ResponseWriterWrapper wrappers underlying ResponseWriter
+// and inherits its Hijacker/Pusher/CloseNotifier/Flusher as well.
+type ResponseWriterWrapper struct {
+	http.ResponseWriter
+}
+
+// Hijack implements http.Hijacker. It simply wraps the underlying
+// ResponseWriter's Hijack method if there is one, or returns an error.
+func (rww *ResponseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) {
+	if hj, ok := rww.ResponseWriter.(http.Hijacker); ok {
+		return hj.Hijack()
+	}
+	return nil, nil, NonHijackerError{Underlying: rww.ResponseWriter}
+}
+
+// Flush implements http.Flusher. It simply wraps the underlying
+// ResponseWriter's Flush method if there is one, or panics.
+func (rww *ResponseWriterWrapper) Flush() {
+	if f, ok := rww.ResponseWriter.(http.Flusher); ok {
+		f.Flush()
+	} else {
+		panic(NonFlusherError{Underlying: rww.ResponseWriter})
+	}
+}
+
+// CloseNotify implements http.CloseNotifier.
+// It just inherits the underlying ResponseWriter's CloseNotify method.
+// It panics if the underlying ResponseWriter is not a CloseNotifier.
+func (rww *ResponseWriterWrapper) CloseNotify() <-chan bool {
+	if cn, ok := rww.ResponseWriter.(http.CloseNotifier); ok {
+		return cn.CloseNotify()
+	}
+	panic(NonCloseNotifierError{Underlying: rww.ResponseWriter})
+}
+
+// Push implements http.Pusher.
+// It just inherits the underlying ResponseWriter's Push method.
+// It panics if the underlying ResponseWriter is not a Pusher.
+func (rww *ResponseWriterWrapper) Push(target string, opts *http.PushOptions) error {
+	if pusher, hasPusher := rww.ResponseWriter.(http.Pusher); hasPusher {
+		return pusher.Push(target, opts)
+	}
+
+	return NonPusherError{Underlying: rww.ResponseWriter}
+}
+
+// HTTPInterfaces mix all the interfaces that middleware ResponseWriters need to support.
+type HTTPInterfaces interface {
+	http.ResponseWriter
+	http.Pusher
+	http.Flusher
+	http.CloseNotifier
+	http.Hijacker
+}
+
+// Interface guards
+var _ HTTPInterfaces = (*ResponseWriterWrapper)(nil)

caddyhttp/httpserver/roller.go

@@ -1,6 +1,7 @@
 package httpserver
 
 import (
+	"errors"
 	"io"
 	"path/filepath"
 	"strconv"
@@ -14,6 +15,7 @@ type LogRoller struct {
 	MaxSize    int
 	MaxAge     int
 	MaxBackups int
+	Compress   bool
 	LocalTime  bool
 }
 
@@ -37,6 +39,7 @@ func (l LogRoller) GetLogWriter() io.Writer {
 			MaxSize:    l.MaxSize,
 			MaxAge:     l.MaxAge,
 			MaxBackups: l.MaxBackups,
+			Compress:   l.Compress,
 			LocalTime:  l.LocalTime,
 		}
 		lumberjacks[absPath] = lj
@@ -48,20 +51,36 @@ func (l LogRoller) GetLogWriter() io.Writer {
 func IsLogRollerSubdirective(subdir string) bool {
 	return subdir == directiveRotateSize ||
 		subdir == directiveRotateAge ||
-		subdir == directiveRotateKeep
+		subdir == directiveRotateKeep ||
+		subdir == directiveRotateCompress
 }
 
+var invalidRollerParameterErr = errors.New("invalid roller parameter")
+
 // ParseRoller parses roller contents out of c.
-func ParseRoller(l *LogRoller, what string, where string) error {
+func ParseRoller(l *LogRoller, what string, where ...string) error {
 	if l == nil {
 		l = DefaultLogRoller()
 	}
-	var value int
-	var err error
-	value, err = strconv.Atoi(where)
-	if err != nil {
-		return err
+
+	// rotate_compress doesn't accept any parameters.
+	// others only accept one parameter
+	if (what == directiveRotateCompress && len(where) != 0) ||
+		(what != directiveRotateCompress && len(where) != 1) {
+		return invalidRollerParameterErr
 	}
+
+	var (
+		value int
+		err   error
+	)
+	if what != directiveRotateCompress {
+		value, err = strconv.Atoi(where[0])
+		if err != nil {
+			return err
+		}
+	}
+
 	switch what {
 	case directiveRotateSize:
 		l.MaxSize = value
@@ -69,6 +88,8 @@ func ParseRoller(l *LogRoller, what string, where string) error {
 		l.MaxAge = value
 	case directiveRotateKeep:
 		l.MaxBackups = value
+	case directiveRotateCompress:
+		l.Compress = true
 	}
 	return nil
 }
@@ -79,6 +100,7 @@ func DefaultLogRoller() *LogRoller {
 		MaxSize:    defaultRotateSize,
 		MaxAge:     defaultRotateAge,
 		MaxBackups: defaultRotateKeep,
+		Compress:   false,
 		LocalTime:  true,
 	}
 }
@@ -89,10 +111,12 @@ const (
 	// defaultRotateAge is 14 days.
 	defaultRotateAge = 14
 	// defaultRotateKeep is 10 files.
-	defaultRotateKeep   = 10
-	directiveRotateSize = "rotate_size"
-	directiveRotateAge  = "rotate_age"
-	directiveRotateKeep = "rotate_keep"
+	defaultRotateKeep = 10
+
+	directiveRotateSize     = "rotate_size"
+	directiveRotateAge      = "rotate_age"
+	directiveRotateKeep     = "rotate_keep"
+	directiveRotateCompress = "rotate_compress"
 )
 
 // lumberjacks maps log filenames to the logger

caddyhttp/httpserver/server.go

@@ -4,8 +4,8 @@ package httpserver
 import (
 	"context"
 	"crypto/tls"
+	"errors"
 	"fmt"
-	"io"
 	"log"
 	"net"
 	"net/http"
@@ -57,6 +57,16 @@ func makeTLSConfig(group []*SiteConfig) (*tls.Config, error) {
 	return caddytls.MakeTLSConfig(tlsConfigs)
 }
 
+func getFallbacks(sites []*SiteConfig) []string {
+	fallbacks := []string{}
+	for _, sc := range sites {
+		if sc.FallbackSite {
+			fallbacks = append(fallbacks, sc.Addr.Host)
+		}
+	}
+	return fallbacks
+}
+
 // NewServer creates a new Server instance that will listen on addr
 // and will serve the sites configured in group.
 func NewServer(addr string, group []*SiteConfig) (*Server, error) {
@@ -66,6 +76,8 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 		sites:       group,
 		connTimeout: GracefulTimeout,
 	}
+	s.vhosts.fallbackHosts = append(s.vhosts.fallbackHosts, getFallbacks(group)...)
+	s.Server = makeHTTPServerWithHeaderLimit(s.Server, group)
 	s.Server.Handler = s // this is weird, but whatever
 
 	// extract TLS settings from each site config to build
@@ -76,14 +88,14 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 	}
 	s.Server.TLSConfig = tlsConfig
 
-	// Enable QUIC if desired
-	if QUIC {
-		s.quicServer = &h2quic.Server{Server: s.Server}
-		s.Server.Handler = s.wrapWithSvcHeaders(s.Server.Handler)
-	}
-
 	// if TLS is enabled, make sure we prepare the Server accordingly
 	if s.Server.TLSConfig != nil {
+		// enable QUIC if desired (requires HTTP/2)
+		if HTTP2 && QUIC {
+			s.quicServer = &h2quic.Server{Server: s.Server}
+			s.Server.Handler = s.wrapWithSvcHeaders(s.Server.Handler)
+		}
+
 		// wrap the HTTP handler with a handler that does MITM detection
 		tlsh := &tlsHandler{next: s.Server.Handler}
 		s.Server.Handler = tlsh // this needs to be the "outer" handler when Serve() is called, for type assertion
@@ -127,6 +139,32 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 	return s, nil
 }
 
+// makeHTTPServerWithHeaderLimit apply minimum header limit within a group to given http.Server
+func makeHTTPServerWithHeaderLimit(s *http.Server, group []*SiteConfig) *http.Server {
+	var min int64
+	for _, cfg := range group {
+		limit := cfg.Limits.MaxRequestHeaderSize
+		if limit == 0 {
+			continue
+		}
+
+		// not set yet
+		if min == 0 {
+			min = limit
+		}
+
+		// find a better one
+		if limit < min {
+			min = limit
+		}
+	}
+
+	if min > 0 {
+		s.MaxHeaderBytes = int(min)
+	}
+	return s
+}
+
 // makeHTTPServerWithTimeouts makes an http.Server from the group of
 // configs in a way that configures timeouts (or, if not set, it uses
 // the default timeouts) by combining the configuration of each
@@ -267,7 +305,7 @@ func (s *Server) Serve(ln net.Listener) error {
 	}
 
 	err := s.Server.Serve(ln)
-	if QUIC {
+	if s.quicServer != nil {
 		s.quicServer.Close()
 	}
 	return err
@@ -275,7 +313,7 @@ func (s *Server) Serve(ln net.Listener) error {
 
 // ServePacket serves QUIC requests on pc until it is closed.
 func (s *Server) ServePacket(pc net.PacketConn) error {
-	if QUIC {
+	if s.quicServer != nil {
 		err := s.quicServer.Serve(pc.(*net.UDPConn))
 		return fmt.Errorf("serving QUIC connections: %v", err)
 	}
@@ -304,7 +342,9 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	c := context.WithValue(r.Context(), OriginalURLCtxKey, urlCopy)
 	r = r.WithContext(c)
 
-	w.Header().Set("Server", "Caddy")
+	w.Header().Set("Server", caddy.AppName)
+	sponsors := "Minio, Uptime Robot, and Sourcegraph"
+	w.Header().Set("Caddy-Sponsors", "This free web server is made possible by its sponsors: "+sponsors)
 
 	status, _ := s.serveHTTP(w, r)
 
@@ -325,6 +365,8 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 
 	// look up the virtualhost; if no match, serve error
 	vhost, pathPrefix := s.vhosts.Match(hostname + r.URL.Path)
+	c := context.WithValue(r.Context(), caddy.CtxKey("path_prefix"), pathPrefix)
+	r = r.WithContext(c)
 
 	if vhost == nil {
 		// check for ACME challenge even if vhost is nil;
@@ -337,7 +379,7 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 		if err != nil {
 			remoteHost = r.RemoteAddr
 		}
-		WriteTextResponse(w, http.StatusNotFound, "No such site at "+s.Server.Addr)
+		WriteSiteNotFound(w, r) // don't add headers outside of this function
 		log.Printf("[INFO] %s - No such site at %s (Remote: %s, Referer: %s)",
 			hostname, s.Server.Addr, remoteHost, r.Header.Get("Referer"))
 		return 0, nil
@@ -359,20 +401,6 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 		}
 	}
 
-	// Apply the path-based request body size limit
-	// The error returned by MaxBytesReader is meant to be handled
-	// by whichever middleware/plugin that receives it when calling
-	// .Read() or a similar method on the request body
-	// TODO: Make this middleware instead?
-	if r.Body != nil {
-		for _, pathlimit := range vhost.MaxRequestBodySizes {
-			if Path(r.URL.Path).Matches(pathlimit.Path) {
-				r.Body = MaxBytesReader(w, r.Body, pathlimit.Limit)
-				break
-			}
-		}
-	}
-
 	return vhost.middlewareChain.ServeHTTP(w, r)
 }
 
@@ -435,9 +463,9 @@ func (s *Server) OnStartupComplete() {
 }
 
 // defaultTimeouts stores the default timeout values to use
-// if left unset by user configuration. NOTE: Default timeouts
-// are disabled (see issue #1464).
-var defaultTimeouts Timeouts
+// if left unset by user configuration. NOTE: Most default
+// timeouts are disabled (see issues #1464 and #1733).
+var defaultTimeouts = Timeouts{IdleTimeout: 5 * time.Minute}
 
 // tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
 // connections. It's used by ListenAndServe and ListenAndServeTLS so
@@ -465,73 +493,9 @@ func (ln tcpKeepAliveListener) File() (*os.File, error) {
 	return ln.TCPListener.File()
 }
 
-// MaxBytesExceeded is the error type returned by MaxBytesReader
+// ErrMaxBytesExceeded is the error returned by MaxBytesReader
 // when the request body exceeds the limit imposed
-type MaxBytesExceeded struct{}
-
-func (err MaxBytesExceeded) Error() string {
-	return "http: request body too large"
-}
-
-// MaxBytesReader and its associated methods are borrowed from the
-// Go Standard library (comments intact). The only difference is that
-// it returns a MaxBytesExceeded error instead of a generic error message
-// when the request body has exceeded the requested limit
-func MaxBytesReader(w http.ResponseWriter, r io.ReadCloser, n int64) io.ReadCloser {
-	return &maxBytesReader{w: w, r: r, n: n}
-}
-
-type maxBytesReader struct {
-	w   http.ResponseWriter
-	r   io.ReadCloser // underlying reader
-	n   int64         // max bytes remaining
-	err error         // sticky error
-}
-
-func (l *maxBytesReader) Read(p []byte) (n int, err error) {
-	if l.err != nil {
-		return 0, l.err
-	}
-	if len(p) == 0 {
-		return 0, nil
-	}
-	// If they asked for a 32KB byte read but only 5 bytes are
-	// remaining, no need to read 32KB. 6 bytes will answer the
-	// question of the whether we hit the limit or go past it.
-	if int64(len(p)) > l.n+1 {
-		p = p[:l.n+1]
-	}
-	n, err = l.r.Read(p)
-
-	if int64(n) <= l.n {
-		l.n -= int64(n)
-		l.err = err
-		return n, err
-	}
-
-	n = int(l.n)
-	l.n = 0
-
-	// The server code and client code both use
-	// maxBytesReader. This "requestTooLarge" check is
-	// only used by the server code. To prevent binaries
-	// which only using the HTTP Client code (such as
-	// cmd/go) from also linking in the HTTP server, don't
-	// use a static type assertion to the server
-	// "*response" type. Check this interface instead:
-	type requestTooLarger interface {
-		requestTooLarge()
-	}
-	if res, ok := l.w.(requestTooLarger); ok {
-		res.requestTooLarge()
-	}
-	l.err = MaxBytesExceeded{}
-	return n, l.err
-}
-
-func (l *maxBytesReader) Close() error {
-	return l.r.Close()
-}
+var ErrMaxBytesExceeded = errors.New("http: request body too large")
 
 // DefaultErrorFunc responds to an HTTP request with a simple description
 // of the specified HTTP status code.
@@ -539,6 +503,19 @@ func DefaultErrorFunc(w http.ResponseWriter, r *http.Request, status int) {
 	WriteTextResponse(w, status, fmt.Sprintf("%d %s\n", status, http.StatusText(status)))
 }
 
+const httpStatusMisdirectedRequest = 421 // RFC 7540, 9.1.2
+
+// WriteSiteNotFound writes appropriate error code to w, signaling that
+// requested host is not served by Caddy on a given port.
+func WriteSiteNotFound(w http.ResponseWriter, r *http.Request) {
+	status := http.StatusNotFound
+	if r.ProtoMajor >= 2 {
+		// TODO: use http.StatusMisdirectedRequest when it gets defined
+		status = httpStatusMisdirectedRequest
+	}
+	WriteTextResponse(w, status, fmt.Sprintf("%d Site %s is not served on this interface\n", status, r.Host))
+}
+
 // WriteTextResponse writes body with code status to w. The body will
 // be interpreted as plain text.
 func WriteTextResponse(w http.ResponseWriter, status int, body string) {

caddyhttp/httpserver/server_test.go

@@ -15,7 +15,7 @@ func TestAddress(t *testing.T) {
 	}
 }
 
-func TestMakeHTTPServer(t *testing.T) {
+func TestMakeHTTPServerWithTimeouts(t *testing.T) {
 	for i, tc := range []struct {
 		group    []*SiteConfig
 		expected Timeouts
@@ -111,3 +111,36 @@ func TestMakeHTTPServer(t *testing.T) {
 		}
 	}
 }
+
+func TestMakeHTTPServerWithHeaderLimit(t *testing.T) {
+	for name, c := range map[string]struct {
+		group  []*SiteConfig
+		expect int
+	}{
+		"disable": {
+			group:  []*SiteConfig{{}},
+			expect: 0,
+		},
+		"oneSite": {
+			group: []*SiteConfig{{Limits: Limits{
+				MaxRequestHeaderSize: 100,
+			}}},
+			expect: 100,
+		},
+		"multiSites": {
+			group: []*SiteConfig{
+				{Limits: Limits{MaxRequestHeaderSize: 100}},
+				{Limits: Limits{MaxRequestHeaderSize: 50}},
+			},
+			expect: 50,
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			actual := makeHTTPServerWithHeaderLimit(&http.Server{}, c.group)
+			if got := actual.MaxHeaderBytes; got != c.expect {
+				t.Errorf("Expect %d, but got %d", c.expect, got)
+			}
+		})
+	}
+}

caddyhttp/httpserver/siteconfig.go

@@ -38,8 +38,8 @@ type SiteConfig struct {
 	// for a request.
 	HiddenFiles []string
 
-	// Max amount of bytes a request can send on a given path
-	MaxRequestBodySizes []PathLimit
+	// Max request's header/body size
+	Limits Limits
 
 	// The path to the Caddyfile used to generate this site config
 	originCaddyfile string
@@ -52,6 +52,10 @@ type SiteConfig struct {
 	// preserving functionality needed for proxying,
 	// websockets, etc.
 	Timeouts Timeouts
+
+	// If true, any requests not matching other site definitions
+	// may be served by this site.
+	FallbackSite bool
 }
 
 // Timeouts specify various timeouts for a server to use.
@@ -71,6 +75,12 @@ type Timeouts struct {
 	IdleTimeoutSet       bool
 }
 
+// Limits specify size limit of request's header and body.
+type Limits struct {
+	MaxRequestHeaderSize int64
+	MaxRequestBodySizes  []PathLimit
+}
+
 // PathLimit is a mapping from a site's path to its corresponding
 // maximum request body size (in bytes)
 type PathLimit struct {

caddyhttp/httpserver/tplcontext.go

@@ -424,12 +424,13 @@ func (c Context) RandomString(minLen, maxLen int) string {
 	return string(result)
 }
 
-// Push adds a preload link in response header for server push
-func (c Context) Push(link string) string {
+// AddLink adds a link header in response
+// see https://www.w3.org/wiki/LinkHeader
+func (c Context) AddLink(link string) string {
 	if c.responseHeader == nil {
 		return ""
 	}
-	c.responseHeader.Add("Link", "<"+link+">; rel=preload")
+	c.responseHeader.Add("Link", link)
 	return ""
 }
 

caddyhttp/httpserver/tplcontext_test.go

@@ -497,7 +497,7 @@ func TestMethod(t *testing.T) {
 
 }
 
-func TestPathMatches(t *testing.T) {
+func TestContextPathMatches(t *testing.T) {
 	context := getContextOrFail(t)
 
 	tests := []struct {
@@ -877,18 +877,18 @@ func TestFiles(t *testing.T) {
 	}
 }
 
-func TestPush(t *testing.T) {
+func TestAddLink(t *testing.T) {
 	for name, c := range map[string]struct {
 		input       string
 		expectLinks []string
 	}{
 		"oneLink": {
-			input:       `{{.Push "/test.css"}}`,
+			input:       `{{.AddLink "</test.css>; rel=preload"}}`,
 			expectLinks: []string{"</test.css>; rel=preload"},
 		},
 		"multipleLinks": {
-			input:       `{{.Push "/test1.css"}} {{.Push "/test2.css"}}`,
-			expectLinks: []string{"</test1.css>; rel=preload", "</test2.css>; rel=preload"},
+			input:       `{{.AddLink "</test1.css>; rel=preload"}} {{.AddLink "</test2.css>; rel=meta"}}`,
+			expectLinks: []string{"</test1.css>; rel=preload", "</test2.css>; rel=meta"},
 		},
 	} {
 		c := c

caddyhttp/httpserver/vhosttrie.go

@@ -10,14 +10,15 @@ import (
 // wildcards as TLS certificates support them), then
 // by longest matching path.
 type vhostTrie struct {
-	edges map[string]*vhostTrie
-	site  *SiteConfig // site to match on this node; also known as a virtual host
-	path  string      // the path portion of the key for the associated site
+	fallbackHosts []string
+	edges         map[string]*vhostTrie
+	site          *SiteConfig // site to match on this node; also known as a virtual host
+	path          string      // the path portion of the key for the associated site
 }
 
 // newVHostTrie returns a new vhostTrie.
 func newVHostTrie() *vhostTrie {
-	return &vhostTrie{edges: make(map[string]*vhostTrie)}
+	return &vhostTrie{edges: make(map[string]*vhostTrie), fallbackHosts: []string{"0.0.0.0", ""}}
 }
 
 // Insert adds stack to t keyed by key. The key should be
@@ -57,13 +58,13 @@ func (t *vhostTrie) insertPath(remainingPath, originalPath string, site *SiteCon
 // A typical key will be in the form "host" or "host/path".
 func (t *vhostTrie) Match(key string) (*SiteConfig, string) {
 	host, path := t.splitHostPath(key)
-	// try the given host, then, if no match, try wildcard hosts
+	// try the given host, then, if no match, try fallback hosts
 	branch := t.matchHost(host)
-	if branch == nil {
-		branch = t.matchHost("0.0.0.0")
-	}
-	if branch == nil {
-		branch = t.matchHost("")
+	for _, h := range t.fallbackHosts {
+		if branch != nil {
+			break
+		}
+		branch = t.matchHost(h)
 	}
 	if branch == nil {
 		return nil, ""

caddyhttp/internalsrv/internal.go

@@ -7,8 +7,6 @@
 package internalsrv
 
 import (
-	"bufio"
-	"net"
 	"net/http"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -34,7 +32,6 @@ func isInternalRedirect(w http.ResponseWriter) bool {
 
 // ServeHTTP implements the httpserver.Handler interface.
 func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-
 	// Internal location requested? -> Not found.
 	for _, prefix := range i.Paths {
 		if httpserver.Path(r.URL.Path).Matches(prefix) {
@@ -44,7 +41,7 @@ func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 
 	// Use internal response writer to ignore responses that will be
 	// redirected to internal locations
-	iw := internalResponseWriter{ResponseWriter: w}
+	iw := internalResponseWriter{ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w}}
 	status, err := i.Next.ServeHTTP(iw, r)
 
 	for c := 0; c < maxRedirectCount && isInternalRedirect(iw); c++ {
@@ -52,7 +49,6 @@ func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 		// "down the chain"
 		r.URL.Path = iw.Header().Get(redirectHeader)
 		iw.ClearHeader()
-
 		status, err = i.Next.ServeHTTP(iw, r)
 	}
 
@@ -69,7 +65,7 @@ func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 // calls to Write and WriteHeader if the response should be redirected to an
 // internal location.
 type internalResponseWriter struct {
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 }
 
 // ClearHeader removes script headers that would interfere with follow up
@@ -84,7 +80,7 @@ func (w internalResponseWriter) ClearHeader() {
 // internal location.
 func (w internalResponseWriter) WriteHeader(code int) {
 	if !isInternalRedirect(w) {
-		w.ResponseWriter.WriteHeader(code)
+		w.ResponseWriterWrapper.WriteHeader(code)
 	}
 }
 
@@ -94,53 +90,8 @@ func (w internalResponseWriter) Write(b []byte) (int, error) {
 	if isInternalRedirect(w) {
 		return 0, nil
 	}
-	return w.ResponseWriter.Write(b)
-}
-
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (w internalResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := w.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: w.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (w internalResponseWriter) Flush() {
-	if f, ok := w.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: w.ResponseWriter})
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-// It panics if the underlying ResponseWriter is not a CloseNotifier.
-func (w internalResponseWriter) CloseNotify() <-chan bool {
-	if cn, ok := w.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: w.ResponseWriter})
-}
-
-// Push implements http.Pusher.
-// It just inherits the underlying ResponseWriter's Push method.
-// It panics if the underlying ResponseWriter is not a Pusher.
-func (w internalResponseWriter) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := w.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return httpserver.NonPusherError{Underlying: w.ResponseWriter}
+	return w.ResponseWriterWrapper.Write(b)
 }
 
 // Interface guards
-var (
-	_ http.Pusher        = internalResponseWriter{}
-	_ http.Flusher       = internalResponseWriter{}
-	_ http.CloseNotifier = internalResponseWriter{}
-	_ http.Hijacker      = internalResponseWriter{}
-)
+var _ httpserver.HTTPInterfaces = internalResponseWriter{}

caddyhttp/internalsrv/setup.go

@@ -30,10 +30,12 @@ func internalParse(c *caddy.Controller) ([]string, error) {
 	var paths []string
 
 	for c.Next() {
-		if !c.NextArg() {
-			return paths, c.ArgErr()
+		if c.NextArg() {
+			paths = append(paths, c.Val())
+		}
+		if c.NextArg() {
+			return nil, c.ArgErr()
 		}
-		paths = append(paths, c.Val())
 	}
 
 	return paths, nil

caddyhttp/internalsrv/setup_test.go

@@ -41,10 +41,14 @@ func TestInternalParse(t *testing.T) {
 		shouldErr             bool
 		expectedInternalPaths []string
 	}{
+		{`internal`, false, []string{}},
+
 		{`internal /internal`, false, []string{"/internal"}},
 
 		{`internal /internal1
 		  internal /internal2`, false, []string{"/internal1", "/internal2"}},
+
+		{`internal /internal1 /internal2`, true, nil},
 	}
 	for i, test := range tests {
 		actualInternalPaths, err := internalParse(caddy.NewTestController("http", test.inputInternalPaths))

caddyhttp/limits/handler.go

@@ -0,0 +1,90 @@
+package limits
+
+import (
+	"io"
+	"net/http"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+// Limit is a middleware to control request body size
+type Limit struct {
+	Next       httpserver.Handler
+	BodyLimits []httpserver.PathLimit
+}
+
+func (l Limit) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
+	if r.Body == nil {
+		return l.Next.ServeHTTP(w, r)
+	}
+
+	// apply the path-based request body size limit.
+	for _, bl := range l.BodyLimits {
+		if httpserver.Path(r.URL.Path).Matches(bl.Path) {
+			r.Body = MaxBytesReader(w, r.Body, bl.Limit)
+			break
+		}
+	}
+
+	return l.Next.ServeHTTP(w, r)
+}
+
+// MaxBytesReader and its associated methods are borrowed from the
+// Go Standard library (comments intact). The only difference is that
+// it returns a ErrMaxBytesExceeded error instead of a generic error message
+// when the request body has exceeded the requested limit
+func MaxBytesReader(w http.ResponseWriter, r io.ReadCloser, n int64) io.ReadCloser {
+	return &maxBytesReader{w: w, r: r, n: n}
+}
+
+type maxBytesReader struct {
+	w   http.ResponseWriter
+	r   io.ReadCloser // underlying reader
+	n   int64         // max bytes remaining
+	err error         // sticky error
+}
+
+func (l *maxBytesReader) Read(p []byte) (n int, err error) {
+	if l.err != nil {
+		return 0, l.err
+	}
+	if len(p) == 0 {
+		return 0, nil
+	}
+	// If they asked for a 32KB byte read but only 5 bytes are
+	// remaining, no need to read 32KB. 6 bytes will answer the
+	// question of the whether we hit the limit or go past it.
+	if int64(len(p)) > l.n+1 {
+		p = p[:l.n+1]
+	}
+	n, err = l.r.Read(p)
+
+	if int64(n) <= l.n {
+		l.n -= int64(n)
+		l.err = err
+		return n, err
+	}
+
+	n = int(l.n)
+	l.n = 0
+
+	// The server code and client code both use
+	// maxBytesReader. This "requestTooLarge" check is
+	// only used by the server code. To prevent binaries
+	// which only using the HTTP Client code (such as
+	// cmd/go) from also linking in the HTTP server, don't
+	// use a static type assertion to the server
+	// "*response" type. Check this interface instead:
+	type requestTooLarger interface {
+		requestTooLarge()
+	}
+	if res, ok := l.w.(requestTooLarger); ok {
+		res.requestTooLarge()
+	}
+	l.err = httpserver.ErrMaxBytesExceeded
+	return n, l.err
+}
+
+func (l *maxBytesReader) Close() error {
+	return l.r.Close()
+}

caddyhttp/limits/handler_test.go

@@ -0,0 +1,35 @@
+package limits
+
+import (
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func TestBodySizeLimit(t *testing.T) {
+	var (
+		gotContent    []byte
+		gotError      error
+		expectContent = "hello"
+	)
+	l := Limit{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			gotContent, gotError = ioutil.ReadAll(r.Body)
+			return 0, nil
+		}),
+		BodyLimits: []httpserver.PathLimit{{Path: "/", Limit: int64(len(expectContent))}},
+	}
+
+	r := httptest.NewRequest("GET", "/", strings.NewReader(expectContent+expectContent))
+	l.ServeHTTP(httptest.NewRecorder(), r)
+	if got := string(gotContent); got != expectContent {
+		t.Errorf("expected content[%s], got[%s]", expectContent, got)
+	}
+	if gotError != httpserver.ErrMaxBytesExceeded {
+		t.Errorf("expect error %v, got %v", httpserver.ErrMaxBytesExceeded, gotError)
+	}
+}

caddyhttp/limits/setup.go

@@ -1,4 +1,4 @@
-package maxrequestbody
+package limits
 
 import (
 	"errors"
@@ -12,13 +12,13 @@ import (
 
 const (
 	serverType = "http"
-	pluginName = "maxrequestbody"
+	pluginName = "limits"
 )
 
 func init() {
 	caddy.RegisterPlugin(pluginName, caddy.Plugin{
 		ServerType: serverType,
-		Action:     setupMaxRequestBody,
+		Action:     setupLimits,
 	})
 }
 
@@ -28,56 +28,97 @@ type pathLimitUnparsed struct {
 	Limit string
 }
 
-func setupMaxRequestBody(c *caddy.Controller) error {
+func setupLimits(c *caddy.Controller) error {
+	bls, err := parseLimits(c)
+	if err != nil {
+		return err
+	}
+
+	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
+		return Limit{Next: next, BodyLimits: bls}
+	})
+	return nil
+}
+
+func parseLimits(c *caddy.Controller) ([]httpserver.PathLimit, error) {
 	config := httpserver.GetConfig(c)
 
 	if !c.Next() {
-		return c.ArgErr()
+		return nil, c.ArgErr()
 	}
 
 	args := c.RemainingArgs()
 	argList := []pathLimitUnparsed{}
+	headerLimit := ""
 
 	switch len(args) {
 	case 0:
-		// Format: { <path> <limit> ... }
+		// Format: limits {
+		//	header <limit>
+		//	body <path> <limit>
+		//	body <limit>
+		//	...
+		// }
 		for c.NextBlock() {
-			path := c.Val()
-			if !c.NextArg() {
-				// Uneven pairing of path/limit
-				return c.ArgErr()
+			kind := c.Val()
+			pathOrLimit := c.RemainingArgs()
+			switch kind {
+			case "header":
+				if len(pathOrLimit) != 1 {
+					return nil, c.ArgErr()
+				}
+				headerLimit = pathOrLimit[0]
+			case "body":
+				if len(pathOrLimit) == 1 {
+					argList = append(argList, pathLimitUnparsed{
+						Path:  "/",
+						Limit: pathOrLimit[0],
+					})
+					break
+				}
+
+				if len(pathOrLimit) == 2 {
+					argList = append(argList, pathLimitUnparsed{
+						Path:  pathOrLimit[0],
+						Limit: pathOrLimit[1],
+					})
+					break
+				}
+
+				fallthrough
+			default:
+				return nil, c.ArgErr()
 			}
-			argList = append(argList, pathLimitUnparsed{
-				Path:  path,
-				Limit: c.Val(),
-			})
 		}
 	case 1:
-		// Format: <limit>
+		// Format: limits <limit>
+		headerLimit = args[0]
 		argList = []pathLimitUnparsed{{
 			Path:  "/",
 			Limit: args[0],
 		}}
-	case 2:
-		// Format: <path> <limit>
-		argList = []pathLimitUnparsed{{
-			Path:  args[0],
-			Limit: args[1],
-		}}
 	default:
-		return c.ArgErr()
+		return nil, c.ArgErr()
 	}
 
-	pathLimit, err := parseArguments(argList)
-	if err != nil {
-		return c.ArgErr()
+	if headerLimit != "" {
+		size := parseSize(headerLimit)
+		if size < 1 { // also disallow size = 0
+			return nil, c.ArgErr()
+		}
+		config.Limits.MaxRequestHeaderSize = size
 	}
 
-	SortPathLimits(pathLimit)
+	if len(argList) > 0 {
+		pathLimit, err := parseArguments(argList)
+		if err != nil {
+			return nil, c.ArgErr()
+		}
+		SortPathLimits(pathLimit)
+		config.Limits.MaxRequestBodySizes = pathLimit
+	}
 
-	config.MaxRequestBodySizes = pathLimit
-
-	return nil
+	return config.Limits.MaxRequestBodySizes, nil
 }
 
 func parseArguments(args []pathLimitUnparsed) ([]httpserver.PathLimit, error) {

caddyhttp/limits/setup_test.go

@@ -1,4 +1,4 @@
-package maxrequestbody
+package limits
 
 import (
 	"reflect"
@@ -14,32 +14,98 @@ const (
 	GB = 1024 * 1024 * 1024
 )
 
-func TestSetupMaxRequestBody(t *testing.T) {
-	cases := []struct {
-		input    string
-		hasError bool
+func TestParseLimits(t *testing.T) {
+	for name, c := range map[string]struct {
+		input     string
+		shouldErr bool
+		expect    httpserver.Limits
 	}{
-		// Format: { <path> <limit> ... }
-		{input: "maxrequestbody / 20MB", hasError: false},
-		// Format: <limit>
-		{input: "maxrequestbody 999KB", hasError: false},
-		// Format: { <path> <limit> ... }
-		{input: "maxrequestbody { /images 50MB /upload 10MB\n/test 10KB }", hasError: false},
-
-		// Wrong formats
-		{input: "maxrequestbody typo { /images 50MB }", hasError: true},
-		{input: "maxrequestbody 999MB /home 20KB", hasError: true},
-	}
-	for caseNum, c := range cases {
-		controller := caddy.NewTestController("", c.input)
-		err := setupMaxRequestBody(controller)
-
-		if c.hasError && (err == nil) {
-			t.Errorf("Expecting error for case %v but none encountered", caseNum)
-		}
-		if !c.hasError && (err != nil) {
-			t.Errorf("Expecting no error for case %v but encountered %v", caseNum, err)
-		}
+		"catchAll": {
+			input: `limits 2kb`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 2 * KB,
+				MaxRequestBodySizes:  []httpserver.PathLimit{{Path: "/", Limit: 2 * KB}},
+			},
+		},
+		"onlyHeader": {
+			input: `limits {
+				header 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 2 * KB,
+			},
+		},
+		"onlyBody": {
+			input: `limits {
+				body 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestBodySizes: []httpserver.PathLimit{{Path: "/", Limit: 2 * KB}},
+			},
+		},
+		"onlyBodyWithPath": {
+			input: `limits {
+				body /test 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestBodySizes: []httpserver.PathLimit{{Path: "/test", Limit: 2 * KB}},
+			},
+		},
+		"mixture": {
+			input: `limits {
+				header 1kb
+				body 2kb
+				body /bar 3kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 1 * KB,
+				MaxRequestBodySizes: []httpserver.PathLimit{
+					{Path: "/bar", Limit: 3 * KB},
+					{Path: "/", Limit: 2 * KB},
+				},
+			},
+		},
+		"invalidFormat": {
+			input:     `limits a b`,
+			shouldErr: true,
+		},
+		"invalidHeaderFormat": {
+			input: `limits {
+				header / 100
+			}`,
+			shouldErr: true,
+		},
+		"invalidBodyFormat": {
+			input: `limits {
+				body / 100 200
+			}`,
+			shouldErr: true,
+		},
+		"invalidKind": {
+			input: `limits {
+				head 100
+			}`,
+			shouldErr: true,
+		},
+		"invalidLimitSize": {
+			input:     `limits 10bk`,
+			shouldErr: true,
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			controller := caddy.NewTestController("", c.input)
+			_, err := parseLimits(controller)
+			if c.shouldErr && err == nil {
+				t.Error("failed to get expected error")
+			}
+			if !c.shouldErr && err != nil {
+				t.Errorf("got unexpected error: %v", err)
+			}
+			if got := httpserver.GetConfig(controller).Limits; !reflect.DeepEqual(got, c.expect) {
+				t.Errorf("expect %#v, but got %#v", c.expect, got)
+			}
+		})
 	}
 }
 

caddyhttp/log/setup.go

@@ -1,6 +1,8 @@
 package log
 
 import (
+	"strings"
+
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
@@ -36,63 +38,48 @@ func logParse(c *caddy.Controller) ([]*Rule, error) {
 
 		for c.NextBlock() {
 			what := c.Val()
-			if !c.NextArg() {
-				return nil, c.ArgErr()
-			}
-			where := c.Val()
+			where := c.RemainingArgs()
 
 			// only support roller related options inside a block
 			if !httpserver.IsLogRollerSubdirective(what) {
 				return nil, c.ArgErr()
 			}
 
-			if err := httpserver.ParseRoller(logRoller, what, where); err != nil {
+			if err := httpserver.ParseRoller(logRoller, what, where...); err != nil {
 				return nil, err
 			}
 		}
 
-		if len(args) == 0 {
-			// Nothing specified; use defaults
-			rules = appendEntry(rules, "/", &Entry{
-				Log: &httpserver.Logger{
-					Output: DefaultLogFilename,
-					Roller: logRoller,
-				},
-				Format: DefaultLogFormat,
-			})
-		} else if len(args) == 1 {
+		path := "/"
+		format := DefaultLogFormat
+		output := DefaultLogFilename
+
+		switch len(args) {
+		case 0:
+			// nothing to change
+		case 1:
 			// Only an output file specified
-			rules = appendEntry(rules, "/", &Entry{
-				Log: &httpserver.Logger{
-					Output: args[0],
-					Roller: logRoller,
-				},
-				Format: DefaultLogFormat,
-			})
-		} else {
+			output = args[0]
+		case 2, 3:
 			// Path scope, output file, and maybe a format specified
-
-			format := DefaultLogFormat
-
+			path = args[0]
+			output = args[1]
 			if len(args) > 2 {
-				switch args[2] {
-				case "{common}":
-					format = CommonLogFormat
-				case "{combined}":
-					format = CombinedLogFormat
-				default:
-					format = args[2]
-				}
+				format = strings.Replace(args[2], "{common}", CommonLogFormat, -1)
+				format = strings.Replace(format, "{combined}", CombinedLogFormat, -1)
 			}
-
-			rules = appendEntry(rules, args[0], &Entry{
-				Log: &httpserver.Logger{
-					Output: args[1],
-					Roller: logRoller,
-				},
-				Format: format,
-			})
+		default:
+			// Maximum number of args in log directive is 3.
+			return nil, c.ArgErr()
 		}
+
+		rules = appendEntry(rules, path, &Entry{
+			Log: &httpserver.Logger{
+				Output: output,
+				Roller: logRoller,
+			},
+			Format: format,
+		})
 	}
 
 	return rules, nil

caddyhttp/log/setup_test.go

@@ -124,6 +124,16 @@ func TestLogParse(t *testing.T) {
 				Format: CommonLogFormat,
 			}},
 		}}},
+		{`log /myapi log.txt "prefix {common} suffix"`, false, []Rule{{
+			PathScope: "/myapi",
+			Entries: []*Entry{{
+				Log: &httpserver.Logger{
+					Output: "log.txt",
+					Roller: httpserver.DefaultLogRoller(),
+				},
+				Format: "prefix " + CommonLogFormat + " suffix",
+			}},
+		}}},
 		{`log /test accesslog.txt {combined}`, false, []Rule{{
 			PathScope: "/test",
 			Entries: []*Entry{{
@@ -134,6 +144,16 @@ func TestLogParse(t *testing.T) {
 				Format: CombinedLogFormat,
 			}},
 		}}},
+		{`log /test accesslog.txt "prefix {combined} suffix"`, false, []Rule{{
+			PathScope: "/test",
+			Entries: []*Entry{{
+				Log: &httpserver.Logger{
+					Output: "accesslog.txt",
+					Roller: httpserver.DefaultLogRoller(),
+				},
+				Format: "prefix " + CombinedLogFormat + " suffix",
+			}},
+		}}},
 		{`log /api1 log.txt
 		  log /api2 accesslog.txt {combined}`, false, []Rule{{
 			PathScope: "/api1",
@@ -174,7 +194,12 @@ func TestLogParse(t *testing.T) {
 				Format: "{when}",
 			}},
 		}}},
-		{`log access.log { rotate_size 2 rotate_age 10 rotate_keep 3 }`, false, []Rule{{
+		{`log access.log {
+			rotate_size 2
+			rotate_age 10
+			rotate_keep 3
+			rotate_compress
+		}`, false, []Rule{{
 			PathScope: "/",
 			Entries: []*Entry{{
 				Log: &httpserver.Logger{
@@ -183,6 +208,7 @@ func TestLogParse(t *testing.T) {
 						MaxSize:    2,
 						MaxAge:     10,
 						MaxBackups: 3,
+						Compress:   true,
 						LocalTime:  true,
 					}},
 				Format: DefaultLogFormat,
@@ -205,8 +231,11 @@ func TestLogParse(t *testing.T) {
 				Format: "{when}",
 			}},
 		}}},
+		{`log access.log { rotate_size 2 rotate_age 10 rotate_keep 3 }`, true, nil},
+		{`log access.log { rotate_compress invalid }`, true, nil},
 		{`log access.log { rotate_size }`, true, nil},
 		{`log access.log { invalid_option 1 }`, true, nil},
+		{`log / acccess.log "{remote} - [{when}] "{method} {port}" {scheme} {mitm} "`, true, nil},
 	}
 	for i, test := range tests {
 		c := caddy.NewTestController("http", test.inputLogRules)

caddyhttp/markdown/markdown.go

@@ -53,6 +53,9 @@ type Config struct {
 
 	// Template(s) to render with
 	Template *template.Template
+
+	// a pair of template's name and its underlying file path
+	TemplateFiles map[string]string
 }
 
 // ServeHTTP implements the http.Handler interface.

caddyhttp/markdown/markdown_test.go

@@ -1,17 +1,15 @@
 package markdown
 
 import (
-	"bufio"
 	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
 	"os"
 	"path/filepath"
-	"strings"
 	"testing"
 	"text/template"
-	"time"
 
+	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 	"github.com/russross/blackfriday"
 )
@@ -79,19 +77,23 @@ func TestMarkdown(t *testing.T) {
 		}),
 	}
 
-	req, err := http.NewRequest("GET", "/blog/test.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
+	get := func(url string) string {
+		req, err := http.NewRequest("GET", url, nil)
+		if err != nil {
+			t.Fatalf("Could not create HTTP request: %v", err)
+		}
+		rec := httptest.NewRecorder()
+		code, err := md.ServeHTTP(rec, req)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if code != http.StatusOK {
+			t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, code)
+		}
+		return rec.Body.String()
 	}
 
-	rec := httptest.NewRecorder()
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-
-	respBody := rec.Body.String()
+	respBody := get("/blog/test.md")
 	expectedBody := `<!DOCTYPE html>
 <html>
 <head>
@@ -99,7 +101,6 @@ func TestMarkdown(t *testing.T) {
 </head>
 <body>
 <h1>Header for: Markdown test 1</h1>
-
 Welcome to A Caddy website!
 <h2>Welcome on the blog</h2>
 
@@ -113,46 +114,26 @@ Welcome to A Caddy website!
 </body>
 </html>
 `
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 
-	req, err = http.NewRequest("GET", "/docflags/test.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
-	}
-	rec = httptest.NewRecorder()
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-	respBody = rec.Body.String()
+	respBody = get("/docflags/test.md")
 	expectedBody = `Doc.var_string hello
-Doc.var_bool <no value>
-DocFlags.var_string <no value>
-DocFlags.var_bool true`
+Doc.var_bool true
+`
 
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 
-	req, err = http.NewRequest("GET", "/log/test.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
-	}
-	rec = httptest.NewRecorder()
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-	respBody = rec.Body.String()
+	respBody = get("/log/test.md")
 	expectedBody = `<!DOCTYPE html>
 <html>
 	<head>
 		<title>Markdown test 2</title>
 		<meta charset="utf-8">
+		
 		<link rel="stylesheet" href="/resources/css/log.css">
 		<link rel="stylesheet" href="/resources/css/default.css">
 		<script src="/resources/js/log.js"></script>
@@ -171,26 +152,11 @@ DocFlags.var_bool true`
 	</body>
 </html>`
 
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 
-	req, err = http.NewRequest("GET", "/og/first.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
-	}
-	rec = httptest.NewRecorder()
-	currenttime := time.Now().Local().Add(-time.Second)
-	_ = os.Chtimes("testdata/og/first.md", currenttime, currenttime)
-	currenttime = time.Now().Local()
-	_ = os.Chtimes("testdata/og_static/og/first.md/index.html", currenttime, currenttime)
-	time.Sleep(time.Millisecond * 200)
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-	respBody = rec.Body.String()
+	respBody = get("/og/first.md")
 	expectedBody = `<!DOCTYPE html>
 <html>
 <head>
@@ -198,32 +164,18 @@ DocFlags.var_bool true`
 </head>
 <body>
 <h1>Header for: first_post</h1>
-
 Welcome to title!
 <h1>Test h1</h1>
 
 </body>
-</html>`
+</html>
+`
 
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 }
 
-func equalStrings(s1, s2 string) bool {
-	s1 = strings.TrimSpace(s1)
-	s2 = strings.TrimSpace(s2)
-	in := bufio.NewScanner(strings.NewReader(s1))
-	for in.Scan() {
-		txt := strings.TrimSpace(in.Text())
-		if !strings.HasPrefix(strings.TrimSpace(s2), txt) {
-			return false
-		}
-		s2 = strings.Replace(s2, txt, "", 1)
-	}
-	return true
-}
-
 func setDefaultTemplate(filename string) *template.Template {
 	buf, err := ioutil.ReadFile(filename)
 	if err != nil {
@@ -232,3 +184,68 @@ func setDefaultTemplate(filename string) *template.Template {
 
 	return template.Must(GetDefaultTemplate().Parse(string(buf)))
 }
+
+func TestTemplateReload(t *testing.T) {
+	const (
+		templateFile = "testdata/test.html"
+		targetFile   = "testdata/hello.md"
+	)
+	c := caddy.NewTestController("http", `markdown {
+		template `+templateFile+`
+	}`)
+
+	err := ioutil.WriteFile(templateFile, []byte("hello {{.Doc.body}}"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = ioutil.WriteFile(targetFile, []byte("caddy"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer func() {
+		os.Remove(templateFile)
+		os.Remove(targetFile)
+	}()
+
+	config, err := markdownParse(c)
+	if err != nil {
+		t.Fatal(err)
+	}
+	md := Markdown{
+		Root:    "./testdata",
+		FileSys: http.Dir("./testdata"),
+		Configs: config,
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			t.Fatalf("Next shouldn't be called")
+			return 0, nil
+		}),
+	}
+
+	req := httptest.NewRequest("GET", "/hello.md", nil)
+	get := func() string {
+		rec := httptest.NewRecorder()
+		code, err := md.ServeHTTP(rec, req)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if code != http.StatusOK {
+			t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, code)
+		}
+		return rec.Body.String()
+	}
+
+	if expect, got := "hello <p>caddy</p>\n", get(); expect != got {
+		t.Fatalf("Expected body:\n%q\nbut got:\n%q", expect, got)
+	}
+
+	// update template
+	err = ioutil.WriteFile(templateFile, []byte("hi {{.Doc.body}}"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if expect, got := "hi <p>caddy</p>\n", get(); expect != got {
+		t.Fatalf("Expected body:\n%q\nbut got:\n%q", expect, got)
+	}
+
+}

caddyhttp/markdown/setup.go

@@ -44,10 +44,11 @@ func markdownParse(c *caddy.Controller) ([]*Config, error) {
 
 	for c.Next() {
 		md := &Config{
-			Renderer:   blackfriday.HtmlRenderer(0, "", ""),
-			Extensions: make(map[string]struct{}),
-			Template:   GetDefaultTemplate(),
-			IndexFiles: []string{},
+			Renderer:      blackfriday.HtmlRenderer(0, "", ""),
+			Extensions:    make(map[string]struct{}),
+			Template:      GetDefaultTemplate(),
+			IndexFiles:    []string{},
+			TemplateFiles: make(map[string]string),
 		}
 
 		// Get the path scope
@@ -115,28 +116,42 @@ func loadParams(c *caddy.Controller, mdc *Config) error {
 			fpath := filepath.ToSlash(filepath.Clean(cfg.Root + string(filepath.Separator) + tArgs[0]))
 
 			if err := SetTemplate(mdc.Template, "", fpath); err != nil {
-				c.Errf("default template parse error: %v", err)
+				return c.Errf("default template parse error: %v", err)
 			}
+
+			mdc.TemplateFiles[""] = fpath
 			return nil
 		case 2:
 			fpath := filepath.ToSlash(filepath.Clean(cfg.Root + string(filepath.Separator) + tArgs[1]))
 
 			if err := SetTemplate(mdc.Template, tArgs[0], fpath); err != nil {
-				c.Errf("template parse error: %v", err)
+				return c.Errf("template parse error: %v", err)
 			}
+
+			mdc.TemplateFiles[tArgs[0]] = fpath
 			return nil
 		}
 	case "templatedir":
 		if !c.NextArg() {
 			return c.ArgErr()
 		}
-		_, err := mdc.Template.ParseGlob(c.Val())
+
+		pattern := c.Val()
+		_, err := mdc.Template.ParseGlob(pattern)
 		if err != nil {
-			c.Errf("template load error: %v", err)
+			return c.Errf("template load error: %v", err)
 		}
 		if c.NextArg() {
 			return c.ArgErr()
 		}
+
+		paths, err := filepath.Glob(pattern)
+		if err != nil {
+			return c.Errf("glob %q failed: %v", pattern, err)
+		}
+		for _, path := range paths {
+			mdc.TemplateFiles[filepath.Base(path)] = path
+		}
 		return nil
 	default:
 		return c.Err("Expected valid markdown configuration property")

caddyhttp/markdown/setup_test.go

@@ -4,6 +4,7 @@ import (
 	"bytes"
 	"fmt"
 	"net/http"
+	"reflect"
 	"testing"
 	"text/template"
 
@@ -59,9 +60,10 @@ func TestMarkdownParse(t *testing.T) {
 				".md":  {},
 				".txt": {},
 			},
-			Styles:   []string{"/resources/css/blog.css"},
-			Scripts:  []string{"/resources/js/blog.js"},
-			Template: GetDefaultTemplate(),
+			Styles:        []string{"/resources/css/blog.css"},
+			Scripts:       []string{"/resources/js/blog.js"},
+			Template:      GetDefaultTemplate(),
+			TemplateFiles: make(map[string]string),
 		}}},
 		{`markdown /blog {
 	ext .md
@@ -71,12 +73,12 @@ func TestMarkdownParse(t *testing.T) {
 			Extensions: map[string]struct{}{
 				".md": {},
 			},
-			Template: GetDefaultTemplate(),
+			Template: setDefaultTemplate("./testdata/tpl_with_include.html"),
+			TemplateFiles: map[string]string{
+				"": "testdata/tpl_with_include.html",
+			},
 		}}},
 	}
-	// Setup the extra template
-	tmpl := tests[1].expectedMarkdownConfig[0].Template
-	SetTemplate(tmpl, "", "./testdata/tpl_with_include.html")
 
 	for i, test := range tests {
 		c := caddy.NewTestController("http", test.inputMarkdownConfig)
@@ -110,6 +112,10 @@ func TestMarkdownParse(t *testing.T) {
 			if ok, tx, ty := equalTemplates(actualMarkdownConfig.Template, test.expectedMarkdownConfig[j].Template); !ok {
 				t.Errorf("Test %d the %dth Markdown Config Templates did not match, expected %s to be %s", i, j, tx, ty)
 			}
+			if expect, got := test.expectedMarkdownConfig[j].TemplateFiles, actualMarkdownConfig.TemplateFiles; !reflect.DeepEqual(expect, got) {
+				t.Errorf("Test %d the %d Markdown config TemplateFiles did not match, expect %v, but got %v", i, j, expect, got)
+			}
+
 		}
 	}
 }

caddyhttp/markdown/template.go

@@ -22,7 +22,8 @@ type Data struct {
 // Include "overrides" the embedded httpserver.Context's Include()
 // method so that included files have access to d's fields.
 // Note: using {{template 'template-name' .}} instead might be better.
-func (d Data) Include(filename string) (string, error) {
+func (d Data) Include(filename string, args ...interface{}) (string, error) {
+	d.Args = args
 	return httpserver.ContextInclude(filename, d, d.Root)
 }
 
@@ -37,8 +38,18 @@ func execTemplate(c *Config, mdata metadata.Metadata, meta map[string]string, fi
 		Files:   files,
 	}
 
+	templateName := mdata.Template
+	// reload template on every request for now
+	// TODO: cache templates by a general plugin
+	if templateFile, ok := c.TemplateFiles[templateName]; ok {
+		err := SetTemplate(c.Template, templateName, templateFile)
+		if err != nil {
+			return nil, err
+		}
+	}
+
 	b := new(bytes.Buffer)
-	if err := c.Template.ExecuteTemplate(b, mdata.Template, mdData); err != nil {
+	if err := c.Template.ExecuteTemplate(b, templateName, mdData); err != nil {
 		return nil, err
 	}
 

caddyhttp/markdown/testdata/blog/test.md

@@ -0,0 +1,14 @@
+---
+title: Markdown test 1
+sitename: A Caddy website
+---
+
+## Welcome on the blog
+
+Body
+
+``` go
+func getTrue() bool {
+    return true
+}
+```

caddyhttp/markdown/testdata/docflags/template.txt

@@ -0,0 +1,2 @@
+Doc.var_string {{.Doc.var_string}}
+Doc.var_bool {{.Doc.var_bool}}

caddyhttp/markdown/testdata/docflags/test.md

@@ -0,0 +1,4 @@
+---
+var_string: hello
+var_bool: true
+---

caddyhttp/markdown/testdata/header.html

@@ -0,0 +1 @@
+<h1>Header for: {{.Doc.title}}</h1>

caddyhttp/markdown/testdata/log/test.md

@@ -0,0 +1,14 @@
+---
+title: Markdown test 2
+sitename: A Caddy website
+---
+
+## Welcome on the blog
+
+Body
+
+``` go
+func getTrue() bool {
+    return true
+}
+```

caddyhttp/markdown/testdata/markdown_tpl.html

@@ -0,0 +1,11 @@
+<!DOCTYPE html>
+<html>
+<head>
+<title>{{.Doc.title}}</title>
+</head>
+<body>
+{{.Include "header.html"}}
+Welcome to {{.Doc.sitename}}!
+{{.Doc.body}}
+</body>
+</html>

caddyhttp/markdown/testdata/og/first.md

@@ -0,0 +1,5 @@
+---
+title: first_post
+sitename: title
+---
+# Test h1

caddyhttp/markdown/testdata/tpl_with_include.html

@@ -0,0 +1,11 @@
+<!DOCTYPE html>
+<html>
+	<head>
+		<title>{{.Doc.title}}</title>
+	</head>
+	<body>
+		Welcome to {{.Doc.sitename}}!
+		<br><br>
+		{{.Doc.body}}
+	</body>
+</html>

caddyhttp/mime/mime_test.go

@@ -57,7 +57,7 @@ func nextFunc(shouldMime bool, contentType string) httpserver.Handler {
 	return httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
 		if shouldMime {
 			if w.Header().Get("Content-Type") != contentType {
-				return 0, fmt.Errorf("expected Content-Type: %v, found %v", contentType, r.Header.Get("Content-Type"))
+				return 0, fmt.Errorf("expected Content-Type: %v, found %v", contentType, w.Header().Get("Content-Type"))
 			}
 			return 0, nil
 		}

caddyhttp/proxy/policy.go

@@ -18,11 +18,13 @@ type Policy interface {
 }
 
 func init() {
-	RegisterPolicy("random", func() Policy { return &Random{} })
-	RegisterPolicy("least_conn", func() Policy { return &LeastConn{} })
-	RegisterPolicy("round_robin", func() Policy { return &RoundRobin{} })
-	RegisterPolicy("ip_hash", func() Policy { return &IPHash{} })
-	RegisterPolicy("first", func() Policy { return &First{} })
+	RegisterPolicy("random", func(arg string) Policy { return &Random{} })
+	RegisterPolicy("least_conn", func(arg string) Policy { return &LeastConn{} })
+	RegisterPolicy("round_robin", func(arg string) Policy { return &RoundRobin{} })
+	RegisterPolicy("ip_hash", func(arg string) Policy { return &IPHash{} })
+	RegisterPolicy("first", func(arg string) Policy { return &First{} })
+	RegisterPolicy("uri_hash", func(arg string) Policy { return &URIHash{} })
+	RegisterPolicy("header", func(arg string) Policy { return &Header{arg} })
 }
 
 // Random is a policy that selects up hosts from a pool at random.
@@ -56,7 +58,7 @@ func (r *Random) Select(pool HostPool, request *http.Request) *UpstreamHost {
 type LeastConn struct{}
 
 // Select selects the up host with the least number of connections in the
-// pool.  If more than one host has the same least number of connections,
+// pool. If more than one host has the same least number of connections,
 // one of the hosts is chosen at random.
 func (r *LeastConn) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	var bestHost *UpstreamHost
@@ -84,13 +86,13 @@ func (r *LeastConn) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	return bestHost
 }
 
-// RoundRobin is a policy that selects hosts based on round robin ordering.
+// RoundRobin is a policy that selects hosts based on round-robin ordering.
 type RoundRobin struct {
 	robin uint32
 	mutex sync.Mutex
 }
 
-// Select selects an up host from the pool using a round robin ordering scheme.
+// Select selects an up host from the pool using a round-robin ordering scheme.
 func (r *RoundRobin) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	poolLen := uint32(len(pool))
 	r.mutex.Lock()
@@ -106,23 +108,10 @@ func (r *RoundRobin) Select(pool HostPool, request *http.Request) *UpstreamHost
 	return nil
 }
 
-// IPHash is a policy that selects hosts based on hashing the request ip
-type IPHash struct{}
-
-func hash(s string) uint32 {
-	h := fnv.New32a()
-	h.Write([]byte(s))
-	return h.Sum32()
-}
-
-// Select selects an up host from the pool using a round robin ordering scheme.
-func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+// hostByHashing returns an available host from pool based on a hashable string
+func hostByHashing(pool HostPool, s string) *UpstreamHost {
 	poolLen := uint32(len(pool))
-	clientIP, _, err := net.SplitHostPort(request.RemoteAddr)
-	if err != nil {
-		clientIP = request.RemoteAddr
-	}
-	index := hash(clientIP) % poolLen
+	index := hash(s) % poolLen
 	for i := uint32(0); i < poolLen; i++ {
 		index += i
 		host := pool[index%poolLen]
@@ -133,10 +122,37 @@ func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	return nil
 }
 
-// First is a policy that selects the fist available host
+// hash calculates a hash based on string s
+func hash(s string) uint32 {
+	h := fnv.New32a()
+	h.Write([]byte(s))
+	return h.Sum32()
+}
+
+// IPHash is a policy that selects hosts based on hashing the request IP
+type IPHash struct{}
+
+// Select selects an up host from the pool based on hashing the request IP
+func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	clientIP, _, err := net.SplitHostPort(request.RemoteAddr)
+	if err != nil {
+		clientIP = request.RemoteAddr
+	}
+	return hostByHashing(pool, clientIP)
+}
+
+// URIHash is a policy that selects the host based on hashing the request URI
+type URIHash struct{}
+
+// Select selects the host based on hashing the URI
+func (r *URIHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	return hostByHashing(pool, request.RequestURI)
+}
+
+// First is a policy that selects the first available host
 type First struct{}
 
-// Select selects the first host from the pool, that is available
+// Select selects the first available host from the pool
 func (r *First) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	for _, host := range pool {
 		if host.Available() {
@@ -145,3 +161,22 @@ func (r *First) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	}
 	return nil
 }
+
+// Header is a policy that selects based on a hash of the given header
+type Header struct {
+	// The name of the request header, the value of which will determine
+	// how the request is routed
+	Name string
+}
+
+// Select selects the host based on hashing the header value
+func (r *Header) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	if r.Name == "" {
+		return nil
+	}
+	val := request.Header.Get(r.Name)
+	if val == "" {
+		return nil
+	}
+	return hostByHashing(pool, val)
+}

caddyhttp/proxy/policy_test.go

@@ -243,3 +243,101 @@ func TestFirstPolicy(t *testing.T) {
 		t.Error("Expected first policy host to be the second host.")
 	}
 }
+
+func TestUriPolicy(t *testing.T) {
+	pool := testPool()
+	uriPolicy := &URIHash{}
+
+	request := httptest.NewRequest(http.MethodGet, "/test", nil)
+	h := uriPolicy.Select(pool, request)
+	if h != pool[0] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	pool[0].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test_2", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the second host.")
+	}
+
+	// We should be able to resize the host pool and still be able to predict
+	// where a request will be routed with the same URI's used above
+	pool = []*UpstreamHost{
+		{
+			Name: workableServer.URL, // this should resolve (healthcheck test)
+		},
+		{
+			Name: "http://localhost:99998", // this shouldn't
+		},
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[0] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	pool[0].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test_2", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the second host.")
+	}
+
+	pool[0].Unhealthy = 1
+	pool[1].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != nil {
+		t.Error("Expected uri policy policy host to be nil.")
+	}
+}
+
+func TestHeaderPolicy(t *testing.T) {
+	pool := testPool()
+	tests := []struct {
+		Policy             *Header
+		RequestHeaderName  string
+		RequestHeaderValue string
+		NilHost            bool
+		HostIndex          int
+	}{
+		{&Header{""}, "", "", true, 0},
+		{&Header{""}, "Affinity", "somevalue", true, 0},
+		{&Header{""}, "Affinity", "", true, 0},
+
+		{&Header{"Affinity"}, "", "", true, 0},
+		{&Header{"Affinity"}, "Affinity", "somevalue", false, 1},
+		{&Header{"Affinity"}, "Affinity", "somevalue2", false, 0},
+		{&Header{"Affinity"}, "Affinity", "somevalue3", false, 2},
+		{&Header{"Affinity"}, "Affinity", "", true, 0},
+	}
+
+	for idx, test := range tests {
+		request, _ := http.NewRequest("GET", "/", nil)
+		if test.RequestHeaderName != "" {
+			request.Header.Add(test.RequestHeaderName, test.RequestHeaderValue)
+		}
+
+		host := test.Policy.Select(pool, request)
+		if test.NilHost && host != nil {
+			t.Errorf("%d: Expected host to be nil", idx)
+		}
+		if !test.NilHost && host == nil {
+			t.Errorf("%d: Did not expect host to be nil", idx)
+		}
+		if !test.NilHost && host != pool[test.HostIndex] {
+			t.Errorf("%d: Expected Header policy to be host %d", idx, test.HostIndex)
+		}
+	}
+}

caddyhttp/proxy/proxy.go

@@ -228,7 +228,7 @@ func (p Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 			return 0, nil
 		}
 
-		if _, ok := backendErr.(httpserver.MaxBytesExceeded); ok {
+		if backendErr == httpserver.ErrMaxBytesExceeded {
 			return http.StatusRequestEntityTooLarge, backendErr
 		}
 

caddyhttp/proxy/proxy_test.go

@@ -14,6 +14,7 @@ import (
 	"net/http/httptest"
 	"net/url"
 	"os"
+	"path"
 	"path/filepath"
 	"reflect"
 	"runtime"
@@ -23,6 +24,7 @@ import (
 	"testing"
 	"time"
 
+	"github.com/lucas-clemente/quic-go/h2quic"
 	"github.com/mholt/caddy/caddyfile"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 
@@ -44,32 +46,62 @@ func TestReverseProxy(t *testing.T) {
 	log.SetOutput(ioutil.Discard)
 	defer log.SetOutput(os.Stderr)
 
-	verifyHeaders := func(headers http.Header, trailers http.Header) {
-		if headers.Get("X-Header") != "header-value" {
-			t.Error("Expected header 'X-Header' to be proxied properly")
+	testHeaderValue := []string{"header-value"}
+	testHeaders := http.Header{
+		"X-Header-1": testHeaderValue,
+		"X-Header-2": testHeaderValue,
+		"X-Header-3": testHeaderValue,
+	}
+	testTrailerValue := []string{"trailer-value"}
+	testTrailers := http.Header{
+		"X-Trailer-1": testTrailerValue,
+		"X-Trailer-2": testTrailerValue,
+		"X-Trailer-3": testTrailerValue,
+	}
+	verifyHeaderValues := func(actual http.Header, expected http.Header) bool {
+		if actual == nil {
+			t.Error("Expected headers")
+			return true
 		}
 
-		if trailers == nil {
-			t.Error("Expected to receive trailers")
+		for k := range expected {
+			if expected.Get(k) != actual.Get(k) {
+				t.Errorf("Expected header '%s' to be proxied properly", k)
+				return true
+			}
 		}
-		if trailers.Get("X-Trailer") != "trailer-value" {
-			t.Error("Expected header 'X-Trailer' to be proxied properly")
+
+		return false
+	}
+	verifyHeadersTrailers := func(headers http.Header, trailers http.Header) {
+		if verifyHeaderValues(headers, testHeaders) || verifyHeaderValues(trailers, testTrailers) {
+			t.FailNow()
 		}
 	}
 
-	var requestReceived bool
+	requestReceived := false
 	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		// read the body (even if it's empty) to make Go parse trailers
 		io.Copy(ioutil.Discard, r.Body)
-		verifyHeaders(r.Header, r.Trailer)
 
+		verifyHeadersTrailers(r.Header, r.Trailer)
 		requestReceived = true
 
-		w.Header().Set("Trailer", "X-Trailer")
-		w.Header().Set("X-Header", "header-value")
+		// Set headers.
+		copyHeader(w.Header(), testHeaders)
+
+		// Only announce one of the trailers to test wether
+		// unannounced trailers are proxied correctly.
+		for k := range testTrailers {
+			w.Header().Set("Trailer", k)
+			break
+		}
+
 		w.WriteHeader(http.StatusOK)
 		w.Write([]byte("Hello, client"))
-		w.Header().Set("X-Trailer", "trailer-value")
+
+		// Set trailers.
+		shallowCopyTrailers(w.Header(), testTrailers, true)
 	}))
 	defer backend.Close()
 
@@ -79,28 +111,43 @@ func TestReverseProxy(t *testing.T) {
 		Upstreams: []Upstream{newFakeUpstream(backend.URL, false)},
 	}
 
-	// create request and response recorder
-	r := httptest.NewRequest("GET", "/", strings.NewReader("test"))
-	w := httptest.NewRecorder()
-
-	r.ContentLength = -1 // force chunked encoding (required for trailers)
-	r.Header.Set("X-Header", "header-value")
-	r.Trailer = map[string][]string{
-		"X-Trailer": {"trailer-value"},
+	// Create the fake request body.
+	// This will copy "trailersToSet" to r.Trailer right before it is closed and
+	// thus test for us wether unannounced client trailers are proxied correctly.
+	body := &trailerTestStringReader{
+		Reader:        *strings.NewReader("test"),
+		trailersToSet: testTrailers,
 	}
 
+	// Create the fake request with the above body.
+	r := httptest.NewRequest("GET", "/", body)
+	r.Trailer = make(http.Header)
+	body.request = r
+
+	copyHeader(r.Header, testHeaders)
+
+	// Only announce one of the trailers to test wether
+	// unannounced trailers are proxied correctly.
+	for k, v := range testTrailers {
+		r.Trailer[k] = v
+		break
+	}
+
+	w := httptest.NewRecorder()
 	p.ServeHTTP(w, r)
+	res := w.Result()
 
 	if !requestReceived {
 		t.Error("Expected backend to receive request, but it didn't")
 	}
 
-	res := w.Result()
-	verifyHeaders(res.Header, res.Trailer)
+	verifyHeadersTrailers(res.Header, res.Trailer)
 
 	// Make sure {upstream} placeholder is set
 	r.Body = ioutil.NopCloser(strings.NewReader("test"))
-	rr := httpserver.NewResponseRecorder(testResponseRecorder{httptest.NewRecorder()})
+	rr := httpserver.NewResponseRecorder(testResponseRecorder{
+		ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: httptest.NewRecorder()},
+	})
 	rr.Replacer = httpserver.NewReplacer(r, rr, "-")
 
 	p.ServeHTTP(rr, r)
@@ -110,6 +157,21 @@ func TestReverseProxy(t *testing.T) {
 	}
 }
 
+// trailerTestStringReader is used to test unannounced trailers coming
+// from a client which should properly be proxied to the upstream.
+type trailerTestStringReader struct {
+	strings.Reader
+	request       *http.Request
+	trailersToSet http.Header
+}
+
+var _ io.ReadCloser = &trailerTestStringReader{}
+
+func (r *trailerTestStringReader) Close() error {
+	copyHeader(r.request.Trailer, r.trailersToSet)
+	return nil
+}
+
 func TestReverseProxyInsecureSkipVerify(t *testing.T) {
 	log.SetOutput(ioutil.Discard)
 	defer log.SetOutput(os.Stderr)
@@ -245,8 +307,8 @@ func TestWebSocketReverseProxyNonHijackerPanic(t *testing.T) {
 func TestWebSocketReverseProxyServeHTTPHandler(t *testing.T) {
 	// No-op websocket backend simply allows the WS connection to be
 	// accepted then it will be immediately closed. Perfect for testing.
-	var connCount int32
-	wsNop := httptest.NewServer(websocket.Handler(func(ws *websocket.Conn) { atomic.AddInt32(&connCount, 1) }))
+	accepted := make(chan struct{})
+	wsNop := httptest.NewServer(websocket.Handler(func(ws *websocket.Conn) { close(accepted) }))
 	defer wsNop.Close()
 
 	// Get proxy to use for the test
@@ -277,8 +339,14 @@ func TestWebSocketReverseProxyServeHTTPHandler(t *testing.T) {
 	if !bytes.Equal(actual, expected) {
 		t.Errorf("Expected backend to accept response:\n'%s'\nActually got:\n'%s'", expected, actual)
 	}
-	if got, want := atomic.LoadInt32(&connCount), int32(1); got != want {
-		t.Errorf("Expected %d websocket connection, got %d", want, got)
+
+	// wait a minute for backend handling, see issue 1654.
+	time.Sleep(10 * time.Millisecond)
+
+	select {
+	case <-accepted:
+	default:
+		t.Error("Expect a accepted websocket connection, but not")
 	}
 }
 
@@ -1315,24 +1383,13 @@ func (c *fakeConn) Write(b []byte) (int, error)        { return c.writeBuf.Write
 // testResponseRecorder wraps `httptest.ResponseRecorder`,
 // also implements `http.CloseNotifier`, `http.Hijacker` and `http.Pusher`.
 type testResponseRecorder struct {
-	*httptest.ResponseRecorder
+	*httpserver.ResponseWriterWrapper
 }
 
 func (testResponseRecorder) CloseNotify() <-chan bool { return nil }
-func (t testResponseRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	return nil, nil, httpserver.NonHijackerError{Underlying: t}
-}
-func (t testResponseRecorder) Push(target string, opts *http.PushOptions) error {
-	return httpserver.NonPusherError{Underlying: t}
-}
 
 // Interface guards
-var (
-	_ http.Pusher        = testResponseRecorder{}
-	_ http.Flusher       = testResponseRecorder{}
-	_ http.CloseNotifier = testResponseRecorder{}
-	_ http.Hijacker      = testResponseRecorder{}
-)
+var _ httpserver.HTTPInterfaces = testResponseRecorder{}
 
 func BenchmarkProxy(b *testing.B) {
 	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -1367,3 +1424,114 @@ func BenchmarkProxy(b *testing.B) {
 		p.ServeHTTP(w, r)
 	}
 }
+
+func TestChunkedWebSocketReverseProxy(t *testing.T) {
+	s := websocket.Server{
+		Handler: websocket.Handler(func(ws *websocket.Conn) {
+			for {
+				select {}
+			}
+		}),
+	}
+	s.Config.Header = http.Header(make(map[string][]string))
+	s.Config.Header.Set("Transfer-Encoding", "chunked")
+
+	wsNop := httptest.NewServer(s)
+	defer wsNop.Close()
+
+	// Get proxy to use for the test
+	p := newWebSocketTestProxy(wsNop.URL, false)
+
+	// Create client request
+	r := httptest.NewRequest("GET", "/", nil)
+
+	r.Header = http.Header{
+		"Connection":            {"Upgrade"},
+		"Upgrade":               {"websocket"},
+		"Origin":                {wsNop.URL},
+		"Sec-WebSocket-Key":     {"x3JJHMbDL1EzLkh9GBhXDw=="},
+		"Sec-WebSocket-Version": {"13"},
+	}
+
+	// Capture the request
+	w := &recorderHijacker{httptest.NewRecorder(), new(fakeConn)}
+
+	// Booya! Do the test.
+	_, err := p.ServeHTTP(w, r)
+
+	// Make sure the backend accepted the WS connection.
+	// Mostly interested in the Upgrade and Connection response headers
+	// and the 101 status code.
+	expected := []byte("HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=\r\nTransfer-Encoding: chunked\r\n\r\n")
+	actual := w.fakeConn.writeBuf.Bytes()
+	if !bytes.Equal(actual, expected) {
+		t.Errorf("Expected backend to accept response:\n'%s'\nActually got:\n'%s'", expected, actual)
+	}
+
+	if err != nil {
+		t.Error(err)
+	}
+}
+
+func TestQuic(t *testing.T) {
+	if strings.ToLower(os.Getenv("CI")) != "true" {
+		// TODO. (#1782) This test requires configuring hosts
+		// file and updating the certificate in testdata. We
+		// should find a more robust way of testing this.
+		return
+	}
+
+	upstream := "quic.clemente.io:8086"
+	config := "proxy / quic://" + upstream
+	content := "Hello, client"
+
+	// make proxy
+	upstreams, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(config)), "")
+	if err != nil {
+		t.Errorf("Expected no error. Got: %s", err.Error())
+	}
+	p := &Proxy{
+		Next:      httpserver.EmptyNext, // prevents panic in some cases when test fails
+		Upstreams: upstreams,
+	}
+
+	// start QUIC server
+	go func() {
+		dir, err := os.Getwd()
+		if err != nil {
+			t.Errorf("Expected no error. Got: %s", err.Error())
+			return
+		}
+		handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			w.Write([]byte(content))
+			w.WriteHeader(200)
+		})
+		err = h2quic.ListenAndServeQUIC(
+			upstream,
+			path.Join(dir, "testdata", "fullchain.pem"),
+			path.Join(dir, "testdata", "privkey.pem"),
+			handler,
+		)
+		if err != nil {
+			t.Errorf("Expected no error. Got: %s", err.Error())
+			return
+		}
+	}()
+
+	r := httptest.NewRequest("GET", "/", nil)
+	w := httptest.NewRecorder()
+	_, err = p.ServeHTTP(w, r)
+	if err != nil {
+		t.Errorf("Expected no error. Got: %s", err.Error())
+		return
+	}
+
+	// check response
+	if w.Code != 200 {
+		t.Errorf("Expected response code 200, got: %d", w.Code)
+	}
+	responseContent := string(w.Body.Bytes())
+	if responseContent != content {
+		t.Errorf("Expected response body, got: %s", responseContent)
+	}
+}

caddyhttp/proxy/reverseproxy.go

@@ -23,6 +23,8 @@ import (
 
 	"golang.org/x/net/http2"
 
+	"github.com/lucas-clemente/quic-go"
+	"github.com/lucas-clemente/quic-go/h2quic"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
@@ -33,6 +35,8 @@ var (
 	}
 
 	bufferPool = sync.Pool{New: createBuffer}
+
+	defaultCryptoHandshakeTimeout = 10 * time.Second
 )
 
 func createBuffer() interface{} {
@@ -180,11 +184,18 @@ func NewSingleHostReverseProxy(target *url.URL, without string, keepalive int) *
 			req.URL.RawQuery = targetQuery + "&" + req.URL.RawQuery
 		}
 	}
+
 	rp := &ReverseProxy{Director: director, FlushInterval: 250 * time.Millisecond} // flushing good for streaming & server-sent events
 	if target.Scheme == "unix" {
 		rp.Transport = &http.Transport{
 			Dial: socketDial(target.String()),
 		}
+	} else if target.Scheme == "quic" {
+		rp.Transport = &h2quic.RoundTripper{
+			QuicConfig: &quic.Config{
+				HandshakeTimeout: defaultCryptoHandshakeTimeout,
+			},
+		}
 	} else if keepalive != http.DefaultMaxIdleConnsPerHost {
 		// if keepalive is equal to the default,
 		// just use default transport, to avoid creating
@@ -192,7 +203,7 @@ func NewSingleHostReverseProxy(target *url.URL, without string, keepalive int) *
 		transport := &http.Transport{
 			Proxy:                 http.ProxyFromEnvironment,
 			Dial:                  defaultDialer.Dial,
-			TLSHandshakeTimeout:   10 * time.Second,
+			TLSHandshakeTimeout:   defaultCryptoHandshakeTimeout,
 			ExpectContinueTimeout: 1 * time.Second,
 		}
 		if keepalive == 0 {
@@ -216,7 +227,7 @@ func (rp *ReverseProxy) UseInsecureTransport() {
 		transport := &http.Transport{
 			Proxy:               http.ProxyFromEnvironment,
 			Dial:                defaultDialer.Dial,
-			TLSHandshakeTimeout: 10 * time.Second,
+			TLSHandshakeTimeout: defaultCryptoHandshakeTimeout,
 			TLSClientConfig:     &tls.Config{InsecureSkipVerify: true},
 		}
 		if httpserver.HTTP2 {
@@ -231,6 +242,11 @@ func (rp *ReverseProxy) UseInsecureTransport() {
 		// No http2.ConfigureTransport() here.
 		// For now this is only added in places where
 		// an http.Transport is actually created.
+	} else if transport, ok := rp.Transport.(*h2quic.RoundTripper); ok {
+		if transport.TLSClientConfig == nil {
+			transport.TLSClientConfig = &tls.Config{}
+		}
+		transport.TLSClientConfig.InsecureSkipVerify = true
 	}
 }
 
@@ -246,6 +262,10 @@ func (rp *ReverseProxy) ServeHTTP(rw http.ResponseWriter, outreq *http.Request,
 
 	rp.Director(outreq)
 
+	if outreq.URL.Scheme == "quic" {
+		outreq.URL.Scheme = "https" // Change scheme back to https for QUIC RoundTripper
+	}
+
 	res, err := transport.RoundTrip(outreq)
 	if err != nil {
 		return err
@@ -272,7 +292,7 @@ func (rp *ReverseProxy) ServeHTTP(rw http.ResponseWriter, outreq *http.Request,
 	}
 
 	if isWebsocket {
-		res.Body.Close()
+		defer res.Body.Close()
 		hj, ok := rw.(http.Hijacker)
 		if !ok {
 			panic(httpserver.NonHijackerError{Underlying: rw})
@@ -318,30 +338,61 @@ func (rp *ReverseProxy) ServeHTTP(rw http.ResponseWriter, outreq *http.Request,
 		}
 		pooledIoCopy(backendConn, conn)
 	} else {
+		// NOTE:
+		//   Closing the Body involves acquiring a mutex, which is a
+		//   unnecessarily heavy operation, considering that this defer will
+		//   pretty much never be executed with the Body still unclosed.
+		bodyOpen := true
+		closeBody := func() {
+			if bodyOpen {
+				res.Body.Close()
+				bodyOpen = false
+			}
+		}
+		defer closeBody()
+
+		// Copy all headers over.
+		// res.Header does not include the "Trailer" header,
+		// which means we will have to do that manually below.
 		copyHeader(rw.Header(), res.Header)
 
-		// The "Trailer" header isn't included in the Transport's response,
-		// at least for *http.Transport. Build it up from Trailer.
-		if len(res.Trailer) > 0 {
-			trailerKeys := make([]string, 0, len(res.Trailer))
+		// The "Trailer" header isn't included in res' Header map, which
+		// is why we have to build one ourselves from res.Trailer.
+		//
+		// But res.Trailer does not necessarily contain all trailer keys at this
+		// point yet. The HTTP spec allows one to send "unannounced trailers"
+		// after a request and certain systems like gRPC make use of that.
+		announcedTrailerKeyCount := len(res.Trailer)
+		if announcedTrailerKeyCount > 0 {
+			vv := make([]string, 0, announcedTrailerKeyCount)
 			for k := range res.Trailer {
-				trailerKeys = append(trailerKeys, k)
+				vv = append(vv, k)
 			}
-			rw.Header().Add("Trailer", strings.Join(trailerKeys, ", "))
+			rw.Header()["Trailer"] = vv
 		}
 
+		// Now copy over the status code as well as the response body.
 		rw.WriteHeader(res.StatusCode)
-		if len(res.Trailer) > 0 {
+		if announcedTrailerKeyCount > 0 {
 			// Force chunking if we saw a response trailer.
-			// This prevents net/http from calculating the length for short
-			// bodies and adding a Content-Length.
+			// This prevents net/http from calculating the length
+			// for short bodies and adding a Content-Length.
 			if fl, ok := rw.(http.Flusher); ok {
 				fl.Flush()
 			}
 		}
 		rp.copyResponse(rw, res.Body)
-		res.Body.Close() // close now, instead of defer, to populate res.Trailer
-		copyHeader(rw.Header(), res.Trailer)
+
+		// Now close the body to fully populate res.Trailer.
+		closeBody()
+
+		// Since Go does not remove keys from res.Trailer we
+		// can safely do a length comparison to check wether
+		// we received further, unannounced trailers.
+		//
+		// Most of the time forceSetTrailers should be false.
+		forceSetTrailers := len(res.Trailer) != announcedTrailerKeyCount
+		shallowCopyTrailers(rw.Header(), res.Trailer, forceSetTrailers)
 	}
 
 	return nil
@@ -382,8 +433,12 @@ func copyHeader(dst, src http.Header) {
 			if _, shouldSkip := skipHeaders[k]; shouldSkip {
 				continue
 			}
-			// otherwise, overwrite
-			dst.Del(k)
+			// otherwise, overwrite to avoid duplicated fields that can be
+			// problematic (see issue #1086) -- however, allow duplicate
+			// Server fields so we can see the reality of the proxying.
+			if k != "Server" {
+				dst.Del(k)
+			}
 		}
 		for _, v := range vv {
 			dst.Add(k, v)
@@ -391,6 +446,22 @@ func copyHeader(dst, src http.Header) {
 	}
 }
 
+// shallowCopyTrailers copies all headers from srcTrailer to dstHeader.
+//
+// If forceSetTrailers is set to true, the http.TrailerPrefix will be added to
+// all srcTrailer key names. Otherwise the Go stdlib will ignore all keys
+// which weren't listed in the Trailer map before submitting the Response.
+//
+// WARNING: Only a shallow copy will be created!
+func shallowCopyTrailers(dstHeader, srcTrailer http.Header, forceSetTrailers bool) {
+	for k, vv := range srcTrailer {
+		if forceSetTrailers {
+			k = http.TrailerPrefix + k
+		}
+		dstHeader[k] = vv
+	}
+}
+
 // Hop-by-hop headers. These are removed when sent to the backend.
 // http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
 var hopHeaders = []string{

caddyhttp/proxy/setup_test.go

@@ -147,6 +147,14 @@ func TestSetup(t *testing.T) {
 				"http://localhost:1984": {},
 			},
 		},
+		// test #14 test QUIC
+		{
+			"proxy / quic://localhost:443",
+			false,
+			map[string]struct{}{
+				"quic://localhost:443": {},
+			},
+		},
 	} {
 		c := caddy.NewTestController("http", test.input)
 		err := setup(c)

caddyhttp/proxy/testdata/fullchain.pem

@@ -0,0 +1,56 @@
+-----BEGIN CERTIFICATE-----
+MIIFAzCCA+ugAwIBAgISA7e2G9wJth5EaqaD5X0RiagYMA0GCSqGSIb3DQEBCwUA
+MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
+ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xNzA3MDMxODU3MDBaFw0x
+NzEwMDExODU3MDBaMBsxGTAXBgNVBAMTEHF1aWMuY2xlbWVudGUuaW8wggEiMA0G
+CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC7UjonSCiB0tyHsenbXZw/QF028EmH
+tvdyMTOlz2DVLqi0K6brqVIh3KSl2gPlsizLHmkoTLVINuGCnDXc4jXu6yCVHrPr
+KOf+ip8SUcqQEmLXmHw5Y+L4/6ZKUE5mFpfqmlMCEb1t86J7FI9z+QA9LGdbziYv
+qQBW8GytX16OJ4h/S1fiPCQ5GfWtkoVgYzgz8Vn4o51lLG2YXAl451BR8+XhGlYS
+OjS6x7RA0F5wqCeGgro7wKbFuyfxrkWkVzn5hNdEkBAABiub6obNBMZ6v+u84bQk
+1rH0oZB5rn3uEPycLmrQF2cYR5b+2F+BymKC0ElkFi3iWQoO98SZZQinAgMBAAGj
+ggIQMIICDDAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsG
+AQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFNjHumfJ0g905MebRnAvNfQh
+3AvEMB8GA1UdIwQYMBaAFKhKamMEfd265tE5t6ZFZe/zqOyhMG8GCCsGAQUFBwEB
+BGMwYTAuBggrBgEFBQcwAYYiaHR0cDovL29jc3AuaW50LXgzLmxldHNlbmNyeXB0
+Lm9yZzAvBggrBgEFBQcwAoYjaHR0cDovL2NlcnQuaW50LXgzLmxldHNlbmNyeXB0
+Lm9yZy8wGwYDVR0RBBQwEoIQcXVpYy5jbGVtZW50ZS5pbzCB/gYDVR0gBIH2MIHz
+MAgGBmeBDAECATCB5gYLKwYBBAGC3xMBAQEwgdYwJgYIKwYBBQUHAgEWGmh0dHA6
+Ly9jcHMubGV0c2VuY3J5cHQub3JnMIGrBggrBgEFBQcCAjCBngyBm1RoaXMgQ2Vy
+dGlmaWNhdGUgbWF5IG9ubHkgYmUgcmVsaWVkIHVwb24gYnkgUmVseWluZyBQYXJ0
+aWVzIGFuZCBvbmx5IGluIGFjY29yZGFuY2Ugd2l0aCB0aGUgQ2VydGlmaWNhdGUg
+UG9saWN5IGZvdW5kIGF0IGh0dHBzOi8vbGV0c2VuY3J5cHQub3JnL3JlcG9zaXRv
+cnkvMA0GCSqGSIb3DQEBCwUAA4IBAQAqs3Mrr/Erqp1rOFkLwKbStWZniCvqhl58
+VnScP2CjiBsaLJUuBlWqC215FtX5CrdkIwYrMMkkOZHZI4mPxN64UVqMY5UJRonL
+GvkeHC5QYsCV09bBHjCei6JDItNH2PCec9+mV9EIQiVzd8xliE3t0eTbjNsa9zf1
+Qwp64THbiyTIXuh4xgFTxU2u58+RkIRbKGRM1X4jgIv8xjNV4P1c0jUVqaEFkCjR
+A03becsSv3wqWvPCNQRdVRdoMMghHenDEAGD621McnaXDoNz8pgn/ss1vzrO36gX
+WZ7CmbgIFdYeMgqQop/252bN2wrNjnxAjLAHo/X1MPEabjoL1C0g
+-----END CERTIFICATE-----
+-----BEGIN CERTIFICATE-----
+MIIEkjCCA3qgAwIBAgIQCgFBQgAAAVOFc2oLheynCDANBgkqhkiG9w0BAQsFADA/
+MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
+DkRTVCBSb290IENBIFgzMB4XDTE2MDMxNzE2NDA0NloXDTIxMDMxNzE2NDA0Nlow
+SjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxIzAhBgNVBAMT
+GkxldCdzIEVuY3J5cHQgQXV0aG9yaXR5IFgzMIIBIjANBgkqhkiG9w0BAQEFAAOC
+AQ8AMIIBCgKCAQEAnNMM8FrlLke3cl03g7NoYzDq1zUmGSXhvb418XCSL7e4S0EF
+q6meNQhY7LEqxGiHC6PjdeTm86dicbp5gWAf15Gan/PQeGdxyGkOlZHP/uaZ6WA8
+SMx+yk13EiSdRxta67nsHjcAHJyse6cF6s5K671B5TaYucv9bTyWaN8jKkKQDIZ0
+Z8h/pZq4UmEUEz9l6YKHy9v6Dlb2honzhT+Xhq+w3Brvaw2VFn3EK6BlspkENnWA
+a6xK8xuQSXgvopZPKiAlKQTGdMDQMc2PMTiVFrqoM7hD8bEfwzB/onkxEz0tNvjj
+/PIzark5McWvxI0NHWQWM6r6hCm21AvA2H3DkwIDAQABo4IBfTCCAXkwEgYDVR0T
+AQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwfwYIKwYBBQUHAQEEczBxMDIG
+CCsGAQUFBzABhiZodHRwOi8vaXNyZy50cnVzdGlkLm9jc3AuaWRlbnRydXN0LmNv
+bTA7BggrBgEFBQcwAoYvaHR0cDovL2FwcHMuaWRlbnRydXN0LmNvbS9yb290cy9k
+c3Ryb290Y2F4My5wN2MwHwYDVR0jBBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAw
+VAYDVR0gBE0wSzAIBgZngQwBAgEwPwYLKwYBBAGC3xMBAQEwMDAuBggrBgEFBQcC
+ARYiaHR0cDovL2Nwcy5yb290LXgxLmxldHNlbmNyeXB0Lm9yZzA8BgNVHR8ENTAz
+MDGgL6AthitodHRwOi8vY3JsLmlkZW50cnVzdC5jb20vRFNUUk9PVENBWDNDUkwu
+Y3JsMB0GA1UdDgQWBBSoSmpjBH3duubRObemRWXv86jsoTANBgkqhkiG9w0BAQsF
+AAOCAQEA3TPXEfNjWDjdGBX7CVW+dla5cEilaUcne8IkCJLxWh9KEik3JHRRHGJo
+uM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/
+wApIvJSwtmVi4MFU5aMqrSDE6ea73Mj2tcMyo5jMd6jmeWUHK8so/joWUoHOUgwu
+X4Po1QYz+3dszkDqMp4fklxBwXRsW10KXzPMTZ+sOPAveyxindmjkW8lGy+QsRlG
+PfZ+G6Z6h7mjem0Y+iWlkYcV4PIWL1iwBi8saCbGS5jN2p8M+X+Q7UNKEkROb3N6
+KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg==
+-----END CERTIFICATE-----

caddyhttp/proxy/testdata/privkey.pem

@@ -0,0 +1,28 @@
+-----BEGIN PRIVATE KEY-----
+MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7UjonSCiB0tyH
+senbXZw/QF028EmHtvdyMTOlz2DVLqi0K6brqVIh3KSl2gPlsizLHmkoTLVINuGC
+nDXc4jXu6yCVHrPrKOf+ip8SUcqQEmLXmHw5Y+L4/6ZKUE5mFpfqmlMCEb1t86J7
+FI9z+QA9LGdbziYvqQBW8GytX16OJ4h/S1fiPCQ5GfWtkoVgYzgz8Vn4o51lLG2Y
+XAl451BR8+XhGlYSOjS6x7RA0F5wqCeGgro7wKbFuyfxrkWkVzn5hNdEkBAABiub
+6obNBMZ6v+u84bQk1rH0oZB5rn3uEPycLmrQF2cYR5b+2F+BymKC0ElkFi3iWQoO
+98SZZQinAgMBAAECggEAbO8EopNz+wuE8+Si+s8VbjMgAjL6j9H3VJEIWASha1gX
+A6/fAm0VNlv54/lFCu7y3axxut3hDn3b5viw2iMy+h4CdLXGK5s+TuiOWTj3c5E9
+qeMjWryb4fHJ4q2Q6g15ixTz8OAgKTDl7G2ofujvGqQX92uLCWxepjBrAufTNRcJ
+OZ3ngqHlKsRXX3nXkAMYrypK7ALF2kuavAGNrDQvPWUZKp3vuvd3Hx/stw0s3Th1
+XrHZnaAMZlxZg32IiVxs3vR2sACJ0YyOBpERBjjBsIaeyNXfZVrEmNzvo6iVhdhN
+ZNxrKSnPEfTdFk5pldFbTzNpvCvjbFAlE0aHXNRJAQKBgQDpAzWGkOTE+wmcWJNk
+oRi4ZJHhK/kckvNg1OZMXAqqZJOPxvwatEFgQ1GZo8rhSzdf64kB9b9I3OjEhd8r
+M90pt57BqRSq5rbytZBdR2BcbNnKkYF204AS2pkEVvkOVnWz5zSVhd8a0gMx3EdE
+LKN0r+DLKune8cnAS0BDvBjf3QKBgQDNzRJe9pI29mxUyuLQuKngaa8KmPy1EpbW
++d21ET4MjZbH6uPOAe1Q+7aCEA7rjvFoOqGk0w1WIN0i8EaIOuwM2W0jw4VS7AVI
+rWXTYy9uSnUuLWL6gHNbehqLs6JaADEvytWdXdqiR/XWxCDn7qg2CrjxwmDB/OUm
+RopmnlkEUwKBgQCreZ4ZUmXYhDmVYiXN5zPO9svYHkkr+wS6HNMCHLYIoQ1qwG/k
+owR9d+0EGOKDm5u7rhTcaWIEl/WAMliCbZ9zRNrC/8/i2PiHcpAz5QQH4F8CUMQq
+kwjsVwxGgk60e3IRG7O52ZPPJAAP4GBdzk/X3lqaiREk7WCgb4BymGjhzQKBgEMF
+mQkCJeXuZKNMm4c7zF8AK/g4kHvrvOHv56sTHXD7H3Kl5WBusjmgb/R1hFZka+v0
+xDWoYfx9oWbCd0XgYoVgvbFa+G1j3eioR7QK5iR17SmHsGdCM89DuadrbeD/lQUq
+elzQduZIpyA1KT4/M9q9rTNWiSpD0OChMmtvADBvAoGAAXF3cARv5w0fSZGSRCOw
+U3LdFNIhBgVdROj2C4ym+uJFErKTkB5kghdUER7UsFH8fVn3JLAb35cQRYGrysYz
+XF5eK0akNhkO9GLNrK0GbSHKZm9vQxixm5W05aVoUofRHqkkKL1ceC2rhwzp3Q5P
+1jLabOA4K0DkhNga0YPKJLQ=
+-----END PRIVATE KEY-----

caddyhttp/proxy/upstream.go

@@ -1,9 +1,11 @@
 package proxy
 
 import (
+	"bytes"
 	"fmt"
 	"io"
 	"io/ioutil"
+	"net"
 	"net/http"
 	"net/url"
 	"path"
@@ -20,7 +22,7 @@ import (
 )
 
 var (
-	supportedPolicies = make(map[string]func() Policy)
+	supportedPolicies = make(map[string]func(string) Policy)
 )
 
 type staticUpstream struct {
@@ -37,11 +39,13 @@ type staticUpstream struct {
 	TryInterval       time.Duration
 	MaxConns          int64
 	HealthCheck       struct {
-		Client   http.Client
-		Path     string
-		Interval time.Duration
-		Timeout  time.Duration
-		Host     string
+		Client        http.Client
+		Path          string
+		Interval      time.Duration
+		Timeout       time.Duration
+		Host          string
+		Port          string
+		ContentString string
 	}
 	WithoutPathPrefix  string
 	IgnoredSubPaths    []string
@@ -146,7 +150,8 @@ func (u *staticUpstream) From() string {
 
 func (u *staticUpstream) NewHost(host string) (*UpstreamHost, error) {
 	if !strings.HasPrefix(host, "http") &&
-		!strings.HasPrefix(host, "unix:") {
+		!strings.HasPrefix(host, "unix:") &&
+		!strings.HasPrefix(host, "quic:") {
 		host = "http://" + host
 	}
 	uh := &UpstreamHost{
@@ -239,7 +244,11 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 		if !ok {
 			return c.ArgErr()
 		}
-		u.Policy = policyCreateFunc()
+		arg := ""
+		if c.NextArg() {
+			arg = c.Val()
+		}
+		u.Policy = policyCreateFunc(arg)
 	case "fail_timeout":
 		if !c.NextArg() {
 			return c.ArgErr()
@@ -321,6 +330,25 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 			return err
 		}
 		u.HealthCheck.Timeout = dur
+	case "health_check_port":
+		if !c.NextArg() {
+			return c.ArgErr()
+		}
+		port := c.Val()
+		n, err := strconv.Atoi(port)
+		if err != nil {
+			return err
+		}
+
+		if n < 0 {
+			return c.Errf("invalid health_check_port '%s'", port)
+		}
+		u.HealthCheck.Port = port
+	case "health_check_contains":
+		if !c.NextArg() {
+			return c.ArgErr()
+		}
+		u.HealthCheck.ContentString = c.Val()
 	case "header_upstream":
 		var header, value string
 		if !c.Args(&header, &value) {
@@ -380,28 +408,48 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 
 func (u *staticUpstream) healthCheck() {
 	for _, host := range u.Hosts {
-		hostURL := host.Name + u.HealthCheck.Path
-		var unhealthy bool
+		hostURL := host.Name
+		if u.HealthCheck.Port != "" {
+			hostURL = replacePort(host.Name, u.HealthCheck.Port)
+		}
+		hostURL += u.HealthCheck.Path
 
-		// set up request, needed to be able to modify headers
-		// possible errors are bad HTTP methods or un-parsable urls
-		req, err := http.NewRequest("GET", hostURL, nil)
-		if err != nil {
-			unhealthy = true
-		} else {
+		unhealthy := func() bool {
+			// set up request, needed to be able to modify headers
+			// possible errors are bad HTTP methods or un-parsable urls
+			req, err := http.NewRequest("GET", hostURL, nil)
+			if err != nil {
+				return true
+			}
 			// set host for request going upstream
 			if u.HealthCheck.Host != "" {
 				req.Host = u.HealthCheck.Host
 			}
-
-			if r, err := u.HealthCheck.Client.Do(req); err == nil {
+			r, err := u.HealthCheck.Client.Do(req)
+			if err != nil {
+				return true
+			}
+			defer func() {
 				io.Copy(ioutil.Discard, r.Body)
 				r.Body.Close()
-				unhealthy = r.StatusCode < 200 || r.StatusCode >= 400
-			} else {
-				unhealthy = true
+			}()
+			if r.StatusCode < 200 || r.StatusCode >= 400 {
+				return true
 			}
-		}
+			if u.HealthCheck.ContentString == "" { // don't check for content string
+				return false
+			}
+			// TODO ReadAll will be replaced if deemed necessary
+			//      See https://github.com/mholt/caddy/pull/1691
+			buf, err := ioutil.ReadAll(r.Body)
+			if err != nil {
+				return true
+			}
+			if bytes.Contains(buf, []byte(u.HealthCheck.ContentString)) {
+				return false
+			}
+			return true
+		}()
 		if unhealthy {
 			atomic.StoreInt32(&host.Unhealthy, 1)
 		} else {
@@ -480,6 +528,22 @@ func (u *staticUpstream) Stop() error {
 }
 
 // RegisterPolicy adds a custom policy to the proxy.
-func RegisterPolicy(name string, policy func() Policy) {
+func RegisterPolicy(name string, policy func(string) Policy) {
 	supportedPolicies[name] = policy
 }
+
+func replacePort(originalURL string, newPort string) string {
+	parsedURL, err := url.Parse(originalURL)
+	if err != nil {
+		return originalURL
+	}
+
+	// handles 'localhost' and 'localhost:8080'
+	parsedHost, _, err := net.SplitHostPort(parsedURL.Host)
+	if err != nil {
+		parsedHost = parsedURL.Host
+	}
+
+	parsedURL.Host = net.JoinHostPort(parsedHost, newPort)
+	return parsedURL.String()
+}

caddyhttp/proxy/upstream_test.go

@@ -2,6 +2,7 @@ package proxy
 
 import (
 	"fmt"
+	"net"
 	"net/http"
 	"net/http/httptest"
 	"strings"
@@ -9,6 +10,7 @@ import (
 	"testing"
 	"time"
 
+	"github.com/lucas-clemente/quic-go/h2quic"
 	"github.com/mholt/caddy/caddyfile"
 )
 
@@ -105,7 +107,7 @@ func TestSelect(t *testing.T) {
 func TestRegisterPolicy(t *testing.T) {
 	name := "custom"
 	customPolicy := &customPolicy{}
-	RegisterPolicy(name, func() Policy { return customPolicy })
+	RegisterPolicy(name, func(string) Policy { return customPolicy })
 	if _, ok := supportedPolicies[name]; !ok {
 		t.Error("Expected supportedPolicies to have a custom policy.")
 	}
@@ -375,3 +377,163 @@ func TestHealthCheckHost(t *testing.T) {
 		}
 	}
 }
+
+func TestHealthCheckPort(t *testing.T) {
+	var counter int64
+
+	healthCounter := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		r.Body.Close()
+		atomic.AddInt64(&counter, 1)
+	}))
+
+	_, healthPort, err := net.SplitHostPort(healthCounter.Listener.Addr().String())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	defer healthCounter.Close()
+
+	tests := []struct {
+		config string
+	}{
+		// Test #1: upstream with port
+		{"proxy / localhost:8080 {\n health_check / health_check_port " + healthPort + "\n}"},
+
+		// Test #2: upstream without port (default to 80)
+		{"proxy / localhost {\n health_check / health_check_port " + healthPort + "\n}"},
+	}
+
+	for i, test := range tests {
+		counterValueAtStart := atomic.LoadInt64(&counter)
+		upstreams, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+		if err != nil {
+			t.Error("Expected no error. Got:", err.Error())
+		}
+
+		// Give some time for healthchecks to hit the server.
+		time.Sleep(500 * time.Millisecond)
+
+		for _, upstream := range upstreams {
+			if err := upstream.Stop(); err != nil {
+				t.Errorf("Test %d: Expected no error stopping upstream. Got: %v", i, err.Error())
+			}
+		}
+
+		counterValueAfterShutdown := atomic.LoadInt64(&counter)
+
+		if counterValueAfterShutdown == counterValueAtStart {
+			t.Errorf("Test %d: Expected healthchecks to hit test server. Got no healthchecks.", i)
+		}
+	}
+
+	t.Run("valid_port", func(t *testing.T) {
+		tests := []struct {
+			config string
+		}{
+			// Test #1: invalid port (nil)
+			{"proxy / localhost {\n health_check / health_check_port\n}"},
+
+			// Test #2: invalid port (string)
+			{"proxy / localhost {\n health_check / health_check_port abc\n}"},
+
+			// Test #3: invalid port (negative)
+			{"proxy / localhost {\n health_check / health_check_port -1\n}"},
+		}
+
+		for i, test := range tests {
+			_, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+			if err == nil {
+				t.Errorf("Test %d accepted invalid config", i)
+			}
+		}
+	})
+
+}
+
+func TestHealthCheckContentString(t *testing.T) {
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprintf(w, "blablabla good blablabla")
+		r.Body.Close()
+	}))
+	_, port, err := net.SplitHostPort(server.Listener.Addr().String())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer server.Close()
+
+	tests := []struct {
+		config        string
+		shouldContain bool
+	}{
+		{"proxy / localhost:" + port +
+			" { health_check /testhealth " +
+			" health_check_contains good\n}",
+			true,
+		},
+		{"proxy / localhost:" + port + " {\n health_check /testhealth health_check_port " + port +
+			" \n health_check_contains bad\n}",
+			false,
+		},
+	}
+	for i, test := range tests {
+		u, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+		if err != nil {
+			t.Errorf("Expected no error. Test %d Got: %s", i, err.Error())
+		}
+		for _, upstream := range u {
+			staticUpstream, ok := upstream.(*staticUpstream)
+			if !ok {
+				t.Errorf("Type mismatch: %#v", upstream)
+				continue
+			}
+			staticUpstream.healthCheck()
+			for _, host := range staticUpstream.Hosts {
+				if test.shouldContain && atomic.LoadInt32(&host.Unhealthy) == 0 {
+					// healthcheck url was hit and the required test string was found
+					continue
+				}
+				if !test.shouldContain && atomic.LoadInt32(&host.Unhealthy) != 0 {
+					// healthcheck url was hit and the required string was not found
+					continue
+				}
+				t.Errorf("Health check bad response")
+			}
+			upstream.Stop()
+		}
+	}
+}
+
+func TestQuicHost(t *testing.T) {
+	// tests for QUIC proxy
+	tests := []struct {
+		config string
+		flag   bool
+	}{
+		// Test #1: without flag
+		{"proxy / quic://localhost:8080", false},
+
+		// Test #2: with flag
+		{"proxy / quic://localhost:8080 {\n insecure_skip_verify \n}", true},
+	}
+
+	for _, test := range tests {
+		upstreams, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+		if err != nil {
+			t.Errorf("Expected no error. Got: %s", err.Error())
+		}
+		for _, upstream := range upstreams {
+			staticUpstream, ok := upstream.(*staticUpstream)
+			if !ok {
+				t.Errorf("Type mismatch: %#v", upstream)
+				continue
+			}
+			for _, host := range staticUpstream.Hosts {
+				_, ok := host.ReverseProxy.Transport.(*h2quic.RoundTripper)
+				if !ok {
+					t.Errorf("Type mismatch: %#v", host.ReverseProxy.Transport)
+					continue
+				}
+			}
+		}
+	}
+}

caddyhttp/push/handler.go

@@ -5,6 +5,7 @@ import (
 	"strings"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
+	"github.com/mholt/caddy/caddyhttp/staticfiles"
 )
 
 func (h Middleware) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
@@ -25,7 +26,16 @@ func (h Middleware) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, erro
 	// push first
 outer:
 	for _, rule := range h.Rules {
-		if httpserver.Path(r.URL.Path).Matches(rule.Path) {
+		urlPath := r.URL.Path
+		matches := httpserver.Path(urlPath).Matches(rule.Path)
+		// Also check IndexPages when requesting a directory
+		if !matches {
+			indexFile, isIndexFile := httpserver.IndexFile(h.Root, urlPath, staticfiles.IndexPages)
+			if isIndexFile {
+				matches = httpserver.Path(indexFile).Matches(rule.Path)
+			}
+		}
+		if matches {
 			for _, resource := range rule.Resources {
 				pushErr := pusher.Push(resource.Path, &http.PushOptions{
 					Method: resource.Method,
@@ -50,27 +60,40 @@ outer:
 	return code, err
 }
 
-func (h Middleware) servePreloadLinks(pusher http.Pusher, headers http.Header, links []string) {
-	for _, link := range links {
-		parts := strings.Split(link, ";")
+// servePreloadLinks parses Link headers from backend and pushes resources found in them.
+// For accepted header formats check parseLinkHeader function.
+//
+// If resource has 'nopush' attribute then it will be omitted.
+func (h Middleware) servePreloadLinks(pusher http.Pusher, headers http.Header, resources []string) {
+outer:
+	for _, resource := range resources {
+		for _, resource := range parseLinkHeader(resource) {
+			if _, exists := resource.params["nopush"]; exists {
+				continue
+			}
 
-		if link == "" || strings.HasSuffix(link, "nopush") {
-			continue
-		}
+			if h.isRemoteResource(resource.uri) {
+				continue
+			}
 
-		target := strings.TrimSuffix(strings.TrimPrefix(parts[0], "<"), ">")
+			err := pusher.Push(resource.uri, &http.PushOptions{
+				Method: http.MethodGet,
+				Header: headers,
+			})
 
-		err := pusher.Push(target, &http.PushOptions{
-			Method: http.MethodGet,
-			Header: headers,
-		})
-
-		if err != nil {
-			break
+			if err != nil {
+				break outer
+			}
 		}
 	}
 }
 
+func (h Middleware) isRemoteResource(resource string) bool {
+	return strings.HasPrefix(resource, "//") ||
+		strings.HasPrefix(resource, "http://") ||
+		strings.HasPrefix(resource, "https://")
+}
+
 func (h Middleware) mergeHeaders(l, r http.Header) http.Header {
 	out := http.Header{}
 

caddyhttp/push/handler_test.go

@@ -2,8 +2,11 @@ package push
 
 import (
 	"errors"
+	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
+	"os"
+	"path/filepath"
 	"reflect"
 	"testing"
 
@@ -266,6 +269,52 @@ func TestMiddlewareShouldInterceptLinkHeader(t *testing.T) {
 	comparePushedResources(t, expectedPushedResources, pushingWriter.pushed)
 }
 
+func TestMiddlewareShouldInterceptLinkHeaderWithMultipleResources(t *testing.T) {
+	// given
+	request, err := http.NewRequest(http.MethodGet, "/index.html", nil)
+	writer := httptest.NewRecorder()
+
+	if err != nil {
+		t.Fatalf("Could not create HTTP request: %v", err)
+	}
+
+	middleware := Middleware{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			w.Header().Add("Link", "</assets/css/screen.css?v=5fc240c512>; rel=preload; as=style,</content/images/2016/06/Timeouts-001.png>; rel=preload; as=image,</content/images/2016/06/Timeouts-002.png>; rel=preload; as=image")
+			w.Header().Add("Link", "<//cdn.bizible.com/scripts/bizible.js>; rel=preload; as=script,</resource.png>; rel=preload; as=script; nopush")
+			return 0, nil
+		}),
+		Rules: []Rule{},
+	}
+
+	pushingWriter := &MockedPusher{ResponseWriter: writer}
+
+	// when
+	_, err2 := middleware.ServeHTTP(pushingWriter, request)
+
+	// then
+	if err2 != nil {
+		t.Error("Should not return error")
+	}
+
+	expectedPushedResources := map[string]*http.PushOptions{
+		"/assets/css/screen.css?v=5fc240c512": {
+			Method: http.MethodGet,
+			Header: http.Header{},
+		},
+		"/content/images/2016/06/Timeouts-001.png": {
+			Method: http.MethodGet,
+			Header: http.Header{},
+		},
+		"/content/images/2016/06/Timeouts-002.png": {
+			Method: http.MethodGet,
+			Header: http.Header{},
+		},
+	}
+
+	comparePushedResources(t, expectedPushedResources, pushingWriter.pushed)
+}
+
 func TestMiddlewareShouldInterceptLinkHeaderPusherError(t *testing.T) {
 	// given
 	expectedHeaders := http.Header{"Accept-Encoding": []string{"br"}}
@@ -307,6 +356,112 @@ func TestMiddlewareShouldInterceptLinkHeaderPusherError(t *testing.T) {
 	comparePushedResources(t, expectedPushedResources, pushingWriter.pushed)
 }
 
+func TestMiddlewareShouldPushIndexFile(t *testing.T) {
+	// given
+	indexFile := "/index.html"
+	request, err := http.NewRequest(http.MethodGet, "/", nil) // Request root directory, not indexfile itself
+	if err != nil {
+		t.Fatalf("Could not create HTTP request: %v", err)
+	}
+
+	root, err := ioutil.TempDir("", "caddy")
+	if err != nil {
+		t.Fatalf("Could not create temporary directory: %v", err)
+	}
+	defer os.Remove(root)
+
+	middleware := Middleware{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			return 0, nil
+		}),
+		Rules: []Rule{
+			{Path: indexFile, Resources: []Resource{
+				{Path: "/index.css", Method: http.MethodGet},
+			}},
+		},
+		Root: http.Dir(root),
+	}
+
+	indexFilePath := filepath.Join(root, indexFile)
+	_, err = os.Create(indexFilePath)
+	if err != nil {
+		t.Fatalf("Could not create index file: %s: %v", indexFile, err)
+	}
+	defer os.Remove(indexFilePath)
+
+	pushingWriter := &MockedPusher{
+		ResponseWriter: httptest.NewRecorder(),
+		returnedError:  errors.New("Cannot push right now"),
+	}
+
+	// when
+	_, err2 := middleware.ServeHTTP(pushingWriter, request)
+
+	// then
+	if err2 != nil {
+		t.Error("Should not return error")
+	}
+
+	expectedPushedResources := map[string]*http.PushOptions{
+		"/index.css": {
+			Method: http.MethodGet,
+			Header: http.Header{},
+		},
+	}
+
+	comparePushedResources(t, expectedPushedResources, pushingWriter.pushed)
+}
+
+func TestMiddlewareShouldNotPushIndexFileWhenNotARule(t *testing.T) {
+	// given
+	indexFile := "/index.html"
+	request, err := http.NewRequest(http.MethodGet, "/", nil) // Request root directory, not indexfile itself
+	if err != nil {
+		t.Fatalf("Could not create HTTP request: %v", err)
+	}
+
+	root, err := ioutil.TempDir("", "caddy")
+	if err != nil {
+		t.Fatalf("Could not create temporary directory: %v", err)
+	}
+	defer os.Remove(root)
+
+	middleware := Middleware{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			return 0, nil
+		}),
+		Rules: []Rule{
+			{Path: "dummy.html", Resources: []Resource{
+				{Path: "/index.css", Method: http.MethodGet},
+			}}},
+		Root: http.Dir(root),
+	}
+
+	indexFilePath := filepath.Join(root, indexFile)
+	_, err = os.Create(indexFilePath)
+	if err != nil {
+		t.Fatalf("Could not create index file: %s: %v", indexFile, err)
+	}
+	defer os.Remove(indexFilePath)
+
+	pushingWriter := &MockedPusher{
+		ResponseWriter: httptest.NewRecorder(),
+		returnedError:  errors.New("Cannot push right now"),
+	}
+
+	// when
+	_, err2 := middleware.ServeHTTP(pushingWriter, request)
+
+	// then
+	if err2 != nil {
+		t.Error("Should not return error")
+	}
+
+	expectedPushedResources := map[string]*http.PushOptions{}
+
+	comparePushedResources(t, expectedPushedResources, pushingWriter.pushed)
+}
+
 func comparePushedResources(t *testing.T, expected, actual map[string]*http.PushOptions) {
 	if len(expected) != len(actual) {
 		t.Errorf("Expected %d pushed resources, actual: %d", len(expected), len(actual))

caddyhttp/push/link_parser.go

@@ -0,0 +1,63 @@
+package push
+
+import (
+	"strings"
+)
+
+const (
+	commaSeparator     = ","
+	semicolonSeparator = ";"
+	equalSeparator     = "="
+)
+
+type linkResource struct {
+	uri    string
+	params map[string]string
+}
+
+// parseLinkHeader is responsible for parsing Link header and returning list of found resources.
+//
+// Accepted formats are:
+// Link: </resource>; as=script
+// Link: </resource>; as=script,</resource2>; as=style
+// Link: </resource>;</resource2>
+func parseLinkHeader(header string) []linkResource {
+	resources := []linkResource{}
+
+	if header == "" {
+		return resources
+	}
+
+	for _, link := range strings.Split(header, commaSeparator) {
+		l := linkResource{params: make(map[string]string)}
+
+		li, ri := strings.Index(link, "<"), strings.Index(link, ">")
+
+		if li == -1 || ri == -1 {
+			continue
+		}
+
+		l.uri = strings.TrimSpace(link[li+1 : ri])
+
+		for _, param := range strings.Split(strings.TrimSpace(link[ri+1:]), semicolonSeparator) {
+			parts := strings.SplitN(strings.TrimSpace(param), equalSeparator, 2)
+			key := strings.TrimSpace(parts[0])
+
+			if key == "" {
+				continue
+			}
+
+			if len(parts) == 1 {
+				l.params[key] = key
+			}
+
+			if len(parts) == 2 {
+				l.params[key] = strings.TrimSpace(parts[1])
+			}
+		}
+
+		resources = append(resources, l)
+	}
+
+	return resources
+}

caddyhttp/push/link_parser_test.go

@@ -0,0 +1,73 @@
+package push
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestDifferentParserInputs(t *testing.T) {
+	testCases := []struct {
+		header            string
+		expectedResources []linkResource
+	}{
+		{
+			header:            "</resource>; as=script",
+			expectedResources: []linkResource{{uri: "/resource", params: map[string]string{"as": "script"}}},
+		},
+		{
+			header:            "</resource>",
+			expectedResources: []linkResource{{uri: "/resource", params: map[string]string{}}},
+		},
+		{
+			header:            "</resource>; nopush",
+			expectedResources: []linkResource{{uri: "/resource", params: map[string]string{"nopush": "nopush"}}},
+		},
+		{
+			header:            "</resource>;nopush;rel=next",
+			expectedResources: []linkResource{{uri: "/resource", params: map[string]string{"nopush": "nopush", "rel": "next"}}},
+		},
+		{
+			header: "</resource>;nopush;rel=next,</resource2>;nopush",
+			expectedResources: []linkResource{
+				{uri: "/resource", params: map[string]string{"nopush": "nopush", "rel": "next"}},
+				{uri: "/resource2", params: map[string]string{"nopush": "nopush"}},
+			},
+		},
+		{
+			header: "</resource>,</resource2>",
+			expectedResources: []linkResource{
+				{uri: "/resource", params: map[string]string{}},
+				{uri: "/resource2", params: map[string]string{}},
+			},
+		},
+		{
+			header:            "malformed",
+			expectedResources: []linkResource{},
+		},
+		{
+			header:            "<malformed",
+			expectedResources: []linkResource{},
+		},
+		{
+			header:            ",",
+			expectedResources: []linkResource{},
+		},
+		{
+			header:            ";",
+			expectedResources: []linkResource{},
+		},
+		{
+			header:            "</resource> ; ",
+			expectedResources: []linkResource{{uri: "/resource", params: map[string]string{}}},
+		},
+	}
+
+	for i, test := range testCases {
+
+		actualResources := parseLinkHeader(test.header)
+
+		if !reflect.DeepEqual(actualResources, test.expectedResources) {
+			t.Errorf("Test %d (header: %s) - expected resources %v, got %v", i, test.header, test.expectedResources, actualResources)
+		}
+	}
+}

caddyhttp/push/push.go

@@ -24,6 +24,7 @@ type (
 	Middleware struct {
 		Next  httpserver.Handler
 		Rules []Rule
+		Root  http.FileSystem
 	}
 
 	ruleOp func([]Resource)

caddyhttp/push/setup.go

@@ -34,8 +34,9 @@ func setup(c *caddy.Controller) error {
 		return err
 	}
 
-	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
-		return Middleware{Next: next, Rules: rules}
+	cfg := httpserver.GetConfig(c)
+	cfg.AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
+		return Middleware{Next: next, Rules: rules, Root: http.Dir(cfg.Root)}
 	})
 
 	return nil

caddyhttp/requestid/requestid.go

@@ -0,0 +1,34 @@
+package requestid
+
+import (
+	"context"
+	"log"
+	"net/http"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+	uuid "github.com/nu7hatch/gouuid"
+)
+
+// Handler is a middleware handler
+type Handler struct {
+	Next httpserver.Handler
+}
+
+func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
+	reqid := UUID()
+	c := context.WithValue(r.Context(), httpserver.RequestIDCtxKey, reqid)
+	r = r.WithContext(c)
+
+	return h.Next.ServeHTTP(w, r)
+}
+
+// UUID returns U4 UUID
+func UUID() string {
+	u4, err := uuid.NewV4()
+	if err != nil {
+		log.Printf("[ERROR] generating request ID: %v", err)
+		return ""
+	}
+
+	return u4.String()
+}

caddyhttp/requestid/requestid_test.go

@@ -0,0 +1,33 @@
+package requestid
+
+import (
+	"context"
+	"net/http"
+	"testing"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func TestRequestID(t *testing.T) {
+	request, err := http.NewRequest("GET", "http://localhost/", nil)
+	if err != nil {
+		t.Fatal("Could not create HTTP request:", err)
+	}
+
+	reqid := UUID()
+
+	c := context.WithValue(request.Context(), httpserver.RequestIDCtxKey, reqid)
+
+	request = request.WithContext(c)
+
+	// See caddyhttp/replacer.go
+	value, _ := request.Context().Value(httpserver.RequestIDCtxKey).(string)
+
+	if value == "" {
+		t.Fatal("Request ID should not be empty")
+	}
+
+	if value != reqid {
+		t.Fatal("Request ID does not match")
+	}
+}

caddyhttp/requestid/setup.go

@@ -0,0 +1,27 @@
+package requestid
+
+import (
+	"github.com/mholt/caddy"
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func init() {
+	caddy.RegisterPlugin("request_id", caddy.Plugin{
+		ServerType: "http",
+		Action:     setup,
+	})
+}
+
+func setup(c *caddy.Controller) error {
+	for c.Next() {
+		if c.NextArg() {
+			return c.ArgErr() //no arg expected.
+		}
+	}
+
+	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
+		return Handler{Next: next}
+	})
+
+	return nil
+}

caddyhttp/requestid/setup_test.go

@@ -0,0 +1,43 @@
+package requestid
+
+import (
+	"testing"
+
+	"github.com/mholt/caddy"
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func TestSetup(t *testing.T) {
+	c := caddy.NewTestController("http", `requestid`)
+	err := setup(c)
+	if err != nil {
+		t.Errorf("Expected no errors, got: %v", err)
+	}
+	mids := httpserver.GetConfig(c).Middleware()
+	if len(mids) == 0 {
+		t.Fatal("Expected middleware, got 0 instead")
+	}
+
+	handler := mids[0](httpserver.EmptyNext)
+	myHandler, ok := handler.(Handler)
+
+	if !ok {
+		t.Fatalf("Expected handler to be type Handler, got: %#v", handler)
+	}
+
+	if !httpserver.SameNext(myHandler.Next, httpserver.EmptyNext) {
+		t.Error("'Next' field of handler was not set properly")
+	}
+}
+
+func TestSetupWithArg(t *testing.T) {
+	c := caddy.NewTestController("http", `requestid abc`)
+	err := setup(c)
+	if err == nil {
+		t.Errorf("Expected an error, got: %v", err)
+	}
+	mids := httpserver.GetConfig(c).Middleware()
+	if len(mids) != 0 {
+		t.Fatal("Expected no middleware")
+	}
+}

caddyhttp/rewrite/rewrite.go

@@ -84,19 +84,19 @@ type ComplexRule struct {
 	// Request matcher
 	httpserver.RequestMatcher
 
-	*regexp.Regexp
+	Regexp *regexp.Regexp
 }
 
 // NewComplexRule creates a new RegexpRule. It returns an error if regexp
 // pattern (pattern) or extensions (ext) are invalid.
-func NewComplexRule(base, pattern, to string, ext []string, matcher httpserver.RequestMatcher) (*ComplexRule, error) {
+func NewComplexRule(base, pattern, to string, ext []string, matcher httpserver.RequestMatcher) (ComplexRule, error) {
 	// validate regexp if present
 	var r *regexp.Regexp
 	if pattern != "" {
 		var err error
 		r, err = regexp.Compile(pattern)
 		if err != nil {
-			return nil, err
+			return ComplexRule{}, err
 		}
 	}
 
@@ -105,7 +105,7 @@ func NewComplexRule(base, pattern, to string, ext []string, matcher httpserver.R
 		if len(v) < 2 || (len(v) < 3 && v[0] == '!') {
 			// check if no extension is specified
 			if v != "/" && v != "!/" {
-				return nil, fmt.Errorf("invalid extension %v", v)
+				return ComplexRule{}, fmt.Errorf("invalid extension %v", v)
 			}
 		}
 	}
@@ -118,7 +118,7 @@ func NewComplexRule(base, pattern, to string, ext []string, matcher httpserver.R
 		httpserver.PathMatcher(base),
 	)
 
-	return &ComplexRule{
+	return ComplexRule{
 		Base:           base,
 		To:             to,
 		Exts:           ext,
@@ -128,13 +128,13 @@ func NewComplexRule(base, pattern, to string, ext []string, matcher httpserver.R
 }
 
 // BasePath satisfies httpserver.Config
-func (r *ComplexRule) BasePath() string { return r.Base }
+func (r ComplexRule) BasePath() string { return r.Base }
 
 // Match satisfies httpserver.Config.
 //
 // Though ComplexRule embeds a RequestMatcher, additional
 // checks are needed which requires a custom implementation.
-func (r *ComplexRule) Match(req *http.Request) bool {
+func (r ComplexRule) Match(req *http.Request) bool {
 	// validate RequestMatcher
 	// includes if and path
 	if !r.RequestMatcher.Match(req) {
@@ -155,7 +155,7 @@ func (r *ComplexRule) Match(req *http.Request) bool {
 }
 
 // Rewrite rewrites the internal location of the current request.
-func (r *ComplexRule) Rewrite(fs http.FileSystem, req *http.Request) (re Result) {
+func (r ComplexRule) Rewrite(fs http.FileSystem, req *http.Request) (re Result) {
 	replacer := newReplacer(req)
 
 	// validate regexp if present
@@ -189,7 +189,7 @@ func (r *ComplexRule) Rewrite(fs http.FileSystem, req *http.Request) (re Result)
 
 // matchExt matches rPath against registered file extensions.
 // Returns true if a match is found and false otherwise.
-func (r *ComplexRule) matchExt(rPath string) bool {
+func (r ComplexRule) matchExt(rPath string) bool {
 	f := filepath.Base(rPath)
 	ext := path.Ext(f)
 	if ext == "" {
@@ -216,14 +216,14 @@ func (r *ComplexRule) matchExt(rPath string) bool {
 	return !mustUse
 }
 
-func (r *ComplexRule) regexpMatches(rPath string) []string {
+func (r ComplexRule) regexpMatches(rPath string) []string {
 	if r.Regexp != nil {
 		// include trailing slash in regexp if present
 		start := len(r.Base)
 		if strings.HasSuffix(r.Base, "/") {
 			start--
 		}
-		return r.FindStringSubmatch(rPath[start:])
+		return r.Regexp.FindStringSubmatch(rPath[start:])
 	}
 	return nil
 }

caddyhttp/rewrite/setup.go

@@ -76,10 +76,6 @@ func rewriteParse(c *caddy.Controller) ([]httpserver.HandlerConfig, error) {
 						return nil, c.ArgErr()
 					}
 					to = strings.Join(args1, " ")
-					// ensure rewrite path begins with /
-					if !strings.HasPrefix(to, "/") {
-						return nil, c.Errf("%s:%d - Syntax error: Rewrite path must begin with '/'. Provided: '%s'", c.File(), c.Line(), c.Val())
-					}
 				case "ext":
 					args1 := c.RemainingArgs()
 					if len(args1) == 0 {
@@ -94,20 +90,14 @@ func rewriteParse(c *caddy.Controller) ([]httpserver.HandlerConfig, error) {
 			if to == "" {
 				return nil, c.ArgErr()
 			}
-
 			if rule, err = NewComplexRule(base, pattern, to, ext, matcher); err != nil {
 				return nil, err
 			}
 			rules = append(rules, rule)
 
-		// handle case of 2 arguments: "from to"
+		// the only unhandled case is 2 and above
 		default:
-			// ensure rewrite path begins with /
-			topath := strings.Join(args[1:], " ")
-			if !strings.HasPrefix(topath, "/") {
-				return nil, c.Errf("%s:%d - Syntax error: Rewrite path must begin with '/'. Provided: '%s'", c.File(), c.Line(), c.Val())
-			}
-			rule = NewSimpleRule(args[0], topath)
+			rule = NewSimpleRule(args[0], strings.Join(args[1:], " "))
 			rules = append(rules, rule)
 		}
 

caddyhttp/rewrite/setup_test.go

@@ -3,6 +3,7 @@ package rewrite
 import (
 	"fmt"
 	"regexp"
+	"strings"
 	"testing"
 
 	"github.com/mholt/caddy"
@@ -45,19 +46,15 @@ func TestRewriteParse(t *testing.T) {
 			SimpleRule{From: "/from", To: "/to"},
 		}},
 		{`rewrite /from /to
-		  rewrite a /b`, false, []Rule{
+		  rewrite a b`, false, []Rule{
 			SimpleRule{From: "/from", To: "/to"},
-			SimpleRule{From: "a", To: "/b"},
+			SimpleRule{From: "a", To: "b"},
 		}},
-		{`rewrite a b`, true, []Rule{}},
 		{`rewrite a`, true, []Rule{}},
 		{`rewrite`, true, []Rule{}},
-		{`rewrite a b c`, true, []Rule{
+		{`rewrite a b c`, false, []Rule{
 			SimpleRule{From: "a", To: "b c"},
 		}},
-		{`rewrite a /b c`, false, []Rule{
-			SimpleRule{From: "a", To: "/b c"},
-		}},
 	}
 
 	for i, test := range simpleTests {
@@ -101,14 +98,14 @@ func TestRewriteParse(t *testing.T) {
 			r	.*
 			to	/to /index.php?
 		 }`, false, []Rule{
-			&ComplexRule{Base: "/", To: "/to /index.php?", Regexp: regexp.MustCompile(".*")},
+			ComplexRule{Base: "/", To: "/to /index.php?", Regexp: regexp.MustCompile(".*")},
 		}},
 		{`rewrite {
 			regexp	.*
 			to		/to
 			ext		/ html txt
 		 }`, false, []Rule{
-			&ComplexRule{Base: "/", To: "/to", Exts: []string{"/", "html", "txt"}, Regexp: regexp.MustCompile(".*")},
+			ComplexRule{Base: "/", To: "/to", Exts: []string{"/", "html", "txt"}, Regexp: regexp.MustCompile(".*")},
 		}},
 		{`rewrite /path {
 			r	rr
@@ -119,27 +116,27 @@ func TestRewriteParse(t *testing.T) {
 		 	to 		/to /to2
 		 }
 		 `, false, []Rule{
-			&ComplexRule{Base: "/path", To: "/dest", Regexp: regexp.MustCompile("rr")},
-			&ComplexRule{Base: "/", To: "/to /to2", Regexp: regexp.MustCompile("[a-z]+")},
+			ComplexRule{Base: "/path", To: "/dest", Regexp: regexp.MustCompile("rr")},
+			ComplexRule{Base: "/", To: "/to /to2", Regexp: regexp.MustCompile("[a-z]+")},
 		}},
 		{`rewrite {
 			r	.*
 		 }`, true, []Rule{
-			&ComplexRule{},
+			ComplexRule{},
 		}},
 		{`rewrite {
 
 		 }`, true, []Rule{
-			&ComplexRule{},
+			ComplexRule{},
 		}},
 		{`rewrite /`, true, []Rule{
-			&ComplexRule{},
+			ComplexRule{},
 		}},
 		{`rewrite {
 			if {path} match /
 			to		/to
 		 }`, false, []Rule{
-			&ComplexRule{Base: "/", To: "/to"},
+			ComplexRule{Base: "/", To: "/to"},
 		}},
 	}
 
@@ -160,8 +157,8 @@ func TestRewriteParse(t *testing.T) {
 		}
 
 		for j, e := range test.expected {
-			actualRule := actual[j].(*ComplexRule)
-			expectedRule := e.(*ComplexRule)
+			actualRule := actual[j].(ComplexRule)
+			expectedRule := e.(ComplexRule)
 
 			if actualRule.Base != expectedRule.Base {
 				t.Errorf("Test %d, rule %d: Expected Base=%s, got %s",
@@ -179,13 +176,15 @@ func TestRewriteParse(t *testing.T) {
 			}
 
 			if actualRule.Regexp != nil {
-				if actualRule.String() != expectedRule.String() {
+				if actualRule.Regexp.String() != expectedRule.Regexp.String() {
 					t.Errorf("Test %d, rule %d: Expected Pattern=%s, got %s",
-						i, j, expectedRule.String(), actualRule.String())
+						i, j, actualRule.Regexp.String(), expectedRule.Regexp.String())
 				}
 			}
+		}
 
+		if rules_fmt := fmt.Sprintf("%v", actual); strings.HasPrefix(rules_fmt, "%!") {
+			t.Errorf("Test %d: Failed to string encode: %#v", i, rules_fmt)
 		}
 	}
-
 }