Update changes and readme for version 0.10.5

Also clarified a comment in SiteConfig

.github/CONTRIBUTING.md

@@ -23,13 +23,13 @@ Other menu items:
 
 ### Contributing code
 
-You can have a direct impact on the project by helping with its code. To contribute code to Caddy, open a [pull request](https://github.com/mholt/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.**
+You can have a direct impact on the project by helping with its code. To contribute code to Caddy, open a [pull request](https://github.com/mholt/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with Caddy's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/mholt/caddy/-/search).
 
 We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :wink: If your change is on the right track, we can guide you to make it mergable.
 
 Here are some of the expectations we have of contributors:
 
-- If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, you can comment on the existing one to claim it.
+- If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, comment on the existing issue to claim it.
 
 - **Keep pull requests small.** Smaller PRs are more likely to be merged because they are easier to review! We might ask you to break up large PRs into smaller ones. [An example of what we DON'T do.](https://twitter.com/iamdevloper/status/397664295875805184)
 
@@ -37,11 +37,11 @@ Here are some of the expectations we have of contributors:
 
 - **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass.
 
-- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven to work better with benchmarks or profiling.
+- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks or profiling.
 
-- **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance.
+- **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance. Interactive rebase can do this, or a simpler way is `git reset --soft <diverging-commit>` then `git commit -s`.
 
-- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors can help maintain their change after it is merged.
+- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors help maintain their change after it is merged.
 
 - **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious.
 
@@ -126,7 +126,9 @@ Collabators have push rights to the repository. We grant this permission after o
 
 - **Do not merge pull requests until they have been approved by one or two other collaborators.** If a project owner approves the PR, it can be merged (as long as the conversation has finished too).
 
-- **Prefer squashed commits over a messy merge.** If there are many little commits, please squash the commits so we don't clutter the commit history.
+- **Prefer squashed commits over a messy merge.** If there are many little commits, please [squash the commits](https://stackoverflow.com/a/11732910/1048862) so we don't clutter the commit history.
+
+- **Don't accept new dependencies lightly.** Dependencies can make the world crash and burn, but they are sometimes necessary. Choose carefully. Extremely small dependencies (a few lines of code) can be inlined. The rest may not be needed. For those that are, Caddy vendors all dependencies with the help of [gvt](https://github.com/FiloSottile/gvt). All external dependencies must be vendored, and _Caddy must not export any types defined by those dependencies_. Check this diligently!
 
 - **Be extra careful in some areas of the code.** There are some critical areas in the Caddy code base that we review extra meticulously: the `caddy` and `caddytls` packages especially.
 

.github/ISSUE_TEMPLATE.md

@@ -1,4 +1,6 @@
-(Are you asking for help with using Caddy? Please use our forum instead: https://caddy.community. If you are filing a bug report, please take a few minutes to carefully answer the following questions. If your issue is not a bug report, you do not need to use this template. Thanks!)
+<!--
+Are you asking for help with using Caddy? Please use our forum instead: https://caddy.community. If you are filing a bug report, please take a few minutes to carefully answer the following questions. If your issue is not a bug report, you do not need to use this template. Thanks!)
+-->
 
 ### 1. What version of Caddy are you using (`caddy -version`)?
 
@@ -16,7 +18,7 @@
 
 ### 5. Please paste any relevant HTTP request(s) here.
 
-(paste curl command, or full HTTP request including headers and body, here)
+<!-- Paste curl command, or full HTTP request including headers and body, here. -->
 
 
 ### 6. What did you expect to see?
@@ -27,4 +29,4 @@
 
 ### 8. How can someone who is starting from scratch reproduce the bug as minimally as possible?
 
-(Please strip away any extra infrastructure such as containers, reverse proxies, upstream apps, caches, dependencies, etc, to prove this is a bug in Caddy and not an external misconfiguration. Your chances of getting this bug fixed go way up the easier it is to replicate. Thank you!)
+<!-- Please strip away any extra infrastructure such as containers, reverse proxies, upstream apps, caches, dependencies, etc, to prove this is a bug in Caddy and not an external misconfiguration. Your chances of getting this bug fixed go way up the easier it is to replicate. Thank you! -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,4 +1,6 @@
-(Thank you for contributing to Caddy! Please fill this out to help us make the most of your pull request.)
+<!--
+Thank you for contributing to Caddy! Please fill this out to help us make the most of your pull request.
+-->
 
 ### 1. What does this change do, exactly?
 

.travis.yml

@@ -1,7 +1,7 @@
 language: go
 
 go:
-  - 1.8
+  - 1.8.3
   - tip
 
 matrix:
@@ -20,15 +20,20 @@ install:
   - if [ "$TRAVIS_PULL_REQUEST" = "false" ]; then bash dist/gitcookie.sh; fi
   - go get -t ./...
   - go get github.com/golang/lint/golint
-  - go get github.com/gordonklaus/ineffassign
+  - go get github.com/FiloSottile/vendorcheck
+  # Install gometalinter and certain linters
+  - go get github.com/alecthomas/gometalinter
   - go get github.com/client9/misspell/cmd/misspell
+  - go get github.com/gordonklaus/ineffassign
+  - go get golang.org/x/tools/cmd/goimports
+  - go get github.com/tsenart/deadcode
 
 script:
-  - diff <(echo -n) <(gofmt -s -d .)
-  - ineffassign .
-  - misspell -error .
-  - go vet ./...
-  - go test -race ./...
+  - gometalinter --disable-all -E vet -E gofmt -E misspell -E ineffassign -E goimports -E deadcode --tests --vendor ./...
+  - vendorcheck ./...
+  # TODO: When Go 1.9 is released, replace $(go list) subcommand with ./... because vendor folder should be ignored
+  - go test -race $(go list ./... | grep -v vendor)
 
 after_script:
-  - golint ./...
+  # TODO: When Go 1.9 is released, replace $(go list) subcommand with ./... because vendor folder should be ignored
+  - golint $(go list ./... | grep -v vendor)

README.md

@@ -1,7 +1,7 @@
 <p align="center">
 	<a href="https://caddyserver.com"><img src="https://cloud.githubusercontent.com/assets/1128849/25305033/12916fce-2731-11e7-86ec-580d4d31cb16.png" alt="Caddy" width="400"></a>
 </p>
-<h3 align="center">Every Site on HTTPS <!-- Serve Confidently. --></h3>
+<h3 align="center">Every Site on HTTPS <!-- Serve Confidently --></h3>
 <p align="center">Caddy is a general-purpose HTTP/2 web server that serves HTTPS by default.</p>
 <p align="center">
 	<a href="https://travis-ci.org/mholt/caddy"><img src="https://img.shields.io/travis/mholt/caddy.svg?label=linux+build"></a>
@@ -123,12 +123,14 @@ The Caddy project does not officially maintain any system-specific integrations
 
 How you choose to run Caddy is up to you. Many users are satisfied with `nohup caddy &`. Others use `screen`. Users who need Caddy to come back up after reboots either do so in the script that caused the reboot, add a command to an init script, or configure a service with their OS.
 
+If you have questions or concerns about Caddy' underlying crypto implementations, consult Go's [crypto packages](https://golang.org/pkg/crypto), starting with their documentation, then issues, then the code itself; as Caddy uses mainly those libraries.
+
 
 ## Contributing
 
-**[Join our forum](https://caddy.community) where you can chat with other Caddy users and developers!**
+**[Join our forum](https://caddy.community) where you can chat with other Caddy users and developers!** To get familiar with the code base, try [Caddy code search on Sourcegraph](https://sourcegraph.com/github.com/mholt/caddy/-/search)!
 
-Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/CONTRIBUTING.md). If you want to write a plugin, check out the [developer wiki](https://github.com/mholt/caddy/wiki).
+Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/.github/CONTRIBUTING.md) for instructions. If you want to write a plugin, check out the [developer wiki](https://github.com/mholt/caddy/wiki).
 
 We use GitHub issues and pull requests only for discussing bug reports and the development of specific changes. We welcome all other topics on the [forum](https://caddy.community)!
 

appveyor.yml

@@ -9,23 +9,31 @@ environment:
 
 install:
   - rmdir c:\go /s /q
-  - appveyor DownloadFile https://storage.googleapis.com/golang/go1.8.windows-amd64.zip
-  - 7z x go1.8.windows-amd64.zip -y -oC:\ > NUL
+  - appveyor DownloadFile https://storage.googleapis.com/golang/go1.8.3.windows-amd64.zip
+  - 7z x go1.8.3.windows-amd64.zip -y -oC:\ > NUL
+  - set PATH=%GOPATH%\bin;%PATH%
   - go version
   - go env
   - go get -t ./...
   - go get github.com/golang/lint/golint
+  - go get github.com/FiloSottile/vendorcheck
+   # Install gometalinter and certain linters
+  - go get github.com/alecthomas/gometalinter
+  - go get github.com/client9/misspell/cmd/misspell
   - go get github.com/gordonklaus/ineffassign
-  - set PATH=%GOPATH%\bin;%PATH%
+  - go get golang.org/x/tools/cmd/goimports
+  - go get github.com/tsenart/deadcode
 
 build: off
 
 test_script:
-  - go vet ./...
-  - go test -race ./...
-  - ineffassign .
+  - gometalinter --disable-all -E vet -E gofmt -E misspell -E ineffassign -E goimports -E deadcode --tests --vendor ./...
+  - vendorcheck ./... 
+  # TODO: When Go 1.9 comes out, replace this whole line with `go test -race ./...` b/c vendor folder should be ignored
+  - for /f "" %%G in ('go list ./... ^| find /i /v "/vendor/"') do (go test -race %%G & IF ERRORLEVEL == 1 EXIT 1)
 
 after_test:
-  - golint ./...
+  # TODO: When Go 1.9 comes out, replace this whole line with `golint ./...` b/c vendor folder should be ignored
+  - for /f "" %%G in ('go list ./... ^| find /i /v "/vendor/"') do (golint %%G & IF ERRORLEVEL == 1 EXIT 1)
 
 deploy: off

caddy/caddymain/run.go

@@ -101,7 +101,7 @@ func Run() {
 	}
 
 	// Executes Startup events
-	caddy.EmitEvent(caddy.StartupEvent)
+	caddy.EmitEvent(caddy.StartupEvent, nil)
 
 	// Get Caddyfile input
 	caddyfileinput, err := caddy.LoadCaddyfile(serverType)

caddy_test.go

@@ -142,13 +142,13 @@ func TestListenerAddrEqual(t *testing.T) {
 		addr   string
 		expect bool
 	}{
-		{ln1, ":1234", false},
-		{ln1, "0.0.0.0:1234", false},
+		{ln1, ":" + ln2port, false},
+		{ln1, "0.0.0.0:" + ln2port, false},
 		{ln1, "0.0.0.0", false},
 		{ln1, ":" + ln1port, true},
 		{ln1, "0.0.0.0:" + ln1port, true},
 		{ln2, ":" + ln2port, false},
-		{ln2, "127.0.0.1:1234", false},
+		{ln2, "127.0.0.1:" + ln1port, false},
 		{ln2, "127.0.0.1", false},
 		{ln2, "127.0.0.1:" + ln2port, true},
 	} {

caddyhttp/basicauth/setup.go

@@ -91,8 +91,9 @@ func basicAuthParse(c *caddy.Controller) ([]Rule, error) {
 }
 
 func passwordMatcher(username, passw, siteRoot string) (PasswordMatcher, error) {
-	if !strings.HasPrefix(passw, "htpasswd=") {
+	htpasswdPrefix := "htpasswd="
+	if !strings.HasPrefix(passw, htpasswdPrefix) {
 		return PlainMatcher(passw), nil
 	}
-	return GetHtpasswdMatcher(passw[9:], username, siteRoot)
+	return GetHtpasswdMatcher(passw[len(htpasswdPrefix):], username, siteRoot)
 }

caddyhttp/browse/browse.go

@@ -9,6 +9,7 @@ import (
 	"net/url"
 	"os"
 	"path"
+	"path/filepath"
 	"sort"
 	"strconv"
 	"strings"
@@ -112,12 +113,13 @@ func (l Listing) Breadcrumbs() []Crumb {
 
 // FileInfo is the info about a particular file or directory
 type FileInfo struct {
-	Name    string
-	Size    int64
-	URL     string
-	ModTime time.Time
-	Mode    os.FileMode
-	IsDir   bool
+	Name      string
+	Size      int64
+	URL       string
+	ModTime   time.Time
+	Mode      os.FileMode
+	IsDir     bool
+	IsSymlink bool
 }
 
 // HumanSize returns the size of the file as a human-readable string
@@ -258,12 +260,13 @@ func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string, config
 		url := url.URL{Path: "./" + name} // prepend with "./" to fix paths with ':' in the name
 
 		fileinfos = append(fileinfos, FileInfo{
-			IsDir:   f.IsDir(),
-			Name:    f.Name(),
-			Size:    f.Size(),
-			URL:     url.String(),
-			ModTime: f.ModTime().UTC(),
-			Mode:    f.Mode(),
+			IsDir:     f.IsDir() || isSymlinkTargetDir(f, urlPath, config),
+			IsSymlink: isSymlink(f),
+			Name:      f.Name(),
+			Size:      f.Size(),
+			URL:       url.String(),
+			ModTime:   f.ModTime().UTC(),
+			Mode:      f.Mode(),
 		})
 	}
 
@@ -277,6 +280,32 @@ func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string, config
 	}, hasIndexFile
 }
 
+// isSymlink return true if f is a symbolic link
+func isSymlink(f os.FileInfo) bool {
+	return f.Mode()&os.ModeSymlink != 0
+}
+
+// isSymlinkTargetDir return true if f's symbolic link target
+// is a directory. Return false if not a symbolic link.
+func isSymlinkTargetDir(f os.FileInfo, urlPath string, config *Config) bool {
+	if !isSymlink(f) {
+		return false
+	}
+	fullPath := func(fileName string) string {
+		fullPath := filepath.Join(string(config.Fs.Root.(http.Dir)), urlPath, fileName)
+		return filepath.Clean(fullPath)
+	}
+	target, err := os.Readlink(fullPath(f.Name()))
+	if err != nil {
+		return false
+	}
+	targetInfo, err := os.Lstat(fullPath(target))
+	if err != nil {
+		return false
+	}
+	return targetInfo.IsDir()
+}
+
 // ServeHTTP determines if the request is for this plugin, and if all prerequisites are met.
 // If so, control is handed over to ServeListing.
 func (b Browse) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {

caddyhttp/browse/setup.go

@@ -150,16 +150,22 @@ h1 {
 	white-space: nowrap;
 	overflow-x: hidden;
 	text-overflow: ellipsis;
+	color: #999;
 }
 
 h1 a {
-	color: inherit;
+	color: #000;
+	margin: 0 4px;
 }
 
 h1 a:hover {
 	text-decoration: underline;
 }
 
+h1 a:first-child {
+	margin: 0;
+}
+
 main {
 	display: block;
 }
@@ -284,6 +290,18 @@ footer {
 		padding-right: 5%;
 		text-align: right;
 	}
+
+	h1 {
+		color: #000;
+	}
+
+	h1 a {
+		margin: 0;
+	}
+
+	#filter {
+		max-width: 100px;
+	}
 }
 </style>
 	</head>
@@ -291,45 +309,32 @@ footer {
 		<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" height="0" width="0" style="position: absolute;">
 			<defs>
 				<!-- Folder -->
-				<linearGradient id="f" y2="640" gradientUnits="userSpaceOnUse" x2="244.84" gradientTransform="matrix(.97319 0 0 1.0135 -.50695 -13.679)" y1="415.75" x1="244.84">
-					<stop stop-color="#b3ddfd" offset="0"/>
-					<stop stop-color="#69c" offset="1"/>
-				</linearGradient>
-				<linearGradient id="e" y2="571.06" gradientUnits="userSpaceOnUse" x2="238.03" gradientTransform="translate(0,2)" y1="346.05" x1="236.26">
-					<stop stop-color="#ace" offset="0"/>
-					<stop stop-color="#369" offset="1"/>
-				</linearGradient>
-				<g id="folder" transform="translate(-266.06 -193.36)">
-					<g transform="matrix(.066019 0 0 .066019 264.2 170.93)">
-						<g transform="matrix(1.4738 0 0 1.4738 -52.053 -166.93)">
-							<path fill="#69c" d="m98.424 343.78c-11.08 0-20 8.92-20 20v48.5 33.719 105.06c0 11.08 8.92 20 20 20h279.22c11.08 0 20-8.92 20-20v-138.78c0-11.08-8.92-20-20-20h-117.12c-7.5478-1.1844-9.7958-6.8483-10.375-11.312v-5.625-11.562c0-11.08-8.92-20-20-20h-131.72z"/>
-							<rect rx="12.885" ry="12.199" height="227.28" width="366.69" y="409.69" x="54.428" fill="#369"/>
-							<path fill="url(#e)" d="m98.424 345.78c-11.08 0-20 8.92-20 20v48.5 33.719 105.06c0 11.08 8.92 20 20 20h279.22c11.08 0 20-8.92 20-20v-138.78c0-11.08-8.92-20-20-20h-117.12c-7.5478-1.1844-9.7958-6.8483-10.375-11.312v-5.625-11.562c0-11.08-8.92-20-20-20h-131.72z"/>
-							<rect rx="12.885" ry="12.199" height="227.28" width="366.69" y="407.69" x="54.428" fill="url(#f)"/>
+				<g id="folder" fill-rule="nonzero" fill="none">
+					<path d="M285.22 37.55h-142.6L110.9 0H31.7C14.25 0 0 16.9 0 37.55v75.1h316.92V75.1c0-20.65-14.26-37.55-31.7-37.55z" fill="#FFA000"/>
+					<path d="M285.22 36H31.7C14.25 36 0 50.28 0 67.74v158.7c0 17.47 14.26 31.75 31.7 31.75H285.2c17.44 0 31.7-14.3 31.7-31.75V67.75c0-17.47-14.26-31.75-31.7-31.75z" fill="#FFCA28"/>
+				</g>
+				<g id="folder-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+					<g id="folder-shortcut-group" fill-rule="nonzero">
+						<g id="folder-shortcut-shape">
+							<path d="M285.224876,37.5486902 L142.612438,37.5486902 L110.920785,0 L31.6916529,0 C14.2612438,0 0,16.8969106 0,37.5486902 L0,112.646071 L316.916529,112.646071 L316.916529,75.0973805 C316.916529,54.4456008 302.655285,37.5486902 285.224876,37.5486902 Z" id="Shape" fill="#FFA000"></path>
+							<path d="M285.224876,36 L31.6916529,36 C14.2612438,36 0,50.2838568 0,67.7419039 L0,226.451424 C0,243.909471 14.2612438,258.193328 31.6916529,258.193328 L285.224876,258.193328 C302.655285,258.193328 316.916529,243.909471 316.916529,226.451424 L316.916529,67.7419039 C316.916529,50.2838568 302.655285,36 285.224876,36 Z" id="Shape" fill="#FFCA28"></path>
 						</g>
+						<path d="M126.154134,250.559184 C126.850974,251.883673 127.300549,253.006122 127.772602,254.106122 C128.469442,255.206122 128.919016,256.104082 129.638335,257.002041 C130.559962,258.326531 131.728855,259 133.100057,259 C134.493737,259 135.415364,258.55102 136.112204,257.67551 C136.809044,257.002041 137.258619,255.902041 137.258619,254.577551 C137.258619,253.904082 137.258619,252.804082 137.033832,251.457143 C136.786566,249.908163 136.561779,249.032653 136.561779,248.583673 C136.089726,242.814286 135.864939,237.920408 135.864939,233.273469 C135.864939,225.057143 136.786566,217.514286 138.180246,210.846939 C139.798713,204.202041 141.889234,198.634694 144.429328,193.763265 C147.216689,188.869388 150.678411,184.873469 154.836973,181.326531 C158.995535,177.779592 163.626149,174.883673 168.481552,172.661224 C173.336954,170.438776 179.113983,168.665306 185.587852,167.340816 C192.061722,166.218367 198.760378,165.342857 205.481514,164.669388 C212.18017,164.220408 219.598146,163.995918 228.162535,163.995918 L246.055591,163.995918 L246.055591,195.514286 C246.055591,197.736735 246.752431,199.510204 248.370899,201.059184 C250.214153,202.608163 252.079886,203.506122 254.372715,203.506122 C256.463236,203.506122 258.531277,202.608163 260.172223,201.059184 L326.102289,137.797959 C327.720757,136.24898 328.642384,134.47551 328.642384,132.253061 C328.642384,130.030612 327.720757,128.257143 326.102289,126.708163 L260.172223,63.4469388 C258.553756,61.8979592 256.463236,61 254.395194,61 C252.079886,61 250.236632,61.8979592 248.393377,63.4469388 C246.77491,64.9959184 246.07807,66.7693878 246.07807,68.9918367 L246.07807,100.510204 L228.162535,100.510204 C166.863084,100.510204 129.166282,117.167347 115.274437,150.459184 C110.666301,161.54898 108.350993,175.310204 108.350993,191.742857 C108.350993,205.279592 113.903236,223.912245 124.760454,247.438776 C125.00772,248.112245 125.457294,249.010204 126.154134,250.559184 Z" id="Shape" fill="#FFFFFF" transform="translate(218.496689, 160.000000) scale(-1, 1) translate(-218.496689, -160.000000) "></path>
 					</g>
 				</g>
 
 				<!-- File -->
-				<linearGradient id="a">
-					<stop stop-color="#cbcbcb" offset="0"/>
-					<stop stop-color="#f0f0f0" offset=".34923"/>
-					<stop stop-color="#e2e2e2" offset="1"/>
-				</linearGradient>
-				<linearGradient id="d" y2="686.15" xlink:href="#a" gradientUnits="userSpaceOnUse" y1="207.83" gradientTransform="matrix(.28346 0 0 .31053 -608.52 485.11)" x2="380.1" x1="749.25"/>
-				<linearGradient id="c" y2="287.74" xlink:href="#a" gradientUnits="userSpaceOnUse" y1="169.44" gradientTransform="matrix(.28342 0 0 .31057 -608.52 485.11)" x2="622.33" x1="741.64"/>
-				<linearGradient id="b" y2="418.54" gradientUnits="userSpaceOnUse" y1="236.13" gradientTransform="matrix(.29343 0 0 .29999 -608.52 485.11)" x2="330.88" x1="687.96">
-					<stop stop-color="#fff" offset="0"/>
-					<stop stop-color="#fff" stop-opacity="0" offset="1"/>
-				</linearGradient>
-				<g id="file" transform="translate(-278.15 -216.59)">
-					<g fill-rule="evenodd" transform="matrix(.19775 0 0 .19775 381.05 112.68)">
-						<path d="m-520.17 525.5v36.739 36.739 36.739 36.739h33.528 33.528 33.528 33.528v-36.739-36.739-36.739l-33.528-36.739h-33.528-33.528-33.528z" stroke-opacity=".36478" stroke-width=".42649" fill="#fff"/>
-						<g>
-							<path d="m-520.11 525.68v36.739 36.739 36.739 36.739h33.528 33.528 33.528 33.528v-36.739-36.739-36.739l-33.528-36.739h-33.528-33.528-33.528z" stroke-opacity=".36478" stroke="#000" stroke-width=".42649" fill="url(#d)"/>
-							<path d="m-386 562.42c-10.108-2.9925-23.206-2.5682-33.101-0.86253 1.7084-10.962 1.922-24.701-0.4271-35.877l33.528 36.739z" stroke-width=".95407pt" fill="url(#c)"/>
-							<path d="m-519.13 537-0.60402 134.7h131.68l0.0755-33.296c-2.9446 1.1325-32.692-40.998-70.141-39.186-37.483 1.8137-27.785-56.777-61.006-62.214z" stroke-width="1pt" fill="url(#b)"/>
+				<g id="file" stroke="#000" stroke-width="25" fill="#FFF" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round">
+					<path d="M13 24.12v274.76c0 6.16 5.87 11.12 13.17 11.12H239c7.3 0 13.17-4.96 13.17-11.12V136.15S132.6 13 128.37 13H26.17C18.87 13 13 17.96 13 24.12z"/>
+					<path d="M129.37 13L129 113.9c0 10.58 7.26 19.1 16.27 19.1H249L129.37 13z"/>
+				</g>
+				<g id="file-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+					<g id="file-shortcut-group" transform="translate(13.000000, 13.000000)">
+						<g id="file-shortcut-shape" stroke="#000000" stroke-width="25" fill="#FFFFFF" stroke-linecap="round" stroke-linejoin="round">
+							<path d="M0,11.1214886 L0,285.878477 C0,292.039924 5.87498876,296.999983 13.1728373,296.999983 L225.997983,296.999983 C233.295974,296.999983 239.17082,292.039942 239.17082,285.878477 L239.17082,123.145388 C239.17082,123.145388 119.58541,2.84217094e-14 115.369423,2.84217094e-14 L13.1728576,2.84217094e-14 C5.87500907,-1.71479982e-05 0,4.96022995 0,11.1214886 Z" id="rect1171"></path>
+							<path d="M116.37005,0 L116,100.904964 C116,111.483663 123.258008,120 132.273377,120 L236,120 L116.37005,0 L116.37005,0 Z" id="rect1794"></path>
 						</g>
+						<path d="M47.803141,294.093878 C48.4999811,295.177551 48.9495553,296.095918 49.4216083,296.995918 C50.1184484,297.895918 50.5680227,298.630612 51.2873415,299.365306 C52.2089688,300.44898 53.3778619,301 54.7490634,301 C56.1427436,301 57.0643709,300.632653 57.761211,299.916327 C58.4580511,299.365306 58.9076254,298.465306 58.9076254,297.381633 C58.9076254,296.830612 58.9076254,295.930612 58.6828382,294.828571 C58.4355724,293.561224 58.2107852,292.844898 58.2107852,292.477551 C57.7387323,287.757143 57.5139451,283.753061 57.5139451,279.95102 C57.5139451,273.228571 58.4355724,267.057143 59.8292526,261.602041 C61.44772,256.165306 63.5382403,251.610204 66.0783349,247.62449 C68.8656954,243.620408 72.3274172,240.35102 76.4859792,237.44898 C80.6445412,234.546939 85.2751561,232.177551 90.1305582,230.359184 C94.9859603,228.540816 100.76299,227.089796 107.236859,226.006122 C113.710728,225.087755 120.409385,224.371429 127.13052,223.820408 C133.829177,223.453061 141.247152,223.269388 149.811542,223.269388 L167.704598,223.269388 L167.704598,249.057143 C167.704598,250.87551 168.401438,252.326531 170.019905,253.593878 C171.86316,254.861224 173.728893,255.595918 176.021722,255.595918 C178.112242,255.595918 180.180284,254.861224 181.82123,253.593878 L247.751296,201.834694 C249.369763,200.567347 250.291391,199.116327 250.291391,197.297959 C250.291391,195.479592 249.369763,194.028571 247.751296,192.761224 L181.82123,141.002041 C180.202763,139.734694 178.112242,139 176.044201,139 C173.728893,139 171.885639,139.734694 170.042384,141.002041 C168.423917,142.269388 167.727077,143.720408 167.727077,145.538776 L167.727077,171.326531 L149.811542,171.326531 C88.5120908,171.326531 50.8152886,184.955102 36.9234437,212.193878 C32.3153075,221.267347 30,232.526531 30,245.971429 C30,257.046939 35.5522422,272.291837 46.4094607,291.540816 C46.6567266,292.091837 47.1063009,292.826531 47.803141,294.093878 Z" id="Shape-Copy" fill="#000000" fill-rule="nonzero" transform="translate(140.145695, 220.000000) scale(-1, 1) translate(-140.145695, -220.000000) "></path>
 					</g>
 				</g>
 
@@ -419,9 +424,9 @@ footer {
 						<td>
 							<a href="{{html .URL}}">
 								{{- if .IsDir}}
-								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 35.678803 28.527945"><use xlink:href="#folder"></use></svg>
+								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 317 259"><use xlink:href="#folder{{if .IsSymlink}}-shortcut{{end}}"></use></svg>
 								{{- else}}
-								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 26.604381 29.144726"><use xlink:href="#file"></use></svg>
+								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 265 323"><use xlink:href="#file{{if .IsSymlink}}-shortcut{{end}}"></use></svg>
 								{{- end}}
 								<span class="name">{{html .Name}}</span>
 							</a>
@@ -443,6 +448,8 @@ footer {
 		</footer>
 		<script>
 			var filterEl = document.getElementById('filter');
+			filterEl.focus();
+
 			function filter() {
 				var q = filterEl.value.trim().toLowerCase();
 				var elems = document.querySelectorAll('tr.file');

caddyhttp/caddyhttp.go

@@ -16,14 +16,15 @@ import (
 	_ "github.com/mholt/caddy/caddyhttp/header"
 	_ "github.com/mholt/caddy/caddyhttp/index"
 	_ "github.com/mholt/caddy/caddyhttp/internalsrv"
+	_ "github.com/mholt/caddy/caddyhttp/limits"
 	_ "github.com/mholt/caddy/caddyhttp/log"
 	_ "github.com/mholt/caddy/caddyhttp/markdown"
-	_ "github.com/mholt/caddy/caddyhttp/maxrequestbody"
 	_ "github.com/mholt/caddy/caddyhttp/mime"
 	_ "github.com/mholt/caddy/caddyhttp/pprof"
 	_ "github.com/mholt/caddy/caddyhttp/proxy"
 	_ "github.com/mholt/caddy/caddyhttp/push"
 	_ "github.com/mholt/caddy/caddyhttp/redirect"
+	_ "github.com/mholt/caddy/caddyhttp/requestid"
 	_ "github.com/mholt/caddy/caddyhttp/rewrite"
 	_ "github.com/mholt/caddy/caddyhttp/root"
 	_ "github.com/mholt/caddy/caddyhttp/status"

caddyhttp/caddyhttp_test.go

@@ -11,7 +11,7 @@ import (
 // ensure that the standard plugins are in fact plugged in
 // and registered properly; this is a quick/naive way to do it.
 func TestStandardPlugins(t *testing.T) {
-	numStandardPlugins := 31 // importing caddyhttp plugs in this many plugins
+	numStandardPlugins := 32 // importing caddyhttp plugs in this many plugins
 	s := caddy.DescribePlugins()
 	if got, want := strings.Count(s, "\n"), numStandardPlugins+5; got != want {
 		t.Errorf("Expected all standard plugins to be plugged in, got:\n%s", s)

caddyhttp/errors/setup.go

@@ -44,18 +44,20 @@ func errorsParse(c *caddy.Controller) (*ErrorHandler, error) {
 		for c.NextBlock() {
 
 			what := c.Val()
-			if !c.NextArg() {
-				return c.ArgErr()
-			}
-			where := c.Val()
+			where := c.RemainingArgs()
 
 			if httpserver.IsLogRollerSubdirective(what) {
 				var err error
-				err = httpserver.ParseRoller(handler.Log.Roller, what, where)
+				err = httpserver.ParseRoller(handler.Log.Roller, what, where...)
 				if err != nil {
 					return err
 				}
 			} else {
+				if len(where) != 1 {
+					return c.ArgErr()
+				}
+				where := where[0]
+
 				// Error page; ensure it exists
 				if !filepath.IsAbs(where) {
 					where = filepath.Join(cfg.Root, where)

caddyhttp/errors/setup_test.go

@@ -85,13 +85,19 @@ func TestErrorsParse(t *testing.T) {
 				Roller: httpserver.DefaultLogRoller(),
 			},
 		}},
-		{`errors errors.txt { rotate_size 2 rotate_age 10 rotate_keep 3 }`, false, ErrorHandler{
+		{`errors errors.txt {
+			rotate_size 2
+			rotate_age 10
+			rotate_keep 3
+			rotate_compress
+		}`, false, ErrorHandler{
 			ErrorPages: map[int]string{},
 			Log: &httpserver.Logger{
 				Output: "errors.txt", Roller: &httpserver.LogRoller{
 					MaxSize:    2,
 					MaxAge:     10,
 					MaxBackups: 3,
+					Compress:   true,
 					LocalTime:  true,
 				},
 			},
@@ -113,6 +119,7 @@ func TestErrorsParse(t *testing.T) {
 					MaxSize:    3,
 					MaxAge:     11,
 					MaxBackups: 5,
+					Compress:   false,
 					LocalTime:  true,
 				},
 			},
@@ -142,6 +149,12 @@ func TestErrorsParse(t *testing.T) {
 				},
 				Log: &httpserver.Logger{},
 			}},
+		{`errors errors.txt { rotate_size 2 rotate_age 10 rotate_keep 3 rotate_compress }`,
+			true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
+		{`errors errors.txt {
+			rotate_compress invalid
+		}`,
+			true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
 		// Next two test cases is the detection of duplicate status codes
 		{`errors {
 			503 503.html

caddyhttp/fastcgi/dialer.go

@@ -1,102 +0,0 @@
-package fastcgi
-
-import (
-	"errors"
-	"sync"
-	"sync/atomic"
-	"time"
-)
-
-type dialer interface {
-	Dial() (Client, error)
-	Close(Client) error
-}
-
-// basicDialer is a basic dialer that wraps default fcgi functions.
-type basicDialer struct {
-	network string
-	address string
-	timeout time.Duration
-}
-
-func (b basicDialer) Dial() (Client, error) {
-	return DialTimeout(b.network, b.address, b.timeout)
-}
-
-func (b basicDialer) Close(c Client) error { return c.Close() }
-
-// persistentDialer keeps a pool of fcgi connections.
-// connections are not closed after use, rather added back to the pool for reuse.
-type persistentDialer struct {
-	size    int
-	network string
-	address string
-	timeout time.Duration
-	pool    []Client
-	sync.Mutex
-}
-
-func (p *persistentDialer) Dial() (Client, error) {
-	p.Lock()
-	// connection is available, return first one.
-	if len(p.pool) > 0 {
-		client := p.pool[0]
-		p.pool = p.pool[1:]
-		p.Unlock()
-
-		return client, nil
-	}
-
-	p.Unlock()
-
-	// no connection available, create new one
-	return DialTimeout(p.network, p.address, p.timeout)
-}
-
-func (p *persistentDialer) Close(client Client) error {
-	p.Lock()
-	if len(p.pool) < p.size {
-		// pool is not full yet, add connection for reuse
-		p.pool = append(p.pool, client)
-		p.Unlock()
-
-		return nil
-	}
-
-	p.Unlock()
-
-	// otherwise, close the connection.
-	return client.Close()
-}
-
-type loadBalancingDialer struct {
-	current int64
-	dialers []dialer
-}
-
-func (m *loadBalancingDialer) Dial() (Client, error) {
-	nextDialerIndex := atomic.AddInt64(&m.current, 1) % int64(len(m.dialers))
-	currentDialer := m.dialers[nextDialerIndex]
-
-	client, err := currentDialer.Dial()
-
-	if err != nil {
-		return nil, err
-	}
-
-	return &dialerAwareClient{Client: client, dialer: currentDialer}, nil
-}
-
-func (m *loadBalancingDialer) Close(c Client) error {
-	// Close the client according to dialer behaviour
-	if da, ok := c.(*dialerAwareClient); ok {
-		return da.dialer.Close(c)
-	}
-
-	return errors.New("Cannot close client")
-}
-
-type dialerAwareClient struct {
-	Client
-	dialer dialer
-}

caddyhttp/fastcgi/dialer_test.go

@@ -1,126 +0,0 @@
-package fastcgi
-
-import (
-	"errors"
-	"testing"
-)
-
-func TestLoadbalancingDialer(t *testing.T) {
-	// given
-	runs := 100
-	mockDialer1 := new(mockDialer)
-	mockDialer2 := new(mockDialer)
-
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1, mockDialer2}}
-
-	// when
-	for i := 0; i < runs; i++ {
-		client, err := dialer.Dial()
-		dialer.Close(client)
-
-		if err != nil {
-			t.Errorf("Expected error to be nil")
-		}
-	}
-
-	// then
-	if mockDialer1.dialCalled != mockDialer2.dialCalled && mockDialer1.dialCalled != 50 {
-		t.Errorf("Expected dialer to call Dial() on multiple backend dialers %d times [actual: %d, %d]", 50, mockDialer1.dialCalled, mockDialer2.dialCalled)
-	}
-
-	if mockDialer1.closeCalled != mockDialer2.closeCalled && mockDialer1.closeCalled != 50 {
-		t.Errorf("Expected dialer to call Close() on multiple backend dialers %d times [actual: %d, %d]", 50, mockDialer1.closeCalled, mockDialer2.closeCalled)
-	}
-}
-
-func TestLoadBalancingDialerShouldReturnDialerAwareClient(t *testing.T) {
-	// given
-	mockDialer1 := new(mockDialer)
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1}}
-
-	// when
-	client, err := dialer.Dial()
-
-	// then
-	if err != nil {
-		t.Errorf("Expected error to be nil")
-	}
-
-	if awareClient, ok := client.(*dialerAwareClient); !ok {
-		t.Error("Expected dialer to wrap client")
-	} else {
-		if awareClient.dialer != mockDialer1 {
-			t.Error("Expected wrapped client to have reference to dialer")
-		}
-	}
-}
-
-func TestLoadBalancingDialerShouldUnderlyingReturnDialerError(t *testing.T) {
-	// given
-	mockDialer1 := new(errorReturningDialer)
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1}}
-
-	// when
-	_, err := dialer.Dial()
-
-	// then
-	if err.Error() != "Error during dial" {
-		t.Errorf("Expected 'Error during dial', got: '%s'", err.Error())
-	}
-}
-
-func TestLoadBalancingDialerShouldCloseClient(t *testing.T) {
-	// given
-	mockDialer1 := new(mockDialer)
-	mockDialer2 := new(mockDialer)
-
-	dialer := &loadBalancingDialer{dialers: []dialer{mockDialer1, mockDialer2}}
-	client, _ := dialer.Dial()
-
-	// when
-	err := dialer.Close(client)
-
-	// then
-	if err != nil {
-		t.Error("Expected error not to occur")
-	}
-
-	// load balancing starts from index 1
-	if mockDialer2.client != client {
-		t.Errorf("Expected Close() to be called on referenced dialer")
-	}
-}
-
-type mockDialer struct {
-	dialCalled  int
-	closeCalled int
-	client      Client
-}
-
-type mockClient struct {
-	Client
-}
-
-func (m *mockDialer) Dial() (Client, error) {
-	m.dialCalled++
-	return mockClient{Client: &FCGIClient{}}, nil
-}
-
-func (m *mockDialer) Close(c Client) error {
-	m.client = c
-	m.closeCalled++
-	return nil
-}
-
-type errorReturningDialer struct {
-	client Client
-}
-
-func (m *errorReturningDialer) Dial() (Client, error) {
-	return mockClient{Client: &FCGIClient{}}, errors.New("Error during dial")
-}
-
-func (m *errorReturningDialer) Close(c Client) error {
-	m.client = c
-	return errors.New("Error during close")
-}

caddyhttp/fastcgi/fastcgi.go

@@ -4,6 +4,7 @@
 package fastcgi
 
 import (
+	"context"
 	"errors"
 	"io"
 	"net"
@@ -14,6 +15,7 @@ import (
 	"path/filepath"
 	"strconv"
 	"strings"
+	"sync/atomic"
 	"time"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -36,9 +38,25 @@ type Handler struct {
 // ServeHTTP satisfies the httpserver.Handler interface.
 func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	for _, rule := range h.Rules {
-
-		// First requirement: Base path must match and the path must be allowed.
-		if !httpserver.Path(r.URL.Path).Matches(rule.Path) || !rule.AllowedPath(r.URL.Path) {
+		// First requirement: Base path must match request path. If it doesn't,
+		// we check to make sure the leading slash is not missing, and if so,
+		// we check again with it prepended. This is in case people forget
+		// a leading slash when performing rewrites, and we don't want to expose
+		// the contents of the (likely PHP) script. See issue #1645.
+		hpath := httpserver.Path(r.URL.Path)
+		if !hpath.Matches(rule.Path) {
+			if strings.HasPrefix(string(hpath), "/") {
+				// this is a normal-looking path, and it doesn't match; try next rule
+				continue
+			}
+			hpath = httpserver.Path("/" + string(hpath)) // prepend leading slash
+			if !hpath.Matches(rule.Path) {
+				// even after fixing the request path, it still doesn't match; try next rule
+				continue
+			}
+		}
+		// The path must also be allowed (not ignored).
+		if !rule.AllowedPath(r.URL.Path) {
 			continue
 		}
 
@@ -74,16 +92,28 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 			}
 
 			// Connect to FastCGI gateway
-			fcgiBackend, err := rule.dialer.Dial()
+			network, address := parseAddress(rule.Address())
+
+			ctx := context.Background()
+			if rule.ConnectTimeout > 0 {
+				var cancel context.CancelFunc
+				ctx, cancel = context.WithTimeout(ctx, rule.ConnectTimeout)
+				defer cancel()
+			}
+
+			fcgiBackend, err := DialContext(ctx, network, address)
 			if err != nil {
-				if err, ok := err.(net.Error); ok && err.Timeout() {
-					return http.StatusGatewayTimeout, err
-				}
 				return http.StatusBadGateway, err
 			}
 			defer fcgiBackend.Close()
-			fcgiBackend.SetReadTimeout(rule.ReadTimeout)
-			fcgiBackend.SetSendTimeout(rule.SendTimeout)
+
+			// read/write timeouts
+			if err := fcgiBackend.SetReadTimeout(rule.ReadTimeout); err != nil {
+				return http.StatusInternalServerError, err
+			}
+			if err := fcgiBackend.SetSendTimeout(rule.SendTimeout); err != nil {
+				return http.StatusInternalServerError, err
+			}
 
 			var resp *http.Response
 
@@ -105,6 +135,10 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 				resp, err = fcgiBackend.Post(env, r.Method, r.Header.Get("Content-Type"), r.Body, contentLength)
 			}
 
+			if resp != nil && resp.Body != nil {
+				defer resp.Body.Close()
+			}
+
 			if err != nil {
 				if err, ok := err.(net.Error); ok && err.Timeout() {
 					return http.StatusGatewayTimeout, err
@@ -123,9 +157,9 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 			}
 
 			// Log any stderr output from upstream
-			if stderr := fcgiBackend.StdErr(); stderr.Len() != 0 {
+			if fcgiBackend.stderr.Len() != 0 {
 				// Remove trailing newline, error logger already does this.
-				err = LogError(strings.TrimSuffix(stderr.String(), "\n"))
+				err = LogError(strings.TrimSuffix(fcgiBackend.stderr.String(), "\n"))
 			}
 
 			// Normally we would return the status code if it is an error status (>= 400),
@@ -287,8 +321,8 @@ type Rule struct {
 	// The base path to match. Required.
 	Path string
 
-	// The address of the FastCGI server. Required.
-	Address string
+	// upstream load balancer
+	balancer
 
 	// Always process files with this extension with fastcgi.
 	Ext string
@@ -313,14 +347,32 @@ type Rule struct {
 	// Ignored paths
 	IgnoredSubPaths []string
 
+	// The duration used to set a deadline when connecting to an upstream.
+	ConnectTimeout time.Duration
+
 	// The duration used to set a deadline when reading from the FastCGI server.
 	ReadTimeout time.Duration
 
 	// The duration used to set a deadline when sending to the FastCGI server.
 	SendTimeout time.Duration
+}
 
-	// FCGI dialer
-	dialer dialer
+// balancer is a fastcgi upstream load balancer.
+type balancer interface {
+	// Address picks an upstream address from the
+	// underlying load balancer.
+	Address() string
+}
+
+// roundRobin is a round robin balancer for fastcgi upstreams.
+type roundRobin struct {
+	addresses []string
+	index     int64
+}
+
+func (r *roundRobin) Address() string {
+	index := atomic.AddInt64(&r.index, 1) % int64(len(r.addresses))
+	return r.addresses[index]
 }
 
 // canSplit checks if path can split into two based on rule.SplitPath.

caddyhttp/fastcgi/fastcgi_test.go

@@ -29,16 +29,9 @@ func TestServeHTTP(t *testing.T) {
 		w.Write([]byte(body))
 	}))
 
-	network, address := parseAddress(listener.Addr().String())
 	handler := Handler{
-		Next: nil,
-		Rules: []Rule{
-			{
-				Path:    "/",
-				Address: listener.Addr().String(),
-				dialer:  basicDialer{network: network, address: address},
-			},
-		},
+		Next:  nil,
+		Rules: []Rule{{Path: "/", balancer: address(listener.Addr().String())}},
 	}
 	r, err := http.NewRequest("GET", "/", nil)
 	if err != nil {
@@ -62,122 +55,25 @@ func TestServeHTTP(t *testing.T) {
 	}
 }
 
-// connectionCounter in fact is a listener with an added counter to keep track
-// of the number of accepted connections.
-type connectionCounter struct {
-	net.Listener
-	sync.Mutex
-	counter int
-}
-
-func (l *connectionCounter) Accept() (net.Conn, error) {
-	l.Lock()
-	l.counter++
-	l.Unlock()
-	return l.Listener.Accept()
-}
-
-// TestPersistent ensures that persistent
-// as well as the non-persistent fastCGI servers
-// send the answers corresnponding to the correct request.
-// It also checks the number of tcp connections used.
-func TestPersistent(t *testing.T) {
-	numberOfRequests := 32
-
-	for _, poolsize := range []int{0, 1, 5, numberOfRequests} {
-		l, err := net.Listen("tcp", "127.0.0.1:0")
-		if err != nil {
-			t.Fatalf("Unable to create listener for test: %v", err)
-		}
-
-		listener := &connectionCounter{l, *new(sync.Mutex), 0}
-
-		// this fcgi server replies with the request URL
-		go fcgi.Serve(listener, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-			body := "This answers a request to " + r.URL.Path
-			bodyLenStr := strconv.Itoa(len(body))
-
-			w.Header().Set("Content-Length", bodyLenStr)
-			w.Write([]byte(body))
-		}))
-
-		network, address := parseAddress(listener.Addr().String())
-		handler := Handler{
-			Next:  nil,
-			Rules: []Rule{{Path: "/", Address: listener.Addr().String(), dialer: &persistentDialer{size: poolsize, network: network, address: address}}},
-		}
-
-		var semaphore sync.WaitGroup
-		serialMutex := new(sync.Mutex)
-
-		serialCounter := 0
-		parallelCounter := 0
-		// make some serial followed by some
-		// parallel requests to challenge the handler
-		for _, serialize := range []bool{true, false, false, false} {
-			if serialize {
-				serialCounter++
-			} else {
-				parallelCounter++
-			}
-			semaphore.Add(numberOfRequests)
-
-			for i := 0; i < numberOfRequests; i++ {
-				go func(i int, serialize bool) {
-					defer semaphore.Done()
-					if serialize {
-						serialMutex.Lock()
-						defer serialMutex.Unlock()
-					}
-					r, err := http.NewRequest("GET", "/"+strconv.Itoa(i), nil)
-					if err != nil {
-						t.Errorf("Unable to create request: %v", err)
-					}
-					ctx := context.WithValue(r.Context(), httpserver.OriginalURLCtxKey, *r.URL)
-					r = r.WithContext(ctx)
-					w := httptest.NewRecorder()
-
-					status, err := handler.ServeHTTP(w, r)
-
-					if status != 0 {
-						t.Errorf("Handler(pool: %v) return status %v", poolsize, status)
-					}
-					if err != nil {
-						t.Errorf("Handler(pool: %v) Error: %v", poolsize, err)
-					}
-					want := "This answers a request to /" + strconv.Itoa(i)
-					if got := w.Body.String(); got != want {
-						t.Errorf("Expected response from handler(pool: %v) to be '%s', got: '%s'", poolsize, want, got)
-					}
-				}(i, serialize)
-			} //next request
-			semaphore.Wait()
-		} // next set of requests (serial/parallel)
-
-		listener.Close()
-		t.Logf("The pool: %v test used %v tcp connections to answer %v * %v serial and %v * %v parallel requests.", poolsize, listener.counter, serialCounter, numberOfRequests, parallelCounter, numberOfRequests)
-	} // next handler (persistent/non-persistent)
-}
-
 func TestRuleParseAddress(t *testing.T) {
 	getClientTestTable := []struct {
 		rule            *Rule
 		expectednetwork string
 		expectedaddress string
 	}{
-		{&Rule{Address: "tcp://172.17.0.1:9000"}, "tcp", "172.17.0.1:9000"},
-		{&Rule{Address: "fastcgi://localhost:9000"}, "tcp", "localhost:9000"},
-		{&Rule{Address: "172.17.0.15"}, "tcp", "172.17.0.15"},
-		{&Rule{Address: "/my/unix/socket"}, "unix", "/my/unix/socket"},
-		{&Rule{Address: "unix:/second/unix/socket"}, "unix", "/second/unix/socket"},
+		{&Rule{balancer: address("tcp://172.17.0.1:9000")}, "tcp", "172.17.0.1:9000"},
+		{&Rule{balancer: address("fastcgi://localhost:9000")}, "tcp", "localhost:9000"},
+		{&Rule{balancer: address("172.17.0.15")}, "tcp", "172.17.0.15"},
+		{&Rule{balancer: address("/my/unix/socket")}, "unix", "/my/unix/socket"},
+		{&Rule{balancer: address("unix:/second/unix/socket")}, "unix", "/second/unix/socket"},
 	}
 
 	for _, entry := range getClientTestTable {
-		if actualnetwork, _ := parseAddress(entry.rule.Address); actualnetwork != entry.expectednetwork {
-			t.Errorf("Unexpected network for address string %v. Got %v, expected %v", entry.rule.Address, actualnetwork, entry.expectednetwork)
+		if actualnetwork, _ := parseAddress(entry.rule.Address()); actualnetwork != entry.expectednetwork {
+			t.Errorf("Unexpected network for address string %v. Got %v, expected %v", entry.rule.Address(), actualnetwork, entry.expectednetwork)
 		}
-		if _, actualaddress := parseAddress(entry.rule.Address); actualaddress != entry.expectedaddress {
-			t.Errorf("Unexpected parsed address for address string %v. Got %v, expected %v", entry.rule.Address, actualaddress, entry.expectedaddress)
+		if _, actualaddress := parseAddress(entry.rule.Address()); actualaddress != entry.expectedaddress {
+			t.Errorf("Unexpected parsed address for address string %v. Got %v, expected %v", entry.rule.Address(), actualaddress, entry.expectedaddress)
 		}
 	}
 }
@@ -332,14 +228,12 @@ func TestReadTimeout(t *testing.T) {
 		}
 		defer listener.Close()
 
-		network, address := parseAddress(listener.Addr().String())
 		handler := Handler{
 			Next: nil,
 			Rules: []Rule{
 				{
 					Path:        "/",
-					Address:     listener.Addr().String(),
-					dialer:      basicDialer{network: network, address: address},
+					balancer:    address(listener.Addr().String()),
 					ReadTimeout: test.readTimeout,
 				},
 			},
@@ -394,14 +288,12 @@ func TestSendTimeout(t *testing.T) {
 		}
 		defer listener.Close()
 
-		network, address := parseAddress(listener.Addr().String())
 		handler := Handler{
 			Next: nil,
 			Rules: []Rule{
 				{
 					Path:        "/",
-					Address:     listener.Addr().String(),
-					dialer:      basicDialer{network: network, address: address},
+					balancer:    address(listener.Addr().String()),
 					SendTimeout: test.sendTimeout,
 				},
 			},
@@ -434,3 +326,28 @@ func TestSendTimeout(t *testing.T) {
 		}
 	}
 }
+
+func TestBalancer(t *testing.T) {
+	tests := [][]string{
+		{"localhost", "host.local"},
+		{"localhost"},
+		{"localhost", "host.local", "example.com"},
+		{"localhost", "host.local", "example.com", "127.0.0.1"},
+	}
+	for i, test := range tests {
+		b := address(test...)
+		for _, host := range test {
+			a := b.Address()
+			if a != host {
+				t.Errorf("Test %d: expected %s, found %s", i, host, a)
+			}
+		}
+	}
+}
+
+func address(addresses ...string) balancer {
+	return &roundRobin{
+		addresses: addresses,
+		index:     -1,
+	}
+}

caddyhttp/fastcgi/fcgiclient.go

@@ -13,6 +13,7 @@ package fastcgi
 import (
 	"bufio"
 	"bytes"
+	"context"
 	"encoding/binary"
 	"errors"
 	"io"
@@ -107,18 +108,6 @@ const (
 	maxPad   = 255
 )
 
-// Client interface
-type Client interface {
-	Get(pair map[string]string) (response *http.Response, err error)
-	Head(pair map[string]string) (response *http.Response, err error)
-	Options(pairs map[string]string) (response *http.Response, err error)
-	Post(pairs map[string]string, method string, bodyType string, body io.Reader, contentLength int64) (response *http.Response, err error)
-	Close() error
-	StdErr() bytes.Buffer
-	SetReadTimeout(time.Duration) error
-	SetSendTimeout(time.Duration) error
-}
-
 type header struct {
 	Version       uint8
 	Type          uint8
@@ -150,7 +139,7 @@ func (rec *record) read(r io.Reader) (buf []byte, err error) {
 		return
 	}
 	if rec.h.Version != 1 {
-		err = errInvalidHeaderVersion
+		err = errors.New("fcgi: invalid header version")
 		return
 	}
 	if rec.h.Type == EndRequest {
@@ -173,7 +162,7 @@ func (rec *record) read(r io.Reader) (buf []byte, err error) {
 // interfacing external applications with Web servers.
 type FCGIClient struct {
 	mutex       sync.Mutex
-	conn        net.Conn
+	rwc         io.ReadWriteCloser
 	h           header
 	buf         bytes.Buffer
 	stderr      bytes.Buffer
@@ -183,53 +172,57 @@ type FCGIClient struct {
 	sendTimeout time.Duration
 }
 
-// DialTimeout connects to the fcgi responder at the specified network address, using default net.Dialer.
+// DialWithDialerContext connects to the fcgi responder at the specified network address, using custom net.Dialer
+// and a context.
 // See func net.Dial for a description of the network and address parameters.
-func DialTimeout(network string, address string, timeout time.Duration) (fcgi *FCGIClient, err error) {
-	conn, err := net.DialTimeout(network, address, timeout)
+func DialWithDialerContext(ctx context.Context, network, address string, dialer net.Dialer) (fcgi *FCGIClient, err error) {
+	var conn net.Conn
+	conn, err = dialer.DialContext(ctx, network, address)
 	if err != nil {
 		return
 	}
 
-	fcgi = &FCGIClient{conn: conn, keepAlive: false, reqID: 1}
+	fcgi = &FCGIClient{
+		rwc:       conn,
+		keepAlive: false,
+		reqID:     1,
+	}
 
-	return fcgi, nil
+	return
 }
 
-// Close closes fcgi connnection.
-func (c *FCGIClient) Close() error {
-	return c.conn.Close()
+// DialContext is like Dial but passes ctx to dialer.Dial.
+func DialContext(ctx context.Context, network, address string) (fcgi *FCGIClient, err error) {
+	return DialWithDialerContext(ctx, network, address, net.Dialer{})
 }
 
-func (c *FCGIClient) writeRecord(recType uint8, content []byte) error {
+// Dial connects to the fcgi responder at the specified network address, using default net.Dialer.
+// See func net.Dial for a description of the network and address parameters.
+func Dial(network, address string) (fcgi *FCGIClient, err error) {
+	return DialContext(context.Background(), network, address)
+}
+
+// Close closes fcgi connnection
+func (c *FCGIClient) Close() {
+	c.rwc.Close()
+}
+
+func (c *FCGIClient) writeRecord(recType uint8, content []byte) (err error) {
 	c.mutex.Lock()
 	defer c.mutex.Unlock()
 	c.buf.Reset()
 	c.h.init(recType, c.reqID, len(content))
-
 	if err := binary.Write(&c.buf, binary.BigEndian, c.h); err != nil {
 		return err
 	}
-
 	if _, err := c.buf.Write(content); err != nil {
 		return err
 	}
-
 	if _, err := c.buf.Write(pad[:c.h.PaddingLength]); err != nil {
 		return err
 	}
-
-	if c.sendTimeout != 0 {
-		if err := c.conn.SetWriteDeadline(time.Now().Add(c.sendTimeout)); err != nil {
-			return err
-		}
-	}
-
-	if _, err := c.conn.Write(c.buf.Bytes()); err != nil {
-		return err
-	}
-
-	return nil
+	_, err = c.rwc.Write(c.buf.Bytes())
+	return err
 }
 
 func (c *FCGIClient) writeBeginRequest(role uint16, flags uint8) error {
@@ -345,14 +338,13 @@ func (w *streamReader) Read(p []byte) (n int, err error) {
 
 	if len(p) > 0 {
 		if len(w.buf) == 0 {
+
 			// filter outputs for error log
 			for {
 				rec := &record{}
 				var buf []byte
-				buf, err = rec.read(w.c.conn)
-				if err == errInvalidHeaderVersion {
-					continue
-				} else if err != nil {
+				buf, err = rec.read(w.c.rwc)
+				if err != nil {
 					return
 				}
 				// standard error output
@@ -376,15 +368,10 @@ func (w *streamReader) Read(p []byte) (n int, err error) {
 	return
 }
 
-// StdErr returns stderr stream
-func (c *FCGIClient) StdErr() bytes.Buffer {
-	return c.stderr
-}
-
 // Do made the request and returns a io.Reader that translates the data read
 // from fcgi responder out of fcgi packet before returning it.
 func (c *FCGIClient) Do(p map[string]string, req io.Reader) (r io.Reader, err error) {
-	err = c.writeBeginRequest(uint16(Responder), FCGIKeepConn)
+	err = c.writeBeginRequest(uint16(Responder), 0)
 	if err != nil {
 		return
 	}
@@ -407,11 +394,11 @@ func (c *FCGIClient) Do(p map[string]string, req io.Reader) (r io.Reader, err er
 // clientCloser is a io.ReadCloser. It wraps a io.Reader with a Closer
 // that closes FCGIClient connection.
 type clientCloser struct {
-	f *FCGIClient
+	*FCGIClient
 	io.Reader
 }
 
-func (c clientCloser) Close() error { return c.f.Close() }
+func (f clientCloser) Close() error { return f.rwc.Close() }
 
 // Request returns a HTTP Response with Header and Body
 // from fcgi responder
@@ -425,12 +412,6 @@ func (c *FCGIClient) Request(p map[string]string, req io.Reader) (resp *http.Res
 	tp := textproto.NewReader(rb)
 	resp = new(http.Response)
 
-	if c.readTimeout != 0 {
-		if err = c.conn.SetReadDeadline(time.Now().Add(c.readTimeout)); err != nil {
-			return
-		}
-	}
-
 	// Parse the response headers.
 	mimeHeader, err := tp.ReadMIMEHeader()
 	if err != nil && err != io.EOF {
@@ -566,18 +547,20 @@ func (c *FCGIClient) PostFile(p map[string]string, data url.Values, file map[str
 // SetReadTimeout sets the read timeout for future calls that read from the
 // fcgi responder. A zero value for t means no timeout will be set.
 func (c *FCGIClient) SetReadTimeout(t time.Duration) error {
-	c.readTimeout = t
+	if conn, ok := c.rwc.(net.Conn); ok && t != 0 {
+		return conn.SetReadDeadline(time.Now().Add(t))
+	}
 	return nil
 }
 
 // SetSendTimeout sets the read timeout for future calls that send data to
 // the fcgi responder. A zero value for t means no timeout will be set.
 func (c *FCGIClient) SetSendTimeout(t time.Duration) error {
-	c.sendTimeout = t
+	if conn, ok := c.rwc.(net.Conn); ok && t != 0 {
+		return conn.SetWriteDeadline(time.Now().Add(t))
+	}
 	return nil
 }
 
 // Checks whether chunked is part of the encodings stack
 func chunked(te []string) bool { return len(te) > 0 && te[0] == "chunked" }
-
-var errInvalidHeaderVersion = errors.New("fcgi: invalid header version")

caddyhttp/fastcgi/fcgiclient_test.go

@@ -103,7 +103,7 @@ func (s FastCGIServer) ServeHTTP(resp http.ResponseWriter, req *http.Request) {
 }
 
 func sendFcgi(reqType int, fcgiParams map[string]string, data []byte, posts map[string]string, files map[string]string) (content []byte) {
-	fcgi, err := DialTimeout("tcp", ipPort, 0)
+	fcgi, err := Dial("tcp", ipPort)
 	if err != nil {
 		log.Println("err:", err)
 		return
@@ -155,7 +155,7 @@ func sendFcgi(reqType int, fcgiParams map[string]string, data []byte, posts map[
 	fcgi.Close()
 	time.Sleep(1 * time.Second)
 
-	if bytes.Contains(content, []byte("FAILED")) {
+	if bytes.Index(content, []byte("FAILED")) >= 0 {
 		globalt.Error("Server return failed message")
 	}
 

caddyhttp/fastcgi/setup.go

@@ -4,8 +4,6 @@ import (
 	"errors"
 	"net/http"
 	"path/filepath"
-	"strconv"
-	"strings"
 	"time"
 
 	"github.com/mholt/caddy"
@@ -61,10 +59,8 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 		}
 
 		rule := Rule{
-			Root:        absRoot,
-			Path:        args[0],
-			ReadTimeout: 60 * time.Second,
-			SendTimeout: 60 * time.Second,
+			Root: absRoot,
+			Path: args[0],
 		}
 		upstreams := []string{args[1]}
 
@@ -75,10 +71,6 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 		}
 
 		var err error
-		var pool int
-		var connectTimeout = 60 * time.Second
-		var dialers []dialer
-		var poolSize = -1
 
 		for c.NextBlock() {
 			switch c.Val() {
@@ -126,24 +118,11 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 				}
 				rule.IgnoredSubPaths = ignoredPaths
 
-			case "pool":
-				if !c.NextArg() {
-					return rules, c.ArgErr()
-				}
-				pool, err = strconv.Atoi(c.Val())
-				if err != nil {
-					return rules, err
-				}
-				if pool >= 0 {
-					poolSize = pool
-				} else {
-					return rules, c.Errf("positive integer expected, found %d", pool)
-				}
 			case "connect_timeout":
 				if !c.NextArg() {
 					return rules, c.ArgErr()
 				}
-				connectTimeout, err = time.ParseDuration(c.Val())
+				rule.ConnectTimeout, err = time.ParseDuration(c.Val())
 				if err != nil {
 					return rules, err
 				}
@@ -168,29 +147,10 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 			}
 		}
 
-		for _, rawAddress := range upstreams {
-			network, address := parseAddress(rawAddress)
-			if poolSize >= 0 {
-				dialers = append(dialers, &persistentDialer{
-					size:    poolSize,
-					network: network,
-					address: address,
-					timeout: connectTimeout,
-				})
-			} else {
-				dialers = append(dialers, basicDialer{
-					network: network,
-					address: address,
-					timeout: connectTimeout,
-				})
-			}
-		}
+		rule.balancer = &roundRobin{addresses: upstreams, index: -1}
 
-		rule.dialer = &loadBalancingDialer{dialers: dialers}
-		rule.Address = strings.Join(upstreams, ",")
 		rules = append(rules, rule)
 	}
-
 	return rules, nil
 }
 

caddyhttp/fastcgi/setup_test.go

@@ -2,10 +2,7 @@ package fastcgi
 
 import (
 	"fmt"
-	"os"
-	"reflect"
 	"testing"
-	"time"
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -32,45 +29,13 @@ func TestSetup(t *testing.T) {
 	if myHandler.Rules[0].Path != "/" {
 		t.Errorf("Expected / as the Path")
 	}
-	if myHandler.Rules[0].Address != "127.0.0.1:9000" {
+	if myHandler.Rules[0].Address() != "127.0.0.1:9000" {
 		t.Errorf("Expected 127.0.0.1:9000 as the Address")
 	}
 
 }
 
-func (p *persistentDialer) Equals(q *persistentDialer) bool {
-	if p.size != q.size {
-		return false
-	}
-	if p.network != q.network {
-		return false
-	}
-	if p.address != q.address {
-		return false
-	}
-
-	if len(p.pool) != len(q.pool) {
-		return false
-	}
-	for i, client := range p.pool {
-		if client != q.pool[i] {
-			return false
-		}
-	}
-	// ignore mutex state
-	return true
-}
-
 func TestFastcgiParse(t *testing.T) {
-	rootPath, err := os.Getwd()
-	if err != nil {
-		t.Errorf("Can't determine current working directory; got '%v'", err)
-	}
-
-	defaultAddress := "127.0.0.1:9001"
-	network, address := parseAddress(defaultAddress)
-	t.Logf("Address '%v' was parsed to network '%v' and address '%v'", defaultAddress, network, address)
-
 	tests := []struct {
 		inputFastcgiConfig    string
 		shouldErr             bool
@@ -79,193 +44,34 @@ func TestFastcgiParse(t *testing.T) {
 
 		{`fastcgi /blog 127.0.0.1:9000 php`,
 			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000",
-				Ext:         ".php",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{"index.php"},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
+				Path:       "/blog",
+				balancer:   &roundRobin{addresses: []string{"127.0.0.1:9000"}},
+				Ext:        ".php",
+				SplitPath:  ".php",
+				IndexFiles: []string{"index.php"},
 			}}},
-		{`fastcgi /blog 127.0.0.1:9000 php {
-			root /tmp
-		}`,
-			false, []Rule{{
-				Root:        "/tmp",
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000",
-				Ext:         ".php",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{"index.php"},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi /blog 127.0.0.1:9000 php {
-			upstream 127.0.0.1:9001
-		}`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000,127.0.0.1:9001",
-				Ext:         ".php",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}, basicDialer{network: "tcp", address: "127.0.0.1:9001", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{"index.php"},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi /blog 127.0.0.1:9000 {
-			upstream 127.0.0.1:9001 
-		}`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/blog",
-				Address:     "127.0.0.1:9000,127.0.0.1:9001",
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}, basicDialer{network: "tcp", address: "127.0.0.1:9001", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / ` + defaultAddress + ` {
+		{`fastcgi / 127.0.0.1:9001 {
 	              split .html
 	              }`,
 			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   ".html",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
+				Path:       "/",
+				balancer:   &roundRobin{addresses: []string{"127.0.0.1:9001"}},
+				Ext:        "",
+				SplitPath:  ".html",
+				IndexFiles: []string{},
 			}}},
-		{`fastcgi / ` + defaultAddress + ` {
+		{`fastcgi / 127.0.0.1:9001 {
 	              split .html
 	              except /admin /user
 	              }`,
 			false, []Rule{{
-				Root:            rootPath,
 				Path:            "/",
-				Address:         "127.0.0.1:9001",
+				balancer:        &roundRobin{addresses: []string{"127.0.0.1:9001"}},
 				Ext:             "",
 				SplitPath:       ".html",
-				dialer:          &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
 				IndexFiles:      []string{},
 				IgnoredSubPaths: []string{"/admin", "/user"},
-				ReadTimeout:     60 * time.Second,
-				SendTimeout:     60 * time.Second,
 			}}},
-		{`fastcgi / ` + defaultAddress + ` {
-	              pool 0
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{&persistentDialer{size: 0, network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / 127.0.0.1:8080  {
-			upstream 127.0.0.1:9000
-	              pool 5
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     "127.0.0.1:8080,127.0.0.1:9000",
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{&persistentDialer{size: 5, network: "tcp", address: "127.0.0.1:8080", timeout: 60 * time.Second}, &persistentDialer{size: 5, network: "tcp", address: "127.0.0.1:9000", timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / ` + defaultAddress + ` {
-	              split .php
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   ".php",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{`fastcgi / ` + defaultAddress + ` {
-	              connect_timeout 5s
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 5 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{
-			`fastcgi / ` + defaultAddress + ` { connect_timeout BADVALUE }`,
-			true,
-			[]Rule{},
-		},
-		{`fastcgi / ` + defaultAddress + ` {
-	              read_timeout 5s
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 5 * time.Second,
-				SendTimeout: 60 * time.Second,
-			}}},
-		{
-			`fastcgi / ` + defaultAddress + ` { read_timeout BADVALUE }`,
-			true,
-			[]Rule{},
-		},
-		{`fastcgi / ` + defaultAddress + ` {
-	              send_timeout 5s
-	              }`,
-			false, []Rule{{
-				Root:        rootPath,
-				Path:        "/",
-				Address:     defaultAddress,
-				Ext:         "",
-				SplitPath:   "",
-				dialer:      &loadBalancingDialer{dialers: []dialer{basicDialer{network: network, address: address, timeout: 60 * time.Second}}},
-				IndexFiles:  []string{},
-				ReadTimeout: 60 * time.Second,
-				SendTimeout: 5 * time.Second,
-			}}},
-		{
-			`fastcgi / ` + defaultAddress + ` { send_timeout BADVALUE }`,
-			true,
-			[]Rule{},
-		},
-		{`fastcgi / {
-
-		              }`,
-			true, []Rule{},
-		},
 	}
 	for i, test := range tests {
 		actualFastcgiConfigs, err := fastcgiParse(caddy.NewTestController("http", test.inputFastcgiConfig))
@@ -281,19 +87,14 @@ func TestFastcgiParse(t *testing.T) {
 		}
 		for j, actualFastcgiConfig := range actualFastcgiConfigs {
 
-			if actualFastcgiConfig.Root != test.expectedFastcgiConfig[j].Root {
-				t.Errorf("Test %d expected %dth FastCGI Root to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].Root, actualFastcgiConfig.Root)
-			}
-
 			if actualFastcgiConfig.Path != test.expectedFastcgiConfig[j].Path {
 				t.Errorf("Test %d expected %dth FastCGI Path to be  %s  , but got %s",
 					i, j, test.expectedFastcgiConfig[j].Path, actualFastcgiConfig.Path)
 			}
 
-			if actualFastcgiConfig.Address != test.expectedFastcgiConfig[j].Address {
+			if actualFastcgiConfig.Address() != test.expectedFastcgiConfig[j].Address() {
 				t.Errorf("Test %d expected %dth FastCGI Address to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].Address, actualFastcgiConfig.Address)
+					i, j, test.expectedFastcgiConfig[j].Address(), actualFastcgiConfig.Address())
 			}
 
 			if actualFastcgiConfig.Ext != test.expectedFastcgiConfig[j].Ext {
@@ -306,16 +107,6 @@ func TestFastcgiParse(t *testing.T) {
 					i, j, test.expectedFastcgiConfig[j].SplitPath, actualFastcgiConfig.SplitPath)
 			}
 
-			if reflect.TypeOf(actualFastcgiConfig.dialer) != reflect.TypeOf(test.expectedFastcgiConfig[j].dialer) {
-				t.Errorf("Test %d expected %dth FastCGI dialer to be of type %T, but got %T",
-					i, j, test.expectedFastcgiConfig[j].dialer, actualFastcgiConfig.dialer)
-			} else {
-				if !areDialersEqual(actualFastcgiConfig.dialer, test.expectedFastcgiConfig[j].dialer, t) {
-					t.Errorf("Test %d expected %dth FastCGI dialer to be %v, but got %v",
-						i, j, test.expectedFastcgiConfig[j].dialer, actualFastcgiConfig.dialer)
-				}
-			}
-
 			if fmt.Sprint(actualFastcgiConfig.IndexFiles) != fmt.Sprint(test.expectedFastcgiConfig[j].IndexFiles) {
 				t.Errorf("Test %d expected %dth FastCGI IndexFiles to be  %s  , but got %s",
 					i, j, test.expectedFastcgiConfig[j].IndexFiles, actualFastcgiConfig.IndexFiles)
@@ -325,43 +116,7 @@ func TestFastcgiParse(t *testing.T) {
 				t.Errorf("Test %d expected %dth FastCGI IgnoredSubPaths to be  %s  , but got %s",
 					i, j, test.expectedFastcgiConfig[j].IgnoredSubPaths, actualFastcgiConfig.IgnoredSubPaths)
 			}
-
-			if fmt.Sprint(actualFastcgiConfig.ReadTimeout) != fmt.Sprint(test.expectedFastcgiConfig[j].ReadTimeout) {
-				t.Errorf("Test %d expected %dth FastCGI ReadTimeout to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].ReadTimeout, actualFastcgiConfig.ReadTimeout)
-			}
-
-			if fmt.Sprint(actualFastcgiConfig.SendTimeout) != fmt.Sprint(test.expectedFastcgiConfig[j].SendTimeout) {
-				t.Errorf("Test %d expected %dth FastCGI SendTimeout to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].SendTimeout, actualFastcgiConfig.SendTimeout)
-			}
 		}
 	}
-}
-
-func areDialersEqual(current, expected dialer, t *testing.T) bool {
-
-	switch actual := current.(type) {
-	case *loadBalancingDialer:
-		if expected, ok := expected.(*loadBalancingDialer); ok {
-			for i := 0; i < len(actual.dialers); i++ {
-				if !areDialersEqual(actual.dialers[i], expected.dialers[i], t) {
-					return false
-				}
-			}
-
-			return true
-		}
-	case basicDialer:
-		return current == expected
-	case *persistentDialer:
-		if expected, ok := expected.(*persistentDialer); ok {
-			return actual.Equals(expected)
-		}
-
-	default:
-		t.Errorf("Unknown dialer type %T", current)
-	}
-
-	return false
+
 }

caddyhttp/gzip/gzip.go

@@ -3,9 +3,7 @@
 package gzip
 
 import (
-	"bufio"
 	"io"
-	"net"
 	"net/http"
 	"strings"
 
@@ -58,7 +56,10 @@ outer:
 		// original form.
 		gzipWriter := getWriter(c.Level)
 		defer putWriter(c.Level, gzipWriter)
-		gz := &gzipResponseWriter{Writer: gzipWriter, ResponseWriter: w}
+		gz := &gzipResponseWriter{
+			Writer:                gzipWriter,
+			ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
+		}
 
 		var rw http.ResponseWriter
 		// if no response filter is used
@@ -92,7 +93,7 @@ outer:
 // with a gzip.Writer to compress the output.
 type gzipResponseWriter struct {
 	io.Writer
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 	statusCodeWritten bool
 }
 
@@ -104,7 +105,11 @@ func (w *gzipResponseWriter) WriteHeader(code int) {
 	w.Header().Del("Content-Length")
 	w.Header().Set("Content-Encoding", "gzip")
 	w.Header().Add("Vary", "Accept-Encoding")
-	w.ResponseWriter.WriteHeader(code)
+	originalEtag := w.Header().Get("ETag")
+	if originalEtag != "" && !strings.HasPrefix(originalEtag, "W/") {
+		w.Header().Set("ETag", "W/"+originalEtag)
+	}
+	w.ResponseWriterWrapper.WriteHeader(code)
 	w.statusCodeWritten = true
 }
 
@@ -120,44 +125,5 @@ func (w *gzipResponseWriter) Write(b []byte) (int, error) {
 	return n, err
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (w *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := w.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: w.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (w *gzipResponseWriter) Flush() {
-	if f, ok := w.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: w.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (w *gzipResponseWriter) CloseNotify() <-chan bool {
-	if cn, ok := w.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: w.ResponseWriter})
-}
-
-func (w *gzipResponseWriter) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := w.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return httpserver.NonFlusherError{Underlying: w.ResponseWriter}
-}
-
 // Interface guards
-var _ http.Pusher = (*gzipResponseWriter)(nil)
-var _ http.Flusher = (*gzipResponseWriter)(nil)
-var _ http.CloseNotifier = (*gzipResponseWriter)(nil)
-var _ http.Hijacker = (*gzipResponseWriter)(nil)
+var _ httpserver.HTTPInterfaces = (*gzipResponseWriter)(nil)

caddyhttp/gzip/gzip_test.go

@@ -38,6 +38,14 @@ func TestGzipHandler(t *testing.T) {
 			t.Error(err)
 		}
 		r.Header.Set("Accept-Encoding", "gzip")
+		w.Header().Set("ETag", `"2n9cd"`)
+		_, err = gz.ServeHTTP(w, r)
+		if err != nil {
+			t.Error(err)
+		}
+
+		// The second pass, test if the ETag is already weak
+		w.Header().Set("ETag", `W/"2n9cd"`)
 		_, err = gz.ServeHTTP(w, r)
 		if err != nil {
 			t.Error(err)
@@ -109,10 +117,14 @@ func nextFunc(shouldGzip bool) httpserver.Handler {
 
 		if shouldGzip {
 			if w.Header().Get("Content-Encoding") != "gzip" {
-				return 0, fmt.Errorf("Content-Encoding must be gzip, found %v", r.Header.Get("Content-Encoding"))
+				return 0, fmt.Errorf("Content-Encoding must be gzip, found %v", w.Header().Get("Content-Encoding"))
 			}
 			if w.Header().Get("Vary") != "Accept-Encoding" {
-				return 0, fmt.Errorf("Vary must be Accept-Encoding, found %v", r.Header.Get("Vary"))
+				return 0, fmt.Errorf("Vary must be Accept-Encoding, found %v", w.Header().Get("Vary"))
+			}
+			etag := w.Header().Get("ETag")
+			if etag != "" && etag != `W/"2n9cd"` {
+				return 0, fmt.Errorf("ETag must be converted to weak Etag, found %v", w.Header().Get("ETag"))
 			}
 			if _, ok := w.(*gzipResponseWriter); !ok {
 				return 0, fmt.Errorf("ResponseWriter should be gzipResponseWriter, found %T", w)

caddyhttp/gzip/responsefilter_test.go

@@ -33,7 +33,7 @@ func TestLengthFilter(t *testing.T) {
 		for j, filter := range filters {
 			r := httptest.NewRecorder()
 			r.Header().Set("Content-Length", fmt.Sprint(ts.length))
-			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), r, false})
+			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), &httpserver.ResponseWriterWrapper{ResponseWriter: r}, false})
 			if filter.ShouldCompress(wWriter) != ts.shouldCompress[j] {
 				t.Errorf("Test %v: Expected %v found %v", i, ts.shouldCompress[j], filter.ShouldCompress(r))
 			}

caddyhttp/gzip/setup.go

@@ -146,11 +146,7 @@ func initWriterPool() {
 
 	// add default writer pool
 	defaultWriterPoolIndex = i
-	writerPool[defaultWriterPoolIndex] = &sync.Pool{
-		New: func() interface{} {
-			return gzip.NewWriter(ioutil.Discard)
-		},
-	}
+	writerPool[defaultWriterPoolIndex] = newWriterPool(gzip.DefaultCompression)
 }
 
 func getWriter(level int) *gzip.Writer {

caddyhttp/header/header.go

@@ -4,8 +4,6 @@
 package header
 
 import (
-	"bufio"
-	"net"
 	"net/http"
 	"strings"
 
@@ -23,7 +21,9 @@ type Headers struct {
 // setting headers on the response according to the configured rules.
 func (h Headers) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	replacer := httpserver.NewReplacer(r, nil, "")
-	rww := &responseWriterWrapper{ResponseWriter: w}
+	rww := &responseWriterWrapper{
+		ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
+	}
 	for _, rule := range h.Rules {
 		if httpserver.Path(r.URL.Path).Matches(rule.Path) {
 			for name := range rule.Headers {
@@ -62,20 +62,20 @@ type headerOperation func(http.Header)
 // responseWriterWrapper wraps the real ResponseWriter.
 // It defers header operations until writeHeader
 type responseWriterWrapper struct {
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 	ops         []headerOperation
 	wroteHeader bool
 }
 
 func (rww *responseWriterWrapper) Header() http.Header {
-	return rww.ResponseWriter.Header()
+	return rww.ResponseWriterWrapper.Header()
 }
 
 func (rww *responseWriterWrapper) Write(d []byte) (int, error) {
 	if !rww.wroteHeader {
 		rww.WriteHeader(http.StatusOK)
 	}
-	return rww.ResponseWriter.Write(d)
+	return rww.ResponseWriterWrapper.Write(d)
 }
 
 func (rww *responseWriterWrapper) WriteHeader(status int) {
@@ -91,7 +91,7 @@ func (rww *responseWriterWrapper) WriteHeader(status int) {
 		op(h)
 	}
 
-	rww.ResponseWriter.WriteHeader(status)
+	rww.ResponseWriterWrapper.WriteHeader(status)
 }
 
 // delHeader deletes the existing header according to the key
@@ -106,45 +106,5 @@ func (rww *responseWriterWrapper) delHeader(key string) {
 	})
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (rww *responseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := rww.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: rww.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (rww *responseWriterWrapper) Flush() {
-	if f, ok := rww.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: rww.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-// It panics if the underlying ResponseWriter is not a CloseNotifier.
-func (rww *responseWriterWrapper) CloseNotify() <-chan bool {
-	if cn, ok := rww.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: rww.ResponseWriter})
-}
-
-func (rww *responseWriterWrapper) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := rww.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return httpserver.NonPusherError{Underlying: rww.ResponseWriter}
-}
-
 // Interface guards
-var _ http.Pusher = (*responseWriterWrapper)(nil)
-var _ http.Flusher = (*responseWriterWrapper)(nil)
-var _ http.CloseNotifier = (*responseWriterWrapper)(nil)
-var _ http.Hijacker = (*responseWriterWrapper)(nil)
+var _ httpserver.HTTPInterfaces = (*responseWriterWrapper)(nil)

caddyhttp/httpserver/condition.go

@@ -48,11 +48,9 @@ const (
 	isOp         = "is"
 	notOp        = "not"
 	hasOp        = "has"
-	notHasOp     = "not_has"
 	startsWithOp = "starts_with"
 	endsWithOp   = "ends_with"
 	matchOp      = "match"
-	notMatchOp   = "not_match"
 )
 
 func operatorError(operator string) error {
@@ -66,11 +64,9 @@ var ifConditions = map[string]ifCondition{
 	isOp:         isFunc,
 	notOp:        notFunc,
 	hasOp:        hasFunc,
-	notHasOp:     notHasFunc,
 	startsWithOp: startsWithFunc,
 	endsWithOp:   endsWithFunc,
 	matchOp:      matchFunc,
-	notMatchOp:   notMatchFunc,
 }
 
 // isFunc is condition for Is operator.
@@ -82,7 +78,7 @@ func isFunc(a, b string) bool {
 // notFunc is condition for Not operator.
 // It checks for inequality.
 func notFunc(a, b string) bool {
-	return a != b
+	return !isFunc(a, b)
 }
 
 // hasFunc is condition for Has operator.
@@ -91,12 +87,6 @@ func hasFunc(a, b string) bool {
 	return strings.Contains(a, b)
 }
 
-// notHasFunc is condition for NotHas operator.
-// It checks if b is not a substring of a.
-func notHasFunc(a, b string) bool {
-	return !strings.Contains(a, b)
-}
-
 // startsWithFunc is condition for StartsWith operator.
 // It checks if b is a prefix of a.
 func startsWithFunc(a, b string) bool {
@@ -117,30 +107,29 @@ func matchFunc(a, b string) bool {
 	return matched
 }
 
-// notMatchFunc is condition for NotMatch operator.
-// It does regexp matching of a against pattern in b
-// and returns if they do not match.
-func notMatchFunc(a, b string) bool {
-	matched, _ := regexp.MatchString(b, a)
-	return !matched
-}
-
 // ifCond is statement for a IfMatcher condition.
 type ifCond struct {
-	a  string
-	op string
-	b  string
+	a   string
+	op  string
+	b   string
+	neg bool
 }
 
 // newIfCond creates a new If condition.
 func newIfCond(a, operator, b string) (ifCond, error) {
+	neg := false
+	if strings.HasPrefix(operator, "not_") {
+		neg = true
+		operator = operator[4:]
+	}
 	if _, ok := ifConditions[operator]; !ok {
 		return ifCond{}, operatorError(operator)
 	}
 	return ifCond{
-		a:  a,
-		op: operator,
-		b:  b,
+		a:   a,
+		op:  operator,
+		b:   b,
+		neg: neg,
 	}, nil
 }
 
@@ -154,9 +143,12 @@ func (i ifCond) True(r *http.Request) bool {
 			a = replacer.Replace(i.a)
 			b = replacer.Replace(i.b)
 		}
+		if i.neg {
+			return !c(a, b)
+		}
 		return c(a, b)
 	}
-	return false
+	return i.neg // false if not negated, true otherwise
 }
 
 // IfMatcher is a RequestMatcher for 'if' conditions.

caddyhttp/httpserver/condition_test.go

@@ -32,9 +32,15 @@ func TestConditions(t *testing.T) {
 		{"bab starts_with bb", false},
 		{"bab starts_with ba", true},
 		{"bab starts_with bab", true},
+		{"bab not_starts_with bb", true},
+		{"bab not_starts_with ba", false},
+		{"bab not_starts_with bab", false},
 		{"bab ends_with bb", false},
 		{"bab ends_with bab", true},
 		{"bab ends_with ab", true},
+		{"bab not_ends_with bb", true},
+		{"bab not_ends_with ab", false},
+		{"bab not_ends_with bab", false},
 		{"a match *", false},
 		{"a match a", true},
 		{"a match .*", true},
@@ -195,7 +201,7 @@ func TestSetupIfMatcher(t *testing.T) {
 			if	a match b
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "a", op: "match", b: "b"},
+				{a: "a", op: "match", b: "b", neg: false},
 			},
 		}},
 		{`test {
@@ -203,7 +209,7 @@ func TestSetupIfMatcher(t *testing.T) {
 			if_op or
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "a", op: "match", b: "b"},
+				{a: "a", op: "match", b: "b", neg: false},
 			},
 			isOr: true,
 		}},
@@ -221,26 +227,26 @@ func TestSetupIfMatcher(t *testing.T) {
 		},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "goal", op: "has", b: "go"},
-				{a: "cook", op: "not_has", b: "go"},
+				{a: "goal", op: "has", b: "go", neg: false},
+				{a: "cook", op: "has", b: "go", neg: true},
 			},
 		}},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 			if_op and
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "goal", op: "has", b: "go"},
-				{a: "cook", op: "not_has", b: "go"},
+				{a: "goal", op: "has", b: "go", neg: false},
+				{a: "cook", op: "has", b: "go", neg: true},
 			},
 		}},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 			if_op not
 		 }`, true, IfMatcher{},
 		},

caddyhttp/httpserver/logger_test.go

@@ -12,7 +12,7 @@ import (
 	"sync"
 	"testing"
 
-	"gopkg.in/mcuadros/go-syslog.v2"
+	syslog "gopkg.in/mcuadros/go-syslog.v2"
 	"gopkg.in/mcuadros/go-syslog.v2/format"
 )
 

caddyhttp/httpserver/middleware.go

@@ -205,4 +205,7 @@ const (
 
 	// MitmCtxKey is the key for the result of MITM detection
 	MitmCtxKey caddy.CtxKey = "mitm"
+
+	// RequestIDCtxKey is the key for the U4 UUID value
+	RequestIDCtxKey caddy.CtxKey = "request_id"
 )

caddyhttp/httpserver/mitm.go

@@ -7,6 +7,7 @@ import (
 	"io"
 	"net"
 	"net/http"
+	"strconv"
 	"strings"
 	"sync"
 )
@@ -65,7 +66,16 @@ func (h *tlsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		mitm = !info.looksLikeChrome() && !info.looksLikeSafari()
 	} else if strings.Contains(ua, "Firefox") {
 		checked = true
-		mitm = !info.looksLikeFirefox()
+		if strings.Contains(ua, "Windows") {
+			ver := getVersion(ua, "Firefox")
+			if ver == 45.0 || ver == 52.0 {
+				mitm = !info.looksLikeTor()
+			} else {
+				mitm = !info.looksLikeFirefox()
+			}
+		} else {
+			mitm = !info.looksLikeFirefox()
+		}
 	} else if strings.Contains(ua, "Safari") {
 		checked = true
 		mitm = !info.looksLikeSafari()
@@ -87,6 +97,36 @@ func (h *tlsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	h.next.ServeHTTP(w, r)
 }
 
+// getVersion returns a (possibly simplified) representation of the version string
+// from a UserAgent string. It returns a float, so it can represent major and minor
+// versions; the rest of the version is just tacked on behind the decimal point.
+// The purpose of this is to stay simple while allowing for basic, fast comparisons.
+// If the version for softwareName is not found in ua, -1 is returned.
+func getVersion(ua, softwareName string) float64 {
+	search := softwareName + "/"
+	start := strings.Index(ua, search)
+	if start < 0 {
+		return -1
+	}
+	start += len(search)
+	end := strings.Index(ua[start:], " ")
+	if end < 0 {
+		end = len(ua)
+	} else {
+		end += start
+	}
+	strVer := strings.Replace(ua[start:end], "-", "", -1)
+	firstDot := strings.Index(strVer, ".")
+	if firstDot >= 0 {
+		strVer = strVer[:firstDot+1] + strings.Replace(strVer[firstDot+1:], ".", "", -1)
+	}
+	ver, err := strconv.ParseFloat(strVer, 64)
+	if err != nil {
+		return -1
+	}
+	return ver
+}
+
 // clientHelloConn reads the ClientHello
 // and stores it in the attached listener.
 type clientHelloConn struct {
@@ -323,27 +363,38 @@ func (info rawHelloInfo) looksLikeFirefox() bool {
 	// "To determine whether a Firefox session has been
 	// intercepted, we check for the presence and order
 	// of extensions, cipher suites, elliptic curves,
-	// EC point formats, and handshake compression methods."
+	// EC point formats, and handshake compression methods." (early 2016)
 
 	// We check for the presence and order of the extensions.
-	// Note: Sometimes padding (21) is present, sometimes not.
+	// Note: Sometimes 0x15 (21, padding) is present, sometimes not.
 	// Note: Firefox 51+ does not advertise 0x3374 (13172, NPN).
 	// Note: Firefox doesn't advertise 0x0 (0, SNI) when connecting to IP addresses.
-	requiredExtensionsOrder := []uint16{23, 65281, 10, 11, 35, 16, 5, 65283, 13}
+	// Note: Firefox 55+ doesn't appear to advertise 0xFF03 (65283, short headers). It used to be between 5 and 13.
+	// Note: Firefox on Fedora (or RedHat) doesn't include ECC suites because of patent liability.
+	requiredExtensionsOrder := []uint16{23, 65281, 10, 11, 35, 16, 5, 13}
 	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
 		return false
 	}
 
 	// We check for both presence of curves and their ordering.
-	expectedCurves := []tls.CurveID{29, 23, 24, 25}
-	if len(info.curves) != len(expectedCurves) {
+	requiredCurves := []tls.CurveID{29, 23, 24, 25}
+	if len(info.curves) < len(requiredCurves) {
 		return false
 	}
-	for i := range expectedCurves {
-		if info.curves[i] != expectedCurves[i] {
+	for i := range requiredCurves {
+		if info.curves[i] != requiredCurves[i] {
 			return false
 		}
 	}
+	if len(info.curves) > len(requiredCurves) {
+		// newer Firefox (55 Nightly?) may have additional curves at end of list
+		allowedCurves := []tls.CurveID{256, 257}
+		for i := range allowedCurves {
+			if info.curves[len(requiredCurves)+i] != allowedCurves[i] {
+				return false
+			}
+		}
+	}
 
 	if hasGreaseCiphers(info.cipherSuites) {
 		return false
@@ -353,6 +404,9 @@ func (info rawHelloInfo) looksLikeFirefox() bool {
 	// according to the paper, cipher suites may be not be added
 	// or reordered by the user, but they may be disabled.
 	expectedCipherSuiteOrder := []uint16{
+		TLS_AES_128_GCM_SHA256,                      // 0x1301
+		TLS_CHACHA20_POLY1305_SHA256,                // 0x1303
+		TLS_AES_256_GCM_SHA384,                      // 0x1302
 		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
 		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,   // 0xc02f
 		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,  // 0xcca9
@@ -379,7 +433,7 @@ func (info rawHelloInfo) looksLikeChrome() bool {
 	// to not support, but do not check for the inclusion of
 	// specific ciphers or extensions, nor do we validate their
 	// order. When appropriate, we check the presence and order
-	// of elliptic curves, compression methods, and EC point formats."
+	// of elliptic curves, compression methods, and EC point formats." (early 2016)
 
 	// Not in Chrome 56, but present in Safari 10 (Feb. 2017):
 	// TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
@@ -401,14 +455,14 @@ func (info rawHelloInfo) looksLikeChrome() bool {
 	// 0xc00a, 0xc014, 0xc009, 0x9c, 0x9d, 0x2f, 0x35, 0xa
 
 	chromeCipherExclusions := map[uint16]struct{}{
-		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:   {}, // 0xc024
-		TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256:   {}, // 0xc023
-		TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:     {}, // 0xc028
-		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256: {}, // 0xc027
-		TLS_RSA_WITH_AES_256_CBC_SHA256:           {}, // 0x3d
-		tls.TLS_RSA_WITH_AES_128_CBC_SHA256:       {}, // 0x3c
-		TLS_DHE_RSA_WITH_AES_128_CBC_SHA:          {}, // 0x33
-		TLS_DHE_RSA_WITH_AES_256_CBC_SHA:          {}, // 0x39
+		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:     {}, // 0xc024
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256: {}, // 0xc023
+		TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:       {}, // 0xc028
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256:   {}, // 0xc027
+		TLS_RSA_WITH_AES_256_CBC_SHA256:             {}, // 0x3d
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA256:         {}, // 0x3c
+		TLS_DHE_RSA_WITH_AES_128_CBC_SHA:            {}, // 0x33
+		TLS_DHE_RSA_WITH_AES_256_CBC_SHA:            {}, // 0x39
 	}
 	for _, ext := range info.cipherSuites {
 		if _, ok := chromeCipherExclusions[ext]; ok {
@@ -436,7 +490,7 @@ func (info rawHelloInfo) looksLikeEdge() bool {
 	// "SChannel connections can by uniquely identified because SChannel
 	// is the only TLS library we tested that includes the OCSP status
 	// request extension before the supported groups and EC point formats
-	// extensions."
+	// extensions." (early 2016)
 	//
 	// More specifically, the OCSP status request extension appears
 	// *directly* before the other two extensions, which occur in that
@@ -482,24 +536,28 @@ func (info rawHelloInfo) looksLikeSafari() bool {
 	// in the HTTP User-Agent header. We allow for any of the
 	// updates when validating handshakes, and we check for the
 	// presence and ordering of ciphers, extensions, elliptic
-	// curves, and compression methods."
+	// curves, and compression methods." (early 2016)
 
 	// Note that any C lib (e.g. curl) compiled on macOS
 	// will probably use Secure Transport which will also
 	// share the TLS handshake characteristics of Safari.
 
-	// Let's do the easy check first... should be sufficient in many cases.
-	if len(info.cipherSuites) < 1 {
-		return false
-	}
-	if info.cipherSuites[0] != scsvRenegotiation {
-		return false
-	}
-
 	// We check for the presence and order of the extensions.
 	requiredExtensionsOrder := []uint16{10, 11, 13, 13172, 16, 5, 18, 23}
 	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
-		return false
+		// Safari on iOS 11 (beta) uses different set/ordering of extensions
+		requiredExtensionsOrderiOS11 := []uint16{65281, 0, 23, 13, 5, 13172, 18, 16, 11, 10}
+		if !assertPresenceAndOrdering(requiredExtensionsOrderiOS11, info.extensions, true) {
+			return false
+		}
+	} else {
+		// For these versions of Safari, expect TLS_EMPTY_RENEGOTIATION_INFO_SCSV first.
+		if len(info.cipherSuites) < 1 {
+			return false
+		}
+		if info.cipherSuites[0] != scsvRenegotiation {
+			return false
+		}
 	}
 
 	if hasGreaseCiphers(info.cipherSuites) {
@@ -511,7 +569,7 @@ func (info rawHelloInfo) looksLikeSafari() bool {
 		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, // 0xc02c
 		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
 		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,     // 0xc024
-		TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,     // 0xc023
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, // 0xc023
 		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,    // 0xc00a
 		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,    // 0xc009
 		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,   // 0xc030
@@ -523,13 +581,77 @@ func (info rawHelloInfo) looksLikeSafari() bool {
 		tls.TLS_RSA_WITH_AES_256_GCM_SHA384,         // 0x9d
 		tls.TLS_RSA_WITH_AES_128_GCM_SHA256,         // 0x9c
 		TLS_RSA_WITH_AES_256_CBC_SHA256,             // 0x3d
-		TLS_RSA_WITH_AES_128_CBC_SHA256,             // 0x3c
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA256,         // 0x3c
 		tls.TLS_RSA_WITH_AES_256_CBC_SHA,            // 0x35
 		tls.TLS_RSA_WITH_AES_128_CBC_SHA,            // 0x2f
 	}
 	return assertPresenceAndOrdering(expectedCipherSuiteOrder, info.cipherSuites, true)
 }
 
+// looksLikeTor returns true if the info looks like a ClientHello from Tor browser
+// (based on Firefox).
+func (info rawHelloInfo) looksLikeTor() bool {
+	requiredExtensionsOrder := []uint16{10, 11, 16, 5, 13}
+	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
+		return false
+	}
+
+	// check for session tickets support; Tor doesn't support them to prevent tracking
+	for _, ext := range info.extensions {
+		if ext == 35 {
+			return false
+		}
+	}
+
+	// We check for both presence of curves and their ordering, including
+	// an optional curve at the beginning (for Tor based on Firefox 52)
+	infoCurves := info.curves
+	if len(info.curves) == 4 {
+		if info.curves[0] != 29 {
+			return false
+		}
+		infoCurves = info.curves[1:]
+	}
+	requiredCurves := []tls.CurveID{23, 24, 25}
+	if len(infoCurves) < len(requiredCurves) {
+		return false
+	}
+	for i := range requiredCurves {
+		if infoCurves[i] != requiredCurves[i] {
+			return false
+		}
+	}
+
+	if hasGreaseCiphers(info.cipherSuites) {
+		return false
+	}
+
+	// We check for order of cipher suites but not presence, since
+	// according to the paper, cipher suites may be not be added
+	// or reordered by the user, but they may be disabled.
+	expectedCipherSuiteOrder := []uint16{
+		TLS_AES_128_GCM_SHA256,                      // 0x1301
+		TLS_CHACHA20_POLY1305_SHA256,                // 0x1303
+		TLS_AES_256_GCM_SHA384,                      // 0x1302
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
+		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,   // 0xc02f
+		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,  // 0xcca9
+		tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,    // 0xcca8
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, // 0xc02c
+		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,   // 0xc030
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,    // 0xc00a
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,    // 0xc009
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,      // 0xc013
+		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,      // 0xc014
+		TLS_DHE_RSA_WITH_AES_128_CBC_SHA,            // 0x33
+		TLS_DHE_RSA_WITH_AES_256_CBC_SHA,            // 0x39
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA,            // 0x2f
+		tls.TLS_RSA_WITH_AES_256_CBC_SHA,            // 0x35
+		tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,           // 0xa
+	}
+	return assertPresenceAndOrdering(expectedCipherSuiteOrder, info.cipherSuites, false)
+}
+
 // assertPresenceAndOrdering will return true if candidateList contains
 // the items in requiredItems in the same order as requiredItems.
 //
@@ -610,11 +732,17 @@ const (
 	// cipher suites missing from the crypto/tls package,
 	// in no particular order here
 	TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 = 0xc024
-	TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 = 0xc023
 	TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384   = 0xc028
-	TLS_RSA_WITH_AES_128_CBC_SHA256         = 0x3c
 	TLS_RSA_WITH_AES_256_CBC_SHA256         = 0x3d
 	TLS_DHE_RSA_WITH_AES_128_CBC_SHA        = 0x33
 	TLS_DHE_RSA_WITH_AES_256_CBC_SHA        = 0x39
 	TLS_RSA_WITH_RC4_128_MD5                = 0x4
+
+	// new PSK ciphers introduced by TLS 1.3, not (yet) in crypto/tls
+	// https://tlswg.github.io/tls13-spec/#rfc.appendix.A.4)
+	TLS_AES_128_GCM_SHA256       = 0x1301
+	TLS_AES_256_GCM_SHA384       = 0x1302
+	TLS_CHACHA20_POLY1305_SHA256 = 0x1303
+	TLS_AES_128_CCM_SHA256       = 0x1304
+	TLS_AES_128_CCM_8_SHA256     = 0x1305
 )

caddyhttp/httpserver/mitm_test.go

@@ -108,20 +108,20 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010000c003031dae75222dae1433a5a283ddcde8ddabaefbf16d84f250eee6fdff48cdfff8a00000201a1ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a010000777a7a0000ff010001000000000e000c0000096c6f63616c686f73740017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a0008aaaa001d001700182a2a000100`,
 				interception: false,
 			},
-			// TODO: Chrome on iOS will use iOS' TLS stack for requests that load
-			// the web page (apparently required by the dev ToS) but will use its
-			// own TLS stack for everything else, it seems. Figure out a decent way
-			// to test this with a nice, unified corpus that allows for this variance.
-			// {
-			// 	// Chrome on iOS
-			// 	userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
-			// 	helloHex:  `010000de030358b062c509b21410a6496b5a82bfec74436cdecebe8ea1da29799939bbd3c17200002c00ffc02cc02bc024c023c00ac009c008c030c02fc028c027c014c013c012009d009c003d003c0035002f000a0100008900000014001200000f66696e6572706978656c732e636f6d000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
-			// },
-			// {
-			// 	// Chrome on iOS (requesting favicon)
-			// 	userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
-			// 	helloHex:  `010000c20303863eb64788e3b9638c261300318411cbdd8f09576d58eec1e744b6ce944f574f0000208a8acca9cca8cc14cc13c02bc02fc02cc030c013c014009c009d002f0035000a01000079baba0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e31000b00020100000a000a00083a3a001d001700184a4a000100`,
-			// },
+			{
+				// Chrome on iOS will use iOS' TLS stack for requests that load
+				// the web page (apparently required by the dev ToS) but will use its
+				// own TLS stack for everything else, it seems.
+
+				// Chrome on iOS
+				userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
+				helloHex:  `010000de030358b062c509b21410a6496b5a82bfec74436cdecebe8ea1da29799939bbd3c17200002c00ffc02cc02bc024c023c00ac009c008c030c02fc028c027c014c013c012009d009c003d003c0035002f000a0100008900000014001200000f66696e6572706978656c732e636f6d000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
+			},
+			{
+				// Chrome on iOS (requesting favicon)
+				userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
+				helloHex:  `010000c20303863eb64788e3b9638c261300318411cbdd8f09576d58eec1e744b6ce944f574f0000208a8acca9cca8cc14cc13c02bc02fc02cc030c013c014009c009d002f0035000a01000079baba0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e31000b00020100000a000a00083a3a001d001700184a4a000100`,
+			},
 			{
 				userAgent:    "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
 				helloHex:     `010000c603036f717a88212c3e9e41940f82c42acb3473e0e4a64e8f52d9af33d34e972e08a30000206a6ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a0100007d7a7a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00087a7a001d001700188a8a000100`,
@@ -132,6 +132,16 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010001fc030383141d213d1bf069171843489faf808028d282c9828e1ba87637c863833c730720a67e76e152f4b704523b72317ef4587e231f02e2395e0ecac6be9f28c35e6ce600208a8ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a010001931a1a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d00170000002300785e85429bf1764f33111cd3ad5d1c56d765976fd962b49dbecbb6f7865e2a8d8536ad854f1fa99a8bbbf998814fee54a63a0bf162869d2bba37e9778304e7c4140825718e191b574c6246a0611de6447bdd80417f83ff9d9b7124069a9f74b90394ecb89bec5f6a1a67c1b89e50b8674782f53dd51807651a000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00081a1a001d001700182a2a0001000015009a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000`,
 				interception: false,
 			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
+				helloHex:     `010000c203034166c97e2016046e0c88ad867c410d0aee470f4d9b4ec8fe41a751d2a6348e3100001c4a4ac02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a0100007dcaca0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00086a6a001d001700187a7a000100`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
+				helloHex:     `010000c203037741795e73cd5b4949f79a0dc9cccc8b006e4c0ec324f965c6fe9f0833909f0100001c7a7ac02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a0100007d7a7a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00084a4a001d001700185a5a000100`,
+				interception: false,
+			},
 		},
 		"Firefox": {
 			{
@@ -139,6 +149,28 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010000bd030375f9022fc3a6562467f3540d68013b2d0b961979de6129e944efe0b35531323500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a010000760000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000ff030000000d0020001e040305030603020308040805080604010501060102010402050206020202`,
 				interception: false,
 			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010001fc0303c99d54ae0628bbb9fea3833a4244c6a712cac9d7738f4930b8b9d8e2f6bd578220f7936cedb48907981c9292fb08ceee6f59bd6fddb3d4271ccd7c12380c5038ab001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a01000195001500af000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b000201000023007886da2d41843ff42131b856982c19a545837b70e604325423a817d925e9d95bd084737682cea6b804dfb7cbe336a3b27b8d520d57520c29cfe5f4f3d3236183b84b05c18f0ca30bf598111e390086fea00d9631f1f78527277eb7838b86e73c4e5d15b55d086b1a4a8aa29f12a55126c6274bcd499bbeb23a0010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010000b1030365d899820b999245d571c2f7d6b850f63ad931d3c68ceb9cf5a508421a871dc500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100006a0000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+			{
+				// this was a Nightly release at the time
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0",
+				helloHex:     `010001fc030331e380b7d12018e1202ef3327607203df5c5732b4fa5ab5abaf0b60034c2fb662070c836b9b89123e37f4f1074d152df438fa8ee8a0f89b036fd952f4fcc0b994f001c130113031302c02bc02fcca9cca8c02cc030c013c014002f0035000a0100019700000014001200000f63616464797365727665722e636f6d00170000ff01000100000a000e000c001d00170018001901000101000b0002010000230078c97e7716a041e2ea824571bef26a3dff2bf50a883cd15d904ab2d17deb514f6e0a079ee7c212c000178387ffafc2e530b6df6662f570aae134330f13c458a0eaad5a96a9696f572110918740b15db1143d19aaaa706942030b433a7e6150f62b443c0564e5b8f7ee9577bf3bf7faec8c67425b648ab54d880010000e000c02683208687474702f312e310005000501000000000028006b0069001d0020aee6e596155ee6f79f943e81ceabe0979d27fbbb8b9189ccb2ebc75226351f32001700410421875a44e510decac11ef1d7cfddd4dfe105d5cd3a2d42fba03ebde23e51e8ce65bda1b48be82d4848d1db2bfce68e94092e925a9ce0dbf5df35479558108489002b0009087f12030303020301000d0018001604030503060308040805080604010501060102030201002d000201010015002500000000000000000000000000000000000000000000000000000000000000000000000000`,
+				interception: false,
+			},
+			{
+				// Firefox on Fedora (RedHat) doesn't include ECC ciphers because of patent liabilities
+				userAgent:    "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010000b70303f5280b74d617d42e39fd77b78a2b537b1d7787ce4fcbcf3604c9fbcd677c6c5500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100007000000014001200000f66696e6572706978656c732e636f6d00170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
 		},
 		"Edge": {
 			{
@@ -153,6 +185,23 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				helloHex:     `010000d2030358a295b513c8140c6ff880f4a8a73cc830ed2dab2c4f2068eb365228d828732e00002600ffc02cc02bc024c023c00ac009c030c02fc028c027c014c013009d009c003d003c0035002f010000830000000e000c0000096c6f63616c686f7374000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
 				interception: false,
 			},
+			{
+				userAgent:    "Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.28 (KHTML, like Gecko) Version/11.0 Mobile/15A5318g Safari/604.1",
+				helloHex:     `010000e10303be294e11847ba01301e0bb6129f4a0d66344602141a8f0a1ab0750a1db145755000028c02cc02bc024c023cca9c00ac009c030c02fc028c027cca8c014c013009d009c003d003c0035002f01000090ff0100010000000014001200000f66696e6572706978656c732e636f6d00170000000d00140012040308040401050308050501080606010201000500050100000000337400000012000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e31000b00020100000a00080006001d00170018`,
+				interception: false,
+			},
+		},
+		"Tor": {
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0",
+				helloHex:     `010000a40303137f05d4151f2d9095aee4254416d9dce73d6a1d857e8097ea20d021c04a7a81000016c02bc02fc00ac009c013c01400330039002f0035000a0100006500000014001200000f66696e6572706978656c732e636f6dff01000100000a00080006001700180019000b00020100337400000010000b000908687474702f312e31000500050100000000000d001600140401050106010201040305030603020304020202`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0",
+				helloHex:     `010000b4030322e1f3aff4c37caba303c2ce53ba1689b3e70117a46f413d44f70a74cb6a496100001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100006d00000014001200000f66696e6572706978656c732e636f6d00170000ff01000100000a000a0008001d001700180019000b000201000010000b000908687474702f312e31000500050100000000ff030000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
 		},
 		"Other": { // these are either non-browser clients or intercepted client hellos
 			{
@@ -221,7 +270,7 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 				interception: true,
 			},
 			{
-				// IE 11 on Windows 10, intercepted by Fortigate (same firewallas above)
+				// IE 11 on Windows 10, intercepted by Fortigate (same firewall as above)
 				userAgent:    "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko",
 				helloHex:     `010000e5030158ac634c5278d7b17421f23a64cc91d68c470c6b247322fe867ba035b373d05c000064003300320039003800160013c013c009c014c00ac012c008002f0035000a00150012003d003c00670040006b006ac011c0070096009a009900410084004500440088008700ba00be00bd00c000c400c3c03cc044c042c03dc045c04300090005000400ff01000058000a003600340000000100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019000b0002010000000014001200000f66696e6572706978656c732e636f6d`,
 				interception: true,
@@ -254,6 +303,7 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 			isFirefox := parsed.looksLikeFirefox()
 			isSafari := parsed.looksLikeSafari()
 			isEdge := parsed.looksLikeEdge()
+			isTor := parsed.looksLikeTor()
 
 			// we want each of the heuristic functions to be as
 			// exclusive but as low-maintenance as possible;
@@ -261,26 +311,17 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 			// should return false, with as little logic as possible,
 			// but with enough logic to force TLS proxies to do a
 			// good job preserving characterstics of the handshake.
-			var correct bool
-			switch client {
-			case "Chrome":
-				correct = isChrome && !isFirefox && !isSafari && !isEdge
-			case "Firefox":
-				correct = !isChrome && isFirefox && !isSafari && !isEdge
-			case "Safari":
-				correct = !isChrome && !isFirefox && isSafari && !isEdge
-			case "Edge":
-				correct = !isChrome && !isFirefox && !isSafari && isEdge
-			case "Other":
-				correct = !isChrome && !isFirefox && !isSafari && !isEdge
+			if (isChrome && (isFirefox || isSafari || isEdge || isTor)) ||
+				(isFirefox && (isChrome || isSafari || isEdge || isTor)) ||
+				(isSafari && (isChrome || isFirefox || isEdge || isTor)) ||
+				(isEdge && (isChrome || isFirefox || isSafari || isTor)) ||
+				(isTor && (isChrome || isFirefox || isSafari || isEdge)) {
+				t.Errorf("[%s] Test %d: Multiple fingerprinting functions matched: "+
+					"Chrome=%v Firefox=%v Safari=%v Edge=%v Tor=%v\n\tparsed hello dec: %+v\n\tparsed hello hex: %#x\n",
+					client, i, isChrome, isFirefox, isSafari, isEdge, isTor, parsed, parsed)
 			}
 
-			if !correct {
-				t.Errorf("[%s] Test %d: Chrome=%v, Firefox=%v, Safari=%v, Edge=%v; parsed hello: %+v",
-					client, i, isChrome, isFirefox, isSafari, isEdge, parsed)
-			}
-
-			// test the handler too
+			// test the handler and detection results
 			var got, checked bool
 			want := ch.interception
 			handler := &tlsHandler{
@@ -305,7 +346,54 @@ func TestHeuristicFunctionsAndHandler(t *testing.T) {
 			if got != want {
 				t.Errorf("[%s] Test %d: Expected MITM=%v but got %v (type assertion OK (checked)=%v)",
 					client, i, want, got, checked)
+				t.Errorf("[%s] Test %d: Looks like Chrome=%v Firefox=%v Safari=%v Edge=%v Tor=%v\n\tparsed hello dec: %+v\n\tparsed hello hex: %#x\n",
+					client, i, isChrome, isFirefox, isSafari, isEdge, isTor, parsed, parsed)
 			}
 		}
 	}
 }
+
+func TestGetVersion(t *testing.T) {
+	for i, test := range []struct {
+		UserAgent    string
+		SoftwareName string
+		Version      float64
+	}{
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0",
+			SoftwareName: "Firefox",
+			Version:      45.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0 more_stuff_here",
+			SoftwareName: "Firefox",
+			Version:      45.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Safari",
+			Version:      537.36,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Chrome",
+			Version:      51.0270479,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Mozilla",
+			Version:      5.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "curl",
+			Version:      -1,
+		},
+	} {
+		actual := getVersion(test.UserAgent, test.SoftwareName)
+		if actual != test.Version {
+			t.Errorf("Test [%d]: Expected version=%f, got version=%f for %s in '%s'",
+				i, test.Version, actual, test.SoftwareName, test.UserAgent)
+		}
+	}
+}

caddyhttp/httpserver/path.go

@@ -5,19 +5,25 @@ import (
 	"strings"
 )
 
-// Path represents a URI path.
+// Path represents a URI path. It should usually be
+// set to the value of a request path.
 type Path string
 
-// Matches checks to see if other matches p.
+// Matches checks to see if base matches p. The correct
+// usage of this method sets p as the request path, and
+// base as a Caddyfile (user-defined) rule path.
 //
 // Path matching will probably not always be a direct
 // comparison; this method assures that paths can be
 // easily and consistently matched.
-func (p Path) Matches(other string) bool {
-	if CaseSensitivePath {
-		return strings.HasPrefix(string(p), other)
+func (p Path) Matches(base string) bool {
+	if base == "/" {
+		return true
 	}
-	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(other))
+	if CaseSensitivePath {
+		return strings.HasPrefix(string(p), base)
+	}
+	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(base))
 }
 
 // PathMatcher is a Path RequestMatcher.

caddyhttp/httpserver/path_test.go

@@ -0,0 +1,96 @@
+package httpserver
+
+import "testing"
+
+func TestPathMatches(t *testing.T) {
+	for i, testcase := range []struct {
+		reqPath         Path
+		rulePath        string // or "base path" as in Caddyfile docs
+		shouldMatch     bool
+		caseInsensitive bool
+	}{
+		{
+			reqPath:     "/",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "/foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/bar",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/Foobar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+
+			reqPath:     "/FooBar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:         "/foobar",
+			rulePath:        "/FooBar",
+			shouldMatch:     true,
+			caseInsensitive: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "/", // a lone forward slash means to match all requests (see issue #1645) - many future test cases related to this issue
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "foobar.php",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/no/leading/slash",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+	} {
+		CaseSensitivePath = !testcase.caseInsensitive
+		if got, want := testcase.reqPath.Matches(testcase.rulePath), testcase.shouldMatch; got != want {
+			t.Errorf("Test %d: For request path '%s' and other path '%s': expected %v, got %v",
+				i, testcase.reqPath, testcase.rulePath, want, got)
+		}
+	}
+}

caddyhttp/httpserver/plugin.go

@@ -436,13 +436,14 @@ var directives = []string{
 	"root",
 	"index",
 	"bind",
-	"maxrequestbody", // TODO: 'limits'
+	"limits",
 	"timeouts",
 	"tls",
 
 	// services/utilities, or other directives that don't necessarily inject handlers
 	"startup",
 	"shutdown",
+	"request_id",
 	"realip", // github.com/captncraig/caddy-realip
 	"git",    // github.com/abiosoft/caddy-git
 
@@ -452,11 +453,13 @@ var directives = []string{
 	// directives that add middleware to the stack
 	"locale", // github.com/simia-tech/caddy-locale
 	"log",
+	"cache", // github.com/nicolasazrak/caddy-cache
 	"rewrite",
 	"ext",
 	"gzip",
 	"header",
 	"errors",
+	"authz",     // github.com/casbin/caddy-authz
 	"filter",    // github.com/echocat/caddy-filter
 	"minify",    // github.com/hacdias/caddy-minify
 	"ipfilter",  // github.com/pyed/ipfilter
@@ -466,8 +469,11 @@ var directives = []string{
 	"basicauth",
 	"redir",
 	"status",
-	"cors", // github.com/captncraig/cors/caddy
+	"cors",   // github.com/captncraig/cors/caddy
+	"nobots", // github.com/Xumeiquer/nobots
 	"mime",
+	"login",     // github.com/tarent/loginsrv/caddy
+	"reauth",    // github.com/freman/caddy-reauth
 	"jwt",       // github.com/BTBurke/caddy-jwt
 	"jsonp",     // github.com/pschlump/caddy-jsonp
 	"upload",    // blitznote.com/src/caddy.upload
@@ -476,18 +482,23 @@ var directives = []string{
 	"pprof",
 	"expvar",
 	"push",
+	"datadog",    // github.com/payintech/caddy-datadog
 	"prometheus", // github.com/miekg/caddy-prometheus
 	"proxy",
 	"fastcgi",
 	"cgi", // github.com/jung-kurt/caddy-cgi
 	"websocket",
 	"filemanager", // github.com/hacdias/caddy-filemanager
+	"webdav",      // github.com/hacdias/caddy-webdav
 	"markdown",
 	"templates",
 	"browse",
 	"hugo",      // github.com/hacdias/caddy-hugo
 	"mailout",   // github.com/SchumacherFM/mailout
 	"awslambda", // github.com/coopernurse/caddy-awslambda
+	"grpc",      // github.com/pieterlouw/caddy-grpc
+	"gopkg",     // github.com/zikes/gopkg
+	"restic",    // github.com/restic/caddy
 }
 
 const (

caddyhttp/httpserver/recorder.go

@@ -1,8 +1,6 @@
 package httpserver
 
 import (
-	"bufio"
-	"net"
 	"net/http"
 	"time"
 )
@@ -20,7 +18,7 @@ import (
 //
 // Beware when accessing the Replacer value; it may be nil!
 type ResponseRecorder struct {
-	http.ResponseWriter
+	*ResponseWriterWrapper
 	Replacer Replacer
 	status   int
 	size     int
@@ -35,9 +33,9 @@ type ResponseRecorder struct {
 // of 200 to cover the default case.
 func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 	return &ResponseRecorder{
-		ResponseWriter: w,
-		status:         http.StatusOK,
-		start:          time.Now(),
+		ResponseWriterWrapper: &ResponseWriterWrapper{ResponseWriter: w},
+		status:                http.StatusOK,
+		start:                 time.Now(),
 	}
 }
 
@@ -45,13 +43,13 @@ func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 // underlying ResponseWriter's WriteHeader method.
 func (r *ResponseRecorder) WriteHeader(status int) {
 	r.status = status
-	r.ResponseWriter.WriteHeader(status)
+	r.ResponseWriterWrapper.WriteHeader(status)
 }
 
 // Write is a wrapper that records the size of the body
 // that gets written.
 func (r *ResponseRecorder) Write(buf []byte) (int, error) {
-	n, err := r.ResponseWriter.Write(buf)
+	n, err := r.ResponseWriterWrapper.Write(buf)
 	if err == nil {
 		r.size += n
 	}
@@ -68,45 +66,5 @@ func (r *ResponseRecorder) Status() int {
 	return r.status
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (r *ResponseRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := r.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, NonHijackerError{Underlying: r.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or does nothing.
-func (r *ResponseRecorder) Flush() {
-	if f, ok := r.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(NonFlusherError{Underlying: r.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (r *ResponseRecorder) CloseNotify() <-chan bool {
-	if cn, ok := r.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(NonCloseNotifierError{Underlying: r.ResponseWriter})
-}
-
-// Push resource to client
-func (r *ResponseRecorder) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := r.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return NonPusherError{Underlying: r.ResponseWriter}
-}
-
 // Interface guards
-var _ http.Pusher = (*ResponseRecorder)(nil)
-var _ http.Flusher = (*ResponseRecorder)(nil)
-var _ http.CloseNotifier = (*ResponseRecorder)(nil)
-var _ http.Hijacker = (*ResponseRecorder)(nil)
+var _ HTTPInterfaces = (*ResponseRecorder)(nil)

caddyhttp/httpserver/replacer.go

@@ -243,6 +243,9 @@ func (r *replacer) getSubstitution(key string) string {
 	case "{path_escaped}":
 		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
 		return url.QueryEscape(u.Path)
+	case "{request_id}":
+		reqid, _ := r.request.Context().Value(RequestIDCtxKey).(string)
+		return reqid
 	case "{rewrite_path}":
 		return r.request.URL.Path
 	case "{rewrite_path_escaped}":
@@ -284,6 +287,8 @@ func (r *replacer) getSubstitution(key string) string {
 		return now().Format(timeFormat)
 	case "{when_iso}":
 		return now().UTC().Format(timeFormatISOUTC)
+	case "{when_unix}":
+		return strconv.FormatInt(now().Unix(), 10)
 	case "{file}":
 		_, file := path.Split(r.request.URL.Path)
 		return file
@@ -302,7 +307,7 @@ func (r *replacer) getSubstitution(key string) string {
 		}
 		_, err := ioutil.ReadAll(r.request.Body)
 		if err != nil {
-			if _, ok := err.(MaxBytesExceeded); ok {
+			if err == ErrMaxBytesExceeded {
 				return r.emptyValue
 			}
 		}
@@ -312,7 +317,6 @@ func (r *replacer) getSubstitution(key string) string {
 			if val {
 				return "likely"
 			}
-
 			return "unlikely"
 		}
 		return "unknown"

caddyhttp/httpserver/replacer_test.go

@@ -75,6 +75,7 @@ func TestReplace(t *testing.T) {
 		{"The response status is {status}.", "The response status is 200."},
 		{"{when}", "02/Jan/2006:15:04:05 +0000"},
 		{"{when_iso}", "2006-01-02T15:04:12Z"},
+		{"{when_unix}", "1136214252"},
 		{"The Custom header is {>Custom}.", "The Custom header is foobarbaz."},
 		{"The CustomAdd header is {>CustomAdd}.", "The CustomAdd header is caddy."},
 		{"The request is {request}.", "The request is POST /?foo=bar HTTP/1.1\\r\\nHost: localhost\\r\\n" +

caddyhttp/httpserver/responsewriterwrapper.go

@@ -0,0 +1,65 @@
+package httpserver
+
+import (
+	"bufio"
+	"net"
+	"net/http"
+)
+
+// ResponseWriterWrapper wrappers underlying ResponseWriter
+// and inherits its Hijacker/Pusher/CloseNotifier/Flusher as well.
+type ResponseWriterWrapper struct {
+	http.ResponseWriter
+}
+
+// Hijack implements http.Hijacker. It simply wraps the underlying
+// ResponseWriter's Hijack method if there is one, or returns an error.
+func (rww *ResponseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) {
+	if hj, ok := rww.ResponseWriter.(http.Hijacker); ok {
+		return hj.Hijack()
+	}
+	return nil, nil, NonHijackerError{Underlying: rww.ResponseWriter}
+}
+
+// Flush implements http.Flusher. It simply wraps the underlying
+// ResponseWriter's Flush method if there is one, or panics.
+func (rww *ResponseWriterWrapper) Flush() {
+	if f, ok := rww.ResponseWriter.(http.Flusher); ok {
+		f.Flush()
+	} else {
+		panic(NonFlusherError{Underlying: rww.ResponseWriter})
+	}
+}
+
+// CloseNotify implements http.CloseNotifier.
+// It just inherits the underlying ResponseWriter's CloseNotify method.
+// It panics if the underlying ResponseWriter is not a CloseNotifier.
+func (rww *ResponseWriterWrapper) CloseNotify() <-chan bool {
+	if cn, ok := rww.ResponseWriter.(http.CloseNotifier); ok {
+		return cn.CloseNotify()
+	}
+	panic(NonCloseNotifierError{Underlying: rww.ResponseWriter})
+}
+
+// Push implements http.Pusher.
+// It just inherits the underlying ResponseWriter's Push method.
+// It panics if the underlying ResponseWriter is not a Pusher.
+func (rww *ResponseWriterWrapper) Push(target string, opts *http.PushOptions) error {
+	if pusher, hasPusher := rww.ResponseWriter.(http.Pusher); hasPusher {
+		return pusher.Push(target, opts)
+	}
+
+	return NonPusherError{Underlying: rww.ResponseWriter}
+}
+
+// HTTPInterfaces mix all the interfaces that middleware ResponseWriters need to support.
+type HTTPInterfaces interface {
+	http.ResponseWriter
+	http.Pusher
+	http.Flusher
+	http.CloseNotifier
+	http.Hijacker
+}
+
+// Interface guards
+var _ HTTPInterfaces = (*ResponseWriterWrapper)(nil)

caddyhttp/httpserver/roller.go

@@ -1,6 +1,7 @@
 package httpserver
 
 import (
+	"errors"
 	"io"
 	"path/filepath"
 	"strconv"
@@ -14,6 +15,7 @@ type LogRoller struct {
 	MaxSize    int
 	MaxAge     int
 	MaxBackups int
+	Compress   bool
 	LocalTime  bool
 }
 
@@ -37,6 +39,7 @@ func (l LogRoller) GetLogWriter() io.Writer {
 			MaxSize:    l.MaxSize,
 			MaxAge:     l.MaxAge,
 			MaxBackups: l.MaxBackups,
+			Compress:   l.Compress,
 			LocalTime:  l.LocalTime,
 		}
 		lumberjacks[absPath] = lj
@@ -48,20 +51,36 @@ func (l LogRoller) GetLogWriter() io.Writer {
 func IsLogRollerSubdirective(subdir string) bool {
 	return subdir == directiveRotateSize ||
 		subdir == directiveRotateAge ||
-		subdir == directiveRotateKeep
+		subdir == directiveRotateKeep ||
+		subdir == directiveRotateCompress
 }
 
+var invalidRollerParameterErr = errors.New("invalid roller parameter")
+
 // ParseRoller parses roller contents out of c.
-func ParseRoller(l *LogRoller, what string, where string) error {
+func ParseRoller(l *LogRoller, what string, where ...string) error {
 	if l == nil {
 		l = DefaultLogRoller()
 	}
-	var value int
-	var err error
-	value, err = strconv.Atoi(where)
-	if err != nil {
-		return err
+
+	// rotate_compress doesn't accept any parameters.
+	// others only accept one parameter
+	if (what == directiveRotateCompress && len(where) != 0) ||
+		(what != directiveRotateCompress && len(where) != 1) {
+		return invalidRollerParameterErr
 	}
+
+	var (
+		value int
+		err   error
+	)
+	if what != directiveRotateCompress {
+		value, err = strconv.Atoi(where[0])
+		if err != nil {
+			return err
+		}
+	}
+
 	switch what {
 	case directiveRotateSize:
 		l.MaxSize = value
@@ -69,6 +88,8 @@ func ParseRoller(l *LogRoller, what string, where string) error {
 		l.MaxAge = value
 	case directiveRotateKeep:
 		l.MaxBackups = value
+	case directiveRotateCompress:
+		l.Compress = true
 	}
 	return nil
 }
@@ -79,6 +100,7 @@ func DefaultLogRoller() *LogRoller {
 		MaxSize:    defaultRotateSize,
 		MaxAge:     defaultRotateAge,
 		MaxBackups: defaultRotateKeep,
+		Compress:   false,
 		LocalTime:  true,
 	}
 }
@@ -89,10 +111,12 @@ const (
 	// defaultRotateAge is 14 days.
 	defaultRotateAge = 14
 	// defaultRotateKeep is 10 files.
-	defaultRotateKeep   = 10
-	directiveRotateSize = "rotate_size"
-	directiveRotateAge  = "rotate_age"
-	directiveRotateKeep = "rotate_keep"
+	defaultRotateKeep = 10
+
+	directiveRotateSize     = "rotate_size"
+	directiveRotateAge      = "rotate_age"
+	directiveRotateKeep     = "rotate_keep"
+	directiveRotateCompress = "rotate_compress"
 )
 
 // lumberjacks maps log filenames to the logger

caddyhttp/httpserver/server.go

@@ -4,8 +4,8 @@ package httpserver
 import (
 	"context"
 	"crypto/tls"
+	"errors"
 	"fmt"
-	"io"
 	"log"
 	"net"
 	"net/http"
@@ -57,6 +57,16 @@ func makeTLSConfig(group []*SiteConfig) (*tls.Config, error) {
 	return caddytls.MakeTLSConfig(tlsConfigs)
 }
 
+func getFallbacks(sites []*SiteConfig) []string {
+	fallbacks := []string{}
+	for _, sc := range sites {
+		if sc.FallbackSite {
+			fallbacks = append(fallbacks, sc.Addr.Host)
+		}
+	}
+	return fallbacks
+}
+
 // NewServer creates a new Server instance that will listen on addr
 // and will serve the sites configured in group.
 func NewServer(addr string, group []*SiteConfig) (*Server, error) {
@@ -66,6 +76,8 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 		sites:       group,
 		connTimeout: GracefulTimeout,
 	}
+	s.vhosts.fallbackHosts = append(s.vhosts.fallbackHosts, getFallbacks(group)...)
+	s.Server = makeHTTPServerWithHeaderLimit(s.Server, group)
 	s.Server.Handler = s // this is weird, but whatever
 
 	// extract TLS settings from each site config to build
@@ -76,14 +88,14 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 	}
 	s.Server.TLSConfig = tlsConfig
 
-	// Enable QUIC if desired
-	if QUIC {
-		s.quicServer = &h2quic.Server{Server: s.Server}
-		s.Server.Handler = s.wrapWithSvcHeaders(s.Server.Handler)
-	}
-
 	// if TLS is enabled, make sure we prepare the Server accordingly
 	if s.Server.TLSConfig != nil {
+		// enable QUIC if desired (requires HTTP/2)
+		if HTTP2 && QUIC {
+			s.quicServer = &h2quic.Server{Server: s.Server}
+			s.Server.Handler = s.wrapWithSvcHeaders(s.Server.Handler)
+		}
+
 		// wrap the HTTP handler with a handler that does MITM detection
 		tlsh := &tlsHandler{next: s.Server.Handler}
 		s.Server.Handler = tlsh // this needs to be the "outer" handler when Serve() is called, for type assertion
@@ -127,6 +139,32 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 	return s, nil
 }
 
+// makeHTTPServerWithHeaderLimit apply minimum header limit within a group to given http.Server
+func makeHTTPServerWithHeaderLimit(s *http.Server, group []*SiteConfig) *http.Server {
+	var min int64
+	for _, cfg := range group {
+		limit := cfg.Limits.MaxRequestHeaderSize
+		if limit == 0 {
+			continue
+		}
+
+		// not set yet
+		if min == 0 {
+			min = limit
+		}
+
+		// find a better one
+		if limit < min {
+			min = limit
+		}
+	}
+
+	if min > 0 {
+		s.MaxHeaderBytes = int(min)
+	}
+	return s
+}
+
 // makeHTTPServerWithTimeouts makes an http.Server from the group of
 // configs in a way that configures timeouts (or, if not set, it uses
 // the default timeouts) by combining the configuration of each
@@ -275,7 +313,7 @@ func (s *Server) Serve(ln net.Listener) error {
 
 // ServePacket serves QUIC requests on pc until it is closed.
 func (s *Server) ServePacket(pc net.PacketConn) error {
-	if QUIC {
+	if s.quicServer != nil {
 		err := s.quicServer.Serve(pc.(*net.UDPConn))
 		return fmt.Errorf("serving QUIC connections: %v", err)
 	}
@@ -304,7 +342,7 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	c := context.WithValue(r.Context(), OriginalURLCtxKey, urlCopy)
 	r = r.WithContext(c)
 
-	w.Header().Set("Server", "Caddy")
+	w.Header().Set("Server", caddy.AppName)
 
 	status, _ := s.serveHTTP(w, r)
 
@@ -325,6 +363,8 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 
 	// look up the virtualhost; if no match, serve error
 	vhost, pathPrefix := s.vhosts.Match(hostname + r.URL.Path)
+	c := context.WithValue(r.Context(), caddy.CtxKey("path_prefix"), pathPrefix)
+	r = r.WithContext(c)
 
 	if vhost == nil {
 		// check for ACME challenge even if vhost is nil;
@@ -337,7 +377,7 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 		if err != nil {
 			remoteHost = r.RemoteAddr
 		}
-		WriteTextResponse(w, http.StatusNotFound, "No such site at "+s.Server.Addr)
+		WriteSiteNotFound(w, r) // don't add headers outside of this function
 		log.Printf("[INFO] %s - No such site at %s (Remote: %s, Referer: %s)",
 			hostname, s.Server.Addr, remoteHost, r.Header.Get("Referer"))
 		return 0, nil
@@ -359,20 +399,6 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 		}
 	}
 
-	// Apply the path-based request body size limit
-	// The error returned by MaxBytesReader is meant to be handled
-	// by whichever middleware/plugin that receives it when calling
-	// .Read() or a similar method on the request body
-	// TODO: Make this middleware instead?
-	if r.Body != nil {
-		for _, pathlimit := range vhost.MaxRequestBodySizes {
-			if Path(r.URL.Path).Matches(pathlimit.Path) {
-				r.Body = MaxBytesReader(w, r.Body, pathlimit.Limit)
-				break
-			}
-		}
-	}
-
 	return vhost.middlewareChain.ServeHTTP(w, r)
 }
 
@@ -435,9 +461,9 @@ func (s *Server) OnStartupComplete() {
 }
 
 // defaultTimeouts stores the default timeout values to use
-// if left unset by user configuration. NOTE: Default timeouts
-// are disabled (see issue #1464).
-var defaultTimeouts Timeouts
+// if left unset by user configuration. NOTE: Most default
+// timeouts are disabled (see issues #1464 and #1733).
+var defaultTimeouts = Timeouts{IdleTimeout: 5 * time.Minute}
 
 // tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
 // connections. It's used by ListenAndServe and ListenAndServeTLS so
@@ -465,73 +491,9 @@ func (ln tcpKeepAliveListener) File() (*os.File, error) {
 	return ln.TCPListener.File()
 }
 
-// MaxBytesExceeded is the error type returned by MaxBytesReader
+// ErrMaxBytesExceeded is the error returned by MaxBytesReader
 // when the request body exceeds the limit imposed
-type MaxBytesExceeded struct{}
-
-func (err MaxBytesExceeded) Error() string {
-	return "http: request body too large"
-}
-
-// MaxBytesReader and its associated methods are borrowed from the
-// Go Standard library (comments intact). The only difference is that
-// it returns a MaxBytesExceeded error instead of a generic error message
-// when the request body has exceeded the requested limit
-func MaxBytesReader(w http.ResponseWriter, r io.ReadCloser, n int64) io.ReadCloser {
-	return &maxBytesReader{w: w, r: r, n: n}
-}
-
-type maxBytesReader struct {
-	w   http.ResponseWriter
-	r   io.ReadCloser // underlying reader
-	n   int64         // max bytes remaining
-	err error         // sticky error
-}
-
-func (l *maxBytesReader) Read(p []byte) (n int, err error) {
-	if l.err != nil {
-		return 0, l.err
-	}
-	if len(p) == 0 {
-		return 0, nil
-	}
-	// If they asked for a 32KB byte read but only 5 bytes are
-	// remaining, no need to read 32KB. 6 bytes will answer the
-	// question of the whether we hit the limit or go past it.
-	if int64(len(p)) > l.n+1 {
-		p = p[:l.n+1]
-	}
-	n, err = l.r.Read(p)
-
-	if int64(n) <= l.n {
-		l.n -= int64(n)
-		l.err = err
-		return n, err
-	}
-
-	n = int(l.n)
-	l.n = 0
-
-	// The server code and client code both use
-	// maxBytesReader. This "requestTooLarge" check is
-	// only used by the server code. To prevent binaries
-	// which only using the HTTP Client code (such as
-	// cmd/go) from also linking in the HTTP server, don't
-	// use a static type assertion to the server
-	// "*response" type. Check this interface instead:
-	type requestTooLarger interface {
-		requestTooLarge()
-	}
-	if res, ok := l.w.(requestTooLarger); ok {
-		res.requestTooLarge()
-	}
-	l.err = MaxBytesExceeded{}
-	return n, l.err
-}
-
-func (l *maxBytesReader) Close() error {
-	return l.r.Close()
-}
+var ErrMaxBytesExceeded = errors.New("http: request body too large")
 
 // DefaultErrorFunc responds to an HTTP request with a simple description
 // of the specified HTTP status code.
@@ -539,6 +501,19 @@ func DefaultErrorFunc(w http.ResponseWriter, r *http.Request, status int) {
 	WriteTextResponse(w, status, fmt.Sprintf("%d %s\n", status, http.StatusText(status)))
 }
 
+const httpStatusMisdirectedRequest = 421 // RFC 7540, 9.1.2
+
+// WriteSiteNotFound writes appropriate error code to w, signaling that
+// requested host is not served by Caddy on a given port.
+func WriteSiteNotFound(w http.ResponseWriter, r *http.Request) {
+	status := http.StatusNotFound
+	if r.ProtoMajor >= 2 {
+		// TODO: use http.StatusMisdirectedRequest when it gets defined
+		status = httpStatusMisdirectedRequest
+	}
+	WriteTextResponse(w, status, fmt.Sprintf("%d Site %s is not served on this interface\n", status, r.Host))
+}
+
 // WriteTextResponse writes body with code status to w. The body will
 // be interpreted as plain text.
 func WriteTextResponse(w http.ResponseWriter, status int, body string) {

caddyhttp/httpserver/server_test.go

@@ -15,7 +15,7 @@ func TestAddress(t *testing.T) {
 	}
 }
 
-func TestMakeHTTPServer(t *testing.T) {
+func TestMakeHTTPServerWithTimeouts(t *testing.T) {
 	for i, tc := range []struct {
 		group    []*SiteConfig
 		expected Timeouts
@@ -111,3 +111,36 @@ func TestMakeHTTPServer(t *testing.T) {
 		}
 	}
 }
+
+func TestMakeHTTPServerWithHeaderLimit(t *testing.T) {
+	for name, c := range map[string]struct {
+		group  []*SiteConfig
+		expect int
+	}{
+		"disable": {
+			group:  []*SiteConfig{{}},
+			expect: 0,
+		},
+		"oneSite": {
+			group: []*SiteConfig{{Limits: Limits{
+				MaxRequestHeaderSize: 100,
+			}}},
+			expect: 100,
+		},
+		"multiSites": {
+			group: []*SiteConfig{
+				{Limits: Limits{MaxRequestHeaderSize: 100}},
+				{Limits: Limits{MaxRequestHeaderSize: 50}},
+			},
+			expect: 50,
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			actual := makeHTTPServerWithHeaderLimit(&http.Server{}, c.group)
+			if got := actual.MaxHeaderBytes; got != c.expect {
+				t.Errorf("Expect %d, but got %d", c.expect, got)
+			}
+		})
+	}
+}

caddyhttp/httpserver/siteconfig.go

@@ -38,8 +38,8 @@ type SiteConfig struct {
 	// for a request.
 	HiddenFiles []string
 
-	// Max amount of bytes a request can send on a given path
-	MaxRequestBodySizes []PathLimit
+	// Max request's header/body size
+	Limits Limits
 
 	// The path to the Caddyfile used to generate this site config
 	originCaddyfile string
@@ -52,6 +52,10 @@ type SiteConfig struct {
 	// preserving functionality needed for proxying,
 	// websockets, etc.
 	Timeouts Timeouts
+
+	// If true, any requests not matching other site definitions
+	// may be served by this site.
+	FallbackSite bool
 }
 
 // Timeouts specify various timeouts for a server to use.
@@ -71,6 +75,12 @@ type Timeouts struct {
 	IdleTimeoutSet       bool
 }
 
+// Limits specify size limit of request's header and body.
+type Limits struct {
+	MaxRequestHeaderSize int64
+	MaxRequestBodySizes  []PathLimit
+}
+
 // PathLimit is a mapping from a site's path to its corresponding
 // maximum request body size (in bytes)
 type PathLimit struct {

caddyhttp/httpserver/tplcontext.go

@@ -424,12 +424,13 @@ func (c Context) RandomString(minLen, maxLen int) string {
 	return string(result)
 }
 
-// Push adds a preload link in response header for server push
-func (c Context) Push(link string) string {
+// AddLink adds a link header in response
+// see https://www.w3.org/wiki/LinkHeader
+func (c Context) AddLink(link string) string {
 	if c.responseHeader == nil {
 		return ""
 	}
-	c.responseHeader.Add("Link", "<"+link+">; rel=preload")
+	c.responseHeader.Add("Link", link)
 	return ""
 }
 

caddyhttp/httpserver/tplcontext_test.go

@@ -497,7 +497,7 @@ func TestMethod(t *testing.T) {
 
 }
 
-func TestPathMatches(t *testing.T) {
+func TestContextPathMatches(t *testing.T) {
 	context := getContextOrFail(t)
 
 	tests := []struct {
@@ -877,18 +877,18 @@ func TestFiles(t *testing.T) {
 	}
 }
 
-func TestPush(t *testing.T) {
+func TestAddLink(t *testing.T) {
 	for name, c := range map[string]struct {
 		input       string
 		expectLinks []string
 	}{
 		"oneLink": {
-			input:       `{{.Push "/test.css"}}`,
+			input:       `{{.AddLink "</test.css>; rel=preload"}}`,
 			expectLinks: []string{"</test.css>; rel=preload"},
 		},
 		"multipleLinks": {
-			input:       `{{.Push "/test1.css"}} {{.Push "/test2.css"}}`,
-			expectLinks: []string{"</test1.css>; rel=preload", "</test2.css>; rel=preload"},
+			input:       `{{.AddLink "</test1.css>; rel=preload"}} {{.AddLink "</test2.css>; rel=meta"}}`,
+			expectLinks: []string{"</test1.css>; rel=preload", "</test2.css>; rel=meta"},
 		},
 	} {
 		c := c

caddyhttp/httpserver/vhosttrie.go

@@ -10,14 +10,15 @@ import (
 // wildcards as TLS certificates support them), then
 // by longest matching path.
 type vhostTrie struct {
-	edges map[string]*vhostTrie
-	site  *SiteConfig // site to match on this node; also known as a virtual host
-	path  string      // the path portion of the key for the associated site
+	fallbackHosts []string
+	edges         map[string]*vhostTrie
+	site          *SiteConfig // site to match on this node; also known as a virtual host
+	path          string      // the path portion of the key for the associated site
 }
 
 // newVHostTrie returns a new vhostTrie.
 func newVHostTrie() *vhostTrie {
-	return &vhostTrie{edges: make(map[string]*vhostTrie)}
+	return &vhostTrie{edges: make(map[string]*vhostTrie), fallbackHosts: []string{"0.0.0.0", ""}}
 }
 
 // Insert adds stack to t keyed by key. The key should be
@@ -57,13 +58,13 @@ func (t *vhostTrie) insertPath(remainingPath, originalPath string, site *SiteCon
 // A typical key will be in the form "host" or "host/path".
 func (t *vhostTrie) Match(key string) (*SiteConfig, string) {
 	host, path := t.splitHostPath(key)
-	// try the given host, then, if no match, try wildcard hosts
+	// try the given host, then, if no match, try fallback hosts
 	branch := t.matchHost(host)
-	if branch == nil {
-		branch = t.matchHost("0.0.0.0")
-	}
-	if branch == nil {
-		branch = t.matchHost("")
+	for _, h := range t.fallbackHosts {
+		if branch != nil {
+			break
+		}
+		branch = t.matchHost(h)
 	}
 	if branch == nil {
 		return nil, ""

caddyhttp/internalsrv/internal.go

@@ -7,8 +7,6 @@
 package internalsrv
 
 import (
-	"bufio"
-	"net"
 	"net/http"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -44,7 +42,7 @@ func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 
 	// Use internal response writer to ignore responses that will be
 	// redirected to internal locations
-	iw := internalResponseWriter{ResponseWriter: w}
+	iw := internalResponseWriter{ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w}}
 	status, err := i.Next.ServeHTTP(iw, r)
 
 	for c := 0; c < maxRedirectCount && isInternalRedirect(iw); c++ {
@@ -69,7 +67,7 @@ func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 // calls to Write and WriteHeader if the response should be redirected to an
 // internal location.
 type internalResponseWriter struct {
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 }
 
 // ClearHeader removes script headers that would interfere with follow up
@@ -84,7 +82,7 @@ func (w internalResponseWriter) ClearHeader() {
 // internal location.
 func (w internalResponseWriter) WriteHeader(code int) {
 	if !isInternalRedirect(w) {
-		w.ResponseWriter.WriteHeader(code)
+		w.ResponseWriterWrapper.WriteHeader(code)
 	}
 }
 
@@ -94,53 +92,8 @@ func (w internalResponseWriter) Write(b []byte) (int, error) {
 	if isInternalRedirect(w) {
 		return 0, nil
 	}
-	return w.ResponseWriter.Write(b)
-}
-
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (w internalResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := w.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: w.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (w internalResponseWriter) Flush() {
-	if f, ok := w.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: w.ResponseWriter})
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-// It panics if the underlying ResponseWriter is not a CloseNotifier.
-func (w internalResponseWriter) CloseNotify() <-chan bool {
-	if cn, ok := w.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: w.ResponseWriter})
-}
-
-// Push implements http.Pusher.
-// It just inherits the underlying ResponseWriter's Push method.
-// It panics if the underlying ResponseWriter is not a Pusher.
-func (w internalResponseWriter) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := w.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return httpserver.NonPusherError{Underlying: w.ResponseWriter}
+	return w.ResponseWriterWrapper.Write(b)
 }
 
 // Interface guards
-var (
-	_ http.Pusher        = internalResponseWriter{}
-	_ http.Flusher       = internalResponseWriter{}
-	_ http.CloseNotifier = internalResponseWriter{}
-	_ http.Hijacker      = internalResponseWriter{}
-)
+var _ httpserver.HTTPInterfaces = internalResponseWriter{}

caddyhttp/limits/handler.go

@@ -0,0 +1,90 @@
+package limits
+
+import (
+	"io"
+	"net/http"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+// Limit is a middleware to control request body size
+type Limit struct {
+	Next       httpserver.Handler
+	BodyLimits []httpserver.PathLimit
+}
+
+func (l Limit) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
+	if r.Body == nil {
+		return l.Next.ServeHTTP(w, r)
+	}
+
+	// apply the path-based request body size limit.
+	for _, bl := range l.BodyLimits {
+		if httpserver.Path(r.URL.Path).Matches(bl.Path) {
+			r.Body = MaxBytesReader(w, r.Body, bl.Limit)
+			break
+		}
+	}
+
+	return l.Next.ServeHTTP(w, r)
+}
+
+// MaxBytesReader and its associated methods are borrowed from the
+// Go Standard library (comments intact). The only difference is that
+// it returns a ErrMaxBytesExceeded error instead of a generic error message
+// when the request body has exceeded the requested limit
+func MaxBytesReader(w http.ResponseWriter, r io.ReadCloser, n int64) io.ReadCloser {
+	return &maxBytesReader{w: w, r: r, n: n}
+}
+
+type maxBytesReader struct {
+	w   http.ResponseWriter
+	r   io.ReadCloser // underlying reader
+	n   int64         // max bytes remaining
+	err error         // sticky error
+}
+
+func (l *maxBytesReader) Read(p []byte) (n int, err error) {
+	if l.err != nil {
+		return 0, l.err
+	}
+	if len(p) == 0 {
+		return 0, nil
+	}
+	// If they asked for a 32KB byte read but only 5 bytes are
+	// remaining, no need to read 32KB. 6 bytes will answer the
+	// question of the whether we hit the limit or go past it.
+	if int64(len(p)) > l.n+1 {
+		p = p[:l.n+1]
+	}
+	n, err = l.r.Read(p)
+
+	if int64(n) <= l.n {
+		l.n -= int64(n)
+		l.err = err
+		return n, err
+	}
+
+	n = int(l.n)
+	l.n = 0
+
+	// The server code and client code both use
+	// maxBytesReader. This "requestTooLarge" check is
+	// only used by the server code. To prevent binaries
+	// which only using the HTTP Client code (such as
+	// cmd/go) from also linking in the HTTP server, don't
+	// use a static type assertion to the server
+	// "*response" type. Check this interface instead:
+	type requestTooLarger interface {
+		requestTooLarge()
+	}
+	if res, ok := l.w.(requestTooLarger); ok {
+		res.requestTooLarge()
+	}
+	l.err = httpserver.ErrMaxBytesExceeded
+	return n, l.err
+}
+
+func (l *maxBytesReader) Close() error {
+	return l.r.Close()
+}

caddyhttp/limits/handler_test.go

@@ -0,0 +1,35 @@
+package limits
+
+import (
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func TestBodySizeLimit(t *testing.T) {
+	var (
+		gotContent    []byte
+		gotError      error
+		expectContent = "hello"
+	)
+	l := Limit{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			gotContent, gotError = ioutil.ReadAll(r.Body)
+			return 0, nil
+		}),
+		BodyLimits: []httpserver.PathLimit{{Path: "/", Limit: int64(len(expectContent))}},
+	}
+
+	r := httptest.NewRequest("GET", "/", strings.NewReader(expectContent+expectContent))
+	l.ServeHTTP(httptest.NewRecorder(), r)
+	if got := string(gotContent); got != expectContent {
+		t.Errorf("expected content[%s], got[%s]", expectContent, got)
+	}
+	if gotError != httpserver.ErrMaxBytesExceeded {
+		t.Errorf("expect error %v, got %v", httpserver.ErrMaxBytesExceeded, gotError)
+	}
+}

caddyhttp/limits/setup.go

@@ -1,4 +1,4 @@
-package maxrequestbody
+package limits
 
 import (
 	"errors"
@@ -12,13 +12,13 @@ import (
 
 const (
 	serverType = "http"
-	pluginName = "maxrequestbody"
+	pluginName = "limits"
 )
 
 func init() {
 	caddy.RegisterPlugin(pluginName, caddy.Plugin{
 		ServerType: serverType,
-		Action:     setupMaxRequestBody,
+		Action:     setupLimits,
 	})
 }
 
@@ -28,56 +28,97 @@ type pathLimitUnparsed struct {
 	Limit string
 }
 
-func setupMaxRequestBody(c *caddy.Controller) error {
+func setupLimits(c *caddy.Controller) error {
+	bls, err := parseLimits(c)
+	if err != nil {
+		return err
+	}
+
+	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
+		return Limit{Next: next, BodyLimits: bls}
+	})
+	return nil
+}
+
+func parseLimits(c *caddy.Controller) ([]httpserver.PathLimit, error) {
 	config := httpserver.GetConfig(c)
 
 	if !c.Next() {
-		return c.ArgErr()
+		return nil, c.ArgErr()
 	}
 
 	args := c.RemainingArgs()
 	argList := []pathLimitUnparsed{}
+	headerLimit := ""
 
 	switch len(args) {
 	case 0:
-		// Format: { <path> <limit> ... }
+		// Format: limits {
+		//	header <limit>
+		//	body <path> <limit>
+		//	body <limit>
+		//	...
+		// }
 		for c.NextBlock() {
-			path := c.Val()
-			if !c.NextArg() {
-				// Uneven pairing of path/limit
-				return c.ArgErr()
+			kind := c.Val()
+			pathOrLimit := c.RemainingArgs()
+			switch kind {
+			case "header":
+				if len(pathOrLimit) != 1 {
+					return nil, c.ArgErr()
+				}
+				headerLimit = pathOrLimit[0]
+			case "body":
+				if len(pathOrLimit) == 1 {
+					argList = append(argList, pathLimitUnparsed{
+						Path:  "/",
+						Limit: pathOrLimit[0],
+					})
+					break
+				}
+
+				if len(pathOrLimit) == 2 {
+					argList = append(argList, pathLimitUnparsed{
+						Path:  pathOrLimit[0],
+						Limit: pathOrLimit[1],
+					})
+					break
+				}
+
+				fallthrough
+			default:
+				return nil, c.ArgErr()
 			}
-			argList = append(argList, pathLimitUnparsed{
-				Path:  path,
-				Limit: c.Val(),
-			})
 		}
 	case 1:
-		// Format: <limit>
+		// Format: limits <limit>
+		headerLimit = args[0]
 		argList = []pathLimitUnparsed{{
 			Path:  "/",
 			Limit: args[0],
 		}}
-	case 2:
-		// Format: <path> <limit>
-		argList = []pathLimitUnparsed{{
-			Path:  args[0],
-			Limit: args[1],
-		}}
 	default:
-		return c.ArgErr()
+		return nil, c.ArgErr()
 	}
 
-	pathLimit, err := parseArguments(argList)
-	if err != nil {
-		return c.ArgErr()
+	if headerLimit != "" {
+		size := parseSize(headerLimit)
+		if size < 1 { // also disallow size = 0
+			return nil, c.ArgErr()
+		}
+		config.Limits.MaxRequestHeaderSize = size
 	}
 
-	SortPathLimits(pathLimit)
+	if len(argList) > 0 {
+		pathLimit, err := parseArguments(argList)
+		if err != nil {
+			return nil, c.ArgErr()
+		}
+		SortPathLimits(pathLimit)
+		config.Limits.MaxRequestBodySizes = pathLimit
+	}
 
-	config.MaxRequestBodySizes = pathLimit
-
-	return nil
+	return config.Limits.MaxRequestBodySizes, nil
 }
 
 func parseArguments(args []pathLimitUnparsed) ([]httpserver.PathLimit, error) {

caddyhttp/limits/setup_test.go

@@ -1,4 +1,4 @@
-package maxrequestbody
+package limits
 
 import (
 	"reflect"
@@ -14,32 +14,98 @@ const (
 	GB = 1024 * 1024 * 1024
 )
 
-func TestSetupMaxRequestBody(t *testing.T) {
-	cases := []struct {
-		input    string
-		hasError bool
+func TestParseLimits(t *testing.T) {
+	for name, c := range map[string]struct {
+		input     string
+		shouldErr bool
+		expect    httpserver.Limits
 	}{
-		// Format: { <path> <limit> ... }
-		{input: "maxrequestbody / 20MB", hasError: false},
-		// Format: <limit>
-		{input: "maxrequestbody 999KB", hasError: false},
-		// Format: { <path> <limit> ... }
-		{input: "maxrequestbody { /images 50MB /upload 10MB\n/test 10KB }", hasError: false},
-
-		// Wrong formats
-		{input: "maxrequestbody typo { /images 50MB }", hasError: true},
-		{input: "maxrequestbody 999MB /home 20KB", hasError: true},
-	}
-	for caseNum, c := range cases {
-		controller := caddy.NewTestController("", c.input)
-		err := setupMaxRequestBody(controller)
-
-		if c.hasError && (err == nil) {
-			t.Errorf("Expecting error for case %v but none encountered", caseNum)
-		}
-		if !c.hasError && (err != nil) {
-			t.Errorf("Expecting no error for case %v but encountered %v", caseNum, err)
-		}
+		"catchAll": {
+			input: `limits 2kb`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 2 * KB,
+				MaxRequestBodySizes:  []httpserver.PathLimit{{Path: "/", Limit: 2 * KB}},
+			},
+		},
+		"onlyHeader": {
+			input: `limits {
+				header 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 2 * KB,
+			},
+		},
+		"onlyBody": {
+			input: `limits {
+				body 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestBodySizes: []httpserver.PathLimit{{Path: "/", Limit: 2 * KB}},
+			},
+		},
+		"onlyBodyWithPath": {
+			input: `limits {
+				body /test 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestBodySizes: []httpserver.PathLimit{{Path: "/test", Limit: 2 * KB}},
+			},
+		},
+		"mixture": {
+			input: `limits {
+				header 1kb
+				body 2kb
+				body /bar 3kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 1 * KB,
+				MaxRequestBodySizes: []httpserver.PathLimit{
+					{Path: "/bar", Limit: 3 * KB},
+					{Path: "/", Limit: 2 * KB},
+				},
+			},
+		},
+		"invalidFormat": {
+			input:     `limits a b`,
+			shouldErr: true,
+		},
+		"invalidHeaderFormat": {
+			input: `limits {
+				header / 100
+			}`,
+			shouldErr: true,
+		},
+		"invalidBodyFormat": {
+			input: `limits {
+				body / 100 200
+			}`,
+			shouldErr: true,
+		},
+		"invalidKind": {
+			input: `limits {
+				head 100
+			}`,
+			shouldErr: true,
+		},
+		"invalidLimitSize": {
+			input:     `limits 10bk`,
+			shouldErr: true,
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			controller := caddy.NewTestController("", c.input)
+			_, err := parseLimits(controller)
+			if c.shouldErr && err == nil {
+				t.Error("failed to get expected error")
+			}
+			if !c.shouldErr && err != nil {
+				t.Errorf("got unexpected error: %v", err)
+			}
+			if got := httpserver.GetConfig(controller).Limits; !reflect.DeepEqual(got, c.expect) {
+				t.Errorf("expect %#v, but got %#v", c.expect, got)
+			}
+		})
 	}
 }
 

caddyhttp/log/setup.go

@@ -1,6 +1,8 @@
 package log
 
 import (
+	"strings"
+
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
@@ -36,63 +38,48 @@ func logParse(c *caddy.Controller) ([]*Rule, error) {
 
 		for c.NextBlock() {
 			what := c.Val()
-			if !c.NextArg() {
-				return nil, c.ArgErr()
-			}
-			where := c.Val()
+			where := c.RemainingArgs()
 
 			// only support roller related options inside a block
 			if !httpserver.IsLogRollerSubdirective(what) {
 				return nil, c.ArgErr()
 			}
 
-			if err := httpserver.ParseRoller(logRoller, what, where); err != nil {
+			if err := httpserver.ParseRoller(logRoller, what, where...); err != nil {
 				return nil, err
 			}
 		}
 
-		if len(args) == 0 {
-			// Nothing specified; use defaults
-			rules = appendEntry(rules, "/", &Entry{
-				Log: &httpserver.Logger{
-					Output: DefaultLogFilename,
-					Roller: logRoller,
-				},
-				Format: DefaultLogFormat,
-			})
-		} else if len(args) == 1 {
+		path := "/"
+		format := DefaultLogFormat
+		output := DefaultLogFilename
+
+		switch len(args) {
+		case 0:
+			// nothing to change
+		case 1:
 			// Only an output file specified
-			rules = appendEntry(rules, "/", &Entry{
-				Log: &httpserver.Logger{
-					Output: args[0],
-					Roller: logRoller,
-				},
-				Format: DefaultLogFormat,
-			})
-		} else {
+			output = args[0]
+		case 2, 3:
 			// Path scope, output file, and maybe a format specified
-
-			format := DefaultLogFormat
-
+			path = args[0]
+			output = args[1]
 			if len(args) > 2 {
-				switch args[2] {
-				case "{common}":
-					format = CommonLogFormat
-				case "{combined}":
-					format = CombinedLogFormat
-				default:
-					format = args[2]
-				}
+				format = strings.Replace(args[2], "{common}", CommonLogFormat, -1)
+				format = strings.Replace(format, "{combined}", CombinedLogFormat, -1)
 			}
-
-			rules = appendEntry(rules, args[0], &Entry{
-				Log: &httpserver.Logger{
-					Output: args[1],
-					Roller: logRoller,
-				},
-				Format: format,
-			})
+		default:
+			// Maximum number of args in log directive is 3.
+			return nil, c.ArgErr()
 		}
+
+		rules = appendEntry(rules, path, &Entry{
+			Log: &httpserver.Logger{
+				Output: output,
+				Roller: logRoller,
+			},
+			Format: format,
+		})
 	}
 
 	return rules, nil

caddyhttp/log/setup_test.go

@@ -124,6 +124,16 @@ func TestLogParse(t *testing.T) {
 				Format: CommonLogFormat,
 			}},
 		}}},
+		{`log /myapi log.txt "prefix {common} suffix"`, false, []Rule{{
+			PathScope: "/myapi",
+			Entries: []*Entry{{
+				Log: &httpserver.Logger{
+					Output: "log.txt",
+					Roller: httpserver.DefaultLogRoller(),
+				},
+				Format: "prefix " + CommonLogFormat + " suffix",
+			}},
+		}}},
 		{`log /test accesslog.txt {combined}`, false, []Rule{{
 			PathScope: "/test",
 			Entries: []*Entry{{
@@ -134,6 +144,16 @@ func TestLogParse(t *testing.T) {
 				Format: CombinedLogFormat,
 			}},
 		}}},
+		{`log /test accesslog.txt "prefix {combined} suffix"`, false, []Rule{{
+			PathScope: "/test",
+			Entries: []*Entry{{
+				Log: &httpserver.Logger{
+					Output: "accesslog.txt",
+					Roller: httpserver.DefaultLogRoller(),
+				},
+				Format: "prefix " + CombinedLogFormat + " suffix",
+			}},
+		}}},
 		{`log /api1 log.txt
 		  log /api2 accesslog.txt {combined}`, false, []Rule{{
 			PathScope: "/api1",
@@ -174,7 +194,12 @@ func TestLogParse(t *testing.T) {
 				Format: "{when}",
 			}},
 		}}},
-		{`log access.log { rotate_size 2 rotate_age 10 rotate_keep 3 }`, false, []Rule{{
+		{`log access.log {
+			rotate_size 2
+			rotate_age 10
+			rotate_keep 3
+			rotate_compress
+		}`, false, []Rule{{
 			PathScope: "/",
 			Entries: []*Entry{{
 				Log: &httpserver.Logger{
@@ -183,6 +208,7 @@ func TestLogParse(t *testing.T) {
 						MaxSize:    2,
 						MaxAge:     10,
 						MaxBackups: 3,
+						Compress:   true,
 						LocalTime:  true,
 					}},
 				Format: DefaultLogFormat,
@@ -205,8 +231,11 @@ func TestLogParse(t *testing.T) {
 				Format: "{when}",
 			}},
 		}}},
+		{`log access.log { rotate_size 2 rotate_age 10 rotate_keep 3 }`, true, nil},
+		{`log access.log { rotate_compress invalid }`, true, nil},
 		{`log access.log { rotate_size }`, true, nil},
 		{`log access.log { invalid_option 1 }`, true, nil},
+		{`log / acccess.log "{remote} - [{when}] "{method} {port}" {scheme} {mitm} "`, true, nil},
 	}
 	for i, test := range tests {
 		c := caddy.NewTestController("http", test.inputLogRules)

caddyhttp/markdown/markdown.go

@@ -53,6 +53,9 @@ type Config struct {
 
 	// Template(s) to render with
 	Template *template.Template
+
+	// a pair of template's name and its underlying file path
+	TemplateFiles map[string]string
 }
 
 // ServeHTTP implements the http.Handler interface.

caddyhttp/markdown/markdown_test.go

@@ -1,17 +1,15 @@
 package markdown
 
 import (
-	"bufio"
 	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
 	"os"
 	"path/filepath"
-	"strings"
 	"testing"
 	"text/template"
-	"time"
 
+	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 	"github.com/russross/blackfriday"
 )
@@ -79,19 +77,23 @@ func TestMarkdown(t *testing.T) {
 		}),
 	}
 
-	req, err := http.NewRequest("GET", "/blog/test.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
+	get := func(url string) string {
+		req, err := http.NewRequest("GET", url, nil)
+		if err != nil {
+			t.Fatalf("Could not create HTTP request: %v", err)
+		}
+		rec := httptest.NewRecorder()
+		code, err := md.ServeHTTP(rec, req)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if code != http.StatusOK {
+			t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, code)
+		}
+		return rec.Body.String()
 	}
 
-	rec := httptest.NewRecorder()
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-
-	respBody := rec.Body.String()
+	respBody := get("/blog/test.md")
 	expectedBody := `<!DOCTYPE html>
 <html>
 <head>
@@ -99,7 +101,6 @@ func TestMarkdown(t *testing.T) {
 </head>
 <body>
 <h1>Header for: Markdown test 1</h1>
-
 Welcome to A Caddy website!
 <h2>Welcome on the blog</h2>
 
@@ -113,46 +114,26 @@ Welcome to A Caddy website!
 </body>
 </html>
 `
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 
-	req, err = http.NewRequest("GET", "/docflags/test.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
-	}
-	rec = httptest.NewRecorder()
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-	respBody = rec.Body.String()
+	respBody = get("/docflags/test.md")
 	expectedBody = `Doc.var_string hello
-Doc.var_bool <no value>
-DocFlags.var_string <no value>
-DocFlags.var_bool true`
+Doc.var_bool true
+`
 
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 
-	req, err = http.NewRequest("GET", "/log/test.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
-	}
-	rec = httptest.NewRecorder()
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-	respBody = rec.Body.String()
+	respBody = get("/log/test.md")
 	expectedBody = `<!DOCTYPE html>
 <html>
 	<head>
 		<title>Markdown test 2</title>
 		<meta charset="utf-8">
+		
 		<link rel="stylesheet" href="/resources/css/log.css">
 		<link rel="stylesheet" href="/resources/css/default.css">
 		<script src="/resources/js/log.js"></script>
@@ -171,26 +152,11 @@ DocFlags.var_bool true`
 	</body>
 </html>`
 
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 
-	req, err = http.NewRequest("GET", "/og/first.md", nil)
-	if err != nil {
-		t.Fatalf("Could not create HTTP request: %v", err)
-	}
-	rec = httptest.NewRecorder()
-	currenttime := time.Now().Local().Add(-time.Second)
-	_ = os.Chtimes("testdata/og/first.md", currenttime, currenttime)
-	currenttime = time.Now().Local()
-	_ = os.Chtimes("testdata/og_static/og/first.md/index.html", currenttime, currenttime)
-	time.Sleep(time.Millisecond * 200)
-
-	md.ServeHTTP(rec, req)
-	if rec.Code != http.StatusOK {
-		t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, rec.Code)
-	}
-	respBody = rec.Body.String()
+	respBody = get("/og/first.md")
 	expectedBody = `<!DOCTYPE html>
 <html>
 <head>
@@ -198,32 +164,18 @@ DocFlags.var_bool true`
 </head>
 <body>
 <h1>Header for: first_post</h1>
-
 Welcome to title!
 <h1>Test h1</h1>
 
 </body>
-</html>`
+</html>
+`
 
-	if !equalStrings(respBody, expectedBody) {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+	if respBody != expectedBody {
+		t.Fatalf("Expected body:\n%q\ngot:\n%q", expectedBody, respBody)
 	}
 }
 
-func equalStrings(s1, s2 string) bool {
-	s1 = strings.TrimSpace(s1)
-	s2 = strings.TrimSpace(s2)
-	in := bufio.NewScanner(strings.NewReader(s1))
-	for in.Scan() {
-		txt := strings.TrimSpace(in.Text())
-		if !strings.HasPrefix(strings.TrimSpace(s2), txt) {
-			return false
-		}
-		s2 = strings.Replace(s2, txt, "", 1)
-	}
-	return true
-}
-
 func setDefaultTemplate(filename string) *template.Template {
 	buf, err := ioutil.ReadFile(filename)
 	if err != nil {
@@ -232,3 +184,68 @@ func setDefaultTemplate(filename string) *template.Template {
 
 	return template.Must(GetDefaultTemplate().Parse(string(buf)))
 }
+
+func TestTemplateReload(t *testing.T) {
+	const (
+		templateFile = "testdata/test.html"
+		targetFile   = "testdata/hello.md"
+	)
+	c := caddy.NewTestController("http", `markdown {
+		template `+templateFile+`
+	}`)
+
+	err := ioutil.WriteFile(templateFile, []byte("hello {{.Doc.body}}"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = ioutil.WriteFile(targetFile, []byte("caddy"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer func() {
+		os.Remove(templateFile)
+		os.Remove(targetFile)
+	}()
+
+	config, err := markdownParse(c)
+	if err != nil {
+		t.Fatal(err)
+	}
+	md := Markdown{
+		Root:    "./testdata",
+		FileSys: http.Dir("./testdata"),
+		Configs: config,
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			t.Fatalf("Next shouldn't be called")
+			return 0, nil
+		}),
+	}
+
+	req := httptest.NewRequest("GET", "/hello.md", nil)
+	get := func() string {
+		rec := httptest.NewRecorder()
+		code, err := md.ServeHTTP(rec, req)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if code != http.StatusOK {
+			t.Fatalf("Wrong status, expected: %d and got %d", http.StatusOK, code)
+		}
+		return rec.Body.String()
+	}
+
+	if expect, got := "hello <p>caddy</p>\n", get(); expect != got {
+		t.Fatalf("Expected body:\n%q\nbut got:\n%q", expect, got)
+	}
+
+	// update template
+	err = ioutil.WriteFile(templateFile, []byte("hi {{.Doc.body}}"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	if expect, got := "hi <p>caddy</p>\n", get(); expect != got {
+		t.Fatalf("Expected body:\n%q\nbut got:\n%q", expect, got)
+	}
+
+}

caddyhttp/markdown/setup.go

@@ -44,10 +44,11 @@ func markdownParse(c *caddy.Controller) ([]*Config, error) {
 
 	for c.Next() {
 		md := &Config{
-			Renderer:   blackfriday.HtmlRenderer(0, "", ""),
-			Extensions: make(map[string]struct{}),
-			Template:   GetDefaultTemplate(),
-			IndexFiles: []string{},
+			Renderer:      blackfriday.HtmlRenderer(0, "", ""),
+			Extensions:    make(map[string]struct{}),
+			Template:      GetDefaultTemplate(),
+			IndexFiles:    []string{},
+			TemplateFiles: make(map[string]string),
 		}
 
 		// Get the path scope
@@ -115,28 +116,42 @@ func loadParams(c *caddy.Controller, mdc *Config) error {
 			fpath := filepath.ToSlash(filepath.Clean(cfg.Root + string(filepath.Separator) + tArgs[0]))
 
 			if err := SetTemplate(mdc.Template, "", fpath); err != nil {
-				c.Errf("default template parse error: %v", err)
+				return c.Errf("default template parse error: %v", err)
 			}
+
+			mdc.TemplateFiles[""] = fpath
 			return nil
 		case 2:
 			fpath := filepath.ToSlash(filepath.Clean(cfg.Root + string(filepath.Separator) + tArgs[1]))
 
 			if err := SetTemplate(mdc.Template, tArgs[0], fpath); err != nil {
-				c.Errf("template parse error: %v", err)
+				return c.Errf("template parse error: %v", err)
 			}
+
+			mdc.TemplateFiles[tArgs[0]] = fpath
 			return nil
 		}
 	case "templatedir":
 		if !c.NextArg() {
 			return c.ArgErr()
 		}
-		_, err := mdc.Template.ParseGlob(c.Val())
+
+		pattern := c.Val()
+		_, err := mdc.Template.ParseGlob(pattern)
 		if err != nil {
-			c.Errf("template load error: %v", err)
+			return c.Errf("template load error: %v", err)
 		}
 		if c.NextArg() {
 			return c.ArgErr()
 		}
+
+		paths, err := filepath.Glob(pattern)
+		if err != nil {
+			return c.Errf("glob %q failed: %v", pattern, err)
+		}
+		for _, path := range paths {
+			mdc.TemplateFiles[filepath.Base(path)] = path
+		}
 		return nil
 	default:
 		return c.Err("Expected valid markdown configuration property")

caddyhttp/markdown/setup_test.go

@@ -4,6 +4,7 @@ import (
 	"bytes"
 	"fmt"
 	"net/http"
+	"reflect"
 	"testing"
 	"text/template"
 
@@ -59,9 +60,10 @@ func TestMarkdownParse(t *testing.T) {
 				".md":  {},
 				".txt": {},
 			},
-			Styles:   []string{"/resources/css/blog.css"},
-			Scripts:  []string{"/resources/js/blog.js"},
-			Template: GetDefaultTemplate(),
+			Styles:        []string{"/resources/css/blog.css"},
+			Scripts:       []string{"/resources/js/blog.js"},
+			Template:      GetDefaultTemplate(),
+			TemplateFiles: make(map[string]string),
 		}}},
 		{`markdown /blog {
 	ext .md
@@ -71,12 +73,12 @@ func TestMarkdownParse(t *testing.T) {
 			Extensions: map[string]struct{}{
 				".md": {},
 			},
-			Template: GetDefaultTemplate(),
+			Template: setDefaultTemplate("./testdata/tpl_with_include.html"),
+			TemplateFiles: map[string]string{
+				"": "testdata/tpl_with_include.html",
+			},
 		}}},
 	}
-	// Setup the extra template
-	tmpl := tests[1].expectedMarkdownConfig[0].Template
-	SetTemplate(tmpl, "", "./testdata/tpl_with_include.html")
 
 	for i, test := range tests {
 		c := caddy.NewTestController("http", test.inputMarkdownConfig)
@@ -110,6 +112,10 @@ func TestMarkdownParse(t *testing.T) {
 			if ok, tx, ty := equalTemplates(actualMarkdownConfig.Template, test.expectedMarkdownConfig[j].Template); !ok {
 				t.Errorf("Test %d the %dth Markdown Config Templates did not match, expected %s to be %s", i, j, tx, ty)
 			}
+			if expect, got := test.expectedMarkdownConfig[j].TemplateFiles, actualMarkdownConfig.TemplateFiles; !reflect.DeepEqual(expect, got) {
+				t.Errorf("Test %d the %d Markdown config TemplateFiles did not match, expect %v, but got %v", i, j, expect, got)
+			}
+
 		}
 	}
 }

caddyhttp/markdown/template.go

@@ -22,7 +22,8 @@ type Data struct {
 // Include "overrides" the embedded httpserver.Context's Include()
 // method so that included files have access to d's fields.
 // Note: using {{template 'template-name' .}} instead might be better.
-func (d Data) Include(filename string) (string, error) {
+func (d Data) Include(filename string, args ...interface{}) (string, error) {
+	d.Args = args
 	return httpserver.ContextInclude(filename, d, d.Root)
 }
 
@@ -37,8 +38,18 @@ func execTemplate(c *Config, mdata metadata.Metadata, meta map[string]string, fi
 		Files:   files,
 	}
 
+	templateName := mdata.Template
+	// reload template on every request for now
+	// TODO: cache templates by a general plugin
+	if templateFile, ok := c.TemplateFiles[templateName]; ok {
+		err := SetTemplate(c.Template, templateName, templateFile)
+		if err != nil {
+			return nil, err
+		}
+	}
+
 	b := new(bytes.Buffer)
-	if err := c.Template.ExecuteTemplate(b, mdata.Template, mdData); err != nil {
+	if err := c.Template.ExecuteTemplate(b, templateName, mdData); err != nil {
 		return nil, err
 	}
 

caddyhttp/markdown/testdata/blog/test.md

@@ -0,0 +1,14 @@
+---
+title: Markdown test 1
+sitename: A Caddy website
+---
+
+## Welcome on the blog
+
+Body
+
+``` go
+func getTrue() bool {
+    return true
+}
+```

caddyhttp/markdown/testdata/docflags/template.txt

@@ -0,0 +1,2 @@
+Doc.var_string {{.Doc.var_string}}
+Doc.var_bool {{.Doc.var_bool}}

caddyhttp/markdown/testdata/docflags/test.md

@@ -0,0 +1,4 @@
+---
+var_string: hello
+var_bool: true
+---

caddyhttp/markdown/testdata/header.html

@@ -0,0 +1 @@
+<h1>Header for: {{.Doc.title}}</h1>

caddyhttp/markdown/testdata/log/test.md

@@ -0,0 +1,14 @@
+---
+title: Markdown test 2
+sitename: A Caddy website
+---
+
+## Welcome on the blog
+
+Body
+
+``` go
+func getTrue() bool {
+    return true
+}
+```

caddyhttp/markdown/testdata/markdown_tpl.html

@@ -0,0 +1,11 @@
+<!DOCTYPE html>
+<html>
+<head>
+<title>{{.Doc.title}}</title>
+</head>
+<body>
+{{.Include "header.html"}}
+Welcome to {{.Doc.sitename}}!
+{{.Doc.body}}
+</body>
+</html>

caddyhttp/markdown/testdata/og/first.md

@@ -0,0 +1,5 @@
+---
+title: first_post
+sitename: title
+---
+# Test h1

caddyhttp/markdown/testdata/tpl_with_include.html

@@ -0,0 +1,11 @@
+<!DOCTYPE html>
+<html>
+	<head>
+		<title>{{.Doc.title}}</title>
+	</head>
+	<body>
+		Welcome to {{.Doc.sitename}}!
+		<br><br>
+		{{.Doc.body}}
+	</body>
+</html>

caddyhttp/mime/mime_test.go

@@ -57,7 +57,7 @@ func nextFunc(shouldMime bool, contentType string) httpserver.Handler {
 	return httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
 		if shouldMime {
 			if w.Header().Get("Content-Type") != contentType {
-				return 0, fmt.Errorf("expected Content-Type: %v, found %v", contentType, r.Header.Get("Content-Type"))
+				return 0, fmt.Errorf("expected Content-Type: %v, found %v", contentType, w.Header().Get("Content-Type"))
 			}
 			return 0, nil
 		}

caddyhttp/proxy/policy.go

@@ -18,11 +18,13 @@ type Policy interface {
 }
 
 func init() {
-	RegisterPolicy("random", func() Policy { return &Random{} })
-	RegisterPolicy("least_conn", func() Policy { return &LeastConn{} })
-	RegisterPolicy("round_robin", func() Policy { return &RoundRobin{} })
-	RegisterPolicy("ip_hash", func() Policy { return &IPHash{} })
-	RegisterPolicy("first", func() Policy { return &First{} })
+	RegisterPolicy("random", func(arg string) Policy { return &Random{} })
+	RegisterPolicy("least_conn", func(arg string) Policy { return &LeastConn{} })
+	RegisterPolicy("round_robin", func(arg string) Policy { return &RoundRobin{} })
+	RegisterPolicy("ip_hash", func(arg string) Policy { return &IPHash{} })
+	RegisterPolicy("first", func(arg string) Policy { return &First{} })
+	RegisterPolicy("uri_hash", func(arg string) Policy { return &URIHash{} })
+	RegisterPolicy("header", func(arg string) Policy { return &Header{arg} })
 }
 
 // Random is a policy that selects up hosts from a pool at random.
@@ -56,7 +58,7 @@ func (r *Random) Select(pool HostPool, request *http.Request) *UpstreamHost {
 type LeastConn struct{}
 
 // Select selects the up host with the least number of connections in the
-// pool.  If more than one host has the same least number of connections,
+// pool. If more than one host has the same least number of connections,
 // one of the hosts is chosen at random.
 func (r *LeastConn) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	var bestHost *UpstreamHost
@@ -84,13 +86,13 @@ func (r *LeastConn) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	return bestHost
 }
 
-// RoundRobin is a policy that selects hosts based on round robin ordering.
+// RoundRobin is a policy that selects hosts based on round-robin ordering.
 type RoundRobin struct {
 	robin uint32
 	mutex sync.Mutex
 }
 
-// Select selects an up host from the pool using a round robin ordering scheme.
+// Select selects an up host from the pool using a round-robin ordering scheme.
 func (r *RoundRobin) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	poolLen := uint32(len(pool))
 	r.mutex.Lock()
@@ -106,23 +108,10 @@ func (r *RoundRobin) Select(pool HostPool, request *http.Request) *UpstreamHost
 	return nil
 }
 
-// IPHash is a policy that selects hosts based on hashing the request ip
-type IPHash struct{}
-
-func hash(s string) uint32 {
-	h := fnv.New32a()
-	h.Write([]byte(s))
-	return h.Sum32()
-}
-
-// Select selects an up host from the pool using a round robin ordering scheme.
-func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+// hostByHashing returns an available host from pool based on a hashable string
+func hostByHashing(pool HostPool, s string) *UpstreamHost {
 	poolLen := uint32(len(pool))
-	clientIP, _, err := net.SplitHostPort(request.RemoteAddr)
-	if err != nil {
-		clientIP = request.RemoteAddr
-	}
-	index := hash(clientIP) % poolLen
+	index := hash(s) % poolLen
 	for i := uint32(0); i < poolLen; i++ {
 		index += i
 		host := pool[index%poolLen]
@@ -133,10 +122,37 @@ func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	return nil
 }
 
-// First is a policy that selects the fist available host
+// hash calculates a hash based on string s
+func hash(s string) uint32 {
+	h := fnv.New32a()
+	h.Write([]byte(s))
+	return h.Sum32()
+}
+
+// IPHash is a policy that selects hosts based on hashing the request IP
+type IPHash struct{}
+
+// Select selects an up host from the pool based on hashing the request IP
+func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	clientIP, _, err := net.SplitHostPort(request.RemoteAddr)
+	if err != nil {
+		clientIP = request.RemoteAddr
+	}
+	return hostByHashing(pool, clientIP)
+}
+
+// URIHash is a policy that selects the host based on hashing the request URI
+type URIHash struct{}
+
+// Select selects the host based on hashing the URI
+func (r *URIHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	return hostByHashing(pool, request.RequestURI)
+}
+
+// First is a policy that selects the first available host
 type First struct{}
 
-// Select selects the first host from the pool, that is available
+// Select selects the first available host from the pool
 func (r *First) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	for _, host := range pool {
 		if host.Available() {
@@ -145,3 +161,22 @@ func (r *First) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	}
 	return nil
 }
+
+// Header is a policy that selects based on a hash of the given header
+type Header struct {
+	// The name of the request header, the value of which will determine
+	// how the request is routed
+	Name string
+}
+
+// Select selects the host based on hashing the header value
+func (r *Header) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	if r.Name == "" {
+		return nil
+	}
+	val := request.Header.Get(r.Name)
+	if val == "" {
+		return nil
+	}
+	return hostByHashing(pool, val)
+}

caddyhttp/proxy/policy_test.go

@@ -243,3 +243,101 @@ func TestFirstPolicy(t *testing.T) {
 		t.Error("Expected first policy host to be the second host.")
 	}
 }
+
+func TestUriPolicy(t *testing.T) {
+	pool := testPool()
+	uriPolicy := &URIHash{}
+
+	request := httptest.NewRequest(http.MethodGet, "/test", nil)
+	h := uriPolicy.Select(pool, request)
+	if h != pool[0] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	pool[0].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test_2", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the second host.")
+	}
+
+	// We should be able to resize the host pool and still be able to predict
+	// where a request will be routed with the same URI's used above
+	pool = []*UpstreamHost{
+		{
+			Name: workableServer.URL, // this should resolve (healthcheck test)
+		},
+		{
+			Name: "http://localhost:99998", // this shouldn't
+		},
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[0] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	pool[0].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test_2", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the second host.")
+	}
+
+	pool[0].Unhealthy = 1
+	pool[1].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != nil {
+		t.Error("Expected uri policy policy host to be nil.")
+	}
+}
+
+func TestHeaderPolicy(t *testing.T) {
+	pool := testPool()
+	tests := []struct {
+		Policy             *Header
+		RequestHeaderName  string
+		RequestHeaderValue string
+		NilHost            bool
+		HostIndex          int
+	}{
+		{&Header{""}, "", "", true, 0},
+		{&Header{""}, "Affinity", "somevalue", true, 0},
+		{&Header{""}, "Affinity", "", true, 0},
+
+		{&Header{"Affinity"}, "", "", true, 0},
+		{&Header{"Affinity"}, "Affinity", "somevalue", false, 1},
+		{&Header{"Affinity"}, "Affinity", "somevalue2", false, 0},
+		{&Header{"Affinity"}, "Affinity", "somevalue3", false, 2},
+		{&Header{"Affinity"}, "Affinity", "", true, 0},
+	}
+
+	for idx, test := range tests {
+		request, _ := http.NewRequest("GET", "/", nil)
+		if test.RequestHeaderName != "" {
+			request.Header.Add(test.RequestHeaderName, test.RequestHeaderValue)
+		}
+
+		host := test.Policy.Select(pool, request)
+		if test.NilHost && host != nil {
+			t.Errorf("%d: Expected host to be nil", idx)
+		}
+		if !test.NilHost && host == nil {
+			t.Errorf("%d: Did not expect host to be nil", idx)
+		}
+		if !test.NilHost && host != pool[test.HostIndex] {
+			t.Errorf("%d: Expected Header policy to be host %d", idx, test.HostIndex)
+		}
+	}
+}

caddyhttp/proxy/proxy.go

@@ -228,7 +228,7 @@ func (p Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 			return 0, nil
 		}
 
-		if _, ok := backendErr.(httpserver.MaxBytesExceeded); ok {
+		if backendErr == httpserver.ErrMaxBytesExceeded {
 			return http.StatusRequestEntityTooLarge, backendErr
 		}
 

caddyhttp/proxy/proxy_test.go

@@ -44,32 +44,62 @@ func TestReverseProxy(t *testing.T) {
 	log.SetOutput(ioutil.Discard)
 	defer log.SetOutput(os.Stderr)
 
-	verifyHeaders := func(headers http.Header, trailers http.Header) {
-		if headers.Get("X-Header") != "header-value" {
-			t.Error("Expected header 'X-Header' to be proxied properly")
+	testHeaderValue := []string{"header-value"}
+	testHeaders := http.Header{
+		"X-Header-1": testHeaderValue,
+		"X-Header-2": testHeaderValue,
+		"X-Header-3": testHeaderValue,
+	}
+	testTrailerValue := []string{"trailer-value"}
+	testTrailers := http.Header{
+		"X-Trailer-1": testTrailerValue,
+		"X-Trailer-2": testTrailerValue,
+		"X-Trailer-3": testTrailerValue,
+	}
+	verifyHeaderValues := func(actual http.Header, expected http.Header) bool {
+		if actual == nil {
+			t.Error("Expected headers")
+			return true
 		}
 
-		if trailers == nil {
-			t.Error("Expected to receive trailers")
+		for k := range expected {
+			if expected.Get(k) != actual.Get(k) {
+				t.Errorf("Expected header '%s' to be proxied properly", k)
+				return true
+			}
 		}
-		if trailers.Get("X-Trailer") != "trailer-value" {
-			t.Error("Expected header 'X-Trailer' to be proxied properly")
+
+		return false
+	}
+	verifyHeadersTrailers := func(headers http.Header, trailers http.Header) {
+		if verifyHeaderValues(headers, testHeaders) || verifyHeaderValues(trailers, testTrailers) {
+			t.FailNow()
 		}
 	}
 
-	var requestReceived bool
+	requestReceived := false
 	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		// read the body (even if it's empty) to make Go parse trailers
 		io.Copy(ioutil.Discard, r.Body)
-		verifyHeaders(r.Header, r.Trailer)
 
+		verifyHeadersTrailers(r.Header, r.Trailer)
 		requestReceived = true
 
-		w.Header().Set("Trailer", "X-Trailer")
-		w.Header().Set("X-Header", "header-value")
+		// Set headers.
+		copyHeader(w.Header(), testHeaders)
+
+		// Only announce one of the trailers to test wether
+		// unannounced trailers are proxied correctly.
+		for k := range testTrailers {
+			w.Header().Set("Trailer", k)
+			break
+		}
+
 		w.WriteHeader(http.StatusOK)
 		w.Write([]byte("Hello, client"))
-		w.Header().Set("X-Trailer", "trailer-value")
+
+		// Set trailers.
+		shallowCopyTrailers(w.Header(), testTrailers, true)
 	}))
 	defer backend.Close()
 
@@ -79,28 +109,43 @@ func TestReverseProxy(t *testing.T) {
 		Upstreams: []Upstream{newFakeUpstream(backend.URL, false)},
 	}
 
-	// create request and response recorder
-	r := httptest.NewRequest("GET", "/", strings.NewReader("test"))
-	w := httptest.NewRecorder()
-
-	r.ContentLength = -1 // force chunked encoding (required for trailers)
-	r.Header.Set("X-Header", "header-value")
-	r.Trailer = map[string][]string{
-		"X-Trailer": {"trailer-value"},
+	// Create the fake request body.
+	// This will copy "trailersToSet" to r.Trailer right before it is closed and
+	// thus test for us wether unannounced client trailers are proxied correctly.
+	body := &trailerTestStringReader{
+		Reader:        *strings.NewReader("test"),
+		trailersToSet: testTrailers,
 	}
 
+	// Create the fake request with the above body.
+	r := httptest.NewRequest("GET", "/", body)
+	r.Trailer = make(http.Header)
+	body.request = r
+
+	copyHeader(r.Header, testHeaders)
+
+	// Only announce one of the trailers to test wether
+	// unannounced trailers are proxied correctly.
+	for k, v := range testTrailers {
+		r.Trailer[k] = v
+		break
+	}
+
+	w := httptest.NewRecorder()
 	p.ServeHTTP(w, r)
+	res := w.Result()
 
 	if !requestReceived {
 		t.Error("Expected backend to receive request, but it didn't")
 	}
 
-	res := w.Result()
-	verifyHeaders(res.Header, res.Trailer)
+	verifyHeadersTrailers(res.Header, res.Trailer)
 
 	// Make sure {upstream} placeholder is set
 	r.Body = ioutil.NopCloser(strings.NewReader("test"))
-	rr := httpserver.NewResponseRecorder(testResponseRecorder{httptest.NewRecorder()})
+	rr := httpserver.NewResponseRecorder(testResponseRecorder{
+		ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: httptest.NewRecorder()},
+	})
 	rr.Replacer = httpserver.NewReplacer(r, rr, "-")
 
 	p.ServeHTTP(rr, r)
@@ -110,6 +155,21 @@ func TestReverseProxy(t *testing.T) {
 	}
 }
 
+// trailerTestStringReader is used to test unannounced trailers coming
+// from a client which should properly be proxied to the upstream.
+type trailerTestStringReader struct {
+	strings.Reader
+	request       *http.Request
+	trailersToSet http.Header
+}
+
+var _ io.ReadCloser = &trailerTestStringReader{}
+
+func (r *trailerTestStringReader) Close() error {
+	copyHeader(r.request.Trailer, r.trailersToSet)
+	return nil
+}
+
 func TestReverseProxyInsecureSkipVerify(t *testing.T) {
 	log.SetOutput(ioutil.Discard)
 	defer log.SetOutput(os.Stderr)
@@ -245,8 +305,8 @@ func TestWebSocketReverseProxyNonHijackerPanic(t *testing.T) {
 func TestWebSocketReverseProxyServeHTTPHandler(t *testing.T) {
 	// No-op websocket backend simply allows the WS connection to be
 	// accepted then it will be immediately closed. Perfect for testing.
-	var connCount int32
-	wsNop := httptest.NewServer(websocket.Handler(func(ws *websocket.Conn) { atomic.AddInt32(&connCount, 1) }))
+	accepted := make(chan struct{})
+	wsNop := httptest.NewServer(websocket.Handler(func(ws *websocket.Conn) { close(accepted) }))
 	defer wsNop.Close()
 
 	// Get proxy to use for the test
@@ -277,8 +337,14 @@ func TestWebSocketReverseProxyServeHTTPHandler(t *testing.T) {
 	if !bytes.Equal(actual, expected) {
 		t.Errorf("Expected backend to accept response:\n'%s'\nActually got:\n'%s'", expected, actual)
 	}
-	if got, want := atomic.LoadInt32(&connCount), int32(1); got != want {
-		t.Errorf("Expected %d websocket connection, got %d", want, got)
+
+	// wait a minute for backend handling, see issue 1654.
+	time.Sleep(10 * time.Millisecond)
+
+	select {
+	case <-accepted:
+	default:
+		t.Error("Expect a accepted websocket connection, but not")
 	}
 }
 
@@ -1315,24 +1381,13 @@ func (c *fakeConn) Write(b []byte) (int, error)        { return c.writeBuf.Write
 // testResponseRecorder wraps `httptest.ResponseRecorder`,
 // also implements `http.CloseNotifier`, `http.Hijacker` and `http.Pusher`.
 type testResponseRecorder struct {
-	*httptest.ResponseRecorder
+	*httpserver.ResponseWriterWrapper
 }
 
 func (testResponseRecorder) CloseNotify() <-chan bool { return nil }
-func (t testResponseRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	return nil, nil, httpserver.NonHijackerError{Underlying: t}
-}
-func (t testResponseRecorder) Push(target string, opts *http.PushOptions) error {
-	return httpserver.NonPusherError{Underlying: t}
-}
 
 // Interface guards
-var (
-	_ http.Pusher        = testResponseRecorder{}
-	_ http.Flusher       = testResponseRecorder{}
-	_ http.CloseNotifier = testResponseRecorder{}
-	_ http.Hijacker      = testResponseRecorder{}
-)
+var _ httpserver.HTTPInterfaces = testResponseRecorder{}
 
 func BenchmarkProxy(b *testing.B) {
 	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -1367,3 +1422,51 @@ func BenchmarkProxy(b *testing.B) {
 		p.ServeHTTP(w, r)
 	}
 }
+
+func TestChunkedWebSocketReverseProxy(t *testing.T) {
+	s := websocket.Server{
+		Handler: websocket.Handler(func(ws *websocket.Conn) {
+			for {
+				select {}
+			}
+		}),
+	}
+	s.Config.Header = http.Header(make(map[string][]string))
+	s.Config.Header.Set("Transfer-Encoding", "chunked")
+
+	wsNop := httptest.NewServer(s)
+	defer wsNop.Close()
+
+	// Get proxy to use for the test
+	p := newWebSocketTestProxy(wsNop.URL, false)
+
+	// Create client request
+	r := httptest.NewRequest("GET", "/", nil)
+
+	r.Header = http.Header{
+		"Connection":            {"Upgrade"},
+		"Upgrade":               {"websocket"},
+		"Origin":                {wsNop.URL},
+		"Sec-WebSocket-Key":     {"x3JJHMbDL1EzLkh9GBhXDw=="},
+		"Sec-WebSocket-Version": {"13"},
+	}
+
+	// Capture the request
+	w := &recorderHijacker{httptest.NewRecorder(), new(fakeConn)}
+
+	// Booya! Do the test.
+	_, err := p.ServeHTTP(w, r)
+
+	// Make sure the backend accepted the WS connection.
+	// Mostly interested in the Upgrade and Connection response headers
+	// and the 101 status code.
+	expected := []byte("HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=\r\nTransfer-Encoding: chunked\r\n\r\n")
+	actual := w.fakeConn.writeBuf.Bytes()
+	if !bytes.Equal(actual, expected) {
+		t.Errorf("Expected backend to accept response:\n'%s'\nActually got:\n'%s'", expected, actual)
+	}
+
+	if err != nil {
+		t.Error(err)
+	}
+}

caddyhttp/proxy/reverseproxy.go

@@ -272,7 +272,7 @@ func (rp *ReverseProxy) ServeHTTP(rw http.ResponseWriter, outreq *http.Request,
 	}
 
 	if isWebsocket {
-		res.Body.Close()
+		defer res.Body.Close()
 		hj, ok := rw.(http.Hijacker)
 		if !ok {
 			panic(httpserver.NonHijackerError{Underlying: rw})
@@ -318,30 +318,61 @@ func (rp *ReverseProxy) ServeHTTP(rw http.ResponseWriter, outreq *http.Request,
 		}
 		pooledIoCopy(backendConn, conn)
 	} else {
+		// NOTE:
+		//   Closing the Body involves acquiring a mutex, which is a
+		//   unnecessarily heavy operation, considering that this defer will
+		//   pretty much never be executed with the Body still unclosed.
+		bodyOpen := true
+		closeBody := func() {
+			if bodyOpen {
+				res.Body.Close()
+				bodyOpen = false
+			}
+		}
+		defer closeBody()
+
+		// Copy all headers over.
+		// res.Header does not include the "Trailer" header,
+		// which means we will have to do that manually below.
 		copyHeader(rw.Header(), res.Header)
 
-		// The "Trailer" header isn't included in the Transport's response,
-		// at least for *http.Transport. Build it up from Trailer.
-		if len(res.Trailer) > 0 {
-			trailerKeys := make([]string, 0, len(res.Trailer))
+		// The "Trailer" header isn't included in res' Header map, which
+		// is why we have to build one ourselves from res.Trailer.
+		//
+		// But res.Trailer does not necessarily contain all trailer keys at this
+		// point yet. The HTTP spec allows one to send "unannounced trailers"
+		// after a request and certain systems like gRPC make use of that.
+		announcedTrailerKeyCount := len(res.Trailer)
+		if announcedTrailerKeyCount > 0 {
+			vv := make([]string, 0, announcedTrailerKeyCount)
 			for k := range res.Trailer {
-				trailerKeys = append(trailerKeys, k)
+				vv = append(vv, k)
 			}
-			rw.Header().Add("Trailer", strings.Join(trailerKeys, ", "))
+			rw.Header()["Trailer"] = vv
 		}
 
+		// Now copy over the status code as well as the response body.
 		rw.WriteHeader(res.StatusCode)
-		if len(res.Trailer) > 0 {
+		if announcedTrailerKeyCount > 0 {
 			// Force chunking if we saw a response trailer.
-			// This prevents net/http from calculating the length for short
-			// bodies and adding a Content-Length.
+			// This prevents net/http from calculating the length
+			// for short bodies and adding a Content-Length.
 			if fl, ok := rw.(http.Flusher); ok {
 				fl.Flush()
 			}
 		}
 		rp.copyResponse(rw, res.Body)
-		res.Body.Close() // close now, instead of defer, to populate res.Trailer
-		copyHeader(rw.Header(), res.Trailer)
+
+		// Now close the body to fully populate res.Trailer.
+		closeBody()
+
+		// Since Go does not remove keys from res.Trailer we
+		// can safely do a length comparison to check wether
+		// we received further, unannounced trailers.
+		//
+		// Most of the time forceSetTrailers should be false.
+		forceSetTrailers := len(res.Trailer) != announcedTrailerKeyCount
+		shallowCopyTrailers(rw.Header(), res.Trailer, forceSetTrailers)
 	}
 
 	return nil
@@ -382,8 +413,12 @@ func copyHeader(dst, src http.Header) {
 			if _, shouldSkip := skipHeaders[k]; shouldSkip {
 				continue
 			}
-			// otherwise, overwrite
-			dst.Del(k)
+			// otherwise, overwrite to avoid duplicated fields that can be
+			// problematic (see issue #1086) -- however, allow duplicate
+			// Server fields so we can see the reality of the proxying.
+			if k != "Server" {
+				dst.Del(k)
+			}
 		}
 		for _, v := range vv {
 			dst.Add(k, v)
@@ -391,6 +426,22 @@ func copyHeader(dst, src http.Header) {
 	}
 }
 
+// shallowCopyTrailers copies all headers from srcTrailer to dstHeader.
+//
+// If forceSetTrailers is set to true, the http.TrailerPrefix will be added to
+// all srcTrailer key names. Otherwise the Go stdlib will ignore all keys
+// which weren't listed in the Trailer map before submitting the Response.
+//
+// WARNING: Only a shallow copy will be created!
+func shallowCopyTrailers(dstHeader, srcTrailer http.Header, forceSetTrailers bool) {
+	for k, vv := range srcTrailer {
+		if forceSetTrailers {
+			k = http.TrailerPrefix + k
+		}
+		dstHeader[k] = vv
+	}
+}
+
 // Hop-by-hop headers. These are removed when sent to the backend.
 // http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
 var hopHeaders = []string{

caddyhttp/proxy/upstream.go

@@ -1,9 +1,11 @@
 package proxy
 
 import (
+	"bytes"
 	"fmt"
 	"io"
 	"io/ioutil"
+	"net"
 	"net/http"
 	"net/url"
 	"path"
@@ -20,7 +22,7 @@ import (
 )
 
 var (
-	supportedPolicies = make(map[string]func() Policy)
+	supportedPolicies = make(map[string]func(string) Policy)
 )
 
 type staticUpstream struct {
@@ -37,11 +39,13 @@ type staticUpstream struct {
 	TryInterval       time.Duration
 	MaxConns          int64
 	HealthCheck       struct {
-		Client   http.Client
-		Path     string
-		Interval time.Duration
-		Timeout  time.Duration
-		Host     string
+		Client        http.Client
+		Path          string
+		Interval      time.Duration
+		Timeout       time.Duration
+		Host          string
+		Port          string
+		ContentString string
 	}
 	WithoutPathPrefix  string
 	IgnoredSubPaths    []string
@@ -239,7 +243,11 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 		if !ok {
 			return c.ArgErr()
 		}
-		u.Policy = policyCreateFunc()
+		arg := ""
+		if c.NextArg() {
+			arg = c.Val()
+		}
+		u.Policy = policyCreateFunc(arg)
 	case "fail_timeout":
 		if !c.NextArg() {
 			return c.ArgErr()
@@ -321,6 +329,25 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 			return err
 		}
 		u.HealthCheck.Timeout = dur
+	case "health_check_port":
+		if !c.NextArg() {
+			return c.ArgErr()
+		}
+		port := c.Val()
+		n, err := strconv.Atoi(port)
+		if err != nil {
+			return err
+		}
+
+		if n < 0 {
+			return c.Errf("invalid health_check_port '%s'", port)
+		}
+		u.HealthCheck.Port = port
+	case "health_check_contains":
+		if !c.NextArg() {
+			return c.ArgErr()
+		}
+		u.HealthCheck.ContentString = c.Val()
 	case "header_upstream":
 		var header, value string
 		if !c.Args(&header, &value) {
@@ -380,28 +407,48 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 
 func (u *staticUpstream) healthCheck() {
 	for _, host := range u.Hosts {
-		hostURL := host.Name + u.HealthCheck.Path
-		var unhealthy bool
+		hostURL := host.Name
+		if u.HealthCheck.Port != "" {
+			hostURL = replacePort(host.Name, u.HealthCheck.Port)
+		}
+		hostURL += u.HealthCheck.Path
 
-		// set up request, needed to be able to modify headers
-		// possible errors are bad HTTP methods or un-parsable urls
-		req, err := http.NewRequest("GET", hostURL, nil)
-		if err != nil {
-			unhealthy = true
-		} else {
+		unhealthy := func() bool {
+			// set up request, needed to be able to modify headers
+			// possible errors are bad HTTP methods or un-parsable urls
+			req, err := http.NewRequest("GET", hostURL, nil)
+			if err != nil {
+				return true
+			}
 			// set host for request going upstream
 			if u.HealthCheck.Host != "" {
 				req.Host = u.HealthCheck.Host
 			}
-
-			if r, err := u.HealthCheck.Client.Do(req); err == nil {
+			r, err := u.HealthCheck.Client.Do(req)
+			if err != nil {
+				return true
+			}
+			defer func() {
 				io.Copy(ioutil.Discard, r.Body)
 				r.Body.Close()
-				unhealthy = r.StatusCode < 200 || r.StatusCode >= 400
-			} else {
-				unhealthy = true
+			}()
+			if r.StatusCode < 200 || r.StatusCode >= 400 {
+				return true
 			}
-		}
+			if u.HealthCheck.ContentString == "" { // don't check for content string
+				return false
+			}
+			// TODO ReadAll will be replaced if deemed necessary
+			//      See https://github.com/mholt/caddy/pull/1691
+			buf, err := ioutil.ReadAll(r.Body)
+			if err != nil {
+				return true
+			}
+			if bytes.Contains(buf, []byte(u.HealthCheck.ContentString)) {
+				return false
+			}
+			return true
+		}()
 		if unhealthy {
 			atomic.StoreInt32(&host.Unhealthy, 1)
 		} else {
@@ -480,6 +527,22 @@ func (u *staticUpstream) Stop() error {
 }
 
 // RegisterPolicy adds a custom policy to the proxy.
-func RegisterPolicy(name string, policy func() Policy) {
+func RegisterPolicy(name string, policy func(string) Policy) {
 	supportedPolicies[name] = policy
 }
+
+func replacePort(originalURL string, newPort string) string {
+	parsedURL, err := url.Parse(originalURL)
+	if err != nil {
+		return originalURL
+	}
+
+	// handles 'localhost' and 'localhost:8080'
+	parsedHost, _, err := net.SplitHostPort(parsedURL.Host)
+	if err != nil {
+		parsedHost = parsedURL.Host
+	}
+
+	parsedURL.Host = net.JoinHostPort(parsedHost, newPort)
+	return parsedURL.String()
+}

caddyhttp/proxy/upstream_test.go

@@ -2,6 +2,7 @@ package proxy
 
 import (
 	"fmt"
+	"net"
 	"net/http"
 	"net/http/httptest"
 	"strings"
@@ -105,7 +106,7 @@ func TestSelect(t *testing.T) {
 func TestRegisterPolicy(t *testing.T) {
 	name := "custom"
 	customPolicy := &customPolicy{}
-	RegisterPolicy(name, func() Policy { return customPolicy })
+	RegisterPolicy(name, func(string) Policy { return customPolicy })
 	if _, ok := supportedPolicies[name]; !ok {
 		t.Error("Expected supportedPolicies to have a custom policy.")
 	}
@@ -375,3 +376,128 @@ func TestHealthCheckHost(t *testing.T) {
 		}
 	}
 }
+
+func TestHealthCheckPort(t *testing.T) {
+	var counter int64
+
+	healthCounter := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		r.Body.Close()
+		atomic.AddInt64(&counter, 1)
+	}))
+
+	_, healthPort, err := net.SplitHostPort(healthCounter.Listener.Addr().String())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	defer healthCounter.Close()
+
+	tests := []struct {
+		config string
+	}{
+		// Test #1: upstream with port
+		{"proxy / localhost:8080 {\n health_check / health_check_port " + healthPort + "\n}"},
+
+		// Test #2: upstream without port (default to 80)
+		{"proxy / localhost {\n health_check / health_check_port " + healthPort + "\n}"},
+	}
+
+	for i, test := range tests {
+		counterValueAtStart := atomic.LoadInt64(&counter)
+		upstreams, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+		if err != nil {
+			t.Error("Expected no error. Got:", err.Error())
+		}
+
+		// Give some time for healthchecks to hit the server.
+		time.Sleep(500 * time.Millisecond)
+
+		for _, upstream := range upstreams {
+			if err := upstream.Stop(); err != nil {
+				t.Errorf("Test %d: Expected no error stopping upstream. Got: %v", i, err.Error())
+			}
+		}
+
+		counterValueAfterShutdown := atomic.LoadInt64(&counter)
+
+		if counterValueAfterShutdown == counterValueAtStart {
+			t.Errorf("Test %d: Expected healthchecks to hit test server. Got no healthchecks.", i)
+		}
+	}
+
+	t.Run("valid_port", func(t *testing.T) {
+		tests := []struct {
+			config string
+		}{
+			// Test #1: invalid port (nil)
+			{"proxy / localhost {\n health_check / health_check_port\n}"},
+
+			// Test #2: invalid port (string)
+			{"proxy / localhost {\n health_check / health_check_port abc\n}"},
+
+			// Test #3: invalid port (negative)
+			{"proxy / localhost {\n health_check / health_check_port -1\n}"},
+		}
+
+		for i, test := range tests {
+			_, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+			if err == nil {
+				t.Errorf("Test %d accepted invalid config", i)
+			}
+		}
+	})
+
+}
+
+func TestHealthCheckContentString(t *testing.T) {
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		fmt.Fprintf(w, "blablabla good blablabla")
+		r.Body.Close()
+	}))
+	_, port, err := net.SplitHostPort(server.Listener.Addr().String())
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer server.Close()
+
+	tests := []struct {
+		config        string
+		shouldContain bool
+	}{
+		{"proxy / localhost:" + port +
+			" { health_check /testhealth " +
+			" health_check_contains good\n}",
+			true,
+		},
+		{"proxy / localhost:" + port + " {\n health_check /testhealth health_check_port " + port +
+			" \n health_check_contains bad\n}",
+			false,
+		},
+	}
+	for i, test := range tests {
+		u, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+		if err != nil {
+			t.Errorf("Expected no error. Test %d Got: %s", i, err.Error())
+		}
+		for _, upstream := range u {
+			staticUpstream, ok := upstream.(*staticUpstream)
+			if !ok {
+				t.Errorf("Type mismatch: %#v", upstream)
+				continue
+			}
+			staticUpstream.healthCheck()
+			for _, host := range staticUpstream.Hosts {
+				if test.shouldContain && atomic.LoadInt32(&host.Unhealthy) == 0 {
+					// healthcheck url was hit and the required test string was found
+					continue
+				}
+				if !test.shouldContain && atomic.LoadInt32(&host.Unhealthy) != 0 {
+					// healthcheck url was hit and the required string was not found
+					continue
+				}
+				t.Errorf("Health check bad response")
+			}
+			upstream.Stop()
+		}
+	}
+}

caddyhttp/push/handler.go

@@ -5,6 +5,7 @@ import (
 	"strings"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
+	"github.com/mholt/caddy/caddyhttp/staticfiles"
 )
 
 func (h Middleware) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
@@ -25,7 +26,13 @@ func (h Middleware) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, erro
 	// push first
 outer:
 	for _, rule := range h.Rules {
-		if httpserver.Path(r.URL.Path).Matches(rule.Path) {
+		urlPath := r.URL.Path
+		matches := httpserver.Path(urlPath).Matches(rule.Path)
+		// Also check IndexPages when requesting a directory
+		if !matches {
+			_, matches = httpserver.IndexFile(h.Root, urlPath, staticfiles.IndexPages)
+		}
+		if matches {
 			for _, resource := range rule.Resources {
 				pushErr := pusher.Push(resource.Path, &http.PushOptions{
 					Method: resource.Method,

caddyhttp/push/handler_test.go

@@ -2,8 +2,11 @@ package push
 
 import (
 	"errors"
+	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
+	"os"
+	"path/filepath"
 	"reflect"
 	"testing"
 
@@ -307,6 +310,63 @@ func TestMiddlewareShouldInterceptLinkHeaderPusherError(t *testing.T) {
 	comparePushedResources(t, expectedPushedResources, pushingWriter.pushed)
 }
 
+func TestMiddlewareShouldPushIndexFile(t *testing.T) {
+	// given
+	indexFile := "/index.html"
+	request, err := http.NewRequest(http.MethodGet, "/", nil) // Request root directory, not indexfile itself
+	if err != nil {
+		t.Fatalf("Could not create HTTP request: %v", err)
+	}
+
+	root, err := ioutil.TempDir("", "caddy")
+	if err != nil {
+		t.Fatalf("Could not create temporary directory: %v", err)
+	}
+	defer os.Remove(root)
+
+	middleware := Middleware{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			return 0, nil
+		}),
+		Rules: []Rule{
+			{Path: indexFile, Resources: []Resource{
+				{Path: "/index.css", Method: http.MethodGet},
+			}},
+		},
+		Root: http.Dir(root),
+	}
+
+	indexFilePath := filepath.Join(root, indexFile)
+	_, err = os.Create(indexFilePath)
+	if err != nil {
+		t.Fatalf("Could not create index file: %s: %v", indexFile, err)
+	}
+	defer os.Remove(indexFilePath)
+
+	pushingWriter := &MockedPusher{
+		ResponseWriter: httptest.NewRecorder(),
+		returnedError:  errors.New("Cannot push right now"),
+	}
+
+	// when
+	_, err2 := middleware.ServeHTTP(pushingWriter, request)
+
+	// then
+	if err2 != nil {
+		t.Error("Should not return error")
+	}
+
+	expectedPushedResources := map[string]*http.PushOptions{
+		"/index.css": {
+			Method: http.MethodGet,
+			Header: http.Header{},
+		},
+	}
+
+	comparePushedResources(t, expectedPushedResources, pushingWriter.pushed)
+
+}
+
 func comparePushedResources(t *testing.T, expected, actual map[string]*http.PushOptions) {
 	if len(expected) != len(actual) {
 		t.Errorf("Expected %d pushed resources, actual: %d", len(expected), len(actual))

caddyhttp/push/push.go

@@ -24,6 +24,7 @@ type (
 	Middleware struct {
 		Next  httpserver.Handler
 		Rules []Rule
+		Root  http.FileSystem
 	}
 
 	ruleOp func([]Resource)

caddyhttp/push/setup.go

@@ -34,8 +34,9 @@ func setup(c *caddy.Controller) error {
 		return err
 	}
 
-	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
-		return Middleware{Next: next, Rules: rules}
+	cfg := httpserver.GetConfig(c)
+	cfg.AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
+		return Middleware{Next: next, Rules: rules, Root: http.Dir(cfg.Root)}
 	})
 
 	return nil

caddyhttp/requestid/requestid.go

@@ -0,0 +1,34 @@
+package requestid
+
+import (
+	"context"
+	"log"
+	"net/http"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+	uuid "github.com/nu7hatch/gouuid"
+)
+
+// Handler is a middleware handler
+type Handler struct {
+	Next httpserver.Handler
+}
+
+func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
+	reqid := UUID()
+	c := context.WithValue(r.Context(), httpserver.RequestIDCtxKey, reqid)
+	r = r.WithContext(c)
+
+	return h.Next.ServeHTTP(w, r)
+}
+
+// UUID returns U4 UUID
+func UUID() string {
+	u4, err := uuid.NewV4()
+	if err != nil {
+		log.Printf("[ERROR] generating request ID: %v", err)
+		return ""
+	}
+
+	return u4.String()
+}

caddyhttp/requestid/requestid_test.go

@@ -0,0 +1,33 @@
+package requestid
+
+import (
+	"context"
+	"net/http"
+	"testing"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func TestRequestID(t *testing.T) {
+	request, err := http.NewRequest("GET", "http://localhost/", nil)
+	if err != nil {
+		t.Fatal("Could not create HTTP request:", err)
+	}
+
+	reqid := UUID()
+
+	c := context.WithValue(request.Context(), httpserver.RequestIDCtxKey, reqid)
+
+	request = request.WithContext(c)
+
+	// See caddyhttp/replacer.go
+	value, _ := request.Context().Value(httpserver.RequestIDCtxKey).(string)
+
+	if value == "" {
+		t.Fatal("Request ID should not be empty")
+	}
+
+	if value != reqid {
+		t.Fatal("Request ID does not match")
+	}
+}

caddyhttp/requestid/setup.go

@@ -0,0 +1,27 @@
+package requestid
+
+import (
+	"github.com/mholt/caddy"
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func init() {
+	caddy.RegisterPlugin("request_id", caddy.Plugin{
+		ServerType: "http",
+		Action:     setup,
+	})
+}
+
+func setup(c *caddy.Controller) error {
+	for c.Next() {
+		if c.NextArg() {
+			return c.ArgErr() //no arg expected.
+		}
+	}
+
+	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
+		return Handler{Next: next}
+	})
+
+	return nil
+}

caddyhttp/requestid/setup_test.go

@@ -0,0 +1,43 @@
+package requestid
+
+import (
+	"testing"
+
+	"github.com/mholt/caddy"
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func TestSetup(t *testing.T) {
+	c := caddy.NewTestController("http", `requestid`)
+	err := setup(c)
+	if err != nil {
+		t.Errorf("Expected no errors, got: %v", err)
+	}
+	mids := httpserver.GetConfig(c).Middleware()
+	if len(mids) == 0 {
+		t.Fatal("Expected middleware, got 0 instead")
+	}
+
+	handler := mids[0](httpserver.EmptyNext)
+	myHandler, ok := handler.(Handler)
+
+	if !ok {
+		t.Fatalf("Expected handler to be type Handler, got: %#v", handler)
+	}
+
+	if !httpserver.SameNext(myHandler.Next, httpserver.EmptyNext) {
+		t.Error("'Next' field of handler was not set properly")
+	}
+}
+
+func TestSetupWithArg(t *testing.T) {
+	c := caddy.NewTestController("http", `requestid abc`)
+	err := setup(c)
+	if err == nil {
+		t.Errorf("Expected an error, got: %v", err)
+	}
+	mids := httpserver.GetConfig(c).Middleware()
+	if len(mids) != 0 {
+		t.Fatal("Expected no middleware")
+	}
+}

caddyhttp/rewrite/setup.go

@@ -76,10 +76,6 @@ func rewriteParse(c *caddy.Controller) ([]httpserver.HandlerConfig, error) {
 						return nil, c.ArgErr()
 					}
 					to = strings.Join(args1, " ")
-					// ensure rewrite path begins with /
-					if !strings.HasPrefix(to, "/") {
-						return nil, c.Errf("%s:%d - Syntax error: Rewrite path must begin with '/'. Provided: '%s'", c.File(), c.Line(), c.Val())
-					}
 				case "ext":
 					args1 := c.RemainingArgs()
 					if len(args1) == 0 {
@@ -94,20 +90,14 @@ func rewriteParse(c *caddy.Controller) ([]httpserver.HandlerConfig, error) {
 			if to == "" {
 				return nil, c.ArgErr()
 			}
-
 			if rule, err = NewComplexRule(base, pattern, to, ext, matcher); err != nil {
 				return nil, err
 			}
 			rules = append(rules, rule)
 
-		// handle case of 2 arguments: "from to"
+		// the only unhandled case is 2 and above
 		default:
-			// ensure rewrite path begins with /
-			topath := strings.Join(args[1:], " ")
-			if !strings.HasPrefix(topath, "/") {
-				return nil, c.Errf("%s:%d - Syntax error: Rewrite path must begin with '/'. Provided: '%s'", c.File(), c.Line(), c.Val())
-			}
-			rule = NewSimpleRule(args[0], topath)
+			rule = NewSimpleRule(args[0], strings.Join(args[1:], " "))
 			rules = append(rules, rule)
 		}
 

caddyhttp/rewrite/setup_test.go

@@ -45,19 +45,15 @@ func TestRewriteParse(t *testing.T) {
 			SimpleRule{From: "/from", To: "/to"},
 		}},
 		{`rewrite /from /to
-		  rewrite a /b`, false, []Rule{
+		  rewrite a b`, false, []Rule{
 			SimpleRule{From: "/from", To: "/to"},
-			SimpleRule{From: "a", To: "/b"},
+			SimpleRule{From: "a", To: "b"},
 		}},
-		{`rewrite a b`, true, []Rule{}},
 		{`rewrite a`, true, []Rule{}},
 		{`rewrite`, true, []Rule{}},
-		{`rewrite a b c`, true, []Rule{
+		{`rewrite a b c`, false, []Rule{
 			SimpleRule{From: "a", To: "b c"},
 		}},
-		{`rewrite a /b c`, false, []Rule{
-			SimpleRule{From: "a", To: "/b c"},
-		}},
 	}
 
 	for i, test := range simpleTests {

caddyhttp/staticfiles/fileserver.go

@@ -7,7 +7,6 @@ package staticfiles
 import (
 	"math/rand"
 	"net/http"
-	"net/url"
 	"os"
 	"path"
 	"path/filepath"
@@ -82,37 +81,41 @@ func (fs FileServer) serveFile(w http.ResponseWriter, r *http.Request) (int, err
 
 	// redirect to canonical path (being careful to preserve other parts of URL and
 	// considering cases where a site is defined with a path prefix that gets stripped)
-	u := r.Context().Value(caddy.CtxKey("original_url")).(url.URL)
-	if u.Path == "" {
-		u.Path = "/"
+	urlCopy := *r.URL
+	pathPrefix, _ := r.Context().Value(caddy.CtxKey("path_prefix")).(string)
+	if pathPrefix != "/" {
+		urlCopy.Path = pathPrefix + urlCopy.Path
+	}
+	if urlCopy.Path == "" {
+		urlCopy.Path = "/"
 	}
 	if d.IsDir() {
 		// ensure there is a trailing slash
-		if u.Path[len(u.Path)-1] != '/' {
-			u.Path += "/"
-			http.Redirect(w, r, u.String(), http.StatusMovedPermanently)
+		if urlCopy.Path[len(urlCopy.Path)-1] != '/' {
+			urlCopy.Path += "/"
+			http.Redirect(w, r, urlCopy.String(), http.StatusMovedPermanently)
 			return http.StatusMovedPermanently, nil
 		}
 	} else {
 		// ensure no trailing slash
 		redir := false
-		if u.Path[len(u.Path)-1] == '/' {
-			u.Path = u.Path[:len(u.Path)-1]
+		if urlCopy.Path[len(urlCopy.Path)-1] == '/' {
+			urlCopy.Path = urlCopy.Path[:len(urlCopy.Path)-1]
 			redir = true
 		}
 
 		// if an index file was explicitly requested, strip file name from the request
 		// ("/foo/index.html" -> "/foo/")
 		for _, indexPage := range IndexPages {
-			if strings.HasSuffix(u.Path, indexPage) {
-				u.Path = u.Path[:len(u.Path)-len(indexPage)]
+			if strings.HasSuffix(urlCopy.Path, indexPage) {
+				urlCopy.Path = urlCopy.Path[:len(urlCopy.Path)-len(indexPage)]
 				redir = true
 				break
 			}
 		}
 
 		if redir {
-			http.Redirect(w, r, u.String(), http.StatusMovedPermanently)
+			http.Redirect(w, r, urlCopy.String(), http.StatusMovedPermanently)
 			return http.StatusMovedPermanently, nil
 		}
 	}

caddyhttp/staticfiles/fileserver_test.go

@@ -230,9 +230,11 @@ func TestServeHTTP(t *testing.T) {
 			continue
 		}
 
-		// set the original URL on the context
+		// set the original URL and path prefix on the context
 		ctx := context.WithValue(request.Context(), caddy.CtxKey("original_url"), *request.URL)
 		request = request.WithContext(ctx)
+		ctx = context.WithValue(request.Context(), caddy.CtxKey("path_prefix"), test.stripPathPrefix)
+		request = request.WithContext(ctx)
 
 		request.Header.Add("Accept-Encoding", test.acceptEncoding)
 

caddytls/config.go

@@ -9,6 +9,7 @@ import (
 	"net/url"
 	"strings"
 
+	"github.com/codahale/aesnicheck"
 	"github.com/mholt/caddy"
 	"github.com/xenolf/lego/acme"
 )
@@ -294,7 +295,7 @@ func (c *Config) buildStandardTLSConfig() error {
 
 	// default cipher suites
 	if len(config.CipherSuites) == 0 {
-		config.CipherSuites = defaultCiphers
+		config.CipherSuites = getPreferredDefaultCiphers()
 	}
 
 	// for security, ensure TLS_FALLBACK_SCSV is always included first
@@ -380,7 +381,7 @@ func RegisterConfigGetter(serverType string, fn ConfigGetter) {
 func SetDefaultTLSParams(config *Config) {
 	// If no ciphers provided, use default list
 	if len(config.Ciphers) == 0 {
-		config.Ciphers = defaultCiphers
+		config.Ciphers = getPreferredDefaultCiphers()
 	}
 
 	// Not a cipher suite, but still important for mitigating protocol downgrade attacks
@@ -464,6 +465,35 @@ var defaultCiphers = []uint16{
 	tls.TLS_RSA_WITH_AES_128_CBC_SHA,
 }
 
+// List of ciphers we should prefer if native AESNI support is missing
+var defaultCiphersNonAESNI = []uint16{
+	tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
+	tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
+	tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
+	tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
+	tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
+	tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
+	tls.TLS_RSA_WITH_AES_256_CBC_SHA,
+	tls.TLS_RSA_WITH_AES_128_CBC_SHA,
+}
+
+// getPreferredDefaultCiphers returns an appropriate cipher suite to use, depending on
+// the hardware support available for AES-NI.
+//
+// See https://github.com/mholt/caddy/issues/1674
+func getPreferredDefaultCiphers() []uint16 {
+	if aesnicheck.HasAESNI() {
+		return defaultCiphers
+	}
+
+	// Return a cipher suite that prefers ChaCha20
+	return defaultCiphersNonAESNI
+}
+
 // Map of supported curves
 // https://golang.org/pkg/crypto/tls/#CurveID
 var supportedCurvesMap = map[string]tls.CurveID{

caddytls/config_test.go

@@ -6,6 +6,8 @@ import (
 	"net/url"
 	"reflect"
 	"testing"
+
+	"github.com/codahale/aesnicheck"
 )
 
 func TestConvertTLSConfigProtocolVersions(t *testing.T) {
@@ -60,10 +62,11 @@ func TestConvertTLSConfigCipherSuites(t *testing.T) {
 		{Enabled: true, Ciphers: nil},
 	}
 
+	defaultCiphersExpected := getPreferredDefaultCiphers()
 	expectedCiphers := [][]uint16{
 		{tls.TLS_FALLBACK_SCSV, 0xc02c, 0xc030},
 		{tls.TLS_FALLBACK_SCSV, 0xc012, 0xc030, 0xc00a},
-		append([]uint16{tls.TLS_FALLBACK_SCSV}, defaultCiphers...),
+		append([]uint16{tls.TLS_FALLBACK_SCSV}, defaultCiphersExpected...),
 	}
 
 	for i, config := range configs {
@@ -79,6 +82,21 @@ func TestConvertTLSConfigCipherSuites(t *testing.T) {
 	}
 }
 
+func TestGetPreferredDefaultCiphers(t *testing.T) {
+	expectedCiphers := defaultCiphers
+	if !aesnicheck.HasAESNI() {
+		expectedCiphers = defaultCiphersNonAESNI
+	}
+
+	// Ensure ordering is correct and ciphers are what we expected.
+	result := getPreferredDefaultCiphers()
+	for i, actual := range result {
+		if actual != expectedCiphers[i] {
+			t.Errorf("Expected cipher in position %d to be %0x, got %0x", i, expectedCiphers[i], actual)
+		}
+	}
+}
+
 func TestStorageForNoURL(t *testing.T) {
 	c := &Config{}
 	if _, err := c.StorageFor(""); err == nil {

caddytls/maintain.go

@@ -25,6 +25,13 @@ const (
 	// RenewDurationBefore is how long before expiration to renew certificates.
 	RenewDurationBefore = (24 * time.Hour) * 30
 
+	// RenewDurationBeforeAtStartup is how long before expiration to require
+	// a renewed certificate when the process is first starting up (see #1680).
+	// A wider window between RenewDurationBefore and this value will allow
+	// Caddy to start under duress but hopefully this duration will give it
+	// enough time for the blockage to be relieved.
+	RenewDurationBeforeAtStartup = (24 * time.Hour) * 7
+
 	// OCSPInterval is how often to check if OCSP stapling needs updating.
 	OCSPInterval = 1 * time.Hour
 )
@@ -126,13 +133,17 @@ func RenewManagedCertificates(allowPrompts bool) (err error) {
 		err := cert.Config.RenewCert(renewName, allowPrompts)
 		if err != nil {
 			if allowPrompts {
-				// Certificate renewal failed and the operator is present; we should stop
-				// immediately and return the error. See a discussion in issue 642
-				// about this. For a while, we only stopped if the certificate was
-				// expired, but in reality, there is no difference between reporting
-				// it now versus later, except that there's somebody present to deal
-				// with it now, so require it.
-				return err
+				// Certificate renewal failed and the operator is present. See a discussion
+				// about this in issue 642. For a while, we only stopped if the certificate
+				// was expired, but in reality, there is no difference between reporting
+				// it now versus later, except that there's somebody present to deal with
+				// it right now.
+				timeLeft := cert.NotAfter.Sub(time.Now().UTC())
+				if timeLeft < RenewDurationBeforeAtStartup {
+					// See issue 1680. Only fail at startup if the certificate is dangerously
+					// close to expiration.
+					return err
+				}
 			}
 			log.Printf("[ERROR] %v", err)
 			if cert.Config.OnDemand {

caddytls/setup.go

@@ -66,6 +66,12 @@ func setupTLS(c *caddy.Controller) error {
 		for c.NextBlock() {
 			hadBlock = true
 			switch c.Val() {
+			case "ca":
+				arg := c.RemainingArgs()
+				if len(arg) != 1 {
+					return c.ArgErr()
+				}
+				config.CAUrl = arg[0]
 			case "key_type":
 				arg := c.RemainingArgs()
 				value, ok := supportedKeyTypes[strings.ToUpper(arg[0])]

caddytls/setup_test.go

@@ -58,21 +58,7 @@ func TestSetupParseBasic(t *testing.T) {
 	}
 
 	// Cipher checks
-	expectedCiphers := []uint16{
-		tls.TLS_FALLBACK_SCSV,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
-		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
-		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
-		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
-		tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
-		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
-		tls.TLS_RSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_RSA_WITH_AES_128_CBC_SHA,
-	}
+	expectedCiphers := append([]uint16{tls.TLS_FALLBACK_SCSV}, getPreferredDefaultCiphers()...)
 
 	// Ensure count is correct (plus one for TLS_FALLBACK_SCSV)
 	if len(cfg.Ciphers) != len(expectedCiphers) {
@@ -291,6 +277,46 @@ func TestSetupParseWithClientAuth(t *testing.T) {
 	}
 }
 
+func TestSetupParseWithCAUrl(t *testing.T) {
+	testURL := "https://acme-staging.api.letsencrypt.org/directory"
+	for caseNumber, caseData := range []struct {
+		params        string
+		expectedErr   bool
+		expectedCAUrl string
+	}{
+		// Test working case
+		{`tls {
+				ca ` + testURL + `
+			}`, false, testURL},
+		// Test too few args
+		{`tls {
+				ca
+			}`, true, ""},
+		// Test too many args
+		{`tls {
+				ca 1 2
+			}`, true, ""},
+	} {
+		cfg := new(Config)
+		RegisterConfigGetter("", func(c *caddy.Controller) *Config { return cfg })
+		c := caddy.NewTestController("", caseData.params)
+		err := setupTLS(c)
+		if caseData.expectedErr {
+			if err == nil {
+				t.Errorf("In case %d: Expected an error, got: %v", caseNumber, err)
+			}
+			continue
+		}
+		if err != nil {
+			t.Errorf("In case %d: Expected no errors, got: %v", caseNumber, err)
+		}
+
+		if cfg.CAUrl != caseData.expectedCAUrl {
+			t.Errorf("Expected '%v' as CAUrl, got %#v", caseData.expectedCAUrl, cfg.CAUrl)
+		}
+	}
+}
+
 func TestSetupParseWithKeyType(t *testing.T) {
 	params := `tls {
             key_type p384

caddytls/tls.go

@@ -140,9 +140,18 @@ func QualifiesForManagedTLS(c ConfigHolder) bool {
 		(HostQualifies(c.Host()) || tlsConfig.OnDemand)
 }
 
+// ChallengeProvider defines an own type that should be used in Caddy plugins
+// over acme.ChallengeProvider. Using acme.ChallengeProvider causes version mismatches
+// with vendored dependencies (see https://github.com/mattfarina/golang-broken-vendor)
+//
+// acme.ChallengeProvider is an interface that allows the implementation of custom
+// challenge providers. For more details, see:
+// https://godoc.org/github.com/xenolf/lego/acme#ChallengeProvider
+type ChallengeProvider acme.ChallengeProvider
+
 // DNSProviderConstructor is a function that takes credentials and
 // returns a type that can solve the ACME DNS challenges.
-type DNSProviderConstructor func(credentials ...string) (acme.ChallengeProvider, error)
+type DNSProviderConstructor func(credentials ...string) (ChallengeProvider, error)
 
 // dnsProviders is the list of DNS providers that have been plugged in.
 var dnsProviders = make(map[string]DNSProviderConstructor)

dist/CHANGES.txt

@@ -1,5 +1,48 @@
 CHANGES
 
+0.10.5 (July 27, 2017)
+- Renamed requestid directive to request_id
+- Set default idle timeout of 5 minutes
+- New 3rd-party plugin directives: cache, nobots, webdav
+- New Unix timestamp placeholder {when_unix}
+- Improved MITM detection on iOS clients
+- errors, log: Fix log rolling parsing
+- gzip: Convert any ETag header to weak etag
+- fastcgi: Reverted persistent connections (issue #1736)
+- proxy: Added header loaded balancing policy
+- proxy: Fix hang on chunked WebSockets (e.g. with HomeAssistant)
+- Several other bug fixes and minor internal improvements
+
+
+0.10.4 (June 28, 2017)
+- Vendor all dependencies
+- Improve MITM detection, add experimental Tor browser support
+- New requestid directive to add request IDs to each request
+- New HTTP plugins supported: authz, grpc, gopkg, reauth, restic
+- browse: Refreshed default UI and added symlink indicators
+- errors, log: Added rotate_compress directive to compress rolled logs
+- markdown: Template files loaded at each request instead of just once
+- proxy: Allow multiple Server header fields on downstream response
+- proxy: Perform health checks by body substring
+- rewrite,redir: Added 'not_starts_with' and 'not_ends_with' operators
+- tls: New ca subdirective to specify CA endpoint per-site
+- Several bug fixes
+
+
+0.10.3 (May 19, 2017)
+- Replace 'maxrequestbody' directive with 'limits' directive
+- proxy: Configurable port for health check
+- proxy: New load balance policy: uri_hash
+- templates: Renamed .Push context action to .AddLink
+- tls: Allow narrower certificate renewal window at startup (#1680)
+- tls: Prefer ChaCha20 if hardware does not have AES-NI
+
+
+0.10.2 (May 2, 2017)
+- Hot fix for rule paths of "/" so that they match every request
+- fastcgi: Match request paths that don't start with "/" even if rule does
+
+
 0.10.1 (May 1, 2017)
 - Reduced memory usage for gzip, templates, and MITM detection
 - Fixed automatic HTTP->HTTPS redirects for sites with wildcard labels

dist/README.txt

@@ -1,4 +1,4 @@
-CADDY 0.10.1
+CADDY 0.10.5
 
 Website
 	https://caddyserver.com

dist/automate.go

@@ -1,161 +0,0 @@
-package main
-
-import (
-	"fmt"
-	"log"
-	"os"
-	"os/exec"
-	"path/filepath"
-	"runtime"
-	"sync"
-
-	"github.com/mholt/archiver"
-)
-
-var buildScript, repoDir, mainDir, distDir, buildDir, releaseDir string
-
-func init() {
-	repoDir = filepath.Join(os.Getenv("GOPATH"), "src", "github.com", "mholt", "caddy")
-	mainDir = filepath.Join(repoDir, "caddy")
-	buildScript = filepath.Join(mainDir, "build.bash")
-	distDir = filepath.Join(repoDir, "dist")
-	buildDir = filepath.Join(distDir, "builds")
-	releaseDir = filepath.Join(distDir, "release")
-}
-
-func main() {
-	// First, clean up
-	err := os.RemoveAll(buildDir)
-	if err != nil {
-		log.Fatal(err)
-	}
-	err = os.RemoveAll(releaseDir)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Then set up
-	err = os.MkdirAll(buildDir, 0755)
-	if err != nil {
-		log.Fatal(err)
-	}
-	err = os.MkdirAll(releaseDir, 0755)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Perform builds and make archives in parallel; only as many
-	// goroutines as we have processors.
-	var wg sync.WaitGroup
-	var throttle = make(chan struct{}, numProcs())
-	for _, p := range platforms {
-		wg.Add(1)
-		throttle <- struct{}{}
-
-		if p.os == "" || p.arch == "" || p.archive == "" {
-			log.Fatalf("Platform OS, architecture, and archive format is required: %+v", p)
-		}
-
-		go func(p platform) {
-			defer wg.Done()
-			defer func() { <-throttle }()
-
-			fmt.Printf("== Building %s\n", p)
-
-			var baseFilename, binFilename string
-			baseFilename = fmt.Sprintf("caddy_%s_%s", p.os, p.arch)
-			if p.arch == "arm" {
-				baseFilename += p.arm
-			}
-			binFilename = baseFilename + p.binExt
-
-			binPath := filepath.Join(buildDir, binFilename)
-			archive := filepath.Join(releaseDir, fmt.Sprintf("%s.%s", baseFilename, p.archive))
-			archiveContents := append(distContents, binPath)
-
-			err := build(p, binPath)
-			if err != nil {
-				log.Fatal(err)
-			}
-
-			fmt.Printf("== Compressing %s\n", baseFilename)
-
-			if p.archive == "zip" {
-				err := archiver.Zip.Make(archive, archiveContents)
-				if err != nil {
-					log.Fatal(err)
-				}
-			} else if p.archive == "tar.gz" {
-				err := archiver.TarGz.Make(archive, archiveContents)
-				if err != nil {
-					log.Fatal(err)
-				}
-			}
-		}(p)
-	}
-
-	wg.Wait()
-}
-
-func build(p platform, out string) error {
-	cmd := exec.Command(buildScript, out)
-	cmd.Dir = mainDir
-	cmd.Env = os.Environ()
-	cmd.Env = append(cmd.Env, "CGO_ENABLED=0")
-	cmd.Env = append(cmd.Env, "GOOS="+p.os)
-	cmd.Env = append(cmd.Env, "GOARCH="+p.arch)
-	cmd.Env = append(cmd.Env, "GOARM="+p.arm)
-	cmd.Stderr = os.Stderr
-	return cmd.Run()
-}
-
-type platform struct {
-	os, arch, arm, binExt, archive string
-}
-
-func (p platform) String() string {
-	outStr := fmt.Sprintf("%s/%s", p.os, p.arch)
-	if p.arch == "arm" {
-		outStr += fmt.Sprintf(" (ARM v%s)", p.arm)
-	}
-	return outStr
-}
-
-func numProcs() int {
-	n := runtime.GOMAXPROCS(0)
-	if n == runtime.NumCPU() && n > 1 {
-		n--
-	}
-	return n
-}
-
-// See: https://golang.org/doc/install/source#environment
-// Not all supported platforms are listed since some are
-// problematic and we only build the most common ones.
-// These are just the pre-made, readily-available static
-// builds, and we can try to add more upon request if there
-// is enough demand.
-var platforms = []platform{
-	{os: "darwin", arch: "amd64", archive: "zip"},
-	{os: "freebsd", arch: "386", archive: "tar.gz"},
-	{os: "freebsd", arch: "amd64", archive: "tar.gz"},
-	{os: "freebsd", arch: "arm", arm: "7", archive: "tar.gz"},
-	{os: "linux", arch: "386", archive: "tar.gz"},
-	{os: "linux", arch: "amd64", archive: "tar.gz"},
-	{os: "linux", arch: "arm", arm: "7", archive: "tar.gz"},
-	{os: "linux", arch: "arm64", archive: "tar.gz"},
-	{os: "netbsd", arch: "386", archive: "tar.gz"},
-	{os: "netbsd", arch: "amd64", archive: "tar.gz"},
-	{os: "openbsd", arch: "386", archive: "tar.gz"},
-	{os: "openbsd", arch: "amd64", archive: "tar.gz"},
-	{os: "solaris", arch: "amd64", archive: "tar.gz"},
-	{os: "windows", arch: "386", binExt: ".exe", archive: "zip"},
-	{os: "windows", arch: "amd64", binExt: ".exe", archive: "zip"},
-}
-
-var distContents = []string{
-	filepath.Join(distDir, "init"),
-	filepath.Join(distDir, "CHANGES.txt"),
-	filepath.Join(distDir, "LICENSES.txt"),
-	filepath.Join(distDir, "README.txt"),
-}

dist/automate_test.go

@@ -1,15 +0,0 @@
-package main
-
-import (
-	"runtime"
-	"testing"
-)
-
-func TestNumProcs(t *testing.T) {
-	num := runtime.NumCPU()
-	n := numProcs()
-	if n > num || n < 1 {
-		t.Errorf("Expected numProcs() to return max(NumCPU-1, 1) or at least some "+
-			"reasonable value (depending on CI environment), but got n=%d (NumCPU=%d)", n, num)
-	}
-}

dist/init/freebsd/caddy

@@ -23,22 +23,35 @@
 # caddy_config_path (str):    Set to "/usr/local/www/Caddyfile" by default.
 #                             Defines the path for the configuration file caddy will load on boot
 #
-# caddy_run_user (str):       Set to "root" by default.
+# caddy_user (str):           Set to "root" by default.
 #                             Defines the user that caddy will run on
 #
+# caddy_group (str):  	      Set to "wheel" by default.
+#                             Defines the group that caddy files will be attached to
+#
+# caddy_logfile (str)	      Set to "/var/log/caddy.log" by default.
+#			      Defines where the process log file is written, this is not a web access log
+#
+# caddy_env (str)	      Set to "" by default.
+#			      This allows environment variable to be set that may be required, for example when using "DNS Challenge" account credentials are required.
+#			      e.g. (in your rc.conf)   caddy_env="CLOUDFLARE_EMAIL=me@domain.com CLOUDFLARE_API_KEY=my_api_key"
+#
 
 . /etc/rc.subr
 
 name="caddy"
 rcvar="${name}_enable"
 
-load_rc_config $name
-: ${caddy_enable:=no}
+load_rc_config ${name}
+
+: ${caddy_enable:="NO"}
 : ${caddy_cert_email=""}
 : ${caddy_bin_path="/usr/local/bin/caddy"}
 : ${caddy_cpu="99%"} # was a bug for me that caused a crash within jails
 : ${caddy_config_path="/usr/local/www/Caddyfile"}
-: ${caddy_run_user="root"}
+: ${caddy_logfile="/var/log/caddy.log"}
+: ${caddy_user="root"}
+: ${caddy_group="wheel"}
 
 if [ "$caddy_cert_email" = "" ]
 then
@@ -46,38 +59,25 @@ then
     exit 1
 fi
 
-pidfile="/var/run/caddy.pid"
-logfile="/var/log/caddy.log"
+pidfile="/var/run/${name}.pid"
+procname="${caddy_bin_path}" #enabled builtin pid checking for start / stop
+command="/usr/sbin/daemon"
+command_args="-u ${caddy_user} -p ${pidfile} /usr/bin/env ${caddy_env} ${procname} -cpu ${caddy_cpu} -log stdout -conf ${caddy_config_path} -agree -email ${caddy_cert_email} < /dev/null >> ${caddy_logfile} 2>&1"
 
-command="${caddy_bin_path} -log ${logfile} -cpu ${caddy_cpu} -conf ${caddy_config_path} -agree -email ${caddy_cert_email}"
+start_precmd="caddy_startprecmd"
 
-start_cmd="caddy_start"
-status_cmd="caddy_status"
-stop_cmd="caddy_stop"
+caddy_startprecmd()
+{
+	if [ ! -e "${pidfile}" ]; then
+		install -o "${caddy_user}" -g "${caddy_group}" "/dev/null" "${pidfile}"
+	fi
 
-caddy_start() {
-    echo "Starting ${name}..."
-    /usr/sbin/daemon -u ${caddy_run_user} -c -p ${pidfile} -f ${command}
+	if [ ! -e "${caddy_logfile}" ]; then
+		install -o "${caddy_user}" -g "${caddy_group}" "dev/null" "${caddy_logfile}"
+	fi
 }
 
-caddy_status() {
-    if [ -f ${pidfile} ]; then
-      echo "${name} is running as $(cat $pidfile)."
-    else
-      echo "${name} is not running."
-      return 1
-    fi
-}
-
-caddy_stop() {
-    if [ ! -f ${pidfile} ]; then
-      echo "${name} is not running."
-      return 1
-    fi
-
-    echo -n "Stopping ${name}..."
-    kill -KILL $(cat $pidfile) 2> /dev/null && echo "stopped"
-    rm -f ${pidfile}
-}
+required_files="${caddy_config_path}"
 
 run_rc_command "$1"
+

dist/init/linux-systemd/caddy.service

@@ -20,6 +20,11 @@ Environment=CADDYPATH=/etc/ssl/caddy
 ExecStart=/usr/local/bin/caddy -log stdout -agree=true -conf=/etc/caddy/Caddyfile -root=/var/tmp
 ExecReload=/bin/kill -USR1 $MAINPID
 
+; Use graceful shutdown with a reasonable timeout
+KillMode=mixed
+KillSignal=SIGQUIT
+TimeoutStopSec=5s
+
 ; Limit the number of file descriptors; see `man systemd.exec` for more limit settings.
 LimitNOFILE=1048576
 ; Unmodified caddy is not expected to use more than that.