Update readme and changelog for v0.10.3

* Add uri policy test cases
 * Add function definition
 * Add uri hashing policy
 * Refactor and extract hostByHashing and use in IP and URI policy
 * Rename to URIHash

Signed-off-by: Jonas Östanbäck <jonas.ostanback@gmail.com>

.github/CONTRIBUTING.md

@@ -0,0 +1,158 @@
+Contributing to Caddy
+=====================
+
+Welcome! Thank you for choosing to be a part of our community. Caddy wouldn't be great without your involvement!
+
+For starters, we invite you to join [the Caddy forum](https://caddy.community) where you can hang out with other Caddy users and developers.
+
+## Common Tasks
+
+- [Contributing code](#contributing-code)
+- [Writing a plugin](#writing-a-plugin)
+- [Asking or answering questions for help using Caddy](#getting-help-using-caddy)
+- [Reporting a bug](#reporting-bugs)
+- [Suggesting an enhancement or a new feature](#suggesting-features)
+- [Improving documentation](#improving-documentation)
+
+Other menu items:
+
+- [Values](#values)
+- [Responsible Disclosure](#responsible-disclosure)
+- [Thank You](#thank-you)
+
+
+### Contributing code
+
+You can have a direct impact on the project by helping with its code. To contribute code to Caddy, open a [pull request](https://github.com/mholt/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.**
+
+We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :wink: If your change is on the right track, we can guide you to make it mergable.
+
+Here are some of the expectations we have of contributors:
+
+- If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, you can comment on the existing one to claim it.
+
+- **Keep pull requests small.** Smaller PRs are more likely to be merged because they are easier to review! We might ask you to break up large PRs into smaller ones. [An example of what we DON'T do.](https://twitter.com/iamdevloper/status/397664295875805184)
+
+- **Keep related commits together in a PR.** We do want pull requests to be small, but you should also keep multiple related commits in the same PR if they rely on each other.
+
+- **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass.
+
+- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven to work better with benchmarks or profiling.
+
+- **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance.
+
+- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors can help maintain their change after it is merged.
+
+- **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious.
+
+We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base!
+
+
+#### HOW TO MAKE A PULL REQUEST TO CADDY
+
+Contributing to Go projects on GitHub is fun and easy. We recommend the following workflow:
+
+1. [Fork this repo](https://github.com/mholt/caddy). This makes a copy of the code you can write to.
+
+2. If you don't already have this repo (mholt/caddy.git) repo on your computer, get it with `go get github.com/mholt/caddy/caddy`.
+
+3. Tell git that it can push the mholt/caddy.git repo to your fork by adding a remote: `git remote add myfork https://github.com/you/caddy.git`
+
+4. Make your changes in the mholt/caddy.git repo on your computer.
+
+5. Push your changes to your fork: `git push myfork`
+
+6. [Create a pull request](https://github.com/mholt/caddy/pull/new/master) to merge your changes into mholt/caddy @ master. (Click "compare across forks" and change the head fork.)
+
+This workflow is nice because you don't have to change import paths. You can get fancier by using different branches if you want.
+
+
+### Writing a plugin
+
+Caddy can do more with plugins! Anyone can write a plugin. Plugins are Go libraries that get compiled into Caddy, extending its feature set. They can add directives to the Caddyfile, change how the Caddyfile is loaded, and even implement new server types (e.g. HTTP, DNS). When it's ready, you can submit your plugin to the Caddy website so others can download it.
+
+[Learn how to write and submit a plugin](https://github.com/mholt/caddy/wiki) on the wiki. You should also share and discuss your plugin idea [on the forums](https://caddy.community) to have people test it out. We don't use the Caddy issue tracker for plugins.
+
+
+### Getting help using Caddy
+
+If you have a question about using Caddy, [ask on our forum](https://caddy.community)! There will be more people there who can help you than just the Caddy developers who follow our issue tracker. Issues are not the place for usage questions.
+
+Many people on the forums could benefit from your experience and expertise, too. Once you've been helped, consider giving back by answering other people's questions and participating in other discussions.
+
+
+### Reporting bugs
+
+Like every software, Caddy has its flaws. If you find one, [search the issues](https://github.com/mholt/caddy/issues) to see if it has already been reported. If not, [open a new issue](https://github.com/mholt/caddy/issues/new) and describe the bug, and somebody will look into it! (This repository is only for Caddy, not plugins.)
+
+**You can help stop bugs in their tracks!** Speed up the patch by identifying the bug in the code. This can sometimes be done by adding `fmt.Println()` statements (or similar) in relevant code paths to narrow down where the problem may be. It's a good way to [introduce yourself to the Go language](https://tour.golang.org), too.
+
+Please follow the issue template so we have all the needed information. Unredacted—yes, actual values matter. We need to be able to repeat the bug using your instructions. Please simplify the issue as much as possible. The burden is on you to convince us that it is actually a bug in Caddy. This is easiest to do when you write clear, concise instructions so we can reproduce the behavior (even if it seems obvious). The more detailed and specific you are, the faster we will be able to help you!
+
+We suggest reading [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
+
+Please be kind. :smile: Remember that Caddy comes at no cost to you, and you're getting free support when we fix your issues. If we helped you, please consider helping someone else!
+
+
+### Suggesting features
+
+First, [search to see if your feature has already been requested](https://github.com/mholt/caddy/issues). If it has, you can add a :+1: reaction to vote for it. If your feature idea is new, open an issue to request the feature. You don't have to follow the bug template for feature requests. Please describe your idea thoroughly so that we know how to implement it! Really vague requests may not be helpful or actionable and without clarification will have to be closed.
+
+While we really do value your requests and implement many of them, not all features are a good fit for Caddy. Most of those [make good plugins](https://github.com/mholt/caddy/wiki), though, which can be made by anyone! But if a feature is not in the best interest of the Caddy project or its users in general, we may politely decline to implement it into Caddy core.
+
+
+### Improving documentation
+
+Caddy's documentation is available at [https://caddyserver.com/docs](https://caddyserver.com/docs). If you would like to make a fix to the docs, feel free to contribute at the [caddyserver/website](https://github.com/caddyserver/website) repository!
+
+Note that plugin documentation is not hosted by the Caddy website, other than basic usage examples. They are managed by the individual plugin authors, and you will have to contact them to change their documentation.
+
+
+
+## Collaborator Instructions
+
+Collabators have push rights to the repository. We grant this permission after one or more successful, high-quality PRs are merged! We thank them for their help.The expectations we have of collaborators are:
+
+- **Help review pull requests.** Be meticulous, but also kind. We love our contributors, but we critique the contribution to make it better. Multiple, thorough reviews make for the best contributions! Here are some questions to consider:
+	- Can the change be made more elegant?
+	- Is this a maintenance burden?
+	- What assumptions does the code make?
+	- Is it well-tested?
+	- Is the change a good fit for the project?
+	- Does it actually fix the problem or is it creating a special case instead?
+	- Does the change incur any new dependencies? (Avoid these!)
+
+- **Answer issues.** If every collaborator helped out with issues, we could count the number of open issues on two hands. This means getting involved in the discussion, investigating the code, and yes, debugging it. It's fun. Really! :smile: Please, please help with open issues. Granted, some issues need to be done before others. And of course some are larger than others: you don't have to do it all yourself. Work with other collaborators as a team!
+
+- **Do not merge pull requests until they have been approved by one or two other collaborators.** If a project owner approves the PR, it can be merged (as long as the conversation has finished too).
+
+- **Prefer squashed commits over a messy merge.** If there are many little commits, please squash the commits so we don't clutter the commit history.
+
+- **Be extra careful in some areas of the code.** There are some critical areas in the Caddy code base that we review extra meticulously: the `caddy` and `caddytls` packages especially.
+
+- **Make sure tests test the actual thing.** Double-check that the tests fail without the change, and pass with it. It's important that they assert what they're purported to assert.
+
+- **Recommended reading**
+	- [CodeReviewComments](https://github.com/golang/go/wiki/CodeReviewComments) for an idea of what we look for in good, clean Go code
+	- [Linus Torvalds describes a good commit message](https://gist.github.com/matthewhudson/1475276)
+	- [Best Practices for Maintainers](https://opensource.guide/best-practices/)
+	- [Shrinking Code Review](https://alexgaynor.net/2015/dec/29/shrinking-code-review/)
+
+
+
+## Values
+
+- A person is always more important than code. People don't like being handled "efficiently". But we can still process issues and pull requests efficiently while being kind, patient, and considerate.
+
+- The ends justify the means, if the means are good. A good tree won't produce bad fruit. But if we cut corners or are hasty in our process, the end result will not be good.
+
+
+## Responsible Disclosure
+
+If you've found a security vulnerability, please email me, the author, directly: Matthew dot Holt at Gmail. I'll need enough information to verify the bug and make a patch. It will speed things up if you suggest a working patch. If your report is valid and a patch is released, we will not reveal your identity by default. If you wish to be credited, please give me the name to use. Thanks for responsibly helping Caddy—and thousands of websites—be more secure!
+
+
+## Thank you
+
+Thanks for your help! Caddy would not be what it is today without your
+contributions.

.github/ISSUE_TEMPLATE.md

@@ -1,6 +1,6 @@
-(Are you asking for help with using Caddy? Please use our forum instead: https://forum.caddyserver.com. If you are filing a bug report, please take a few minutes to carefully answer the following questions. If your issue is not a bug report, you do not need to use this template. Thanks!)
+(Are you asking for help with using Caddy? Please use our forum instead: https://caddy.community. If you are filing a bug report, please take a few minutes to carefully answer the following questions. If your issue is not a bug report, you do not need to use this template. Thanks!)
 
-### 1. What version of Caddy are you running (`caddy -version`)?
+### 1. What version of Caddy are you using (`caddy -version`)?
 
 
 ### 2. What are you trying to do?
@@ -8,7 +8,7 @@
 
 ### 3. What is your entire Caddyfile?
 ```text
-(Put Caddyfile here)
+(paste Caddyfile here)
 ```
 
 ### 4. How did you run Caddy (give the full command and describe the execution environment)?
@@ -27,4 +27,4 @@
 
 ### 8. How can someone who is starting from scratch reproduce the bug as minimally as possible?
 
-(Please strip away any extra infrastructure such as containers, reverse proxies, upstream apps, dependencies, etc, to prove this is a bug in Caddy and not an external misconfiguration. Your chances of getting this bug fixed go way up the easier it is to replicate. Thank you!)
+(Please strip away any extra infrastructure such as containers, reverse proxies, upstream apps, caches, dependencies, etc, to prove this is a bug in Caddy and not an external misconfiguration. Your chances of getting this bug fixed go way up the easier it is to replicate. Thank you!)

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,17 @@
+(Thank you for contributing to Caddy! Please fill this out to help us make the most of your pull request.)
+
+### 1. What does this change do, exactly?
+
+
+### 2. Please link to the relevant issues.
+
+
+### 3. Which documentation changes (if any) need to be made because of this PR?
+
+
+### 4. Checklist
+
+- [ ] I have written tests and verified that they fail without my change
+- [ ] I have squashed any insignificant commits
+- [ ] This change has comments for package types, values, functions, and non-obvious lines of code
+- [ ] I am willing to help maintain this change if there are issues with it later

CONTRIBUTING.md

@@ -1,117 +0,0 @@
-## Contributing to Caddy
-
-Welcome! Our community focuses on helping others and making Caddy the best it
-can be. We gladly accept contributions and encourage you to get involved!
-
-
-### Join us in the forum
-
-The [Caddy forum](https://forum.caddyserver.com) is the place for all discussion
-that doesn't belong in issues or pull requests. Feel free to participate with us!
-
-If you want to file a bug report or make an improvement to Caddy, however, you
-should submit an issue or pull request.
-
-
-### Bug reports
-
-Please [search this repository](https://github.com/mholt/caddy/search?q=&type=Issues&utf8=%E2%9C%93)
-with a variety of keywords to ensure your bug is not already reported.
-
-If unique, [open an issue](https://github.com/mholt/caddy/issues) and answer the
-questions so we can understand and reproduce the problematic behavior.
-
-The burden is on you to convince us that it is actually a bug in Caddy. This is
-easiest to do when you write clear, concise instructions so we can reproduce
-the behavior (even if it seems obvious). The more detailed and specific you are,
-the faster we will be able to help you. Check out
-[How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
-
-Please be kind. :smile: Remember that Caddy comes at no cost to you, and you're
-getting free help. If we helped you, please consider
-[donating](https://caddyserver.com/donate) - it keeps us motivated!
-
-
-### Minor improvements and new tests
-
-Submit [pull requests](https://github.com/mholt/caddy/pulls) at any time for
-minor changes or new tests. Make sure to write tests to assert your change is
-working properly and is thoroughly covered. We'll ask most pull requests to be
-[squashed](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html),
-especially with small commits.
-
-Your pull request may be thoroughly reviewed. This is because if we accept the
-PR, we also assume responsibility for it, although we would prefer you to
-help maintain your code after it gets merged. 
-
-
-### Proposals, suggestions, ideas, new features
-
-First, please [search](https://github.com/mholt/caddy/search?q=&type=Issues&utf8=%E2%9C%93)
-with a variety of keywords to ensure your suggestion/proposal is new.
-
-If so, you may open either an issue or a pull request for discussion and
-feedback.
-
-The advantage of issues is that you don't have to spend time implementing your
-idea, but you should still describe it thoroughly as if someone reading it would
-implement the whole thing starting from scratch.
-
-The advantage of pull requests is that we can immediately see the impact the
-change will have on the project, what the code will look like, and how to
-improve it. The disadvantage of pull requests is that they are unlikely to get
-accepted without significant changes first, or it may be rejected entirely.
-Don't worry, that won't happen without an open discussion first.
-
-If you are going to spend significant time writing code for a new pull request,
-best to open an issue to "claim" it and get feedback before you invest a lot of
-time. Not all pull requests are merged, and that's okay,
-[Read why.](https://github.com/turbolinks/turbolinks/pull/124#issuecomment-239826060)
-
-Remember: pull requests should always be thoroughly documented both via godoc
-and with at least a rough draft of documentation that might go on the website
-for users to read.
-
-
-### Collaborator status
-
-If your pull request is merged, congratulations! You're technically a
-collaborator. We may also grant you "Collaborator status" which means you can
-push to the repository and merge other pull requests. We hope that you will
-stay involved by reviewing pull requests, submitting more of your own, and
-resolving issues as you are able to. Thanks for making Caddy amazing!
-
-We ask that collaborators will conduct thorough code reviews and be nice to
-new contributors. Before merging a PR, it's best to get the approval of
-at least one or two other collaborators and/or the project owner. We prefer
-squashed commits instead of many little, semantically-unimportant commits. Also,
-CI and other post-commit hooks must pass before being merged except in certain
-unusual circumstances.
-
-Collaborator status may be removed for inactive users from time to time as
-we see fit; this is not an insult, just a basic security precaution in case
-the account becomes inactive or abandoned. Privileges can always be restored
-later.
-
-**Reviewing pull requests:** Please help submit and review pull requests as
-you are able! We would ask that every pull request be reviewed by at least
-one collaborator who did not open the pull request before merging. This will
-help ensure high code quality as new collaborators are added to the project.
-
-Read [CodeReviewComments](https://github.com/golang/go/wiki/CodeReviewComments)
-on the Go wiki for an idea of what we look for in good, clean Go code, and
-check out [what Linus suggests](https://gist.github.com/matthewhudson/1475276)
-for good commit messages.
-
-
-
-### Vulnerabilities
-
-If you've found a vulnerability that is serious, please email me: Matthew dot 
-Holt at Gmail. If it's not a big deal, a pull request will probably be faster.
-
-
-## Thank you
-
-Thanks for your help! Caddy would not be what it is today without your
-contributions.

README.md

@@ -1,73 +1,95 @@
-<a href="https://caddyserver.com"><img src="https://caddyserver.com/resources/images/caddy-lower.png" alt="Caddy" width="350"></a>
+<p align="center">
+	<a href="https://caddyserver.com"><img src="https://cloud.githubusercontent.com/assets/1128849/25305033/12916fce-2731-11e7-86ec-580d4d31cb16.png" alt="Caddy" width="400"></a>
+</p>
+<h3 align="center">Every Site on HTTPS <!-- Serve Confidently --></h3>
+<p align="center">Caddy is a general-purpose HTTP/2 web server that serves HTTPS by default.</p>
+<p align="center">
+	<a href="https://travis-ci.org/mholt/caddy"><img src="https://img.shields.io/travis/mholt/caddy.svg?label=linux+build"></a>
+	<a href="https://ci.appveyor.com/project/mholt/caddy"><img src="https://img.shields.io/appveyor/ci/mholt/caddy.svg?label=windows+build"></a>
+	<a href="https://godoc.org/github.com/mholt/caddy"><img src="https://img.shields.io/badge/godoc-reference-blue.svg"></a>
+	<a href="https://goreportcard.com/report/mholt/caddy"><img src="https://goreportcard.com/badge/github.com/mholt/caddy"></a>
+	<br>
+	<a href="https://twitter.com/caddyserver" title="@caddyserver on Twitter"><img src="https://img.shields.io/badge/twitter-@caddyserver-55acee.svg" alt="@caddyserver on Twitter"></a>
+	<a href="https://caddy.community" title="Caddy Forum"><img src="https://img.shields.io/badge/community-forum-ff69b4.svg" alt="Caddy Forum"></a>
+	<a href="https://sourcegraph.com/github.com/mholt/caddy?badge" title="Caddy on Sourcegraph"><img src="https://sourcegraph.com/github.com/mholt/caddy/-/badge.svg" alt="Caddy on Sourcegraph"></a>
+</p>
+<p align="center">
+	<a href="https://caddyserver.com/download">Download</a> ·
+	<a href="https://caddyserver.com/docs">Documentation</a> ·
+	<a href="https://caddy.community">Community</a>
+</p>
 
-[![community](https://img.shields.io/badge/community-forum-ff69b4.svg?style=flat-square)](https://forum.caddyserver.com) [![twitter](https://img.shields.io/badge/twitter-@caddyserver-55acee.svg?style=flat-square)](https://twitter.com/caddyserver) [![Documentation](https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square)](https://godoc.org/github.com/mholt/caddy) [![Linux Build Status](https://img.shields.io/travis/mholt/caddy.svg?style=flat-square&label=linux+build)](https://travis-ci.org/mholt/caddy) [![Windows Build Status](https://img.shields.io/appveyor/ci/mholt/caddy.svg?style=flat-square&label=windows+build)](https://ci.appveyor.com/project/mholt/caddy)
-[![Go Report Card](https://goreportcard.com/badge/github.com/mholt/caddy?style=flat-square)](https://goreportcard.com/report/mholt/caddy)
-[![Sourcegraph Badge](https://sourcegraph.com/github.com/mholt/caddy/-/badge.svg)](https://sourcegraph.com/github.com/mholt/caddy?badge)
+---
 
+Caddy is fast, easy to use, and makes you more productive.
 
-Caddy is a general-purpose web server for Windows, Mac, Linux, BSD, and
-[Android](https://github.com/mholt/caddy/wiki/Running-Caddy-on-Android). It is
-a capable but easier alternative to other popular web servers.
-
-[Releases](https://github.com/mholt/caddy/releases) ·
-[User Guide](https://caddyserver.com/docs) ·
-[Community](https://forum.caddyserver.com)
-
-**Attend the [Caddy launch event](https://www.facebook.com/events/1413078512092363/) on April 20! It's free, and we will try to get a live stream going.**
-
-<a href="https://www.facebook.com/events/1413078512092363/"><img src="http://i.imgur.com/CjGICWU.png" alt="Caddy launch event" width="500"></a>
-
-Try browsing [the code on Sourcegraph](https://sourcegraph.com/github.com/mholt/caddy)!
+Available for Windows, Mac, Linux, BSD, Solaris, and [Android](https://github.com/mholt/caddy/wiki/Running-Caddy-on-Android).
 
 ## Menu
 
 - [Features](#features)
+- [Install](#install)
 - [Quick Start](#quick-start)
-- [Running from Source](#running-from-source)
 - [Running in Production](#running-in-production)
 - [Contributing](#contributing)
+- [Donors](#donors)
 - [About the Project](#about-the-project)
 
-
-
 ## Features
 
-- **Easy configuration** with Caddyfile
-- **Automatic HTTPS** via [Let's Encrypt](https://letsencrypt.org); Caddy
-obtains and manages all cryptographic assets for you
-- **HTTP/2** enabled by default (powered by Go standard library)
-- **Virtual hosting** for hundreds of sites per server instance, including TLS
-SNI
+- **Easy configuration** with the Caddyfile
+- **Automatic HTTPS** on by default (via [Let's Encrypt](https://letsencrypt.org))
+- **HTTP/2** by default
+- **Virtual hosting** so multiple sites just work
 - Experimental **QUIC support** for those that like speed
 - TLS session ticket **key rotation** for more secure connections
-- **Brilliant extensibility** so Caddy can be customized for your needs
+- **Extensible with plugins** because a convenient web server is a helpful one
 - **Runs anywhere** with **no external dependencies** (not even libc)
 
+There's way more, too! [See all features built into Caddy.](https://caddyserver.com/features) On top of all those, Caddy does even more with plugins: choose which plugins you want at [download](https://caddyserver.com/download).
 
 
-## Quick Start
+## Install
 
-Caddy binaries have no dependencies and are available for every platform.
-Install Caddy any one of these ways:
+Caddy binaries have no dependencies and are available for every platform. Get Caddy any one of these ways:
 
 - **[Download page](https://caddyserver.com/download)** allows you to
 customize your build in the browser
 - **[Latest release](https://github.com/mholt/caddy/releases/latest)** for
-pre-built binaries
-- **curl [getcaddy.com](https://getcaddy.com)** for auto install:
-`curl https://getcaddy.com | bash`
+pre-built, vanilla binaries
+- **go get** to build from source: `go get github.com/mholt/caddy/caddy` (requires Go 1.8 or newer)
 
-Once `caddy` is in your PATH, you can `cd` to your website's folder and run
-`caddy` to serve it. By default, Caddy serves the current directory at
-[localhost:2015](http://localhost:2015).
+Then make sure the `caddy` binary is in your PATH.
 
-To customize how your site is served, create a file named Caddyfile by your
-site and paste this into it:
+
+## Quick Start
+
+To serve static files from the current working directory, run:
+
+```
+caddy
+```
+
+Caddy's default port is 2015, so open your browser to [http://localhost:2015](http://localhost:2015).
+
+### Go from 0 to HTTPS in 5 seconds
+
+If the `caddy` binary has permission to bind to low ports and your domain name's DNS records point to the machine you're on:
+
+```
+caddy -host example.com
+```
+
+This command serves static files from the current directory over HTTPS. Certificates are automatically obtained and renewed for you!
+
+### Customizing your site
+
+To customize how your site is served, create a file named Caddyfile by your site and paste this into it:
 
 ```plain
 localhost
 
-gzip
+push
 browse
 websocket /echo cat
 ext    .html
@@ -76,93 +98,59 @@ proxy  /api 127.0.0.1:7005
 header /api Access-Control-Allow-Origin *
 ```
 
-When you run `caddy` in that directory, it will automatically find and use
-that Caddyfile to configure itself.
+When you run `caddy` in that directory, it will automatically find and use that Caddyfile.
 
-This simple file enables compression, allows directory browsing (for folders
-without an index file), hosts a WebSocket echo server at /echo, serves clean
-URLs, logs requests to access.log, proxies all API requests to a backend on
-port 7005, and adds the coveted  `Access-Control-Allow-Origin: *` header for
-all responses from the API.
+This simple file enables server push (via Link headers), allows directory browsing (for folders without an index file), hosts a WebSocket echo server at /echo, serves clean URLs, logs requests to an access log, proxies all API requests to a backend on port 7005, and adds the coveted  `Access-Control-Allow-Origin: *` header for all responses from the API.
 
 Wow! Caddy can do a lot with just a few lines.
 
-To host multiple sites and do more with the Caddyfile, please see the
-[Caddyfile documentation](https://caddyserver.com/docs/caddyfile).
+### Doing more with Caddy
 
-Note that production sites are served over
-[HTTPS by default](https://caddyserver.com/docs/automatic-https).
+To host multiple sites and do more with the Caddyfile, please see the [Caddyfile tutorial](https://caddyserver.com/tutorial/caddyfile).
 
-Caddy has a command line interface. Run `caddy -h` to view basic help or see
-the [CLI documentation](https://caddyserver.com/docs/cli) for details.
-
-**Running as root:** We advise against this. You can still listen on ports
-< 1024 using setcap like so: `sudo setcap cap_net_bind_service=+ep ./caddy`
-
-
-
-## Running from Source
-
-Note: You will need **[Go 1.8](https://golang.org/dl/)** or newer.
-
-1. `go get github.com/mholt/caddy/caddy`
-2. `cd` into your website's directory
-3. Run `caddy` (assuming `$GOPATH/bin` is in your `$PATH`)
-
-Caddy's `main()` is in the caddy subfolder. To recompile Caddy, use
-`build.bash` found in that folder.
+Sites with qualifying hostnames are served over [HTTPS by default](https://caddyserver.com/docs/automatic-https).
 
+Caddy has a command line interface. Run `caddy -h` to view basic help or see the [CLI documentation](https://caddyserver.com/docs/cli) for details.
 
 
 ## Running in Production
 
-The Caddy project does not officially maintain any system-specific
-integrations, but your download file includes
-[unofficial resources](https://github.com/mholt/caddy/tree/master/dist/init)
-contributed by the community that you may find helpful for running Caddy in
-production.
+Caddy is production-ready if you find it to be a good fit for your site and workflow.
 
-How you choose to run Caddy is up to you. Many users are satisfied with
-`nohup caddy &`. Others use `screen`. Users who need Caddy to come back up
-after reboots either do so in the script that caused the reboot, add a command
-to an init script, or configure a service with their OS.
+**Running as root:** We advise against this. You can still listen on ports < 1024 on Linux using setcap like so: `sudo setcap cap_net_bind_service=+ep ./caddy`
 
+The Caddy project does not officially maintain any system-specific integrations nor suggest how to administer your own system. But your download file includes [unofficial resources](https://github.com/mholt/caddy/tree/master/dist/init) contributed by the community that you may find helpful for running Caddy in production.
+
+How you choose to run Caddy is up to you. Many users are satisfied with `nohup caddy &`. Others use `screen`. Users who need Caddy to come back up after reboots either do so in the script that caused the reboot, add a command to an init script, or configure a service with their OS.
 
 
 ## Contributing
 
-**[Join our community](https://forum.caddyserver.com) where you can chat with
-other Caddy users and developers!**
+**[Join our forum](https://caddy.community) where you can chat with other Caddy users and developers!**
 
-Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/CONTRIBUTING.md)
-and check out the [developer wiki](https://github.com/mholt/caddy/wiki).
+Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/.github/CONTRIBUTING.md). If you want to write a plugin, check out the [developer wiki](https://github.com/mholt/caddy/wiki).
 
-We use GitHub issues and pull requests only for discussing bug reports and
-the development of specific changes. We welcome all other topics on the
-[forum](https://forum.caddyserver.com)!
+We use GitHub issues and pull requests only for discussing bug reports and the development of specific changes. We welcome all other topics on the [forum](https://caddy.community)!
 
-If you want to contribute to the documentation, please submit pull requests to [caddyserver/caddyserver.com](https://github.com/caddyserver/caddyserver.com).
+If you want to contribute to the documentation, please submit pull requests to [caddyserver/website](https://github.com/caddyserver/website).
 
 Thanks for making Caddy -- and the Web -- better!
 
-Special thanks to
-[![DigitalOcean](https://i.imgur.com/sfGr0eY.png)](https://www.digitalocean.com)
-for hosting the Caddy project.
 
+## Donors
+
+- [DigitalOcean](https://m.do.co/c/6d7bdafccf96) is hosting the Caddy project.
+- [DNSimple](https://dnsimple.link/resolving-caddy) provides DNS services for Caddy's sites.
+- [DNS Spy](https://dnsspy.io) keeps an eye on Caddy's DNS properties.
+
+We thank them for their services. **If you want to help keep Caddy free, please [become a sponsor](https://caddyserver.com/pricing)!**
 
 
 ## About the Project
 
-Caddy was born out of the need for a "batteries-included" web server that runs
-anywhere and doesn't have to take its configuration with it. Caddy took
-inspiration from [spark](https://github.com/rif/spark),
-[nginx](https://github.com/nginx/nginx), lighttpd,
-[Websocketd](https://github.com/joewalnes/websocketd)
-and [Vagrant](https://www.vagrantup.com/),
-which provides a pleasant mixture of features from each of them.
+Caddy was born out of the need for a "batteries-included" web server that runs anywhere and doesn't have to take its configuration with it. Caddy took inspiration from [spark](https://github.com/rif/spark), [nginx](https://github.com/nginx/nginx), lighttpd,
+[Websocketd](https://github.com/joewalnes/websocketd) and [Vagrant](https://www.vagrantup.com/), which provides a pleasant mixture of features from each of them.
 
-**The name "Caddy":** The name of the software is "Caddy", not "Caddy Server"
-or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the
-Caddy web server". See [brand guidelines](https://caddyserver.com/brand).
+**The name "Caddy":** The name of the software is "Caddy", not "Caddy Server" or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the Caddy web server". See [brand guidelines](https://caddyserver.com/brand).
 
 *Author on Twitter: [@mholt6](https://twitter.com/mholt6)*

caddy.go

@@ -768,7 +768,7 @@ func IsLoopback(addr string) bool {
 // be an IP or an IP:port combination.
 // Loopback addresses are considered false.
 func IsInternal(addr string) bool {
-	private_networks := []string{
+	privateNetworks := []string{
 		"10.0.0.0/8",
 		"172.16.0.0/12",
 		"192.168.0.0/16",
@@ -777,14 +777,17 @@ func IsInternal(addr string) bool {
 
 	host, _, err := net.SplitHostPort(addr)
 	if err != nil {
-		host = addr // happens if the addr is just a hostname
+		host = addr // happens if the addr is just a hostname, missing port
+		// if we encounter an error, the brackets need to be stripped
+		// because SplitHostPort didn't do it for us
+		host = strings.Trim(host, "[]")
 	}
 	ip := net.ParseIP(host)
 	if ip == nil {
 		return false
 	}
-	for _, private_network := range private_networks {
-		_, ipnet, _ := net.ParseCIDR(private_network)
+	for _, privateNetwork := range privateNetworks {
+		_, ipnet, _ := net.ParseCIDR(privateNetwork)
 		if ipnet.Contains(ip) {
 			return true
 		}

caddy/main.go

@@ -1,7 +1,7 @@
 // By moving the application's package main logic into
 // a package other than main, it becomes much easier to
 // wrap caddy for custom builds that are go-gettable.
-// https://forum.caddyserver.com/t/my-wish-for-0-9-go-gettable-custom-builds/59?u=matt
+// https://caddy.community/t/my-wish-for-0-9-go-gettable-custom-builds/59?u=matt
 
 package main
 

caddy_test.go

@@ -94,6 +94,8 @@ func TestIsInternal(t *testing.T) {
 		{"fbff:ffff:ffff:ffff:ffff:ffff:ffff:ffff", false},
 		{"fc00::", true},
 		{"fc00::1", true},
+		{"[fc00::1]", true},
+		{"[fc00::1]:8888", true},
 		{"fdff:ffff:ffff:ffff:ffff:ffff:ffff:fffe", true},
 		{"fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff", true},
 		{"fe00::", false},

caddyhttp/browse/browse.go

@@ -156,10 +156,10 @@ func (l byNameDirFirst) Less(i, j int) bool {
 	// if both are dir or file sort normally
 	if l.Items[i].IsDir == l.Items[j].IsDir {
 		return strings.ToLower(l.Items[i].Name) < strings.ToLower(l.Items[j].Name)
-	} else {
-		// always sort dir ahead of file
-		return l.Items[i].IsDir
 	}
+
+	// always sort dir ahead of file
+	return l.Items[i].IsDir
 }
 
 // By Size
@@ -333,9 +333,14 @@ func (b Browse) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 
 	// Browsing navigation gets messed up if browsing a directory
 	// that doesn't end in "/" (which it should, anyway)
-	if !strings.HasSuffix(r.URL.Path, "/") {
-		staticfiles.RedirectToDir(w, r)
-		return 0, nil
+	u := *r.URL
+	if u.Path == "" {
+		u.Path = "/"
+	}
+	if u.Path[len(u.Path)-1] != '/' {
+		u.Path += "/"
+		http.Redirect(w, r, u.String(), http.StatusMovedPermanently)
+		return http.StatusMovedPermanently, nil
 	}
 
 	return b.ServeListing(w, r, requestedFilepath, bc)

caddyhttp/browse/browse_test.go

@@ -142,6 +142,8 @@ func TestBrowseHTTPMethods(t *testing.T) {
 		if err != nil {
 			t.Fatalf("Test: Could not create HTTP request: %v", err)
 		}
+		ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+		req = req.WithContext(ctx)
 
 		code, _ := b.ServeHTTP(rec, req)
 		if code != expected {
@@ -177,6 +179,8 @@ func TestBrowseTemplate(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Test: Could not create HTTP request: %v", err)
 	}
+	ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+	req = req.WithContext(ctx)
 
 	rec := httptest.NewRecorder()
 
@@ -322,6 +326,8 @@ func TestBrowseJson(t *testing.T) {
 		if err != nil && !test.shouldErr {
 			t.Errorf("Test %d errored when making request, but it shouldn't have; got '%v'", i, err)
 		}
+		ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+		req = req.WithContext(ctx)
 
 		req.Header.Set("Accept", "application/json")
 		rec := httptest.NewRecorder()
@@ -394,13 +400,13 @@ func TestBrowseRedirect(t *testing.T) {
 		{
 			"http://www.example.com/photos",
 			http.StatusMovedPermanently,
-			0,
+			http.StatusMovedPermanently,
 			"http://www.example.com/photos/",
 		},
 		{
 			"/photos",
 			http.StatusMovedPermanently,
-			0,
+			http.StatusMovedPermanently,
 			"/photos/",
 		},
 	}
@@ -422,12 +428,11 @@ func TestBrowseRedirect(t *testing.T) {
 		}
 
 		req, err := http.NewRequest("GET", tc.url, nil)
-		u, _ := url.Parse(tc.url)
-		ctx := context.WithValue(req.Context(), staticfiles.URLPathCtxKey, u.Path)
-		req = req.WithContext(ctx)
 		if err != nil {
 			t.Fatalf("Test %d - could not create HTTP request: %v", i, err)
 		}
+		ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+		req = req.WithContext(ctx)
 
 		rec := httptest.NewRecorder()
 

caddyhttp/browse/setup.go

@@ -65,7 +65,7 @@ func browseParse(c *caddy.Controller) ([]Config, error) {
 
 		bc.Fs = staticfiles.FileServer{
 			Root: http.Dir(cfg.Root),
-			Hide: httpserver.GetConfig(c).HiddenFiles,
+			Hide: cfg.HiddenFiles,
 		}
 
 		// Second argument would be the template file to use

caddyhttp/caddyhttp.go

@@ -16,9 +16,9 @@ import (
 	_ "github.com/mholt/caddy/caddyhttp/header"
 	_ "github.com/mholt/caddy/caddyhttp/index"
 	_ "github.com/mholt/caddy/caddyhttp/internalsrv"
+	_ "github.com/mholt/caddy/caddyhttp/limits"
 	_ "github.com/mholt/caddy/caddyhttp/log"
 	_ "github.com/mholt/caddy/caddyhttp/markdown"
-	_ "github.com/mholt/caddy/caddyhttp/maxrequestbody"
 	_ "github.com/mholt/caddy/caddyhttp/mime"
 	_ "github.com/mholt/caddy/caddyhttp/pprof"
 	_ "github.com/mholt/caddy/caddyhttp/proxy"

caddyhttp/extensions/ext.go

@@ -31,9 +31,10 @@ type Ext struct {
 // ServeHTTP implements the httpserver.Handler interface.
 func (e Ext) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	urlpath := strings.TrimSuffix(r.URL.Path, "/")
-	if path.Ext(urlpath) == "" && len(r.URL.Path) > 0 && r.URL.Path[len(r.URL.Path)-1] != '/' {
+	if len(r.URL.Path) > 0 && path.Ext(urlpath) == "" && r.URL.Path[len(r.URL.Path)-1] != '/' {
 		for _, ext := range e.Extensions {
-			if resourceExists(e.Root, urlpath+ext) {
+			_, err := os.Stat(httpserver.SafePath(e.Root, urlpath) + ext)
+			if err == nil {
 				r.URL.Path = urlpath + ext
 				break
 			}
@@ -41,12 +42,3 @@ func (e Ext) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	}
 	return e.Next.ServeHTTP(w, r)
 }
-
-// resourceExists returns true if the file specified at
-// root + path exists; false otherwise.
-func resourceExists(root, path string) bool {
-	_, err := os.Stat(root + path)
-	// technically we should use os.IsNotExist(err)
-	// but we don't handle any other kinds of errors anyway
-	return err == nil
-}

caddyhttp/fastcgi/fastcgi.go

@@ -8,6 +8,7 @@ import (
 	"io"
 	"net"
 	"net/http"
+	"net/url"
 	"os"
 	"path"
 	"path/filepath"
@@ -35,9 +36,25 @@ type Handler struct {
 // ServeHTTP satisfies the httpserver.Handler interface.
 func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	for _, rule := range h.Rules {
-
-		// First requirement: Base path must match and the path must be allowed.
-		if !httpserver.Path(r.URL.Path).Matches(rule.Path) || !rule.AllowedPath(r.URL.Path) {
+		// First requirement: Base path must match request path. If it doesn't,
+		// we check to make sure the leading slash is not missing, and if so,
+		// we check again with it prepended. This is in case people forget
+		// a leading slash when performing rewrites, and we don't want to expose
+		// the contents of the (likely PHP) script. See issue #1645.
+		hpath := httpserver.Path(r.URL.Path)
+		if !hpath.Matches(rule.Path) {
+			if strings.HasPrefix(string(hpath), "/") {
+				// this is a normal-looking path, and it doesn't match; try next rule
+				continue
+			}
+			hpath = httpserver.Path("/" + string(hpath)) // prepend leading slash
+			if !hpath.Matches(rule.Path) {
+				// even after fixing the request path, it still doesn't match; try next rule
+				continue
+			}
+		}
+		// The path must also be allowed (not ignored).
+		if !rule.AllowedPath(r.URL.Path) {
 			continue
 		}
 
@@ -213,23 +230,19 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 	// Strip PATH_INFO from SCRIPT_NAME
 	scriptName = strings.TrimSuffix(scriptName, pathInfo)
 
-	// Get the request URI from context. The request URI might be as it came in over the wire,
-	// or it might have been rewritten internally by the rewrite middleware (see issue #256).
-	// If it was rewritten, there will be a context value with the original URL,
-	// which is needed to get the correct RequestURI value for PHP apps.
-	reqURI := r.URL.RequestURI()
-	if origURI, _ := r.Context().Value(httpserver.URIxRewriteCtxKey).(string); origURI != "" {
-		reqURI = origURI
-	}
+	// Get the request URI from context. The context stores the original URI in case
+	// it was changed by a middleware such as rewrite. By default, we pass the
+	// original URI in as the value of REQUEST_URI (the user can overwrite this
+	// if desired). Most PHP apps seem to want the original URI. Besides, this is
+	// how nginx defaults: http://stackoverflow.com/a/12485156/1048862
+	reqURL, _ := r.Context().Value(httpserver.OriginalURLCtxKey).(url.URL)
 
-	// Retrieve name of remote user that was set by some downstream middleware,
-	// possibly basicauth.
-	remoteUser, _ := r.Context().Value(httpserver.RemoteUserCtxKey).(string) // Blank if not set
+	// Retrieve name of remote user that was set by some downstream middleware such as basicauth.
+	remoteUser, _ := r.Context().Value(httpserver.RemoteUserCtxKey).(string)
 
 	// Some variables are unused but cleared explicitly to prevent
 	// the parent environment from interfering.
 	env = map[string]string{
-
 		// Variables defined in CGI 1.1 spec
 		"AUTH_TYPE":         "", // Not used
 		"CONTENT_LENGTH":    r.Header.Get("Content-Length"),
@@ -252,13 +265,13 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 		"DOCUMENT_ROOT":   rule.Root,
 		"DOCUMENT_URI":    docURI,
 		"HTTP_HOST":       r.Host, // added here, since not always part of headers
-		"REQUEST_URI":     reqURI,
+		"REQUEST_URI":     reqURL.RequestURI(),
 		"SCRIPT_FILENAME": scriptFilename,
 		"SCRIPT_NAME":     scriptName,
 	}
 
-	// compliance with the CGI specification that PATH_TRANSLATED
-	// should only exist if PATH_INFO is defined.
+	// compliance with the CGI specification requires that
+	// PATH_TRANSLATED should only exist if PATH_INFO is defined.
 	// Info: https://www.ietf.org/rfc/rfc3875 Page 14
 	if env["PATH_INFO"] != "" {
 		env["PATH_TRANSLATED"] = filepath.Join(rule.Root, pathInfo) // Info: http://www.oreilly.com/openbook/cgi/ch02_04.html
@@ -269,10 +282,9 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 		env["HTTPS"] = "on"
 	}
 
+	// Add env variables from config (with support for placeholders in values)
 	replacer := httpserver.NewReplacer(r, nil, "")
-	// Add env variables from config
 	for _, envVar := range rule.EnvVars {
-		// replace request placeholders in environment variables
 		env[envVar[0]] = replacer.Replace(envVar[1])
 	}
 

caddyhttp/fastcgi/fastcgi_test.go

@@ -1,6 +1,7 @@
 package fastcgi
 
 import (
+	"context"
 	"net"
 	"net/http"
 	"net/http/fcgi"
@@ -10,6 +11,8 @@ import (
 	"sync"
 	"testing"
 	"time"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
 func TestServeHTTP(t *testing.T) {
@@ -130,6 +133,8 @@ func TestPersistent(t *testing.T) {
 					if err != nil {
 						t.Errorf("Unable to create request: %v", err)
 					}
+					ctx := context.WithValue(r.Context(), httpserver.OriginalURLCtxKey, *r.URL)
+					r = r.WithContext(ctx)
 					w := httptest.NewRecorder()
 
 					status, err := handler.ServeHTTP(w, r)
@@ -222,13 +227,13 @@ func TestBuildEnv(t *testing.T) {
 	}
 
 	rule := Rule{}
-	url, err := url.Parse("http://localhost:2015/fgci_test.php?test=blabla")
+	url, err := url.Parse("http://localhost:2015/fgci_test.php?test=foobar")
 	if err != nil {
 		t.Error("Unexpected error:", err.Error())
 	}
 
 	var newReq = func() *http.Request {
-		return &http.Request{
+		r := http.Request{
 			Method:     "GET",
 			URL:        url,
 			Proto:      "HTTP/1.1",
@@ -241,6 +246,8 @@ func TestBuildEnv(t *testing.T) {
 				"Foo": {"Bar", "two"},
 			},
 		}
+		ctx := context.WithValue(r.Context(), httpserver.OriginalURLCtxKey, *r.URL)
+		return r.WithContext(ctx)
 	}
 
 	fpath := "/fgci_test.php"
@@ -250,7 +257,7 @@ func TestBuildEnv(t *testing.T) {
 			"REMOTE_ADDR":     "2b02:1810:4f2d:9400:70ab:f822:be8a:9093",
 			"REMOTE_PORT":     "51688",
 			"SERVER_PROTOCOL": "HTTP/1.1",
-			"QUERY_STRING":    "test=blabla",
+			"QUERY_STRING":    "test=foobar",
 			"REQUEST_METHOD":  "GET",
 			"HTTP_HOST":       "localhost:2015",
 		}
@@ -300,8 +307,8 @@ func TestBuildEnv(t *testing.T) {
 	}
 	envExpected = newEnv()
 	envExpected["HTTP_HOST"] = "localhost:2015"
-	envExpected["CUSTOM_URI"] = "custom_uri/fgci_test.php?test=blabla"
-	envExpected["CUSTOM_QUERY"] = "custom=true&test=blabla"
+	envExpected["CUSTOM_URI"] = "custom_uri/fgci_test.php?test=foobar"
+	envExpected["CUSTOM_QUERY"] = "custom=true&test=foobar"
 	testBuildEnv(r, rule, fpath, envExpected)
 }
 

caddyhttp/gzip/gzip.go

@@ -3,16 +3,10 @@
 package gzip
 
 import (
-	"bufio"
-	"compress/gzip"
 	"io"
-	"io/ioutil"
-	"net"
 	"net/http"
 	"strings"
 
-	"errors"
-
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
@@ -22,6 +16,8 @@ func init() {
 		ServerType: "http",
 		Action:     setup,
 	})
+
+	initWriterPool()
 }
 
 // Gzip is a middleware type which gzips HTTP responses. It is
@@ -58,13 +54,12 @@ outer:
 		// gzipWriter modifies underlying writer at init,
 		// use a discard writer instead to leave ResponseWriter in
 		// original form.
-		gzipWriter, err := newWriter(c, ioutil.Discard)
-		if err != nil {
-			// should not happen
-			return http.StatusInternalServerError, err
+		gzipWriter := getWriter(c.Level)
+		defer putWriter(c.Level, gzipWriter)
+		gz := &gzipResponseWriter{
+			Writer:                gzipWriter,
+			ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
 		}
-		defer gzipWriter.Close()
-		gz := &gzipResponseWriter{Writer: gzipWriter, ResponseWriter: w}
 
 		var rw http.ResponseWriter
 		// if no response filter is used
@@ -94,21 +89,11 @@ outer:
 	return g.Next.ServeHTTP(w, r)
 }
 
-// newWriter create a new Gzip Writer based on the compression level.
-// If the level is valid (i.e. between 1 and 9), it uses the level.
-// Otherwise, it uses default compression level.
-func newWriter(c Config, w io.Writer) (*gzip.Writer, error) {
-	if c.Level >= gzip.BestSpeed && c.Level <= gzip.BestCompression {
-		return gzip.NewWriterLevel(w, c.Level)
-	}
-	return gzip.NewWriter(w), nil
-}
-
 // gzipResponeWriter wraps the underlying Write method
 // with a gzip.Writer to compress the output.
 type gzipResponseWriter struct {
 	io.Writer
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 	statusCodeWritten bool
 }
 
@@ -120,7 +105,7 @@ func (w *gzipResponseWriter) WriteHeader(code int) {
 	w.Header().Del("Content-Length")
 	w.Header().Set("Content-Encoding", "gzip")
 	w.Header().Add("Vary", "Accept-Encoding")
-	w.ResponseWriter.WriteHeader(code)
+	w.ResponseWriterWrapper.WriteHeader(code)
 	w.statusCodeWritten = true
 }
 
@@ -136,44 +121,5 @@ func (w *gzipResponseWriter) Write(b []byte) (int, error) {
 	return n, err
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (w *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := w.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: w.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (w *gzipResponseWriter) Flush() {
-	if f, ok := w.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: w.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (w *gzipResponseWriter) CloseNotify() <-chan bool {
-	if cn, ok := w.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: w.ResponseWriter})
-}
-
-func (w *gzipResponseWriter) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := w.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return errors.New("push is unavailable (probably chained http.ResponseWriter does not implement http.Pusher)")
-}
-
 // Interface guards
-var _ http.Pusher = (*gzipResponseWriter)(nil)
-var _ http.Flusher = (*gzipResponseWriter)(nil)
-var _ http.CloseNotifier = (*gzipResponseWriter)(nil)
-var _ http.Hijacker = (*gzipResponseWriter)(nil)
+var _ httpserver.HTTPInterfaces = (*gzipResponseWriter)(nil)

caddyhttp/gzip/gzip_test.go

@@ -1,6 +1,7 @@
 package gzip
 
 import (
+	"compress/gzip"
 	"fmt"
 	"io/ioutil"
 	"net/http"
@@ -77,6 +78,22 @@ func TestGzipHandler(t *testing.T) {
 			t.Error(err)
 		}
 	}
+
+	// test all levels
+	w = httptest.NewRecorder()
+	gz.Next = nextFunc(true)
+	for i := 0; i <= gzip.BestCompression; i++ {
+		gz.Configs[0].Level = i
+		r, err := http.NewRequest("GET", "/file.txt", nil)
+		if err != nil {
+			t.Error(err)
+		}
+		r.Header.Set("Accept-Encoding", "gzip")
+		_, err = gz.ServeHTTP(w, r)
+		if err != nil {
+			t.Error(err)
+		}
+	}
 }
 
 func nextFunc(shouldGzip bool) httpserver.Handler {
@@ -117,3 +134,37 @@ func nextFunc(shouldGzip bool) httpserver.Handler {
 		return 0, nil
 	})
 }
+
+func BenchmarkGzip(b *testing.B) {
+	pathFilter := PathFilter{make(Set)}
+	badPaths := []string{"/bad", "/nogzip", "/nongzip"}
+	for _, p := range badPaths {
+		pathFilter.IgnoredPaths.Add(p)
+	}
+	extFilter := ExtFilter{make(Set)}
+	for _, e := range []string{".txt", ".html", ".css", ".md"} {
+		extFilter.Exts.Add(e)
+	}
+	gz := Gzip{Configs: []Config{
+		{
+			RequestFilters: []RequestFilter{pathFilter, extFilter},
+		},
+	}}
+
+	w := httptest.NewRecorder()
+	gz.Next = nextFunc(true)
+	url := "/file.txt"
+	r, err := http.NewRequest("GET", url, nil)
+	if err != nil {
+		b.Fatal(err)
+	}
+	r.Header.Set("Accept-Encoding", "gzip")
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		_, err = gz.ServeHTTP(w, r)
+		if err != nil {
+			b.Fatal(err)
+		}
+	}
+}

caddyhttp/gzip/responsefilter_test.go

@@ -33,7 +33,7 @@ func TestLengthFilter(t *testing.T) {
 		for j, filter := range filters {
 			r := httptest.NewRecorder()
 			r.Header().Set("Content-Length", fmt.Sprint(ts.length))
-			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), r, false})
+			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), &httpserver.ResponseWriterWrapper{ResponseWriter: r}, false})
 			if filter.ShouldCompress(wWriter) != ts.shouldCompress[j] {
 				t.Errorf("Test %v: Expected %v found %v", i, ts.shouldCompress[j], filter.ShouldCompress(r))
 			}

caddyhttp/gzip/setup.go

@@ -1,9 +1,12 @@
 package gzip
 
 import (
+	"compress/gzip"
 	"fmt"
+	"io/ioutil"
 	"strconv"
 	"strings"
+	"sync"
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -119,3 +122,52 @@ func gzipParse(c *caddy.Controller) ([]Config, error) {
 
 	return configs, nil
 }
+
+// pool gzip.Writer according to compress level
+// so we can reuse allocations over time
+var (
+	writerPool             = map[int]*sync.Pool{}
+	defaultWriterPoolIndex int
+)
+
+func initWriterPool() {
+	var i int
+	newWriterPool := func(level int) *sync.Pool {
+		return &sync.Pool{
+			New: func() interface{} {
+				w, _ := gzip.NewWriterLevel(ioutil.Discard, level)
+				return w
+			},
+		}
+	}
+	for i = gzip.BestSpeed; i <= gzip.BestCompression; i++ {
+		writerPool[i] = newWriterPool(i)
+	}
+
+	// add default writer pool
+	defaultWriterPoolIndex = i
+	writerPool[defaultWriterPoolIndex] = &sync.Pool{
+		New: func() interface{} {
+			return gzip.NewWriter(ioutil.Discard)
+		},
+	}
+}
+
+func getWriter(level int) *gzip.Writer {
+	index := defaultWriterPoolIndex
+	if level >= gzip.BestSpeed && level <= gzip.BestCompression {
+		index = level
+	}
+	w := writerPool[index].Get().(*gzip.Writer)
+	w.Reset(ioutil.Discard)
+	return w
+}
+
+func putWriter(level int, w *gzip.Writer) {
+	index := defaultWriterPoolIndex
+	if level >= gzip.BestSpeed && level <= gzip.BestCompression {
+		index = level
+	}
+	w.Close()
+	writerPool[index].Put(w)
+}

caddyhttp/header/header.go

@@ -4,12 +4,9 @@
 package header
 
 import (
-	"bufio"
-	"net"
 	"net/http"
 	"strings"
 
-	"errors"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
@@ -24,7 +21,9 @@ type Headers struct {
 // setting headers on the response according to the configured rules.
 func (h Headers) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	replacer := httpserver.NewReplacer(r, nil, "")
-	rww := &responseWriterWrapper{ResponseWriter: w}
+	rww := &responseWriterWrapper{
+		ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
+	}
 	for _, rule := range h.Rules {
 		if httpserver.Path(r.URL.Path).Matches(rule.Path) {
 			for name := range rule.Headers {
@@ -63,20 +62,20 @@ type headerOperation func(http.Header)
 // responseWriterWrapper wraps the real ResponseWriter.
 // It defers header operations until writeHeader
 type responseWriterWrapper struct {
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 	ops         []headerOperation
 	wroteHeader bool
 }
 
 func (rww *responseWriterWrapper) Header() http.Header {
-	return rww.ResponseWriter.Header()
+	return rww.ResponseWriterWrapper.Header()
 }
 
 func (rww *responseWriterWrapper) Write(d []byte) (int, error) {
 	if !rww.wroteHeader {
 		rww.WriteHeader(http.StatusOK)
 	}
-	return rww.ResponseWriter.Write(d)
+	return rww.ResponseWriterWrapper.Write(d)
 }
 
 func (rww *responseWriterWrapper) WriteHeader(status int) {
@@ -92,7 +91,7 @@ func (rww *responseWriterWrapper) WriteHeader(status int) {
 		op(h)
 	}
 
-	rww.ResponseWriter.WriteHeader(status)
+	rww.ResponseWriterWrapper.WriteHeader(status)
 }
 
 // delHeader deletes the existing header according to the key
@@ -107,45 +106,5 @@ func (rww *responseWriterWrapper) delHeader(key string) {
 	})
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (rww *responseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := rww.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, httpserver.NonHijackerError{Underlying: rww.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (rww *responseWriterWrapper) Flush() {
-	if f, ok := rww.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(httpserver.NonFlusherError{Underlying: rww.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-// It panics if the underlying ResponseWriter is not a CloseNotifier.
-func (rww *responseWriterWrapper) CloseNotify() <-chan bool {
-	if cn, ok := rww.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(httpserver.NonCloseNotifierError{Underlying: rww.ResponseWriter})
-}
-
-func (rww *responseWriterWrapper) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := rww.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return errors.New("push is unavailable (probably chained http.ResponseWriter does not implement http.Pusher)")
-}
-
 // Interface guards
-var _ http.Pusher = (*responseWriterWrapper)(nil)
-var _ http.Flusher = (*responseWriterWrapper)(nil)
-var _ http.CloseNotifier = (*responseWriterWrapper)(nil)
-var _ http.Hijacker = (*responseWriterWrapper)(nil)
+var _ httpserver.HTTPInterfaces = (*responseWriterWrapper)(nil)

caddyhttp/httpserver/condition_test.go

@@ -1,6 +1,7 @@
 package httpserver
 
 import (
+	"context"
 	"fmt"
 	"net/http"
 	"strings"
@@ -94,16 +95,21 @@ func TestConditions(t *testing.T) {
 	for i, test := range replaceTests {
 		r, err := http.NewRequest("GET", test.url, nil)
 		if err != nil {
-			t.Error(err)
+			t.Errorf("Test %d: failed to create request: %v", i, err)
+			continue
 		}
+		ctx := context.WithValue(r.Context(), OriginalURLCtxKey, *r.URL)
+		r = r.WithContext(ctx)
 		str := strings.Fields(test.condition)
 		ifCond, err := newIfCond(str[0], str[1], str[2])
 		if err != nil {
-			t.Error(err)
+			t.Errorf("Test %d: failed to create 'if' condition %v", i, err)
+			continue
 		}
 		isTrue := ifCond.True(r)
 		if isTrue != test.isTrue {
 			t.Errorf("Test %v: expected %v found %v", i, test.isTrue, isTrue)
+			continue
 		}
 	}
 }

caddyhttp/httpserver/error.go

@@ -8,6 +8,7 @@ var (
 	_ error = NonHijackerError{}
 	_ error = NonFlusherError{}
 	_ error = NonCloseNotifierError{}
+	_ error = NonPusherError{}
 )
 
 // NonHijackerError is more descriptive error caused by a non hijacker
@@ -42,3 +43,14 @@ type NonCloseNotifierError struct {
 func (c NonCloseNotifierError) Error() string {
 	return fmt.Sprintf("%T is not a closeNotifier", c.Underlying)
 }
+
+// NonPusherError is more descriptive error caused by a non pusher
+type NonPusherError struct {
+	// underlying type which doesn't implement pusher
+	Underlying interface{}
+}
+
+// Implement Error
+func (c NonPusherError) Error() string {
+	return fmt.Sprintf("%T is not a pusher", c.Underlying)
+}

caddyhttp/httpserver/https.go

@@ -146,13 +146,20 @@ func redirPlaintextHost(cfg *SiteConfig) *SiteConfig {
 	}
 	redirMiddleware := func(next Handler) Handler {
 		return HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			// Construct the URL to which to redirect. Note that the Host in a request might
+			// contain a port, but we just need the hostname; we'll set the port if needed.
 			toURL := "https://"
+			requestHost, _, err := net.SplitHostPort(r.Host)
+			if err != nil {
+				requestHost = r.Host // Host did not contain a port; great
+			}
 			if redirPort == "" {
-				toURL += cfg.Addr.Host // don't use r.Host as it may have a port included
+				toURL += requestHost
 			} else {
-				toURL += net.JoinHostPort(cfg.Addr.Host, redirPort)
+				toURL += net.JoinHostPort(requestHost, redirPort)
 			}
 			toURL += r.URL.RequestURI()
+
 			w.Header().Set("Connection", "close")
 			http.Redirect(w, r, toURL, http.StatusMovedPermanently)
 			return 0, nil

caddyhttp/httpserver/https_test.go

@@ -1,6 +1,8 @@
 package httpserver
 
 import (
+	"fmt"
+	"net"
 	"net/http"
 	"net/http/httptest"
 	"testing"
@@ -9,75 +11,107 @@ import (
 )
 
 func TestRedirPlaintextHost(t *testing.T) {
-	cfg := redirPlaintextHost(&SiteConfig{
-		Addr: Address{
+	for i, testcase := range []struct {
+		Host        string // used for the site config
+		Port        string
+		ListenHost  string
+		RequestHost string // if different from Host
+	}{
+		{
+			Host: "foohost",
+		},
+		{
+			Host: "foohost",
+			Port: "80",
+		},
+		{
 			Host: "foohost",
 			Port: "1234",
 		},
-		ListenHost: "93.184.216.34",
-		TLS:        new(caddytls.Config),
-	})
+		{
+			Host:       "foohost",
+			ListenHost: "93.184.216.34",
+		},
+		{
+			Host:       "foohost",
+			Port:       "1234",
+			ListenHost: "93.184.216.34",
+		},
+		{
+			Host: "foohost",
+			Port: "443", // since this is the default HTTPS port, should not be included in Location value
+		},
+		{
+			Host:        "*.example.com",
+			RequestHost: "foo.example.com",
+		},
+		{
+			Host:        "*.example.com",
+			Port:        "1234",
+			RequestHost: "foo.example.com:1234",
+		},
+	} {
+		cfg := redirPlaintextHost(&SiteConfig{
+			Addr: Address{
+				Host: testcase.Host,
+				Port: testcase.Port,
+			},
+			ListenHost: testcase.ListenHost,
+			TLS:        new(caddytls.Config),
+		})
 
-	// Check host and port
-	if actual, expected := cfg.Addr.Host, "foohost"; actual != expected {
-		t.Errorf("Expected redir config to have host %s but got %s", expected, actual)
-	}
-	if actual, expected := cfg.ListenHost, "93.184.216.34"; actual != expected {
-		t.Errorf("Expected redir config to have bindhost %s but got %s", expected, actual)
-	}
-	if actual, expected := cfg.Addr.Port, "80"; actual != expected {
-		t.Errorf("Expected redir config to have port '%s' but got '%s'", expected, actual)
-	}
+		// Check host and port
+		if actual, expected := cfg.Addr.Host, testcase.Host; actual != expected {
+			t.Errorf("Test %d: Expected redir config to have host %s but got %s", i, expected, actual)
+		}
+		if actual, expected := cfg.ListenHost, testcase.ListenHost; actual != expected {
+			t.Errorf("Test %d: Expected redir config to have bindhost %s but got %s", i, expected, actual)
+		}
+		if actual, expected := cfg.Addr.Port, HTTPPort; actual != expected {
+			t.Errorf("Test %d: Expected redir config to have port '%s' but got '%s'", i, expected, actual)
+		}
 
-	// Make sure redirect handler is set up properly
-	if cfg.middleware == nil || len(cfg.middleware) != 1 {
-		t.Fatalf("Redir config middleware not set up properly; got: %#v", cfg.middleware)
-	}
+		// Make sure redirect handler is set up properly
+		if cfg.middleware == nil || len(cfg.middleware) != 1 {
+			t.Fatalf("Test %d: Redir config middleware not set up properly; got: %#v", i, cfg.middleware)
+		}
 
-	handler := cfg.middleware[0](nil)
+		handler := cfg.middleware[0](nil)
 
-	// Check redirect for correctness
-	rec := httptest.NewRecorder()
-	req, err := http.NewRequest("GET", "http://foohost/bar?q=1", nil)
-	if err != nil {
-		t.Fatal(err)
-	}
-	status, err := handler.ServeHTTP(rec, req)
-	if status != 0 {
-		t.Errorf("Expected status return to be 0, but was %d", status)
-	}
-	if err != nil {
-		t.Errorf("Expected returned error to be nil, but was %v", err)
-	}
-	if rec.Code != http.StatusMovedPermanently {
-		t.Errorf("Expected status %d but got %d", http.StatusMovedPermanently, rec.Code)
-	}
-	if got, want := rec.Header().Get("Location"), "https://foohost:1234/bar?q=1"; got != want {
-		t.Errorf("Expected Location: '%s' but got '%s'", want, got)
-	}
+		// Check redirect for correctness, first by inspecting error and status code
+		requestHost := testcase.Host // hostname of request might be different than in config (e.g. wildcards)
+		if testcase.RequestHost != "" {
+			requestHost = testcase.RequestHost
+		}
+		rec := httptest.NewRecorder()
+		req, err := http.NewRequest("GET", "http://"+requestHost+"/bar?q=1", nil)
+		if err != nil {
+			t.Fatalf("Test %d: %v", i, err)
+		}
+		status, err := handler.ServeHTTP(rec, req)
+		if status != 0 {
+			t.Errorf("Test %d: Expected status return to be 0, but was %d", i, status)
+		}
+		if err != nil {
+			t.Errorf("Test %d: Expected returned error to be nil, but was %v", i, err)
+		}
+		if rec.Code != http.StatusMovedPermanently {
+			t.Errorf("Test %d: Expected status %d but got %d", http.StatusMovedPermanently, i, rec.Code)
+		}
 
-	// browsers can infer a default port from scheme, so make sure the port
-	// doesn't get added in explicitly for default ports like 443 for https.
-	cfg = redirPlaintextHost(&SiteConfig{Addr: Address{Host: "foohost", Port: "443"}, TLS: new(caddytls.Config)})
-	handler = cfg.middleware[0](nil)
-
-	rec = httptest.NewRecorder()
-	req, err = http.NewRequest("GET", "http://foohost/bar?q=1", nil)
-	if err != nil {
-		t.Fatal(err)
-	}
-	status, err = handler.ServeHTTP(rec, req)
-	if status != 0 {
-		t.Errorf("Expected status return to be 0, but was %d", status)
-	}
-	if err != nil {
-		t.Errorf("Expected returned error to be nil, but was %v", err)
-	}
-	if rec.Code != http.StatusMovedPermanently {
-		t.Errorf("Expected status %d but got %d", http.StatusMovedPermanently, rec.Code)
-	}
-	if got, want := rec.Header().Get("Location"), "https://foohost/bar?q=1"; got != want {
-		t.Errorf("Expected Location: '%s' but got '%s'", want, got)
+		// Now check the Location value. It should mirror the hostname and port of the request
+		// unless the port is redundant, in which case it should be dropped.
+		locationHost, _, err := net.SplitHostPort(requestHost)
+		if err != nil {
+			locationHost = requestHost
+		}
+		expectedLoc := fmt.Sprintf("https://%s/bar?q=1", locationHost)
+		if testcase.Port != "" && testcase.Port != DefaultHTTPSPort {
+			expectedLoc = fmt.Sprintf("https://%s:%s/bar?q=1", locationHost, testcase.Port)
+		}
+		if got, want := rec.Header().Get("Location"), expectedLoc; got != want {
+			t.Errorf("Test %d: Expected Location: '%s' but got '%s'", i, want, got)
+		}
 	}
 }
 

caddyhttp/httpserver/middleware.go

@@ -198,15 +198,11 @@ func SameNext(next1, next2 Handler) bool {
 	return fmt.Sprintf("%v", next1) == fmt.Sprintf("%v", next2)
 }
 
-// Context key constants
+// Context key constants.
 const (
-	// URIxRewriteCtxKey is a context key used to store original unrewritten
-	// URI in context.WithValue
-	URIxRewriteCtxKey caddy.CtxKey = "caddy_rewrite_original_uri"
-
-	// RemoteUserCtxKey is a context key used to store remote user for request
+	// RemoteUserCtxKey is the key for the remote user of the request, if any (basicauth).
 	RemoteUserCtxKey caddy.CtxKey = "remote_user"
 
-	// MitmCtxKey stores Mitm result
+	// MitmCtxKey is the key for the result of MITM detection
 	MitmCtxKey caddy.CtxKey = "mitm"
 )

caddyhttp/httpserver/mitm.go

@@ -133,7 +133,7 @@ func (c *clientHelloConn) Read(b []byte) (n int, err error) {
 	if err != nil {
 		return
 	}
-	c.buf = nil // buffer no longer needed
+	bufpool.Put(c.buf) // buffer no longer needed
 
 	// parse the ClientHello and store it in the map
 	rawParsed := parseRawClientHello(hello)
@@ -161,11 +161,11 @@ func parseRawClientHello(data []byte) (info rawHelloInfo) {
 	if len(data) < 42 {
 		return
 	}
-	sessionIdLen := int(data[38])
-	if sessionIdLen > 32 || len(data) < 39+sessionIdLen {
+	sessionIDLen := int(data[38])
+	if sessionIDLen > 32 || len(data) < 39+sessionIDLen {
 		return
 	}
-	data = data[39+sessionIdLen:]
+	data = data[39+sessionIDLen:]
 	if len(data) < 2 {
 		return
 	}
@@ -285,7 +285,9 @@ func (l *tlsHelloListener) Accept() (net.Conn, error) {
 	if err != nil {
 		return nil, err
 	}
-	helloConn := &clientHelloConn{Conn: conn, listener: l, buf: new(bytes.Buffer)}
+	buf := bufpool.Get().(*bytes.Buffer)
+	buf.Reset()
+	helloConn := &clientHelloConn{Conn: conn, listener: l, buf: buf}
 	return tls.Server(helloConn, l.config), nil
 }
 
@@ -570,6 +572,13 @@ func hasGreaseCiphers(cipherSuites []uint16) bool {
 	return false
 }
 
+// pool buffers so we can reuse allocations over time
+var bufpool = sync.Pool{
+	New: func() interface{} {
+		return new(bytes.Buffer)
+	},
+}
+
 var greaseCiphers = map[uint16]struct{}{
 	0x0A0A: {},
 	0x1A1A: {},
@@ -589,6 +598,7 @@ var greaseCiphers = map[uint16]struct{}{
 	0xFAFA: {},
 }
 
+// Define variables used for TLS communication
 const (
 	extensionOCSPStatusRequest = 5
 	extensionSupportedCurves   = 10 // also called "SupportedGroups"

caddyhttp/httpserver/path.go

@@ -5,19 +5,25 @@ import (
 	"strings"
 )
 
-// Path represents a URI path.
+// Path represents a URI path. It should usually be
+// set to the value of a request path.
 type Path string
 
-// Matches checks to see if other matches p.
+// Matches checks to see if base matches p. The correct
+// usage of this method sets p as the request path, and
+// base as a Caddyfile (user-defined) rule path.
 //
 // Path matching will probably not always be a direct
 // comparison; this method assures that paths can be
 // easily and consistently matched.
-func (p Path) Matches(other string) bool {
-	if CaseSensitivePath {
-		return strings.HasPrefix(string(p), other)
+func (p Path) Matches(base string) bool {
+	if base == "/" {
+		return true
 	}
-	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(other))
+	if CaseSensitivePath {
+		return strings.HasPrefix(string(p), base)
+	}
+	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(base))
 }
 
 // PathMatcher is a Path RequestMatcher.

caddyhttp/httpserver/path_test.go

@@ -0,0 +1,96 @@
+package httpserver
+
+import "testing"
+
+func TestPathMatches(t *testing.T) {
+	for i, testcase := range []struct {
+		reqPath         Path
+		rulePath        string // or "base path" as in Caddyfile docs
+		shouldMatch     bool
+		caseInsensitive bool
+	}{
+		{
+			reqPath:     "/",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "/foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/bar",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/Foobar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+
+			reqPath:     "/FooBar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:         "/foobar",
+			rulePath:        "/FooBar",
+			shouldMatch:     true,
+			caseInsensitive: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "/", // a lone forward slash means to match all requests (see issue #1645) - many future test cases related to this issue
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "foobar.php",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/no/leading/slash",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+	} {
+		CaseSensitivePath = !testcase.caseInsensitive
+		if got, want := testcase.reqPath.Matches(testcase.rulePath), testcase.shouldMatch; got != want {
+			t.Errorf("Test %d: For request path '%s' and other path '%s': expected %v, got %v",
+				i, testcase.reqPath, testcase.rulePath, want, got)
+		}
+	}
+}

caddyhttp/httpserver/pathcleaner.go

@@ -1,76 +0,0 @@
-package httpserver
-
-import (
-	"math/rand"
-	"path"
-	"strings"
-	"time"
-)
-
-// CleanMaskedPath prevents one or more of the path cleanup operations:
-//   - collapse multiple slashes into one
-//   - eliminate "/." (current directory)
-//   - eliminate "<parent_directory>/.."
-// by masking certain patterns in the path with a temporary random string.
-// This could be helpful when certain patterns in the path are desired to be preserved
-// that would otherwise be changed by path.Clean().
-// One such use case is the presence of the double slashes as protocol separator
-// (e.g., /api/endpoint/http://example.com).
-// This is a common pattern in many applications to allow passing URIs as path argument.
-func CleanMaskedPath(reqPath string, masks ...string) string {
-	var replacerVal string
-	maskMap := make(map[string]string)
-
-	// Iterate over supplied masks and create temporary replacement strings
-	// only for the masks that are present in the path, then replace all occurrences
-	for _, mask := range masks {
-		if strings.Index(reqPath, mask) >= 0 {
-			replacerVal = "/_caddy" + generateRandomString() + "__"
-			maskMap[mask] = replacerVal
-			reqPath = strings.Replace(reqPath, mask, replacerVal, -1)
-		}
-	}
-
-	reqPath = path.Clean(reqPath)
-
-	// Revert the replaced masks after path cleanup
-	for mask, replacerVal := range maskMap {
-		reqPath = strings.Replace(reqPath, replacerVal, mask, -1)
-	}
-	return reqPath
-}
-
-// CleanPath calls CleanMaskedPath() with the default mask of "://"
-// to preserve double slashes of protocols
-// such as "http://", "https://", and "ftp://" etc.
-func CleanPath(reqPath string) string {
-	return CleanMaskedPath(reqPath, "://")
-}
-
-// An efficient and fast method for random string generation.
-// Inspired by http://stackoverflow.com/a/31832326.
-const randomStringLength = 4
-const letterBytes = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
-const (
-	letterIdxBits = 6
-	letterIdxMask = 1<<letterIdxBits - 1
-	letterIdxMax  = 63 / letterIdxBits
-)
-
-var src = rand.NewSource(time.Now().UnixNano())
-
-func generateRandomString() string {
-	b := make([]byte, randomStringLength)
-	for i, cache, remain := randomStringLength-1, src.Int63(), letterIdxMax; i >= 0; {
-		if remain == 0 {
-			cache, remain = src.Int63(), letterIdxMax
-		}
-		if idx := int(cache & letterIdxMask); idx < len(letterBytes) {
-			b[i] = letterBytes[idx]
-			i--
-		}
-		cache >>= letterIdxBits
-		remain--
-	}
-	return string(b)
-}

caddyhttp/httpserver/pathcleaner_test.go

@@ -1,120 +0,0 @@
-package httpserver
-
-import (
-	"path"
-	"testing"
-)
-
-var paths = map[string]map[string]string{
-	"/../a/b/../././/c": {
-		"preserve_all":      "/../a/b/../././/c",
-		"preserve_protocol": "/a/c",
-		"preserve_slashes":  "/a//c",
-		"preserve_dots":     "/../a/b/../././c",
-		"clean_all":         "/a/c",
-	},
-	"/path/https://www.google.com": {
-		"preserve_all":      "/path/https://www.google.com",
-		"preserve_protocol": "/path/https://www.google.com",
-		"preserve_slashes":  "/path/https://www.google.com",
-		"preserve_dots":     "/path/https:/www.google.com",
-		"clean_all":         "/path/https:/www.google.com",
-	},
-	"/a/b/../././/c/http://example.com/foo//bar/../blah": {
-		"preserve_all":      "/a/b/../././/c/http://example.com/foo//bar/../blah",
-		"preserve_protocol": "/a/c/http://example.com/foo/blah",
-		"preserve_slashes":  "/a//c/http://example.com/foo/blah",
-		"preserve_dots":     "/a/b/../././c/http:/example.com/foo/bar/../blah",
-		"clean_all":         "/a/c/http:/example.com/foo/blah",
-	},
-}
-
-func assertEqual(t *testing.T, expected, received string) {
-	if expected != received {
-		t.Errorf("\tExpected: %s\n\t\t\tReceived: %s", expected, received)
-	}
-}
-
-func maskedTestRunner(t *testing.T, variation string, masks ...string) {
-	for reqPath, transformation := range paths {
-		assertEqual(t, transformation[variation], CleanMaskedPath(reqPath, masks...))
-	}
-}
-
-// No need to test the built-in path.Clean() function.
-// However, it could be useful to cross-examine the test dataset.
-func TestPathClean(t *testing.T) {
-	for reqPath, transformation := range paths {
-		assertEqual(t, transformation["clean_all"], path.Clean(reqPath))
-	}
-}
-
-func TestCleanAll(t *testing.T) {
-	maskedTestRunner(t, "clean_all")
-}
-
-func TestPreserveAll(t *testing.T) {
-	maskedTestRunner(t, "preserve_all", "//", "/..", "/.")
-}
-
-func TestPreserveProtocol(t *testing.T) {
-	maskedTestRunner(t, "preserve_protocol", "://")
-}
-
-func TestPreserveSlashes(t *testing.T) {
-	maskedTestRunner(t, "preserve_slashes", "//")
-}
-
-func TestPreserveDots(t *testing.T) {
-	maskedTestRunner(t, "preserve_dots", "/..", "/.")
-}
-
-func TestDefaultMask(t *testing.T) {
-	for reqPath, transformation := range paths {
-		assertEqual(t, transformation["preserve_protocol"], CleanPath(reqPath))
-	}
-}
-
-func maskedBenchmarkRunner(b *testing.B, masks ...string) {
-	for n := 0; n < b.N; n++ {
-		for reqPath := range paths {
-			CleanMaskedPath(reqPath, masks...)
-		}
-	}
-}
-
-func BenchmarkPathClean(b *testing.B) {
-	for n := 0; n < b.N; n++ {
-		for reqPath := range paths {
-			path.Clean(reqPath)
-		}
-	}
-}
-
-func BenchmarkCleanAll(b *testing.B) {
-	maskedBenchmarkRunner(b)
-}
-
-func BenchmarkPreserveAll(b *testing.B) {
-	maskedBenchmarkRunner(b, "//", "/..", "/.")
-}
-
-func BenchmarkPreserveProtocol(b *testing.B) {
-	maskedBenchmarkRunner(b, "://")
-}
-
-func BenchmarkPreserveSlashes(b *testing.B) {
-	maskedBenchmarkRunner(b, "//")
-}
-
-func BenchmarkPreserveDots(b *testing.B) {
-	maskedBenchmarkRunner(b, "/..", "/.")
-}
-
-func BenchmarkDefaultMask(b *testing.B) {
-	for n := 0; n < b.N; n++ {
-		for reqPath := range paths {
-			CleanPath(reqPath)
-		}
-	}
-}

caddyhttp/httpserver/plugin.go

@@ -70,7 +70,7 @@ func hideCaddyfile(cctx caddy.Context) error {
 			return err
 		}
 		if strings.HasPrefix(absOriginCaddyfile, absRoot) {
-			cfg.HiddenFiles = append(cfg.HiddenFiles, strings.TrimPrefix(absOriginCaddyfile, absRoot))
+			cfg.HiddenFiles = append(cfg.HiddenFiles, filepath.ToSlash(strings.TrimPrefix(absOriginCaddyfile, absRoot)))
 		}
 	}
 	return nil
@@ -254,7 +254,7 @@ func groupSiteConfigsByListenAddr(configs []*SiteConfig) (map[string][]*SiteConf
 		// We would add a special case here so that localhost addresses
 		// bind to 127.0.0.1 if conf.ListenHost is not already set, which
 		// would prevent outsiders from even connecting; but that was problematic:
-		// https://forum.caddyserver.com/t/wildcard-virtual-domains-with-wildcard-roots/221/5?u=matt
+		// https://caddy.community/t/wildcard-virtual-domains-with-wildcard-roots/221/5?u=matt
 
 		if conf.Addr.Port == "" {
 			conf.Addr.Port = Port
@@ -436,7 +436,7 @@ var directives = []string{
 	"root",
 	"index",
 	"bind",
-	"maxrequestbody", // TODO: 'limits'
+	"limits",
 	"timeouts",
 	"tls",
 
@@ -468,6 +468,7 @@ var directives = []string{
 	"status",
 	"cors", // github.com/captncraig/cors/caddy
 	"mime",
+	"login",     // github.com/tarent/loginsrv/caddy
 	"jwt",       // github.com/BTBurke/caddy-jwt
 	"jsonp",     // github.com/pschlump/caddy-jsonp
 	"upload",    // blitznote.com/src/caddy.upload
@@ -476,6 +477,7 @@ var directives = []string{
 	"pprof",
 	"expvar",
 	"push",
+	"datadog",    // github.com/payintech/caddy-datadog
 	"prometheus", // github.com/miekg/caddy-prometheus
 	"proxy",
 	"fastcgi",

caddyhttp/httpserver/plugin_test.go

@@ -209,3 +209,27 @@ func TestContextSaveConfig(t *testing.T) {
 		t.Errorf("Expected len(siteConfigs) == %d, but was %d", want, got)
 	}
 }
+
+// Test to make sure we are correctly hiding the Caddyfile
+func TestHideCaddyfile(t *testing.T) {
+	ctx := newContext().(*httpContext)
+	ctx.saveConfig("test", &SiteConfig{
+		Root:            Root,
+		originCaddyfile: "Testfile",
+	})
+	err := hideCaddyfile(ctx)
+	if err != nil {
+		t.Fatalf("Failed to hide Caddyfile, got: %v", err)
+		return
+	}
+	if len(ctx.siteConfigs[0].HiddenFiles) == 0 {
+		t.Fatal("Failed to add Caddyfile to HiddenFiles.")
+		return
+	}
+	for _, file := range ctx.siteConfigs[0].HiddenFiles {
+		if file == "/Testfile" {
+			return
+		}
+	}
+	t.Fatal("Caddyfile missing from HiddenFiles")
+}

caddyhttp/httpserver/recorder.go

@@ -1,9 +1,6 @@
 package httpserver
 
 import (
-	"bufio"
-	"errors"
-	"net"
 	"net/http"
 	"time"
 )
@@ -21,7 +18,7 @@ import (
 //
 // Beware when accessing the Replacer value; it may be nil!
 type ResponseRecorder struct {
-	http.ResponseWriter
+	*ResponseWriterWrapper
 	Replacer Replacer
 	status   int
 	size     int
@@ -36,9 +33,9 @@ type ResponseRecorder struct {
 // of 200 to cover the default case.
 func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 	return &ResponseRecorder{
-		ResponseWriter: w,
-		status:         http.StatusOK,
-		start:          time.Now(),
+		ResponseWriterWrapper: &ResponseWriterWrapper{ResponseWriter: w},
+		status:                http.StatusOK,
+		start:                 time.Now(),
 	}
 }
 
@@ -46,13 +43,13 @@ func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 // underlying ResponseWriter's WriteHeader method.
 func (r *ResponseRecorder) WriteHeader(status int) {
 	r.status = status
-	r.ResponseWriter.WriteHeader(status)
+	r.ResponseWriterWrapper.WriteHeader(status)
 }
 
 // Write is a wrapper that records the size of the body
 // that gets written.
 func (r *ResponseRecorder) Write(buf []byte) (int, error) {
-	n, err := r.ResponseWriter.Write(buf)
+	n, err := r.ResponseWriterWrapper.Write(buf)
 	if err == nil {
 		r.size += n
 	}
@@ -69,45 +66,5 @@ func (r *ResponseRecorder) Status() int {
 	return r.status
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (r *ResponseRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := r.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, NonHijackerError{Underlying: r.ResponseWriter}
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or does nothing.
-func (r *ResponseRecorder) Flush() {
-	if f, ok := r.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic(NonFlusherError{Underlying: r.ResponseWriter}) // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (r *ResponseRecorder) CloseNotify() <-chan bool {
-	if cn, ok := r.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic(NonCloseNotifierError{Underlying: r.ResponseWriter})
-}
-
-// Push resource to client
-func (r *ResponseRecorder) Push(target string, opts *http.PushOptions) error {
-	if pusher, hasPusher := r.ResponseWriter.(http.Pusher); hasPusher {
-		return pusher.Push(target, opts)
-	}
-
-	return errors.New("push is unavailable (probably chained http.ResponseWriter does not implement http.Pusher)")
-}
-
 // Interface guards
-var _ http.Pusher = (*ResponseRecorder)(nil)
-var _ http.Flusher = (*ResponseRecorder)(nil)
-var _ http.CloseNotifier = (*ResponseRecorder)(nil)
-var _ http.Hijacker = (*ResponseRecorder)(nil)
+var _ HTTPInterfaces = (*ResponseRecorder)(nil)

caddyhttp/httpserver/replacer.go

@@ -238,37 +238,24 @@ func (r *replacer) getSubstitution(key string) string {
 		}
 		return host
 	case "{path}":
-		// if a rewrite has happened, the original URI should be used as the path
-		// rather than the rewritten URI
-		var path string
-		origpath, _ := r.request.Context().Value(URIxRewriteCtxKey).(string)
-		if origpath == "" {
-			path = r.request.URL.Path
-		} else {
-			parsedURL, _ := url.Parse(origpath)
-			path = parsedURL.Path
-		}
-		return path
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.Path
 	case "{path_escaped}":
-		var path string
-		origpath, _ := r.request.Context().Value(URIxRewriteCtxKey).(string)
-		if origpath == "" {
-			path = r.request.URL.Path
-		} else {
-			parsedURL, _ := url.Parse(origpath)
-			path = parsedURL.Path
-		}
-		return url.QueryEscape(path)
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return url.QueryEscape(u.Path)
 	case "{rewrite_path}":
 		return r.request.URL.Path
 	case "{rewrite_path_escaped}":
 		return url.QueryEscape(r.request.URL.Path)
 	case "{query}":
-		return r.request.URL.RawQuery
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.RawQuery
 	case "{query_escaped}":
-		return url.QueryEscape(r.request.URL.RawQuery)
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return url.QueryEscape(u.RawQuery)
 	case "{fragment}":
-		return r.request.URL.Fragment
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.Fragment
 	case "{proto}":
 		return r.request.Proto
 	case "{remote}":
@@ -284,17 +271,11 @@ func (r *replacer) getSubstitution(key string) string {
 		}
 		return port
 	case "{uri}":
-		uri, _ := r.request.Context().Value(URIxRewriteCtxKey).(string)
-		if uri == "" {
-			uri = r.request.URL.RequestURI()
-		}
-		return uri
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.RequestURI()
 	case "{uri_escaped}":
-		uri, _ := r.request.Context().Value(URIxRewriteCtxKey).(string)
-		if uri == "" {
-			uri = r.request.URL.RequestURI()
-		}
-		return url.QueryEscape(uri)
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return url.QueryEscape(u.RequestURI())
 	case "{rewrite_uri}":
 		return r.request.URL.RequestURI()
 	case "{rewrite_uri_escaped}":
@@ -321,7 +302,7 @@ func (r *replacer) getSubstitution(key string) string {
 		}
 		_, err := ioutil.ReadAll(r.request.Body)
 		if err != nil {
-			if _, ok := err.(MaxBytesExceeded); ok {
+			if err == ErrMaxBytesExceeded {
 				return r.emptyValue
 			}
 		}
@@ -330,9 +311,9 @@ func (r *replacer) getSubstitution(key string) string {
 		if val, ok := r.request.Context().Value(caddy.CtxKey("mitm")).(bool); ok {
 			if val {
 				return "likely"
-			} else {
-				return "unlikely"
 			}
+
+			return "unlikely"
 		}
 		return "unknown"
 	case "{status}":

caddyhttp/httpserver/replacer_test.go

@@ -41,8 +41,11 @@ func TestReplace(t *testing.T) {
 
 	request, err := http.NewRequest("POST", "http://localhost/?foo=bar", reader)
 	if err != nil {
-		t.Fatal("Request Formation Failed\n")
+		t.Fatalf("Failed to make request: %v", err)
 	}
+	ctx := context.WithValue(request.Context(), OriginalURLCtxKey, *request.URL)
+	request = request.WithContext(ctx)
+
 	request.Header.Set("Custom", "foobarbaz")
 	request.Header.Set("ShorterVal", "1")
 	repl := NewReplacer(request, recordRequest, "-")
@@ -52,7 +55,7 @@ func TestReplace(t *testing.T) {
 
 	hostname, err := os.Hostname()
 	if err != nil {
-		t.Fatal("Failed to determine hostname\n")
+		t.Fatalf("Failed to determine hostname: %v", err)
 	}
 
 	old := now
@@ -161,25 +164,26 @@ func TestPathRewrite(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Request Formation Failed: %s\n", err.Error())
 	}
-
-	ctx := context.WithValue(request.Context(), URIxRewriteCtxKey, "a/custom/path.php?key=value")
+	urlCopy := *request.URL
+	urlCopy.Path = "a/custom/path.php"
+	ctx := context.WithValue(request.Context(), OriginalURLCtxKey, urlCopy)
 	request = request.WithContext(ctx)
 
 	repl := NewReplacer(request, recordRequest, "")
 
-	if repl.Replace("This path is '{path}'") != "This path is 'a/custom/path.php'" {
-		t.Error("Expected host {path} replacement failed  (" + repl.Replace("This path is '{path}'") + ")")
+	if got, want := repl.Replace("This path is '{path}'"), "This path is 'a/custom/path.php'"; got != want {
+		t.Errorf("{path} replacement failed; got '%s', want '%s'", got, want)
 	}
 
-	if repl.Replace("This path is {rewrite_path}") != "This path is /index.php" {
-		t.Error("Expected host {rewrite_path} replacement failed (" + repl.Replace("This path is {rewrite_path}") + ")")
+	if got, want := repl.Replace("This path is {rewrite_path}"), "This path is /index.php"; got != want {
+		t.Errorf("{rewrite_path} replacement failed; got '%s', want '%s'", got, want)
 	}
-	if repl.Replace("This path is '{uri}'") != "This path is 'a/custom/path.php?key=value'" {
-		t.Error("Expected host {uri} replacement failed  (" + repl.Replace("This path is '{uri}'") + ")")
+	if got, want := repl.Replace("This path is '{uri}'"), "This path is 'a/custom/path.php?key=value'"; got != want {
+		t.Errorf("{uri} replacement failed; got '%s', want '%s'", got, want)
 	}
 
-	if repl.Replace("This path is {rewrite_uri}") != "This path is /index.php?key=value" {
-		t.Error("Expected host {rewrite_uri} replacement failed (" + repl.Replace("This path is {rewrite_uri}") + ")")
+	if got, want := repl.Replace("This path is {rewrite_uri}"), "This path is /index.php?key=value"; got != want {
+		t.Errorf("{rewrite_uri} replacement failed; got '%s', want '%s'", got, want)
 	}
 
 }

caddyhttp/httpserver/responsewriterwrapper.go

@@ -0,0 +1,65 @@
+package httpserver
+
+import (
+	"bufio"
+	"net"
+	"net/http"
+)
+
+// ResponseWriterWrapper wrappers underlying ResponseWriter
+// and inherits its Hijacker/Pusher/CloseNotifier/Flusher as well.
+type ResponseWriterWrapper struct {
+	http.ResponseWriter
+}
+
+// Hijack implements http.Hijacker. It simply wraps the underlying
+// ResponseWriter's Hijack method if there is one, or returns an error.
+func (rww *ResponseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) {
+	if hj, ok := rww.ResponseWriter.(http.Hijacker); ok {
+		return hj.Hijack()
+	}
+	return nil, nil, NonHijackerError{Underlying: rww.ResponseWriter}
+}
+
+// Flush implements http.Flusher. It simply wraps the underlying
+// ResponseWriter's Flush method if there is one, or panics.
+func (rww *ResponseWriterWrapper) Flush() {
+	if f, ok := rww.ResponseWriter.(http.Flusher); ok {
+		f.Flush()
+	} else {
+		panic(NonFlusherError{Underlying: rww.ResponseWriter})
+	}
+}
+
+// CloseNotify implements http.CloseNotifier.
+// It just inherits the underlying ResponseWriter's CloseNotify method.
+// It panics if the underlying ResponseWriter is not a CloseNotifier.
+func (rww *ResponseWriterWrapper) CloseNotify() <-chan bool {
+	if cn, ok := rww.ResponseWriter.(http.CloseNotifier); ok {
+		return cn.CloseNotify()
+	}
+	panic(NonCloseNotifierError{Underlying: rww.ResponseWriter})
+}
+
+// Push implements http.Pusher.
+// It just inherits the underlying ResponseWriter's Push method.
+// It panics if the underlying ResponseWriter is not a Pusher.
+func (rww *ResponseWriterWrapper) Push(target string, opts *http.PushOptions) error {
+	if pusher, hasPusher := rww.ResponseWriter.(http.Pusher); hasPusher {
+		return pusher.Push(target, opts)
+	}
+
+	return NonPusherError{Underlying: rww.ResponseWriter}
+}
+
+// HTTPInterfaces mix all the interfaces that middleware ResponseWriters need to support.
+type HTTPInterfaces interface {
+	http.ResponseWriter
+	http.Pusher
+	http.Flusher
+	http.CloseNotifier
+	http.Hijacker
+}
+
+// Interface guards
+var _ HTTPInterfaces = (*ResponseWriterWrapper)(nil)

caddyhttp/httpserver/server.go

@@ -4,12 +4,15 @@ package httpserver
 import (
 	"context"
 	"crypto/tls"
+	"errors"
 	"fmt"
-	"io"
 	"log"
 	"net"
 	"net/http"
+	"net/url"
 	"os"
+	"path"
+	"path/filepath"
 	"runtime"
 	"strings"
 	"sync"
@@ -63,6 +66,7 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 		sites:       group,
 		connTimeout: GracefulTimeout,
 	}
+	s.Server = makeHTTPServerWithHeaderLimit(s.Server, group)
 	s.Server.Handler = s // this is weird, but whatever
 
 	// extract TLS settings from each site config to build
@@ -124,6 +128,32 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 	return s, nil
 }
 
+// makeHTTPServerWithHeaderLimit apply minimum header limit within a group to given http.Server
+func makeHTTPServerWithHeaderLimit(s *http.Server, group []*SiteConfig) *http.Server {
+	var min int64
+	for _, cfg := range group {
+		limit := cfg.Limits.MaxRequestHeaderSize
+		if limit == 0 {
+			continue
+		}
+
+		// not set yet
+		if min == 0 {
+			min = limit
+		}
+
+		// find a better one
+		if limit < min {
+			min = limit
+		}
+	}
+
+	if min > 0 {
+		s.MaxHeaderBytes = int(min)
+	}
+	return s
+}
+
 // makeHTTPServerWithTimeouts makes an http.Server from the group of
 // configs in a way that configures timeouts (or, if not set, it uses
 // the default timeouts) by combining the configuration of each
@@ -290,11 +320,18 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		}
 	}()
 
-	w.Header().Set("Server", "Caddy")
-	c := context.WithValue(r.Context(), staticfiles.URLPathCtxKey, r.URL.Path)
+	// copy the original, unchanged URL into the context
+	// so it can be referenced by middlewares
+	urlCopy := *r.URL
+	if r.URL.User != nil {
+		userInfo := new(url.Userinfo)
+		*userInfo = *r.URL.User
+		urlCopy.User = userInfo
+	}
+	c := context.WithValue(r.Context(), OriginalURLCtxKey, urlCopy)
 	r = r.WithContext(c)
 
-	sanitizePath(r)
+	w.Header().Set("Server", "Caddy")
 
 	status, _ := s.serveHTTP(w, r)
 
@@ -349,19 +386,6 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 		}
 	}
 
-	// Apply the path-based request body size limit
-	// The error returned by MaxBytesReader is meant to be handled
-	// by whichever middleware/plugin that receives it when calling
-	// .Read() or a similar method on the request body
-	if r.Body != nil {
-		for _, pathlimit := range vhost.MaxRequestBodySizes {
-			if Path(r.URL.Path).Matches(pathlimit.Path) {
-				r.Body = MaxBytesReader(w, r.Body, pathlimit.Limit)
-				break
-			}
-		}
-	}
-
 	return vhost.middlewareChain.ServeHTTP(w, r)
 }
 
@@ -407,28 +431,6 @@ func (s *Server) Stop() error {
 	return nil
 }
 
-// sanitizePath collapses any ./ ../ /// madness which helps prevent
-// path traversal attacks. Note to middleware: use the value within the
-// request's context at key caddy.URLPathContextKey to access the
-// "original" URL.Path value.
-func sanitizePath(r *http.Request) {
-	if r.URL.Path == "/" {
-		return
-	}
-	cleanedPath := CleanPath(r.URL.Path)
-	if cleanedPath == "." {
-		r.URL.Path = "/"
-	} else {
-		if !strings.HasPrefix(cleanedPath, "/") {
-			cleanedPath = "/" + cleanedPath
-		}
-		if strings.HasSuffix(r.URL.Path, "/") && !strings.HasSuffix(cleanedPath, "/") {
-			cleanedPath = cleanedPath + "/"
-		}
-		r.URL.Path = cleanedPath
-	}
-}
-
 // OnStartupComplete lists the sites served by this server
 // and any relevant information, assuming caddy.Quiet == false.
 func (s *Server) OnStartupComplete() {
@@ -476,73 +478,9 @@ func (ln tcpKeepAliveListener) File() (*os.File, error) {
 	return ln.TCPListener.File()
 }
 
-// MaxBytesExceeded is the error type returned by MaxBytesReader
+// ErrMaxBytesExceeded is the error returned by MaxBytesReader
 // when the request body exceeds the limit imposed
-type MaxBytesExceeded struct{}
-
-func (err MaxBytesExceeded) Error() string {
-	return "http: request body too large"
-}
-
-// MaxBytesReader and its associated methods are borrowed from the
-// Go Standard library (comments intact). The only difference is that
-// it returns a MaxBytesExceeded error instead of a generic error message
-// when the request body has exceeded the requested limit
-func MaxBytesReader(w http.ResponseWriter, r io.ReadCloser, n int64) io.ReadCloser {
-	return &maxBytesReader{w: w, r: r, n: n}
-}
-
-type maxBytesReader struct {
-	w   http.ResponseWriter
-	r   io.ReadCloser // underlying reader
-	n   int64         // max bytes remaining
-	err error         // sticky error
-}
-
-func (l *maxBytesReader) Read(p []byte) (n int, err error) {
-	if l.err != nil {
-		return 0, l.err
-	}
-	if len(p) == 0 {
-		return 0, nil
-	}
-	// If they asked for a 32KB byte read but only 5 bytes are
-	// remaining, no need to read 32KB. 6 bytes will answer the
-	// question of the whether we hit the limit or go past it.
-	if int64(len(p)) > l.n+1 {
-		p = p[:l.n+1]
-	}
-	n, err = l.r.Read(p)
-
-	if int64(n) <= l.n {
-		l.n -= int64(n)
-		l.err = err
-		return n, err
-	}
-
-	n = int(l.n)
-	l.n = 0
-
-	// The server code and client code both use
-	// maxBytesReader. This "requestTooLarge" check is
-	// only used by the server code. To prevent binaries
-	// which only using the HTTP Client code (such as
-	// cmd/go) from also linking in the HTTP server, don't
-	// use a static type assertion to the server
-	// "*response" type. Check this interface instead:
-	type requestTooLarger interface {
-		requestTooLarge()
-	}
-	if res, ok := l.w.(requestTooLarger); ok {
-		res.requestTooLarge()
-	}
-	l.err = MaxBytesExceeded{}
-	return n, l.err
-}
-
-func (l *maxBytesReader) Close() error {
-	return l.r.Close()
-}
+var ErrMaxBytesExceeded = errors.New("http: request body too large")
 
 // DefaultErrorFunc responds to an HTTP request with a simple description
 // of the specified HTTP status code.
@@ -558,3 +496,20 @@ func WriteTextResponse(w http.ResponseWriter, status int, body string) {
 	w.WriteHeader(status)
 	w.Write([]byte(body))
 }
+
+// SafePath joins siteRoot and reqPath and converts it to a path that can
+// be used to access a path on the local disk. It ensures the path does
+// not traverse outside of the site root.
+//
+// If opening a file, use http.Dir instead.
+func SafePath(siteRoot, reqPath string) string {
+	reqPath = filepath.ToSlash(reqPath)
+	reqPath = strings.Replace(reqPath, "\x00", "", -1) // NOTE: Go 1.9 checks for null bytes in the syscall package
+	if siteRoot == "" {
+		siteRoot = "."
+	}
+	return filepath.Join(siteRoot, filepath.FromSlash(path.Clean("/"+reqPath)))
+}
+
+// OriginalURLCtxKey is the key for accessing the original, incoming URL on an HTTP request.
+const OriginalURLCtxKey = caddy.CtxKey("original_url")

caddyhttp/httpserver/server_test.go

@@ -15,7 +15,7 @@ func TestAddress(t *testing.T) {
 	}
 }
 
-func TestMakeHTTPServer(t *testing.T) {
+func TestMakeHTTPServerWithTimeouts(t *testing.T) {
 	for i, tc := range []struct {
 		group    []*SiteConfig
 		expected Timeouts
@@ -111,3 +111,36 @@ func TestMakeHTTPServer(t *testing.T) {
 		}
 	}
 }
+
+func TestMakeHTTPServerWithHeaderLimit(t *testing.T) {
+	for name, c := range map[string]struct {
+		group  []*SiteConfig
+		expect int
+	}{
+		"disable": {
+			group:  []*SiteConfig{{}},
+			expect: 0,
+		},
+		"oneSite": {
+			group: []*SiteConfig{{Limits: Limits{
+				MaxRequestHeaderSize: 100,
+			}}},
+			expect: 100,
+		},
+		"multiSites": {
+			group: []*SiteConfig{
+				{Limits: Limits{MaxRequestHeaderSize: 100}},
+				{Limits: Limits{MaxRequestHeaderSize: 50}},
+			},
+			expect: 50,
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			actual := makeHTTPServerWithHeaderLimit(&http.Server{}, c.group)
+			if got := actual.MaxHeaderBytes; got != c.expect {
+				t.Errorf("Expect %d, but got %d", c.expect, got)
+			}
+		})
+	}
+}

caddyhttp/httpserver/siteconfig.go

@@ -38,8 +38,8 @@ type SiteConfig struct {
 	// for a request.
 	HiddenFiles []string
 
-	// Max amount of bytes a request can send on a given path
-	MaxRequestBodySizes []PathLimit
+	// Max request's header/body size
+	Limits Limits
 
 	// The path to the Caddyfile used to generate this site config
 	originCaddyfile string
@@ -71,6 +71,12 @@ type Timeouts struct {
 	IdleTimeoutSet       bool
 }
 
+// Limits specify size limit of request's header and body.
+type Limits struct {
+	MaxRequestHeaderSize int64
+	MaxRequestBodySizes  []PathLimit
+}
+
 // PathLimit is a mapping from a site's path to its corresponding
 // maximum request body size (in bytes)
 type PathLimit struct {

caddyhttp/httpserver/tplcontext.go

@@ -11,6 +11,7 @@ import (
 	"net/url"
 	"path"
 	"strings"
+	"sync"
 	"text/template"
 	"time"
 
@@ -28,6 +29,20 @@ type Context struct {
 	Req  *http.Request
 	URL  *url.URL
 	Args []interface{} // defined by arguments to .Include
+
+	// just used for adding preload links for server push
+	responseHeader http.Header
+}
+
+// NewContextWithHeader creates a context with given response header.
+//
+// To plugin developer:
+// The returned context's exported fileds remain empty,
+// you should then initialize them if you want.
+func NewContextWithHeader(rh http.Header) Context {
+	return Context{
+		responseHeader: rh,
+	}
 }
 
 // Include returns the contents of filename relative to the site root.
@@ -256,9 +271,6 @@ func (c Context) Markdown(filename string) (string, error) {
 	return string(markdown), nil
 }
 
-// TemplateFuncs contains user defined functions
-var TemplateFuncs = template.FuncMap{}
-
 // ContextInclude opens filename using fs and executes a template with the context ctx.
 // This does the same thing that Context.Include() does, but with the ability to provide
 // your own context so that the included files can have access to additional fields your
@@ -281,8 +293,10 @@ func ContextInclude(filename string, ctx interface{}, fs http.FileSystem) (strin
 		return "", err
 	}
 
-	var buf bytes.Buffer
-	err = tpl.Execute(&buf, ctx)
+	buf := includeBufs.Get().(*bytes.Buffer)
+	buf.Reset()
+	defer includeBufs.Put(buf)
+	err = tpl.Execute(buf, ctx)
 	if err != nil {
 		return "", err
 	}
@@ -409,3 +423,24 @@ func (c Context) RandomString(minLen, maxLen int) string {
 
 	return string(result)
 }
+
+// AddLink adds a link header in response
+// see https://www.w3.org/wiki/LinkHeader
+func (c Context) AddLink(link string) string {
+	if c.responseHeader == nil {
+		return ""
+	}
+	c.responseHeader.Add("Link", link)
+	return ""
+}
+
+// buffer pool for .Include context actions
+var includeBufs = sync.Pool{
+	New: func() interface{} {
+		return new(bytes.Buffer)
+	},
+}
+
+// TemplateFuncs contains user-defined functions
+// for execution in templates.
+var TemplateFuncs = template.FuncMap{}

caddyhttp/httpserver/tplcontext_test.go

@@ -6,6 +6,7 @@ import (
 	"io/ioutil"
 	"net"
 	"net/http"
+	"net/http/httptest"
 	"net/url"
 	"os"
 	"path/filepath"
@@ -251,14 +252,16 @@ func TestHostname(t *testing.T) {
 		inputRemoteAddr  string
 		expectedHostname string
 	}{
+		// TODO(mholt): Fix these tests, they're not portable. i.e. my resolver
+		// returns "fwdr-8.fwdr-8.fwdr-8.fwdr-8." instead of these google ones.
 		// Test 0 - ipv4 with port
-		{"8.8.8.8:1111", "google-public-dns-a.google.com."},
-		// Test 1 - ipv4 without port
-		{"8.8.8.8", "google-public-dns-a.google.com."},
-		// Test 2 - ipv6 with port
-		{"[2001:4860:4860::8888]:11", "google-public-dns-a.google.com."},
-		// Test 3 - ipv6 without port and brackets
-		{"2001:4860:4860::8888", "google-public-dns-a.google.com."},
+		// {"8.8.8.8:1111", "google-public-dns-a.google.com."},
+		// // Test 1 - ipv4 without port
+		// {"8.8.8.8", "google-public-dns-a.google.com."},
+		// // Test 2 - ipv6 with port
+		// {"[2001:4860:4860::8888]:11", "google-public-dns-a.google.com."},
+		// // Test 3 - ipv6 without port and brackets
+		// {"2001:4860:4860::8888", "google-public-dns-a.google.com."},
 		// Test 4 - no hostname available
 		{"1.1.1.1", "1.1.1.1"},
 	}
@@ -494,7 +497,7 @@ func TestMethod(t *testing.T) {
 
 }
 
-func TestPathMatches(t *testing.T) {
+func TestContextPathMatches(t *testing.T) {
 	context := getContextOrFail(t)
 
 	tests := []struct {
@@ -729,8 +732,9 @@ func initTestContext() (Context, error) {
 	if err != nil {
 		return Context{}, err
 	}
+	res := httptest.NewRecorder()
 
-	return Context{Root: http.Dir(os.TempDir()), Req: request}, nil
+	return Context{Root: http.Dir(os.TempDir()), responseHeader: res.Header(), Req: request}, nil
 }
 
 func getContextOrFail(t *testing.T) Context {
@@ -872,3 +876,35 @@ func TestFiles(t *testing.T) {
 		}
 	}
 }
+
+func TestAddLink(t *testing.T) {
+	for name, c := range map[string]struct {
+		input       string
+		expectLinks []string
+	}{
+		"oneLink": {
+			input:       `{{.AddLink "</test.css>; rel=preload"}}`,
+			expectLinks: []string{"</test.css>; rel=preload"},
+		},
+		"multipleLinks": {
+			input:       `{{.AddLink "</test1.css>; rel=preload"}} {{.AddLink "</test2.css>; rel=meta"}}`,
+			expectLinks: []string{"</test1.css>; rel=preload", "</test2.css>; rel=meta"},
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			ctx := getContextOrFail(t)
+			tmpl, err := template.New("").Parse(c.input)
+			if err != nil {
+				t.Fatal(err)
+			}
+			err = tmpl.Execute(ioutil.Discard, ctx)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if got := ctx.responseHeader["Link"]; !reflect.DeepEqual(got, c.expectLinks) {
+				t.Errorf("Result not match: expect %v, but got %v", c.expectLinks, got)
+			}
+		})
+	}
+}

caddyhttp/internalsrv/internal.go

@@ -42,7 +42,7 @@ func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 
 	// Use internal response writer to ignore responses that will be
 	// redirected to internal locations
-	iw := internalResponseWriter{ResponseWriter: w}
+	iw := internalResponseWriter{ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w}}
 	status, err := i.Next.ServeHTTP(iw, r)
 
 	for c := 0; c < maxRedirectCount && isInternalRedirect(iw); c++ {
@@ -67,7 +67,7 @@ func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 // calls to Write and WriteHeader if the response should be redirected to an
 // internal location.
 type internalResponseWriter struct {
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 }
 
 // ClearHeader removes script headers that would interfere with follow up
@@ -82,7 +82,7 @@ func (w internalResponseWriter) ClearHeader() {
 // internal location.
 func (w internalResponseWriter) WriteHeader(code int) {
 	if !isInternalRedirect(w) {
-		w.ResponseWriter.WriteHeader(code)
+		w.ResponseWriterWrapper.WriteHeader(code)
 	}
 }
 
@@ -92,5 +92,8 @@ func (w internalResponseWriter) Write(b []byte) (int, error) {
 	if isInternalRedirect(w) {
 		return 0, nil
 	}
-	return w.ResponseWriter.Write(b)
+	return w.ResponseWriterWrapper.Write(b)
 }
+
+// Interface guards
+var _ httpserver.HTTPInterfaces = internalResponseWriter{}

caddyhttp/limits/handler.go

@@ -0,0 +1,90 @@
+package limits
+
+import (
+	"io"
+	"net/http"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+// Limit is a middleware to control request body size
+type Limit struct {
+	Next       httpserver.Handler
+	BodyLimits []httpserver.PathLimit
+}
+
+func (l Limit) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
+	if r.Body == nil {
+		return l.Next.ServeHTTP(w, r)
+	}
+
+	// apply the path-based request body size limit.
+	for _, bl := range l.BodyLimits {
+		if httpserver.Path(r.URL.Path).Matches(bl.Path) {
+			r.Body = MaxBytesReader(w, r.Body, bl.Limit)
+			break
+		}
+	}
+
+	return l.Next.ServeHTTP(w, r)
+}
+
+// MaxBytesReader and its associated methods are borrowed from the
+// Go Standard library (comments intact). The only difference is that
+// it returns a ErrMaxBytesExceeded error instead of a generic error message
+// when the request body has exceeded the requested limit
+func MaxBytesReader(w http.ResponseWriter, r io.ReadCloser, n int64) io.ReadCloser {
+	return &maxBytesReader{w: w, r: r, n: n}
+}
+
+type maxBytesReader struct {
+	w   http.ResponseWriter
+	r   io.ReadCloser // underlying reader
+	n   int64         // max bytes remaining
+	err error         // sticky error
+}
+
+func (l *maxBytesReader) Read(p []byte) (n int, err error) {
+	if l.err != nil {
+		return 0, l.err
+	}
+	if len(p) == 0 {
+		return 0, nil
+	}
+	// If they asked for a 32KB byte read but only 5 bytes are
+	// remaining, no need to read 32KB. 6 bytes will answer the
+	// question of the whether we hit the limit or go past it.
+	if int64(len(p)) > l.n+1 {
+		p = p[:l.n+1]
+	}
+	n, err = l.r.Read(p)
+
+	if int64(n) <= l.n {
+		l.n -= int64(n)
+		l.err = err
+		return n, err
+	}
+
+	n = int(l.n)
+	l.n = 0
+
+	// The server code and client code both use
+	// maxBytesReader. This "requestTooLarge" check is
+	// only used by the server code. To prevent binaries
+	// which only using the HTTP Client code (such as
+	// cmd/go) from also linking in the HTTP server, don't
+	// use a static type assertion to the server
+	// "*response" type. Check this interface instead:
+	type requestTooLarger interface {
+		requestTooLarge()
+	}
+	if res, ok := l.w.(requestTooLarger); ok {
+		res.requestTooLarge()
+	}
+	l.err = httpserver.ErrMaxBytesExceeded
+	return n, l.err
+}
+
+func (l *maxBytesReader) Close() error {
+	return l.r.Close()
+}

caddyhttp/limits/handler_test.go

@@ -0,0 +1,35 @@
+package limits
+
+import (
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
+)
+
+func TestBodySizeLimit(t *testing.T) {
+	var (
+		gotContent    []byte
+		gotError      error
+		expectContent = "hello"
+	)
+	l := Limit{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			gotContent, gotError = ioutil.ReadAll(r.Body)
+			return 0, nil
+		}),
+		BodyLimits: []httpserver.PathLimit{{Path: "/", Limit: int64(len(expectContent))}},
+	}
+
+	r := httptest.NewRequest("GET", "/", strings.NewReader(expectContent+expectContent))
+	l.ServeHTTP(httptest.NewRecorder(), r)
+	if got := string(gotContent); got != expectContent {
+		t.Errorf("expected content[%s], got[%s]", expectContent, got)
+	}
+	if gotError != httpserver.ErrMaxBytesExceeded {
+		t.Errorf("expect error %v, got %v", httpserver.ErrMaxBytesExceeded, gotError)
+	}
+}

caddyhttp/limits/setup.go

@@ -1,4 +1,4 @@
-package maxrequestbody
+package limits
 
 import (
 	"errors"
@@ -12,13 +12,13 @@ import (
 
 const (
 	serverType = "http"
-	pluginName = "maxrequestbody"
+	pluginName = "limits"
 )
 
 func init() {
 	caddy.RegisterPlugin(pluginName, caddy.Plugin{
 		ServerType: serverType,
-		Action:     setupMaxRequestBody,
+		Action:     setupLimits,
 	})
 }
 
@@ -28,56 +28,97 @@ type pathLimitUnparsed struct {
 	Limit string
 }
 
-func setupMaxRequestBody(c *caddy.Controller) error {
+func setupLimits(c *caddy.Controller) error {
+	bls, err := parseLimits(c)
+	if err != nil {
+		return err
+	}
+
+	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
+		return Limit{Next: next, BodyLimits: bls}
+	})
+	return nil
+}
+
+func parseLimits(c *caddy.Controller) ([]httpserver.PathLimit, error) {
 	config := httpserver.GetConfig(c)
 
 	if !c.Next() {
-		return c.ArgErr()
+		return nil, c.ArgErr()
 	}
 
 	args := c.RemainingArgs()
 	argList := []pathLimitUnparsed{}
+	headerLimit := ""
 
 	switch len(args) {
 	case 0:
-		// Format: { <path> <limit> ... }
+		// Format: limits {
+		//	header <limit>
+		//	body <path> <limit>
+		//	body <limit>
+		//	...
+		// }
 		for c.NextBlock() {
-			path := c.Val()
-			if !c.NextArg() {
-				// Uneven pairing of path/limit
-				return c.ArgErr()
+			kind := c.Val()
+			pathOrLimit := c.RemainingArgs()
+			switch kind {
+			case "header":
+				if len(pathOrLimit) != 1 {
+					return nil, c.ArgErr()
+				}
+				headerLimit = pathOrLimit[0]
+			case "body":
+				if len(pathOrLimit) == 1 {
+					argList = append(argList, pathLimitUnparsed{
+						Path:  "/",
+						Limit: pathOrLimit[0],
+					})
+					break
+				}
+
+				if len(pathOrLimit) == 2 {
+					argList = append(argList, pathLimitUnparsed{
+						Path:  pathOrLimit[0],
+						Limit: pathOrLimit[1],
+					})
+					break
+				}
+
+				fallthrough
+			default:
+				return nil, c.ArgErr()
 			}
-			argList = append(argList, pathLimitUnparsed{
-				Path:  path,
-				Limit: c.Val(),
-			})
 		}
 	case 1:
-		// Format: <limit>
+		// Format: limits <limit>
+		headerLimit = args[0]
 		argList = []pathLimitUnparsed{{
 			Path:  "/",
 			Limit: args[0],
 		}}
-	case 2:
-		// Format: <path> <limit>
-		argList = []pathLimitUnparsed{{
-			Path:  args[0],
-			Limit: args[1],
-		}}
 	default:
-		return c.ArgErr()
+		return nil, c.ArgErr()
 	}
 
-	pathLimit, err := parseArguments(argList)
-	if err != nil {
-		return c.ArgErr()
+	if headerLimit != "" {
+		size := parseSize(headerLimit)
+		if size < 1 { // also disallow size = 0
+			return nil, c.ArgErr()
+		}
+		config.Limits.MaxRequestHeaderSize = size
 	}
 
-	SortPathLimits(pathLimit)
+	if len(argList) > 0 {
+		pathLimit, err := parseArguments(argList)
+		if err != nil {
+			return nil, c.ArgErr()
+		}
+		SortPathLimits(pathLimit)
+		config.Limits.MaxRequestBodySizes = pathLimit
+	}
 
-	config.MaxRequestBodySizes = pathLimit
-
-	return nil
+	return config.Limits.MaxRequestBodySizes, nil
 }
 
 func parseArguments(args []pathLimitUnparsed) ([]httpserver.PathLimit, error) {

caddyhttp/limits/setup_test.go

@@ -1,4 +1,4 @@
-package maxrequestbody
+package limits
 
 import (
 	"reflect"
@@ -14,32 +14,98 @@ const (
 	GB = 1024 * 1024 * 1024
 )
 
-func TestSetupMaxRequestBody(t *testing.T) {
-	cases := []struct {
-		input    string
-		hasError bool
+func TestParseLimits(t *testing.T) {
+	for name, c := range map[string]struct {
+		input     string
+		shouldErr bool
+		expect    httpserver.Limits
 	}{
-		// Format: { <path> <limit> ... }
-		{input: "maxrequestbody / 20MB", hasError: false},
-		// Format: <limit>
-		{input: "maxrequestbody 999KB", hasError: false},
-		// Format: { <path> <limit> ... }
-		{input: "maxrequestbody { /images 50MB /upload 10MB\n/test 10KB }", hasError: false},
-
-		// Wrong formats
-		{input: "maxrequestbody typo { /images 50MB }", hasError: true},
-		{input: "maxrequestbody 999MB /home 20KB", hasError: true},
-	}
-	for caseNum, c := range cases {
-		controller := caddy.NewTestController("", c.input)
-		err := setupMaxRequestBody(controller)
-
-		if c.hasError && (err == nil) {
-			t.Errorf("Expecting error for case %v but none encountered", caseNum)
-		}
-		if !c.hasError && (err != nil) {
-			t.Errorf("Expecting no error for case %v but encountered %v", caseNum, err)
-		}
+		"catchAll": {
+			input: `limits 2kb`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 2 * KB,
+				MaxRequestBodySizes:  []httpserver.PathLimit{{Path: "/", Limit: 2 * KB}},
+			},
+		},
+		"onlyHeader": {
+			input: `limits {
+				header 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 2 * KB,
+			},
+		},
+		"onlyBody": {
+			input: `limits {
+				body 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestBodySizes: []httpserver.PathLimit{{Path: "/", Limit: 2 * KB}},
+			},
+		},
+		"onlyBodyWithPath": {
+			input: `limits {
+				body /test 2kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestBodySizes: []httpserver.PathLimit{{Path: "/test", Limit: 2 * KB}},
+			},
+		},
+		"mixture": {
+			input: `limits {
+				header 1kb
+				body 2kb
+				body /bar 3kb
+			}`,
+			expect: httpserver.Limits{
+				MaxRequestHeaderSize: 1 * KB,
+				MaxRequestBodySizes: []httpserver.PathLimit{
+					{Path: "/bar", Limit: 3 * KB},
+					{Path: "/", Limit: 2 * KB},
+				},
+			},
+		},
+		"invalidFormat": {
+			input:     `limits a b`,
+			shouldErr: true,
+		},
+		"invalidHeaderFormat": {
+			input: `limits {
+				header / 100
+			}`,
+			shouldErr: true,
+		},
+		"invalidBodyFormat": {
+			input: `limits {
+				body / 100 200
+			}`,
+			shouldErr: true,
+		},
+		"invalidKind": {
+			input: `limits {
+				head 100
+			}`,
+			shouldErr: true,
+		},
+		"invalidLimitSize": {
+			input:     `limits 10bk`,
+			shouldErr: true,
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			controller := caddy.NewTestController("", c.input)
+			_, err := parseLimits(controller)
+			if c.shouldErr && err == nil {
+				t.Error("failed to get expected error")
+			}
+			if !c.shouldErr && err != nil {
+				t.Errorf("got unexpected error: %v", err)
+			}
+			if got := httpserver.GetConfig(controller).Limits; !reflect.DeepEqual(got, c.expect) {
+				t.Errorf("expect %#v, but got %#v", c.expect, got)
+			}
+		})
 	}
 }
 

caddyhttp/log/setup.go

@@ -1,6 +1,8 @@
 package log
 
 import (
+	"strings"
+
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
@@ -51,48 +53,36 @@ func logParse(c *caddy.Controller) ([]*Rule, error) {
 			}
 		}
 
-		if len(args) == 0 {
-			// Nothing specified; use defaults
-			rules = appendEntry(rules, "/", &Entry{
-				Log: &httpserver.Logger{
-					Output: DefaultLogFilename,
-					Roller: logRoller,
-				},
-				Format: DefaultLogFormat,
-			})
-		} else if len(args) == 1 {
+		path := "/"
+		format := DefaultLogFormat
+		output := DefaultLogFilename
+
+		switch len(args) {
+		case 0:
+			// nothing to change
+		case 1:
 			// Only an output file specified
-			rules = appendEntry(rules, "/", &Entry{
-				Log: &httpserver.Logger{
-					Output: args[0],
-					Roller: logRoller,
-				},
-				Format: DefaultLogFormat,
-			})
-		} else {
+			output = args[0]
+		case 2, 3:
 			// Path scope, output file, and maybe a format specified
-
-			format := DefaultLogFormat
-
+			path = args[0]
+			output = args[1]
 			if len(args) > 2 {
-				switch args[2] {
-				case "{common}":
-					format = CommonLogFormat
-				case "{combined}":
-					format = CombinedLogFormat
-				default:
-					format = args[2]
-				}
+				format = strings.Replace(args[2], "{common}", CommonLogFormat, -1)
+				format = strings.Replace(format, "{combined}", CombinedLogFormat, -1)
 			}
-
-			rules = appendEntry(rules, args[0], &Entry{
-				Log: &httpserver.Logger{
-					Output: args[1],
-					Roller: logRoller,
-				},
-				Format: format,
-			})
+		default:
+			// Maximum number of args in log directive is 3.
+			return nil, c.ArgErr()
 		}
+
+		rules = appendEntry(rules, path, &Entry{
+			Log: &httpserver.Logger{
+				Output: output,
+				Roller: logRoller,
+			},
+			Format: format,
+		})
 	}
 
 	return rules, nil

caddyhttp/log/setup_test.go

@@ -124,6 +124,16 @@ func TestLogParse(t *testing.T) {
 				Format: CommonLogFormat,
 			}},
 		}}},
+		{`log /myapi log.txt "prefix {common} suffix"`, false, []Rule{{
+			PathScope: "/myapi",
+			Entries: []*Entry{{
+				Log: &httpserver.Logger{
+					Output: "log.txt",
+					Roller: httpserver.DefaultLogRoller(),
+				},
+				Format: "prefix " + CommonLogFormat + " suffix",
+			}},
+		}}},
 		{`log /test accesslog.txt {combined}`, false, []Rule{{
 			PathScope: "/test",
 			Entries: []*Entry{{
@@ -134,6 +144,16 @@ func TestLogParse(t *testing.T) {
 				Format: CombinedLogFormat,
 			}},
 		}}},
+		{`log /test accesslog.txt "prefix {combined} suffix"`, false, []Rule{{
+			PathScope: "/test",
+			Entries: []*Entry{{
+				Log: &httpserver.Logger{
+					Output: "accesslog.txt",
+					Roller: httpserver.DefaultLogRoller(),
+				},
+				Format: "prefix " + CombinedLogFormat + " suffix",
+			}},
+		}}},
 		{`log /api1 log.txt
 		  log /api2 accesslog.txt {combined}`, false, []Rule{{
 			PathScope: "/api1",
@@ -207,6 +227,7 @@ func TestLogParse(t *testing.T) {
 		}}},
 		{`log access.log { rotate_size }`, true, nil},
 		{`log access.log { invalid_option 1 }`, true, nil},
+		{`log / acccess.log "{remote} - [{when}] "{method} {port}" {scheme} {mitm} "`, true, nil},
 	}
 	for i, test := range tests {
 		c := caddy.NewTestController("http", test.inputLogRules)

caddyhttp/markdown/markdown.go

@@ -133,11 +133,10 @@ func (md Markdown) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error
 	}
 	lastModTime = latest(lastModTime, fs.ModTime())
 
-	ctx := httpserver.Context{
-		Root: md.FileSys,
-		Req:  r,
-		URL:  r.URL,
-	}
+	ctx := httpserver.NewContextWithHeader(w.Header())
+	ctx.Root = md.FileSys
+	ctx.Req = r
+	ctx.URL = r.URL
 	html, err := cfg.Markdown(title(fpath), f, dirents, ctx)
 	if err != nil {
 		return http.StatusInternalServerError, err

caddyhttp/markdown/template.go

@@ -22,7 +22,8 @@ type Data struct {
 // Include "overrides" the embedded httpserver.Context's Include()
 // method so that included files have access to d's fields.
 // Note: using {{template 'template-name' .}} instead might be better.
-func (d Data) Include(filename string) (string, error) {
+func (d Data) Include(filename string, args ...interface{}) (string, error) {
+	d.Args = args
 	return httpserver.ContextInclude(filename, d, d.Root)
 }
 

caddyhttp/proxy/policy.go

@@ -23,6 +23,7 @@ func init() {
 	RegisterPolicy("round_robin", func() Policy { return &RoundRobin{} })
 	RegisterPolicy("ip_hash", func() Policy { return &IPHash{} })
 	RegisterPolicy("first", func() Policy { return &First{} })
+	RegisterPolicy("uri_hash", func() Policy { return &URIHash{} })
 }
 
 // Random is a policy that selects up hosts from a pool at random.
@@ -56,7 +57,7 @@ func (r *Random) Select(pool HostPool, request *http.Request) *UpstreamHost {
 type LeastConn struct{}
 
 // Select selects the up host with the least number of connections in the
-// pool.  If more than one host has the same least number of connections,
+// pool. If more than one host has the same least number of connections,
 // one of the hosts is chosen at random.
 func (r *LeastConn) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	var bestHost *UpstreamHost
@@ -84,13 +85,13 @@ func (r *LeastConn) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	return bestHost
 }
 
-// RoundRobin is a policy that selects hosts based on round robin ordering.
+// RoundRobin is a policy that selects hosts based on round-robin ordering.
 type RoundRobin struct {
 	robin uint32
 	mutex sync.Mutex
 }
 
-// Select selects an up host from the pool using a round robin ordering scheme.
+// Select selects an up host from the pool using a round-robin ordering scheme.
 func (r *RoundRobin) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	poolLen := uint32(len(pool))
 	r.mutex.Lock()
@@ -106,23 +107,10 @@ func (r *RoundRobin) Select(pool HostPool, request *http.Request) *UpstreamHost
 	return nil
 }
 
-// IPHash is a policy that selects hosts based on hashing the request ip
-type IPHash struct{}
-
-func hash(s string) uint32 {
-	h := fnv.New32a()
-	h.Write([]byte(s))
-	return h.Sum32()
-}
-
-// Select selects an up host from the pool using a round robin ordering scheme.
-func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+// hostByHashing returns an available host from pool based on a hashable string
+func hostByHashing(pool HostPool, s string) *UpstreamHost {
 	poolLen := uint32(len(pool))
-	clientIP, _, err := net.SplitHostPort(request.RemoteAddr)
-	if err != nil {
-		clientIP = request.RemoteAddr
-	}
-	index := hash(clientIP) % poolLen
+	index := hash(s) % poolLen
 	for i := uint32(0); i < poolLen; i++ {
 		index += i
 		host := pool[index%poolLen]
@@ -133,10 +121,37 @@ func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	return nil
 }
 
-// First is a policy that selects the fist available host
+// hash calculates a hash based on string s
+func hash(s string) uint32 {
+	h := fnv.New32a()
+	h.Write([]byte(s))
+	return h.Sum32()
+}
+
+// IPHash is a policy that selects hosts based on hashing the request IP
+type IPHash struct{}
+
+// Select selects an up host from the pool based on hashing the request IP
+func (r *IPHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	clientIP, _, err := net.SplitHostPort(request.RemoteAddr)
+	if err != nil {
+		clientIP = request.RemoteAddr
+	}
+	return hostByHashing(pool, clientIP)
+}
+
+// URIHash is a policy that selects the host based on hashing the request URI
+type URIHash struct{}
+
+// Select selects the host based on hashing the URI
+func (r *URIHash) Select(pool HostPool, request *http.Request) *UpstreamHost {
+	return hostByHashing(pool, request.RequestURI)
+}
+
+// First is a policy that selects the first available host
 type First struct{}
 
-// Select selects the first host from the pool, that is available
+// Select selects the first available host from the pool
 func (r *First) Select(pool HostPool, request *http.Request) *UpstreamHost {
 	for _, host := range pool {
 		if host.Available() {

caddyhttp/proxy/policy_test.go

@@ -243,3 +243,62 @@ func TestFirstPolicy(t *testing.T) {
 		t.Error("Expected first policy host to be the second host.")
 	}
 }
+
+func TestUriPolicy(t *testing.T) {
+	pool := testPool()
+	uriPolicy := &URIHash{}
+
+	request := httptest.NewRequest(http.MethodGet, "/test", nil)
+	h := uriPolicy.Select(pool, request)
+	if h != pool[0] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	pool[0].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test_2", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the second host.")
+	}
+
+	// We should be able to resize the host pool and still be able to predict
+	// where a request will be routed with the same URI's used above
+	pool = []*UpstreamHost{
+		{
+			Name: workableServer.URL, // this should resolve (healthcheck test)
+		},
+		{
+			Name: "http://localhost:99998", // this shouldn't
+		},
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[0] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	pool[0].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the first host.")
+	}
+
+	request = httptest.NewRequest(http.MethodGet, "/test_2", nil)
+	h = uriPolicy.Select(pool, request)
+	if h != pool[1] {
+		t.Error("Expected uri policy host to be the second host.")
+	}
+
+	pool[0].Unhealthy = 1
+	pool[1].Unhealthy = 1
+	h = uriPolicy.Select(pool, request)
+	if h != nil {
+		t.Error("Expected uri policy policy host to be nil.")
+	}
+}

caddyhttp/proxy/proxy.go

@@ -2,6 +2,7 @@
 package proxy
 
 import (
+	"context"
 	"errors"
 	"net"
 	"net/http"
@@ -103,7 +104,8 @@ func (p Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	replacer := httpserver.NewReplacer(r, nil, "")
 
 	// outreq is the request that makes a roundtrip to the backend
-	outreq := createUpstreamRequest(r)
+	outreq, cancel := createUpstreamRequest(w, r)
+	defer cancel()
 
 	// If we have more than one upstream host defined and if retrying is enabled
 	// by setting try_duration to a non-zero value, caddy will try to
@@ -131,7 +133,11 @@ func (p Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	// loop and try to select another host, or false if we
 	// should break and stop retrying.
 	start := time.Now()
-	keepRetrying := func() bool {
+	keepRetrying := func(backendErr error) bool {
+		// if downstream has canceled the request, break
+		if backendErr == context.Canceled {
+			return false
+		}
 		// if we've tried long enough, break
 		if time.Since(start) >= upstream.GetTryDuration() {
 			return false
@@ -150,7 +156,7 @@ func (p Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 			if backendErr == nil {
 				backendErr = errors.New("no hosts available upstream")
 			}
-			if !keepRetrying() {
+			if !keepRetrying(backendErr) {
 				break
 			}
 			continue
@@ -222,7 +228,7 @@ func (p Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 			return 0, nil
 		}
 
-		if _, ok := backendErr.(httpserver.MaxBytesExceeded); ok {
+		if backendErr == httpserver.ErrMaxBytesExceeded {
 			return http.StatusRequestEntityTooLarge, backendErr
 		}
 
@@ -238,7 +244,7 @@ func (p Proxy) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 		}
 
 		// if we've tried long enough, break
-		if !keepRetrying() {
+		if !keepRetrying(backendErr) {
 			break
 		}
 	}
@@ -267,9 +273,23 @@ func (p Proxy) match(r *http.Request) Upstream {
 // that can be sent upstream.
 //
 // Derived from reverseproxy.go in the standard Go httputil package.
-func createUpstreamRequest(r *http.Request) *http.Request {
-	outreq := new(http.Request)
-	*outreq = *r // includes shallow copies of maps, but okay
+func createUpstreamRequest(rw http.ResponseWriter, r *http.Request) (*http.Request, context.CancelFunc) {
+	// Original incoming server request may be canceled by the
+	// user or by std lib(e.g. too many idle connections).
+	ctx, cancel := context.WithCancel(r.Context())
+	if cn, ok := rw.(http.CloseNotifier); ok {
+		notifyChan := cn.CloseNotify()
+		go func() {
+			select {
+			case <-notifyChan:
+				cancel()
+			case <-ctx.Done():
+			}
+		}()
+	}
+
+	outreq := r.WithContext(ctx) // includes shallow copies of maps, but okay
+
 	// We should set body to nil explicitly if request body is empty.
 	// For server requests the Request Body is always non-nil.
 	if r.ContentLength == 0 {
@@ -319,7 +339,7 @@ func createUpstreamRequest(r *http.Request) *http.Request {
 		outreq.Header.Set("X-Forwarded-For", clientIP)
 	}
 
-	return outreq
+	return outreq, cancel
 }
 
 func createRespHeaderUpdateFn(rules http.Header, replacer httpserver.Replacer) respUpdateFn {

caddyhttp/proxy/proxy_test.go

@@ -12,7 +12,6 @@ import (
 	"net"
 	"net/http"
 	"net/http/httptest"
-	"net/http/httptrace"
 	"net/url"
 	"os"
 	"path/filepath"
@@ -26,7 +25,6 @@ import (
 
 	"github.com/mholt/caddy/caddyfile"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
-	"github.com/mholt/caddy/caddyhttp/staticfiles"
 
 	"golang.org/x/net/websocket"
 )
@@ -46,32 +44,62 @@ func TestReverseProxy(t *testing.T) {
 	log.SetOutput(ioutil.Discard)
 	defer log.SetOutput(os.Stderr)
 
-	verifyHeaders := func(headers http.Header, trailers http.Header) {
-		if headers.Get("X-Header") != "header-value" {
-			t.Error("Expected header 'X-Header' to be proxied properly")
+	testHeaderValue := []string{"header-value"}
+	testHeaders := http.Header{
+		"X-Header-1": testHeaderValue,
+		"X-Header-2": testHeaderValue,
+		"X-Header-3": testHeaderValue,
+	}
+	testTrailerValue := []string{"trailer-value"}
+	testTrailers := http.Header{
+		"X-Trailer-1": testTrailerValue,
+		"X-Trailer-2": testTrailerValue,
+		"X-Trailer-3": testTrailerValue,
+	}
+	verifyHeaderValues := func(actual http.Header, expected http.Header) bool {
+		if actual == nil {
+			t.Error("Expected headers")
+			return true
 		}
 
-		if trailers == nil {
-			t.Error("Expected to receive trailers")
+		for k := range expected {
+			if expected.Get(k) != actual.Get(k) {
+				t.Errorf("Expected header '%s' to be proxied properly", k)
+				return true
+			}
 		}
-		if trailers.Get("X-Trailer") != "trailer-value" {
-			t.Error("Expected header 'X-Trailer' to be proxied properly")
+
+		return false
+	}
+	verifyHeadersTrailers := func(headers http.Header, trailers http.Header) {
+		if verifyHeaderValues(headers, testHeaders) || verifyHeaderValues(trailers, testTrailers) {
+			t.FailNow()
 		}
 	}
 
-	var requestReceived bool
+	requestReceived := false
 	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		// read the body (even if it's empty) to make Go parse trailers
 		io.Copy(ioutil.Discard, r.Body)
-		verifyHeaders(r.Header, r.Trailer)
 
+		verifyHeadersTrailers(r.Header, r.Trailer)
 		requestReceived = true
 
-		w.Header().Set("Trailer", "X-Trailer")
-		w.Header().Set("X-Header", "header-value")
+		// Set headers.
+		copyHeader(w.Header(), testHeaders)
+
+		// Only announce one of the trailers to test wether
+		// unannounced trailers are proxied correctly.
+		for k := range testTrailers {
+			w.Header().Set("Trailer", k)
+			break
+		}
+
 		w.WriteHeader(http.StatusOK)
 		w.Write([]byte("Hello, client"))
-		w.Header().Set("X-Trailer", "trailer-value")
+
+		// Set trailers.
+		shallowCopyTrailers(w.Header(), testTrailers, true)
 	}))
 	defer backend.Close()
 
@@ -81,28 +109,43 @@ func TestReverseProxy(t *testing.T) {
 		Upstreams: []Upstream{newFakeUpstream(backend.URL, false)},
 	}
 
-	// create request and response recorder
-	r := httptest.NewRequest("GET", "/", strings.NewReader("test"))
-	w := httptest.NewRecorder()
-
-	r.ContentLength = -1 // force chunked encoding (required for trailers)
-	r.Header.Set("X-Header", "header-value")
-	r.Trailer = map[string][]string{
-		"X-Trailer": {"trailer-value"},
+	// Create the fake request body.
+	// This will copy "trailersToSet" to r.Trailer right before it is closed and
+	// thus test for us wether unannounced client trailers are proxied correctly.
+	body := &trailerTestStringReader{
+		Reader:        *strings.NewReader("test"),
+		trailersToSet: testTrailers,
 	}
 
+	// Create the fake request with the above body.
+	r := httptest.NewRequest("GET", "/", body)
+	r.Trailer = make(http.Header)
+	body.request = r
+
+	copyHeader(r.Header, testHeaders)
+
+	// Only announce one of the trailers to test wether
+	// unannounced trailers are proxied correctly.
+	for k, v := range testTrailers {
+		r.Trailer[k] = v
+		break
+	}
+
+	w := httptest.NewRecorder()
 	p.ServeHTTP(w, r)
+	res := w.Result()
 
 	if !requestReceived {
 		t.Error("Expected backend to receive request, but it didn't")
 	}
 
-	res := w.Result()
-	verifyHeaders(res.Header, res.Trailer)
+	verifyHeadersTrailers(res.Header, res.Trailer)
 
 	// Make sure {upstream} placeholder is set
 	r.Body = ioutil.NopCloser(strings.NewReader("test"))
-	rr := httpserver.NewResponseRecorder(httptest.NewRecorder())
+	rr := httpserver.NewResponseRecorder(testResponseRecorder{
+		ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: httptest.NewRecorder()},
+	})
 	rr.Replacer = httpserver.NewReplacer(r, rr, "-")
 
 	p.ServeHTTP(rr, r)
@@ -112,6 +155,21 @@ func TestReverseProxy(t *testing.T) {
 	}
 }
 
+// trailerTestStringReader is used to test unannounced trailers coming
+// from a client which should properly be proxied to the upstream.
+type trailerTestStringReader struct {
+	strings.Reader
+	request       *http.Request
+	trailersToSet http.Header
+}
+
+var _ io.ReadCloser = &trailerTestStringReader{}
+
+func (r *trailerTestStringReader) Close() error {
+	copyHeader(r.request.Trailer, r.trailersToSet)
+	return nil
+}
+
 func TestReverseProxyInsecureSkipVerify(t *testing.T) {
 	log.SetOutput(ioutil.Discard)
 	defer log.SetOutput(os.Stderr)
@@ -247,8 +305,8 @@ func TestWebSocketReverseProxyNonHijackerPanic(t *testing.T) {
 func TestWebSocketReverseProxyServeHTTPHandler(t *testing.T) {
 	// No-op websocket backend simply allows the WS connection to be
 	// accepted then it will be immediately closed. Perfect for testing.
-	var connCount int32
-	wsNop := httptest.NewServer(websocket.Handler(func(ws *websocket.Conn) { atomic.AddInt32(&connCount, 1) }))
+	accepted := make(chan struct{})
+	wsNop := httptest.NewServer(websocket.Handler(func(ws *websocket.Conn) { close(accepted) }))
 	defer wsNop.Close()
 
 	// Get proxy to use for the test
@@ -279,8 +337,14 @@ func TestWebSocketReverseProxyServeHTTPHandler(t *testing.T) {
 	if !bytes.Equal(actual, expected) {
 		t.Errorf("Expected backend to accept response:\n'%s'\nActually got:\n'%s'", expected, actual)
 	}
-	if got, want := atomic.LoadInt32(&connCount), int32(1); got != want {
-		t.Errorf("Expected %d websocket connection, got %d", want, got)
+
+	// wait a minute for backend handling, see issue 1654.
+	time.Sleep(10 * time.Millisecond)
+
+	select {
+	case <-accepted:
+	default:
+		t.Error("Expect a accepted websocket connection, but not")
 	}
 }
 
@@ -897,11 +961,10 @@ func basicAuthTestcase(t *testing.T, upstreamUser, clientUser *url.Userinfo) {
 
 func TestProxyDirectorURL(t *testing.T) {
 	for i, c := range []struct {
-		originalPath string
-		requestURL   string
-		targetURL    string
-		without      string
-		expectURL    string
+		requestURL string
+		targetURL  string
+		without    string
+		expectURL  string
 	}{
 		{
 			requestURL: `http://localhost:2020/test`,
@@ -972,10 +1035,15 @@ func TestProxyDirectorURL(t *testing.T) {
 			expectURL:  `https://localhost:2021/%2C/%2C`,
 		},
 		{
-			originalPath: `///test`,
-			requestURL:   `http://localhost:2020/%2F/test`,
-			targetURL:    `https://localhost:2021/`,
-			expectURL:    `https://localhost:2021/%2F/test`,
+			requestURL: `http://localhost:2020/%2F/test`,
+			targetURL:  `https://localhost:2021/`,
+			expectURL:  `https://localhost:2021/%2F/test`,
+		},
+		{
+			requestURL: `http://localhost:2020/test/%2F/mypath`,
+			targetURL:  `https://localhost:2021/t/`,
+			expectURL:  `https://localhost:2021/t/%2F/mypath`,
+			without:    "/test",
 		},
 	} {
 		targetURL, err := url.Parse(c.targetURL)
@@ -988,8 +1056,6 @@ func TestProxyDirectorURL(t *testing.T) {
 			t.Errorf("case %d failed to create request: %s", i, err)
 			continue
 		}
-		req = req.WithContext(context.WithValue(req.Context(),
-			staticfiles.URLPathCtxKey, c.originalPath))
 
 		NewSingleHostReverseProxy(targetURL, c.without, 0).Director(req)
 		if expect, got := c.expectURL, req.URL.String(); expect != got {
@@ -1119,7 +1185,18 @@ func TestReverseProxyLargeBody(t *testing.T) {
 }
 
 func TestCancelRequest(t *testing.T) {
+	reqInFlight := make(chan struct{})
 	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		close(reqInFlight) // cause the client to cancel its request
+
+		select {
+		case <-time.After(10 * time.Second):
+			t.Error("Handler never saw CloseNotify")
+			return
+		case <-w.(http.CloseNotifier).CloseNotify():
+		}
+
+		w.WriteHeader(http.StatusOK)
 		w.Write([]byte("Hello, client"))
 	}))
 	defer backend.Close()
@@ -1136,26 +1213,21 @@ func TestCancelRequest(t *testing.T) {
 	defer cancel()
 	req = req.WithContext(ctx)
 
-	// add GotConn hook to cancel the request
-	gotC := make(chan struct{})
-	defer close(gotC)
-	trace := &httptrace.ClientTrace{
-		GotConn: func(connInfo httptrace.GotConnInfo) {
-			gotC <- struct{}{}
-		},
-	}
-	req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
-
 	// wait for canceling the request
 	go func() {
-		<-gotC
+		<-reqInFlight
 		cancel()
 	}()
 
-	status, err := p.ServeHTTP(httptest.NewRecorder(), req)
-	if status != 0 || err != nil {
-		t.Errorf("expect proxy handle normally, but not, status:%d, err:%q",
-			status, err)
+	rec := httptest.NewRecorder()
+	status, err := p.ServeHTTP(rec, req)
+	expectedStatus, expectErr := http.StatusBadGateway, context.Canceled
+	if status != expectedStatus || err != expectErr {
+		t.Errorf("expect proxy handle return status[%d] with error[%v], but got status[%d] with error[%v]",
+			expectedStatus, expectErr, status, err)
+	}
+	if body := rec.Body.String(); body != "" {
+		t.Errorf("expect a blank response, but got %q", body)
 	}
 }
 
@@ -1306,6 +1378,17 @@ func (c *fakeConn) Close() error                       { return nil }
 func (c *fakeConn) Read(b []byte) (int, error)         { return c.readBuf.Read(b) }
 func (c *fakeConn) Write(b []byte) (int, error)        { return c.writeBuf.Write(b) }
 
+// testResponseRecorder wraps `httptest.ResponseRecorder`,
+// also implements `http.CloseNotifier`, `http.Hijacker` and `http.Pusher`.
+type testResponseRecorder struct {
+	*httpserver.ResponseWriterWrapper
+}
+
+func (testResponseRecorder) CloseNotify() <-chan bool { return nil }
+
+// Interface guards
+var _ httpserver.HTTPInterfaces = testResponseRecorder{}
+
 func BenchmarkProxy(b *testing.B) {
 	backend := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
 		w.Write([]byte("Hello, client"))

caddyhttp/proxy/reverseproxy.go

@@ -12,7 +12,6 @@
 package proxy
 
 import (
-	"context"
 	"crypto/tls"
 	"io"
 	"net"
@@ -25,7 +24,6 @@ import (
 	"golang.org/x/net/http2"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
-	"github.com/mholt/caddy/caddyhttp/staticfiles"
 )
 
 var (
@@ -120,7 +118,7 @@ func NewSingleHostReverseProxy(target *url.URL, without string, keepalive int) *
 			req.URL.Host = target.Host
 		}
 
-		// We should remove the `without` prefix at first.
+		// remove the `without` prefix
 		if without != "" {
 			req.URL.Path = strings.TrimPrefix(req.URL.Path, without)
 			if req.URL.Opaque != "" {
@@ -138,6 +136,7 @@ func NewSingleHostReverseProxy(target *url.URL, without string, keepalive int) *
 			}
 			return def
 		}
+
 		// Make up the final URL by concatenating the request and target URL.
 		//
 		// If there is encoded part in request or target URL,
@@ -155,13 +154,7 @@ func NewSingleHostReverseProxy(target *url.URL, without string, keepalive int) *
 				prefer(target.RawPath, target.Path),
 				prefer(req.URL.RawPath, req.URL.Path))
 		}
-		untouchedPath, _ := req.Context().Value(staticfiles.URLPathCtxKey).(string)
-		req.URL.Path = singleJoiningSlash(target.Path,
-			prefer(untouchedPath, req.URL.Path))
-		// req.URL.Path must be consistent with decoded form of req.URL.RawPath if any
-		if req.URL.RawPath != "" && req.URL.RawPath != req.URL.EscapedPath() {
-			panic("RawPath doesn't match Path")
-		}
+		req.URL.Path = singleJoiningSlash(target.Path, req.URL.Path)
 
 		// Trims the path of the socket from the URL path.
 		// This is done because req.URL passed to your proxied service
@@ -253,14 +246,6 @@ func (rp *ReverseProxy) ServeHTTP(rw http.ResponseWriter, outreq *http.Request,
 
 	rp.Director(outreq)
 
-	// Original incoming server request may be canceled by the
-	// user or by std lib(e.g. too many idle connections).
-	// Now we issue the new outgoing client request which
-	// doesn't depend on the original one. (issue 1345)
-	ctx, cancel := context.WithCancel(context.Background())
-	defer cancel()
-	outreq = outreq.WithContext(ctx)
-
 	res, err := transport.RoundTrip(outreq)
 	if err != nil {
 		return err
@@ -333,30 +318,61 @@ func (rp *ReverseProxy) ServeHTTP(rw http.ResponseWriter, outreq *http.Request,
 		}
 		pooledIoCopy(backendConn, conn)
 	} else {
+		// NOTE:
+		//   Closing the Body involves acquiring a mutex, which is a
+		//   unnecessarily heavy operation, considering that this defer will
+		//   pretty much never be executed with the Body still unclosed.
+		bodyOpen := true
+		closeBody := func() {
+			if bodyOpen {
+				res.Body.Close()
+				bodyOpen = false
+			}
+		}
+		defer closeBody()
+
+		// Copy all headers over.
+		// res.Header does not include the "Trailer" header,
+		// which means we will have to do that manually below.
 		copyHeader(rw.Header(), res.Header)
 
-		// The "Trailer" header isn't included in the Transport's response,
-		// at least for *http.Transport. Build it up from Trailer.
-		if len(res.Trailer) > 0 {
-			trailerKeys := make([]string, 0, len(res.Trailer))
+		// The "Trailer" header isn't included in res' Header map, which
+		// is why we have to build one ourselves from res.Trailer.
+		//
+		// But res.Trailer does not necessarily contain all trailer keys at this
+		// point yet. The HTTP spec allows one to send "unannounced trailers"
+		// after a request and certain systems like gRPC make use of that.
+		announcedTrailerKeyCount := len(res.Trailer)
+		if announcedTrailerKeyCount > 0 {
+			vv := make([]string, 0, announcedTrailerKeyCount)
 			for k := range res.Trailer {
-				trailerKeys = append(trailerKeys, k)
+				vv = append(vv, k)
 			}
-			rw.Header().Add("Trailer", strings.Join(trailerKeys, ", "))
+			rw.Header()["Trailer"] = vv
 		}
 
+		// Now copy over the status code as well as the response body.
 		rw.WriteHeader(res.StatusCode)
-		if len(res.Trailer) > 0 {
+		if announcedTrailerKeyCount > 0 {
 			// Force chunking if we saw a response trailer.
-			// This prevents net/http from calculating the length for short
-			// bodies and adding a Content-Length.
+			// This prevents net/http from calculating the length
+			// for short bodies and adding a Content-Length.
 			if fl, ok := rw.(http.Flusher); ok {
 				fl.Flush()
 			}
 		}
 		rp.copyResponse(rw, res.Body)
-		res.Body.Close() // close now, instead of defer, to populate res.Trailer
-		copyHeader(rw.Header(), res.Trailer)
+
+		// Now close the body to fully populate res.Trailer.
+		closeBody()
+
+		// Since Go does not remove keys from res.Trailer we
+		// can safely do a length comparison to check wether
+		// we received further, unannounced trailers.
+		//
+		// Most of the time forceSetTrailers should be false.
+		forceSetTrailers := len(res.Trailer) != announcedTrailerKeyCount
+		shallowCopyTrailers(rw.Header(), res.Trailer, forceSetTrailers)
 	}
 
 	return nil
@@ -406,6 +422,22 @@ func copyHeader(dst, src http.Header) {
 	}
 }
 
+// shallowCopyTrailers copies all headers from srcTrailer to dstHeader.
+//
+// If forceSetTrailers is set to true, the http.TrailerPrefix will be added to
+// all srcTrailer key names. Otherwise the Go stdlib will ignore all keys
+// which weren't listed in the Trailer map before submitting the Response.
+//
+// WARNING: Only a shallow copy will be created!
+func shallowCopyTrailers(dstHeader, srcTrailer http.Header, forceSetTrailers bool) {
+	for k, vv := range srcTrailer {
+		if forceSetTrailers {
+			k = http.TrailerPrefix + k
+		}
+		dstHeader[k] = vv
+	}
+}
+
 // Hop-by-hop headers. These are removed when sent to the backend.
 // http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html
 var hopHeaders = []string{

caddyhttp/proxy/upstream.go

@@ -4,6 +4,7 @@ import (
 	"fmt"
 	"io"
 	"io/ioutil"
+	"net"
 	"net/http"
 	"net/url"
 	"path"
@@ -42,6 +43,7 @@ type staticUpstream struct {
 		Interval time.Duration
 		Timeout  time.Duration
 		Host     string
+		Port     string
 	}
 	WithoutPathPrefix  string
 	IgnoredSubPaths    []string
@@ -321,6 +323,20 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 			return err
 		}
 		u.HealthCheck.Timeout = dur
+	case "health_check_port":
+		if !c.NextArg() {
+			return c.ArgErr()
+		}
+		port := c.Val()
+		n, err := strconv.Atoi(port)
+		if err != nil {
+			return err
+		}
+
+		if n < 0 {
+			return c.Errf("invalid health_check_port '%s'", port)
+		}
+		u.HealthCheck.Port = port
 	case "header_upstream":
 		var header, value string
 		if !c.Args(&header, &value) {
@@ -380,7 +396,12 @@ func parseBlock(c *caddyfile.Dispenser, u *staticUpstream) error {
 
 func (u *staticUpstream) healthCheck() {
 	for _, host := range u.Hosts {
-		hostURL := host.Name + u.HealthCheck.Path
+		hostURL := host.Name
+		if u.HealthCheck.Port != "" {
+			hostURL = replacePort(host.Name, u.HealthCheck.Port)
+		}
+		hostURL += u.HealthCheck.Path
+
 		var unhealthy bool
 
 		// set up request, needed to be able to modify headers
@@ -483,3 +504,19 @@ func (u *staticUpstream) Stop() error {
 func RegisterPolicy(name string, policy func() Policy) {
 	supportedPolicies[name] = policy
 }
+
+func replacePort(originalURL string, newPort string) string {
+	parsedURL, err := url.Parse(originalURL)
+	if err != nil {
+		return originalURL
+	}
+
+	// handles 'localhost' and 'localhost:8080'
+	parsedHost, _, err := net.SplitHostPort(parsedURL.Host)
+	if err != nil {
+		parsedHost = parsedURL.Host
+	}
+
+	parsedURL.Host = net.JoinHostPort(parsedHost, newPort)
+	return parsedURL.String()
+}

caddyhttp/proxy/upstream_test.go

@@ -2,6 +2,7 @@ package proxy
 
 import (
 	"fmt"
+	"net"
 	"net/http"
 	"net/http/httptest"
 	"strings"
@@ -375,3 +376,75 @@ func TestHealthCheckHost(t *testing.T) {
 		}
 	}
 }
+
+func TestHealthCheckPort(t *testing.T) {
+	var counter int64
+
+	healthCounter := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		r.Body.Close()
+		atomic.AddInt64(&counter, 1)
+	}))
+
+	_, healthPort, err := net.SplitHostPort(healthCounter.Listener.Addr().String())
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	defer healthCounter.Close()
+
+	tests := []struct {
+		config string
+	}{
+		// Test #1: upstream with port
+		{"proxy / localhost:8080 {\n health_check / health_check_port " + healthPort + "\n}"},
+
+		// Test #2: upstream without port (default to 80)
+		{"proxy / localhost {\n health_check / health_check_port " + healthPort + "\n}"},
+	}
+
+	for i, test := range tests {
+		counterValueAtStart := atomic.LoadInt64(&counter)
+		upstreams, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+		if err != nil {
+			t.Error("Expected no error. Got:", err.Error())
+		}
+
+		// Give some time for healthchecks to hit the server.
+		time.Sleep(500 * time.Millisecond)
+
+		for _, upstream := range upstreams {
+			if err := upstream.Stop(); err != nil {
+				t.Errorf("Test %d: Expected no error stopping upstream. Got: %v", i, err.Error())
+			}
+		}
+
+		counterValueAfterShutdown := atomic.LoadInt64(&counter)
+
+		if counterValueAfterShutdown == counterValueAtStart {
+			t.Errorf("Test %d: Expected healthchecks to hit test server. Got no healthchecks.", i)
+		}
+	}
+
+	t.Run("valid_port", func(t *testing.T) {
+		tests := []struct {
+			config string
+		}{
+			// Test #1: invalid port (nil)
+			{"proxy / localhost {\n health_check / health_check_port\n}"},
+
+			// Test #2: invalid port (string)
+			{"proxy / localhost {\n health_check / health_check_port abc\n}"},
+
+			// Test #3: invalid port (negative)
+			{"proxy / localhost {\n health_check / health_check_port -1\n}"},
+		}
+
+		for i, test := range tests {
+			_, err := NewStaticUpstreams(caddyfile.NewDispenser("Testfile", strings.NewReader(test.config)), "")
+			if err == nil {
+				t.Errorf("Test %d accepted invalid config", i)
+			}
+		}
+	})
+
+}

caddyhttp/redirect/redirect_test.go

@@ -2,6 +2,7 @@ package redirect
 
 import (
 	"bytes"
+	"context"
 	"crypto/tls"
 	"io/ioutil"
 	"net/http"
@@ -96,14 +97,16 @@ func TestParametersRedirect(t *testing.T) {
 
 	req, err := http.NewRequest("GET", "/a?b=c", nil)
 	if err != nil {
-		t.Fatalf("Test: Could not create HTTP request: %v", err)
+		t.Fatalf("Test 1: Could not create HTTP request: %v", err)
 	}
+	ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+	req = req.WithContext(ctx)
 
 	rec := httptest.NewRecorder()
 	re.ServeHTTP(rec, req)
 
-	if rec.Header().Get("Location") != "http://example.com/a?b=c" {
-		t.Fatalf("Test: expected location header %q but was %q", "http://example.com/a?b=c", rec.Header().Get("Location"))
+	if got, want := rec.Header().Get("Location"), "http://example.com/a?b=c"; got != want {
+		t.Fatalf("Test 1: expected location header %s but was %s", want, got)
 	}
 
 	re = Redirect{
@@ -114,13 +117,15 @@ func TestParametersRedirect(t *testing.T) {
 
 	req, err = http.NewRequest("GET", "/d?e=f", nil)
 	if err != nil {
-		t.Fatalf("Test: Could not create HTTP request: %v", err)
+		t.Fatalf("Test 2: Could not create HTTP request: %v", err)
 	}
+	ctx = context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+	req = req.WithContext(ctx)
 
 	re.ServeHTTP(rec, req)
 
-	if "http://example.com/a/d?b=c&e=f" != rec.Header().Get("Location") {
-		t.Fatalf("Test: expected location header %q but was %q", "http://example.com/a/d?b=c&e=f", rec.Header().Get("Location"))
+	if got, want := rec.Header().Get("Location"), "http://example.com/a/d?b=c&e=f"; got != want {
+		t.Fatalf("Test 2: expected location header %s but was %s", want, got)
 	}
 }
 

caddyhttp/rewrite/rewrite_test.go

@@ -1,6 +1,7 @@
 package rewrite
 
 import (
+	"context"
 	"fmt"
 	"net/http"
 	"net/http/httptest"
@@ -104,13 +105,14 @@ func TestRewrite(t *testing.T) {
 		if err != nil {
 			t.Fatalf("Test %d: Could not create HTTP request: %v", i, err)
 		}
+		ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+		req = req.WithContext(ctx)
 
 		rec := httptest.NewRecorder()
 		rw.ServeHTTP(rec, req)
 
-		if rec.Body.String() != test.expectedTo {
-			t.Errorf("Test %d: Expected URL to be '%s' but was '%s'",
-				i, test.expectedTo, rec.Body.String())
+		if got, want := rec.Body.String(), test.expectedTo; got != want {
+			t.Errorf("Test %d: Expected URL to be '%s' but was '%s'", i, want, got)
 		}
 	}
 }

caddyhttp/rewrite/to.go

@@ -1,7 +1,6 @@
 package rewrite
 
 import (
-	"context"
 	"log"
 	"net/http"
 	"net/url"
@@ -48,10 +47,6 @@ func To(fs http.FileSystem, r *http.Request, to string, replacer httpserver.Repl
 		return RewriteIgnored
 	}
 
-	// take note of this rewrite for internal use by fastcgi
-	// all we need is the URI, not full URL
-	*r = *r.WithContext(context.WithValue(r.Context(), httpserver.URIxRewriteCtxKey, r.URL.RequestURI()))
-
 	// perform rewrite
 	r.URL.Path = u.Path
 	if query != "" {

caddyhttp/rewrite/to_test.go

@@ -1,9 +1,12 @@
 package rewrite
 
 import (
+	"context"
 	"net/http"
 	"net/url"
 	"testing"
+
+	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
 func TestTo(t *testing.T) {
@@ -39,6 +42,8 @@ func TestTo(t *testing.T) {
 		if err != nil {
 			t.Error(err)
 		}
+		ctx := context.WithValue(r.Context(), httpserver.OriginalURLCtxKey, *r.URL)
+		r = r.WithContext(ctx)
 		To(fs, r, test.to, newReplacer(r))
 		if uri(r.URL) != test.expected {
 			t.Errorf("Test %v: expected %v found %v", i, test.expected, uri(r.URL))

caddyhttp/staticfiles/fileserver.go

@@ -1,3 +1,7 @@
+// Package staticfiles provides middleware for serving static files from disk.
+// Its handler is the default HTTP handler for the HTTP server.
+//
+// TODO: Should this package be rolled into the httpserver package?
 package staticfiles
 
 import (
@@ -5,6 +9,7 @@ import (
 	"net/http"
 	"net/url"
 	"os"
+	"path"
 	"path/filepath"
 	"runtime"
 	"strconv"
@@ -25,47 +30,30 @@ import (
 // Use of this source code is governed by a BSD-style
 // license that can be found in the LICENSE file.
 type FileServer struct {
-	// Jailed disk access
-	Root http.FileSystem
-
-	// List of files to treat as "Not Found"
-	Hide []string
+	Root http.FileSystem // jailed access to the file system
+	Hide []string        // list of files for which to respond with "Not Found"
 }
 
 // ServeHTTP serves static files for r according to fs's configuration.
 func (fs FileServer) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	// r.URL.Path has already been cleaned by Caddy.
-	if r.URL.Path == "" {
-		r.URL.Path = "/"
-	}
-	return fs.serveFile(w, r, r.URL.Path)
-}
-
-// calculateEtag produces a strong etag by default. Prefix the result with "W/" to convert this into a weak one.
-// see https://tools.ietf.org/html/rfc7232#section-2.3
-func calculateEtag(d os.FileInfo) string {
-	t := strconv.FormatInt(d.ModTime().Unix(), 36)
-	s := strconv.FormatInt(d.Size(), 36)
-	return `"` + t + s + `"`
+	return fs.serveFile(w, r)
 }
 
 // serveFile writes the specified file to the HTTP response.
 // name is '/'-separated, not filepath.Separator.
-func (fs FileServer) serveFile(w http.ResponseWriter, r *http.Request, name string) (int, error) {
-
-	location := name
+func (fs FileServer) serveFile(w http.ResponseWriter, r *http.Request) (int, error) {
+	reqPath := r.URL.Path
 
 	// Prevent absolute path access on Windows.
 	// TODO remove when stdlib http.Dir fixes this.
-	if runtime.GOOS == "windows" {
-		if filepath.IsAbs(name[1:]) {
-			return http.StatusNotFound, nil
-		}
+	if runtime.GOOS == "windows" && len(reqPath) > 0 && filepath.IsAbs(reqPath[1:]) {
+		return http.StatusNotFound, nil
 	}
 
-	f, err := fs.Root.Open(name)
+	// open the requested file
+	f, err := fs.Root.Open(reqPath)
 	if err != nil {
-		// TODO: remove when http.Dir handles this
+		// TODO: remove when http.Dir handles this (Go 1.9?)
 		// Go issue #18984
 		err = mapFSRootOpenErr(err)
 		if os.IsNotExist(err) {
@@ -73,13 +61,14 @@ func (fs FileServer) serveFile(w http.ResponseWriter, r *http.Request, name stri
 		} else if os.IsPermission(err) {
 			return http.StatusForbidden, err
 		}
-		// Likely the server is under load and ran out of file descriptors
+		// otherwise, maybe the server is under load and ran out of file descriptors?
 		backoff := int(3 + rand.Int31()%3) // 3–5 seconds to prevent a stampede
 		w.Header().Set("Retry-After", strconv.Itoa(backoff))
 		return http.StatusServiceUnavailable, err
 	}
 	defer f.Close()
 
+	// get information about the file
 	d, err := f.Stat()
 	if err != nil {
 		if os.IsNotExist(err) {
@@ -87,84 +76,105 @@ func (fs FileServer) serveFile(w http.ResponseWriter, r *http.Request, name stri
 		} else if os.IsPermission(err) {
 			return http.StatusForbidden, err
 		}
-		// Return a different status code than above so as to distinguish these cases
+		// return a different status code than above to distinguish these cases
 		return http.StatusInternalServerError, err
 	}
 
-	// redirect to canonical path
+	// redirect to canonical path (being careful to preserve other parts of URL and
+	// considering cases where a site is defined with a path prefix that gets stripped)
+	u := r.Context().Value(caddy.CtxKey("original_url")).(url.URL)
+	if u.Path == "" {
+		u.Path = "/"
+	}
 	if d.IsDir() {
-		// Ensure / at end of directory url. If the original URL path is
-		// used then ensure / exists as well.
-		if !strings.HasSuffix(r.URL.Path, "/") {
-			RedirectToDir(w, r)
+		// ensure there is a trailing slash
+		if u.Path[len(u.Path)-1] != '/' {
+			u.Path += "/"
+			http.Redirect(w, r, u.String(), http.StatusMovedPermanently)
 			return http.StatusMovedPermanently, nil
 		}
 	} else {
-		// Ensure no / at end of file url. If the original URL path is
-		// used then ensure no / exists as well.
-		if strings.HasSuffix(r.URL.Path, "/") {
-			RedirectToFile(w, r)
+		// ensure no trailing slash
+		redir := false
+		if u.Path[len(u.Path)-1] == '/' {
+			u.Path = u.Path[:len(u.Path)-1]
+			redir = true
+		}
+
+		// if an index file was explicitly requested, strip file name from the request
+		// ("/foo/index.html" -> "/foo/")
+		for _, indexPage := range IndexPages {
+			if strings.HasSuffix(u.Path, indexPage) {
+				u.Path = u.Path[:len(u.Path)-len(indexPage)]
+				redir = true
+				break
+			}
+		}
+
+		if redir {
+			http.Redirect(w, r, u.String(), http.StatusMovedPermanently)
 			return http.StatusMovedPermanently, nil
 		}
 	}
 
-	// use contents of an index file, if present, for directory
+	// use contents of an index file, if present, for directory requests
 	if d.IsDir() {
 		for _, indexPage := range IndexPages {
-			index := strings.TrimSuffix(name, "/") + "/" + indexPage
-			ff, err := fs.Root.Open(index)
+			indexPath := path.Join(reqPath, indexPage)
+			indexFile, err := fs.Root.Open(indexPath)
 			if err != nil {
 				continue
 			}
 
-			// this defer does not leak fds because previous iterations
-			// of the loop must have had an err, so nothing to close
-			defer ff.Close()
-
-			dd, err := ff.Stat()
+			indexInfo, err := indexFile.Stat()
 			if err != nil {
-				ff.Close()
+				indexFile.Close()
 				continue
 			}
 
-			// Close previous file - release fd immediately
+			// this defer does not leak fds even though we are in a loop,
+			// because previous iterations of the loop must have had an
+			// err, so there's nothing to close from earlier iterations.
+			defer indexFile.Close()
+
+			// close previously-opened file immediately to release fd
 			f.Close()
 
-			d = dd
-			f = ff
-			location = index
+			// switch to using the index file, and we're done here
+			d = indexInfo
+			f = indexFile
+			reqPath = indexPath
 			break
 		}
 	}
 
-	// Still a directory? (we didn't find an index file)
-	// Return 404 to hide the fact that the folder exists
-	if d.IsDir() {
+	// return Not Found if we either did not find an index file (and thus are
+	// still a directory) or if this file is supposed to be hidden
+	if d.IsDir() || fs.IsHidden(d) {
 		return http.StatusNotFound, nil
 	}
 
-	if fs.IsHidden(d) {
-		return http.StatusNotFound, nil
-	}
-
-	filename := d.Name()
-	etag := calculateEtag(d) // strong
+	etag := calculateEtag(d)
 
+	// look for compressed versions of the file on disk, if the client supports that encoding
 	for _, encoding := range staticEncodingPriority {
+		// see if the client accepts a compressed encoding we offer
 		acceptEncoding := strings.Split(r.Header.Get("Accept-Encoding"), ",")
-
 		accepted := false
 		for _, acc := range acceptEncoding {
-			if accepted || strings.TrimSpace(acc) == encoding {
+			if strings.TrimSpace(acc) == encoding {
 				accepted = true
+				break
 			}
 		}
 
+		// if client doesn't support this encoding, don't even bother; try next one
 		if !accepted {
 			continue
 		}
 
-		encodedFile, err := fs.Root.Open(location + staticEncoding[encoding])
+		// see if the compressed version of this file exists
+		encodedFile, err := fs.Root.Open(reqPath + staticEncoding[encoding])
 		if err != nil {
 			continue
 		}
@@ -175,19 +185,17 @@ func (fs FileServer) serveFile(w http.ResponseWriter, r *http.Request, name stri
 			continue
 		}
 
-		// Close previous file - release fd
+		// close the encoded file when we're done, and close the
+		// previously-opened file immediately to release the fd
+		defer encodedFile.Close()
 		f.Close()
 
-		etag = calculateEtag(encodedFileInfo)
-
-		// Encoded file will be served
+		// the encoded file is now what we're serving
 		f = encodedFile
-
+		etag = calculateEtag(encodedFileInfo)
 		w.Header().Add("Vary", "Accept-Encoding")
 		w.Header().Set("Content-Encoding", encoding)
 		w.Header().Set("Content-Length", strconv.FormatInt(encodedFileInfo.Size(), 10))
-
-		defer f.Close()
 		break
 	}
 
@@ -197,16 +205,17 @@ func (fs FileServer) serveFile(w http.ResponseWriter, r *http.Request, name stri
 
 	// Note: Errors generated by ServeContent are written immediately
 	// to the response. This usually only happens if seeking fails (rare).
-	http.ServeContent(w, r, filename, d.ModTime(), f)
+	// Its signature does not bubble the error up to us, so we cannot
+	// return it for any logging middleware to record. Oh well.
+	http.ServeContent(w, r, d.Name(), d.ModTime(), f)
 
 	return http.StatusOK, nil
 }
 
 // IsHidden checks if file with FileInfo d is on hide list.
 func (fs FileServer) IsHidden(d os.FileInfo) bool {
-	// If the file is supposed to be hidden, return a 404
 	for _, hiddenPath := range fs.Hide {
-		// Check if the served file is exactly the hidden file.
+		// TODO: Could these FileInfos be stored instead of their paths, to avoid opening them all the time?
 		if hFile, err := fs.Root.Open(hiddenPath); err == nil {
 			fs, _ := hFile.Stat()
 			hFile.Close()
@@ -218,34 +227,15 @@ func (fs FileServer) IsHidden(d os.FileInfo) bool {
 	return false
 }
 
-// RedirectToDir replies to the request with a redirect to the URL in r, which
-// has been transformed to indicate that the resource being requested is a
-// directory.
-func RedirectToDir(w http.ResponseWriter, r *http.Request) {
-	toURL, _ := url.Parse(r.URL.String())
-
-	path, ok := r.Context().Value(URLPathCtxKey).(string)
-	if ok && !strings.HasSuffix(path, "/") {
-		toURL.Path = path
-	}
-	toURL.Path += "/"
-
-	http.Redirect(w, r, toURL.String(), http.StatusMovedPermanently)
-}
-
-// RedirectToFile replies to the request with a redirect to the URL in r, which
-// has been transformed to indicate that the resource being requested is a
-// file.
-func RedirectToFile(w http.ResponseWriter, r *http.Request) {
-	toURL, _ := url.Parse(r.URL.String())
-
-	path, ok := r.Context().Value(URLPathCtxKey).(string)
-	if ok && strings.HasSuffix(path, "/") {
-		toURL.Path = path
-	}
-	toURL.Path = strings.TrimSuffix(toURL.Path, "/")
-
-	http.Redirect(w, r, toURL.String(), http.StatusMovedPermanently)
+// calculateEtag produces a strong etag by default, although, for
+// efficiency reasons, it does not actually consume the contents
+// of the file to make a hash of all the bytes. ¯\_(ツ)_/¯
+// Prefix the etag with "W/" to convert it into a weak etag.
+// See: https://tools.ietf.org/html/rfc7232#section-2.3
+func calculateEtag(d os.FileInfo) string {
+	t := strconv.FormatInt(d.ModTime().Unix(), 36)
+	s := strconv.FormatInt(d.Size(), 36)
+	return `"` + t + s + `"`
 }
 
 // IndexPages is a list of pages that may be understood as
@@ -277,7 +267,7 @@ var staticEncodingPriority = []string{
 // to a possibly better non-nil error. In particular, it turns OS-specific errors
 // about opening files in non-directories into os.ErrNotExist.
 //
-// TODO: remove when http.Dir handles this
+// TODO: remove when http.Dir handles this (slated for Go 1.9)
 // Go issue #18984
 func mapFSRootOpenErr(originalErr error) error {
 	if os.IsNotExist(originalErr) || os.IsPermission(originalErr) {
@@ -304,8 +294,3 @@ func mapFSRootOpenErr(originalErr error) error {
 	}
 	return originalErr
 }
-
-// URLPathCtxKey is a context key. It can be used in HTTP handlers with
-// context.WithValue to access the original request URI that accompanied the
-// server request. The associated value will be of type string.
-const URLPathCtxKey caddy.CtxKey = "url_path"

caddyhttp/staticfiles/fileserver_test.go

@@ -3,72 +3,26 @@ package staticfiles
 import (
 	"context"
 	"errors"
+	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
-	"net/url"
 	"os"
 	"path/filepath"
 	"strconv"
 	"strings"
 	"testing"
 	"time"
+
+	"github.com/mholt/caddy"
 )
 
-var (
-	ErrCustom = errors.New("Custom Error")
-
-	testDir     = filepath.Join(os.TempDir(), "caddy_testdir")
-	testWebRoot = filepath.Join(testDir, "webroot")
-)
-
-var (
-	webrootFile1HTML                   = filepath.Join("webroot", "file1.html")
-	webrootDirFile2HTML                = filepath.Join("webroot", "dir", "file2.html")
-	webrootDirHiddenHTML               = filepath.Join("webroot", "dir", "hidden.html")
-	webrootDirwithindexIndeHTML        = filepath.Join("webroot", "dirwithindex", "index.html")
-	webrootSubGzippedHTML              = filepath.Join("webroot", "sub", "gzipped.html")
-	webrootSubGzippedHTMLGz            = filepath.Join("webroot", "sub", "gzipped.html.gz")
-	webrootSubGzippedHTMLBr            = filepath.Join("webroot", "sub", "gzipped.html.br")
-	webrootSubBrotliHTML               = filepath.Join("webroot", "sub", "brotli.html")
-	webrootSubBrotliHTMLGz             = filepath.Join("webroot", "sub", "brotli.html.gz")
-	webrootSubBrotliHTMLBr             = filepath.Join("webroot", "sub", "brotli.html.br")
-	webrootSubBarDirWithIndexIndexHTML = filepath.Join("webroot", "bar", "dirwithindex", "index.html")
-)
-
-// testFiles is a map with relative paths to test files as keys and file content as values.
-// The map represents the following structure:
-// - $TEMP/caddy_testdir/
-// '-- unreachable.html
-// '-- webroot/
-// '---- file1.html
-// '---- dirwithindex/
-// '------ index.html
-// '---- dir/
-// '------ file2.html
-// '------ hidden.html
-var testFiles = map[string]string{
-	"unreachable.html":                 "<h1>must not leak</h1>",
-	webrootFile1HTML:                   "<h1>file1.html</h1>",
-	webrootDirFile2HTML:                "<h1>dir/file2.html</h1>",
-	webrootDirwithindexIndeHTML:        "<h1>dirwithindex/index.html</h1>",
-	webrootDirHiddenHTML:               "<h1>dir/hidden.html</h1>",
-	webrootSubGzippedHTML:              "<h1>gzipped.html</h1>",
-	webrootSubGzippedHTMLGz:            "1.gzipped.html.gz",
-	webrootSubGzippedHTMLBr:            "2.gzipped.html.br",
-	webrootSubBrotliHTML:               "3.brotli.html",
-	webrootSubBrotliHTMLGz:             "4.brotli.html.gz",
-	webrootSubBrotliHTMLBr:             "5.brotli.html.br",
-	webrootSubBarDirWithIndexIndexHTML: "<h1>bar/dirwithindex/index.html</h1>",
-}
-
 // TestServeHTTP covers positive scenarios when serving files.
 func TestServeHTTP(t *testing.T) {
-
-	beforeServeHTTPTest(t)
-	defer afterServeHTTPTest(t)
+	tmpWebRootDir := beforeServeHTTPTest(t)
+	defer afterServeHTTPTest(t, tmpWebRootDir)
 
 	fileserver := FileServer{
-		Root: http.Dir(testWebRoot),
+		Root: http.Dir(filepath.Join(tmpWebRootDir, webrootName)),
 		Hide: []string{"dir/hidden.html"},
 	}
 
@@ -76,7 +30,7 @@ func TestServeHTTP(t *testing.T) {
 
 	tests := []struct {
 		url                   string
-		cleanedPath           string
+		stripPathPrefix       string // for when sites are defined with a path (e.g. "example.com/foo/")
 		acceptEncoding        string
 		expectedLocation      string
 		expectedStatus        int
@@ -148,53 +102,62 @@ func TestServeHTTP(t *testing.T) {
 			url:            "https://foo/dir/hidden.html",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 10 - access a index file directly
+		// Test 10 - access an index file directly
 		{
-			url:                   "https://foo/dirwithindex/index.html",
-			expectedStatus:        http.StatusOK,
-			expectedBodyContent:   testFiles[webrootDirwithindexIndeHTML],
-			expectedEtag:          `"2n9cw"`,
-			expectedContentLength: strconv.Itoa(len(testFiles[webrootDirwithindexIndeHTML])),
+			url:              "https://foo/dirwithindex/index.html",
+			expectedStatus:   http.StatusMovedPermanently,
+			expectedLocation: "https://foo/dirwithindex/",
 		},
-		// Test 11 - send a request with query params
+		// Test 11 - access an index file with a trailing slash
+		{
+			url:              "https://foo/dirwithindex/index.html/",
+			expectedStatus:   http.StatusMovedPermanently,
+			expectedLocation: "https://foo/dirwithindex/",
+		},
+		// Test 12 - send a request with query params
 		{
 			url:                 "https://foo/dir?param1=val",
 			expectedStatus:      http.StatusMovedPermanently,
 			expectedLocation:    "https://foo/dir/?param1=val",
 			expectedBodyContent: movedPermanently,
 		},
-		// Test 12 - attempt to bypass hidden file
+		// Test 13 - attempt to bypass hidden file
 		{
 			url:            "https://foo/dir/hidden.html%20",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 13 - attempt to bypass hidden file
+		// Test 14 - attempt to bypass hidden file
 		{
 			url:            "https://foo/dir/hidden.html.",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 14 - attempt to bypass hidden file
+		// Test 15 - attempt to bypass hidden file
 		{
 			url:            "https://foo/dir/hidden.html.%20",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 15 - attempt to bypass hidden file
+		// Test 16 - attempt to bypass hidden file
 		{
 			url:            "https://foo/dir/hidden.html%20.",
 			acceptEncoding: "br, gzip",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 16 - serve another file with same name as hidden file.
+		// Test 17 - serve another file with same name as hidden file.
 		{
 			url:            "https://foo/hidden.html",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 17 - try to get below the root directory.
+		// Test 18 - try to get below the root directory.
 		{
-			url:            "https://foo/%2f..%2funreachable.html",
+			url:            "https://foo/../unreachable.html",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 18 - try to get pre-gzipped file.
+		// Test 19 - try to get below the root directory (encoded slashes).
+		{
+			url:            "https://foo/..%2funreachable.html",
+			expectedStatus: http.StatusNotFound,
+		},
+		// Test 20 - try to get pre-gzipped file.
 		{
 			url:                   "https://foo/sub/gzipped.html",
 			acceptEncoding:        "gzip",
@@ -205,7 +168,7 @@ func TestServeHTTP(t *testing.T) {
 			expectedEncoding:      "gzip",
 			expectedContentLength: strconv.Itoa(len(testFiles[webrootSubGzippedHTMLGz])),
 		},
-		// Test 19 - try to get pre-brotli encoded file.
+		// Test 21 - try to get pre-brotli encoded file.
 		{
 			url:                   "https://foo/sub/brotli.html",
 			acceptEncoding:        "br,gzip",
@@ -216,7 +179,7 @@ func TestServeHTTP(t *testing.T) {
 			expectedEncoding:      "br",
 			expectedContentLength: strconv.Itoa(len(testFiles[webrootSubBrotliHTMLBr])),
 		},
-		// Test 20 - not allowed to get pre-brotli encoded file.
+		// Test 22 - not allowed to get pre-brotli encoded file.
 		{
 			url:                   "https://foo/sub/brotli.html",
 			acceptEncoding:        "nicebrew", // contains "br" substring but not "br"
@@ -227,33 +190,31 @@ func TestServeHTTP(t *testing.T) {
 			expectedEncoding:      "",
 			expectedContentLength: strconv.Itoa(len(testFiles[webrootSubBrotliHTML])),
 		},
-		// Test 20 - treat existing file as a directory.
+		// Test 23 - treat existing file as a directory.
 		{
 			url:            "https://foo/file1.html/other",
 			expectedStatus: http.StatusNotFound,
 		},
-		// Test 20 - access folder with index file without trailing slash, with
-		// cleaned path
+		// Test 24 - access folder with index file without trailing slash, with stripped path
 		{
 			url:                 "https://foo/bar/dirwithindex",
-			cleanedPath:         "/dirwithindex",
+			stripPathPrefix:     "/bar/",
 			expectedStatus:      http.StatusMovedPermanently,
 			expectedLocation:    "https://foo/bar/dirwithindex/",
 			expectedBodyContent: movedPermanently,
 		},
-		// Test 21 - access folder with index file without trailing slash, with
-		// cleaned path and query params
+		// Test 25 - access folder with index file without trailing slash, with stripped path and query params
 		{
 			url:                 "https://foo/bar/dirwithindex?param1=val",
-			cleanedPath:         "/dirwithindex",
+			stripPathPrefix:     "/bar/",
 			expectedStatus:      http.StatusMovedPermanently,
 			expectedLocation:    "https://foo/bar/dirwithindex/?param1=val",
 			expectedBodyContent: movedPermanently,
 		},
-		// Test 22 - access file with trailing slash with cleaned path
+		// Test 26 - site defined with path ("bar"), which has that prefix stripped
 		{
 			url:                 "https://foo/bar/file1.html/",
-			cleanedPath:         "file1.html/",
+			stripPathPrefix:     "/bar/",
 			expectedStatus:      http.StatusMovedPermanently,
 			expectedLocation:    "https://foo/bar/file1.html",
 			expectedBodyContent: movedPermanently,
@@ -261,26 +222,26 @@ func TestServeHTTP(t *testing.T) {
 	}
 
 	for i, test := range tests {
+		// set up response writer and rewuest
 		responseRecorder := httptest.NewRecorder()
 		request, err := http.NewRequest("GET", test.url, nil)
-		ctx := context.WithValue(request.Context(), URLPathCtxKey, request.URL.Path)
+		if err != nil {
+			t.Errorf("Test %d: Error making request: %v", i, err)
+			continue
+		}
+
+		// set the original URL on the context
+		ctx := context.WithValue(request.Context(), caddy.CtxKey("original_url"), *request.URL)
 		request = request.WithContext(ctx)
 
 		request.Header.Add("Accept-Encoding", test.acceptEncoding)
 
-		if err != nil {
-			t.Errorf("Test %d: Error making request: %v", i, err)
-		}
-		// prevent any URL sanitization within Go: we need unmodified paths here
-		if u, _ := url.Parse(test.url); u.RawPath != "" {
-			request.URL.Path = u.RawPath
-		}
-		// Caddy may trim a request's URL path. Overwrite the path with
-		// the cleanedPath to test redirects when the path has been
-		// modified.
-		if test.cleanedPath != "" {
-			request.URL.Path = test.cleanedPath
+		// simulate cases where a site is defined with a path prefix (e.g. "localhost/foo/")
+		if test.stripPathPrefix != "" {
+			request.URL.Path = strings.TrimPrefix(request.URL.Path, test.stripPathPrefix)
 		}
+
+		// perform the test
 		status, err := fileserver.ServeHTTP(responseRecorder, request)
 		etag := responseRecorder.Header().Get("Etag")
 		body := responseRecorder.Body.String()
@@ -318,6 +279,7 @@ func TestServeHTTP(t *testing.T) {
 			t.Errorf("Test %d: Expected body to contain %q, found %q", i, test.expectedBodyContent, body)
 		}
 
+		// check Location header
 		if test.expectedLocation != "" {
 			l := responseRecorder.Header().Get("Location")
 			if test.expectedLocation != l {
@@ -334,20 +296,16 @@ func TestServeHTTP(t *testing.T) {
 }
 
 // beforeServeHTTPTest creates a test directory with the structure, defined in the variable testFiles
-func beforeServeHTTPTest(t *testing.T) {
-	// make the root test dir
-	err := os.MkdirAll(testWebRoot, os.ModePerm)
+func beforeServeHTTPTest(t *testing.T) string {
+	tmpdir, err := ioutil.TempDir("", testDirPrefix)
 	if err != nil {
-		if !os.IsExist(err) {
-			t.Fatalf("Failed to create test dir. Error was: %v", err)
-			return
-		}
+		t.Fatalf("failed to create test directory: %v", err)
 	}
 
 	fixedTime := time.Unix(123456, 0)
 
 	for relFile, fileContent := range testFiles {
-		absFile := filepath.Join(testDir, relFile)
+		absFile := filepath.Join(tmpdir, relFile)
 
 		// make sure the parent directories exist
 		parentDir := filepath.Dir(absFile)
@@ -360,14 +318,12 @@ func beforeServeHTTPTest(t *testing.T) {
 		f, err := os.Create(absFile)
 		if err != nil {
 			t.Fatalf("Failed to create test file %s. Error was: %v", absFile, err)
-			return
 		}
 
 		// and fill them with content
 		_, err = f.WriteString(fileContent)
 		if err != nil {
 			t.Fatalf("Failed to write to %s. Error was: %v", absFile, err)
-			return
 		}
 		f.Close()
 
@@ -378,14 +334,18 @@ func beforeServeHTTPTest(t *testing.T) {
 		}
 	}
 
+	return tmpdir
 }
 
 // afterServeHTTPTest removes the test dir and all its content
-func afterServeHTTPTest(t *testing.T) {
+func afterServeHTTPTest(t *testing.T, webroot string) {
+	if !strings.Contains(webroot, testDirPrefix) {
+		t.Fatalf("Cannot clean up after test because webroot is: %s", webroot)
+	}
 	// cleans up everything under the test dir. No need to clean the individual files.
-	err := os.RemoveAll(testDir)
+	err := os.RemoveAll(webroot)
 	if err != nil {
-		t.Fatalf("Failed to clean up test dir %s. Error was: %v", testDir, err)
+		t.Fatalf("Failed to clean up test dir %s. Error was: %v", webroot, err)
 	}
 }
 
@@ -416,9 +376,9 @@ func (ff failingFile) Close() error {
 	return nil
 }
 
-// TestServeHTTPFailingFS tests error cases where the Open function fails with various errors.
+// TestServeHTTPFailingFS tests error cases where the Open
+// function fails with various errors.
 func TestServeHTTPFailingFS(t *testing.T) {
-
 	tests := []struct {
 		fsErr           error
 		expectedStatus  int
@@ -436,9 +396,9 @@ func TestServeHTTPFailingFS(t *testing.T) {
 			expectedErr:    os.ErrPermission,
 		},
 		{
-			fsErr:           ErrCustom,
+			fsErr:           errCustom,
 			expectedStatus:  http.StatusServiceUnavailable,
-			expectedErr:     ErrCustom,
+			expectedErr:     errCustom,
 			expectedHeaders: map[string]string{"Retry-After": "5"},
 		},
 	}
@@ -478,9 +438,9 @@ func TestServeHTTPFailingFS(t *testing.T) {
 	}
 }
 
-// TestServeHTTPFailingStat tests error cases where the initial Open function succeeds, but the Stat method on the opened file fails.
+// TestServeHTTPFailingStat tests error cases where the initial Open function succeeds,
+// but the Stat method on the opened file fails.
 func TestServeHTTPFailingStat(t *testing.T) {
-
 	tests := []struct {
 		statErr        error
 		expectedStatus int
@@ -497,9 +457,9 @@ func TestServeHTTPFailingStat(t *testing.T) {
 			expectedErr:    os.ErrPermission,
 		},
 		{
-			statErr:        ErrCustom,
+			statErr:        errCustom,
 			expectedStatus: http.StatusInternalServerError,
-			expectedErr:    ErrCustom,
+			expectedErr:    errCustom,
 		},
 	}
 
@@ -528,6 +488,54 @@ func TestServeHTTPFailingStat(t *testing.T) {
 	}
 }
 
+// Paths for the fake site used temporarily during testing.
+var (
+	webrootFile1HTML                   = filepath.Join(webrootName, "file1.html")
+	webrootDirFile2HTML                = filepath.Join(webrootName, "dir", "file2.html")
+	webrootDirHiddenHTML               = filepath.Join(webrootName, "dir", "hidden.html")
+	webrootDirwithindexIndeHTML        = filepath.Join(webrootName, "dirwithindex", "index.html")
+	webrootSubGzippedHTML              = filepath.Join(webrootName, "sub", "gzipped.html")
+	webrootSubGzippedHTMLGz            = filepath.Join(webrootName, "sub", "gzipped.html.gz")
+	webrootSubGzippedHTMLBr            = filepath.Join(webrootName, "sub", "gzipped.html.br")
+	webrootSubBrotliHTML               = filepath.Join(webrootName, "sub", "brotli.html")
+	webrootSubBrotliHTMLGz             = filepath.Join(webrootName, "sub", "brotli.html.gz")
+	webrootSubBrotliHTMLBr             = filepath.Join(webrootName, "sub", "brotli.html.br")
+	webrootSubBarDirWithIndexIndexHTML = filepath.Join(webrootName, "bar", "dirwithindex", "index.html")
+)
+
+// testFiles is a map with relative paths to test files as keys and file content as values.
+// The map represents the following structure:
+// - $TEMP/caddy_testdir/
+// '-- unreachable.html
+// '-- webroot/
+// '---- file1.html
+// '---- dirwithindex/
+// '------ index.html
+// '---- dir/
+// '------ file2.html
+// '------ hidden.html
+var testFiles = map[string]string{
+	"unreachable.html":                 "<h1>must not leak</h1>",
+	webrootFile1HTML:                   "<h1>file1.html</h1>",
+	webrootDirFile2HTML:                "<h1>dir/file2.html</h1>",
+	webrootDirwithindexIndeHTML:        "<h1>dirwithindex/index.html</h1>",
+	webrootDirHiddenHTML:               "<h1>dir/hidden.html</h1>",
+	webrootSubGzippedHTML:              "<h1>gzipped.html</h1>",
+	webrootSubGzippedHTMLGz:            "1.gzipped.html.gz",
+	webrootSubGzippedHTMLBr:            "2.gzipped.html.br",
+	webrootSubBrotliHTML:               "3.brotli.html",
+	webrootSubBrotliHTMLGz:             "4.brotli.html.gz",
+	webrootSubBrotliHTMLBr:             "5.brotli.html.br",
+	webrootSubBarDirWithIndexIndexHTML: "<h1>bar/dirwithindex/index.html</h1>",
+}
+
+var errCustom = errors.New("custom error")
+
+const (
+	testDirPrefix = "caddy_fileserver_test"
+	webrootName   = "webroot" // name of the folder inside the tmp dir that has the site
+)
+
 //-------------------------------------------------------------------------------------------------
 
 type fileInfo struct {
@@ -564,8 +572,6 @@ func (fi fileInfo) Sys() interface{} {
 
 var _ os.FileInfo = fileInfo{}
 
-//-------------------------------------------------------------------------------------------------
-
 func BenchmarkEtag(b *testing.B) {
 	d := fileInfo{
 		size:    1234567890,

caddyhttp/templates/setup.go

@@ -1,7 +1,9 @@
 package templates
 
 import (
+	"bytes"
 	"net/http"
+	"sync"
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -27,6 +29,11 @@ func setup(c *caddy.Controller) error {
 		Rules:   rules,
 		Root:    cfg.Root,
 		FileSys: http.Dir(cfg.Root),
+		BufPool: &sync.Pool{
+			New: func() interface{} {
+				return new(bytes.Buffer)
+			},
+		},
 	}
 
 	cfg.AddMiddleware(func(next httpserver.Handler) httpserver.Handler {

caddyhttp/templates/templates.go

@@ -9,6 +9,7 @@ import (
 	"os"
 	"path"
 	"path/filepath"
+	"sync"
 	"text/template"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -33,7 +34,10 @@ func (t Templates) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error
 		for _, ext := range rule.Extensions {
 			if reqExt == ext {
 				// Create execution context
-				ctx := httpserver.Context{Root: t.FileSys, Req: r, URL: r.URL}
+				ctx := httpserver.NewContextWithHeader(w.Header())
+				ctx.Root = t.FileSys
+				ctx.Req = r
+				ctx.URL = r.URL
 
 				// New template
 				templateName := filepath.Base(fpath)
@@ -60,8 +64,10 @@ func (t Templates) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error
 				}
 
 				// Execute it
-				var buf bytes.Buffer
-				err = tpl.Execute(&buf, ctx)
+				buf := t.BufPool.Get().(*bytes.Buffer)
+				buf.Reset()
+				defer t.BufPool.Put(buf)
+				err = tpl.Execute(buf, ctx)
 				if err != nil {
 					return http.StatusInternalServerError, err
 				}
@@ -97,6 +103,7 @@ type Templates struct {
 	Rules   []Rule
 	Root    string
 	FileSys http.FileSystem
+	BufPool *sync.Pool // docs: "A Pool must not be copied after first use."
 }
 
 // Rule represents a template rule. A template will only execute

caddyhttp/templates/templates_test.go

@@ -1,8 +1,10 @@
 package templates
 
 import (
+	"bytes"
 	"net/http"
 	"net/http/httptest"
+	"sync"
 	"testing"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -28,6 +30,7 @@ func TestTemplates(t *testing.T) {
 		},
 		Root:    "./testdata",
 		FileSys: http.Dir("./testdata"),
+		BufPool: &sync.Pool{New: func() interface{} { return new(bytes.Buffer) }},
 	}
 
 	tmplroot := Templates{
@@ -43,6 +46,7 @@ func TestTemplates(t *testing.T) {
 		},
 		Root:    "./testdata",
 		FileSys: http.Dir("./testdata"),
+		BufPool: &sync.Pool{New: func() interface{} { return new(bytes.Buffer) }},
 	}
 
 	// Test tmpl on /photos/test.html

caddytls/config.go

@@ -9,6 +9,7 @@ import (
 	"net/url"
 	"strings"
 
+	"github.com/codahale/aesnicheck"
 	"github.com/mholt/caddy"
 	"github.com/xenolf/lego/acme"
 )
@@ -232,8 +233,8 @@ func (c *Config) StorageFor(caURL string) (Storage, error) {
 // buildStandardTLSConfig converts cfg (*caddytls.Config) to a *tls.Config
 // and stores it in cfg so it can be used in servers. If TLS is disabled,
 // no tls.Config is created.
-func (cfg *Config) buildStandardTLSConfig() error {
-	if !cfg.Enabled {
+func (c *Config) buildStandardTLSConfig() error {
+	if !c.Enabled {
 		return nil
 	}
 
@@ -243,35 +244,35 @@ func (cfg *Config) buildStandardTLSConfig() error {
 	curvesAdded := make(map[tls.CurveID]struct{})
 
 	// add cipher suites
-	for _, ciph := range cfg.Ciphers {
+	for _, ciph := range c.Ciphers {
 		if _, ok := ciphersAdded[ciph]; !ok {
 			ciphersAdded[ciph] = struct{}{}
 			config.CipherSuites = append(config.CipherSuites, ciph)
 		}
 	}
 
-	config.PreferServerCipherSuites = cfg.PreferServerCipherSuites
+	config.PreferServerCipherSuites = c.PreferServerCipherSuites
 
 	// add curve preferences
-	for _, curv := range cfg.CurvePreferences {
+	for _, curv := range c.CurvePreferences {
 		if _, ok := curvesAdded[curv]; !ok {
 			curvesAdded[curv] = struct{}{}
 			config.CurvePreferences = append(config.CurvePreferences, curv)
 		}
 	}
 
-	config.MinVersion = cfg.ProtocolMinVersion
-	config.MaxVersion = cfg.ProtocolMaxVersion
-	config.ClientAuth = cfg.ClientAuth
-	config.NextProtos = cfg.ALPN
-	config.GetCertificate = cfg.GetCertificate
+	config.MinVersion = c.ProtocolMinVersion
+	config.MaxVersion = c.ProtocolMaxVersion
+	config.ClientAuth = c.ClientAuth
+	config.NextProtos = c.ALPN
+	config.GetCertificate = c.GetCertificate
 
 	// set up client authentication if enabled
 	if config.ClientAuth != tls.NoClientCert {
 		pool := x509.NewCertPool()
 		clientCertsAdded := make(map[string]struct{})
 
-		for _, caFile := range cfg.ClientCerts {
+		for _, caFile := range c.ClientCerts {
 			// don't add cert to pool more than once
 			if _, ok := clientCertsAdded[caFile]; ok {
 				continue
@@ -294,7 +295,7 @@ func (cfg *Config) buildStandardTLSConfig() error {
 
 	// default cipher suites
 	if len(config.CipherSuites) == 0 {
-		config.CipherSuites = defaultCiphers
+		config.CipherSuites = getPreferredDefaultCiphers()
 	}
 
 	// for security, ensure TLS_FALLBACK_SCSV is always included first
@@ -303,7 +304,7 @@ func (cfg *Config) buildStandardTLSConfig() error {
 	}
 
 	// store the resulting new tls.Config
-	cfg.tlsConfig = config
+	c.tlsConfig = config
 
 	return nil
 }
@@ -380,7 +381,7 @@ func RegisterConfigGetter(serverType string, fn ConfigGetter) {
 func SetDefaultTLSParams(config *Config) {
 	// If no ciphers provided, use default list
 	if len(config.Ciphers) == 0 {
-		config.Ciphers = defaultCiphers
+		config.Ciphers = getPreferredDefaultCiphers()
 	}
 
 	// Not a cipher suite, but still important for mitigating protocol downgrade attacks
@@ -464,6 +465,35 @@ var defaultCiphers = []uint16{
 	tls.TLS_RSA_WITH_AES_128_CBC_SHA,
 }
 
+// List of ciphers we should prefer if native AESNI support is missing
+var defaultCiphersNonAESNI = []uint16{
+	tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
+	tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
+	tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
+	tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
+	tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
+	tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
+	tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
+	tls.TLS_RSA_WITH_AES_256_CBC_SHA,
+	tls.TLS_RSA_WITH_AES_128_CBC_SHA,
+}
+
+// getPreferredDefaultCiphers returns an appropriate cipher suite to use, depending on
+// the hardware support available for AES-NI.
+//
+// See https://github.com/mholt/caddy/issues/1674
+func getPreferredDefaultCiphers() []uint16 {
+	if aesnicheck.HasAESNI() {
+		return defaultCiphers
+	}
+
+	// Return a cipher suite that prefers ChaCha20
+	return defaultCiphersNonAESNI
+}
+
 // Map of supported curves
 // https://golang.org/pkg/crypto/tls/#CurveID
 var supportedCurvesMap = map[string]tls.CurveID{

caddytls/config_test.go

@@ -6,6 +6,8 @@ import (
 	"net/url"
 	"reflect"
 	"testing"
+
+	"github.com/codahale/aesnicheck"
 )
 
 func TestConvertTLSConfigProtocolVersions(t *testing.T) {
@@ -60,10 +62,11 @@ func TestConvertTLSConfigCipherSuites(t *testing.T) {
 		{Enabled: true, Ciphers: nil},
 	}
 
+	defaultCiphersExpected := getPreferredDefaultCiphers()
 	expectedCiphers := [][]uint16{
 		{tls.TLS_FALLBACK_SCSV, 0xc02c, 0xc030},
 		{tls.TLS_FALLBACK_SCSV, 0xc012, 0xc030, 0xc00a},
-		append([]uint16{tls.TLS_FALLBACK_SCSV}, defaultCiphers...),
+		append([]uint16{tls.TLS_FALLBACK_SCSV}, defaultCiphersExpected...),
 	}
 
 	for i, config := range configs {
@@ -79,6 +82,21 @@ func TestConvertTLSConfigCipherSuites(t *testing.T) {
 	}
 }
 
+func TestGetPreferredDefaultCiphers(t *testing.T) {
+	expectedCiphers := defaultCiphers
+	if !aesnicheck.HasAESNI() {
+		expectedCiphers = defaultCiphersNonAESNI
+	}
+
+	// Ensure ordering is correct and ciphers are what we expected.
+	result := getPreferredDefaultCiphers()
+	for i, actual := range result {
+		if actual != expectedCiphers[i] {
+			t.Errorf("Expected cipher in position %d to be %0x, got %0x", i, expectedCiphers[i], actual)
+		}
+	}
+}
+
 func TestStorageForNoURL(t *testing.T) {
 	c := &Config{}
 	if _, err := c.StorageFor(""); err == nil {

caddytls/maintain.go

@@ -25,6 +25,13 @@ const (
 	// RenewDurationBefore is how long before expiration to renew certificates.
 	RenewDurationBefore = (24 * time.Hour) * 30
 
+	// RenewDurationBeforeAtStartup is how long before expiration to require
+	// a renewed certificate when the process is first starting up (see #1680).
+	// A wider window between RenewDurationBefore and this value will allow
+	// Caddy to start under duress but hopefully this duration will give it
+	// enough time for the blockage to be relieved.
+	RenewDurationBeforeAtStartup = (24 * time.Hour) * 7
+
 	// OCSPInterval is how often to check if OCSP stapling needs updating.
 	OCSPInterval = 1 * time.Hour
 )
@@ -126,13 +133,17 @@ func RenewManagedCertificates(allowPrompts bool) (err error) {
 		err := cert.Config.RenewCert(renewName, allowPrompts)
 		if err != nil {
 			if allowPrompts {
-				// Certificate renewal failed and the operator is present; we should stop
-				// immediately and return the error. See a discussion in issue 642
-				// about this. For a while, we only stopped if the certificate was
-				// expired, but in reality, there is no difference between reporting
-				// it now versus later, except that there's somebody present to deal
-				// with it now, so require it.
-				return err
+				// Certificate renewal failed and the operator is present. See a discussion
+				// about this in issue 642. For a while, we only stopped if the certificate
+				// was expired, but in reality, there is no difference between reporting
+				// it now versus later, except that there's somebody present to deal with
+				// it right now.
+				timeLeft := cert.NotAfter.Sub(time.Now().UTC())
+				if timeLeft < RenewDurationBeforeAtStartup {
+					// See issue 1680. Only fail at startup if the certificate is dangerously
+					// close to expiration.
+					return err
+				}
 			}
 			log.Printf("[ERROR] %v", err)
 			if cert.Config.OnDemand {
@@ -141,7 +152,6 @@ func RenewManagedCertificates(allowPrompts bool) (err error) {
 		} else {
 			// successful renewal, so update in-memory cache by loading
 			// renewed certificate so it will be used with handshakes
-			// TODO: Not until CA has valid OCSP response ready for the new cert... sigh.
 			if cert.Names[len(cert.Names)-1] == "" {
 				// Special case: This is the default certificate. We must
 				// flush it out of the cache so that we no longer point to

caddytls/setup_test.go

@@ -58,21 +58,7 @@ func TestSetupParseBasic(t *testing.T) {
 	}
 
 	// Cipher checks
-	expectedCiphers := []uint16{
-		tls.TLS_FALLBACK_SCSV,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
-		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
-		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
-		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,
-		tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,
-		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
-		tls.TLS_RSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_RSA_WITH_AES_128_CBC_SHA,
-	}
+	expectedCiphers := append([]uint16{tls.TLS_FALLBACK_SCSV}, getPreferredDefaultCiphers()...)
 
 	// Ensure count is correct (plus one for TLS_FALLBACK_SCSV)
 	if len(cfg.Ciphers) != len(expectedCiphers) {

dist/CHANGES.txt

@@ -1,5 +1,21 @@
 CHANGES
 
+0.10.3 (May 19, 2017)
+- Several minor changes and bug fixes; see commit log for details
+- tls: Allow narrower certificate renewal window at startup (#1680)
+
+0.10.2 (May 2, 2017)
+- Hot fix for rule paths of "/" so that they match every request
+- fastcgi: Match request paths that don't start with "/" even if rule does
+
+
+0.10.1 (May 1, 2017)
+- Reduced memory usage for gzip, templates, and MITM detection
+- Fixed automatic HTTP->HTTPS redirects for sites with wildcard labels
+- proxy: Fix 'without' subdirective
+- A few other minor bug fixes and improvements
+
+
 0.10 (April 20, 2017)
 - Built on Go 1.8.1
 - HTTPS interception detection

dist/README.txt

@@ -1,10 +1,10 @@
-CADDY 0.10
+CADDY 0.10.3
 
 Website
 	https://caddyserver.com
 
-Community
-	https://forum.caddyserver.com
+Community Forum
+	https://caddy.community
 
 Twitter
 	@caddyserver

dist/automate.go

@@ -1,161 +0,0 @@
-package main
-
-import (
-	"fmt"
-	"log"
-	"os"
-	"os/exec"
-	"path/filepath"
-	"runtime"
-	"sync"
-
-	"github.com/mholt/archiver"
-)
-
-var buildScript, repoDir, mainDir, distDir, buildDir, releaseDir string
-
-func init() {
-	repoDir = filepath.Join(os.Getenv("GOPATH"), "src", "github.com", "mholt", "caddy")
-	mainDir = filepath.Join(repoDir, "caddy")
-	buildScript = filepath.Join(mainDir, "build.bash")
-	distDir = filepath.Join(repoDir, "dist")
-	buildDir = filepath.Join(distDir, "builds")
-	releaseDir = filepath.Join(distDir, "release")
-}
-
-func main() {
-	// First, clean up
-	err := os.RemoveAll(buildDir)
-	if err != nil {
-		log.Fatal(err)
-	}
-	err = os.RemoveAll(releaseDir)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Then set up
-	err = os.MkdirAll(buildDir, 0755)
-	if err != nil {
-		log.Fatal(err)
-	}
-	err = os.MkdirAll(releaseDir, 0755)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Perform builds and make archives in parallel; only as many
-	// goroutines as we have processors.
-	var wg sync.WaitGroup
-	var throttle = make(chan struct{}, numProcs())
-	for _, p := range platforms {
-		wg.Add(1)
-		throttle <- struct{}{}
-
-		if p.os == "" || p.arch == "" || p.archive == "" {
-			log.Fatalf("Platform OS, architecture, and archive format is required: %+v", p)
-		}
-
-		go func(p platform) {
-			defer wg.Done()
-			defer func() { <-throttle }()
-
-			fmt.Printf("== Building %s\n", p)
-
-			var baseFilename, binFilename string
-			baseFilename = fmt.Sprintf("caddy_%s_%s", p.os, p.arch)
-			if p.arch == "arm" {
-				baseFilename += p.arm
-			}
-			binFilename = baseFilename + p.binExt
-
-			binPath := filepath.Join(buildDir, binFilename)
-			archive := filepath.Join(releaseDir, fmt.Sprintf("%s.%s", baseFilename, p.archive))
-			archiveContents := append(distContents, binPath)
-
-			err := build(p, binPath)
-			if err != nil {
-				log.Fatal(err)
-			}
-
-			fmt.Printf("== Compressing %s\n", baseFilename)
-
-			if p.archive == "zip" {
-				err := archiver.Zip.Make(archive, archiveContents)
-				if err != nil {
-					log.Fatal(err)
-				}
-			} else if p.archive == "tar.gz" {
-				err := archiver.TarGz.Make(archive, archiveContents)
-				if err != nil {
-					log.Fatal(err)
-				}
-			}
-		}(p)
-	}
-
-	wg.Wait()
-}
-
-func build(p platform, out string) error {
-	cmd := exec.Command(buildScript, out)
-	cmd.Dir = mainDir
-	cmd.Env = os.Environ()
-	cmd.Env = append(cmd.Env, "CGO_ENABLED=0")
-	cmd.Env = append(cmd.Env, "GOOS="+p.os)
-	cmd.Env = append(cmd.Env, "GOARCH="+p.arch)
-	cmd.Env = append(cmd.Env, "GOARM="+p.arm)
-	cmd.Stderr = os.Stderr
-	return cmd.Run()
-}
-
-type platform struct {
-	os, arch, arm, binExt, archive string
-}
-
-func (p platform) String() string {
-	outStr := fmt.Sprintf("%s/%s", p.os, p.arch)
-	if p.arch == "arm" {
-		outStr += fmt.Sprintf(" (ARM v%s)", p.arm)
-	}
-	return outStr
-}
-
-func numProcs() int {
-	n := runtime.GOMAXPROCS(0)
-	if n == runtime.NumCPU() && n > 1 {
-		n--
-	}
-	return n
-}
-
-// See: https://golang.org/doc/install/source#environment
-// Not all supported platforms are listed since some are
-// problematic and we only build the most common ones.
-// These are just the pre-made, readily-available static
-// builds, and we can try to add more upon request if there
-// is enough demand.
-var platforms = []platform{
-	{os: "darwin", arch: "amd64", archive: "zip"},
-	{os: "freebsd", arch: "386", archive: "tar.gz"},
-	{os: "freebsd", arch: "amd64", archive: "tar.gz"},
-	{os: "freebsd", arch: "arm", arm: "7", archive: "tar.gz"},
-	{os: "linux", arch: "386", archive: "tar.gz"},
-	{os: "linux", arch: "amd64", archive: "tar.gz"},
-	{os: "linux", arch: "arm", arm: "7", archive: "tar.gz"},
-	{os: "linux", arch: "arm64", archive: "tar.gz"},
-	{os: "netbsd", arch: "386", archive: "tar.gz"},
-	{os: "netbsd", arch: "amd64", archive: "tar.gz"},
-	{os: "openbsd", arch: "386", archive: "tar.gz"},
-	{os: "openbsd", arch: "amd64", archive: "tar.gz"},
-	{os: "solaris", arch: "amd64", archive: "tar.gz"},
-	{os: "windows", arch: "386", binExt: ".exe", archive: "zip"},
-	{os: "windows", arch: "amd64", binExt: ".exe", archive: "zip"},
-}
-
-var distContents = []string{
-	filepath.Join(distDir, "init"),
-	filepath.Join(distDir, "CHANGES.txt"),
-	filepath.Join(distDir, "LICENSES.txt"),
-	filepath.Join(distDir, "README.txt"),
-}

dist/automate_test.go

@@ -1,15 +0,0 @@
-package main
-
-import (
-	"runtime"
-	"testing"
-)
-
-func TestNumProcs(t *testing.T) {
-	num := runtime.NumCPU()
-	n := numProcs()
-	if n > num || n < 1 {
-		t.Errorf("Expected numProcs() to return max(NumCPU-1, 1) or at least some "+
-			"reasonable value (depending on CI environment), but got n=%d (NumCPU=%d)", n, num)
-	}
-}

dist/init/linux-systemd/README.md

@@ -7,8 +7,7 @@ the username of whoever touched the file most recently, for example
 `@wmark re systemd: …`.
 
 The provided file should work with systemd version 219 or later. It might work with earlier versions.
-The easiest way to check your systemd version is to look at the version of the installed package
-(e.g. 'sudo yum info systemd' on RedHat/Fedora systems).
+The easiest way to check your systemd version is to run `systemctl --version`.
 
 ## Instructions
 

dist/init/mac-launchd/README.md

@@ -1,12 +1,42 @@
 launchd service for macOS
 =========================
 
-This is a sample file for a *launchd* service on Mac.
-Edit the paths and email in the plist file to match your info.
+This is a working sample file for a *launchd* service on Mac, which should be placed here:
+
+```bash
+/Library/LaunchDaemons/com.caddyserver.web.plist
+```
+
+To create the proper directories as used in the example file:
+
+```bash
+sudo mkdir -p /etc/caddy /etc/ssl/caddy /var/log/caddy /usr/local/bin /var/tmp /srv/www/localhost
+sudo touch /etc/caddy/Caddyfile
+sudo chown root:wheel -R /usr/local/bin/caddy /Library/LaunchDaemons/
+sudo chown _www:_www -R /etc/caddy /etc/ssl/caddy /var/log/caddy
+sudo chmod 0750 /etc/ssl/caddy
+```
+
+Create a simple web page and Caddyfile
+
+```bash
+sudo bash -c 'echo "Hello, World!" > /srv/www/localhost/index.html'
+sudo bash -c 'echo "http://localhost {
+    root /srv/www/localhost
+}" >> /etc/caddy/Caddyfile'
+```
 
 Start and Stop the Caddy launchd service using the following commands:
 
-    $ launchctl load ~/Library/LaunchAgents/com.caddyserver.web.plist
-    $ launchctl unload ~/Library/LaunchAgents/com.caddyserver.web.plist
+```bash
+launchctl load /Library/LaunchDaemons/com.caddyserver.web.plist
+launchctl unload /Library/LaunchDaemons/com.caddyserver.web.plist
+```
 
-More information can be found in this blogpost: [Running Caddy as a service on macOS X server](https://denbeke.be/blog/software/running-caddy-as-a-service-on-macos-os-x-server/)
+To start on every boot use the `-w` flag (to write):
+
+```bash
+launchctl load -w /Library/LaunchAgents/com.caddyserver.web.plist
+```
+
+More information can be found in this blogpost: [Running Caddy as a service on macOS X server](https://denbeke.be/blog/software/running-caddy-as-a-service-on-macos-os-x-server/)

dist/init/mac-launchd/com.caddyserver.web.plist

@@ -1,31 +1,53 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 <plist version="1.0">
-    <dict>
-        <key>Label</key>
-        <string>com.caddyserver.web</string>
-        <key>EnvironmentVariables</key>
-        <dict>
-            <key>HOME</key>
-            <string>/Users/mathias</string>
-        </dict>
-        <key>ProgramArguments</key>
-        <array>
-            <string>sh</string>
-            <string>-c</string>
-            <string>ulimit -n 8192; cd /Users/mathias/Sites; ./caddy -agree -email my_email@domain.com -conf=/Users/mathias/Sites/Caddyfile</string>
-        </array>
-        <key>UserName</key>
-        <string>www</string>
-        <key>RunAtLoad</key>
-        <true/>
-        <key>KeepAlive</key>
-        <true/>
-        <key>WorkingDirectory</key>
-        <string>/Users/mathias/Sites</string>
-        <key>StandardOutPath</key>
-        <string>/Users/mathias/Sites/caddy.log</string>
-        <key>StandardErrorPath</key>
-        <string>/Users/mathias/Sites/caddy_error.log</string>
-    </dict>
-</plist>
+<dict>
+	<key>Label</key>
+	<string>Caddy</string>
+	<key>ProgramArguments</key>
+	<array>
+		<string>/usr/local/bin/caddy</string>
+		<string>-agree</string>
+		<string>-conf</string>
+		<string>/etc/caddy/Caddyfile</string>
+		<string>-root</string>
+		<string>/var/tmp</string>
+	</array>
+	<key>EnvironmentVariables</key>
+	<dict>
+		<key>CADDYPATH</key>
+		<string>/etc/ssl/caddy</string>
+	</dict>
+
+	<key>UserName</key>
+	<string>root</string>
+	<key>GroupName</key>
+	<string>wheel</string>
+	<key>InitGroups</key>
+	<true/>
+
+	<key>RunAtLoad</key>
+	<true/>
+	<key>KeepAlive</key>
+	<dict>
+		<key>Crashed</key>
+		<true/>
+	</dict>
+
+	<key>SoftResourceLimits</key>
+	<dict>
+		<key>NumberOfFiles</key>
+		<integer>8192</integer>
+	</dict>
+	<key>HardResourceLimits</key>
+	<dict/>
+
+	<key>WorkingDirectory</key>
+	<string>/etc/ssl/caddy</string>
+
+	<key>StandardErrorPath</key>
+	<string>/var/log/caddy/error.log</string>
+	<key>StandardOutPath</key>
+	<string>/var/log/caddy/info.log</string>
+</dict>
+</plist>

plugins.go

@@ -217,6 +217,7 @@ func RegisterPlugin(name string, plugin Plugin) {
 // EventName represents the name of an event used with event hooks.
 type EventName string
 
+// Define the event names for the startup and shutdown events
 const (
 	StartupEvent  EventName = "startup"
 	ShutdownEvent EventName = "shutdown"