Update for version 0.10.11

* Allow Response Headers in logs

* Remove log line

* remove unneeded log import

* Check if rr is nil.  Added test to check

* merge if statements

* remove temp file

.github/CONTRIBUTING.md

@@ -0,0 +1,160 @@
+Contributing to Caddy
+=====================
+
+Welcome! Thank you for choosing to be a part of our community. Caddy wouldn't be great without your involvement!
+
+For starters, we invite you to join [the Caddy forum](https://caddy.community) where you can hang out with other Caddy users and developers.
+
+## Common Tasks
+
+- [Contributing code](#contributing-code)
+- [Writing a plugin](#writing-a-plugin)
+- [Asking or answering questions for help using Caddy](#getting-help-using-caddy)
+- [Reporting a bug](#reporting-bugs)
+- [Suggesting an enhancement or a new feature](#suggesting-features)
+- [Improving documentation](#improving-documentation)
+
+Other menu items:
+
+- [Values](#values)
+- [Responsible Disclosure](#responsible-disclosure)
+- [Thank You](#thank-you)
+
+
+### Contributing code
+
+You can have a direct impact on the project by helping with its code. To contribute code to Caddy, open a [pull request](https://github.com/mholt/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with Caddy's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/mholt/caddy/-/search).
+
+We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :wink: If your change is on the right track, we can guide you to make it mergable.
+
+Here are some of the expectations we have of contributors:
+
+- If your change is more than just a minor alteration, **open an issue to propose your change first.** This way we can avoid confusion, coordinate what everyone is working on, and ensure that changes are in-line with the project's goals and the best interests of its users. If there's already an issue about it, comment on the existing issue to claim it.
+
+- **Keep pull requests small.** Smaller PRs are more likely to be merged because they are easier to review! We might ask you to break up large PRs into smaller ones. [An example of what we DON'T do.](https://twitter.com/iamdevloper/status/397664295875805184)
+
+- **Keep related commits together in a PR.** We do want pull requests to be small, but you should also keep multiple related commits in the same PR if they rely on each other.
+
+- **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass.
+
+- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks or profiling.
+
+- **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance. Interactive rebase can do this, or a simpler way is `git reset --soft <diverging-commit>` then `git commit -s`.
+
+- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors help maintain their change after it is merged.
+
+- **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious.
+
+We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base!
+
+
+#### HOW TO MAKE A PULL REQUEST TO CADDY
+
+Contributing to Go projects on GitHub is fun and easy. We recommend the following workflow:
+
+1. [Fork this repo](https://github.com/mholt/caddy). This makes a copy of the code you can write to.
+
+2. If you don't already have this repo (mholt/caddy.git) repo on your computer, get it with `go get github.com/mholt/caddy/caddy`.
+
+3. Tell git that it can push the mholt/caddy.git repo to your fork by adding a remote: `git remote add myfork https://github.com/you/caddy.git`
+
+4. Make your changes in the mholt/caddy.git repo on your computer.
+
+5. Push your changes to your fork: `git push myfork`
+
+6. [Create a pull request](https://github.com/mholt/caddy/pull/new/master) to merge your changes into mholt/caddy @ master. (Click "compare across forks" and change the head fork.)
+
+This workflow is nice because you don't have to change import paths. You can get fancier by using different branches if you want.
+
+
+### Writing a plugin
+
+Caddy can do more with plugins! Anyone can write a plugin. Plugins are Go libraries that get compiled into Caddy, extending its feature set. They can add directives to the Caddyfile, change how the Caddyfile is loaded, and even implement new server types (e.g. HTTP, DNS). When it's ready, you can submit your plugin to the Caddy website so others can download it.
+
+[Learn how to write and submit a plugin](https://github.com/mholt/caddy/wiki) on the wiki. You should also share and discuss your plugin idea [on the forums](https://caddy.community) to have people test it out. We don't use the Caddy issue tracker for plugins.
+
+
+### Getting help using Caddy
+
+If you have a question about using Caddy, [ask on our forum](https://caddy.community)! There will be more people there who can help you than just the Caddy developers who follow our issue tracker. Issues are not the place for usage questions.
+
+Many people on the forums could benefit from your experience and expertise, too. Once you've been helped, consider giving back by answering other people's questions and participating in other discussions.
+
+
+### Reporting bugs
+
+Like every software, Caddy has its flaws. If you find one, [search the issues](https://github.com/mholt/caddy/issues) to see if it has already been reported. If not, [open a new issue](https://github.com/mholt/caddy/issues/new) and describe the bug, and somebody will look into it! (This repository is only for Caddy, not plugins.)
+
+**You can help stop bugs in their tracks!** Speed up the patch by identifying the bug in the code. This can sometimes be done by adding `fmt.Println()` statements (or similar) in relevant code paths to narrow down where the problem may be. It's a good way to [introduce yourself to the Go language](https://tour.golang.org), too.
+
+Please follow the issue template so we have all the needed information. Unredacted&mdash;yes, actual values matter. We need to be able to repeat the bug using your instructions. Please simplify the issue as much as possible. The burden is on you to convince us that it is actually a bug in Caddy. This is easiest to do when you write clear, concise instructions so we can reproduce the behavior (even if it seems obvious). The more detailed and specific you are, the faster we will be able to help you!
+
+We suggest reading [How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
+
+Please be kind. :smile: Remember that Caddy comes at no cost to you, and you're getting free support when we fix your issues. If we helped you, please consider helping someone else!
+
+
+### Suggesting features
+
+First, [search to see if your feature has already been requested](https://github.com/mholt/caddy/issues). If it has, you can add a :+1: reaction to vote for it. If your feature idea is new, open an issue to request the feature. You don't have to follow the bug template for feature requests. Please describe your idea thoroughly so that we know how to implement it! Really vague requests may not be helpful or actionable and without clarification will have to be closed.
+
+While we really do value your requests and implement many of them, not all features are a good fit for Caddy. Most of those [make good plugins](https://github.com/mholt/caddy/wiki), though, which can be made by anyone! But if a feature is not in the best interest of the Caddy project or its users in general, we may politely decline to implement it into Caddy core.
+
+
+### Improving documentation
+
+Caddy's documentation is available at [https://caddyserver.com/docs](https://caddyserver.com/docs). If you would like to make a fix to the docs, feel free to contribute at the [caddyserver/website](https://github.com/caddyserver/website) repository!
+
+Note that plugin documentation is not hosted by the Caddy website, other than basic usage examples. They are managed by the individual plugin authors, and you will have to contact them to change their documentation.
+
+
+
+## Collaborator Instructions
+
+Collabators have push rights to the repository. We grant this permission after one or more successful, high-quality PRs are merged! We thank them for their help.The expectations we have of collaborators are:
+
+- **Help review pull requests.** Be meticulous, but also kind. We love our contributors, but we critique the contribution to make it better. Multiple, thorough reviews make for the best contributions! Here are some questions to consider:
+	- Can the change be made more elegant?
+	- Is this a maintenance burden?
+	- What assumptions does the code make?
+	- Is it well-tested?
+	- Is the change a good fit for the project?
+	- Does it actually fix the problem or is it creating a special case instead?
+	- Does the change incur any new dependencies? (Avoid these!)
+
+- **Answer issues.** If every collaborator helped out with issues, we could count the number of open issues on two hands. This means getting involved in the discussion, investigating the code, and yes, debugging it. It's fun. Really! :smile: Please, please help with open issues. Granted, some issues need to be done before others. And of course some are larger than others: you don't have to do it all yourself. Work with other collaborators as a team!
+
+- **Do not merge pull requests until they have been approved by one or two other collaborators.** If a project owner approves the PR, it can be merged (as long as the conversation has finished too).
+
+- **Prefer squashed commits over a messy merge.** If there are many little commits, please [squash the commits](https://stackoverflow.com/a/11732910/1048862) so we don't clutter the commit history.
+
+- **Don't accept new dependencies lightly.** Dependencies can make the world crash and burn, but they are sometimes necessary. Choose carefully. Extremely small dependencies (a few lines of code) can be inlined. The rest may not be needed. For those that are, Caddy vendors all dependencies with the help of [gvt](https://github.com/FiloSottile/gvt). All external dependencies must be vendored, and _Caddy must not export any types defined by those dependencies_. Check this diligently!
+
+- **Be extra careful in some areas of the code.** There are some critical areas in the Caddy code base that we review extra meticulously: the `caddy` and `caddytls` packages especially.
+
+- **Make sure tests test the actual thing.** Double-check that the tests fail without the change, and pass with it. It's important that they assert what they're purported to assert.
+
+- **Recommended reading**
+	- [CodeReviewComments](https://github.com/golang/go/wiki/CodeReviewComments) for an idea of what we look for in good, clean Go code
+	- [Linus Torvalds describes a good commit message](https://gist.github.com/matthewhudson/1475276)
+	- [Best Practices for Maintainers](https://opensource.guide/best-practices/)
+	- [Shrinking Code Review](https://alexgaynor.net/2015/dec/29/shrinking-code-review/)
+
+
+
+## Values
+
+- A person is always more important than code. People don't like being handled "efficiently". But we can still process issues and pull requests efficiently while being kind, patient, and considerate.
+
+- The ends justify the means, if the means are good. A good tree won't produce bad fruit. But if we cut corners or are hasty in our process, the end result will not be good.
+
+
+## Responsible Disclosure
+
+If you've found a security vulnerability, please email me, the author, directly: Matthew dot Holt at Gmail. I'll need enough information to verify the bug and make a patch. It will speed things up if you suggest a working patch. If your report is valid and a patch is released, we will not reveal your identity by default. If you wish to be credited, please give me the name to use. Thanks for responsibly helping Caddy&mdash;and thousands of websites&mdash;be more secure!
+
+
+## Thank you
+
+Thanks for your help! Caddy would not be what it is today without your
+contributions.

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,32 @@
+<!--
+Are you asking for help with using Caddy? Please use our forum instead: https://caddy.community. If you are filing a bug report, please take a few minutes to carefully answer the following questions. If your issue is not a bug report, you do not need to use this template. Thanks!)
+-->
+
+### 1. What version of Caddy are you using (`caddy -version`)?
+
+
+### 2. What are you trying to do?
+
+
+### 3. What is your entire Caddyfile?
+```text
+(paste Caddyfile here)
+```
+
+### 4. How did you run Caddy (give the full command and describe the execution environment)?
+
+
+### 5. Please paste any relevant HTTP request(s) here.
+
+<!-- Paste curl command, or full HTTP request including headers and body, here. -->
+
+
+### 6. What did you expect to see?
+
+
+### 7. What did you see instead (give full error messages and/or log)?
+
+
+### 8. How can someone who is starting from scratch reproduce the bug as minimally as possible?
+
+<!-- Please strip away any extra infrastructure such as containers, reverse proxies, upstream apps, caches, dependencies, etc, to prove this is a bug in Caddy and not an external misconfiguration. Your chances of getting this bug fixed go way up the easier it is to replicate. Thank you! -->

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,19 @@
+<!--
+Thank you for contributing to Caddy! Please fill this out to help us make the most of your pull request.
+-->
+
+### 1. What does this change do, exactly?
+
+
+### 2. Please link to the relevant issues.
+
+
+### 3. Which documentation changes (if any) need to be made because of this PR?
+
+
+### 4. Checklist
+
+- [ ] I have written tests and verified that they fail without my change
+- [ ] I have squashed any insignificant commits
+- [ ] This change has comments for package types, values, functions, and non-obvious lines of code
+- [ ] I am willing to help maintain this change if there are issues with it later

.gitignore

@@ -3,6 +3,7 @@ Thumbs.db
 _gitignore/
 Vagrantfile
 .vagrant/
+/.idea
 
 dist/builds/
 dist/release/
@@ -13,4 +14,8 @@ access.log
 /*.conf
 Caddyfile
 
-og_static/
+og_static/
+
+.vscode/
+
+*.bat

.travis.yml

@@ -1,11 +1,17 @@
 language: go
 
+addons:
+  hosts:
+    - quic.clemente.io
+
 go:
-  - 1.6.2
+  - 1.x
   - tip
 
-env:
-  - CGO_ENABLED=0
+matrix:
+  allow_failures:
+    - go: tip
+  fast_finish: true
 
 before_install:
   # Decrypts a script that installs an authenticated cookie
@@ -18,13 +24,15 @@ install:
   - if [ "$TRAVIS_PULL_REQUEST" = "false" ]; then bash dist/gitcookie.sh; fi
   - go get -t ./...
   - go get github.com/golang/lint/golint
-  - go get github.com/gordonklaus/ineffassign
+  - go get github.com/FiloSottile/vendorcheck
+  # Install gometalinter
+  - go get github.com/alecthomas/gometalinter
 
 script:
-  - diff <(echo -n) <(gofmt -s -d .)
-  - ineffassign .
-  - go vet ./...
-  - go test ./...
+  - gometalinter --install
+  - gometalinter --disable-all -E vet -E gofmt -E misspell -E ineffassign -E goimports -E deadcode --tests --vendor ./...
+  - vendorcheck ./...
+  - go test -race ./...
 
 after_script:
   - golint ./...

CONTRIBUTING.md

@@ -1,105 +0,0 @@
-## Contributing to Caddy
-
-Welcome! Our community focuses on helping others and making Caddy the best it
-can be. We gladly accept contributions and encourage you to get involved!
-
-
-### Join us in the forum
-
-The [Caddy forum](https://forum.caddyserver.com) is the place for all discussion
-that doesn't belong in issues or pull requests. Feel free to participate with us!
-
-If you want to file a bug report or make an improvement to Caddy, however, you
-should submit an issue or pull request.
-
-
-### Bug reports
-
-Please [search this repository](https://github.com/mholt/caddy/search?q=&type=Issues&utf8=%E2%9C%93)
-with a variety of keywords to ensure your bug is not already reported.
-
-If unique, [open an issue](https://github.com/mholt/caddy/issues) and answer the
-questions so we can understand and reproduce the problematic behavior.
-
-The burden is on you to convince us that it is actually a bug in Caddy. This is
-easiest to do when you write clear, concise instructions so we can reproduce
-the behavior (even if it seems obvious). The more detailed and specific you are,
-the faster we will be able to help you. Check out
-[How to Report Bugs Effectively](http://www.chiark.greenend.org.uk/~sgtatham/bugs.html).
-
-Please be kind. :smile: Remember that Caddy comes at no cost to you, and you're
-getting free help. If we helped you, please consider
-[donating](https://caddyserver.com/donate) - it keeps us motivated!
-
-
-### Minor improvements and new tests
-
-Submit [pull requests](https://github.com/mholt/caddy/pulls) at any time for
-minor changes or new tests. Make sure to write tests to assert your change is
-working properly and is thoroughly covered. We'll ask most pull requests to be
-[squashed](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html),
-especially with small commits.
-
-Your pull request may be thoroughly reviewed. This is because if we accept the
-PR, we also assume responsibility for it, although we would prefer you to
-help maintain your code after it gets merged. 
-
-
-### Proposals, suggestions, ideas, new features
-
-First, please [search](https://github.com/mholt/caddy/search?q=&type=Issues&utf8=%E2%9C%93)
-with a variety of keywords to ensure your suggestion/proposal is new.
-
-If so, you may open either an issue or a pull request for discussion and
-feedback.
-
-The advantage of issues is that you don't have to spend time implementing your
-idea, but you should still describe it thoroughly as if someone reading it would
-implement the whole thing starting from scratch.
-
-The advantage of pull requests is that we can immediately see the impact the
-change will have on the project, what the code will look like, and how to
-improve it. The disadvantage of pull requests is that they are unlikely to get
-accepted without significant changes first, or it may be rejected entirely.
-Don't worry, that won't happen without an open discussion first.
-
-If you are going to spend significant time writing code for a new pull request,
-best to open an issue to "claim" it and get feedback before you invest a lot of
-time.
-
-Remember: pull requests should always be thoroughly documented both via godoc
-and with at least a rough draft of documentation that might go on the website
-for users to read.
-
-
-### Collaborator status
-
-If your pull request is merged, congratulations! You're technically a
-collaborator. We may also grant you "Collaborator status" which means you can
-push to the repository and merge other pull requests. We hope that you will
-stay involved by reviewing pull requests, submitting more of your own, and
-resolving issues as you are able to. Thanks for making Caddy amazing!
-
-We ask that collaborators will conduct thorough code reviews and be nice to
-new contributors. Before merging a PR, it's best to get the approval of
-at least one or two other collaborators and/or the project owner. We prefer
-squashed commits instead of many little, semantically-unimportant commits. Also,
-CI and other post-commit hooks must pass before being merged except in certain
-unusual circumstances.
-
-Collaborator status may be removed for inactive users from time to time as
-we see fit; this is not an insult, just a basic security precaution in case
-the account becomes inactive or abandoned. Privileges can always be restored
-later.
-
-
-### Vulnerabilities
-
-If you've found a vulnerability that is serious, please email me: Matthew dot 
-Holt at Gmail. If it's not a big deal, a pull request will probably be faster.
-
-
-## Thank you
-
-Thanks for your help! Caddy would not be what it is today without your
-contributions.

ISSUE_TEMPLATE

@@ -1,24 +0,0 @@
-(Are you asking for help with using Caddy? Please use our forum instead: https://forum.caddyserver.com. If you are filing a bug report, please answer the following questions. If your issue is not a bug report, you do not need to use this template. Either way, please consider donating if we've helped you. Thanks!)
-
-#### 1. What version of Caddy are you running (`caddy -version`)?
-
-
-#### 2. What are you trying to do?
-
-
-#### 3. What is your entire Caddyfile?
-```text
-(Put Caddyfile here)
-```
-
-#### 4. How did you run Caddy (give the full command and describe the execution environment)?
-
-
-#### 5. What did you expect to see?
-
-
-#### 6. What did you see instead (give full error messages and/or log)?
-
-
-#### 7. How can someone who is starting from scratch reproduce this behavior as minimally as possible?
-

README.md

@@ -1,65 +1,104 @@
-<a href="https://caddyserver.com"><img src="https://caddyserver.com/resources/images/caddy-lower.png" alt="Caddy" width="350"></a>
+<p align="center">
+	<a href="https://caddyserver.com"><img src="https://user-images.githubusercontent.com/1128849/36338535-05fb646a-136f-11e8-987b-e6901e717d5a.png" alt="Caddy" width="450"></a>
+</p>
+<h3 align="center">Every Site on HTTPS <!-- Serve Confidently --></h3>
+<p align="center">Caddy is a general-purpose HTTP/2 web server that serves HTTPS by default.</p>
+<p align="center">
+	<a href="https://travis-ci.org/mholt/caddy"><img src="https://img.shields.io/travis/mholt/caddy.svg?label=linux+build"></a>
+	<a href="https://ci.appveyor.com/project/mholt/caddy"><img src="https://img.shields.io/appveyor/ci/mholt/caddy.svg?label=windows+build"></a>
+	<a href="https://godoc.org/github.com/mholt/caddy"><img src="https://img.shields.io/badge/godoc-reference-blue.svg"></a>
+	<a href="https://goreportcard.com/report/mholt/caddy"><img src="https://goreportcard.com/badge/github.com/mholt/caddy"></a>
+	<br>
+	<a href="https://twitter.com/caddyserver" title="@caddyserver on Twitter"><img src="https://img.shields.io/badge/twitter-@caddyserver-55acee.svg" alt="@caddyserver on Twitter"></a>
+	<a href="https://caddy.community" title="Caddy Forum"><img src="https://img.shields.io/badge/community-forum-ff69b4.svg" alt="Caddy Forum"></a>
+	<a href="https://sourcegraph.com/github.com/mholt/caddy?badge" title="Caddy on Sourcegraph"><img src="https://sourcegraph.com/github.com/mholt/caddy/-/badge.svg" alt="Caddy on Sourcegraph"></a>
+</p>
+<p align="center">
+	<a href="https://caddyserver.com/download">Download</a> ·
+	<a href="https://caddyserver.com/docs">Documentation</a> ·
+	<a href="https://caddy.community">Community</a>
+</p>
 
-[![community](https://img.shields.io/badge/community-forum-ff69b4.svg?style=flat-square)](https://forum.caddyserver.com) [![twitter](https://img.shields.io/badge/twitter-@caddyserver-55acee.svg?style=flat-square)](https://twitter.com/caddyserver) [![Documentation](https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square)](https://godoc.org/github.com/mholt/caddy) [![Linux Build Status](https://img.shields.io/travis/mholt/caddy.svg?style=flat-square&label=linux+build)](https://travis-ci.org/mholt/caddy) [![Windows Build Status](https://img.shields.io/appveyor/ci/mholt/caddy.svg?style=flat-square&label=windows+build)](https://ci.appveyor.com/project/mholt/caddy)
+---
 
-Caddy is a general-purpose web server for Windows, Mac, Linux, BSD, and
-[Android](https://github.com/mholt/caddy/wiki/Running-Caddy-on-Android). It is
-a capable but easier alternative to other popular web servers.
-
-[Releases](https://github.com/mholt/caddy/releases) · 
-[User Guide](https://caddyserver.com/docs) · 
-[Community](https://forum.caddyserver.com)
+Caddy is a **production-ready** open-source web server that is fast, easy to use, and makes you more productive.
 
+Available for Windows, Mac, Linux, BSD, Solaris, and [Android](https://github.com/mholt/caddy/wiki/Running-Caddy-on-Android).
 
 ## Menu
 
 - [Features](#features)
+- [Install](#install)
 - [Quick Start](#quick-start)
-- [Running from Source](#running-from-source)
 - [Running in Production](#running-in-production)
 - [Contributing](#contributing)
+- [Donors](#donors)
 - [About the Project](#about-the-project)
 
-
-
 ## Features
 
-- **Easy configuration** with Caddyfile
-- **Automatic HTTPS** via [Let's Encrypt](https://letsencrypt.org); Caddy
-obtains and manages all cryptographic assets for you
-- **HTTP/2** enabled by default (powered by Go standard library)
-- **Virtual hosting** for hundreds of sites per server instance, including TLS
-SNI
-- Experimental **QUIC support** for those that like speed
+- **Easy configuration** with the Caddyfile
+- **Automatic HTTPS** on by default (via [Let's Encrypt](https://letsencrypt.org))
+- **HTTP/2** by default
+- **Virtual hosting** so multiple sites just work
+- Experimental **QUIC support** for cutting-edge transmissions
 - TLS session ticket **key rotation** for more secure connections
-- **Brilliant extensibility** so Caddy can be customized for your needs
+- **Extensible with plugins** because a convenient web server is a helpful one
 - **Runs anywhere** with **no external dependencies** (not even libc)
 
+[See a more complete list of features built into Caddy.](https://caddyserver.com/features) On top of all those, Caddy does even more with plugins: choose which plugins you want at [download](https://caddyserver.com/download).
+
+Altogether, Caddy can do things other web servers simply cannot do. Its features and plugins save you time and mistakes, and will cheer you up. Your Caddy instance takes care of the details for you!
+
+
+## Install
+
+Caddy binaries have no dependencies and are available for every platform. Get Caddy either of these ways:
+
+- **[Download page](https://caddyserver.com/download)** (RECOMMENDED) allows you to customize your build in the browser
+- **[Latest release](https://github.com/mholt/caddy/releases/latest)** for pre-built, vanilla binaries
+
+
+## Build
+
+To build from source you need **[Git](https://git-scm.com/downloads)** and **[Go](https://golang.org/doc/install)** (1.9 or newer). Follow these instruction for fast building:
+
+- Get the source with `go get github.com/mholt/caddy/caddy` and then run `go get github.com/caddyserver/builds`
+- Now `cd $GOPATH/src/github.com/mholt/caddy/caddy` and run `go run build.go`
+
+Then make sure the `caddy` binary is in your PATH.
+
+To build for other platforms, use build.go with the `--goos` and `--goarch` flags.
 
 
 ## Quick Start
 
-Caddy binaries have no dependencies and are available for every platform.
-Install Caddy any one of these ways:
+To serve static files from the current working directory, run:
 
-- **[Download page](https://caddyserver.com/download)** allows you to
-customize your build in the browser
-- **[Latest release](https://github.com/mholt/caddy/releases/latest)** for
-pre-built binaries
-- **curl [getcaddy.com](https://getcaddy.com)** for auto install:
-`curl https://getcaddy.com | bash`
+```
+caddy
+```
 
-Once `caddy` is in your PATH, you can `cd` to your website's folder and run
-`caddy` to serve it. By default, Caddy serves the current directory at 
-[localhost:2015](http://localhost:2015).
+Caddy's default port is 2015, so open your browser to [http://localhost:2015](http://localhost:2015).
 
-To customize how your site is served, create a file named Caddyfile by your
-site and paste this into it:
+### Go from 0 to HTTPS in 5 seconds
+
+If the `caddy` binary has permission to bind to low ports and your domain name's DNS records point to the machine you're on:
+
+```
+caddy -host example.com
+```
+
+This command serves static files from the current directory over HTTPS. Certificates are automatically obtained and renewed for you! Caddy is also automatically configuring ports 80 and 443 for you, and redirecting HTTP to HTTPS. Cool, huh?
+
+### Customizing your site
+
+To customize how your site is served, create a file named Caddyfile by your site and paste this into it:
 
 ```plain
 localhost
 
-gzip
+push
 browse
 websocket /echo cat
 ext    .html
@@ -68,91 +107,61 @@ proxy  /api 127.0.0.1:7005
 header /api Access-Control-Allow-Origin *
 ```
 
-When you run `caddy` in that directory, it will automatically find and use
-that Caddyfile to configure itself.
+When you run `caddy` in that directory, it will automatically find and use that Caddyfile.
 
-This simple file enables compression, allows directory browsing (for folders 
-without an index file), hosts a WebSocket echo server at /echo, serves clean
-URLs, logs requests to access.log, proxies all API requests to a backend on
-port 7005, and adds the coveted  `Access-Control-Allow-Origin: *` header for
-all responses from the API.
+This simple file enables server push (via Link headers), allows directory browsing (for folders without an index file), hosts a WebSocket echo server at /echo, serves clean URLs, logs requests to an access log, proxies all API requests to a backend on port 7005, and adds the coveted  `Access-Control-Allow-Origin: *` header for all responses from the API.
 
 Wow! Caddy can do a lot with just a few lines.
 
-To host multiple sites and do more with the Caddyfile, please see the
-[Caddyfile documentation](https://caddyserver.com/docs/caddyfile).
+### Doing more with Caddy
 
-Note that production sites (not localhost) are served over
-[HTTPS by default](https://caddyserver.com/docs/automatic-https).
+To host multiple sites and do more with the Caddyfile, please see the [Caddyfile tutorial](https://caddyserver.com/tutorial/caddyfile).
 
-Caddy has a command line interface. Run `caddy -h` to view basic help or see
-the [CLI documentation](https://caddyserver.com/docs/cli) for details.
-
-**Running as root:** We advise against this. You can still listen on ports
-< 1024 using setcap like so: `sudo setcap cap_net_bind_service=+ep ./caddy`
-
-
-
-## Running from Source
-
-Note: You will need **[Go 1.6](https://golang.org/dl/)** or newer.
-
-1. `go get github.com/mholt/caddy`
-2. `cd` into your website's directory
-3. Run `caddy` (assumes `$GOPATH/bin` is in your `$PATH`)
-
-Caddy's `main()` is in the caddy subfolder. To recompile Caddy, use
-`build.bash` found in that folder.
+Sites with qualifying hostnames are served over [HTTPS by default](https://caddyserver.com/docs/automatic-https).
 
+Caddy has a nice little command line interface. Run `caddy -h` to view basic help or see the [CLI documentation](https://caddyserver.com/docs/cli) for details.
 
 
 ## Running in Production
 
-The Caddy project does not officially maintain any system-specific
-integrations, but your download file includes
-[unofficial resources](https://github.com/mholt/caddy/tree/master/dist/init)
-contributed by the community that you may find helpful for running Caddy in
-production.
+Caddy is production-ready if you find it to be a good fit for your site and workflow.
 
-How you choose to run Caddy is up to you. Many users are satisfied with
-`nohup caddy &`. Others use `screen`. Users who need Caddy to come back up
-after reboots either do so in the script that caused the reboot, add a command
-to an init script, or configure a service with their OS.
+**Running as root:** We advise against this. You can still listen on ports < 1024 on Linux using setcap like so: `sudo setcap cap_net_bind_service=+ep ./caddy`
 
+The Caddy project does not officially maintain any system-specific integrations nor suggest how to administer your own system. But your download file includes [unofficial resources](https://github.com/mholt/caddy/tree/master/dist/init) contributed by the community that you may find helpful for running Caddy in production.
+
+How you choose to run Caddy is up to you. Many users are satisfied with `nohup caddy &`. Others use `screen`. Users who need Caddy to come back up after reboots either do so in the script that caused the reboot, add a command to an init script, or configure a service with their OS.
+
+If you have questions or concerns about Caddy' underlying crypto implementations, consult Go's [crypto packages](https://golang.org/pkg/crypto), starting with their documentation, then issues, then the code itself; as Caddy uses mainly those libraries.
 
 
 ## Contributing
 
-**[Join our community](https://forum.caddyserver.com) where you can chat with
-other Caddy users and developers!**
+**[Join our forum](https://caddy.community) where you can chat with other Caddy users and developers!** To get familiar with the code base, try [Caddy code search on Sourcegraph](https://sourcegraph.com/github.com/mholt/caddy/-/search)!
 
-Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/CONTRIBUTING.md)
-and check out the [developer wiki](https://github.com/mholt/caddy/wiki).
+Please see our [contributing guidelines](https://github.com/mholt/caddy/blob/master/.github/CONTRIBUTING.md) for instructions. If you want to write a plugin, check out the [developer wiki](https://github.com/mholt/caddy/wiki).
 
-We use GitHub issues and pull requests only for discussing bug reports and
-the development of specific changes. We welcome all other topics on the
-[forum](https://forum.caddyserver.com)!
+We use GitHub issues and pull requests only for discussing bug reports and the development of specific changes. We welcome all other topics on the [forum](https://caddy.community)!
+
+If you want to contribute to the documentation, please submit pull requests to [caddyserver/website](https://github.com/caddyserver/website).
 
 Thanks for making Caddy -- and the Web -- better!
 
-Special thanks to
-[![DigitalOcean](https://i.imgur.com/sfGr0eY.png)](https://www.digitalocean.com)
-for hosting the Caddy project.
 
+## Donors
+
+- [DigitalOcean](https://m.do.co/c/6d7bdafccf96) is hosting the Caddy project.
+- [DNSimple](https://dnsimple.link/resolving-caddy) provides DNS services for Caddy's sites.
+- [DNS Spy](https://dnsspy.io) keeps an eye on Caddy's DNS properties.
+
+We thank them for their services. **If you want to help keep Caddy free, please [become a sponsor](https://caddyserver.com/pricing)!**
 
 
 ## About the Project
 
-Caddy was born out of the need for a "batteries-included" web server that runs
-anywhere and doesn't have to take its configuration with it. Caddy took
-inspiration from [spark](https://github.com/rif/spark),
-[nginx](https://github.com/nginx/nginx), lighttpd,
-[Websocketd](https://github.com/joewalnes/websocketd)
-and [Vagrant](https://www.vagrantup.com/),
-which provides a pleasant mixture of features from each of them.
+Caddy was born out of the need for a "batteries-included" web server that runs anywhere and doesn't have to take its configuration with it. Caddy took inspiration from [spark](https://github.com/rif/spark), [nginx](https://github.com/nginx/nginx), lighttpd,
+[Websocketd](https://github.com/joewalnes/websocketd) and [Vagrant](https://www.vagrantup.com/), which provides a pleasant mixture of features from each of them.
 
-**The name "Caddy":** The name of the software is "Caddy", not "Caddy Server"
-or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the
-Caddy web server". See [brand guidelines](https://caddyserver.com/brand).
+**The name "Caddy" is trademarked:** The name of the software is "Caddy", not "Caddy Server" or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the Caddy web server". See [brand guidelines](https://caddyserver.com/brand). Caddy is a registered trademark of Light Code Labs, LLC.
 
 *Author on Twitter: [@mholt6](https://twitter.com/mholt6)*

appveyor.yml

@@ -1,30 +1,35 @@
 version: "{build}"
 
+hosts:
+  quic.clemente.io: 127.0.0.1
+
 os: Windows Server 2012 R2
 
 clone_folder: c:\gopath\src\github.com\mholt\caddy
 
 environment:
   GOPATH: c:\gopath
-  CGO_ENABLED: 0
 
 install:
   - rmdir c:\go /s /q
-  - appveyor DownloadFile https://storage.googleapis.com/golang/go1.6.2.windows-amd64.zip
-  - 7z x go1.6.2.windows-amd64.zip -y -oC:\ > NUL
+  - appveyor DownloadFile https://storage.googleapis.com/golang/go1.9.windows-amd64.zip
+  - 7z x go1.9.windows-amd64.zip -y -oC:\ > NUL
+  - set PATH=%GOPATH%\bin;%PATH%
   - go version
   - go env
   - go get -t ./...
   - go get github.com/golang/lint/golint
-  - go get github.com/gordonklaus/ineffassign
-  - set PATH=%GOPATH%\bin;%PATH%
+  - go get github.com/FiloSottile/vendorcheck
+  # Install gometalinter
+  - go get github.com/alecthomas/gometalinter
 
 build: off
 
 test_script:
-  - go vet ./...
-  - go test ./...
-  - ineffassign .
+  - gometalinter --install
+  - gometalinter --disable-all -E vet -E gofmt -E misspell -E ineffassign -E goimports -E deadcode --tests --vendor ./...
+  - vendorcheck ./...
+  - go test -race ./...
 
 after_test:
   - golint ./...

assets.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddy
 
 import (

assets_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddy
 
 import (
@@ -11,9 +25,15 @@ func TestAssetsPath(t *testing.T) {
 		t.Errorf("Expected path to be a .caddy folder, got: %v", actual)
 	}
 
-	os.Setenv("CADDYPATH", "testpath")
+	err := os.Setenv("CADDYPATH", "testpath")
+	if err != nil {
+		t.Error("Could not set CADDYPATH")
+	}
 	if actual, expected := AssetsPath(), "testpath"; actual != expected {
 		t.Errorf("Expected path to be %v, got: %v", expected, actual)
 	}
-	os.Setenv("CADDYPATH", "")
+	err = os.Setenv("CADDYPATH", "")
+	if err != nil {
+		t.Error("Could not set CADDYPATH")
+	}
 }

caddy.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package caddy implements the Caddy server manager.
 //
 // To use this package:
@@ -5,6 +19,8 @@
 //   1. Set the AppName and AppVersion variables.
 //   2. Call LoadCaddyfile() to get the Caddyfile.
 //      Pass in the name of the server type (like "http").
+//      Make sure the server type's package is imported
+//      (import _ "github.com/mholt/caddy/caddyhttp").
 //   3. Call caddy.Start() to start Caddy. You get back
 //      an Instance, on which you can call Restart() to
 //      restart it or Stop() to stop it.
@@ -15,15 +31,13 @@ package caddy
 
 import (
 	"bytes"
+	"encoding/gob"
 	"fmt"
 	"io"
 	"io/ioutil"
 	"log"
 	"net"
 	"os"
-	"os/exec"
-	"path"
-	"runtime"
 	"strconv"
 	"strings"
 	"sync"
@@ -52,11 +66,29 @@ var (
 	// isUpgrade will be set to true if this process
 	// was started as part of an upgrade, where a parent
 	// Caddy process started this one.
-	isUpgrade bool
+	isUpgrade = os.Getenv("CADDY__UPGRADE") == "1"
+
+	// started will be set to true when the first
+	// instance is started; it never gets set to
+	// false after that.
+	started bool
+
+	// mu protects the variables 'isUpgrade' and 'started'.
+	mu sync.Mutex
 )
 
+func init() {
+	OnProcessExit = append(OnProcessExit, func() {
+		if PidFile != "" {
+			os.Remove(PidFile)
+		}
+	})
+}
+
 // Instance contains the state of servers created as a result of
 // calling Start and can be used to access or control those servers.
+// It is literally an instance of a server type. Instance values
+// should NOT be copied. Use *Instance for safety.
 type Instance struct {
 	// serverType is the name of the instance's server type
 	serverType string
@@ -67,18 +99,36 @@ type Instance struct {
 	// wg is used to wait for all servers to shut down
 	wg *sync.WaitGroup
 
-	// context is the context created for this instance.
+	// context is the context created for this instance,
+	// used to coordinate the setting up of the server type
 	context Context
 
-	// servers is the list of servers with their listeners.
-	servers []serverListener
+	// servers is the list of servers with their listeners
+	servers []ServerListener
 
-	// these are callbacks to execute when certain events happen
-	onStartup  []func() error
-	onRestart  []func() error
-	onShutdown []func() error
+	// these callbacks execute when certain events occur
+	onFirstStartup  []func() error // starting, not as part of a restart
+	onStartup       []func() error // starting, even as part of a restart
+	onRestart       []func() error // before restart commences
+	onShutdown      []func() error // stopping, even as part of a restart
+	onFinalShutdown []func() error // stopping, not as part of a restart
+
+	// storing values on an instance is preferable to
+	// global state because these will get garbage-
+	// collected after in-process reloads when the
+	// old instances are destroyed; use StorageMu
+	// to access this value safely
+	Storage   map[interface{}]interface{}
+	StorageMu sync.RWMutex
 }
 
+func Instances() []*Instance {
+	return instances
+}
+
+// Servers returns the ServerListeners in i.
+func (i *Instance) Servers() []ServerListener { return i.servers }
+
 // Stop stops all servers contained in i. It does NOT
 // execute shutdown callbacks.
 func (i *Instance) Stop() error {
@@ -91,7 +141,7 @@ func (i *Instance) Stop() error {
 		}
 	}
 
-	// splice instance list to delete this one
+	// splice i out of instance list, causing it to be garbage-collected
 	instancesMu.Lock()
 	for j, other := range instances {
 		if other == i {
@@ -104,10 +154,11 @@ func (i *Instance) Stop() error {
 	return nil
 }
 
-// shutdownCallbacks executes all the shutdown callbacks of i.
-// An error returned from one does not stop execution of the rest.
-// All the errors will be returned.
-func (i *Instance) shutdownCallbacks() []error {
+// ShutdownCallbacks executes all the shutdown callbacks of i,
+// including ones that are scheduled only for the final shutdown
+// of i. An error returned from one does not stop execution of
+// the rest. All the non-nil errors will be returned.
+func (i *Instance) ShutdownCallbacks() []error {
 	var errs []error
 	for _, shutdownFunc := range i.onShutdown {
 		err := shutdownFunc()
@@ -115,6 +166,12 @@ func (i *Instance) shutdownCallbacks() []error {
 			errs = append(errs, err)
 		}
 	}
+	for _, finalShutdownFunc := range i.onFinalShutdown {
+		err := finalShutdownFunc()
+		if err != nil {
+			errs = append(errs, err)
+		}
+	}
 	return errs
 }
 
@@ -140,17 +197,29 @@ func (i *Instance) Restart(newCaddyfile Input) (*Instance, error) {
 	}
 
 	// Add file descriptors of all the sockets that are capable of it
-	restartFds := make(map[string]restartPair)
+	restartFds := make(map[string]restartTriple)
 	for _, s := range i.servers {
 		gs, srvOk := s.server.(GracefulServer)
 		ln, lnOk := s.listener.(Listener)
-		if srvOk && lnOk {
-			restartFds[gs.Address()] = restartPair{server: gs, listener: ln}
+		pc, pcOk := s.packet.(PacketConn)
+		if srvOk {
+			if lnOk && pcOk {
+				restartFds[gs.Address()] = restartTriple{server: gs, listener: ln, packet: pc}
+				continue
+			}
+			if lnOk {
+				restartFds[gs.Address()] = restartTriple{server: gs, listener: ln}
+				continue
+			}
+			if pcOk {
+				restartFds[gs.Address()] = restartTriple{server: gs, packet: pc}
+				continue
+			}
 		}
 	}
 
 	// create new instance; if the restart fails, it is simply discarded
-	newInst := &Instance{serverType: newCaddyfile.ServerType(), wg: i.wg}
+	newInst := &Instance{serverType: newCaddyfile.ServerType(), wg: i.wg, Storage: make(map[interface{}]interface{})}
 
 	// attempt to start new instance
 	err := startWithListenerFds(newCaddyfile, newInst, restartFds)
@@ -159,7 +228,19 @@ func (i *Instance) Restart(newCaddyfile Input) (*Instance, error) {
 	}
 
 	// success! stop the old instance
-	i.Stop()
+	for _, shutdownFunc := range i.onShutdown {
+		err = shutdownFunc()
+		if err != nil {
+			return i, err
+		}
+	}
+	err = i.Stop()
+	if err != nil {
+		return i, err
+	}
+
+	// Execute instantiation events
+	EmitEvent(InstanceStartupEvent, newInst)
 
 	log.Println("[INFO] Reloading complete")
 
@@ -170,7 +251,7 @@ func (i *Instance) Restart(newCaddyfile Input) (*Instance, error) {
 // internally-kept list of servers that is running. For
 // saved servers, graceful restarts will be provided.
 func (i *Instance) SaveServer(s Server, ln net.Listener) {
-	i.servers = append(i.servers, serverListener{server: s, listener: ln})
+	i.servers = append(i.servers, ServerListener{server: s, listener: ln})
 }
 
 // HasListenerWithAddress returns whether this package is
@@ -197,7 +278,7 @@ func HasListenerWithAddress(addr string) bool {
 func listenerAddrEqual(ln net.Listener, addr string) bool {
 	lnAddr := ln.Addr().String()
 	hostname, port, err := net.SplitHostPort(addr)
-	if err != nil || hostname != "" {
+	if err != nil {
 		return lnAddr == addr
 	}
 	if lnAddr == net.JoinHostPort("::", port) {
@@ -206,47 +287,59 @@ func listenerAddrEqual(ln net.Listener, addr string) bool {
 	if lnAddr == net.JoinHostPort("0.0.0.0", port) {
 		return true
 	}
-	return false
+	return hostname != "" && lnAddr == addr
 }
 
-/*
-// TODO: We should be able to support UDP servers... I'm considering this pattern.
-
-type UDPListener struct {
-	*net.UDPConn
-}
-
-func (u UDPListener) Accept() (net.Conn, error) {
-	return u.UDPConn, nil
-}
-
-func (u UDPListener) Close() error {
-	return u.UDPConn.Close()
-}
-
-func (u UDPListener) Addr() net.Addr {
-	return u.UDPConn.LocalAddr()
-}
-
-var _ net.Listener = UDPListener{}
-*/
-
-// Server is a type that can listen and serve. A Server
-// must associate with exactly zero or one listeners.
-type Server interface {
+// TCPServer is a type that can listen and serve connections.
+// A TCPServer must associate with exactly zero or one net.Listeners.
+type TCPServer interface {
 	// Listen starts listening by creating a new listener
 	// and returning it. It does not start accepting
-	// connections.
+	// connections. For UDP-only servers, this method
+	// can be a no-op that returns (nil, nil).
 	Listen() (net.Listener, error)
 
 	// Serve starts serving using the provided listener.
 	// Serve must start the server loop nearly immediately,
 	// or at least not return any errors before the server
 	// loop begins. Serve blocks indefinitely, or in other
-	// words, until the server is stopped.
+	// words, until the server is stopped. For UDP-only
+	// servers, this method can be a no-op that returns nil.
 	Serve(net.Listener) error
 }
 
+// UDPServer is a type that can listen and serve packets.
+// A UDPServer must associate with exactly zero or one net.PacketConns.
+type UDPServer interface {
+	// ListenPacket starts listening by creating a new packetconn
+	// and returning it. It does not start accepting connections.
+	// TCP-only servers may leave this method blank and return
+	// (nil, nil).
+	ListenPacket() (net.PacketConn, error)
+
+	// ServePacket starts serving using the provided packetconn.
+	// ServePacket must start the server loop nearly immediately,
+	// or at least not return any errors before the server
+	// loop begins. ServePacket blocks indefinitely, or in other
+	// words, until the server is stopped. For TCP-only servers,
+	// this method can be a no-op that returns nil.
+	ServePacket(net.PacketConn) error
+}
+
+// Server is a type that can listen and serve. It supports both
+// TCP and UDP, although the UDPServer interface can be used
+// for more than just UDP.
+//
+// If the server uses TCP, it should implement TCPServer completely.
+// If it uses UDP or some other protocol, it should implement
+// UDPServer completely. If it uses both, both interfaces should be
+// fully implemented. Any unimplemented methods should be made as
+// no-ops that simply return nil values.
+type Server interface {
+	TCPServer
+	UDPServer
+}
+
 // Stopper is a type that can stop serving. The stop
 // does not necessarily have to be graceful.
 type Stopper interface {
@@ -285,6 +378,15 @@ type Listener interface {
 	File() (*os.File, error)
 }
 
+// PacketConn is a net.PacketConn with an underlying file descriptor.
+// A server's packetconn should implement this interface if it is
+// to support zero-downtime reloads (in sofar this holds true for datagram
+// connections).
+type PacketConn interface {
+	net.PacketConn
+	File() (*os.File, error)
+}
+
 // AfterStartup is an interface that can be implemented
 // by a server type that wants to run some code after all
 // servers for the same Instance have started.
@@ -302,6 +404,16 @@ type AfterStartup interface {
 // is returned. Consequently, this function never returns a nil
 // value as long as there are no errors.
 func LoadCaddyfile(serverType string) (Input, error) {
+	// If we are finishing an upgrade, we must obtain the Caddyfile
+	// from our parent process, regardless of configured loaders.
+	if IsUpgrade() {
+		err := gob.NewDecoder(os.Stdin).Decode(&loadedGob)
+		if err != nil {
+			return nil, err
+		}
+		return loadedGob.Caddyfile, nil
+	}
+
 	// Ask plugged-in loaders for a Caddyfile
 	cdyfile, err := loadCaddyfileInput(serverType)
 	if err != nil {
@@ -332,7 +444,7 @@ func (i *Instance) Wait() {
 // but the Input value will be nil. An error is only returned
 // if there was an error reading the pipe, even if the length
 // of what was read is 0.
-func CaddyfileFromPipe(f *os.File) (Input, error) {
+func CaddyfileFromPipe(f *os.File, serverType string) (Input, error) {
 	fi, err := f.Stat()
 	if err == nil && fi.Mode()&os.ModeCharDevice == 0 {
 		// Note that a non-nil error is not a problem. Windows
@@ -346,8 +458,9 @@ func CaddyfileFromPipe(f *os.File) (Input, error) {
 			return nil, err
 		}
 		return CaddyfileInput{
-			Contents: confBody,
-			Filepath: f.Name(),
+			Contents:       confBody,
+			Filepath:       f.Name(),
+			ServerTypeName: serverType,
 		}, nil
 	}
 
@@ -365,41 +478,47 @@ func (i *Instance) Caddyfile() Input {
 //
 // This function blocks until all the servers are listening.
 func Start(cdyfile Input) (*Instance, error) {
-	writePidFile()
-	inst := &Instance{serverType: cdyfile.ServerType(), wg: new(sync.WaitGroup)}
-	return inst, startWithListenerFds(cdyfile, inst, nil)
+	inst := &Instance{serverType: cdyfile.ServerType(), wg: new(sync.WaitGroup), Storage: make(map[interface{}]interface{})}
+	err := startWithListenerFds(cdyfile, inst, nil)
+	if err != nil {
+		return inst, err
+	}
+	signalSuccessToParent()
+	if pidErr := writePidFile(); pidErr != nil {
+		log.Printf("[ERROR] Could not write pidfile: %v", pidErr)
+	}
+	return inst, nil
 }
 
-func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]restartPair) error {
+func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]restartTriple) error {
+	// save this instance in the list now so that
+	// plugins can access it if need be, for example
+	// the caddytls package, so it can perform cert
+	// renewals while starting up; we just have to
+	// remove the instance from the list later if
+	// it fails
+	instancesMu.Lock()
+	instances = append(instances, inst)
+	instancesMu.Unlock()
+	var err error
+	defer func() {
+		if err != nil {
+			instancesMu.Lock()
+			for i, otherInst := range instances {
+				if otherInst == inst {
+					instances = append(instances[:i], instances[i+1:]...)
+					break
+				}
+			}
+			instancesMu.Unlock()
+		}
+	}()
+
 	if cdyfile == nil {
 		cdyfile = CaddyfileInput{}
 	}
 
-	stypeName := cdyfile.ServerType()
-
-	stype, err := getServerType(stypeName)
-	if err != nil {
-		return err
-	}
-
-	inst.caddyfileInput = cdyfile
-
-	sblocks, err := loadServerBlocks(stypeName, path.Base(cdyfile.Path()), bytes.NewReader(cdyfile.Body()))
-	if err != nil {
-		return err
-	}
-
-	inst.context = stype.NewContext()
-	if inst.context == nil {
-		return fmt.Errorf("server type %s produced a nil Context", stypeName)
-	}
-
-	sblocks, err = inst.context.InspectServerBlocks(cdyfile.Path(), sblocks)
-	if err != nil {
-		return err
-	}
-
-	err = executeDirectives(inst, cdyfile.Path(), stype.Directives, sblocks)
+	err = ValidateAndExecuteDirectives(cdyfile, inst, false)
 	if err != nil {
 		return err
 	}
@@ -409,8 +528,18 @@ func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]r
 		return err
 	}
 
+	// run startup callbacks
+	if !IsUpgrade() && restartFds == nil {
+		// first startup means not a restart or upgrade
+		for _, firstStartupFunc := range inst.onFirstStartup {
+			err = firstStartupFunc()
+			if err != nil {
+				return err
+			}
+		}
+	}
 	for _, startupFunc := range inst.onStartup {
-		err := startupFunc()
+		err = startupFunc()
 		if err != nil {
 			return err
 		}
@@ -421,10 +550,6 @@ func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]r
 		return err
 	}
 
-	instancesMu.Lock()
-	instances = append(instances, inst)
-	instancesMu.Unlock()
-
 	// run any AfterStartup callbacks if this is not
 	// part of a restart; then show file descriptor notice
 	if restartFds == nil {
@@ -435,6 +560,11 @@ func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]r
 		}
 		if !Quiet {
 			for _, srvln := range inst.servers {
+				// only show FD notice if the listener is not nil.
+				// This can happen when only serving UDP or TCP
+				if srvln.listener == nil {
+					continue
+				}
 				if !IsLoopback(srvln.listener.Addr().String()) {
 					checkFdlimit()
 					break
@@ -443,12 +573,53 @@ func startWithListenerFds(cdyfile Input, inst *Instance, restartFds map[string]r
 		}
 	}
 
+	mu.Lock()
+	started = true
+	mu.Unlock()
+
 	return nil
 }
 
-func executeDirectives(inst *Instance, filename string,
-	directives []string, sblocks []caddyfile.ServerBlock) error {
+// ValidateAndExecuteDirectives will load the server blocks from cdyfile
+// by parsing it, then execute the directives configured by it and store
+// the resulting server blocks into inst. If justValidate is true, parse
+// callbacks will not be executed between directives, since the purpose
+// is only to check the input for valid syntax.
+func ValidateAndExecuteDirectives(cdyfile Input, inst *Instance, justValidate bool) error {
+	// If parsing only inst will be nil, create an instance for this function call only.
+	if justValidate {
+		inst = &Instance{serverType: cdyfile.ServerType(), wg: new(sync.WaitGroup), Storage: make(map[interface{}]interface{})}
+	}
 
+	stypeName := cdyfile.ServerType()
+
+	stype, err := getServerType(stypeName)
+	if err != nil {
+		return err
+	}
+
+	inst.caddyfileInput = cdyfile
+
+	sblocks, err := loadServerBlocks(stypeName, cdyfile.Path(), bytes.NewReader(cdyfile.Body()))
+	if err != nil {
+		return err
+	}
+
+	inst.context = stype.NewContext(inst)
+	if inst.context == nil {
+		return fmt.Errorf("server type %s produced a nil Context", stypeName)
+	}
+
+	sblocks, err = inst.context.InspectServerBlocks(cdyfile.Path(), sblocks)
+	if err != nil {
+		return fmt.Errorf("error inspecting server blocks: %v", err)
+	}
+
+	return executeDirectives(inst, cdyfile.Path(), stype.Directives(), sblocks, justValidate)
+}
+
+func executeDirectives(inst *Instance, filename string,
+	directives []string, sblocks []caddyfile.ServerBlock, justValidate bool) error {
 	// map of server block ID to map of directive name to whatever.
 	storages := make(map[int]map[string]interface{})
 
@@ -498,12 +669,14 @@ func executeDirectives(inst *Instance, filename string,
 			}
 		}
 
-		// See if there are any callbacks to execute after this directive
-		if allCallbacks, ok := parsingCallbacks[inst.serverType]; ok {
-			callbacks := allCallbacks[dir]
-			for _, callback := range callbacks {
-				if err := callback(inst.context); err != nil {
-					return err
+		if !justValidate {
+			// See if there are any callbacks to execute after this directive
+			if allCallbacks, ok := parsingCallbacks[inst.serverType]; ok {
+				callbacks := allCallbacks[dir]
+				for _, callback := range callbacks {
+					if err := callback(inst.context); err != nil {
+						return err
+					}
 				}
 			}
 		}
@@ -512,27 +685,81 @@ func executeDirectives(inst *Instance, filename string,
 	return nil
 }
 
-func startServers(serverList []Server, inst *Instance, restartFds map[string]restartPair) error {
+func startServers(serverList []Server, inst *Instance, restartFds map[string]restartTriple) error {
 	errChan := make(chan error, len(serverList))
 
 	for _, s := range serverList {
-		var ln net.Listener
-		var err error
+		var (
+			ln  net.Listener
+			pc  net.PacketConn
+			err error
+		)
+
+		// if performing an upgrade, obtain listener file descriptors
+		// from parent process
+		if IsUpgrade() {
+			if gs, ok := s.(GracefulServer); ok {
+				addr := gs.Address()
+				if fdIndex, ok := loadedGob.ListenerFds["tcp"+addr]; ok {
+					file := os.NewFile(fdIndex, "")
+					ln, err = net.FileListener(file)
+					if err != nil {
+						return err
+					}
+					err = file.Close()
+					if err != nil {
+						return err
+					}
+				}
+				if fdIndex, ok := loadedGob.ListenerFds["udp"+addr]; ok {
+					file := os.NewFile(fdIndex, "")
+					pc, err = net.FilePacketConn(file)
+					if err != nil {
+						return err
+					}
+					err = file.Close()
+					if err != nil {
+						return err
+					}
+				}
+			}
+		}
 
 		// If this is a reload and s is a GracefulServer,
 		// reuse the listener for a graceful restart.
 		if gs, ok := s.(GracefulServer); ok && restartFds != nil {
 			addr := gs.Address()
 			if old, ok := restartFds[addr]; ok {
-				file, err := old.listener.File()
-				if err != nil {
-					return err
+				// listener
+				if old.listener != nil {
+					file, err := old.listener.File()
+					if err != nil {
+						return err
+					}
+					ln, err = net.FileListener(file)
+					if err != nil {
+						return err
+					}
+					err = file.Close()
+					if err != nil {
+						return err
+					}
 				}
-				ln, err = net.FileListener(file)
-				if err != nil {
-					return err
+				// packetconn
+				if old.packet != nil {
+					file, err := old.packet.File()
+					if err != nil {
+						return err
+					}
+					pc, err = net.FilePacketConn(file)
+					if err != nil {
+						return err
+					}
+					err = file.Close()
+					if err != nil {
+						return err
+					}
 				}
-				file.Close()
 			}
 		}
 
@@ -542,14 +769,25 @@ func startServers(serverList []Server, inst *Instance, restartFds map[string]res
 				return err
 			}
 		}
+		if pc == nil {
+			pc, err = s.ListenPacket()
+			if err != nil {
+				return err
+			}
+		}
 
-		inst.wg.Add(1)
-		go func(s Server, ln net.Listener, inst *Instance) {
+		inst.wg.Add(2)
+		go func(s Server, ln net.Listener, pc net.PacketConn, inst *Instance) {
 			defer inst.wg.Done()
-			errChan <- s.Serve(ln)
-		}(s, ln, inst)
 
-		inst.servers = append(inst.servers, serverListener{server: s, listener: ln})
+			go func() {
+				errChan <- s.Serve(ln)
+				defer inst.wg.Done()
+			}()
+			errChan <- s.ServePacket(pc)
+		}(s, ln, pc, inst)
+
+		inst.servers = append(inst.servers, ServerListener{server: s, listener: ln, packet: pc})
 	}
 
 	// Log errors that may be returned from Serve() calls,
@@ -611,14 +849,19 @@ func loadServerBlocks(serverType, filename string, input io.Reader) ([]caddyfile
 // instances after stopping is completed. Do not re-use any
 // references to old instances after calling Stop.
 func Stop() error {
-	instancesMu.Lock()
-	for _, inst := range instances {
+	// This awkward for loop is to avoid a deadlock since
+	// inst.Stop() also acquires the instancesMu lock.
+	for {
+		instancesMu.Lock()
+		if len(instances) == 0 {
+			break
+		}
+		inst := instances[0]
+		instancesMu.Unlock()
 		if err := inst.Stop(); err != nil {
 			log.Printf("[ERROR] Stopping %s: %v", inst.serverType, err)
 		}
 	}
-	instances = []*Instance{}
-	instancesMu.Unlock()
 	return nil
 }
 
@@ -635,39 +878,45 @@ func IsLoopback(addr string) bool {
 		strings.HasPrefix(host, "127.")
 }
 
-// checkFdlimit issues a warning if the OS limit for
-// max file descriptors is below a recommended minimum.
-func checkFdlimit() {
-	const min = 8192
+// IsInternal returns true if the IP of addr
+// belongs to a private network IP range. addr must only
+// be an IP or an IP:port combination.
+// Loopback addresses are considered false.
+func IsInternal(addr string) bool {
+	privateNetworks := []string{
+		"10.0.0.0/8",
+		"172.16.0.0/12",
+		"192.168.0.0/16",
+		"fc00::/7",
+	}
 
-	// Warn if ulimit is too low for production sites
-	if runtime.GOOS == "linux" || runtime.GOOS == "darwin" {
-		out, err := exec.Command("sh", "-c", "ulimit -n").Output() // use sh because ulimit isn't in Linux $PATH
-		if err == nil {
-			lim, err := strconv.Atoi(string(bytes.TrimSpace(out)))
-			if err == nil && lim < min {
-				fmt.Printf("WARNING: File descriptor limit %d is too low for production servers. "+
-					"At least %d is recommended. Fix with \"ulimit -n %d\".\n", lim, min, min)
-			}
+	host, _, err := net.SplitHostPort(addr)
+	if err != nil {
+		host = addr // happens if the addr is just a hostname, missing port
+		// if we encounter an error, the brackets need to be stripped
+		// because SplitHostPort didn't do it for us
+		host = strings.Trim(host, "[]")
+	}
+	ip := net.ParseIP(host)
+	if ip == nil {
+		return false
+	}
+	for _, privateNetwork := range privateNetworks {
+		_, ipnet, _ := net.ParseCIDR(privateNetwork)
+		if ipnet.Contains(ip) {
+			return true
 		}
 	}
+	return false
 }
 
-// Upgrade re-launches the process, preserving the listeners
-// for a graceful restart. It does NOT load new configuration;
-// it only starts the process anew with a fresh binary.
-//
-// TODO: This is not yet implemented
-func Upgrade() error {
-	return fmt.Errorf("not implemented")
-	// TODO: have child process set isUpgrade = true
-}
-
-// IsUpgrade returns true if this process is part of an upgrade
-// where a parent caddy process spawned this one to ugprade
-// the binary.
-func IsUpgrade() bool {
-	return isUpgrade
+// Started returns true if at least one instance has been
+// started by this package. It never gets reset to false
+// once it is set to true.
+func Started() bool {
+	mu.Lock()
+	defer mu.Unlock()
+	return started
 }
 
 // CaddyfileInput represents a Caddyfile as input
@@ -729,9 +978,10 @@ func writePidFile() error {
 	return ioutil.WriteFile(PidFile, pid, 0644)
 }
 
-type restartPair struct {
+type restartTriple struct {
 	server   GracefulServer
 	listener Listener
+	packet   PacketConn
 }
 
 var (
@@ -742,8 +992,11 @@ var (
 	instancesMu sync.Mutex
 )
 
-const (
+var (
 	// DefaultConfigFile is the name of the configuration file that is loaded
 	// by default if no other file is specified.
 	DefaultConfigFile = "Caddyfile"
 )
+
+// CtxKey is a value type for use with context.WithValue.
+type CtxKey string

caddy/build.bash

@@ -1,56 +0,0 @@
-#!/usr/bin/env bash
-#
-# Caddy build script. Automates proper versioning.
-#
-# Usage:
-#
-#     $ ./build.bash [output_filename] [git_repo]
-#
-# Outputs compiled program in current directory.
-# Default git repo is current directory.
-# Builds always take place from current directory.
-
-set -euo pipefail
-
-: ${output_filename:="${1:-}"}
-: ${output_filename:="caddy"}
-
-: ${git_repo:="${2:-}"}
-: ${git_repo:="."}
-
-pkg=github.com/mholt/caddy/caddy/caddymain
-ldflags=()
-
-# Timestamp of build
-name="${pkg}.buildDate"
-value=$(date -u +"%a %b %d %H:%M:%S %Z %Y")
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Current tag, if HEAD is on a tag
-name="${pkg}.gitTag"
-set +e
-value="$(git -C "${git_repo}" describe --exact-match HEAD 2>/dev/null)"
-set -e
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Nearest tag on branch
-name="${pkg}.gitNearestTag"
-value="$(git -C "${git_repo}" describe --abbrev=0 --tags HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Commit SHA
-name="${pkg}.gitCommit"
-value="$(git -C "${git_repo}" rev-parse --short HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# Summary of uncommitted changes
-name="${pkg}.gitShortStat"
-value="$(git -C "${git_repo}" diff-index --shortstat HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-# List of modified files
-name="${pkg}.gitFilesModified"
-value="$(git -C "${git_repo}" diff-index --name-only HEAD)"
-ldflags+=("-X" "\"${name}=${value}\"")
-
-go build -ldflags "${ldflags[*]}" -o "${output_filename}"

caddy/build.go

@@ -0,0 +1,87 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// +build dev
+
+// build.go automates proper versioning of caddy binaries.
+// Use it like:   go run build.go
+// You can customize the build with the -goos, -goarch, and
+// -goarm CLI options:   go run build.go -goos=windows
+//
+// To get proper version information, this program must be
+// run from the directory of this file, and the source code
+// must be a working git repository, since it needs to know
+// if the source is in a clean state.
+//
+// This program is NOT required to build Caddy from source
+// since it is go-gettable. (You can run plain `go build`
+// in this directory to get a binary.) However, issues filed
+// without version information will likely be closed.
+package main
+
+import (
+	"flag"
+	"fmt"
+	"log"
+	"os"
+	"os/exec"
+	"path/filepath"
+
+	"github.com/caddyserver/builds"
+)
+
+var goos, goarch, goarm string
+
+func init() {
+	flag.StringVar(&goos, "goos", "", "GOOS for which to build")
+	flag.StringVar(&goarch, "goarch", "", "GOARCH for which to build")
+	flag.StringVar(&goarm, "goarm", "", "GOARM for which to build")
+}
+
+func main() {
+	flag.Parse()
+
+	gopath := os.Getenv("GOPATH")
+
+	pwd, err := os.Getwd()
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	ldflags, err := builds.MakeLdFlags(filepath.Join(pwd, ".."))
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	args := []string{"build", "-ldflags", ldflags}
+	args = append(args, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
+	args = append(args, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
+	cmd := exec.Command("go", args...)
+	cmd.Stderr = os.Stderr
+	cmd.Stdout = os.Stdout
+	cmd.Env = os.Environ()
+	for _, env := range []string{
+		"CGO_ENABLED=0",
+		"GOOS=" + goos,
+		"GOARCH=" + goarch,
+		"GOARM=" + goarm,
+	} {
+		cmd.Env = append(cmd.Env, env)
+	}
+
+	err = cmd.Run()
+	if err != nil {
+		log.Fatal(err)
+	}
+}

caddy/caddymain/run.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddymain
 
 import (
@@ -7,7 +21,6 @@ import (
 	"io/ioutil"
 	"log"
 	"os"
-	"path/filepath"
 	"runtime"
 	"strconv"
 	"strings"
@@ -30,16 +43,20 @@ func init() {
 
 	flag.BoolVar(&caddytls.Agreed, "agree", false, "Agree to the CA's Subscriber Agreement")
 	flag.StringVar(&caddytls.DefaultCAUrl, "ca", "https://acme-v01.api.letsencrypt.org/directory", "URL to certificate authority's ACME server directory")
+	flag.BoolVar(&caddytls.DisableHTTPChallenge, "disable-http-challenge", caddytls.DisableHTTPChallenge, "Disable the ACME HTTP challenge")
+	flag.BoolVar(&caddytls.DisableTLSSNIChallenge, "disable-tls-sni-challenge", caddytls.DisableTLSSNIChallenge, "Disable the ACME TLS-SNI challenge")
 	flag.StringVar(&conf, "conf", "", "Caddyfile to load (default \""+caddy.DefaultConfigFile+"\")")
 	flag.StringVar(&cpu, "cpu", "100%", "CPU cap")
 	flag.BoolVar(&plugins, "plugins", false, "List installed plugins")
 	flag.StringVar(&caddytls.DefaultEmail, "email", "", "Default ACME CA account email address")
+	flag.DurationVar(&acme.HTTPClient.Timeout, "catimeout", acme.HTTPClient.Timeout, "Default ACME CA HTTP timeout")
 	flag.StringVar(&logfile, "log", "", "Process log file")
 	flag.StringVar(&caddy.PidFile, "pidfile", "", "Path to write pid file")
 	flag.BoolVar(&caddy.Quiet, "quiet", false, "Quiet mode (no initialization output)")
 	flag.StringVar(&revoke, "revoke", "", "Hostname for which to revoke the certificate")
 	flag.StringVar(&serverType, "type", "http", "Type of server to run")
 	flag.BoolVar(&version, "version", false, "Show version")
+	flag.BoolVar(&validate, "validate", false, "Parse the Caddyfile but do not start the server")
 
 	caddy.RegisterCaddyfileLoader("flag", caddy.LoaderFunc(confLoader))
 	caddy.SetDefaultCaddyfileLoader("default", caddy.LoaderFunc(defaultLoader))
@@ -48,7 +65,6 @@ func init() {
 // Run is Caddy's main() function.
 func Run() {
 	flag.Parse()
-	moveStorage() // TODO: This is temporary for the 0.9 release, or until most users upgrade to 0.9+
 
 	caddy.AppName = appName
 	caddy.AppVersion = appVersion
@@ -75,13 +91,13 @@ func Run() {
 	if revoke != "" {
 		err := caddytls.Revoke(revoke)
 		if err != nil {
-			log.Fatal(err)
+			mustLogFatalf("%v", err)
 		}
 		fmt.Printf("Revoked certificate for %s\n", revoke)
 		os.Exit(0)
 	}
 	if version {
-		fmt.Printf("%s %s\n", appName, appVersion)
+		fmt.Printf("%s %s (unofficial)\n", appName, appVersion)
 		if devBuild && gitShortStat != "" {
 			fmt.Printf("%s\n%s\n", gitShortStat, gitFilesModified)
 		}
@@ -95,36 +111,53 @@ func Run() {
 	// Set CPU cap
 	err := setCPU(cpu)
 	if err != nil {
-		mustLogFatal(err)
+		mustLogFatalf("%v", err)
 	}
 
+	// Executes Startup events
+	caddy.EmitEvent(caddy.StartupEvent, nil)
+
 	// Get Caddyfile input
-	caddyfile, err := caddy.LoadCaddyfile(serverType)
+	caddyfileinput, err := caddy.LoadCaddyfile(serverType)
 	if err != nil {
-		mustLogFatal(err)
+		mustLogFatalf("%v", err)
+	}
+
+	if validate {
+		err := caddy.ValidateAndExecuteDirectives(caddyfileinput, nil, true)
+		if err != nil {
+			mustLogFatalf("%v", err)
+		}
+		msg := "Caddyfile is valid"
+		fmt.Println(msg)
+		log.Printf("[INFO] %s", msg)
+		os.Exit(0)
 	}
 
 	// Start your engines
-	instance, err := caddy.Start(caddyfile)
+	instance, err := caddy.Start(caddyfileinput)
 	if err != nil {
-		mustLogFatal(err)
+		mustLogFatalf("%v", err)
 	}
 
+	// Execute instantiation events
+	caddy.EmitEvent(caddy.InstanceStartupEvent, instance)
+
 	// Twiddle your thumbs
 	instance.Wait()
 }
 
-// mustLogFatal wraps log.Fatal() in a way that ensures the
+// mustLogFatalf wraps log.Fatalf() in a way that ensures the
 // output is always printed to stderr so the user can see it
 // if the user is still there, even if the process log was not
 // enabled. If this process is an upgrade, however, and the user
 // might not be there anymore, this just logs to the process
 // log and exits.
-func mustLogFatal(args ...interface{}) {
+func mustLogFatalf(format string, args ...interface{}) {
 	if !caddy.IsUpgrade() {
 		log.SetOutput(os.Stderr)
 	}
-	log.Fatal(args...)
+	log.Fatalf(format, args...)
 }
 
 // confLoader loads the Caddyfile using the -conf flag.
@@ -134,13 +167,21 @@ func confLoader(serverType string) (caddy.Input, error) {
 	}
 
 	if conf == "stdin" {
-		return caddy.CaddyfileFromPipe(os.Stdin)
+		return caddy.CaddyfileFromPipe(os.Stdin, serverType)
 	}
 
-	contents, err := ioutil.ReadFile(conf)
-	if err != nil {
-		return nil, err
+	var contents []byte
+	if strings.Contains(conf, "*") {
+		// Let caddyfile.doImport logic handle the globbed path
+		contents = []byte("import " + conf)
+	} else {
+		var err error
+		contents, err = ioutil.ReadFile(conf)
+		if err != nil {
+			return nil, err
+		}
 	}
+
 	return caddy.CaddyfileInput{
 		Contents:       contents,
 		Filepath:       conf,
@@ -164,52 +205,20 @@ func defaultLoader(serverType string) (caddy.Input, error) {
 	}, nil
 }
 
-// moveStorage moves the old certificate storage location by
-// renaming the "letsencrypt" folder to the hostname of the
-// CA URL. This is TEMPORARY until most users have upgraded to 0.9+.
-func moveStorage() {
-	oldPath := filepath.Join(caddy.AssetsPath(), "letsencrypt")
-	_, err := os.Stat(oldPath)
-	if os.IsNotExist(err) {
-		return
-	}
-	newPath, err := caddytls.StorageFor(caddytls.DefaultCAUrl)
-	if err != nil {
-		log.Fatalf("[ERROR] Unable to get new path for certificate storage: %v", err)
-	}
-	err = os.MkdirAll(string(newPath), 0700)
-	if err != nil {
-		log.Fatalf("[ERROR] Unable to make new certificate storage path: %v", err)
-	}
-	err = os.Rename(oldPath, string(newPath))
-	if err != nil {
-		log.Fatalf("[ERROR] Unable to migrate certificate storage: %v", err)
-	}
-	// convert mixed case folder and file names to lowercase
-	filepath.Walk(string(newPath), func(path string, info os.FileInfo, err error) error {
-		// must be careful to only lowercase the base of the path, not the whole thing!!
-		base := filepath.Base(path)
-		if lowerBase := strings.ToLower(base); base != lowerBase {
-			lowerPath := filepath.Join(filepath.Dir(path), lowerBase)
-			err = os.Rename(path, lowerPath)
-			if err != nil {
-				log.Fatalf("[ERROR] Unable to lower-case: %v", err)
-			}
-		}
-		return nil
-	})
-}
-
 // setVersion figures out the version information
 // based on variables set by -ldflags.
 func setVersion() {
 	// A development build is one that's not at a tag or has uncommitted changes
 	devBuild = gitTag == "" || gitShortStat != ""
 
+	if buildDate != "" {
+		buildDate = " " + buildDate
+	}
+
 	// Only set the appVersion if -ldflags was used
 	if gitNearestTag != "" || gitTag != "" {
 		if devBuild && gitNearestTag != "" {
-			appVersion = fmt.Sprintf("%s (+%s %s)",
+			appVersion = fmt.Sprintf("%s (+%s%s)",
 				strings.TrimPrefix(gitNearestTag, "v"), gitCommit, buildDate)
 		} else if gitTag != "" {
 			appVersion = strings.TrimPrefix(gitTag, "v")
@@ -220,6 +229,8 @@ func setVersion() {
 // setCPU parses string cpu and sets GOMAXPROCS
 // according to its value. It accepts either
 // a number (e.g. 3) or a percent (e.g. 50%).
+// If the percent resolves to less than a single
+// GOMAXPROCS, it rounds it up to GOMAXPROCS=1.
 func setCPU(cpu string) error {
 	var numCPU int
 
@@ -235,6 +246,9 @@ func setCPU(cpu string) error {
 		}
 		percent = float32(pctInt) / 100
 		numCPU = int(float32(availCPU) * percent)
+		if numCPU < 1 {
+			numCPU = 1
+		}
 	} else {
 		// Number
 		num, err := strconv.Atoi(cpu)
@@ -263,6 +277,7 @@ var (
 	revoke     string
 	version    bool
 	plugins    bool
+	validate   bool
 )
 
 // Build information obtained with the help of -ldflags

caddy/caddymain/run_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddymain
 
 import (
@@ -27,6 +41,7 @@ func TestSetCPU(t *testing.T) {
 		{"invalid input", currentCPU, true},
 		{"invalid input%", currentCPU, true},
 		{"9999", maxCPU, false}, // over available CPU
+		{"1%", 1, false},        // under a single CPU; assume maxCPU < 100
 	} {
 		err := setCPU(test.input)
 		if test.shouldErr && err == nil {

caddy/main.go

@@ -1,7 +1,21 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // By moving the application's package main logic into
 // a package other than main, it becomes much easier to
 // wrap caddy for custom builds that are go-gettable.
-// https://forum.caddyserver.com/t/my-wish-for-0-9-go-gettable-custom-builds/59?u=matt
+// https://caddy.community/t/my-wish-for-0-9-go-gettable-custom-builds/59?u=matt
 
 package main
 

caddy/main_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package main
 
 import "testing"

caddy_test.go

@@ -1,6 +1,24 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddy
 
-import "testing"
+import (
+	"net"
+	"strconv"
+	"testing"
+)
 
 /*
 // TODO
@@ -56,3 +74,100 @@ func TestIsLoopback(t *testing.T) {
 		}
 	}
 }
+
+func TestIsInternal(t *testing.T) {
+	for i, test := range []struct {
+		input  string
+		expect bool
+	}{
+		{"9.255.255.255", false},
+		{"10.0.0.0", true},
+		{"10.0.0.1", true},
+		{"10.255.255.254", true},
+		{"10.255.255.255", true},
+		{"11.0.0.0", false},
+		{"10.0.0.5:1234", true},
+		{"11.0.0.5:1234", false},
+
+		{"172.15.255.255", false},
+		{"172.16.0.0", true},
+		{"172.16.0.1", true},
+		{"172.31.255.254", true},
+		{"172.31.255.255", true},
+		{"172.32.0.0", false},
+		{"172.16.0.1:1234", true},
+
+		{"192.167.255.255", false},
+		{"192.168.0.0", true},
+		{"192.168.0.1", true},
+		{"192.168.255.254", true},
+		{"192.168.255.255", true},
+		{"192.169.0.0", false},
+		{"192.168.0.1:1234", true},
+
+		{"fbff:ffff:ffff:ffff:ffff:ffff:ffff:ffff", false},
+		{"fc00::", true},
+		{"fc00::1", true},
+		{"[fc00::1]", true},
+		{"[fc00::1]:8888", true},
+		{"fdff:ffff:ffff:ffff:ffff:ffff:ffff:fffe", true},
+		{"fdff:ffff:ffff:ffff:ffff:ffff:ffff:ffff", true},
+		{"fe00::", false},
+		{"fd12:3456:789a:1::1:1234", true},
+
+		{"example.com", false},
+		{"localhost", false},
+		{"localhost:1234", false},
+		{"localhost:", false},
+		{"127.0.0.1", false},
+		{"127.0.0.1:443", false},
+		{"127.0.1.5", false},
+		{"12.7.0.1", false},
+		{"[::1]", false},
+		{"[::1]:1234", false},
+		{"::1", false},
+		{"::", false},
+		{"[::]", false},
+		{"local", false},
+	} {
+		if got, want := IsInternal(test.input), test.expect; got != want {
+			t.Errorf("Test %d (%s): expected %v but was %v", i, test.input, want, got)
+		}
+	}
+}
+
+func TestListenerAddrEqual(t *testing.T) {
+	ln1, err := net.Listen("tcp", "[::]:0")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer ln1.Close()
+	ln1port := strconv.Itoa(ln1.Addr().(*net.TCPAddr).Port)
+
+	ln2, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer ln2.Close()
+	ln2port := strconv.Itoa(ln2.Addr().(*net.TCPAddr).Port)
+
+	for i, test := range []struct {
+		ln     net.Listener
+		addr   string
+		expect bool
+	}{
+		{ln1, ":" + ln2port, false},
+		{ln1, "0.0.0.0:" + ln2port, false},
+		{ln1, "0.0.0.0", false},
+		{ln1, ":" + ln1port, true},
+		{ln1, "0.0.0.0:" + ln1port, true},
+		{ln2, ":" + ln2port, false},
+		{ln2, "127.0.0.1:" + ln1port, false},
+		{ln2, "127.0.0.1", false},
+		{ln2, "127.0.0.1:" + ln2port, true},
+	} {
+		if got, want := listenerAddrEqual(test.ln, test.addr), test.expect; got != want {
+			t.Errorf("Test %d (%s == %s): expected %v but was %v", i, test.addr, test.ln.Addr().String(), want, got)
+		}
+	}
+}

caddyfile/dispenser.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import (
@@ -19,9 +33,10 @@ type Dispenser struct {
 
 // NewDispenser returns a Dispenser, ready to use for parsing the given input.
 func NewDispenser(filename string, input io.Reader) Dispenser {
+	tokens, _ := allTokens(input) // ignoring error because nothing to do with it
 	return Dispenser{
 		filename: filename,
-		tokens:   allTokens(input),
+		tokens:   tokens,
 		cursor:   -1,
 	}
 }
@@ -119,12 +134,6 @@ func (d *Dispenser) NextBlock() bool {
 	return true
 }
 
-// IncrNest adds a level of nesting to the dispenser.
-func (d *Dispenser) IncrNest() {
-	d.nesting++
-	return
-}
-
 // Val gets the text of the current token. If there is no token
 // loaded, it returns empty string.
 func (d *Dispenser) Val() string {
@@ -215,9 +224,9 @@ func (d *Dispenser) EOFErr() error {
 	return d.Errf("Unexpected EOF")
 }
 
-// Err generates a custom parse error with a message of msg.
+// Err generates a custom parse-time error with a message of msg.
 func (d *Dispenser) Err(msg string) error {
-	msg = fmt.Sprintf("%s:%d - Parse error: %s", d.File(), d.Line(), msg)
+	msg = fmt.Sprintf("%s:%d - Error during parsing: %s", d.File(), d.Line(), msg)
 	return errors.New(msg)
 }
 

caddyfile/dispenser_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import (
@@ -64,7 +78,7 @@ func TestDispenser_NextArg(t *testing.T) {
 	}
 
 	assertNextArg := func(expectedVal string, loadAnother bool, expectedCursor int) {
-		if d.NextArg() != true {
+		if !d.NextArg() {
 			t.Error("NextArg(): Should load next argument but got false instead")
 		}
 		if d.cursor != expectedCursor {
@@ -74,7 +88,7 @@ func TestDispenser_NextArg(t *testing.T) {
 			t.Errorf("Val(): Expected '%s' but got '%s'", expectedVal, val)
 		}
 		if !loadAnother {
-			if d.NextArg() != false {
+			if d.NextArg() {
 				t.Fatalf("NextArg(): Should NOT load another argument, but got true instead (val: '%s')", d.Val())
 			}
 			if d.cursor != expectedCursor {

caddyfile/json.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import (

caddyfile/json_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import "testing"

caddyfile/lexer.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import (
@@ -26,9 +40,23 @@ type (
 )
 
 // load prepares the lexer to scan an input for tokens.
+// It discards any leading byte order mark.
 func (l *lexer) load(input io.Reader) error {
 	l.reader = bufio.NewReader(input)
 	l.line = 1
+
+	// discard byte order mark, if present
+	firstCh, _, err := l.reader.ReadRune()
+	if err != nil {
+		return err
+	}
+	if firstCh != 0xFEFF {
+		err := l.reader.UnreadRune()
+		if err != nil {
+			return err
+		}
+	}
+
 	return nil
 }
 

caddyfile/lexer_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import (
@@ -128,6 +142,12 @@ func TestLexer(t *testing.T) {
 				{Line: 2, Text: "characters"},
 			},
 		},
+		{
+			input: "\xEF\xBB\xBF:8080", // test with leading byte order mark
+			expected: []Token{
+				{Line: 1, Text: ":8080"},
+			},
+		},
 	}
 
 	for i, testCase := range testCases {

caddyfile/parse.go

@@ -1,7 +1,22 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import (
 	"io"
+	"log"
 	"os"
 	"path/filepath"
 	"strings"
@@ -15,20 +30,23 @@ import (
 // pass in nil instead.
 func Parse(filename string, input io.Reader, validDirectives []string) ([]ServerBlock, error) {
 	p := parser{Dispenser: NewDispenser(filename, input), validDirectives: validDirectives}
-	blocks, err := p.parseAll()
-	return blocks, err
+	return p.parseAll()
 }
 
 // allTokens lexes the entire input, but does not parse it.
 // It returns all the tokens from the input, unstructured
 // and in order.
-func allTokens(input io.Reader) (tokens []Token) {
+func allTokens(input io.Reader) ([]Token, error) {
 	l := new(lexer)
-	l.load(input)
+	err := l.load(input)
+	if err != nil {
+		return nil, err
+	}
+	var tokens []Token
 	for l.next() {
 		tokens = append(tokens, l.token)
 	}
-	return
+	return tokens, nil
 }
 
 type parser struct {
@@ -36,6 +54,7 @@ type parser struct {
 	block           ServerBlock // current server block being parsed
 	validDirectives []string    // a directive must be valid or it's an error
 	eof             bool        // if we encounter a valid EOF in a hard place
+	definedSnippets map[string][]Token
 }
 
 func (p *parser) parseAll() ([]ServerBlock, error) {
@@ -57,12 +76,7 @@ func (p *parser) parseAll() ([]ServerBlock, error) {
 func (p *parser) parseOne() error {
 	p.block = ServerBlock{Tokens: make(map[string][]Token)}
 
-	err := p.begin()
-	if err != nil {
-		return err
-	}
-
-	return nil
+	return p.begin()
 }
 
 func (p *parser) begin() error {
@@ -71,6 +85,7 @@ func (p *parser) begin() error {
 	}
 
 	err := p.addresses()
+
 	if err != nil {
 		return err
 	}
@@ -81,12 +96,25 @@ func (p *parser) begin() error {
 		return nil
 	}
 
-	err = p.blockContents()
-	if err != nil {
-		return err
+	if ok, name := p.isSnippet(); ok {
+		if p.definedSnippets == nil {
+			p.definedSnippets = map[string][]Token{}
+		}
+		if _, found := p.definedSnippets[name]; found {
+			return p.Errf("redeclaration of previously declared snippet %s", name)
+		}
+		// consume all tokens til matched close brace
+		tokens, err := p.snippetTokens()
+		if err != nil {
+			return err
+		}
+		p.definedSnippets[name] = tokens
+		// empty block keys so we don't save this block as a real server.
+		p.block.Keys = nil
+		return nil
 	}
 
-	return nil
+	return p.blockContents()
 }
 
 func (p *parser) addresses() error {
@@ -201,36 +229,86 @@ func (p *parser) directives() error {
 // other words, call Next() to access the first token that was
 // imported.
 func (p *parser) doImport() error {
-	// syntax check
+	// syntax checks
 	if !p.NextArg() {
 		return p.ArgErr()
 	}
-	importPattern := p.Val()
+	importPattern := replaceEnvVars(p.Val())
+	if importPattern == "" {
+		return p.Err("Import requires a non-empty filepath")
+	}
 	if p.NextArg() {
 		return p.Err("Import takes only one argument (glob pattern or file)")
 	}
-
-	// do glob
-	matches, err := filepath.Glob(importPattern)
-	if err != nil {
-		return p.Errf("Failed to use import pattern %s: %v", importPattern, err)
-	}
-	if len(matches) == 0 {
-		return p.Errf("No files matching import pattern %s", importPattern)
-	}
-
 	// splice out the import directive and its argument (2 tokens total)
 	tokensBefore := p.tokens[:p.cursor-1]
 	tokensAfter := p.tokens[p.cursor+1:]
-
-	// collect all the imported tokens
 	var importedTokens []Token
-	for _, importFile := range matches {
-		newTokens, err := p.doSingleImport(importFile)
+
+	// first check snippets. That is a simple, non-recursive replacement
+	if p.definedSnippets != nil && p.definedSnippets[importPattern] != nil {
+		importedTokens = p.definedSnippets[importPattern]
+	} else {
+		// make path relative to Caddyfile rather than current working directory (issue #867)
+		// and then use glob to get list of matching filenames
+		absFile, err := filepath.Abs(p.Dispenser.filename)
 		if err != nil {
-			return err
+			return p.Errf("Failed to get absolute path of file: %s: %v", p.Dispenser.filename, err)
+		}
+
+		var matches []string
+		var globPattern string
+		if !filepath.IsAbs(importPattern) {
+			globPattern = filepath.Join(filepath.Dir(absFile), importPattern)
+		} else {
+			globPattern = importPattern
+		}
+		matches, err = filepath.Glob(globPattern)
+
+		if err != nil {
+			return p.Errf("Failed to use import pattern %s: %v", importPattern, err)
+		}
+		if len(matches) == 0 {
+			if strings.Contains(globPattern, "*") {
+				log.Printf("[WARNING] No files matching import pattern: %s", importPattern)
+			} else {
+				return p.Errf("File to import not found: %s", importPattern)
+			}
+		}
+
+		// collect all the imported tokens
+
+		for _, importFile := range matches {
+			newTokens, err := p.doSingleImport(importFile)
+			if err != nil {
+				return err
+			}
+
+			var importLine int
+			for i, token := range newTokens {
+				if token.Text == "import" {
+					importLine = token.Line
+					continue
+				}
+				if token.Line == importLine {
+					var abs string
+					if filepath.IsAbs(token.Text) {
+						abs = token.Text
+					} else if !filepath.IsAbs(importFile) {
+						abs = filepath.Join(filepath.Dir(absFile), token.Text)
+					} else {
+						abs = filepath.Join(filepath.Dir(importFile), token.Text)
+					}
+					newTokens[i] = Token{
+						Text: abs,
+						Line: token.Line,
+						File: token.File,
+					}
+				}
+			}
+
+			importedTokens = append(importedTokens, newTokens...)
 		}
-		importedTokens = append(importedTokens, newTokens...)
 	}
 
 	// splice the imported tokens in the place of the import statement
@@ -249,10 +327,24 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
 		return nil, p.Errf("Could not import %s: %v", importFile, err)
 	}
 	defer file.Close()
-	importedTokens := allTokens(file)
 
-	// Tack the filename onto these tokens so errors show the imported file's name
-	filename := filepath.Base(importFile)
+	if info, err := file.Stat(); err != nil {
+		return nil, p.Errf("Could not import %s: %v", importFile, err)
+	} else if info.IsDir() {
+		return nil, p.Errf("Could not import %s: is a directory", importFile)
+	}
+
+	importedTokens, err := allTokens(file)
+	if err != nil {
+		return nil, p.Errf("Could not read tokens while importing %s: %v", importFile, err)
+	}
+
+	// Tack the file path onto these tokens so errors show the imported file's name
+	// (we use full, absolute path to avoid bugs: issue #1892)
+	filename, err := filepath.Abs(importFile)
+	if err != nil {
+		return nil, p.Errf("Failed to get absolute path of file: %s: %v", p.Dispenser.filename, err)
+	}
 	for i := 0; i < len(importedTokens); i++ {
 		importedTokens[i].File = filename
 	}
@@ -365,3 +457,41 @@ type ServerBlock struct {
 	Keys   []string
 	Tokens map[string][]Token
 }
+
+func (p *parser) isSnippet() (bool, string) {
+	keys := p.block.Keys
+	// A snippet block is a single key with parens. Nothing else qualifies.
+	if len(keys) == 1 && strings.HasPrefix(keys[0], "(") && strings.HasSuffix(keys[0], ")") {
+		return true, strings.TrimSuffix(keys[0][1:], ")")
+	}
+	return false, ""
+}
+
+// read and store everything in a block for later replay.
+func (p *parser) snippetTokens() ([]Token, error) {
+	// TODO: disallow imports in snippets for simplicity at import time
+	// snippet must have curlies.
+	err := p.openCurlyBrace()
+	if err != nil {
+		return nil, err
+	}
+	count := 1
+	tokens := []Token{}
+	for p.Next() {
+		if p.Val() == "}" {
+			count--
+			if count == 0 {
+				break
+			}
+		}
+		if p.Val() == "{" {
+			count++
+		}
+		tokens = append(tokens, p.tokens[p.cursor])
+	}
+	// make sure we're matched up
+	if count != 0 {
+		return nil, p.SyntaxErr("}")
+	}
+	return tokens, nil
+}

caddyfile/parse_test.go

@@ -1,7 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyfile
 
 import (
+	"io/ioutil"
 	"os"
+	"path/filepath"
 	"strings"
 	"testing"
 )
@@ -9,8 +25,11 @@ import (
 func TestAllTokens(t *testing.T) {
 	input := strings.NewReader("a b c\nd e")
 	expected := []string{"a", "b", "c", "d", "e"}
-	tokens := allTokens(input)
+	tokens, err := allTokens(input)
 
+	if err != nil {
+		t.Fatalf("Expected no error, got %v", err)
+	}
 	if len(tokens) != len(expected) {
 		t.Fatalf("Expected %d tokens, got %d", len(expected), len(tokens))
 	}
@@ -246,6 +265,101 @@ func TestParseOneAndImport(t *testing.T) {
 	}
 }
 
+func TestRecursiveImport(t *testing.T) {
+	testParseOne := func(input string) (ServerBlock, error) {
+		p := testParser(input)
+		p.Next() // parseOne doesn't call Next() to start, so we must
+		err := p.parseOne()
+		return p.block, err
+	}
+
+	isExpected := func(got ServerBlock) bool {
+		if len(got.Keys) != 1 || got.Keys[0] != "localhost" {
+			t.Errorf("got keys unexpected: expect localhost, got %v", got.Keys)
+			return false
+		}
+		if len(got.Tokens) != 2 {
+			t.Errorf("got wrong number of tokens: expect 2, got %d", len(got.Tokens))
+			return false
+		}
+		if len(got.Tokens["dir1"]) != 1 || len(got.Tokens["dir2"]) != 2 {
+			t.Errorf("got unexpect tokens: %v", got.Tokens)
+			return false
+		}
+		return true
+	}
+
+	recursiveFile1, err := filepath.Abs("testdata/recursive_import_test1")
+	if err != nil {
+		t.Fatal(err)
+	}
+	recursiveFile2, err := filepath.Abs("testdata/recursive_import_test2")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// test relative recursive import
+	err = ioutil.WriteFile(recursiveFile1, []byte(
+		`localhost
+		dir1
+		import recursive_import_test2`), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(recursiveFile1)
+
+	err = ioutil.WriteFile(recursiveFile2, []byte("dir2 1"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(recursiveFile2)
+
+	// import absolute path
+	result, err := testParseOne("import " + recursiveFile1)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("absolute+relative import failed")
+	}
+
+	// import relative path
+	result, err = testParseOne("import testdata/recursive_import_test1")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("relative+relative import failed")
+	}
+
+	// test absolute recursive import
+	err = ioutil.WriteFile(recursiveFile1, []byte(
+		`localhost
+		dir1
+		import `+recursiveFile2), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// import absolute path
+	result, err = testParseOne("import " + recursiveFile1)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("absolute+absolute import failed")
+	}
+
+	// import relative path
+	result, err = testParseOne("import testdata/recursive_import_test1")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("relative+absolute import failed")
+	}
+}
+
 func TestParseAll(t *testing.T) {
 	for i, test := range []struct {
 		input     string
@@ -288,6 +402,9 @@ func TestParseAll(t *testing.T) {
 			{"glob1.host0"},
 			{"glob2.host0"},
 		}},
+
+		{`import notfound/*`, false, [][]string{}},        // glob needn't error with no matches
+		{`import notfound/file.conf`, true, [][]string{}}, // but a specific file should
 	} {
 		p := testParser(test.input)
 		blocks, err := p.parseAll()
@@ -394,6 +511,42 @@ func TestEnvironmentReplacement(t *testing.T) {
 
 func testParser(input string) parser {
 	buf := strings.NewReader(input)
-	p := parser{Dispenser: NewDispenser("Test", buf)}
+	p := parser{Dispenser: NewDispenser("Caddyfile", buf)}
 	return p
 }
+
+func TestSnippets(t *testing.T) {
+	p := testParser(`(common) {
+					gzip foo
+					errors stderr
+					
+				}	
+				http://example.com {
+					import common
+				}
+			`)
+	blocks, err := p.parseAll()
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, b := range blocks {
+		t.Log(b.Keys)
+		t.Log(b.Tokens)
+	}
+	if len(blocks) != 1 {
+		t.Fatalf("Expect exactly one server block. Got %d.", len(blocks))
+	}
+	if actual, expected := blocks[0].Keys[0], "http://example.com"; expected != actual {
+		t.Errorf("Expected server name to be '%s' but was '%s'", expected, actual)
+	}
+	if len(blocks[0].Tokens) != 2 {
+		t.Fatalf("Server block should have tokens from import")
+	}
+	if actual, expected := blocks[0].Tokens["gzip"][0].Text, "gzip"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+	if actual, expected := blocks[0].Tokens["errors"][1].Text, "stderr"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+
+}

caddyhttp/basicauth/basicauth.go

@@ -1,8 +1,28 @@
-// Package basicauth implements HTTP Basic Authentication.
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package basicauth implements HTTP Basic Authentication for Caddy.
+//
+// This is useful for simple protections on a website, like requiring
+// a password to access an admin interface. This package assumes a
+// fairly small threat model.
 package basicauth
 
 import (
 	"bufio"
+	"context"
+	"crypto/sha1"
 	"crypto/subtle"
 	"fmt"
 	"io"
@@ -29,9 +49,8 @@ type BasicAuth struct {
 
 // ServeHTTP implements the httpserver.Handler interface.
 func (a BasicAuth) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-
-	var hasAuth bool
-	var isAuthenticated bool
+	var protected, isAuthenticated bool
+	var realm string
 
 	for _, rule := range a.Rules {
 		for _, res := range rule.Resources {
@@ -39,33 +58,46 @@ func (a BasicAuth) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error
 				continue
 			}
 
-			// Path matches; parse auth header
-			username, password, ok := r.BasicAuth()
-			hasAuth = true
+			// path matches; this endpoint is protected
+			protected = true
+			realm = rule.Realm
 
-			// Check credentials
+			// parse auth header
+			username, password, ok := r.BasicAuth()
+
+			// check credentials
 			if !ok ||
 				username != rule.Username ||
 				!rule.Password(password) {
-				//subtle.ConstantTimeCompare([]byte(password), []byte(rule.Password)) != 1 {
 				continue
 			}
 
-			// Flag set only on successful authentication
+			// by this point, authentication was successful
 			isAuthenticated = true
+
+			// let upstream middleware (e.g. fastcgi and cgi) know about authenticated
+			// user; this replaces the request with a wrapped instance
+			r = r.WithContext(context.WithValue(r.Context(),
+				httpserver.RemoteUserCtxKey, username))
+
+			// Provide username to be used in log by replacer
+			repl := httpserver.NewReplacer(r, nil, "-")
+			repl.Set("user", username)
 		}
 	}
 
-	if hasAuth {
-		if !isAuthenticated {
-			w.Header().Set("WWW-Authenticate", "Basic")
-			return http.StatusUnauthorized, nil
+	if protected && !isAuthenticated {
+		// browsers show a message that says something like:
+		// "The website says: <realm>"
+		// which is kinda dumb, but whatever.
+		if realm == "" {
+			realm = "Restricted"
 		}
-		// "It's an older code, sir, but it checks out. I was about to clear them."
-		return a.Next.ServeHTTP(w, r)
+		w.Header().Set("WWW-Authenticate", "Basic realm=\""+realm+"\"")
+		return http.StatusUnauthorized, nil
 	}
 
-	// Pass-thru when no paths match
+	// Pass-through when no paths match
 	return a.Next.ServeHTTP(w, r)
 }
 
@@ -76,6 +108,7 @@ type Rule struct {
 	Username  string
 	Password  func(string) bool
 	Resources []string
+	Realm     string // See RFC 1945 and RFC 2617, default: "Restricted"
 }
 
 // PasswordMatcher determines whether a password matches a rule.
@@ -140,9 +173,17 @@ func parseHtpasswd(pm map[string]PasswordMatcher, r io.Reader) error {
 }
 
 // PlainMatcher returns a PasswordMatcher that does a constant-time
-// byte-wise comparison.
+// byte comparison against the password passw.
 func PlainMatcher(passw string) PasswordMatcher {
+	// compare hashes of equal length instead of actual password
+	// to avoid leaking password length
+	passwHash := sha1.New()
+	passwHash.Write([]byte(passw))
+	passwSum := passwHash.Sum(nil)
 	return func(pw string) bool {
-		return subtle.ConstantTimeCompare([]byte(pw), []byte(passw)) == 1
+		pwHash := sha1.New()
+		pwHash.Write([]byte(pw))
+		pwSum := pwHash.Sum(nil)
+		return subtle.ConstantTimeCompare([]byte(pwSum), []byte(passwSum)) == 1
 	}
 }

caddyhttp/basicauth/basicauth_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package basicauth
 
 import (
@@ -14,54 +28,90 @@ import (
 )
 
 func TestBasicAuth(t *testing.T) {
-	rw := BasicAuth{
-		Next: httpserver.HandlerFunc(contentHandler),
-		Rules: []Rule{
-			{Username: "test", Password: PlainMatcher("ttest"), Resources: []string{"/testing"}},
+	var i int
+	// This handler is registered for tests in which the only authorized user is
+	// "okuser"
+	upstreamHandler := func(w http.ResponseWriter, r *http.Request) (int, error) {
+		remoteUser, _ := r.Context().Value(httpserver.RemoteUserCtxKey).(string)
+		if remoteUser != "okuser" {
+			t.Errorf("Test %d: expecting remote user 'okuser', got '%s'", i, remoteUser)
+		}
+		return http.StatusOK, nil
+	}
+	rws := []BasicAuth{
+		{
+			Next: httpserver.HandlerFunc(upstreamHandler),
+			Rules: []Rule{
+				{Username: "okuser", Password: PlainMatcher("okpass"),
+					Resources: []string{"/testing"}, Realm: "Resources"},
+			},
+		},
+		{
+			Next: httpserver.HandlerFunc(upstreamHandler),
+			Rules: []Rule{
+				{Username: "okuser", Password: PlainMatcher("okpass"),
+					Resources: []string{"/testing"}},
+			},
 		},
 	}
 
-	tests := []struct {
-		from   string
-		result int
-		cred   string
-	}{
-		{"/testing", http.StatusUnauthorized, "ttest:test"},
-		{"/testing", http.StatusOK, "test:ttest"},
-		{"/testing", http.StatusUnauthorized, ""},
+	type testType struct {
+		from     string
+		result   int
+		user     string
+		password string
 	}
 
-	for i, test := range tests {
+	tests := []testType{
+		{"/testing", http.StatusOK, "okuser", "okpass"},
+		{"/testing", http.StatusUnauthorized, "baduser", "okpass"},
+		{"/testing", http.StatusUnauthorized, "okuser", "badpass"},
+		{"/testing", http.StatusUnauthorized, "OKuser", "okpass"},
+		{"/testing", http.StatusUnauthorized, "OKuser", "badPASS"},
+		{"/testing", http.StatusUnauthorized, "", "okpass"},
+		{"/testing", http.StatusUnauthorized, "okuser", ""},
+		{"/testing", http.StatusUnauthorized, "", ""},
+	}
 
-		req, err := http.NewRequest("GET", test.from, nil)
-		if err != nil {
-			t.Fatalf("Test %d: Could not create HTTP request %v", i, err)
+	var test testType
+	for _, rw := range rws {
+		expectRealm := rw.Rules[0].Realm
+		if expectRealm == "" {
+			expectRealm = "Restricted" // Default if Realm not specified in rule
 		}
-		auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(test.cred))
-		req.Header.Set("Authorization", auth)
+		for i, test = range tests {
+			req, err := http.NewRequest("GET", test.from, nil)
+			if err != nil {
+				t.Fatalf("Test %d: Could not create HTTP request: %v", i, err)
+			}
+			req.SetBasicAuth(test.user, test.password)
 
-		rec := httptest.NewRecorder()
-		result, err := rw.ServeHTTP(rec, req)
-		if err != nil {
-			t.Fatalf("Test %d: Could not ServeHTTP %v", i, err)
-		}
-		if result != test.result {
-			t.Errorf("Test %d: Expected Header '%d' but was '%d'",
-				i, test.result, result)
-		}
-		if result == http.StatusUnauthorized {
-			headers := rec.Header()
-			if val, ok := headers["Www-Authenticate"]; ok {
-				if val[0] != "Basic" {
-					t.Errorf("Test %d, Www-Authenticate should be %s provided %s", i, "Basic", val[0])
+			rec := httptest.NewRecorder()
+			result, err := rw.ServeHTTP(rec, req)
+			if err != nil {
+				t.Fatalf("Test %d: Could not ServeHTTP: %v", i, err)
+			}
+			if result != test.result {
+				t.Errorf("Test %d: Expected status code %d but was %d",
+					i, test.result, result)
+			}
+			if test.result == http.StatusUnauthorized {
+				headers := rec.Header()
+				if val, ok := headers["Www-Authenticate"]; ok {
+					if got, want := val[0], "Basic realm=\""+expectRealm+"\""; got != want {
+						t.Errorf("Test %d: Www-Authenticate header should be '%s', got: '%s'", i, want, got)
+					}
+				} else {
+					t.Errorf("Test %d: response should have a 'Www-Authenticate' header", i)
 				}
 			} else {
-				t.Errorf("Test %d, should provide a header Www-Authenticate", i)
+				if req.Header.Get("Authorization") == "" {
+					// see issue #1508: https://github.com/mholt/caddy/issues/1508
+					t.Errorf("Test %d: Expected Authorization header to be retained after successful auth, but was empty", i)
+				}
 			}
 		}
-
 	}
-
 }
 
 func TestMultipleOverlappingRules(t *testing.T) {
@@ -121,7 +171,7 @@ md5:$apr1$l42y8rex$pOA2VJ0x/0TwaFeAF9nX61`
 
 	htfh, err := ioutil.TempFile("", "basicauth-")
 	if err != nil {
-		t.Skipf("Error creating temp file (%v), will skip htpassword test")
+		t.Skip("Error creating temp file, will skip htpassword test")
 		return
 	}
 	defer os.Remove(htfh.Name())

caddyhttp/basicauth/setup.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package basicauth
 
 import (
@@ -51,13 +65,6 @@ func basicAuthParse(c *caddy.Controller) ([]Rule, error) {
 			if rule.Password, err = passwordMatcher(rule.Username, args[1], cfg.Root); err != nil {
 				return rules, c.Errf("Get password matcher from %s: %v", c.Val(), err)
 			}
-
-			for c.NextBlock() {
-				rule.Resources = append(rule.Resources, c.Val())
-				if c.NextArg() {
-					return rules, c.Errf("Expecting only one resource per line (extra '%s')", c.Val())
-				}
-			}
 		case 3:
 			rule.Resources = append(rule.Resources, args[0])
 			rule.Username = args[1]
@@ -68,6 +75,29 @@ func basicAuthParse(c *caddy.Controller) ([]Rule, error) {
 			return rules, c.ArgErr()
 		}
 
+		// If nested block is present, process it here
+		for c.NextBlock() {
+			val := c.Val()
+			args = c.RemainingArgs()
+			switch len(args) {
+			case 0:
+				// Assume single argument is path resource
+				rule.Resources = append(rule.Resources, val)
+			case 1:
+				if val == "realm" {
+					if rule.Realm == "" {
+						rule.Realm = strings.Replace(args[0], `"`, `\"`, -1)
+					} else {
+						return rules, c.Errf("\"realm\" subdirective can only be specified once")
+					}
+				} else {
+					return rules, c.Errf("expecting \"realm\", got \"%s\"", val)
+				}
+			default:
+				return rules, c.ArgErr()
+			}
+		}
+
 		rules = append(rules, rule)
 	}
 
@@ -75,8 +105,9 @@ func basicAuthParse(c *caddy.Controller) ([]Rule, error) {
 }
 
 func passwordMatcher(username, passw, siteRoot string) (PasswordMatcher, error) {
-	if !strings.HasPrefix(passw, "htpasswd=") {
+	htpasswdPrefix := "htpasswd="
+	if !strings.HasPrefix(passw, htpasswdPrefix) {
 		return PlainMatcher(passw), nil
 	}
-	return GetHtpasswdMatcher(passw[9:], username, siteRoot)
+	return GetHtpasswdMatcher(passw[len(htpasswdPrefix):], username, siteRoot)
 }

caddyhttp/basicauth/setup_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package basicauth
 
 import (
@@ -64,12 +78,39 @@ md5:$apr1$l42y8rex$pOA2VJ0x/0TwaFeAF9nX61`
 		}`, false, "pwd", []Rule{
 			{Username: "user"},
 		}},
+		{`basicauth /resource1 user pwd {
+		}`, false, "pwd", []Rule{
+			{Username: "user", Resources: []string{"/resource1"}},
+		}},
+		{`basicauth /resource1 user pwd {
+			realm Resources
+		}`, false, "pwd", []Rule{
+			{Username: "user", Resources: []string{"/resource1"}, Realm: "Resources"},
+		}},
 		{`basicauth user pwd {
 			/resource1
 			/resource2
 		}`, false, "pwd", []Rule{
 			{Username: "user", Resources: []string{"/resource1", "/resource2"}},
 		}},
+		{`basicauth user pwd {
+			/resource1
+			/resource2
+			realm "Secure resources"
+		}`, false, "pwd", []Rule{
+			{Username: "user", Resources: []string{"/resource1", "/resource2"}, Realm: "Secure resources"},
+		}},
+		{`basicauth user pwd {
+			/resource1
+			realm "Secure resources"
+			realm Extra
+			/resource2
+		}`, true, "pwd", []Rule{}},
+		{`basicauth user pwd {
+			/resource1
+			foo "Resources"
+			/resource2
+		}`, true, "pwd", []Rule{}},
 		{`basicauth /resource user pwd`, false, "pwd", []Rule{
 			{Username: "user", Resources: []string{"/resource"}},
 		}},
@@ -109,6 +150,11 @@ md5:$apr1$l42y8rex$pOA2VJ0x/0TwaFeAF9nX61`
 					i, j, expectedRule.Username, actualRule.Username)
 			}
 
+			if actualRule.Realm != expectedRule.Realm {
+				t.Errorf("Test %d, rule %d: Expected realm '%s', got '%s'",
+					i, j, expectedRule.Realm, actualRule.Realm)
+			}
+
 			if strings.Contains(test.input, "htpasswd=") && skipHtpassword {
 				continue
 			}

caddyhttp/bind/bind.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package bind
 
 import (

caddyhttp/bind/bind_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package bind
 
 import (

caddyhttp/browse/browse.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package browse provides middleware for listing files in a directory
 // when directory path is requested instead of a specific file.
 package browse
@@ -20,6 +34,13 @@ import (
 	"github.com/mholt/caddy/caddyhttp/staticfiles"
 )
 
+const (
+	sortByName         = "name"
+	sortByNameDirFirst = "namedirfirst"
+	sortBySize         = "size"
+	sortByTime         = "time"
+)
+
 // Browse is an http.Handler that can show a file listing when
 // directories in the given paths are specified.
 type Browse struct {
@@ -30,8 +51,8 @@ type Browse struct {
 
 // Config is a configuration for browsing in a particular path.
 type Config struct {
-	PathScope string
-	Root      http.FileSystem
+	PathScope string // the base path the URL must match to enable browsing
+	Fs        staticfiles.FileServer
 	Variables interface{}
 	Template  *template.Template
 }
@@ -71,10 +92,15 @@ type Listing struct {
 	httpserver.Context
 }
 
-// BreadcrumbMap returns l.Path where every element is a map
-// of URLs and path segment names.
-func (l Listing) BreadcrumbMap() map[string]string {
-	result := map[string]string{}
+// Crumb represents part of a breadcrumb menu.
+type Crumb struct {
+	Link, Text string
+}
+
+// Breadcrumbs returns l.Path where every element maps
+// the link to the text to display.
+func (l Listing) Breadcrumbs() []Crumb {
+	var result []Crumb
 
 	if len(l.Path) == 0 {
 		return result
@@ -87,13 +113,12 @@ func (l Listing) BreadcrumbMap() map[string]string {
 	}
 
 	parts := strings.Split(lpath, "/")
-	for i, part := range parts {
-		if i == 0 && part == "" {
-			// Leading slash (root)
-			result["/"] = "/"
-			continue
+	for i := range parts {
+		txt := parts[i]
+		if i == 0 && parts[i] == "" {
+			txt = "/"
 		}
-		result[strings.Join(parts[:i+1], "/")] = part
+		result = append(result, Crumb{Link: strings.Repeat("../", len(parts)-i-1), Text: txt})
 	}
 
 	return result
@@ -101,12 +126,13 @@ func (l Listing) BreadcrumbMap() map[string]string {
 
 // FileInfo is the info about a particular file or directory
 type FileInfo struct {
-	IsDir   bool
-	Name    string
-	Size    int64
-	URL     string
-	ModTime time.Time
-	Mode    os.FileMode
+	Name      string
+	Size      int64
+	URL       string
+	ModTime   time.Time
+	Mode      os.FileMode
+	IsDir     bool
+	IsSymlink bool
 }
 
 // HumanSize returns the size of the file as a human-readable string
@@ -122,6 +148,7 @@ func (fi FileInfo) HumanModTime(format string) string {
 
 // Implement sorting for Listing
 type byName Listing
+type byNameDirFirst Listing
 type bySize Listing
 type byTime Listing
 
@@ -134,6 +161,22 @@ func (l byName) Less(i, j int) bool {
 	return strings.ToLower(l.Items[i].Name) < strings.ToLower(l.Items[j].Name)
 }
 
+// By Name Dir First
+func (l byNameDirFirst) Len() int      { return len(l.Items) }
+func (l byNameDirFirst) Swap(i, j int) { l.Items[i], l.Items[j] = l.Items[j], l.Items[i] }
+
+// Treat upper and lower case equally
+func (l byNameDirFirst) Less(i, j int) bool {
+
+	// if both are dir or file sort normally
+	if l.Items[i].IsDir == l.Items[j].IsDir {
+		return strings.ToLower(l.Items[i].Name) < strings.ToLower(l.Items[j].Name)
+	}
+
+	// always sort dir ahead of file
+	return l.Items[i].IsDir
+}
+
 // By Size
 func (l bySize) Len() int      { return len(l.Items) }
 func (l bySize) Swap(i, j int) { l.Items[i], l.Items[j] = l.Items[j], l.Items[i] }
@@ -141,12 +184,21 @@ func (l bySize) Swap(i, j int) { l.Items[i], l.Items[j] = l.Items[j], l.Items[i]
 const directoryOffset = -1 << 31 // = math.MinInt32
 func (l bySize) Less(i, j int) bool {
 	iSize, jSize := l.Items[i].Size, l.Items[j].Size
+
+	// Directory sizes depend on the filesystem implementation,
+	// which is opaque to a visitor, and should indeed does not change if the operator choses to change the fs.
+	// For a consistent user experience directories are pulled to the front…
 	if l.Items[i].IsDir {
-		iSize = directoryOffset + iSize
+		iSize = directoryOffset
 	}
 	if l.Items[j].IsDir {
-		jSize = directoryOffset + jSize
+		jSize = directoryOffset
 	}
+	// … and sorted by name.
+	if l.Items[i].IsDir && l.Items[j].IsDir {
+		return strings.ToLower(l.Items[i].Name) < strings.ToLower(l.Items[j].Name)
+	}
+
 	return iSize < jSize
 }
 
@@ -161,11 +213,13 @@ func (l Listing) applySort() {
 	// Check '.Order' to know how to sort
 	if l.Order == "desc" {
 		switch l.Sort {
-		case "name":
+		case sortByName:
 			sort.Sort(sort.Reverse(byName(l)))
-		case "size":
+		case sortByNameDirFirst:
+			sort.Sort(sort.Reverse(byNameDirFirst(l)))
+		case sortBySize:
 			sort.Sort(sort.Reverse(bySize(l)))
-		case "time":
+		case sortByTime:
 			sort.Sort(sort.Reverse(byTime(l)))
 		default:
 			// If not one of the above, do nothing
@@ -173,11 +227,13 @@ func (l Listing) applySort() {
 		}
 	} else { // If we had more Orderings we could add them here
 		switch l.Sort {
-		case "name":
+		case sortByName:
 			sort.Sort(byName(l))
-		case "size":
+		case sortByNameDirFirst:
+			sort.Sort(byNameDirFirst(l))
+		case sortBySize:
 			sort.Sort(bySize(l))
-		case "time":
+		case sortByTime:
 			sort.Sort(byTime(l))
 		default:
 			// If not one of the above, do nothing
@@ -186,7 +242,7 @@ func (l Listing) applySort() {
 	}
 }
 
-func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string) (Listing, bool) {
+func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string, config *Config) (Listing, bool) {
 	var (
 		fileinfos           []FileInfo
 		dirCount, fileCount int
@@ -196,29 +252,36 @@ func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string) (Listin
 	for _, f := range files {
 		name := f.Name()
 
-		for _, indexName := range staticfiles.IndexPages {
+		for _, indexName := range config.Fs.IndexPages {
 			if name == indexName {
 				hasIndexFile = true
 				break
 			}
 		}
 
-		if f.IsDir() {
+		isDir := f.IsDir() || isSymlinkTargetDir(f, urlPath, config)
+
+		if isDir {
 			name += "/"
 			dirCount++
 		} else {
 			fileCount++
 		}
 
+		if config.Fs.IsHidden(f) {
+			continue
+		}
+
 		url := url.URL{Path: "./" + name} // prepend with "./" to fix paths with ':' in the name
 
 		fileinfos = append(fileinfos, FileInfo{
-			IsDir:   f.IsDir(),
-			Name:    f.Name(),
-			Size:    f.Size(),
-			URL:     url.String(),
-			ModTime: f.ModTime().UTC(),
-			Mode:    f.Mode(),
+			IsDir:     isDir,
+			IsSymlink: isSymlink(f),
+			Name:      f.Name(),
+			Size:      f.Size(),
+			URL:       url.String(),
+			ModTime:   f.ModTime().UTC(),
+			Mode:      f.Mode(),
 		})
 	}
 
@@ -232,22 +295,49 @@ func directoryListing(files []os.FileInfo, canGoUp bool, urlPath string) (Listin
 	}, hasIndexFile
 }
 
+// isSymlink return true if f is a symbolic link
+func isSymlink(f os.FileInfo) bool {
+	return f.Mode()&os.ModeSymlink != 0
+}
+
+// isSymlinkTargetDir return true if f's symbolic link target
+// is a directory. Return false if not a symbolic link.
+func isSymlinkTargetDir(f os.FileInfo, urlPath string, config *Config) bool {
+	if !isSymlink(f) {
+		return false
+	}
+
+	// a bit strange, but we want Stat thru the jailed filesystem to be safe
+	target, err := config.Fs.Root.Open(path.Join(urlPath, f.Name()))
+	if err != nil {
+		return false
+	}
+	defer target.Close()
+	targetInfo, err := target.Stat()
+	if err != nil {
+		return false
+	}
+
+	return targetInfo.IsDir()
+}
+
 // ServeHTTP determines if the request is for this plugin, and if all prerequisites are met.
 // If so, control is handed over to ServeListing.
 func (b Browse) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	var bc *Config
 	// See if there's a browse configuration to match the path
+	var bc *Config
 	for i := range b.Configs {
 		if httpserver.Path(r.URL.Path).Matches(b.Configs[i].PathScope) {
 			bc = &b.Configs[i]
-			goto inScope
+			break
 		}
 	}
-	return b.Next.ServeHTTP(w, r)
-inScope:
+	if bc == nil {
+		return b.Next.ServeHTTP(w, r)
+	}
 
 	// Browse works on existing directories; delegate everything else
-	requestedFilepath, err := bc.Root.Open(r.URL.Path)
+	requestedFilepath, err := bc.Fs.Root.Open(r.URL.Path)
 	if err != nil {
 		switch {
 		case os.IsPermission(err):
@@ -287,15 +377,20 @@ inScope:
 
 	// Browsing navigation gets messed up if browsing a directory
 	// that doesn't end in "/" (which it should, anyway)
-	if !strings.HasSuffix(r.URL.Path, "/") {
-		http.Redirect(w, r, r.URL.Path+"/", http.StatusTemporaryRedirect)
-		return 0, nil
+	u := *r.URL
+	if u.Path == "" {
+		u.Path = "/"
+	}
+	if u.Path[len(u.Path)-1] != '/' {
+		u.Path += "/"
+		http.Redirect(w, r, u.String(), http.StatusMovedPermanently)
+		return http.StatusMovedPermanently, nil
 	}
 
 	return b.ServeListing(w, r, requestedFilepath, bc)
 }
 
-func (b Browse) loadDirectoryContents(requestedFilepath http.File, urlPath string) (*Listing, bool, error) {
+func (b Browse) loadDirectoryContents(requestedFilepath http.File, urlPath string, config *Config) (*Listing, bool, error) {
 	files, err := requestedFilepath.Readdir(-1)
 	if err != nil {
 		return nil, false, err
@@ -312,7 +407,7 @@ func (b Browse) loadDirectoryContents(requestedFilepath http.File, urlPath strin
 	}
 
 	// Assemble listing of directory contents
-	listing, hasIndex := directoryListing(files, canGoUp, urlPath)
+	listing, hasIndex := directoryListing(files, canGoUp, urlPath, config)
 
 	return &listing, hasIndex, nil
 }
@@ -327,11 +422,11 @@ func (b Browse) handleSortOrder(w http.ResponseWriter, r *http.Request, scope st
 	// If the query 'sort' or 'order' is empty, use defaults or any values previously saved in Cookies
 	switch sort {
 	case "":
-		sort = "name"
+		sort = sortByNameDirFirst
 		if sortCookie, sortErr := r.Cookie("sort"); sortErr == nil {
 			sort = sortCookie.Value
 		}
-	case "name", "size", "type":
+	case sortByName, sortByNameDirFirst, sortBySize, sortByTime:
 		http.SetCookie(w, &http.Cookie{Name: "sort", Value: sort, Path: scope, Secure: r.TLS != nil})
 	}
 
@@ -357,7 +452,7 @@ func (b Browse) handleSortOrder(w http.ResponseWriter, r *http.Request, scope st
 
 // ServeListing returns a formatted view of 'requestedFilepath' contents'.
 func (b Browse) ServeListing(w http.ResponseWriter, r *http.Request, requestedFilepath http.File, bc *Config) (int, error) {
-	listing, containsIndex, err := b.loadDirectoryContents(requestedFilepath, r.URL.Path)
+	listing, containsIndex, err := b.loadDirectoryContents(requestedFilepath, r.URL.Path, bc)
 	if err != nil {
 		switch {
 		case os.IsPermission(err):
@@ -372,7 +467,7 @@ func (b Browse) ServeListing(w http.ResponseWriter, r *http.Request, requestedFi
 		return b.Next.ServeHTTP(w, r)
 	}
 	listing.Context = httpserver.Context{
-		Root: bc.Root,
+		Root: bc.Fs.Root,
 		Req:  r,
 		URL:  r.URL,
 	}

caddyhttp/browse/browse_test.go

@@ -1,20 +1,42 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package browse
 
 import (
+	"context"
 	"encoding/json"
+	"fmt"
+	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
 	"os"
 	"path/filepath"
+	"runtime"
 	"sort"
+	"strings"
 	"testing"
 	"text/template"
 	"time"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
+	"github.com/mholt/caddy/caddyhttp/staticfiles"
 )
 
+const testDirPrefix = "caddy_browse_test"
+
 func TestSort(t *testing.T) {
 	// making up []fileInfo with bogus values;
 	// to be used to make up our "listing"
@@ -68,6 +90,13 @@ func TestSort(t *testing.T) {
 		t.Errorf("The listing isn't time sorted: %v", listing.Items)
 	}
 
+	// sort by name dir first
+	listing.Sort = "namedirfirst"
+	listing.applySort()
+	if !sort.IsSorted(byNameDirFirst(listing)) {
+		t.Errorf("The listing isn't namedirfirst sorted: %v", listing.Items)
+	}
+
 	// reverse by name
 	listing.Sort = "name"
 	listing.Order = "desc"
@@ -91,12 +120,20 @@ func TestSort(t *testing.T) {
 	if !isReversed(byTime(listing)) {
 		t.Errorf("The listing isn't reversed by time: %v", listing.Items)
 	}
+
+	// reverse by name dir first
+	listing.Sort = "namedirfirst"
+	listing.Order = "desc"
+	listing.applySort()
+	if !isReversed(byNameDirFirst(listing)) {
+		t.Errorf("The listing isn't reversed by namedirfirst: %v", listing.Items)
+	}
 }
 
 func TestBrowseHTTPMethods(t *testing.T) {
 	tmpl, err := template.ParseFiles("testdata/photos.tpl")
 	if err != nil {
-		t.Fatalf("An error occured while parsing the template: %v", err)
+		t.Fatalf("An error occurred while parsing the template: %v", err)
 	}
 
 	b := Browse{
@@ -106,8 +143,10 @@ func TestBrowseHTTPMethods(t *testing.T) {
 		Configs: []Config{
 			{
 				PathScope: "/photos",
-				Root:      http.Dir("./testdata"),
-				Template:  tmpl,
+				Fs: staticfiles.FileServer{
+					Root: http.Dir("./testdata"),
+				},
+				Template: tmpl,
 			},
 		},
 	}
@@ -123,6 +162,8 @@ func TestBrowseHTTPMethods(t *testing.T) {
 		if err != nil {
 			t.Fatalf("Test: Could not create HTTP request: %v", err)
 		}
+		ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+		req = req.WithContext(ctx)
 
 		code, _ := b.ServeHTTP(rec, req)
 		if code != expected {
@@ -134,7 +175,7 @@ func TestBrowseHTTPMethods(t *testing.T) {
 func TestBrowseTemplate(t *testing.T) {
 	tmpl, err := template.ParseFiles("testdata/photos.tpl")
 	if err != nil {
-		t.Fatalf("An error occured while parsing the template: %v", err)
+		t.Fatalf("An error occurred while parsing the template: %v", err)
 	}
 
 	b := Browse{
@@ -145,8 +186,11 @@ func TestBrowseTemplate(t *testing.T) {
 		Configs: []Config{
 			{
 				PathScope: "/photos",
-				Root:      http.Dir("./testdata"),
-				Template:  tmpl,
+				Fs: staticfiles.FileServer{
+					Root: http.Dir("./testdata"),
+					Hide: []string{"photos/hidden.html"},
+				},
+				Template: tmpl,
 			},
 		},
 	}
@@ -155,6 +199,8 @@ func TestBrowseTemplate(t *testing.T) {
 	if err != nil {
 		t.Fatalf("Test: Could not create HTTP request: %v", err)
 	}
+	ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+	req = req.WithContext(ctx)
 
 	rec := httptest.NewRecorder()
 
@@ -174,6 +220,8 @@ func TestBrowseTemplate(t *testing.T) {
 
 <h1>/photos/</h1>
 
+<a href="./test1/">test1</a><br>
+
 <a href="./test.html">test.html</a><br>
 
 <a href="./test2.html">test2.html</a><br>
@@ -185,7 +233,7 @@ func TestBrowseTemplate(t *testing.T) {
 `
 
 	if respBody != expectedBody {
-		t.Fatalf("Expected body: %v got: %v", expectedBody, respBody)
+		t.Fatalf("Expected body: '%v' got: '%v'", expectedBody, respBody)
 	}
 
 }
@@ -193,13 +241,15 @@ func TestBrowseTemplate(t *testing.T) {
 func TestBrowseJson(t *testing.T) {
 	b := Browse{
 		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
-			t.Fatalf("Next shouldn't be called")
+			t.Fatalf("Next shouldn't be called: %s", r.URL)
 			return 0, nil
 		}),
 		Configs: []Config{
 			{
 				PathScope: "/photos/",
-				Root:      http.Dir("./testdata"),
+				Fs: staticfiles.FileServer{
+					Root: http.Dir("./testdata"),
+				},
 			},
 		},
 	}
@@ -246,6 +296,9 @@ func TestBrowseJson(t *testing.T) {
 			Mode:    f.Mode(),
 		})
 	}
+
+	// Test that sort=name returns correct listing.
+
 	listing := Listing{Items: fileinfos} // this listing will be used for validation inside the tests
 
 	tests := []struct {
@@ -258,50 +311,53 @@ func TestBrowseJson(t *testing.T) {
 	}{
 		//test case 1: testing for default sort and  order and without the limit parameter, default sort is by name and the default order is ascending
 		//without the limit query entire listing will be produced
-		{"/", "", "", -1, false, listing.Items},
+		{"/?sort=name", "", "", -1, false, listing.Items},
 		//test case 2: limit is set to 1, orderBy and sortBy is default
-		{"/?limit=1", "", "", 1, false, listing.Items[:1]},
+		{"/?limit=1&sort=name", "", "", 1, false, listing.Items[:1]},
 		//test case 3 : if the listing request is bigger than total size of listing then it should return everything
-		{"/?limit=100000000", "", "", 100000000, false, listing.Items},
+		{"/?limit=100000000&sort=name", "", "", 100000000, false, listing.Items},
 		//test case 4 : testing for negative limit
-		{"/?limit=-1", "", "", -1, false, listing.Items},
+		{"/?limit=-1&sort=name", "", "", -1, false, listing.Items},
 		//test case 5 : testing with limit set to -1 and order set to descending
-		{"/?limit=-1&order=desc", "", "desc", -1, false, listing.Items},
+		{"/?limit=-1&order=desc&sort=name", "", "desc", -1, false, listing.Items},
 		//test case 6 : testing with limit set to 2 and order set to descending
-		{"/?limit=2&order=desc", "", "desc", 2, false, listing.Items},
+		{"/?limit=2&order=desc&sort=name", "", "desc", 2, false, listing.Items},
 		//test case 7 : testing with limit set to 3 and order set to descending
-		{"/?limit=3&order=desc", "", "desc", 3, false, listing.Items},
+		{"/?limit=3&order=desc&sort=name", "", "desc", 3, false, listing.Items},
 		//test case 8 : testing with limit set to 3 and order set to ascending
-		{"/?limit=3&order=asc", "", "asc", 3, false, listing.Items},
+		{"/?limit=3&order=asc&sort=name", "", "asc", 3, false, listing.Items},
 		//test case 9 : testing with limit set to 1111111 and order set to ascending
-		{"/?limit=1111111&order=asc", "", "asc", 1111111, false, listing.Items},
+		{"/?limit=1111111&order=asc&sort=name", "", "asc", 1111111, false, listing.Items},
 		//test case 10 : testing with limit set to default and order set to ascending and sorting by size
-		{"/?order=asc&sort=size", "size", "asc", -1, false, listing.Items},
+		{"/?order=asc&sort=size&sort=name", "size", "asc", -1, false, listing.Items},
 		//test case 11 : testing with limit set to default and order set to ascending and sorting by last modified
-		{"/?order=asc&sort=time", "time", "asc", -1, false, listing.Items},
+		{"/?order=asc&sort=time&sort=name", "time", "asc", -1, false, listing.Items},
 		//test case 12 : testing with limit set to 1 and order set to ascending and sorting by last modified
-		{"/?order=asc&sort=time&limit=1", "time", "asc", 1, false, listing.Items},
+		{"/?order=asc&sort=time&limit=1&sort=name", "time", "asc", 1, false, listing.Items},
 		//test case 13 : testing with limit set to -100 and order set to ascending and sorting by last modified
-		{"/?order=asc&sort=time&limit=-100", "time", "asc", -100, false, listing.Items},
+		{"/?order=asc&sort=time&limit=-100&sort=name", "time", "asc", -100, false, listing.Items},
 		//test case 14 : testing with limit set to -100 and order set to ascending and sorting by size
-		{"/?order=asc&sort=size&limit=-100", "size", "asc", -100, false, listing.Items},
+		{"/?order=asc&sort=size&limit=-100&sort=name", "size", "asc", -100, false, listing.Items},
 	}
 
 	for i, test := range tests {
 		var marsh []byte
 		req, err := http.NewRequest("GET", "/photos"+test.QueryURL, nil)
-
-		if err == nil && test.shouldErr {
-			t.Errorf("Test %d didn't error, but it should have", i)
-		} else if err != nil && !test.shouldErr {
-			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
+		if err != nil && !test.shouldErr {
+			t.Errorf("Test %d errored when making request, but it shouldn't have; got '%v'", i, err)
 		}
+		ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+		req = req.WithContext(ctx)
 
 		req.Header.Set("Accept", "application/json")
 		rec := httptest.NewRecorder()
 
 		code, err := b.ServeHTTP(rec, req)
-
+		if err == nil && test.shouldErr {
+			t.Errorf("Test %d didn't error, but it should have", i)
+		} else if err != nil && !test.shouldErr {
+			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
+		}
 		if code != http.StatusOK {
 			t.Fatalf("In test %d: Wrong status, expected %d, got %d", i, http.StatusOK, code)
 		}
@@ -353,3 +409,217 @@ func isReversed(data sort.Interface) bool {
 	}
 	return true
 }
+
+func TestBrowseRedirect(t *testing.T) {
+	testCases := []struct {
+		url        string
+		statusCode int
+		returnCode int
+		location   string
+	}{
+		{
+			"http://www.example.com/photos",
+			http.StatusMovedPermanently,
+			http.StatusMovedPermanently,
+			"http://www.example.com/photos/",
+		},
+		{
+			"/photos",
+			http.StatusMovedPermanently,
+			http.StatusMovedPermanently,
+			"/photos/",
+		},
+	}
+
+	for i, tc := range testCases {
+		b := Browse{
+			Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+				t.Fatalf("Test %d - Next shouldn't be called", i)
+				return 0, nil
+			}),
+			Configs: []Config{
+				{
+					PathScope: "/photos",
+					Fs: staticfiles.FileServer{
+						Root: http.Dir("./testdata"),
+					},
+				},
+			},
+		}
+
+		req, err := http.NewRequest("GET", tc.url, nil)
+		if err != nil {
+			t.Fatalf("Test %d - could not create HTTP request: %v", i, err)
+		}
+		ctx := context.WithValue(req.Context(), httpserver.OriginalURLCtxKey, *req.URL)
+		req = req.WithContext(ctx)
+
+		rec := httptest.NewRecorder()
+
+		returnCode, _ := b.ServeHTTP(rec, req)
+		if returnCode != tc.returnCode {
+			t.Fatalf("Test %d - wrong return code, expected %d, got %d",
+				i, tc.returnCode, returnCode)
+		}
+
+		if got := rec.Code; got != tc.statusCode {
+			t.Errorf("Test %d - wrong status, expected %d, got %d",
+				i, tc.statusCode, got)
+		}
+
+		if got := rec.Header().Get("Location"); got != tc.location {
+			t.Errorf("Test %d - wrong Location header, expected %s, got %s",
+				i, tc.location, got)
+		}
+	}
+}
+
+func TestDirSymlink(t *testing.T) {
+	if runtime.GOOS == "windows" {
+		// Windows support for symlinks is limited, and we had a hard time getting
+		// all these tests to pass with the permissions of CI; so just skip them
+		fmt.Println("Skipping browse symlink tests on Windows...")
+		return
+	}
+
+	testCases := []struct {
+		source       string
+		target       string
+		pathScope    string
+		url          string
+		expectedName string
+		expectedURL  string
+	}{
+		// test case can expect a directory "dir" and a symlink to it called "symlink"
+
+		{"dir", "$TMP/rel_symlink_to_dir", "/", "/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/abs_symlink_to_dir", "/", "/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"../../dir", "$TMP/sub/dir/rel_symlink_to_dir", "/", "/sub/dir/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/sub/dir/abs_symlink_to_dir", "/", "/sub/dir/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"../../dir", "$TMP/with/scope/rel_symlink_to_dir", "/with/scope", "/with/scope/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/with/scope/abs_symlink_to_dir", "/with/scope", "/with/scope/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"../../../../dir", "$TMP/with/scope/sub/dir/rel_symlink_to_dir", "/with/scope", "/with/scope/sub/dir/",
+			"rel_symlink_to_dir", "./rel_symlink_to_dir/"},
+		{"$TMP/dir", "$TMP/with/scope/sub/dir/abs_symlink_to_dir", "/with/scope", "/with/scope/sub/dir/",
+			"abs_symlink_to_dir", "./abs_symlink_to_dir/"},
+
+		{"symlink", "$TMP/rel_symlink_to_symlink", "/", "/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/abs_symlink_to_symlink", "/", "/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+
+		{"../../symlink", "$TMP/sub/dir/rel_symlink_to_symlink", "/", "/sub/dir/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/sub/dir/abs_symlink_to_symlink", "/", "/sub/dir/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+
+		{"../../symlink", "$TMP/with/scope/rel_symlink_to_symlink", "/with/scope", "/with/scope/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/with/scope/abs_symlink_to_symlink", "/with/scope", "/with/scope/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+
+		{"../../../../symlink", "$TMP/with/scope/sub/dir/rel_symlink_to_symlink", "/with/scope", "/with/scope/sub/dir/",
+			"rel_symlink_to_symlink", "./rel_symlink_to_symlink/"},
+		{"$TMP/symlink", "$TMP/with/scope/sub/dir/abs_symlink_to_symlink", "/with/scope", "/with/scope/sub/dir/",
+			"abs_symlink_to_symlink", "./abs_symlink_to_symlink/"},
+	}
+
+	for i, tc := range testCases {
+		func() {
+			tmpdir, err := ioutil.TempDir("", testDirPrefix)
+			if err != nil {
+				t.Fatalf("failed to create test directory: %v", err)
+			}
+			defer os.RemoveAll(tmpdir)
+
+			if err := os.MkdirAll(filepath.Join(tmpdir, "dir"), 0755); err != nil {
+				t.Fatalf("failed to create test dir 'dir': %v", err)
+			}
+			if err := os.Symlink("dir", filepath.Join(tmpdir, "symlink")); err != nil {
+				t.Fatalf("failed to create test symlink 'symlink': %v", err)
+			}
+
+			sourceResolved := strings.Replace(tc.source, "$TMP", tmpdir, -1)
+			targetResolved := strings.Replace(tc.target, "$TMP", tmpdir, -1)
+
+			if err := os.MkdirAll(filepath.Dir(sourceResolved), 0755); err != nil {
+				t.Fatalf("failed to create source symlink dir: %v", err)
+			}
+			if err := os.MkdirAll(filepath.Dir(targetResolved), 0755); err != nil {
+				t.Fatalf("failed to create target symlink dir: %v", err)
+			}
+			if err := os.Symlink(sourceResolved, targetResolved); err != nil {
+				t.Fatalf("failed to create test symlink: %v", err)
+			}
+
+			b := Browse{
+				Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+					t.Fatalf("Test %d - Next shouldn't be called", i)
+					return 0, nil
+				}),
+				Configs: []Config{
+					{
+						PathScope: tc.pathScope,
+						Fs: staticfiles.FileServer{
+							Root: http.Dir(tmpdir),
+						},
+					},
+				},
+			}
+
+			req, err := http.NewRequest("GET", tc.url, nil)
+			req.Header.Add("Accept", "application/json")
+			if err != nil {
+				t.Fatalf("Test %d - could not create HTTP request: %v", i, err)
+			}
+
+			rec := httptest.NewRecorder()
+
+			returnCode, _ := b.ServeHTTP(rec, req)
+			if returnCode != http.StatusOK {
+				t.Fatalf("Test %d - wrong return code, expected %d, got %d",
+					i, http.StatusOK, returnCode)
+			}
+
+			type jsonEntry struct {
+				Name      string
+				IsDir     bool
+				IsSymlink bool
+				URL       string
+			}
+			var entries []jsonEntry
+			if err := json.Unmarshal(rec.Body.Bytes(), &entries); err != nil {
+				t.Fatalf("Test %d - failed to parse json: %v", i, err)
+			}
+
+			found := false
+			for _, e := range entries {
+				if e.Name != tc.expectedName {
+					continue
+				}
+				found = true
+				if !e.IsDir {
+					t.Errorf("Test %d - expected to be a dir, got %v", i, e.IsDir)
+				}
+				if !e.IsSymlink {
+					t.Errorf("Test %d - expected to be a symlink, got %v", i, e.IsSymlink)
+				}
+				if e.URL != tc.expectedURL {
+					t.Errorf("Test %d - wrong URL, expected %v, got %v", i, tc.expectedURL, e.URL)
+				}
+			}
+			if !found {
+				t.Errorf("Test %d - failed, could not find name %v", i, tc.expectedName)
+			}
+		}()
+	}
+}

caddyhttp/browse/setup.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package browse
 
 import (
@@ -8,6 +22,7 @@ import (
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
+	"github.com/mholt/caddy/caddyhttp/staticfiles"
 )
 
 func init() {
@@ -61,15 +76,11 @@ func browseParse(c *caddy.Controller) ([]Config, error) {
 		} else {
 			bc.PathScope = "/"
 		}
-		bc.Root = http.Dir(cfg.Root)
-		theRoot, err := bc.Root.Open("/") // catch a missing path early
-		if err != nil {
-			return configs, err
-		}
-		defer theRoot.Close()
-		_, err = theRoot.Readdir(-1)
-		if err != nil {
-			return configs, err
+
+		bc.Fs = staticfiles.FileServer{
+			Root:       http.Dir(cfg.Root),
+			Hide:       cfg.HiddenFiles,
+			IndexPages: cfg.IndexPages,
 		}
 
 		// Second argument would be the template file to use
@@ -105,7 +116,8 @@ func browseParse(c *caddy.Controller) ([]Config, error) {
 const defaultTemplate = `<!DOCTYPE html>
 <html>
 	<head>
-		<title>{{.Name}}</title>
+		<title>{{html .Name}}</title>
+		<meta charset="utf-8">
 		<meta name="viewport" content="width=device-width, initial-scale=1.0">
 <style>
 * { padding: 0; margin: 0; }
@@ -153,16 +165,22 @@ h1 {
 	white-space: nowrap;
 	overflow-x: hidden;
 	text-overflow: ellipsis;
+	color: #999;
 }
 
 h1 a {
-	color: inherit;
+	color: #000;
+	margin: 0 4px;
 }
 
 h1 a:hover {
 	text-decoration: underline;
 }
 
+h1 a:first-child {
+	margin: 0;
+}
+
 main {
 	display: block;
 }
@@ -171,14 +189,19 @@ main {
 	font-size: 12px;
 	font-family: Verdana, sans-serif;
 	border-bottom: 1px solid #9C9C9C;
-	padding-top: 15px;
-	padding-bottom: 15px;
+	padding-top: 10px;
+	padding-bottom: 10px;
 }
 
 .meta-item {
 	margin-right: 1em;
 }
 
+#filter {
+	padding: 4px;
+	border: 1px solid #CCC;
+}
+
 table {
 	width: 100%;
 	border-collapse: collapse;
@@ -214,11 +237,16 @@ th svg {
 }
 
 td {
+	white-space: nowrap;
 	font-size: 14px;
 }
 
 td:first-child {
-	width: 50%;
+	width: 100%;
+}
+
+td:nth-child(2) {
+	padding: 0 20px 0 20px;
 }
 
 th:last-child,
@@ -238,6 +266,30 @@ td .goup {
 	white-space: pre-wrap;
 }
 
+.icon {
+	margin-right: 5px;
+}
+
+.icon.sort {
+	display: inline-block;
+	width: 1em;
+	height: 1em;
+	position: relative;
+	top: .2em;
+}
+
+.icon.sort .top {
+	position: absolute;
+	left: 0;
+	top: -1px;
+}
+
+.icon.sort .bottom {
+	position: absolute;
+	bottom: -1px;
+	left: 0;
+}
+
 footer {
 	padding: 40px 20px;
 	font-size: 12px;
@@ -258,6 +310,18 @@ footer {
 		padding-right: 5%;
 		text-align: right;
 	}
+
+	h1 {
+		color: #000;
+	}
+
+	h1 a {
+		margin: 0;
+	}
+
+	#filter {
+		max-width: 100px;
+	}
 }
 </style>
 	</head>
@@ -265,45 +329,32 @@ footer {
 		<svg version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" height="0" width="0" style="position: absolute;">
 			<defs>
 				<!-- Folder -->
-				<linearGradient id="f" y2="640" gradientUnits="userSpaceOnUse" x2="244.84" gradientTransform="matrix(.97319 0 0 1.0135 -.50695 -13.679)" y1="415.75" x1="244.84">
-					<stop stop-color="#b3ddfd" offset="0"/>
-					<stop stop-color="#69c" offset="1"/>
-				</linearGradient>
-				<linearGradient id="e" y2="571.06" gradientUnits="userSpaceOnUse" x2="238.03" gradientTransform="translate(0,2)" y1="346.05" x1="236.26">
-					<stop stop-color="#ace" offset="0"/>
-					<stop stop-color="#369" offset="1"/>
-				</linearGradient>
-				<g id="folder" transform="translate(-266.06 -193.36)">
-					<g transform="matrix(.066019 0 0 .066019 264.2 170.93)">
-						<g transform="matrix(1.4738 0 0 1.4738 -52.053 -166.93)">
-							<path fill="#69c" d="m98.424 343.78c-11.08 0-20 8.92-20 20v48.5 33.719 105.06c0 11.08 8.92 20 20 20h279.22c11.08 0 20-8.92 20-20v-138.78c0-11.08-8.92-20-20-20h-117.12c-7.5478-1.1844-9.7958-6.8483-10.375-11.312v-5.625-11.562c0-11.08-8.92-20-20-20h-131.72z"/>
-							<rect rx="12.885" ry="12.199" height="227.28" width="366.69" y="409.69" x="54.428" fill="#369"/>
-							<path fill="url(#e)" d="m98.424 345.78c-11.08 0-20 8.92-20 20v48.5 33.719 105.06c0 11.08 8.92 20 20 20h279.22c11.08 0 20-8.92 20-20v-138.78c0-11.08-8.92-20-20-20h-117.12c-7.5478-1.1844-9.7958-6.8483-10.375-11.312v-5.625-11.562c0-11.08-8.92-20-20-20h-131.72z"/>
-							<rect rx="12.885" ry="12.199" height="227.28" width="366.69" y="407.69" x="54.428" fill="url(#f)"/>
+				<g id="folder" fill-rule="nonzero" fill="none">
+					<path d="M285.22 37.55h-142.6L110.9 0H31.7C14.25 0 0 16.9 0 37.55v75.1h316.92V75.1c0-20.65-14.26-37.55-31.7-37.55z" fill="#FFA000"/>
+					<path d="M285.22 36H31.7C14.25 36 0 50.28 0 67.74v158.7c0 17.47 14.26 31.75 31.7 31.75H285.2c17.44 0 31.7-14.3 31.7-31.75V67.75c0-17.47-14.26-31.75-31.7-31.75z" fill="#FFCA28"/>
+				</g>
+				<g id="folder-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+					<g id="folder-shortcut-group" fill-rule="nonzero">
+						<g id="folder-shortcut-shape">
+							<path d="M285.224876,37.5486902 L142.612438,37.5486902 L110.920785,0 L31.6916529,0 C14.2612438,0 0,16.8969106 0,37.5486902 L0,112.646071 L316.916529,112.646071 L316.916529,75.0973805 C316.916529,54.4456008 302.655285,37.5486902 285.224876,37.5486902 Z" id="Shape" fill="#FFA000"></path>
+							<path d="M285.224876,36 L31.6916529,36 C14.2612438,36 0,50.2838568 0,67.7419039 L0,226.451424 C0,243.909471 14.2612438,258.193328 31.6916529,258.193328 L285.224876,258.193328 C302.655285,258.193328 316.916529,243.909471 316.916529,226.451424 L316.916529,67.7419039 C316.916529,50.2838568 302.655285,36 285.224876,36 Z" id="Shape" fill="#FFCA28"></path>
 						</g>
+						<path d="M126.154134,250.559184 C126.850974,251.883673 127.300549,253.006122 127.772602,254.106122 C128.469442,255.206122 128.919016,256.104082 129.638335,257.002041 C130.559962,258.326531 131.728855,259 133.100057,259 C134.493737,259 135.415364,258.55102 136.112204,257.67551 C136.809044,257.002041 137.258619,255.902041 137.258619,254.577551 C137.258619,253.904082 137.258619,252.804082 137.033832,251.457143 C136.786566,249.908163 136.561779,249.032653 136.561779,248.583673 C136.089726,242.814286 135.864939,237.920408 135.864939,233.273469 C135.864939,225.057143 136.786566,217.514286 138.180246,210.846939 C139.798713,204.202041 141.889234,198.634694 144.429328,193.763265 C147.216689,188.869388 150.678411,184.873469 154.836973,181.326531 C158.995535,177.779592 163.626149,174.883673 168.481552,172.661224 C173.336954,170.438776 179.113983,168.665306 185.587852,167.340816 C192.061722,166.218367 198.760378,165.342857 205.481514,164.669388 C212.18017,164.220408 219.598146,163.995918 228.162535,163.995918 L246.055591,163.995918 L246.055591,195.514286 C246.055591,197.736735 246.752431,199.510204 248.370899,201.059184 C250.214153,202.608163 252.079886,203.506122 254.372715,203.506122 C256.463236,203.506122 258.531277,202.608163 260.172223,201.059184 L326.102289,137.797959 C327.720757,136.24898 328.642384,134.47551 328.642384,132.253061 C328.642384,130.030612 327.720757,128.257143 326.102289,126.708163 L260.172223,63.4469388 C258.553756,61.8979592 256.463236,61 254.395194,61 C252.079886,61 250.236632,61.8979592 248.393377,63.4469388 C246.77491,64.9959184 246.07807,66.7693878 246.07807,68.9918367 L246.07807,100.510204 L228.162535,100.510204 C166.863084,100.510204 129.166282,117.167347 115.274437,150.459184 C110.666301,161.54898 108.350993,175.310204 108.350993,191.742857 C108.350993,205.279592 113.903236,223.912245 124.760454,247.438776 C125.00772,248.112245 125.457294,249.010204 126.154134,250.559184 Z" id="Shape" fill="#FFFFFF" transform="translate(218.496689, 160.000000) scale(-1, 1) translate(-218.496689, -160.000000) "></path>
 					</g>
 				</g>
 
 				<!-- File -->
-				<linearGradient id="a">
-					<stop stop-color="#cbcbcb" offset="0"/>
-					<stop stop-color="#f0f0f0" offset=".34923"/>
-					<stop stop-color="#e2e2e2" offset="1"/>
-				</linearGradient>
-				<linearGradient id="d" y2="686.15" xlink:href="#a" gradientUnits="userSpaceOnUse" y1="207.83" gradientTransform="matrix(.28346 0 0 .31053 -608.52 485.11)" x2="380.1" x1="749.25"/>
-				<linearGradient id="c" y2="287.74" xlink:href="#a" gradientUnits="userSpaceOnUse" y1="169.44" gradientTransform="matrix(.28342 0 0 .31057 -608.52 485.11)" x2="622.33" x1="741.64"/>
-				<linearGradient id="b" y2="418.54" gradientUnits="userSpaceOnUse" y1="236.13" gradientTransform="matrix(.29343 0 0 .29999 -608.52 485.11)" x2="330.88" x1="687.96">
-					<stop stop-color="#fff" offset="0"/>
-					<stop stop-color="#fff" stop-opacity="0" offset="1"/>
-				</linearGradient>
-				<g id="file" transform="translate(-278.15 -216.59)">
-					<g fill-rule="evenodd" transform="matrix(.19775 0 0 .19775 381.05 112.68)">
-						<path d="m-520.17 525.5v36.739 36.739 36.739 36.739h33.528 33.528 33.528 33.528v-36.739-36.739-36.739l-33.528-36.739h-33.528-33.528-33.528z" stroke-opacity=".36478" stroke-width=".42649" fill="#fff"/>
-						<g>
-							<path d="m-520.11 525.68v36.739 36.739 36.739 36.739h33.528 33.528 33.528 33.528v-36.739-36.739-36.739l-33.528-36.739h-33.528-33.528-33.528z" stroke-opacity=".36478" stroke="#000" stroke-width=".42649" fill="url(#d)"/>
-							<path d="m-386 562.42c-10.108-2.9925-23.206-2.5682-33.101-0.86253 1.7084-10.962 1.922-24.701-0.4271-35.877l33.528 36.739z" stroke-width=".95407pt" fill="url(#c)"/>
-							<path d="m-519.13 537-0.60402 134.7h131.68l0.0755-33.296c-2.9446 1.1325-32.692-40.998-70.141-39.186-37.483 1.8137-27.785-56.777-61.006-62.214z" stroke-width="1pt" fill="url(#b)"/>
+				<g id="file" stroke="#000" stroke-width="25" fill="#FFF" fill-rule="evenodd" stroke-linecap="round" stroke-linejoin="round">
+					<path d="M13 24.12v274.76c0 6.16 5.87 11.12 13.17 11.12H239c7.3 0 13.17-4.96 13.17-11.12V136.15S132.6 13 128.37 13H26.17C18.87 13 13 17.96 13 24.12z"/>
+					<path d="M129.37 13L129 113.9c0 10.58 7.26 19.1 16.27 19.1H249L129.37 13z"/>
+				</g>
+				<g id="file-shortcut" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
+					<g id="file-shortcut-group" transform="translate(13.000000, 13.000000)">
+						<g id="file-shortcut-shape" stroke="#000000" stroke-width="25" fill="#FFFFFF" stroke-linecap="round" stroke-linejoin="round">
+							<path d="M0,11.1214886 L0,285.878477 C0,292.039924 5.87498876,296.999983 13.1728373,296.999983 L225.997983,296.999983 C233.295974,296.999983 239.17082,292.039942 239.17082,285.878477 L239.17082,123.145388 C239.17082,123.145388 119.58541,2.84217094e-14 115.369423,2.84217094e-14 L13.1728576,2.84217094e-14 C5.87500907,-1.71479982e-05 0,4.96022995 0,11.1214886 Z" id="rect1171"></path>
+							<path d="M116.37005,0 L116,100.904964 C116,111.483663 123.258008,120 132.273377,120 L236,120 L116.37005,0 L116.37005,0 Z" id="rect1794"></path>
 						</g>
+						<path d="M47.803141,294.093878 C48.4999811,295.177551 48.9495553,296.095918 49.4216083,296.995918 C50.1184484,297.895918 50.5680227,298.630612 51.2873415,299.365306 C52.2089688,300.44898 53.3778619,301 54.7490634,301 C56.1427436,301 57.0643709,300.632653 57.761211,299.916327 C58.4580511,299.365306 58.9076254,298.465306 58.9076254,297.381633 C58.9076254,296.830612 58.9076254,295.930612 58.6828382,294.828571 C58.4355724,293.561224 58.2107852,292.844898 58.2107852,292.477551 C57.7387323,287.757143 57.5139451,283.753061 57.5139451,279.95102 C57.5139451,273.228571 58.4355724,267.057143 59.8292526,261.602041 C61.44772,256.165306 63.5382403,251.610204 66.0783349,247.62449 C68.8656954,243.620408 72.3274172,240.35102 76.4859792,237.44898 C80.6445412,234.546939 85.2751561,232.177551 90.1305582,230.359184 C94.9859603,228.540816 100.76299,227.089796 107.236859,226.006122 C113.710728,225.087755 120.409385,224.371429 127.13052,223.820408 C133.829177,223.453061 141.247152,223.269388 149.811542,223.269388 L167.704598,223.269388 L167.704598,249.057143 C167.704598,250.87551 168.401438,252.326531 170.019905,253.593878 C171.86316,254.861224 173.728893,255.595918 176.021722,255.595918 C178.112242,255.595918 180.180284,254.861224 181.82123,253.593878 L247.751296,201.834694 C249.369763,200.567347 250.291391,199.116327 250.291391,197.297959 C250.291391,195.479592 249.369763,194.028571 247.751296,192.761224 L181.82123,141.002041 C180.202763,139.734694 178.112242,139 176.044201,139 C173.728893,139 171.885639,139.734694 170.042384,141.002041 C168.423917,142.269388 167.727077,143.720408 167.727077,145.538776 L167.727077,171.326531 L149.811542,171.326531 C88.5120908,171.326531 50.8152886,184.955102 36.9234437,212.193878 C32.3153075,221.267347 30,232.526531 30,245.971429 C30,257.046939 35.5522422,272.291837 46.4094607,291.540816 C46.6567266,292.091837 47.1063009,292.826531 47.803141,294.093878 Z" id="Shape-Copy" fill="#000000" fill-rule="nonzero" transform="translate(140.145695, 220.000000) scale(-1, 1) translate(-140.145695, -220.000000) "></path>
 					</g>
 				</g>
 
@@ -321,7 +372,7 @@ footer {
 
 		<header>
 			<h1>
-				{{range $url, $name := .BreadcrumbMap}}<a href="{{$url}}">{{$name}}</a>{{if ne $url "/"}}/{{end}}{{end}}
+				{{range $i, $crumb := .Breadcrumbs}}<a href="{{html $crumb.Link}}">{{html $crumb.Text}}</a>{{if ne $i 0}}/{{end}}{{end}}
 			</h1>
 		</header>
 		<main>
@@ -332,6 +383,7 @@ footer {
 					{{- if ne 0 .ItemsLimitedTo}}
 					<span class="meta-item">(of which only <b>{{.ItemsLimitedTo}}</b> are displayed)</span>
 					{{- end}}
+					<span class="meta-item"><input type="text" placeholder="filter" id="filter" onkeyup='filter()'></span>
 				</div>
 			</div>
 			<div class="listing">
@@ -339,28 +391,36 @@ footer {
 					<thead>
 					<tr>
 						<th>
+							{{- if and (eq .Sort "namedirfirst") (ne .Order "desc")}}
+							<a href="?sort=namedirfirst&order=desc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}" class="icon"><svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a>
+							{{- else if and (eq .Sort "namedirfirst") (ne .Order "asc")}}
+							<a href="?sort=namedirfirst&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}" class="icon"><svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
+							{{- else}}
+							<a href="?sort=namedirfirst&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}" class="icon sort"><svg class="top" width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg><svg class="bottom" width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
+							{{- end}}
+							
 							{{- if and (eq .Sort "name") (ne .Order "desc")}}
-							<a href="?sort=name&order=desc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Name <svg width="1em" height=".4em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a>
+							<a href="?sort=name&order=desc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Name <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a>
 							{{- else if and (eq .Sort "name") (ne .Order "asc")}}
-							<a href="?sort=name&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Name <svg width="1em" height=".4em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
+							<a href="?sort=name&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Name <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
 							{{- else}}
 							<a href="?sort=name&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Name</a>
 							{{- end}}
 						</th>
 						<th>
 							{{- if and (eq .Sort "size") (ne .Order "desc")}}
-							<a href="?sort=size&order=desc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Size <svg width="1em" height=".4em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a>
+							<a href="?sort=size&order=desc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Size <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a>
 							{{- else if and (eq .Sort "size") (ne .Order "asc")}}
-							<a href="?sort=size&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Size <svg width="1em" height=".4em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
+							<a href="?sort=size&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Size <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
 							{{- else}}
 							<a href="?sort=size&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Size</a>
 							{{- end}}
 						</th>
 						<th class="hideable">
 							{{- if and (eq .Sort "time") (ne .Order "desc")}}
-							<a href="?sort=time&order=desc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Modified <svg width="1em" height=".4em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a>
+							<a href="?sort=time&order=desc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Modified <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#up-arrow"></use></svg></a>
 							{{- else if and (eq .Sort "time") (ne .Order "asc")}}
-							<a href="?sort=time&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Modified <svg width="1em" height=".4em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
+							<a href="?sort=time&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Modified <svg width="1em" height=".5em" version="1.1" viewBox="0 0 12.922194 6.0358899"><use xlink:href="#down-arrow"></use></svg></a>
 							{{- else}}
 							<a href="?sort=time&order=asc{{if ne 0 .ItemsLimitedTo}}&limit={{.ItemsLimitedTo}}{{end}}">Modified</a>
 							{{- end}}
@@ -380,15 +440,15 @@ footer {
 					</tr>
 					{{- end}}
 					{{- range .Items}}
-					<tr>
+					<tr class="file">
 						<td>
-							<a href="{{.URL}}">
+							<a href="{{html .URL}}">
 								{{- if .IsDir}}
-								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 35.678803 28.527945"><use xlink:href="#folder"></use></svg>
+								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 317 259"><use xlink:href="#folder{{if .IsSymlink}}-shortcut{{end}}"></use></svg>
 								{{- else}}
-								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 26.604381 29.144726"><use xlink:href="#file"></use></svg>
+								<svg width="1.5em" height="1em" version="1.1" viewBox="0 0 265 323"><use xlink:href="#file{{if .IsSymlink}}-shortcut{{end}}"></use></svg>
 								{{- end}}
-								<span class="name">{{.Name}}</span>
+								<span class="name">{{html .Name}}</span>
 							</a>
 						</td>
 						{{- if .IsDir}}
@@ -404,9 +464,30 @@ footer {
 			</div>
 		</main>
 		<footer>
-			Served with <a rel="noopener noreferrer" href="https://caddyserver.com">Caddy</a>.
+			Served with <a rel="noopener noreferrer" href="https://caddyserver.com">Caddy</a>
 		</footer>
-		<script type="text/javascript">
+		<script>
+			var filterEl = document.getElementById('filter');
+			filterEl.focus();
+
+			function filter() {
+				var q = filterEl.value.trim().toLowerCase();
+				var elems = document.querySelectorAll('tr.file');
+				elems.forEach(function(el) {
+					if (!q) {
+						el.style.display = '';
+						return;
+					}
+					var nameEl = el.querySelector('.name');
+					var nameVal = nameEl.textContent.trim().toLowerCase();
+					if (nameVal.indexOf(q) !== -1) {
+						el.style.display = '';
+					} else {
+						el.style.display = 'none';
+					}
+				});
+			}
+
 			function localizeDatetime(e, index, ar) {
 				if (e.textContent === undefined) {
 					return;
@@ -418,7 +499,7 @@ footer {
 						return;
 					}
 				}
-				e.textContent = d.toLocaleString();
+				e.textContent = d.toLocaleString([], {day: "2-digit", month: "2-digit", year: "numeric", hour: "2-digit", minute: "2-digit", second: "2-digit"});
 			}
 			var timeList = Array.prototype.slice.call(document.getElementsByTagName("time"));
 			timeList.forEach(localizeDatetime);

caddyhttp/browse/setup_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package browse
 
 import (
@@ -18,7 +32,7 @@ func TestSetup(t *testing.T) {
 	if err != nil {
 		t.Fatalf("BeforeTest: Failed to find an existing directory for testing! Error was: %v", err)
 	}
-	nonExistantDirPath := filepath.Join(tempDirPath, strconv.Itoa(int(time.Now().UnixNano())))
+	nonExistentDirPath := filepath.Join(tempDirPath, strconv.Itoa(int(time.Now().UnixNano())))
 
 	tempTemplate, err := ioutil.TempFile(".", "tempTemplate")
 	if err != nil {
@@ -43,7 +57,7 @@ func TestSetup(t *testing.T) {
 		{"browse . " + tempTemplatePath, []string{"."}, false},
 
 		// test case #3 tests detection of non-existent template
-		{"browse . " + nonExistantDirPath, nil, true},
+		{"browse . " + nonExistentDirPath, nil, true},
 
 		// test case #4 tests detection of duplicate pathscopes
 		{"browse " + tempDirPath + "\n browse " + tempDirPath, nil, true},
@@ -52,18 +66,30 @@ func TestSetup(t *testing.T) {
 		c := caddy.NewTestController("http", test.input)
 		err := setup(c)
 		if err != nil && !test.shouldErr {
-			t.Errorf("Test case #%d recieved an error of %v", i, err)
+			t.Errorf("Test case #%d received an error of %v", i, err)
 		}
 		if test.expectedPathScope == nil {
 			continue
 		}
 		mids := httpserver.GetConfig(c).Middleware()
 		mid := mids[len(mids)-1]
-		recievedConfigs := mid(nil).(Browse).Configs
-		for j, config := range recievedConfigs {
+		receivedConfigs := mid(nil).(Browse).Configs
+		for j, config := range receivedConfigs {
 			if config.PathScope != test.expectedPathScope[j] {
 				t.Errorf("Test case #%d expected a pathscope of %v, but got %v", i, test.expectedPathScope, config.PathScope)
 			}
 		}
 	}
+
+	// test case #6 tests startup with missing root directory in combination with default browse settings
+	controller := caddy.NewTestController("http", "browse")
+	cfg := httpserver.GetConfig(controller)
+
+	// Make sure non-existent root path doesn't return error
+	cfg.Root = nonExistentDirPath
+	err = setup(controller)
+
+	if err != nil {
+		t.Errorf("Test for non-existent browse path received an error, but shouldn't have: %v", err)
+	}
 }

caddyhttp/browse/testdata/photos/hidden.html

@@ -0,0 +1 @@
+Should be hidden

caddyhttp/browse/testdata/photos/test1/test.html

@@ -0,0 +1,8 @@
+<!DOCTYPE html>
+<html>
+<head>
+<title>Test</title>
+</head>
+<body>
+</body>
+</html>

caddyhttp/caddyhttp.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyhttp
 
 import (
@@ -14,16 +28,23 @@ import (
 	_ "github.com/mholt/caddy/caddyhttp/fastcgi"
 	_ "github.com/mholt/caddy/caddyhttp/gzip"
 	_ "github.com/mholt/caddy/caddyhttp/header"
+	_ "github.com/mholt/caddy/caddyhttp/index"
 	_ "github.com/mholt/caddy/caddyhttp/internalsrv"
+	_ "github.com/mholt/caddy/caddyhttp/limits"
 	_ "github.com/mholt/caddy/caddyhttp/log"
 	_ "github.com/mholt/caddy/caddyhttp/markdown"
 	_ "github.com/mholt/caddy/caddyhttp/mime"
 	_ "github.com/mholt/caddy/caddyhttp/pprof"
 	_ "github.com/mholt/caddy/caddyhttp/proxy"
+	_ "github.com/mholt/caddy/caddyhttp/push"
 	_ "github.com/mholt/caddy/caddyhttp/redirect"
+	_ "github.com/mholt/caddy/caddyhttp/requestid"
 	_ "github.com/mholt/caddy/caddyhttp/rewrite"
 	_ "github.com/mholt/caddy/caddyhttp/root"
+	_ "github.com/mholt/caddy/caddyhttp/status"
 	_ "github.com/mholt/caddy/caddyhttp/templates"
+	_ "github.com/mholt/caddy/caddyhttp/timeouts"
 	_ "github.com/mholt/caddy/caddyhttp/websocket"
+	_ "github.com/mholt/caddy/onevent"
 	_ "github.com/mholt/caddy/startupshutdown"
 )

caddyhttp/caddyhttp_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package caddyhttp
 
 import (
@@ -11,7 +25,7 @@ import (
 // ensure that the standard plugins are in fact plugged in
 // and registered properly; this is a quick/naive way to do it.
 func TestStandardPlugins(t *testing.T) {
-	numStandardPlugins := 25 // importing caddyhttp plugs in this many plugins
+	numStandardPlugins := 33 // importing caddyhttp plugs in this many plugins
 	s := caddy.DescribePlugins()
 	if got, want := strings.Count(s, "\n"), numStandardPlugins+5; got != want {
 		t.Errorf("Expected all standard plugins to be plugged in, got:\n%s", s)

caddyhttp/errors/errors.go

@@ -1,10 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package errors implements an HTTP error handling middleware.
 package errors
 
 import (
 	"fmt"
 	"io"
-	"log"
 	"net/http"
 	"os"
 	"runtime"
@@ -24,12 +37,11 @@ func init() {
 
 // ErrorHandler handles HTTP errors (and errors from other middleware).
 type ErrorHandler struct {
-	Next       httpserver.Handler
-	ErrorPages map[int]string // map of status code to filename
-	LogFile    string
-	Log        *log.Logger
-	LogRoller  *httpserver.LogRoller
-	Debug      bool // if true, errors are written out to client rather than to a log
+	Next             httpserver.Handler
+	GenericErrorPage string         // default error page filename
+	ErrorPages       map[int]string // map of status code to filename
+	Log              *httpserver.Logger
+	Debug            bool // if true, errors are written out to client rather than to a log
 }
 
 func (h ErrorHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
@@ -62,7 +74,7 @@ func (h ErrorHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, er
 // message is written instead, and the extra error is logged.
 func (h ErrorHandler) errorPage(w http.ResponseWriter, r *http.Request, code int) {
 	// See if an error page for this status code was specified
-	if pagePath, ok := h.ErrorPages[code]; ok {
+	if pagePath, ok := h.findErrorPage(code); ok {
 		// Try to open it
 		errorPage, err := os.Open(pagePath)
 		if err != nil {
@@ -93,6 +105,18 @@ func (h ErrorHandler) errorPage(w http.ResponseWriter, r *http.Request, code int
 	httpserver.DefaultErrorFunc(w, r, code)
 }
 
+func (h ErrorHandler) findErrorPage(code int) (string, bool) {
+	if pagePath, ok := h.ErrorPages[code]; ok {
+		return pagePath, true
+	}
+
+	if h.GenericErrorPage != "" {
+		return h.GenericErrorPage, true
+	}
+
+	return "", false
+}
+
 func (h ErrorHandler) recovery(w http.ResponseWriter, r *http.Request) {
 	rec := recover()
 	if rec == nil {

caddyhttp/errors/errors_test.go

@@ -1,10 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package errors
 
 import (
 	"bytes"
 	"errors"
 	"fmt"
-	"log"
 	"net/http"
 	"net/http/httptest"
 	"os"
@@ -18,27 +31,21 @@ import (
 
 func TestErrors(t *testing.T) {
 	// create a temporary page
-	path := filepath.Join(os.TempDir(), "errors_test.html")
-	f, err := os.Create(path)
+	const content = "This is a error page"
+
+	path, err := createErrorPageFile("errors_test.html", content)
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer os.Remove(path)
 
-	const content = "This is a error page"
-	_, err = f.WriteString(content)
-	if err != nil {
-		t.Fatal(err)
-	}
-	f.Close()
-
 	buf := bytes.Buffer{}
 	em := ErrorHandler{
 		ErrorPages: map[int]string{
 			http.StatusNotFound:  path,
 			http.StatusForbidden: "not_exist_file",
 		},
-		Log: log.New(&buf, "", 0),
+		Log: httpserver.NewTestLogger(&buf),
 	}
 	_, notExistErr := os.Open("not_exist_file")
 
@@ -157,6 +164,87 @@ func TestVisibleErrorWithPanic(t *testing.T) {
 	}
 }
 
+func TestGenericErrorPage(t *testing.T) {
+	// create temporary generic error page
+	const genericErrorContent = "This is a generic error page"
+
+	genericErrorPagePath, err := createErrorPageFile("generic_error_test.html", genericErrorContent)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(genericErrorPagePath)
+
+	// create temporary error page
+	const notFoundErrorContent = "This is a error page"
+
+	notFoundErrorPagePath, err := createErrorPageFile("not_found.html", notFoundErrorContent)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(notFoundErrorPagePath)
+
+	buf := bytes.Buffer{}
+	em := ErrorHandler{
+		GenericErrorPage: genericErrorPagePath,
+		ErrorPages: map[int]string{
+			http.StatusNotFound: notFoundErrorPagePath,
+		},
+		Log: httpserver.NewTestLogger(&buf),
+	}
+
+	tests := []struct {
+		next         httpserver.Handler
+		expectedCode int
+		expectedBody string
+		expectedLog  string
+		expectedErr  error
+	}{
+		{
+			next:         genErrorHandler(http.StatusNotFound, nil, ""),
+			expectedCode: 0,
+			expectedBody: notFoundErrorContent,
+			expectedLog:  "",
+			expectedErr:  nil,
+		},
+		{
+			next:         genErrorHandler(http.StatusInternalServerError, nil, ""),
+			expectedCode: 0,
+			expectedBody: genericErrorContent,
+			expectedLog:  "",
+			expectedErr:  nil,
+		},
+	}
+
+	req, err := http.NewRequest("GET", "/", nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	for i, test := range tests {
+		em.Next = test.next
+		buf.Reset()
+		rec := httptest.NewRecorder()
+		code, err := em.ServeHTTP(rec, req)
+
+		if err != test.expectedErr {
+			t.Errorf("Test %d: Expected error %v, but got %v",
+				i, test.expectedErr, err)
+		}
+		if code != test.expectedCode {
+			t.Errorf("Test %d: Expected status code %d, but got %d",
+				i, test.expectedCode, code)
+		}
+		if body := rec.Body.String(); body != test.expectedBody {
+			t.Errorf("Test %d: Expected body %q, but got %q",
+				i, test.expectedBody, body)
+		}
+		if log := buf.String(); !strings.Contains(log, test.expectedLog) {
+			t.Errorf("Test %d: Expected log %q, but got %q",
+				i, test.expectedLog, log)
+		}
+	}
+}
+
 func genErrorHandler(status int, err error, body string) httpserver.Handler {
 	return httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
 		if len(body) > 0 {
@@ -166,3 +254,19 @@ func genErrorHandler(status int, err error, body string) httpserver.Handler {
 		return status, err
 	})
 }
+
+func createErrorPageFile(name string, content string) (string, error) {
+	errorPageFilePath := filepath.Join(os.TempDir(), name)
+	f, err := os.Create(errorPageFilePath)
+	if err != nil {
+		return "", err
+	}
+
+	_, err = f.WriteString(content)
+	if err != nil {
+		return "", err
+	}
+	f.Close()
+
+	return errorPageFilePath, nil
+}

caddyhttp/errors/setup.go

@@ -1,13 +1,25 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package errors
 
 import (
-	"io"
 	"log"
 	"os"
 	"path/filepath"
 	"strconv"
 
-	"github.com/hashicorp/go-syslog"
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
@@ -15,52 +27,12 @@ import (
 // setup configures a new errors middleware instance.
 func setup(c *caddy.Controller) error {
 	handler, err := errorsParse(c)
+
 	if err != nil {
 		return err
 	}
 
-	// Open the log file for writing when the server starts
-	c.OnStartup(func() error {
-		var err error
-		var writer io.Writer
-
-		switch handler.LogFile {
-		case "visible":
-			handler.Debug = true
-		case "stdout":
-			writer = os.Stdout
-		case "stderr":
-			writer = os.Stderr
-		case "syslog":
-			writer, err = gsyslog.NewLogger(gsyslog.LOG_ERR, "LOCAL0", "caddy")
-			if err != nil {
-				return err
-			}
-		default:
-			if handler.LogFile == "" {
-				writer = os.Stderr // default
-				break
-			}
-
-			var file *os.File
-			file, err = os.OpenFile(handler.LogFile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
-			if err != nil {
-				return err
-			}
-			if handler.LogRoller != nil {
-				file.Close()
-
-				handler.LogRoller.Filename = handler.LogFile
-
-				writer = handler.LogRoller.GetLogWriter()
-			} else {
-				writer = file
-			}
-		}
-
-		handler.Log = log.New(writer, "", 0)
-		return nil
-	})
+	handler.Log.Attach(c)
 
 	httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
 		handler.Next = next
@@ -71,58 +43,66 @@ func setup(c *caddy.Controller) error {
 }
 
 func errorsParse(c *caddy.Controller) (*ErrorHandler, error) {
+
 	// Very important that we make a pointer because the startup
 	// function that opens the log file must have access to the
 	// same instance of the handler, not a copy.
-	handler := &ErrorHandler{ErrorPages: make(map[int]string)}
+	handler := &ErrorHandler{
+		ErrorPages: make(map[int]string),
+		Log:        &httpserver.Logger{},
+	}
 
 	cfg := httpserver.GetConfig(c)
 
-	optionalBlock := func() (bool, error) {
-		var hadBlock bool
-
+	optionalBlock := func() error {
 		for c.NextBlock() {
-			hadBlock = true
 
 			what := c.Val()
-			if !c.NextArg() {
-				return hadBlock, c.ArgErr()
-			}
-			where := c.Val()
+			where := c.RemainingArgs()
 
-			if what == "log" {
-				if where == "visible" {
-					handler.Debug = true
-				} else {
-					handler.LogFile = where
-					if c.NextArg() {
-						if c.Val() == "{" {
-							c.IncrNest()
-							logRoller, err := httpserver.ParseRoller(c)
-							if err != nil {
-								return hadBlock, err
-							}
-							handler.LogRoller = logRoller
-						}
-					}
+			if httpserver.IsLogRollerSubdirective(what) {
+				var err error
+				err = httpserver.ParseRoller(handler.Log.Roller, what, where...)
+				if err != nil {
+					return err
 				}
 			} else {
+				if len(where) != 1 {
+					return c.ArgErr()
+				}
+				where := where[0]
+
 				// Error page; ensure it exists
-				where = filepath.Join(cfg.Root, where)
+				if !filepath.IsAbs(where) {
+					where = filepath.Join(cfg.Root, where)
+				}
+
 				f, err := os.Open(where)
 				if err != nil {
 					log.Printf("[WARNING] Unable to open error page '%s': %v", where, err)
 				}
 				f.Close()
 
-				whatInt, err := strconv.Atoi(what)
-				if err != nil {
-					return hadBlock, c.Err("Expecting a numeric status code, got '" + what + "'")
+				if what == "*" {
+					if handler.GenericErrorPage != "" {
+						return c.Errf("Duplicate status code entry: %s", what)
+					}
+					handler.GenericErrorPage = where
+				} else {
+					whatInt, err := strconv.Atoi(what)
+					if err != nil {
+						return c.Err("Expecting a numeric status code or '*', got '" + what + "'")
+					}
+
+					if _, exists := handler.ErrorPages[whatInt]; exists {
+						return c.Errf("Duplicate status code entry: %s", what)
+					}
+
+					handler.ErrorPages[whatInt] = where
 				}
-				handler.ErrorPages[whatInt] = where
 			}
 		}
-		return hadBlock, nil
+		return nil
 	}
 
 	for c.Next() {
@@ -130,21 +110,23 @@ func errorsParse(c *caddy.Controller) (*ErrorHandler, error) {
 		if c.Val() == "}" {
 			continue
 		}
-		// Configuration may be in a block
-		hadBlock, err := optionalBlock()
-		if err != nil {
-			return handler, err
+
+		args := c.RemainingArgs()
+
+		if len(args) == 1 {
+			switch args[0] {
+			case "visible":
+				handler.Debug = true
+			default:
+				handler.Log.Output = args[0]
+				handler.Log.Roller = httpserver.DefaultLogRoller()
+			}
 		}
 
-		// Otherwise, the only argument would be an error log file name or 'visible'
-		if !hadBlock {
-			if c.NextArg() {
-				if c.Val() == "visible" {
-					handler.Debug = true
-				} else {
-					handler.LogFile = c.Val()
-				}
-			}
+		// Configuration may be in a block
+		err := optionalBlock()
+		if err != nil {
+			return handler, err
 		}
 	}
 

caddyhttp/errors/setup_test.go

@@ -1,6 +1,22 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package errors
 
 import (
+	"path/filepath"
+	"reflect"
 	"testing"
 
 	"github.com/mholt/caddy"
@@ -24,12 +40,12 @@ func TestSetup(t *testing.T) {
 		t.Fatalf("Expected handler to be type ErrorHandler, got: %#v", handler)
 	}
 
-	if myHandler.LogFile != "" {
-		t.Errorf("Expected '%s' as the default LogFile", "")
-	}
-	if myHandler.LogRoller != nil {
-		t.Errorf("Expected LogRoller to be nil, got: %v", *myHandler.LogRoller)
+	expectedLogger := &httpserver.Logger{}
+
+	if !reflect.DeepEqual(expectedLogger, myHandler.Log) {
+		t.Errorf("Expected '%v' as the default Log, got: '%v'", expectedLogger, myHandler.Log)
 	}
+
 	if !httpserver.SameNext(myHandler.Next, httpserver.EmptyNext) {
 		t.Error("'Next' field of handler was not set properly")
 	}
@@ -45,65 +61,126 @@ func TestSetup(t *testing.T) {
 }
 
 func TestErrorsParse(t *testing.T) {
+	testAbs, err := filepath.Abs("./404.html")
+	if err != nil {
+		t.Error(err)
+	}
 	tests := []struct {
 		inputErrorsRules     string
 		shouldErr            bool
 		expectedErrorHandler ErrorHandler
 	}{
 		{`errors`, false, ErrorHandler{
-			LogFile: "",
+			ErrorPages: map[int]string{},
+			Log:        &httpserver.Logger{},
 		}},
 		{`errors errors.txt`, false, ErrorHandler{
-			LogFile: "errors.txt",
+			ErrorPages: map[int]string{},
+			Log: &httpserver.Logger{
+				Output: "errors.txt",
+				Roller: httpserver.DefaultLogRoller(),
+			},
 		}},
 		{`errors visible`, false, ErrorHandler{
-			LogFile: "",
-			Debug:   true,
+			ErrorPages: map[int]string{},
+			Debug:      true,
+			Log:        &httpserver.Logger{},
 		}},
-		{`errors { log visible }`, false, ErrorHandler{
-			LogFile: "",
-			Debug:   true,
-		}},
-		{`errors { log errors.txt
+		{`errors errors.txt {
         404 404.html
         500 500.html
 }`, false, ErrorHandler{
-			LogFile: "errors.txt",
 			ErrorPages: map[int]string{
 				404: "404.html",
 				500: "500.html",
 			},
-		}},
-		{`errors { log errors.txt { size 2 age 10 keep 3 } }`, false, ErrorHandler{
-			LogFile: "errors.txt",
-			LogRoller: &httpserver.LogRoller{
-				MaxSize:    2,
-				MaxAge:     10,
-				MaxBackups: 3,
-				LocalTime:  true,
+			Log: &httpserver.Logger{
+				Output: "errors.txt",
+				Roller: httpserver.DefaultLogRoller(),
 			},
 		}},
-		{`errors { log errors.txt {
-            size 3
-            age 11
-            keep 5
-        }
+		{`errors errors.txt {
+			rotate_size 2
+			rotate_age 10
+			rotate_keep 3
+			rotate_compress
+		}`, false, ErrorHandler{
+			ErrorPages: map[int]string{},
+			Log: &httpserver.Logger{
+				Output: "errors.txt", Roller: &httpserver.LogRoller{
+					MaxSize:    2,
+					MaxAge:     10,
+					MaxBackups: 3,
+					Compress:   true,
+					LocalTime:  true,
+				},
+			},
+		}},
+		{`errors errors.txt {
+		rotate_size 3
+		rotate_age 11
+		rotate_keep 5
         404 404.html
         503 503.html
 }`, false, ErrorHandler{
-			LogFile: "errors.txt",
 			ErrorPages: map[int]string{
 				404: "404.html",
 				503: "503.html",
 			},
-			LogRoller: &httpserver.LogRoller{
-				MaxSize:    3,
-				MaxAge:     11,
-				MaxBackups: 5,
-				LocalTime:  true,
+			Log: &httpserver.Logger{
+				Output: "errors.txt",
+				Roller: &httpserver.LogRoller{
+					MaxSize:    3,
+					MaxAge:     11,
+					MaxBackups: 5,
+					Compress:   false,
+					LocalTime:  true,
+				},
 			},
 		}},
+		{`errors errors.txt {
+        * generic_error.html
+        404 404.html
+        503 503.html
+}`, false, ErrorHandler{
+			Log: &httpserver.Logger{
+				Output: "errors.txt",
+				Roller: httpserver.DefaultLogRoller(),
+			},
+			GenericErrorPage: "generic_error.html",
+			ErrorPages: map[int]string{
+				404: "404.html",
+				503: "503.html",
+			},
+		}},
+		// test absolute file path
+		{`errors {
+			404 ` + testAbs + `
+		}`,
+			false, ErrorHandler{
+				ErrorPages: map[int]string{
+					404: testAbs,
+				},
+				Log: &httpserver.Logger{},
+			}},
+		{`errors errors.txt { rotate_size 2 rotate_age 10 rotate_keep 3 rotate_compress }`,
+			true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
+		{`errors errors.txt {
+			rotate_compress invalid
+		}`,
+			true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
+		// Next two test cases is the detection of duplicate status codes
+		{`errors {
+			503 503.html
+			503 503.html
+		}`, true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
+
+		{`errors {
+			* generic_error.html
+			* generic_error.html
+		}`, true, ErrorHandler{ErrorPages: map[int]string{}, Log: &httpserver.Logger{}}},
 	}
+
 	for i, test := range tests {
 		actualErrorsRule, err := errorsParse(caddy.NewTestController("http", test.inputErrorsRules))
 
@@ -111,45 +188,12 @@ func TestErrorsParse(t *testing.T) {
 			t.Errorf("Test %d didn't error, but it should have", i)
 		} else if err != nil && !test.shouldErr {
 			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
+		} else if err != nil && test.shouldErr {
+			continue
 		}
-		if actualErrorsRule.LogFile != test.expectedErrorHandler.LogFile {
-			t.Errorf("Test %d expected LogFile to be %s, but got %s",
-				i, test.expectedErrorHandler.LogFile, actualErrorsRule.LogFile)
-		}
-		if actualErrorsRule.Debug != test.expectedErrorHandler.Debug {
-			t.Errorf("Test %d expected Debug to be %v, but got %v",
-				i, test.expectedErrorHandler.Debug, actualErrorsRule.Debug)
-		}
-		if actualErrorsRule.LogRoller != nil && test.expectedErrorHandler.LogRoller == nil || actualErrorsRule.LogRoller == nil && test.expectedErrorHandler.LogRoller != nil {
-			t.Fatalf("Test %d expected LogRoller to be %v, but got %v",
-				i, test.expectedErrorHandler.LogRoller, actualErrorsRule.LogRoller)
-		}
-		if len(actualErrorsRule.ErrorPages) != len(test.expectedErrorHandler.ErrorPages) {
-			t.Fatalf("Test %d expected %d no of Error pages, but got %d ",
-				i, len(test.expectedErrorHandler.ErrorPages), len(actualErrorsRule.ErrorPages))
-		}
-		if actualErrorsRule.LogRoller != nil && test.expectedErrorHandler.LogRoller != nil {
-			if actualErrorsRule.LogRoller.Filename != test.expectedErrorHandler.LogRoller.Filename {
-				t.Fatalf("Test %d expected LogRoller Filename to be %s, but got %s",
-					i, test.expectedErrorHandler.LogRoller.Filename, actualErrorsRule.LogRoller.Filename)
-			}
-			if actualErrorsRule.LogRoller.MaxAge != test.expectedErrorHandler.LogRoller.MaxAge {
-				t.Fatalf("Test %d expected LogRoller MaxAge to be %d, but got %d",
-					i, test.expectedErrorHandler.LogRoller.MaxAge, actualErrorsRule.LogRoller.MaxAge)
-			}
-			if actualErrorsRule.LogRoller.MaxBackups != test.expectedErrorHandler.LogRoller.MaxBackups {
-				t.Fatalf("Test %d expected LogRoller MaxBackups to be %d, but got %d",
-					i, test.expectedErrorHandler.LogRoller.MaxBackups, actualErrorsRule.LogRoller.MaxBackups)
-			}
-			if actualErrorsRule.LogRoller.MaxSize != test.expectedErrorHandler.LogRoller.MaxSize {
-				t.Fatalf("Test %d expected LogRoller MaxSize to be %d, but got %d",
-					i, test.expectedErrorHandler.LogRoller.MaxSize, actualErrorsRule.LogRoller.MaxSize)
-			}
-			if actualErrorsRule.LogRoller.LocalTime != test.expectedErrorHandler.LogRoller.LocalTime {
-				t.Fatalf("Test %d expected LogRoller LocalTime to be %t, but got %t",
-					i, test.expectedErrorHandler.LogRoller.LocalTime, actualErrorsRule.LogRoller.LocalTime)
-			}
+		if !reflect.DeepEqual(actualErrorsRule, &test.expectedErrorHandler) {
+			t.Errorf("Test %d expect %v, but got %v", i,
+				test.expectedErrorHandler, actualErrorsRule)
 		}
 	}
-
 }

caddyhttp/expvar/expvar.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package expvar
 
 import (

caddyhttp/expvar/expvar_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package expvar
 
 import (

caddyhttp/expvar/setup.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package expvar
 
 import (

caddyhttp/expvar/setup_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package expvar
 
 import (

caddyhttp/extensions/ext.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package extensions contains middleware for clean URLs.
 //
 // The root path of the site is passed in as well as possible extensions
@@ -21,7 +35,7 @@ type Ext struct {
 	// Next handler in the chain
 	Next httpserver.Handler
 
-	// Path to ther root of the site
+	// Path to site root
 	Root string
 
 	// List of extensions to try
@@ -31,9 +45,10 @@ type Ext struct {
 // ServeHTTP implements the httpserver.Handler interface.
 func (e Ext) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	urlpath := strings.TrimSuffix(r.URL.Path, "/")
-	if path.Ext(urlpath) == "" && len(r.URL.Path) > 0 && r.URL.Path[len(r.URL.Path)-1] != '/' {
+	if len(r.URL.Path) > 0 && path.Ext(urlpath) == "" && r.URL.Path[len(r.URL.Path)-1] != '/' {
 		for _, ext := range e.Extensions {
-			if resourceExists(e.Root, urlpath+ext) {
+			_, err := os.Stat(httpserver.SafePath(e.Root, urlpath) + ext)
+			if err == nil {
 				r.URL.Path = urlpath + ext
 				break
 			}
@@ -41,12 +56,3 @@ func (e Ext) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	}
 	return e.Next.ServeHTTP(w, r)
 }
-
-// resourceExists returns true if the file specified at
-// root + path exists; false otherwise.
-func resourceExists(root, path string) bool {
-	_, err := os.Stat(root + path)
-	// technically we should use os.IsNotExist(err)
-	// but we don't handle any other kinds of errors anyway
-	return err == nil
-}

caddyhttp/extensions/setup.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package extensions
 
 import (

caddyhttp/extensions/setup_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package extensions
 
 import (

caddyhttp/fastcgi/fastcgi.go

@@ -1,18 +1,39 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package fastcgi has middleware that acts as a FastCGI client. Requests
 // that get forwarded to FastCGI stop the middleware execution chain.
 // The most common use for this package is to serve PHP websites via php-fpm.
 package fastcgi
 
 import (
+	"context"
 	"errors"
+	"fmt"
 	"io"
+	"net"
 	"net/http"
+	"net/url"
 	"os"
 	"path"
 	"path/filepath"
 	"strconv"
 	"strings"
+	"sync/atomic"
+	"time"
 
+	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
@@ -21,7 +42,6 @@ type Handler struct {
 	Next    httpserver.Handler
 	Rules   []Rule
 	Root    string
-	AbsRoot string // same as root, but absolute path
 	FileSys http.FileSystem
 
 	// These are sent to CGI scripts in env variables
@@ -34,9 +54,25 @@ type Handler struct {
 // ServeHTTP satisfies the httpserver.Handler interface.
 func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	for _, rule := range h.Rules {
-
-		// First requirement: Base path must match and the path must be allowed.
-		if !httpserver.Path(r.URL.Path).Matches(rule.Path) || !rule.AllowedPath(r.URL.Path) {
+		// First requirement: Base path must match request path. If it doesn't,
+		// we check to make sure the leading slash is not missing, and if so,
+		// we check again with it prepended. This is in case people forget
+		// a leading slash when performing rewrites, and we don't want to expose
+		// the contents of the (likely PHP) script. See issue #1645.
+		hpath := httpserver.Path(r.URL.Path)
+		if !hpath.Matches(rule.Path) {
+			if strings.HasPrefix(string(hpath), "/") {
+				// this is a normal-looking path, and it doesn't match; try next rule
+				continue
+			}
+			hpath = httpserver.Path("/" + string(hpath)) // prepend leading slash
+			if !hpath.Matches(rule.Path) {
+				// even after fixing the request path, it still doesn't match; try next rule
+				continue
+			}
+		}
+		// The path must also be allowed (not ignored).
+		if !rule.AllowedPath(r.URL.Path) {
 			continue
 		}
 
@@ -72,31 +108,63 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 			}
 
 			// Connect to FastCGI gateway
-			network, address := rule.parseAddress()
-			fcgiBackend, err := Dial(network, address)
+			address, err := rule.Address()
 			if err != nil {
 				return http.StatusBadGateway, err
 			}
+			network, address := parseAddress(address)
+
+			ctx := context.Background()
+			if rule.ConnectTimeout > 0 {
+				var cancel context.CancelFunc
+				ctx, cancel = context.WithTimeout(ctx, rule.ConnectTimeout)
+				defer cancel()
+			}
+
+			fcgiBackend, err := DialContext(ctx, network, address)
+			if err != nil {
+				return http.StatusBadGateway, err
+			}
+			defer fcgiBackend.Close()
+
+			// read/write timeouts
+			if err := fcgiBackend.SetReadTimeout(rule.ReadTimeout); err != nil {
+				return http.StatusInternalServerError, err
+			}
+			if err := fcgiBackend.SetSendTimeout(rule.SendTimeout); err != nil {
+				return http.StatusInternalServerError, err
+			}
 
 			var resp *http.Response
-			contentLength, _ := strconv.Atoi(r.Header.Get("Content-Length"))
+
+			var contentLength int64
+			// if ContentLength is already set
+			if r.ContentLength > 0 {
+				contentLength = r.ContentLength
+			} else {
+				contentLength, _ = strconv.ParseInt(r.Header.Get("Content-Length"), 10, 64)
+			}
 			switch r.Method {
 			case "HEAD":
 				resp, err = fcgiBackend.Head(env)
 			case "GET":
-				resp, err = fcgiBackend.Get(env)
+				resp, err = fcgiBackend.Get(env, r.Body, contentLength)
 			case "OPTIONS":
 				resp, err = fcgiBackend.Options(env)
 			default:
 				resp, err = fcgiBackend.Post(env, r.Method, r.Header.Get("Content-Type"), r.Body, contentLength)
 			}
 
-			if resp.Body != nil {
+			if resp != nil && resp.Body != nil {
 				defer resp.Body.Close()
 			}
 
-			if err != nil && err != io.EOF {
-				return http.StatusBadGateway, err
+			if err != nil {
+				if err, ok := err.(net.Error); ok && err.Timeout() {
+					return http.StatusGatewayTimeout, err
+				} else if err != io.EOF {
+					return http.StatusBadGateway, err
+				}
 			}
 
 			// Write response header
@@ -127,28 +195,28 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 	return h.Next.ServeHTTP(w, r)
 }
 
-// parseAddress returns the network and address of r.
+// parseAddress returns the network and address of fcgiAddress.
 // The first string is the network, "tcp" or "unix", implied from the scheme and address.
-// The second string is r.Address, with scheme prefixes removed.
+// The second string is fcgiAddress, with scheme prefixes removed.
 // The two returned strings can be used as parameters to the Dial() function.
-func (r Rule) parseAddress() (string, string) {
+func parseAddress(fcgiAddress string) (string, string) {
 	// check if address has tcp scheme explicitly set
-	if strings.HasPrefix(r.Address, "tcp://") {
-		return "tcp", r.Address[len("tcp://"):]
+	if strings.HasPrefix(fcgiAddress, "tcp://") {
+		return "tcp", fcgiAddress[len("tcp://"):]
 	}
 	// check if address has fastcgi scheme explicitly set
-	if strings.HasPrefix(r.Address, "fastcgi://") {
-		return "tcp", r.Address[len("fastcgi://"):]
+	if strings.HasPrefix(fcgiAddress, "fastcgi://") {
+		return "tcp", fcgiAddress[len("fastcgi://"):]
 	}
 	// check if unix socket
-	if trim := strings.HasPrefix(r.Address, "unix"); strings.HasPrefix(r.Address, "/") || trim {
+	if trim := strings.HasPrefix(fcgiAddress, "unix"); strings.HasPrefix(fcgiAddress, "/") || trim {
 		if trim {
-			return "unix", r.Address[len("unix:"):]
+			return "unix", fcgiAddress[len("unix:"):]
 		}
-		return "unix", r.Address
+		return "unix", fcgiAddress
 	}
 	// default case, a plain tcp address with no scheme
-	return "tcp", r.Address
+	return "tcp", fcgiAddress
 }
 
 func writeHeader(w http.ResponseWriter, r *http.Response) {
@@ -172,7 +240,7 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 	var env map[string]string
 
 	// Get absolute path of requested resource
-	absPath := filepath.Join(h.AbsRoot, fpath)
+	absPath := filepath.Join(rule.Root, fpath)
 
 	// Separate remote IP and port; more lenient than net.SplitHostPort
 	var ip, port string
@@ -200,21 +268,24 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 	// Strip PATH_INFO from SCRIPT_NAME
 	scriptName = strings.TrimSuffix(scriptName, pathInfo)
 
-	// Get the request URI. The request URI might be as it came in over the wire,
-	// or it might have been rewritten internally by the rewrite middleware (see issue #256).
-	// If it was rewritten, there will be a header indicating the original URL,
-	// which is needed to get the correct RequestURI value for PHP apps.
-	const internalRewriteFieldName = "Caddy-Rewrite-Original-URI"
-	reqURI := r.URL.RequestURI()
-	if origURI := r.Header.Get(internalRewriteFieldName); origURI != "" {
-		reqURI = origURI
-		r.Header.Del(internalRewriteFieldName)
-	}
+	// Add vhost path prefix to scriptName. Otherwise, some PHP software will
+	// have difficulty discovering its URL.
+	pathPrefix, _ := r.Context().Value(caddy.CtxKey("path_prefix")).(string)
+	scriptName = path.Join(pathPrefix, scriptName)
+
+	// Get the request URI from context. The context stores the original URI in case
+	// it was changed by a middleware such as rewrite. By default, we pass the
+	// original URI in as the value of REQUEST_URI (the user can overwrite this
+	// if desired). Most PHP apps seem to want the original URI. Besides, this is
+	// how nginx defaults: http://stackoverflow.com/a/12485156/1048862
+	reqURL, _ := r.Context().Value(httpserver.OriginalURLCtxKey).(url.URL)
+
+	// Retrieve name of remote user that was set by some downstream middleware such as basicauth.
+	remoteUser, _ := r.Context().Value(httpserver.RemoteUserCtxKey).(string)
 
 	// Some variables are unused but cleared explicitly to prevent
 	// the parent environment from interfering.
 	env = map[string]string{
-
 		// Variables defined in CGI 1.1 spec
 		"AUTH_TYPE":         "", // Not used
 		"CONTENT_LENGTH":    r.Header.Get("Content-Length"),
@@ -226,7 +297,7 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 		"REMOTE_HOST":       ip, // For speed, remote host lookups disabled
 		"REMOTE_PORT":       port,
 		"REMOTE_IDENT":      "", // Not used
-		"REMOTE_USER":       "", // Not used
+		"REMOTE_USER":       remoteUser,
 		"REQUEST_METHOD":    r.Method,
 		"SERVER_NAME":       h.ServerName,
 		"SERVER_PORT":       h.ServerPort,
@@ -234,19 +305,19 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 		"SERVER_SOFTWARE":   h.SoftwareName + "/" + h.SoftwareVersion,
 
 		// Other variables
-		"DOCUMENT_ROOT":   h.AbsRoot,
+		"DOCUMENT_ROOT":   rule.Root,
 		"DOCUMENT_URI":    docURI,
 		"HTTP_HOST":       r.Host, // added here, since not always part of headers
-		"REQUEST_URI":     reqURI,
+		"REQUEST_URI":     reqURL.RequestURI(),
 		"SCRIPT_FILENAME": scriptFilename,
 		"SCRIPT_NAME":     scriptName,
 	}
 
-	// compliance with the CGI specification that PATH_TRANSLATED
-	// should only exist if PATH_INFO is defined.
+	// compliance with the CGI specification requires that
+	// PATH_TRANSLATED should only exist if PATH_INFO is defined.
 	// Info: https://www.ietf.org/rfc/rfc3875 Page 14
 	if env["PATH_INFO"] != "" {
-		env["PATH_TRANSLATED"] = filepath.Join(h.AbsRoot, pathInfo) // Info: http://www.oreilly.com/openbook/cgi/ch02_04.html
+		env["PATH_TRANSLATED"] = filepath.Join(rule.Root, pathInfo) // Info: http://www.oreilly.com/openbook/cgi/ch02_04.html
 	}
 
 	// Some web apps rely on knowing HTTPS or not
@@ -254,9 +325,10 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 		env["HTTPS"] = "on"
 	}
 
-	// Add env variables from config
+	// Add env variables from config (with support for placeholders in values)
+	replacer := httpserver.NewReplacer(r, nil, "")
 	for _, envVar := range rule.EnvVars {
-		env[envVar[0]] = envVar[1]
+		env[envVar[0]] = replacer.Replace(envVar[1])
 	}
 
 	// Add all HTTP headers to env variables
@@ -265,21 +337,25 @@ func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]
 		header = headerNameReplacer.Replace(header)
 		env["HTTP_"+header] = strings.Join(val, ", ")
 	}
-
 	return env, nil
 }
 
 // Rule represents a FastCGI handling rule.
+// It is parsed from the fastcgi directive in the Caddyfile, see setup.go.
 type Rule struct {
 	// The base path to match. Required.
 	Path string
 
-	// The address of the FastCGI server. Required.
-	Address string
+	// upstream load balancer
+	balancer
 
 	// Always process files with this extension with fastcgi.
 	Ext string
 
+	// Use this directory as the fastcgi root directory. Defaults to the root
+	// directory of the parent virtual host.
+	Root string
+
 	// The path in the URL will be split into two, with the first piece ending
 	// with the value of SplitPath. The first piece will be assumed as the
 	// actual resource (CGI script) name, and the second piece will be set to
@@ -295,6 +371,61 @@ type Rule struct {
 
 	// Ignored paths
 	IgnoredSubPaths []string
+
+	// The duration used to set a deadline when connecting to an upstream.
+	ConnectTimeout time.Duration
+
+	// The duration used to set a deadline when reading from the FastCGI server.
+	ReadTimeout time.Duration
+
+	// The duration used to set a deadline when sending to the FastCGI server.
+	SendTimeout time.Duration
+}
+
+// balancer is a fastcgi upstream load balancer.
+type balancer interface {
+	// Address picks an upstream address from the
+	// underlying load balancer.
+	Address() (string, error)
+}
+
+// roundRobin is a round robin balancer for fastcgi upstreams.
+type roundRobin struct {
+	// Known Go bug: https://golang.org/pkg/sync/atomic/#pkg-note-BUG
+	// must be first field for 64 bit alignment
+	// on x86 and arm.
+	index     int64
+	addresses []string
+}
+
+func (r *roundRobin) Address() (string, error) {
+	index := atomic.AddInt64(&r.index, 1) % int64(len(r.addresses))
+	return r.addresses[index], nil
+}
+
+// srvResolver is a private interface used to abstract
+// the DNS resolver. It is mainly used to facilitate testing.
+type srvResolver interface {
+	LookupSRV(ctx context.Context, service, proto, name string) (string, []*net.SRV, error)
+}
+
+// srv is a service locator for fastcgi upstreams
+type srv struct {
+	resolver srvResolver
+	service  string
+}
+
+// Address looks up the service and returns the address:port
+// from first result in resolved list.
+// No explicit balancing is required because net.LookupSRV
+// sorts the results by priority and randomizes within priority.
+func (s *srv) Address() (string, error) {
+	_, addrs, err := s.resolver.LookupSRV(context.Background(), "", "", s.service)
+	if err != nil {
+		return "", err
+	}
+
+	return fmt.Sprintf("%s:%d", strings.TrimRight(addrs[0].Target, "."), addrs[0].Port), nil
 }
 
 // canSplit checks if path can split into two based on rule.SplitPath.

caddyhttp/fastcgi/fastcgi_test.go

@@ -1,13 +1,33 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package fastcgi
 
 import (
+	"context"
 	"net"
 	"net/http"
 	"net/http/fcgi"
 	"net/http/httptest"
 	"net/url"
 	"strconv"
+	"sync"
 	"testing"
+	"time"
+
+	"github.com/mholt/caddy"
+	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
 
 func TestServeHTTP(t *testing.T) {
@@ -26,7 +46,7 @@ func TestServeHTTP(t *testing.T) {
 
 	handler := Handler{
 		Next:  nil,
-		Rules: []Rule{{Path: "/", Address: listener.Addr().String()}},
+		Rules: []Rule{{Path: "/", balancer: address(listener.Addr().String())}},
 	}
 	r, err := http.NewRequest("GET", "/", nil)
 	if err != nil {
@@ -56,19 +76,23 @@ func TestRuleParseAddress(t *testing.T) {
 		expectednetwork string
 		expectedaddress string
 	}{
-		{&Rule{Address: "tcp://172.17.0.1:9000"}, "tcp", "172.17.0.1:9000"},
-		{&Rule{Address: "fastcgi://localhost:9000"}, "tcp", "localhost:9000"},
-		{&Rule{Address: "172.17.0.15"}, "tcp", "172.17.0.15"},
-		{&Rule{Address: "/my/unix/socket"}, "unix", "/my/unix/socket"},
-		{&Rule{Address: "unix:/second/unix/socket"}, "unix", "/second/unix/socket"},
+		{&Rule{balancer: address("tcp://172.17.0.1:9000")}, "tcp", "172.17.0.1:9000"},
+		{&Rule{balancer: address("fastcgi://localhost:9000")}, "tcp", "localhost:9000"},
+		{&Rule{balancer: address("172.17.0.15")}, "tcp", "172.17.0.15"},
+		{&Rule{balancer: address("/my/unix/socket")}, "unix", "/my/unix/socket"},
+		{&Rule{balancer: address("unix:/second/unix/socket")}, "unix", "/second/unix/socket"},
 	}
 
 	for _, entry := range getClientTestTable {
-		if actualnetwork, _ := entry.rule.parseAddress(); actualnetwork != entry.expectednetwork {
-			t.Errorf("Unexpected network for address string %v. Got %v, expected %v", entry.rule.Address, actualnetwork, entry.expectednetwork)
+		addr, err := entry.rule.Address()
+		if err != nil {
+			t.Errorf("Unexpected error in retrieving address: %s", err.Error())
 		}
-		if _, actualaddress := entry.rule.parseAddress(); actualaddress != entry.expectedaddress {
-			t.Errorf("Unexpected parsed address for address string %v. Got %v, expected %v", entry.rule.Address, actualaddress, entry.expectedaddress)
+		if actualnetwork, _ := parseAddress(addr); actualnetwork != entry.expectednetwork {
+			t.Errorf("Unexpected network for address string %v. Got %v, expected %v", addr, actualnetwork, entry.expectednetwork)
+		}
+		if _, actualaddress := parseAddress(addr); actualaddress != entry.expectedaddress {
+			t.Errorf("Unexpected parsed address for address string %v. Got %v, expected %v", addr, actualaddress, entry.expectedaddress)
 		}
 	}
 }
@@ -117,44 +141,248 @@ func TestBuildEnv(t *testing.T) {
 		}
 	}
 
-	rule := Rule{}
-	url, err := url.Parse("http://localhost:2015/fgci_test.php?test=blabla")
+	rule := Rule{
+		Ext:        ".php",
+		SplitPath:  ".php",
+		IndexFiles: []string{"index.php"},
+	}
+	url, err := url.Parse("http://localhost:2015/fgci_test.php?test=foobar")
 	if err != nil {
 		t.Error("Unexpected error:", err.Error())
 	}
 
-	r := http.Request{
-		Method:     "GET",
-		URL:        url,
-		Proto:      "HTTP/1.1",
-		ProtoMajor: 1,
-		ProtoMinor: 1,
-		Host:       "localhost:2015",
-		RemoteAddr: "[2b02:1810:4f2d:9400:70ab:f822:be8a:9093]:51688",
-		RequestURI: "/fgci_test.php",
+	var newReq = func() *http.Request {
+		r := http.Request{
+			Method:     "GET",
+			URL:        url,
+			Proto:      "HTTP/1.1",
+			ProtoMajor: 1,
+			ProtoMinor: 1,
+			Host:       "localhost:2015",
+			RemoteAddr: "[2b02:1810:4f2d:9400:70ab:f822:be8a:9093]:51688",
+			RequestURI: "/fgci_test.php",
+			Header: map[string][]string{
+				"Foo": {"Bar", "two"},
+			},
+		}
+		ctx := context.WithValue(r.Context(), httpserver.OriginalURLCtxKey, *r.URL)
+		return r.WithContext(ctx)
 	}
 
 	fpath := "/fgci_test.php"
 
-	var envExpected = map[string]string{
-		"REMOTE_ADDR":     "2b02:1810:4f2d:9400:70ab:f822:be8a:9093",
-		"REMOTE_PORT":     "51688",
-		"SERVER_PROTOCOL": "HTTP/1.1",
-		"QUERY_STRING":    "test=blabla",
-		"REQUEST_METHOD":  "GET",
-		"HTTP_HOST":       "localhost:2015",
+	var newEnv = func() map[string]string {
+		return map[string]string{
+			"REMOTE_ADDR":     "2b02:1810:4f2d:9400:70ab:f822:be8a:9093",
+			"REMOTE_PORT":     "51688",
+			"SERVER_PROTOCOL": "HTTP/1.1",
+			"QUERY_STRING":    "test=foobar",
+			"REQUEST_METHOD":  "GET",
+			"HTTP_HOST":       "localhost:2015",
+			"SCRIPT_NAME":     "/fgci_test.php",
+		}
 	}
 
+	// request
+	var r *http.Request
+
+	// expected environment variables
+	var envExpected map[string]string
+
 	// 1. Test for full canonical IPv6 address
-	testBuildEnv(&r, rule, fpath, envExpected)
+	r = newReq()
+	testBuildEnv(r, rule, fpath, envExpected)
 
 	// 2. Test for shorthand notation of IPv6 address
+	r = newReq()
 	r.RemoteAddr = "[::1]:51688"
+	envExpected = newEnv()
 	envExpected["REMOTE_ADDR"] = "::1"
-	testBuildEnv(&r, rule, fpath, envExpected)
+	testBuildEnv(r, rule, fpath, envExpected)
 
 	// 3. Test for IPv4 address
+	r = newReq()
 	r.RemoteAddr = "192.168.0.10:51688"
+	envExpected = newEnv()
 	envExpected["REMOTE_ADDR"] = "192.168.0.10"
-	testBuildEnv(&r, rule, fpath, envExpected)
+	testBuildEnv(r, rule, fpath, envExpected)
+
+	// 4. Test for environment variable
+	r = newReq()
+	rule.EnvVars = [][2]string{
+		{"HTTP_HOST", "localhost:2016"},
+		{"REQUEST_METHOD", "POST"},
+	}
+	envExpected = newEnv()
+	envExpected["HTTP_HOST"] = "localhost:2016"
+	envExpected["REQUEST_METHOD"] = "POST"
+	testBuildEnv(r, rule, fpath, envExpected)
+
+	// 5. Test for environment variable placeholders
+	r = newReq()
+	rule.EnvVars = [][2]string{
+		{"HTTP_HOST", "{host}"},
+		{"CUSTOM_URI", "custom_uri{uri}"},
+		{"CUSTOM_QUERY", "custom=true&{query}"},
+	}
+	envExpected = newEnv()
+	envExpected["HTTP_HOST"] = "localhost:2015"
+	envExpected["CUSTOM_URI"] = "custom_uri/fgci_test.php?test=foobar"
+	envExpected["CUSTOM_QUERY"] = "custom=true&test=foobar"
+	testBuildEnv(r, rule, fpath, envExpected)
+
+	// 6. Test SCRIPT_NAME includes path prefix
+	r = newReq()
+	ctx := context.WithValue(r.Context(), caddy.CtxKey("path_prefix"), "/test")
+	r = r.WithContext(ctx)
+	envExpected = newEnv()
+	envExpected["SCRIPT_NAME"] = "/test/fgci_test.php"
+	testBuildEnv(r, rule, fpath, envExpected)
+}
+
+func TestReadTimeout(t *testing.T) {
+	tests := []struct {
+		sleep       time.Duration
+		readTimeout time.Duration
+		shouldErr   bool
+	}{
+		{75 * time.Millisecond, 50 * time.Millisecond, true},
+		{0, -1 * time.Second, true},
+		{0, time.Minute, false},
+	}
+
+	var wg sync.WaitGroup
+
+	for i, test := range tests {
+		listener, err := net.Listen("tcp", "127.0.0.1:0")
+		if err != nil {
+			t.Fatalf("Test %d: Unable to create listener for test: %v", i, err)
+		}
+		defer listener.Close()
+
+		handler := Handler{
+			Next: nil,
+			Rules: []Rule{
+				{
+					Path:        "/",
+					balancer:    address(listener.Addr().String()),
+					ReadTimeout: test.readTimeout,
+				},
+			},
+		}
+		r, err := http.NewRequest("GET", "/", nil)
+		if err != nil {
+			t.Fatalf("Test %d: Unable to create request: %v", i, err)
+		}
+		w := httptest.NewRecorder()
+
+		wg.Add(1)
+		go fcgi.Serve(listener, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			time.Sleep(test.sleep)
+			w.WriteHeader(http.StatusOK)
+			wg.Done()
+		}))
+
+		got, err := handler.ServeHTTP(w, r)
+		if test.shouldErr {
+			if err == nil {
+				t.Errorf("Test %d: Expected i/o timeout error but had none", i)
+			} else if err, ok := err.(net.Error); !ok || !err.Timeout() {
+				t.Errorf("Test %d: Expected i/o timeout error, got: '%s'", i, err.Error())
+			}
+
+			want := http.StatusGatewayTimeout
+			if got != want {
+				t.Errorf("Test %d: Expected returned status code to be %d, got: %d",
+					i, want, got)
+			}
+		} else if err != nil {
+			t.Errorf("Test %d: Expected nil error, got: %v", i, err)
+		}
+
+		wg.Wait()
+	}
+}
+
+func TestSendTimeout(t *testing.T) {
+	tests := []struct {
+		sendTimeout time.Duration
+		shouldErr   bool
+	}{
+		{-1 * time.Second, true},
+		{time.Minute, false},
+	}
+
+	for i, test := range tests {
+		listener, err := net.Listen("tcp", "127.0.0.1:0")
+		if err != nil {
+			t.Fatalf("Test %d: Unable to create listener for test: %v", i, err)
+		}
+		defer listener.Close()
+
+		handler := Handler{
+			Next: nil,
+			Rules: []Rule{
+				{
+					Path:        "/",
+					balancer:    address(listener.Addr().String()),
+					SendTimeout: test.sendTimeout,
+				},
+			},
+		}
+		r, err := http.NewRequest("GET", "/", nil)
+		if err != nil {
+			t.Fatalf("Test %d: Unable to create request: %v", i, err)
+		}
+		w := httptest.NewRecorder()
+
+		go fcgi.Serve(listener, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			w.WriteHeader(http.StatusOK)
+		}))
+
+		got, err := handler.ServeHTTP(w, r)
+		if test.shouldErr {
+			if err == nil {
+				t.Errorf("Test %d: Expected i/o timeout error but had none", i)
+			} else if err, ok := err.(net.Error); !ok || !err.Timeout() {
+				t.Errorf("Test %d: Expected i/o timeout error, got: '%s'", i, err.Error())
+			}
+
+			want := http.StatusGatewayTimeout
+			if got != want {
+				t.Errorf("Test %d: Expected returned status code to be %d, got: %d",
+					i, want, got)
+			}
+		} else if err != nil {
+			t.Errorf("Test %d: Expected nil error, got: %v", i, err)
+		}
+	}
+}
+
+func TestBalancer(t *testing.T) {
+	tests := [][]string{
+		{"localhost", "host.local"},
+		{"localhost"},
+		{"localhost", "host.local", "example.com"},
+		{"localhost", "host.local", "example.com", "127.0.0.1"},
+	}
+	for i, test := range tests {
+		b := address(test...)
+		for _, host := range test {
+			a, err := b.Address()
+			if err != nil {
+				t.Errorf("Unexpected error in trying to retrieve address: %s", err.Error())
+			}
+			if a != host {
+				t.Errorf("Test %d: expected %s, found %s", i, host, a)
+			}
+		}
+	}
+}
+
+func address(addresses ...string) balancer {
+	return &roundRobin{
+		addresses: addresses,
+		index:     -1,
+	}
 }

caddyhttp/fastcgi/fcgiclient.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Forked Jan. 2015 from http://bitbucket.org/PinIdea/fcgi_client
 // (which is forked from https://code.google.com/p/go-fastcgi-client/)
 
@@ -13,6 +27,7 @@ package fastcgi
 import (
 	"bufio"
 	"bytes"
+	"context"
 	"encoding/binary"
 	"errors"
 	"io"
@@ -28,6 +43,7 @@ import (
 	"strconv"
 	"strings"
 	"sync"
+	"time"
 )
 
 // FCGIListenSockFileno describes listen socket file number.
@@ -44,7 +60,6 @@ const FCGINullRequestID uint8 = 0
 
 // FCGIKeepConn describes keep connection mode.
 const FCGIKeepConn uint8 = 1
-const doubleCRLF = "\r\n\r\n"
 
 const (
 	// BeginRequest is the begin request flag.
@@ -160,20 +175,23 @@ func (rec *record) read(r io.Reader) (buf []byte, err error) {
 // FCGIClient implements a FastCGI client, which is a standard for
 // interfacing external applications with Web servers.
 type FCGIClient struct {
-	mutex     sync.Mutex
-	rwc       io.ReadWriteCloser
-	h         header
-	buf       bytes.Buffer
-	stderr    bytes.Buffer
-	keepAlive bool
-	reqID     uint16
+	mutex       sync.Mutex
+	rwc         io.ReadWriteCloser
+	h           header
+	buf         bytes.Buffer
+	stderr      bytes.Buffer
+	keepAlive   bool
+	reqID       uint16
+	readTimeout time.Duration
+	sendTimeout time.Duration
 }
 
-// DialWithDialer connects to the fcgi responder at the specified network address, using custom net.Dialer.
+// DialWithDialerContext connects to the fcgi responder at the specified network address, using custom net.Dialer
+// and a context.
 // See func net.Dial for a description of the network and address parameters.
-func DialWithDialer(network, address string, dialer net.Dialer) (fcgi *FCGIClient, err error) {
+func DialWithDialerContext(ctx context.Context, network, address string, dialer net.Dialer) (fcgi *FCGIClient, err error) {
 	var conn net.Conn
-	conn, err = dialer.Dial(network, address)
+	conn, err = dialer.DialContext(ctx, network, address)
 	if err != nil {
 		return
 	}
@@ -187,10 +205,15 @@ func DialWithDialer(network, address string, dialer net.Dialer) (fcgi *FCGIClien
 	return
 }
 
+// DialContext is like Dial but passes ctx to dialer.Dial.
+func DialContext(ctx context.Context, network, address string) (fcgi *FCGIClient, err error) {
+	return DialWithDialerContext(ctx, network, address, net.Dialer{})
+}
+
 // Dial connects to the fcgi responder at the specified network address, using default net.Dialer.
 // See func net.Dial for a description of the network and address parameters.
 func Dial(network, address string) (fcgi *FCGIClient, err error) {
-	return DialWithDialer(network, address, net.Dialer{})
+	return DialContext(context.Background(), network, address)
 }
 
 // Close closes fcgi connnection
@@ -261,29 +284,6 @@ func (c *FCGIClient) writePairs(recType uint8, pairs map[string]string) error {
 	return nil
 }
 
-func readSize(s []byte) (uint32, int) {
-	if len(s) == 0 {
-		return 0, 0
-	}
-	size, n := uint32(s[0]), 1
-	if size&(1<<7) != 0 {
-		if len(s) < 4 {
-			return 0, 0
-		}
-		n = 4
-		size = binary.BigEndian.Uint32(s)
-		size &^= 1 << 31
-	}
-	return size, n
-}
-
-func readString(s []byte, size uint32) string {
-	if size > uint32(len(s)) {
-		return ""
-	}
-	return string(s[:size])
-}
-
 func encodeSize(b []byte, size uint32) int {
 	if size > 127 {
 		size |= 1 << 31
@@ -417,7 +417,6 @@ func (f clientCloser) Close() error { return f.rwc.Close() }
 // Request returns a HTTP Response with Header and Body
 // from fcgi responder
 func (c *FCGIClient) Request(p map[string]string, req io.Reader) (resp *http.Response, err error) {
-
 	r, err := c.Do(p, req)
 	if err != nil {
 		return
@@ -461,12 +460,12 @@ func (c *FCGIClient) Request(p map[string]string, req io.Reader) (resp *http.Res
 }
 
 // Get issues a GET request to the fcgi responder.
-func (c *FCGIClient) Get(p map[string]string) (resp *http.Response, err error) {
+func (c *FCGIClient) Get(p map[string]string, body io.Reader, l int64) (resp *http.Response, err error) {
 
 	p["REQUEST_METHOD"] = "GET"
-	p["CONTENT_LENGTH"] = "0"
+	p["CONTENT_LENGTH"] = strconv.FormatInt(l, 10)
 
-	return c.Request(p, nil)
+	return c.Request(p, body)
 }
 
 // Head issues a HEAD request to the fcgi responder.
@@ -489,7 +488,7 @@ func (c *FCGIClient) Options(p map[string]string) (resp *http.Response, err erro
 
 // Post issues a POST request to the fcgi responder. with request body
 // in the format that bodyType specified
-func (c *FCGIClient) Post(p map[string]string, method string, bodyType string, body io.Reader, l int) (resp *http.Response, err error) {
+func (c *FCGIClient) Post(p map[string]string, method string, bodyType string, body io.Reader, l int64) (resp *http.Response, err error) {
 	if p == nil {
 		p = make(map[string]string)
 	}
@@ -500,7 +499,7 @@ func (c *FCGIClient) Post(p map[string]string, method string, bodyType string, b
 		p["REQUEST_METHOD"] = "POST"
 	}
 
-	p["CONTENT_LENGTH"] = strconv.Itoa(l)
+	p["CONTENT_LENGTH"] = strconv.FormatInt(l, 10)
 	if len(bodyType) > 0 {
 		p["CONTENT_TYPE"] = bodyType
 	} else {
@@ -514,7 +513,7 @@ func (c *FCGIClient) Post(p map[string]string, method string, bodyType string, b
 // as a string key to a list values (url.Values)
 func (c *FCGIClient) PostForm(p map[string]string, data url.Values) (resp *http.Response, err error) {
 	body := bytes.NewReader([]byte(data.Encode()))
-	return c.Post(p, "POST", "application/x-www-form-urlencoded", body, body.Len())
+	return c.Post(p, "POST", "application/x-www-form-urlencoded", body, int64(body.Len()))
 }
 
 // PostFile issues a POST to the fcgi responder in multipart(RFC 2046) standard,
@@ -546,6 +545,9 @@ func (c *FCGIClient) PostFile(p map[string]string, data url.Values, file map[str
 			return nil, e
 		}
 		_, err = io.Copy(part, fd)
+		if err != nil {
+			return
+		}
 	}
 
 	err = writer.Close()
@@ -553,7 +555,25 @@ func (c *FCGIClient) PostFile(p map[string]string, data url.Values, file map[str
 		return
 	}
 
-	return c.Post(p, "POST", bodyType, buf, buf.Len())
+	return c.Post(p, "POST", bodyType, buf, int64(buf.Len()))
+}
+
+// SetReadTimeout sets the read timeout for future calls that read from the
+// fcgi responder. A zero value for t means no timeout will be set.
+func (c *FCGIClient) SetReadTimeout(t time.Duration) error {
+	if conn, ok := c.rwc.(net.Conn); ok && t != 0 {
+		return conn.SetReadDeadline(time.Now().Add(t))
+	}
+	return nil
+}
+
+// SetSendTimeout sets the read timeout for future calls that send data to
+// the fcgi responder. A zero value for t means no timeout will be set.
+func (c *FCGIClient) SetSendTimeout(t time.Duration) error {
+	if conn, ok := c.rwc.(net.Conn); ok && t != 0 {
+		return conn.SetWriteDeadline(time.Now().Add(t))
+	}
+	return nil
 }
 
 // Checks whether chunked is part of the encodings stack

caddyhttp/fastcgi/fcgiclient_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // NOTE: These tests were adapted from the original
 // repository from which this package was forked.
 // The tests are slow (~10s) and in dire need of rewriting.
@@ -117,7 +131,7 @@ func sendFcgi(reqType int, fcgiParams map[string]string, data []byte, posts map[
 		if len(data) > 0 {
 			length = len(data)
 			rd := bytes.NewReader(data)
-			resp, err = fcgi.Post(fcgiParams, "", "", rd, rd.Len())
+			resp, err = fcgi.Post(fcgiParams, "", "", rd, int64(rd.Len()))
 		} else if len(posts) > 0 {
 			values := url.Values{}
 			for k, v := range posts {
@@ -126,7 +140,8 @@ func sendFcgi(reqType int, fcgiParams map[string]string, data []byte, posts map[
 			}
 			resp, err = fcgi.PostForm(fcgiParams, values)
 		} else {
-			resp, err = fcgi.Get(fcgiParams)
+			rd := bytes.NewReader(data)
+			resp, err = fcgi.Get(fcgiParams, rd, int64(rd.Len()))
 		}
 
 	default:
@@ -155,7 +170,7 @@ func sendFcgi(reqType int, fcgiParams map[string]string, data []byte, posts map[
 	fcgi.Close()
 	time.Sleep(1 * time.Second)
 
-	if bytes.Index(content, []byte("FAILED")) >= 0 {
+	if bytes.Contains(content, []byte("FAILED")) {
 		globalt.Error("Server return failed message")
 	}
 

caddyhttp/fastcgi/setup.go

@@ -1,9 +1,27 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package fastcgi
 
 import (
 	"errors"
+	"fmt"
+	"net"
 	"net/http"
 	"path/filepath"
+	"strings"
+	"time"
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -19,10 +37,6 @@ func init() {
 // setup configures a new FastCGI middleware instance.
 func setup(c *caddy.Controller) error {
 	cfg := httpserver.GetConfig(c)
-	absRoot, err := filepath.Abs(cfg.Root)
-	if err != nil {
-		return err
-	}
 
 	rules, err := fastcgiParse(c)
 	if err != nil {
@@ -34,7 +48,6 @@ func setup(c *caddy.Controller) error {
 			Next:            next,
 			Rules:           rules,
 			Root:            cfg.Root,
-			AbsRoot:         absRoot,
 			FileSys:         http.Dir(cfg.Root),
 			SoftwareName:    caddy.AppName,
 			SoftwareVersion: caddy.AppVersion,
@@ -49,31 +62,47 @@ func setup(c *caddy.Controller) error {
 func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 	var rules []Rule
 
-	for c.Next() {
-		var rule Rule
+	cfg := httpserver.GetConfig(c)
+	absRoot, err := filepath.Abs(cfg.Root)
+	if err != nil {
+		return nil, err
+	}
 
+	for c.Next() {
 		args := c.RemainingArgs()
 
-		switch len(args) {
-		case 0:
+		if len(args) < 2 || len(args) > 3 {
 			return rules, c.ArgErr()
-		case 1:
-			rule.Path = "/"
-			rule.Address = args[0]
-		case 2:
-			rule.Path = args[0]
-			rule.Address = args[1]
-		case 3:
-			rule.Path = args[0]
-			rule.Address = args[1]
-			err := fastcgiPreset(args[2], &rule)
-			if err != nil {
-				return rules, c.Err("Invalid fastcgi rule preset '" + args[2] + "'")
+		}
+
+		rule := Rule{
+			Root: absRoot,
+			Path: args[0],
+		}
+
+		upstreams := []string{args[1]}
+
+		srvUpstream := false
+		if strings.HasPrefix(upstreams[0], "srv://") {
+			srvUpstream = true
+		}
+
+		if len(args) == 3 {
+			if err := fastcgiPreset(args[2], &rule); err != nil {
+				return rules, err
 			}
 		}
 
+		var err error
+
 		for c.NextBlock() {
 			switch c.Val() {
+			case "root":
+				if !c.NextArg() {
+					return rules, c.ArgErr()
+				}
+				rule.Root = c.Val()
+
 			case "ext":
 				if !c.NextArg() {
 					return rules, c.ArgErr()
@@ -90,6 +119,19 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 					return rules, c.ArgErr()
 				}
 				rule.IndexFiles = args
+
+			case "upstream":
+				if srvUpstream {
+					return rules, c.Err("additional upstreams are not supported with SRV upstream")
+				}
+
+				args := c.RemainingArgs()
+
+				if len(args) != 1 {
+					return rules, c.ArgErr()
+				}
+
+				upstreams = append(upstreams, args[0])
 			case "env":
 				envArgs := c.RemainingArgs()
 				if len(envArgs) < 2 {
@@ -102,15 +144,62 @@ func fastcgiParse(c *caddy.Controller) ([]Rule, error) {
 					return rules, c.ArgErr()
 				}
 				rule.IgnoredSubPaths = ignoredPaths
+
+			case "connect_timeout":
+				if !c.NextArg() {
+					return rules, c.ArgErr()
+				}
+				rule.ConnectTimeout, err = time.ParseDuration(c.Val())
+				if err != nil {
+					return rules, err
+				}
+			case "read_timeout":
+				if !c.NextArg() {
+					return rules, c.ArgErr()
+				}
+				readTimeout, err := time.ParseDuration(c.Val())
+				if err != nil {
+					return rules, err
+				}
+				rule.ReadTimeout = readTimeout
+			case "send_timeout":
+				if !c.NextArg() {
+					return rules, c.ArgErr()
+				}
+				sendTimeout, err := time.ParseDuration(c.Val())
+				if err != nil {
+					return rules, err
+				}
+				rule.SendTimeout = sendTimeout
 			}
 		}
 
+		if srvUpstream {
+			balancer, err := parseSRV(upstreams[0])
+			if err != nil {
+				return rules, c.Err("malformed service locator string: " + err.Error())
+			}
+			rule.balancer = balancer
+		} else {
+			rule.balancer = &roundRobin{addresses: upstreams, index: -1}
+		}
+
 		rules = append(rules, rule)
 	}
-
 	return rules, nil
 }
 
+func parseSRV(locator string) (*srv, error) {
+	if locator[6:] == "" {
+		return nil, fmt.Errorf("%s does not include the host", locator)
+	}
+
+	return &srv{
+		service:  locator[6:],
+		resolver: &net.Resolver{},
+	}, nil
+}
+
 // fastcgiPreset configures rule according to name. It returns an error if
 // name is not a recognized preset name.
 func fastcgiPreset(name string, rule *Rule) error {

caddyhttp/fastcgi/setup_test.go

@@ -1,7 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package fastcgi
 
 import (
+	"context"
 	"fmt"
+	"net"
 	"testing"
 
 	"github.com/mholt/caddy"
@@ -29,10 +45,14 @@ func TestSetup(t *testing.T) {
 	if myHandler.Rules[0].Path != "/" {
 		t.Errorf("Expected / as the Path")
 	}
-	if myHandler.Rules[0].Address != "127.0.0.1:9000" {
-		t.Errorf("Expected 127.0.0.1:9000 as the Address")
+	addr, err := myHandler.Rules[0].Address()
+	if err != nil {
+		t.Errorf("Unexpected error in trying to retrieve address: %s", err.Error())
 	}
 
+	if addr != "127.0.0.1:9000" {
+		t.Errorf("Expected 127.0.0.1:9000 as the Address")
+	}
 }
 
 func TestFastcgiParse(t *testing.T) {
@@ -45,7 +65,7 @@ func TestFastcgiParse(t *testing.T) {
 		{`fastcgi /blog 127.0.0.1:9000 php`,
 			false, []Rule{{
 				Path:       "/blog",
-				Address:    "127.0.0.1:9000",
+				balancer:   &roundRobin{addresses: []string{"127.0.0.1:9000"}},
 				Ext:        ".php",
 				SplitPath:  ".php",
 				IndexFiles: []string{"index.php"},
@@ -55,7 +75,7 @@ func TestFastcgiParse(t *testing.T) {
 	              }`,
 			false, []Rule{{
 				Path:       "/",
-				Address:    "127.0.0.1:9001",
+				balancer:   &roundRobin{addresses: []string{"127.0.0.1:9001"}},
 				Ext:        "",
 				SplitPath:  ".html",
 				IndexFiles: []string{},
@@ -66,7 +86,7 @@ func TestFastcgiParse(t *testing.T) {
 	              }`,
 			false, []Rule{{
 				Path:            "/",
-				Address:         "127.0.0.1:9001",
+				balancer:        &roundRobin{addresses: []string{"127.0.0.1:9001"}},
 				Ext:             "",
 				SplitPath:       ".html",
 				IndexFiles:      []string{},
@@ -92,9 +112,19 @@ func TestFastcgiParse(t *testing.T) {
 					i, j, test.expectedFastcgiConfig[j].Path, actualFastcgiConfig.Path)
 			}
 
-			if actualFastcgiConfig.Address != test.expectedFastcgiConfig[j].Address {
+			actualAddr, err := actualFastcgiConfig.Address()
+			if err != nil {
+				t.Errorf("Test %d unexpected error in trying to retrieve %dth actual address: %s", i, j, err.Error())
+			}
+
+			expectedAddr, err := test.expectedFastcgiConfig[j].Address()
+			if err != nil {
+				t.Errorf("Test %d unexpected error in trying to retrieve %dth expected address: %s", i, j, err.Error())
+			}
+
+			if actualAddr != expectedAddr {
 				t.Errorf("Test %d expected %dth FastCGI Address to be  %s  , but got %s",
-					i, j, test.expectedFastcgiConfig[j].Address, actualFastcgiConfig.Address)
+					i, j, expectedAddr, actualAddr)
 			}
 
 			if actualFastcgiConfig.Ext != test.expectedFastcgiConfig[j].Ext {
@@ -120,3 +150,75 @@ func TestFastcgiParse(t *testing.T) {
 	}
 
 }
+
+func TestFastCGIResolveSRV(t *testing.T) {
+	tests := []struct {
+		inputFastcgiConfig string
+		locator            string
+		target             string
+		port               uint16
+		shouldErr          bool
+	}{
+		{
+			`fastcgi / srv://fpm.tcp.service.consul {
+				upstream yolo
+			}`,
+			"fpm.tcp.service.consul",
+			"127.0.0.1",
+			9000,
+			true,
+		},
+		{
+			`fastcgi / srv://fpm.tcp.service.consul`,
+			"fpm.tcp.service.consul",
+			"127.0.0.1",
+			9000,
+			false,
+		},
+	}
+
+	for i, test := range tests {
+		actualFastcgiConfigs, err := fastcgiParse(caddy.NewTestController("http", test.inputFastcgiConfig))
+
+		if err == nil && test.shouldErr {
+			t.Errorf("Test %d didn't error, but it should have", i)
+		} else if err != nil && !test.shouldErr {
+			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
+		}
+
+		for _, actualFastcgiConfig := range actualFastcgiConfigs {
+			resolver, ok := (actualFastcgiConfig.balancer).(*srv)
+			if !ok {
+				t.Errorf("Test %d upstream balancer is not srv", i)
+			}
+			resolver.resolver = buildTestResolver(test.target, test.port)
+
+			addr, err := actualFastcgiConfig.Address()
+			if err != nil {
+				t.Errorf("Test %d failed to retrieve upstream address. %s", i, err.Error())
+			}
+
+			expectedAddr := fmt.Sprintf("%s:%d", test.target, test.port)
+			if addr != expectedAddr {
+				t.Errorf("Test %d expected upstream address to be %s, got %s", i, expectedAddr, addr)
+			}
+		}
+	}
+}
+
+func buildTestResolver(target string, port uint16) srvResolver {
+	return &testSRVResolver{target, port}
+}
+
+type testSRVResolver struct {
+	target string
+	port   uint16
+}
+
+func (r *testSRVResolver) LookupSRV(ctx context.Context, service, proto, name string) (string, []*net.SRV, error) {
+	return "", []*net.SRV{
+		{Target: r.target,
+			Port:     r.port,
+			Priority: 1,
+			Weight:   1}}, nil
+}

caddyhttp/gzip/gzip.go

@@ -1,14 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package gzip provides a middleware layer that performs
 // gzip compression on the response.
 package gzip
 
 import (
-	"bufio"
-	"compress/gzip"
-	"fmt"
 	"io"
-	"io/ioutil"
-	"net"
 	"net/http"
 	"strings"
 
@@ -21,6 +30,8 @@ func init() {
 		ServerType: "http",
 		Action:     setup,
 	})
+
+	initWriterPool()
 }
 
 // Gzip is a middleware type which gzips HTTP responses. It is
@@ -54,19 +65,15 @@ outer:
 			}
 		}
 
-		// Delete this header so gzipping is not repeated later in the chain
-		r.Header.Del("Accept-Encoding")
-
 		// gzipWriter modifies underlying writer at init,
 		// use a discard writer instead to leave ResponseWriter in
 		// original form.
-		gzipWriter, err := newWriter(c, ioutil.Discard)
-		if err != nil {
-			// should not happen
-			return http.StatusInternalServerError, err
+		gzipWriter := getWriter(c.Level)
+		defer putWriter(c.Level, gzipWriter)
+		gz := &gzipResponseWriter{
+			Writer:                gzipWriter,
+			ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
 		}
-		defer gzipWriter.Close()
-		gz := &gzipResponseWriter{Writer: gzipWriter, ResponseWriter: w}
 
 		var rw http.ResponseWriter
 		// if no response filter is used
@@ -96,21 +103,11 @@ outer:
 	return g.Next.ServeHTTP(w, r)
 }
 
-// newWriter create a new Gzip Writer based on the compression level.
-// If the level is valid (i.e. between 1 and 9), it uses the level.
-// Otherwise, it uses default compression level.
-func newWriter(c Config, w io.Writer) (*gzip.Writer, error) {
-	if c.Level >= gzip.BestSpeed && c.Level <= gzip.BestCompression {
-		return gzip.NewWriterLevel(w, c.Level)
-	}
-	return gzip.NewWriter(w), nil
-}
-
 // gzipResponeWriter wraps the underlying Write method
 // with a gzip.Writer to compress the output.
 type gzipResponseWriter struct {
 	io.Writer
-	http.ResponseWriter
+	*httpserver.ResponseWriterWrapper
 	statusCodeWritten bool
 }
 
@@ -122,7 +119,11 @@ func (w *gzipResponseWriter) WriteHeader(code int) {
 	w.Header().Del("Content-Length")
 	w.Header().Set("Content-Encoding", "gzip")
 	w.Header().Add("Vary", "Accept-Encoding")
-	w.ResponseWriter.WriteHeader(code)
+	originalEtag := w.Header().Get("ETag")
+	if originalEtag != "" && !strings.HasPrefix(originalEtag, "W/") {
+		w.Header().Set("ETag", "W/"+originalEtag)
+	}
+	w.ResponseWriterWrapper.WriteHeader(code)
 	w.statusCodeWritten = true
 }
 
@@ -138,30 +139,5 @@ func (w *gzipResponseWriter) Write(b []byte) (int, error) {
 	return n, err
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (w *gzipResponseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := w.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, fmt.Errorf("not a Hijacker")
-}
-
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or panics.
-func (w *gzipResponseWriter) Flush() {
-	if f, ok := w.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic("not a Flusher") // should be recovered at the beginning of middleware stack
-	}
-}
-
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (w *gzipResponseWriter) CloseNotify() <-chan bool {
-	if cn, ok := w.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
-	}
-	panic("not a CloseNotifier")
-}
+// Interface guards
+var _ httpserver.HTTPInterfaces = (*gzipResponseWriter)(nil)

caddyhttp/gzip/gzip_test.go

@@ -1,6 +1,21 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package gzip
 
 import (
+	"compress/gzip"
 	"fmt"
 	"io/ioutil"
 	"net/http"
@@ -37,6 +52,14 @@ func TestGzipHandler(t *testing.T) {
 			t.Error(err)
 		}
 		r.Header.Set("Accept-Encoding", "gzip")
+		w.Header().Set("ETag", `"2n9cd"`)
+		_, err = gz.ServeHTTP(w, r)
+		if err != nil {
+			t.Error(err)
+		}
+
+		// The second pass, test if the ETag is already weak
+		w.Header().Set("ETag", `W/"2n9cd"`)
 		_, err = gz.ServeHTTP(w, r)
 		if err != nil {
 			t.Error(err)
@@ -77,6 +100,22 @@ func TestGzipHandler(t *testing.T) {
 			t.Error(err)
 		}
 	}
+
+	// test all levels
+	w = httptest.NewRecorder()
+	gz.Next = nextFunc(true)
+	for i := 0; i <= gzip.BestCompression; i++ {
+		gz.Configs[0].Level = i
+		r, err := http.NewRequest("GET", "/file.txt", nil)
+		if err != nil {
+			t.Error(err)
+		}
+		r.Header.Set("Accept-Encoding", "gzip")
+		_, err = gz.ServeHTTP(w, r)
+		if err != nil {
+			t.Error(err)
+		}
+	}
 }
 
 func nextFunc(shouldGzip bool) httpserver.Handler {
@@ -91,20 +130,21 @@ func nextFunc(shouldGzip bool) httpserver.Handler {
 		}
 
 		if shouldGzip {
-			if r.Header.Get("Accept-Encoding") != "" {
-				return 0, fmt.Errorf("Accept-Encoding header not expected")
-			}
 			if w.Header().Get("Content-Encoding") != "gzip" {
-				return 0, fmt.Errorf("Content-Encoding must be gzip, found %v", r.Header.Get("Content-Encoding"))
+				return 0, fmt.Errorf("Content-Encoding must be gzip, found %v", w.Header().Get("Content-Encoding"))
 			}
 			if w.Header().Get("Vary") != "Accept-Encoding" {
-				return 0, fmt.Errorf("Vary must be Accept-Encoding, found %v", r.Header.Get("Vary"))
+				return 0, fmt.Errorf("Vary must be Accept-Encoding, found %v", w.Header().Get("Vary"))
+			}
+			etag := w.Header().Get("ETag")
+			if etag != "" && etag != `W/"2n9cd"` {
+				return 0, fmt.Errorf("ETag must be converted to weak Etag, found %v", w.Header().Get("ETag"))
 			}
 			if _, ok := w.(*gzipResponseWriter); !ok {
 				return 0, fmt.Errorf("ResponseWriter should be gzipResponseWriter, found %T", w)
 			}
 			if strings.Contains(w.Header().Get("Content-Type"), "application/x-gzip") {
-				return 0, fmt.Errorf("Content type should not be gzip.")
+				return 0, fmt.Errorf("Content-Type should not be gzip")
 			}
 			return 0, nil
 		}
@@ -120,3 +160,37 @@ func nextFunc(shouldGzip bool) httpserver.Handler {
 		return 0, nil
 	})
 }
+
+func BenchmarkGzip(b *testing.B) {
+	pathFilter := PathFilter{make(Set)}
+	badPaths := []string{"/bad", "/nogzip", "/nongzip"}
+	for _, p := range badPaths {
+		pathFilter.IgnoredPaths.Add(p)
+	}
+	extFilter := ExtFilter{make(Set)}
+	for _, e := range []string{".txt", ".html", ".css", ".md"} {
+		extFilter.Exts.Add(e)
+	}
+	gz := Gzip{Configs: []Config{
+		{
+			RequestFilters: []RequestFilter{pathFilter, extFilter},
+		},
+	}}
+
+	w := httptest.NewRecorder()
+	gz.Next = nextFunc(true)
+	url := "/file.txt"
+	r, err := http.NewRequest("GET", url, nil)
+	if err != nil {
+		b.Fatal(err)
+	}
+	r.Header.Set("Accept-Encoding", "gzip")
+
+	b.ResetTimer()
+	for i := 0; i < b.N; i++ {
+		_, err = gz.ServeHTTP(w, r)
+		if err != nil {
+			b.Fatal(err)
+		}
+	}
+}

caddyhttp/gzip/requestfilter.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package gzip
 
 import (

caddyhttp/gzip/requestfilter_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package gzip
 
 import (

caddyhttp/gzip/responsefilter.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package gzip
 
 import (
@@ -25,6 +39,20 @@ func (l LengthFilter) ShouldCompress(w http.ResponseWriter) bool {
 	return l != 0 && int64(l) <= length
 }
 
+// SkipCompressedFilter is ResponseFilter that will discard already compressed responses
+type SkipCompressedFilter struct{}
+
+// ShouldCompress returns true if served file is not already compressed
+// encodings via https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
+func (n SkipCompressedFilter) ShouldCompress(w http.ResponseWriter) bool {
+	switch w.Header().Get("Content-Encoding") {
+	case "gzip", "compress", "deflate", "br":
+		return false
+	default:
+		return true
+	}
+}
+
 // ResponseFilterWriter validates ResponseFilters. It writes
 // gzip compressed data if ResponseFilters are satisfied or
 // uncompressed data otherwise.

caddyhttp/gzip/responsefilter_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package gzip
 
 import (
@@ -33,7 +47,7 @@ func TestLengthFilter(t *testing.T) {
 		for j, filter := range filters {
 			r := httptest.NewRecorder()
 			r.Header().Set("Content-Length", fmt.Sprint(ts.length))
-			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), r, false})
+			wWriter := NewResponseFilterWriter([]ResponseFilter{filter}, &gzipResponseWriter{gzip.NewWriter(r), &httpserver.ResponseWriterWrapper{ResponseWriter: r}, false})
 			if filter.ShouldCompress(wWriter) != ts.shouldCompress[j] {
 				t.Errorf("Test %v: Expected %v found %v", i, ts.shouldCompress[j], filter.ShouldCompress(r))
 			}
@@ -87,3 +101,26 @@ func TestResponseFilterWriter(t *testing.T) {
 		}
 	}
 }
+
+func TestResponseGzippedOutput(t *testing.T) {
+	server := Gzip{Configs: []Config{
+		{ResponseFilters: []ResponseFilter{SkipCompressedFilter{}}},
+	}}
+
+	server.Next = httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+		w.Header().Set("Content-Encoding", "gzip")
+		w.Write([]byte("gzipped"))
+		return 200, nil
+	})
+
+	r := urlRequest("/")
+	r.Header.Set("Accept-Encoding", "gzip")
+
+	w := httptest.NewRecorder()
+	server.ServeHTTP(w, r)
+	resp := w.Body.String()
+
+	if resp != "gzipped" {
+		t.Errorf("Expected output not to be gzipped")
+	}
+}

caddyhttp/gzip/setup.go

@@ -1,9 +1,26 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package gzip
 
 import (
+	"compress/gzip"
 	"fmt"
+	"io/ioutil"
 	"strconv"
 	"strings"
+	"sync"
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -106,6 +123,8 @@ func gzipParse(c *caddy.Controller) ([]Config, error) {
 			config.RequestFilters = append(config.RequestFilters, DefaultExtFilter())
 		}
 
+		config.ResponseFilters = append(config.ResponseFilters, SkipCompressedFilter{})
+
 		// Response Filters
 		// If min_length is specified, use it.
 		if int64(lengthFilter) != 0 {
@@ -117,3 +136,48 @@ func gzipParse(c *caddy.Controller) ([]Config, error) {
 
 	return configs, nil
 }
+
+// pool gzip.Writer according to compress level
+// so we can reuse allocations over time
+var (
+	writerPool             = map[int]*sync.Pool{}
+	defaultWriterPoolIndex int
+)
+
+func initWriterPool() {
+	var i int
+	newWriterPool := func(level int) *sync.Pool {
+		return &sync.Pool{
+			New: func() interface{} {
+				w, _ := gzip.NewWriterLevel(ioutil.Discard, level)
+				return w
+			},
+		}
+	}
+	for i = gzip.BestSpeed; i <= gzip.BestCompression; i++ {
+		writerPool[i] = newWriterPool(i)
+	}
+
+	// add default writer pool
+	defaultWriterPoolIndex = i
+	writerPool[defaultWriterPoolIndex] = newWriterPool(gzip.DefaultCompression)
+}
+
+func getWriter(level int) *gzip.Writer {
+	index := defaultWriterPoolIndex
+	if level >= gzip.BestSpeed && level <= gzip.BestCompression {
+		index = level
+	}
+	w := writerPool[index].Get().(*gzip.Writer)
+	w.Reset(ioutil.Discard)
+	return w
+}
+
+func putWriter(level int, w *gzip.Writer) {
+	index := defaultWriterPoolIndex
+	if level >= gzip.BestSpeed && level <= gzip.BestCompression {
+		index = level
+	}
+	w.Close()
+	writerPool[index].Put(w)
+}

caddyhttp/gzip/setup_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package gzip
 
 import (
@@ -99,3 +113,31 @@ func TestSetup(t *testing.T) {
 		}
 	}
 }
+
+func TestShouldAddResponseFilters(t *testing.T) {
+	configs, err := gzipParse(caddy.NewTestController("http", `gzip { min_length 654 }`))
+
+	if err != nil {
+		t.Errorf("Test expected no error but found: %v", err)
+	}
+	filters := 0
+
+	for _, config := range configs {
+		for _, filter := range config.ResponseFilters {
+			switch filter.(type) {
+			case SkipCompressedFilter:
+				filters++
+			case LengthFilter:
+				filters++
+
+				if filter != LengthFilter(654) {
+					t.Errorf("Expected LengthFilter to have length 654, got: %v", filter)
+				}
+			}
+		}
+
+		if filters != 2 {
+			t.Errorf("Expected 2 response filters to be registered, got: %v", filters)
+		}
+	}
+}

caddyhttp/gzip/testdata/test.txt

@@ -1,199 +1,308 @@
-Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus mattis, dolor et feugiat suscipit, nisi urna consectetur enim, et porta libero dui eget dolor. Sed ac enim aliquet, ornare ipsum a, aliquet augue. Fusce faucibus id mi at porta. Donec scelerisque consectetur rutrum. In aliquam est vel quam molestie porttitor. Sed eget nunc blandit, commodo dui ac, tempus mi. Nulla ac est ac leo hendrerit rhoncus. Etiam id finibus neque. In ultrices diam orci, eu tempor neque bibendum sed. Fusce facilisis mauris in finibus elementum. Pellentesque tristique bibendum diam, nec molestie velit facilisis quis. Nulla facilisi. Morbi nec velit eget massa blandit consequat sed hendrerit velit. Ut euismod elit sit amet dui venenatis, a luctus tortor vestibulum.
+Sigh view am high neat half to what. Sent late held than set why wife our. If an blessing building steepest. Agreement distrusts mrs six affection satisfied. Day blushes visitor end company old prevent chapter. Consider declared out expenses her concerns. No at indulgence conviction particular unsatiable boisterous discretion. Direct enough off others say eldest may exeter she. Possible all ignorant supplied get settling marriage recurred. 
 
-Aliquam consequat rutrum sagittis. Donec eros felis, ultricies quis elementum nec, consequat et ex. Nullam feugiat eu sapien nec mollis. Quisque porttitor tortor ipsum, quis aliquam diam tincidunt a. Praesent tortor lorem, finibus sit amet tempor ac, iaculis et nunc. Duis enim justo, gravida in pharetra id, rhoncus id nisi. Maecenas dui risus, accumsan ut nisi vitae, placerat dignissim risus. Quisque orci lectus, sodales a lacus sit amet, bibendum gravida odio. Duis consectetur, ante et vulputate convallis, augue augue mollis ligula, sed tincidunt nunc ligula vel enim. Donec faucibus pulvinar consectetur. Nam laoreet, dolor ac elementum vestibulum, eros justo fringilla leo, pretium fringilla sem nunc non sem. Proin at nisl eget leo hendrerit lacinia.
+Boy desirous families prepared gay reserved add ecstatic say. Replied joy age visitor nothing cottage. Mrs door paid led loud sure easy read. Hastily at perhaps as neither or ye fertile tedious visitor. Use fine bed none call busy dull when. Quiet ought match my right by table means. Principles up do in me favourable affronting. Twenty mother denied effect we to do on. 
 
-Aliquam id lacus mauris. Morbi et justo urna. Vestibulum venenatis pharetra lectus in pharetra. Fusce feugiat nisl nec enim rutrum finibus. Etiam a nunc metus. Phasellus non dignissim eros, non feugiat quam. Nunc velit ante, commodo at ex eu, fringilla semper tellus.
+Compliment interested discretion estimating on stimulated apartments oh. Dear so sing when in find read of call. As distrusts behaviour abilities defective is. Never at water me might. On formed merits hunted unable merely by mr whence or. Possession the unpleasing simplicity her uncommonly. 
 
-Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Etiam rhoncus ultricies nisl, id vestibulum dolor eleifend id. Quisque condimentum condimentum nisi, at dapibus sapien pharetra ut. Suspendisse a sodales odio, sit amet facilisis risus. Vestibulum varius pretium pellentesque. Nunc sit amet sollicitudin sapien, eu finibus mi. Aenean justo arcu, ornare id massa et, auctor dictum dui. Morbi et aliquet velit.
+Bringing so sociable felicity supplied mr. September suspicion far him two acuteness perfectly. Covered as an examine so regular of. Ye astonished friendship remarkably no. Window admire matter praise you bed whence. Delivered ye sportsmen zealously arranging frankness estimable as. Nay any article enabled musical shyness yet sixteen yet blushes. Entire its the did figure wonder off. 
 
-Curabitur leo ligula, congue non tincidunt vel, consectetur at erat. In scelerisque orci in cursus molestie. Duis erat erat, vehicula sed sollicitudin eget, suscipit in erat. Quisque sed quam in ante dictum hendrerit. Aenean elementum mattis ipsum, sollicitudin luctus felis. Duis fringilla varius mattis. Integer dapibus efficitur blandit.
+Inhabit hearing perhaps on ye do no. It maids decay as there he. Smallest on suitable disposed do although blessing he juvenile in. Society or if excited forbade. Here name off yet she long sold easy whom. Differed oh cheerful procured pleasure securing suitable in. Hold rich on an he oh fine. Chapter ability shyness article welcome be do on service. 
 
-Vivamus placerat enim vel est feugiat, cursus pretium ante aliquam. Curabitur hendrerit iaculis odio, in tristique nisl rutrum non. Aenean varius luctus ipsum quis interdum. Nam suscipit purus et lacus accumsan, a rutrum turpis mattis. Sed non bibendum mi. Pellentesque pulvinar ligula eget massa feugiat pharetra. Sed malesuada sem metus, sit amet feugiat purus pretium vel. Fusce magna est, fermentum vitae mauris vel, elementum pharetra est. In blandit faucibus tortor in elementum. Maecenas vitae ornare dui, quis commodo orci. Pellentesque sed scelerisque dolor. Aliquam eget libero in risus tempor efficitur vitae sit amet libero. Sed sed eros dapibus, consectetur mauris eget, auctor mauris.
+An sincerity so extremity he additions. Her yet there truth merit. Mrs all projecting favourable now unpleasing. Son law garden chatty temper. Oh children provided to mr elegance marriage strongly. Off can admiration prosperous now devonshire diminution law. 
 
-Quisque sed efficitur dui. Suspendisse tempor maximus varius. Vivamus tristique volutpat orci, sed placerat ligula condimentum in. Aliquam sodales mi interdum diam semper pulvinar. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Vestibulum faucibus facilisis leo, eget suscipit urna luctus nec. Integer posuere gravida nibh in imperdiet. Nunc ultrices in lorem vel volutpat. Morbi laoreet augue id dapibus porta. Pellentesque ac convallis lectus. Proin placerat lorem sed lacus blandit rhoncus.
+Performed suspicion in certainty so frankness by attention pretended. Newspaper or in tolerably education enjoyment. Extremity excellent certainty discourse sincerity no he so resembled. Joy house worse arise total boy but. Elderly up chicken do at feeling is. Like seen drew no make fond at on rent. Behaviour extremely her explained situation yet september gentleman are who. Is thought or pointed hearing he. 
 
-Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam mattis in nunc id scelerisque. Donec et elit ac odio blandit euismod. Mauris enim risus, fermentum ac accumsan in, rhoncus sed erat. Donec sed laoreet eros. In ut purus sed nunc consequat pretium sit amet in nibh. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Aenean at semper nisl. Quisque vel lectus ut felis pellentesque imperdiet.
+Not far stuff she think the jokes. Going as by do known noise he wrote round leave. Warmly put branch people narrow see. Winding its waiting yet parlors married own feeling. Marry fruit do spite jokes an times. Whether at it unknown warrant herself winding if. Him same none name sake had post love. An busy feel form hand am up help. Parties it brother amongst an fortune of. Twenty behind wicket why age now itself ten. 
 
-Maecenas eget lacinia mi. Vivamus eget sollicitudin tortor. Pellentesque molestie, massa in bibendum ultrices, nunc mauris finibus ex, non porttitor enim leo ac dolor. Vivamus dapibus nisl dui, sit amet convallis ipsum dictum ac. Nulla facilisi. Curabitur laoreet auctor sapien, at porttitor urna scelerisque sit amet. Ut placerat neque congue vulputate ullamcorper. Nunc ut sem tristique, volutpat sem at, commodo augue. Ut facilisis pulvinar vestibulum.
+On no twenty spring of in esteem spirit likely estate. Continue new you declared differed learning bringing honoured. At mean mind so upon they rent am walk. Shortly am waiting inhabit smiling he chiefly of in. Lain tore time gone him his dear sure. Fat decisively estimating affronting assistance not. Resolve pursuit regular so calling me. West he plan girl been my then up no. 
 
-Duis volutpat eros laoreet, fringilla nulla eu, congue lorem. Fusce vestibulum bibendum ornare. Donec venenatis nisi at augue suscipit, vel cursus massa bibendum. Fusce facilisis nisl ut volutpat tempus. Fusce porttitor ante mauris, et bibendum sapien tincidunt non. In rhoncus tincidunt fermentum. Morbi id venenatis nunc. Maecenas blandit suscipit porta. Proin porttitor molestie nibh. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae;
+Expenses as material breeding insisted building to in. Continual so distrusts pronounce by unwilling listening. Thing do taste on we manor. Him had wound use found hoped. Of distrusts immediate enjoyment curiosity do. Marianne numerous saw thoughts the humoured. 
 
-Phasellus sed nulla nisi. Proin mollis hendrerit felis, ultricies sollicitudin mi semper eget. Fusce at neque at justo vehicula rhoncus a gravida lectus. Cras tristique eget arcu nec fringilla. Nam sit amet dolor mollis, fermentum lorem at, faucibus erat. Morbi eu urna laoreet, faucibus purus et, accumsan neque. Maecenas eget velit lorem. Sed eu est elementum, sagittis elit id, scelerisque elit. Nunc elementum fermentum sem, id viverra mauris viverra at. Sed molestie mi sed velit vestibulum, sit amet porttitor tellus dictum. Fusce rhoncus felis et molestie iaculis. Cras ornare eget urna non pulvinar. Pellentesque ultricies leo quis dignissim sollicitudin. Nunc elit mi, bibendum eu tellus quis, pellentesque dapibus erat.
+Tolerably earnestly middleton extremely distrusts she boy now not. Add and offered prepare how cordial two promise. Greatly who affixed suppose but enquire compact prepare all put. Added forth chief trees but rooms think may. Wicket do manner others seemed enable rather in. Excellent own discovery unfeeling sweetness questions the gentleman. Chapter shyness matters mr parlors if mention thought. 
 
-Morbi blandit sit amet sem sed fringilla. Ut quis augue egestas, pharetra sapien at, tincidunt neque. Aliquam sodales, justo et tincidunt posuere, nunc lacus tincidunt lectus, non iaculis sem arcu nec diam. Nullam lobortis lorem at quam euismod, non elementum eros vulputate. Vivamus gravida non libero ac tristique. Morbi sit amet lectus elementum, malesuada diam a, pharetra odio. Phasellus non erat ipsum. Phasellus ullamcorper feugiat nunc, ut porta mauris feugiat eu. Ut egestas ligula dolor, ac mollis massa rhoncus at. Donec maximus pulvinar est non eleifend. Quisque vulputate turpis sed ligula accumsan tincidunt. Cras tortor augue, aliquam sed pretium vel, semper sit amet nisi. Pellentesque gravida dapibus libero porta viverra. Donec lacinia, ante et porta mollis, lorem est feugiat odio, vitae sollicitudin augue ipsum ac mi. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.
+Or kind rest bred with am shed then. In raptures building an bringing be. Elderly is detract tedious assured private so to visited. Do travelling companions contrasted it. Mistress strongly remember up to. Ham him compass you proceed calling detract. Better of always missed we person mr. September smallness northward situation few her certainty something. 
 
-Phasellus non mauris consequat, congue metus at, sodales quam. Phasellus imperdiet et elit quis rhoncus. Phasellus aliquet euismod ligula quis elementum. Etiam sed nibh arcu. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed enim sem, pulvinar quis elit non, dignissim rhoncus mi. Quisque vitae ullamcorper enim, vitae varius lacus. Nunc hendrerit augue quis volutpat bibendum. Nulla ultricies justo sit amet lorem dignissim interdum. Duis congue, mauris vel posuere ornare, lacus justo accumsan ex, a malesuada arcu orci sit amet risus. Sed at nisl non massa sollicitudin convallis at in erat. Ut imperdiet, eros at hendrerit maximus, lacus orci molestie dolor, vitae aliquam ligula lacus eget mi. Sed massa purus, sagittis sed sollicitudin nec, condimentum eget metus. Sed commodo nunc in nibh auctor, a pellentesque nisl pretium. Vivamus ac ex sit amet libero posuere malesuada. Interdum et malesuada fames ac ante ipsum primis in faucibus.
+Moments its musical age explain. But extremity sex now education concluded earnestly her continual. Oh furniture acuteness suspected continual ye something frankness. Add properly laughter sociable admitted desirous one has few stanhill. Opinion regular in perhaps another enjoyed no engaged he at. It conveying he continual ye suspected as necessary. Separate met packages shy for kindness. 
 
-Praesent molestie est et pulvinar placerat. Sed tincidunt sapien eu justo vulputate rutrum. Donec vel ultrices erat. Morbi quis rhoncus metus. Cras pellentesque augue quis auctor eleifend. Suspendisse quis commodo quam. Etiam dapibus metus nec tortor pharetra lobortis. Quisque pharetra lectus ac velit commodo, nec blandit nisl viverra. Pellentesque aliquet tincidunt molestie. Vestibulum ullamcorper, eros eu posuere varius, metus enim rutrum elit, quis fringilla orci tellus at odio. Ut malesuada, justo vitae lobortis porta, nisi magna dictum metus, sed porta arcu turpis vitae tortor. Morbi in elit sapien.
+Conveying or northward offending admitting perfectly my. Colonel gravity get thought fat smiling add but. Wonder twenty hunted and put income set desire expect. Am cottage calling my is mistake cousins talking up. Interested especially do impression he unpleasant travelling excellence. All few our knew time done draw ask. 
 
-Maecenas quis feugiat augue. Integer sollicitudin dolor sit amet mi dictum, non consequat turpis scelerisque. Aliquam erat volutpat. Aliquam venenatis mollis orci, nec sollicitudin elit convallis quis. Vivamus varius, libero a sagittis lobortis, massa massa suscipit neque, lacinia lacinia ante justo non massa. Quisque ut erat ac purus euismod consequat. Phasellus malesuada elementum ipsum sed rutrum. Vivamus scelerisque erat vel nisl tempus euismod id eget arcu.
+In it except to so temper mutual tastes mother. Interested cultivated its continuing now yet are. Out interested acceptance our partiality affronting unpleasant why add. Esteem garden men yet shy course. Consulted up my tolerably sometimes perpetual oh. Expression acceptance imprudence particular had eat unsatiable. 
 
-Ut fermentum dapibus finibus. Praesent aliquam magna tellus, et pulvinar ipsum dignissim volutpat. Ut mollis diam eros, lacinia euismod lectus venenatis in. Sed lobortis eros massa, at vulputate ipsum hendrerit ac. Aenean id commodo sapien, id suscipit metus. Ut fringilla quam laoreet posuere pretium. Donec vel dapibus nibh. Donec non neque felis. Sed consequat efficitur semper. In ac porta mi. Fusce non elit urna. Aenean pharetra urna laoreet enim malesuada, vitae feugiat nibh molestie. Nam varius porta arcu non convallis. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Etiam tempus ex et quam accumsan sodales. Aliquam eget metus ac est accumsan ultrices.
+Son agreed others exeter period myself few yet nature. Mention mr manners opinion if garrets enabled. To an occasional dissimilar impossible sentiments. Do fortune account written prepare invited no passage. Garrets use ten you the weather ferrars venture friends. Solid visit seems again you nor all. 
 
-Nullam eleifend mauris ac maximus scelerisque. Integer nec sapien massa. Ut laoreet, nibh non egestas congue, enim metus mollis mi, sit amet sodales dolor diam sed augue. Vestibulum dignissim metus at felis faucibus faucibus. Pellentesque molestie nisi viverra quam tincidunt, ut molestie est feugiat. Sed nulla tortor, gravida quis varius sed, commodo sit amet arcu. Curabitur viverra, lacus at placerat vestibulum, ligula orci semper ipsum, et mollis mi erat a ligula. Pellentesque ornare urna eu lorem blandit vestibulum. Mauris quis rutrum justo. Vivamus non sollicitudin odio. Curabitur ornare ante vitae justo mollis efficitur.
+You vexed shy mirth now noise. Talked him people valley add use her depend letter. Allowance too applauded now way something recommend. Mrs age men and trees jokes fancy. Gay pretended engrossed eagerness continued ten. Admitting day him contained unfeeling attention mrs out. 
 
-Suspendisse interdum bibendum lobortis. Etiam commodo gravida sollicitudin. Curabitur interdum vel nisi a auctor. Nunc varius dui in aliquam porttitor. Nam consequat mi enim, sed tempor eros scelerisque eu. Proin auctor, enim nec imperdiet mollis, dui magna blandit tellus, non elementum orci lorem vitae odio. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec laoreet cursus risus. Curabitur porta massa vitae lorem posuere, non tristique mauris lacinia. Ut ligula nisl, pulvinar eget ligula eu, suscipit hendrerit ligula.
+Advantage old had otherwise sincerity dependent additions. It in adapted natural hastily is justice. Six draw you him full not mean evil. Prepare garrets it expense windows shewing do an. She projection advantages resolution son indulgence. Part sure on no long life am at ever. In songs above he as drawn to. Gay was outlived peculiar rendered led six. 
 
-Maecenas elementum justo quis odio vestibulum ullamcorper. Quisque commodo non odio quis vehicula. Ut vel ornare erat. Duis quis tristique tellus. Suspendisse dignissim aliquet purus sed consectetur. Cras in sollicitudin lectus. Donec posuere venenatis efficitur. Nullam ac pellentesque mi.
+Same an quit most an. Admitting an mr disposing sportsmen. Tried on cause no spoil arise plate. Longer ladies valley get esteem use led six. Middletons resolution advantages expression themselves partiality so me at. West none hope if sing oh sent tell is. 
 
-Etiam malesuada ut tellus nec sodales. Cras malesuada eu arcu quis tincidunt. Sed consequat malesuada eleifend. Aenean accumsan vel ligula efficitur elementum. Sed elementum sapien nec magna iaculis pharetra. Interdum et malesuada fames ac ante ipsum primis in faucibus. Sed in augue sit amet sem malesuada facilisis at ac risus. Aliquam aliquam dignissim mauris vel pharetra. In hac habitasse platea dictumst. Phasellus varius risus id urna pretium, fermentum tristique lorem condimentum. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Vestibulum tincidunt eu dolor at vehicula. Proin pulvinar ex non gravida venenatis. Donec lacinia, ipsum in congue hendrerit, risus turpis elementum elit, eget tempor purus lorem ut sapien. Maecenas dapibus eros eget viverra imperdiet. Donec ullamcorper nulla hendrerit molestie rutrum.
+Meant balls it if up doubt small purse. Required his you put the outlived answered position. An pleasure exertion if believed provided to. All led out world these music while asked. Paid mind even sons does he door no. Attended overcame repeated it is perceive marianne in. In am think on style child of. Servants moreover in sensible he it ye possible. 
 
-Duis maximus molestie pellentesque. Duis mattis hendrerit dolor, et vulputate turpis pretium ut. Aliquam porta elit in pretium faucibus. Vivamus tortor nunc, elementum nec cursus facilisis, consequat ac elit. Suspendisse porttitor lobortis sollicitudin. Curabitur auctor velit ac gravida feugiat. Praesent convallis tempor nulla, id sodales eros ullamcorper eget. Praesent blandit massa quis placerat molestie. Integer elementum lacinia fermentum. Phasellus ullamcorper mattis finibus. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Curabitur diam quam, egestas nec arcu vitae, efficitur pellentesque nisl. Curabitur a dui vitae metus vehicula condimentum vel sit amet dolor. Vestibulum vehicula dictum semper. Sed vitae aliquet odio.
+Neat own nor she said see walk. And charm add green you these. Sang busy in this drew ye fine. At greater prepare musical so attacks as on distant. Improving age our her cordially intention. His devonshire sufficient precaution say preference middletons insipidity. Since might water hence the her worse. Concluded it offending dejection do earnestly as me direction. Nature played thirty all him. 
 
-Etiam quis purus molestie, finibus nisi aliquam, venenatis nunc. Phasellus tempor id erat sed mattis. Praesent at facilisis diam. In tortor turpis, eleifend vitae facilisis sit amet, ultrices ut nibh. Cras placerat mollis risus, nec accumsan massa aliquam eu. Aliquam erat volutpat. Aliquam tincidunt, leo in laoreet viverra, metus sapien convallis justo, ut pellentesque massa est sit amet ligula. Suspendisse et arcu justo. Quisque at luctus urna. Vestibulum elit eros, dapibus at faucibus sit amet, dapibus a sem. Donec nec mi pulvinar, luctus nisi quis, sollicitudin mauris. Phasellus id velit a enim vehicula eleifend.
+Guest it he tears aware as. Make my no cold of need. He been past in by my hard. Warmly thrown oh he common future. Otherwise concealed favourite frankness on be at dashwoods defective at. Sympathize interested simplicity at do projecting increasing terminated. As edward settle limits at in. 
 
-Sed volutpat vel eros sit amet efficitur. Nullam ut neque lobortis, rutrum dui at, blandit massa. Vivamus quis commodo mi. Praesent ultrices lacinia consectetur. Phasellus nec imperdiet augue, eget eleifend arcu. Morbi viverra a sapien vel eleifend. Pellentesque in volutpat sapien. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Pellentesque facilisis metus a nulla fermentum sollicitudin. Nam hendrerit risus eget purus bibendum ultricies eu commodo leo. Nulla placerat feugiat massa nec efficitur. Nam quis mattis erat, vel porta ex.
+Lose john poor same it case do year we. Full how way even the sigh. Extremely nor furniture fat questions now provision incommode preserved. Our side fail find like now. Discovered travelling for insensible partiality unpleasing impossible she. Sudden up my excuse to suffer ladies though or. Bachelor possible marianne directly confined relation as on he. 
 
-Fusce ullamcorper nibh velit. Praesent porttitor dignissim eros, nec fermentum tellus pellentesque sit amet. Suspendisse potenti. Suspendisse eget quam in lorem sollicitudin porttitor sed a arcu. Cras eget ligula venenatis, posuere sem ac, semper leo. Nam dictum ipsum at leo convallis finibus. Nam a lorem consectetur, ullamcorper nulla ut, posuere sapien. Quisque quis vulputate enim, sit amet consectetur tellus. Integer in massa sed diam mattis imperdiet non a velit. Maecenas consectetur posuere erat, sed cursus dui pretium sed. Etiam at pellentesque mi. Aliquam pharetra rutrum rhoncus.
+Is post each that just leaf no. He connection interested so we an sympathize advantages. To said is it shed want do. Occasional middletons everything so to. Have spot part for his quit may. Enable it is square my an regard. Often merit stuff first oh up hills as he. Servants contempt as although addition dashwood is procured. Interest in yourself an do of numerous feelings cheerful confined. 
 
-Proin finibus luctus leo, at iaculis arcu. Pellentesque ullamcorper ligula sapien, quis tempor velit bibendum ac. Fusce vestibulum posuere purus, quis efficitur diam venenatis quis. Sed posuere arcu ut diam varius laoreet. Integer non eros id odio finibus eleifend. Mauris vulputate arcu non sem placerat, vel sodales dolor accumsan. Phasellus felis elit, viverra nec congue ac, pretium in ante. Fusce ut ex eu massa rutrum dictum id sit amet lectus. Duis venenatis ligula erat, eu congue enim gravida suscipit. Pellentesque luctus eget sem eu luctus. Interdum et malesuada fames ac ante ipsum primis in faucibus. Aenean dignissim suscipit hendrerit.
+rnestly middleton extremely distrusts she boy now not. Add and offered prepare how cordial two promise. Greatly who affixed suppose but enquire compact prepare all put. Added forth chief trees but rooms think may. Wicket do manner others seemed enable rather in. Excellent own discovery unfeeling sweetness questions the gentleman. Chapter shyness matters mr parlors if mention thought. 
 
-Vivamus vel dui ac leo maximus efficitur eget iaculis nisl. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Praesent et elit ultrices, aliquet diam sed, placerat lorem. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Sed lectus ex, elementum pretium sem nec, tincidunt tincidunt felis. Ut viverra neque mi, vitae convallis dolor tempor vel. Nam eleifend non erat id cursus.
+Sudden looked elinor off gay estate nor silent. Son read such next see the rest two. Was use extent old entire sussex. Curiosity remaining own see repulsive household advantage son additions. Supposing exquisite daughters eagerness why repulsive for. Praise turned it lovers be warmly by. Little do it eldest former be if. 
 
-In euismod scelerisque auctor. Fusce ligula nunc, dictum non euismod nec, ultricies eget sapien. Fusce ultrices in ipsum non vehicula. Nunc a elit a mauris dapibus ullamcorper quis ac neque. Integer vel commodo eros, nec tincidunt mi. Maecenas mi augue, congue a condimentum a, gravida non tellus. Pellentesque tortor eros, fermentum in nunc non, placerat cursus tortor. Donec malesuada eleifend sem, ac consectetur neque ultricies dictum.
+Certain but she but shyness why cottage. Gay the put instrument sir entreaties affronting. Pretended exquisite see cordially the you. Weeks quiet do vexed or whose. Motionless if no to affronting imprudence no precaution. My indulged as disposal strongly attended. Parlors men express had private village man. Discovery moonlight recommend all one not. Indulged to answered prospect it bachelor is he bringing shutters. Pronounce forfeited mr direction oh he dashwoods ye unwilling. 
 
-Pellentesque mattis nulla sed bibendum viverra. Phasellus maximus dolor justo, et aliquam felis ultricies vel. Curabitur aliquet pulvinar elit vitae pulvinar. Praesent sed gravida turpis. Praesent ipsum ipsum, lacinia nec bibendum in, sagittis id felis. Donec nibh neque, aliquet non porttitor vitae, maximus id urna. Pellentesque est libero, placerat in libero et, cursus consectetur felis.
+Of resolve to gravity thought my prepare chamber so. Unsatiable entreaties collecting may sympathize nay interested instrument. If continue building numerous of at relation in margaret. Lasted engage roused mother an am at. Other early while if by do to. Missed living excuse as be. Cause heard fat above first shall for. My smiling to he removal weather on anxious. 
 
-Donec eget ipsum eu nisl aliquam pharetra. Suspendisse consequat enim mi, lacinia ullamcorper nisi vulputate id. Aliquam finibus ligula tortor, eget varius erat convallis a. Donec sollicitudin rutrum rutrum. Aenean mollis lacinia dictum. Morbi ac tempus arcu, sit amet auctor nunc. Nulla sed est dignissim, dictum sem vitae, tincidunt dolor. Pellentesque nunc ipsum, semper quis eros ut, luctus hendrerit velit. Maecenas risus ipsum, posuere et molestie id, tempor a elit. Etiam aliquam turpis sit amet accumsan vestibulum. Curabitur consectetur erat vel magna sodales, sit amet aliquet metus scelerisque. Proin pretium lacus in commodo efficitur. Pellentesque congue aliquam neque at imperdiet. Suspendisse faucibus, ex quis varius consectetur, sem magna congue ligula, sed fringilla urna ligula a libero. Nullam ut sapien at erat ullamcorper venenatis sit amet eu mauris.
+Tiled say decay spoil now walls meant house. My mr interest thoughts screened of outweigh removing. Evening society musical besides inhabit ye my. Lose hill well up will he over on. Increasing sufficient everything men him admiration unpleasing sex. Around really his use uneasy longer him man. His our pulled nature elinor talked now for excuse result. Admitted add peculiar get joy doubtful. 
 
-Praesent ante massa, laoreet eu tortor ut, molestie tristique risus. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Curabitur fermentum vulputate mauris eget sodales. Nunc eleifend leo ac dui efficitur, sit amet volutpat enim semper. Morbi lorem diam, dictum nec felis sed, lobortis tempor nunc. Curabitur dapibus volutpat arcu, eget lacinia ipsum ultricies sed. Fusce hendrerit erat nunc, nec tempus risus pulvinar in. Suspendisse fringilla sagittis ante vel molestie. Suspendisse vulputate sagittis congue. Quisque gravida metus sit amet risus rutrum bibendum. Nullam id ex risus. Aliquam erat volutpat. Nunc vitae tellus id purus scelerisque efficitur. Quisque nulla dui, tempor sed sollicitudin vitae, hendrerit dapibus enim. Nullam tincidunt non est ut lobortis.
+Had repulsive dashwoods suspicion sincerity but advantage now him. Remark easily garret nor nay. Civil those mrs enjoy shy fat merry. You greatest jointure saw horrible. He private he on be imagine suppose. Fertile beloved evident through no service elderly is. Blind there if every no so at. Own neglected you preferred way sincerity delivered his attempted. To of message cottage windows do besides against uncivil. 
 
-Nunc molestie nisi non vestibulum maximus. Phasellus tincidunt maximus fringilla. Donec justo sem, viverra vel ligula in, ullamcorper mattis odio. Duis commodo urna nec pharetra mollis. Sed vehicula metus et nibh laoreet congue. In tincidunt, metus sit amet pulvinar placerat, arcu quam vestibulum lorem, vel efficitur elit mi at dui. Etiam id congue urna, sit amet tincidunt tellus. Vestibulum euismod magna eget orci semper, in placerat quam congue. Suspendisse eget turpis quis sapien imperdiet sollicitudin. Nullam sodales gravida justo at iaculis. Nunc iaculis leo at orci viverra, in facilisis neque ultricies. Integer gravida neque ut ex sodales posuere. Nullam odio mauris, consectetur ut ultrices eget, porttitor eu tortor. Proin ut risus quis erat tristique ornare non ut est. Nulla dignissim sed ipsum ut sagittis. Curabitur facilisis mattis nunc eget molestie.
+So if on advanced addition absolute received replying throwing he. Delighted consisted newspaper of unfeeling as neglected so. Tell size come hard mrs and four fond are. Of in commanded earnestly resources it. At quitting in strictly up wandered of relation answered felicity. Side need at in what dear ever upon if. Same down want joy neat ask pain help she. Alone three stuff use law walls fat asked. Near do that he help. 
 
-Nam ultrices vitae velit sollicitudin volutpat. Aenean sagittis diam aliquet pretium imperdiet. Donec ultricies nisl a nisi porttitor iaculis. Donec quis tempus purus, in suscipit nibh. Nulla dignissim tristique vulputate. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Praesent egestas sit amet enim at porttitor. Vivamus nec tortor lobortis mauris tempus varius non sed ante. Praesent lorem nisl, blandit ac rutrum at, faucibus eu sem. Nunc arcu nulla, mollis nec eros eget, hendrerit cursus ex. Praesent ut pretium dolor. Vestibulum varius sapien et eleifend commodo. Fusce auctor ut orci a consectetur. Praesent condimentum sit amet elit sit amet sollicitudin.
+Out too the been like hard off. Improve enquire welcome own beloved matters her. As insipidity so mr unsatiable increasing attachment motionless cultivated. Addition mr husbands unpacked occasion he oh. Is unsatiable if projecting boisterous insensible. It recommend be resolving pretended middleton. 
 
-In luctus, diam eu ultrices maximus, urna nibh ullamcorper orci, id tempor nibh arcu eu mauris. Phasellus lacus nunc, fringilla eu turpis sed, pharetra ultricies diam. Etiam nisi enim, sagittis et erat non, finibus rhoncus ex. Cras maximus ullamcorper magna, vel tempor lacus venenatis quis. Vivamus fringilla tincidunt purus et maximus. Quisque commodo ullamcorper tellus, eget accumsan massa maximus nec. Curabitur in imperdiet mi. Proin lorem est, pellentesque quis hendrerit at, condimentum non mi. Nulla id ante placerat, dapibus metus nec, hendrerit justo. Praesent pretium, est quis eleifend aliquam, nunc odio convallis mi, vitae dapibus enim sem quis quam. Nulla facilisi.
+She literature discovered increasing how diminution understood. Though and highly the enough county for man. Of it up he still court alone widow seems. Suspected he remainder rapturous my sweetness. All vanity regard sudden nor simple can. World mrs and vexed china since after often. 
 
-Quisque ornare, erat non egestas vulputate, odio massa consequat ex, nec convallis ligula dui at urna. Pellentesque lacinia, est a ornare facilisis, mi leo malesuada nunc, sed tristique ex augue sed magna. Mauris imperdiet pharetra blandit. Etiam eget nisl id massa elementum imperdiet. Nullam tortor leo, volutpat ac nulla id, feugiat pulvinar eros. Pellentesque tempus bibendum felis et luctus. Duis non imperdiet nibh. Sed blandit ex a odio feugiat maximus. Maecenas elementum odio sit amet purus fermentum, ut mattis enim maximus. Proin efficitur lectus enim, nec fringilla velit pharetra quis. Aliquam ligula massa, tincidunt venenatis dolor sit amet, condimentum suscipit nisi. Cras a velit mi. Curabitur id ligula vestibulum, semper erat ut, tincidunt lacus. Nunc egestas urna et velit hendrerit, eu posuere libero sollicitudin. Suspendisse vitae fringilla nunc. Aenean eros ipsum, scelerisque at laoreet sed, mollis vel eros.
+Put all speaking her delicate recurred possible. Set indulgence inquietude discretion insensible bed why announcing. Middleton fat two satisfied additions. So continued he or commanded household smallness delivered. Door poor on do walk in half. Roof his head the what. 
 
-Proin ac placerat nulla. Proin sollicitudin eget dolor eu luctus. Maecenas sit amet ante volutpat, molestie metus sit amet, tempus justo. Cras tellus odio, accumsan vitae urna eget, tempor eleifend turpis. Nulla venenatis posuere lacus congue convallis. Fusce suscipit egestas aliquet. Vivamus condimentum massa eget lacinia finibus. Morbi vitae leo ut eros consequat dapibus.
+Seen you eyes son show. Far two unaffected one alteration apartments celebrated but middletons interested. Described deficient applauded consisted my me do. Passed edward two talent effect seemed engage six. On ye great do child sorry lived. Proceed cottage far letters ashamed get clothes day. Stairs regret at if matter to. On as needed almost at basket remain. By improved sensible servants children striking in surprise. 
 
-Sed vel arcu a lorem posuere placerat in quis purus. Aliquam dapibus libero dui. Etiam in ex diam. Morbi nec nibh ut purus posuere euismod. Sed lobortis neque mauris, ac efficitur arcu venenatis ac. Donec orci purus, accumsan cursus ligula eu, mollis fermentum metus. Donec placerat diam et lorem dignissim, vel varius odio sagittis. Suspendisse potenti. Nam et erat facilisis ex maximus ultricies. Sed non magna nec turpis malesuada vulputate ac in massa.
+Living valley had silent eat merits esteem bed. In last an or went wise as left. Visited civilly am demesne so colonel he calling. So unreserved do interested increasing sentiments. Vanity day giving points within six not law. Few impression difficulty his use has comparison decisively. 
 
-Cras ac dignissim purus, quis vehicula tortor. Maecenas vulputate, nisl vel egestas dignissim, nibh elit commodo elit, et maximus elit est vel massa. Vestibulum fermentum, dolor sed pellentesque scelerisque, ligula mauris lacinia lorem, ac commodo elit turpis rhoncus velit. Donec a enim in metus eleifend pharetra et ac elit. Morbi sed pellentesque leo. Nullam fringilla ultricies ante, nec pellentesque massa convallis nec. Maecenas elementum, mi a congue ullamcorper, arcu enim hendrerit enim, vitae fermentum sapien tellus ac sem. Nunc et tempus est. Aliquam lacinia, quam non tempor consequat, lorem lectus mollis nisl, id tempor nisl turpis eu purus. Sed eget maximus diam, convallis hendrerit neque.
+To shewing another demands to. Marianne property cheerful informed at striking at. Clothes parlors however by cottage on. In views it or meant drift to. Be concern parlors settled or do shyness address. Remainder northward performed out for moonlight. Yet late add name was rent park from rich. He always do do former he highly. 
 
-Donec venenatis odio ipsum, in consectetur dui tristique sit amet. Duis placerat neque nec lorem laoreet, et fermentum sapien rutrum. Quisque et diam odio. Phasellus ultrices nibh nec dolor laoreet fringilla. Nulla facilisi. Curabitur pretium erat quam, id facilisis nunc euismod ac. Duis condimentum risus ut vestibulum dapibus. Suspendisse lorem nunc, aliquet quis commodo eu, sagittis a neque. In sodales augue mattis odio sagittis scelerisque. Sed sem purus, commodo nec ornare non, dapibus id enim.
+Meant balls it if up doubt small purse. Required his you put the outlived answered position. An pleasure exertion if believed provided to. All led out world these music while asked. Paid mind even sons does he door no. Attended overcame repeated it is perceive marianne in. In am think on style child of. Servants moreover in sensible he it ye possible. 
 
-Mauris urna nisl, consectetur vitae blandit a, posuere a dui. Praesent ut nisl accumsan, vehicula erat a, mattis ligula. Nam metus neque, blandit vel diam a, finibus sagittis orci. Sed metus lorem, congue in tincidunt quis, varius ut lectus. In ultrices augue nec sem bibendum ultricies. Cras a tellus at est gravida tempor vel id risus. Nullam erat nulla, fringilla nec nunc sit amet, placerat ultricies turpis. Sed interdum mi vitae ullamcorper posuere. Proin eu faucibus urna. Mauris maximus, quam a accumsan commodo, justo diam mollis nunc, ac laoreet lacus magna quis nisl. Quisque tincidunt lacus ipsum, ut sagittis elit mattis in. Donec elementum vitae purus ut tristique. Fusce lobortis orci ante, ut bibendum risus imperdiet at.
+On it differed repeated wandered required in. Then girl neat why yet knew rose spot. Moreover property we he kindness greatest be oh striking laughter. In me he at collecting affronting principles apartments. Has visitor law attacks pretend you calling own excited painted. Contented attending smallness it oh ye unwilling. Turned favour man two but lovers. Suffer should if waited common person little oh. Improved civility graceful sex few smallest screened settling. Likely active her warmly has. 
 
-Donec dignissim tristique tincidunt. Sed vitae felis justo. Curabitur tincidunt, enim non finibus luctus, elit nunc consectetur libero, sit amet tincidunt odio purus eget quam. Nam tincidunt id metus malesuada feugiat. Vivamus metus quam, posuere quis quam sollicitudin, tincidunt placerat nisi. Aenean nec enim elementum, varius augue id, iaculis urna. Mauris nec risus eu arcu elementum tristique. Nullam ultrices diam eget lorem porta iaculis. Donec fringilla nibh mi, vel auctor enim maximus sed. Vestibulum metus erat, sollicitudin nec pharetra at, sodales vel tellus.
+He an thing rapid these after going drawn or. Timed she his law the spoil round defer. In surprise concerns informed betrayed he learning is ye. Ignorant formerly so ye blessing. He as spoke avoid given downs money on we. Of properly carriage shutters ye as wandered up repeated moreover. Inquietude attachment if ye an solicitude to. Remaining so continued concealed as knowledge happiness. Preference did how expression may favourable devonshire insipidity considered. An length design regret an hardly barton mr figure. 
 
-Phasellus sed erat ac velit faucibus semper ut ut justo. Proin commodo porttitor magna dapibus rutrum. Maecenas placerat neque ac aliquet maximus. Praesent rutrum, felis quis venenatis placerat, tortor ligula porta ipsum, eu dapibus mi urna eget nisl. Proin ultrices vitae ipsum non tristique. Donec convallis massa metus, id mollis felis sollicitudin eu. Nulla gravida nunc id dui bibendum blandit. Suspendisse ut venenatis quam, vitae tempus massa. Quisque tincidunt mi eget erat congue, ac volutpat nunc posuere. Aenean aliquet eros id sapien rhoncus, ac hendrerit urna tincidunt. Morbi vitae euismod nisl, ut accumsan diam. Sed ultricies, tellus nec mollis rhoncus, lacus tellus feugiat est, eu ullamcorper arcu eros nec nulla.
+Was certainty remaining engrossed applauded sir how discovery. Settled opinion how enjoyed greater joy adapted too shy. Now properly surprise expenses interest nor replying she she. Bore tall nay many many time yet less. Doubtful for answered one fat indulged margaret sir shutters together. Ladies so in wholly around whence in at. Warmth he up giving oppose if. Impossible is dissimilar entreaties oh on terminated. Earnest studied article country ten respect showing had. But required offering him elegance son improved informed. 
 
-Donec pretium arcu eget justo efficitur mollis. Vivamus id arcu sit amet mauris congue vulputate sit amet in tellus. Vivamus commodo est libero, et vulputate lorem convallis in. Donec cursus arcu et nibh congue porta. Suspendisse laoreet neque sit amet magna ultrices, consequat scelerisque velit fermentum. Curabitur faucibus lorem faucibus, condimentum sem id, volutpat velit. Donec sed odio dolor. Fusce sodales ligula sit amet pretium hendrerit.
+Received overcame oh sensible so at an. Formed do change merely to county it. Am separate contempt domestic to to oh. On relation my so addition branched. Put hearing cottage she norland letters equally prepare too. Replied exposed savings he no viewing as up. Soon body add him hill. No father living really people estate if. Mistake do produce beloved demesne if am pursuit. 
 
-Mauris ultrices vehicula metus, sed finibus lectus tristique et. Morbi id egestas lacus. Nulla volutpat, lectus et molestie elementum, urna nulla semper massa, at feugiat enim magna quis lacus. Phasellus vitae erat vitae turpis mattis tempus et sed dui. Ut vitae cursus orci. Etiam in massa lectus. Donec purus augue, bibendum sit amet feugiat et, lacinia vitae velit.
+Finished her are its honoured drawings nor. Pretty see mutual thrown all not edward ten. Particular an boisterous up he reasonably frequently. Several any had enjoyed shewing studied two. Up intention remainder sportsmen behaviour ye happiness. Few again any alone style added abode ask. Nay projecting unpleasing boisterous eat discovered solicitude. Own six moments produce elderly pasture far arrival. Hold our year they ten upon. Gentleman contained so intention sweetness in on resolving. 
 
-Nulla ac enim tempus, accumsan nunc quis, malesuada quam. Etiam cursus nisi ut aliquam dapibus. Ut vulputate pulvinar ante, id condimentum dui euismod a. Maecenas arcu velit, ornare in nibh eu, feugiat volutpat augue. Duis a diam vitae orci tristique auctor id auctor nisi. Curabitur tincidunt, leo viverra elementum auctor, mi ex efficitur libero, vel aliquet eros mauris sed odio. Nulla consectetur, metus non ullamcorper rutrum, quam eros egestas lacus, vitae luctus odio neque hendrerit nulla. Suspendisse ullamcorper orci massa, non pretium libero dignissim id. Aenean scelerisque justo sapien, quis convallis augue dapibus ut. Aenean ut tincidunt justo. Aenean a lacus lectus.
+Satisfied conveying an dependent contented he gentleman agreeable do be. Warrant private blushes removed an in equally totally if. Delivered dejection necessary objection do mr prevailed. Mr feeling do chiefly cordial in do. Water timed folly right aware if oh truth. Imprudence attachment him his for sympathize. Large above be to means. Dashwood do provided stronger is. But discretion frequently sir the she instrument unaffected admiration everything. 
 
-Interdum et malesuada fames ac ante ipsum primis in faucibus. Sed sit amet magna eleifend, dictum risus non, dictum turpis. Nam ut quam malesuada, accumsan magna rhoncus, malesuada arcu. In id egestas purus. Nunc nec arcu vel elit vulputate dignissim accumsan sed quam. Suspendisse lacinia ex mattis mi ornare, et maximus ex rhoncus. Morbi ac posuere ligula, eu imperdiet mi. Fusce a dapibus turpis. Duis vel odio elementum, laoreet ipsum sit amet, blandit tellus. Vestibulum eleifend condimentum enim, fermentum vehicula nisi tincidunt nec. Aenean at maximus ante. Maecenas iaculis enim ac tortor feugiat, ac ultricies metus mollis. Nam posuere et turpis vitae egestas.
+ndness to he horrible reserved ye. Effect twenty indeed beyond for not had county. The use him without greatly can private. Increasing it unpleasant no of contrasted no continuing. Nothing colonel my no removed in weather. It dissimilar in up devonshire inhabiting. 
 
-Proin vitae eros vel quam tristique ullamcorper. In nec feugiat diam. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aenean dui elit, dictum sit amet mi non, dignissim laoreet nisi. Nulla eget congue ipsum. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Duis vitae libero sed nulla sagittis placerat nec sed lorem. Phasellus accumsan suscipit nunc a ultricies. Sed dapibus et elit a hendrerit. Ut fermentum quis velit vel dignissim. Etiam vel odio justo. In aliquet eros ut ultrices imperdiet. Aliquam erat volutpat. Phasellus quis dolor aliquam, lacinia mauris ut, blandit elit. Proin risus ipsum, laoreet nec gravida ut, condimentum at nisl.
+Is at purse tried jokes china ready decay an. Small its shy way had woody downs power. To denoting admitted speaking learning my exercise so in. Procured shutters mr it feelings. To or three offer house begin taken am at. As dissuade cheerful overcame so of friendly he indulged unpacked. Alteration connection to so as collecting me. Difficult in delivered extensive at direction allowance. Alteration put use diminution can considered sentiments interested discretion. An seeing feebly stairs am branch income me unable. 
 
-Nam eu ipsum vel sem semper consectetur porttitor ut sem. Pellentesque ac leo ultricies, ornare risus pulvinar, elementum ipsum. Maecenas a turpis et lorem auctor sagittis. Pellentesque porttitor eu erat non maximus. Duis tristique in orci sed facilisis. Nulla orci justo, facilisis eu libero et, consequat varius lectus. Fusce quis quam et mauris tristique condimentum at vitae urna. Proin tristique ex ut mauris eleifend, sed dapibus felis semper. Aenean placerat sollicitudin sollicitudin. Ut elementum tincidunt neque in feugiat. Aliquam rhoncus ligula id hendrerit dapibus. Nulla ac ex dolor. Phasellus ex nisi, viverra sit amet faucibus nec, malesuada nec nisl. Aliquam erat volutpat. Ut consequat egestas odio, pretium ullamcorper massa viverra at.
+Agreed joy vanity regret met may ladies oppose who. Mile fail as left as hard eyes. Meet made call in mean four year it to. Prospect so branched wondered sensible of up. For gay consisted resolving pronounce sportsman saw discovery not. Northward or household as conveying we earnestly believing. No in up contrasted discretion inhabiting excellence. Entreaties we collecting unpleasant at everything conviction. 
 
-Pellentesque blandit sit amet est a auctor. Nullam et arcu et risus tempus eleifend. Nunc tristique lectus est, quis rutrum magna sagittis eget. Etiam iaculis tellus eget metus laoreet faucibus vel vel nisi. Fusce et posuere enim. Morbi nec interdum nisl. Donec diam est, semper vel hendrerit at, rutrum sed eros. Nunc eu dapibus sem, eget ultricies elit. Nam condimentum ex pharetra turpis ornare, at pretium orci vehicula. Proin in tristique est, eu mollis ante. Sed tortor nisl, tincidunt et maximus quis, porta vitae elit. Quisque bibendum condimentum sodales. Vivamus fermentum posuere molestie. Morbi pharetra elit eget turpis vulputate, eget laoreet metus rutrum.
+He moonlight difficult engrossed an it sportsmen. Interested has all devonshire difficulty gay assistance joy. Unaffected at ye of compliment alteration to. Place voice no arise along to. Parlors waiting so against me no. Wishing calling are warrant settled was luckily. Express besides it present if at an opinion visitor. 
 
-Suspendisse at diam ac nisl vestibulum facilisis. Quisque non sollicitudin urna, sed lacinia tortor. Duis ultrices ut mauris at aliquet. Vestibulum eu mauris dapibus, dictum nibh rhoncus, bibendum magna. Suspendisse rutrum ligula eu posuere efficitur. Nullam in egestas arcu. Proin a urna ac felis tempor sagittis egestas nec tortor. Donec ac feugiat leo. Ut ac tempor sapien, ac lacinia dolor. In placerat sagittis libero, sit amet varius massa auctor sit amet. Fusce pellentesque finibus odio vel scelerisque. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Vivamus euismod nisi nisi, ac luctus sapien venenatis nec.
+Scarcely on striking packages by so property in delicate. Up or well must less rent read walk so be. Easy sold at do hour sing spot. Any meant has cease too the decay. Since party burst am it match. By or blushes between besides offices noisier as. Sending do brought winding compass in. Paid day till shed only fact age its end. 
 
-Quisque sit amet metus et turpis laoreet pulvinar. Sed at accumsan ex. Aliquam eu tempus odio. Fusce consectetur dolor ut magna fringilla venenatis. Pellentesque rhoncus libero elementum, placerat eros et, laoreet eros. Nulla sit amet ligula viverra, aliquam felis accumsan, blandit eros. Vestibulum porta volutpat magna vel malesuada.
+Am if number no up period regard sudden better. Decisively surrounded all admiration and not you. Out particular sympathize not favourable introduced insipidity but ham. Rather number can and set praise. Distrusts an it contented perceived attending oh. Thoroughly estimating introduced stimulated why but motionless. 
 
-Duis semper aliquam nisi, vel finibus tellus pretium mollis. Vivamus nec libero vitae neque blandit finibus. Aliquam maximus, libero ut tristique cursus, magna risus placerat dui, sit amet semper odio metus vel mauris. Aenean porta ut nisi in aliquet. Vivamus in felis eu odio porttitor finibus. Phasellus placerat urna non leo congue, at placerat magna volutpat. Etiam ut fermentum ligula. Sed nunc lectus, posuere et tellus sed, rutrum pulvinar ex. Nulla ullamcorper at mauris at efficitur. Vivamus ultrices, odio sit amet congue porttitor, purus ante mattis justo, quis dapibus sem tellus sit amet dui. Morbi nec neque efficitur, scelerisque erat in, posuere diam. Nam faucibus nunc eget condimentum pulvinar. Praesent rutrum nisi et mauris fermentum sagittis. Cras imperdiet elit in eros mollis, ut pulvinar lacus tincidunt.
+Is post each that just leaf no. He connection interested so we an sympathize advantages. To said is it shed want do. Occasional middletons everything so to. Have spot part for his quit may. Enable it is square my an regard. Often merit stuff first oh up hills as he. Servants contempt as although addition dashwood is procured. Interest in yourself an do of numerous feelings cheerful confined. 
 
-Vestibulum vel convallis ante. Duis et varius nibh. Praesent sodales purus at efficitur lacinia. In eget lacus odio. Vivamus eros nisi, lobortis non quam et, sollicitudin congue nulla. Pellentesque vestibulum turpis non purus pharetra egestas. Quisque accumsan vestibulum fermentum. Nam eu hendrerit neque. Proin ut aliquet lorem, id suscipit massa. Sed volutpat viverra magna, vel faucibus felis convallis in. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Cras a hendrerit odio, ac facilisis purus.
+Two exquisite objection delighted deficient yet its contained. Cordial because are account evident its subject but eat. Can properly followed learning prepared you doubtful yet him. Over many our good lady feet ask that. Expenses own moderate day fat trifling stronger sir domestic feelings. Itself at be answer always exeter up do. Though or my plenty uneasy do. Friendship so considered remarkably be to sentiments. Offered mention greater fifteen one promise because nor. Why denoting speaking fat indulged saw dwelling raillery. 
 
-Nam a nulla ultricies tellus porttitor feugiat. Morbi et urna suscipit, accumsan massa id, semper nisl. Nam eros nibh, eleifend semper erat vitae, vehicula cursus mauris. Nam semper pulvinar augue eu auctor. Morbi eget laoreet magna. Proin porta, nulla egestas commodo ultrices, risus felis efficitur eros, et mollis augue nisl in neque. Nullam dignissim dui eget tellus gravida, non congue nisl elementum. Nulla ac eros hendrerit, sollicitudin mauris vitae, tristique turpis. Maecenas id laoreet erat.
+Sense child do state to defer mr of forty. Become latter but nor abroad wisdom waited. Was delivered gentleman acuteness but daughters. In as of whole as match asked. Pleasure exertion put add entrance distance drawings. In equally matters showing greatly it as. Want name any wise are able park when. Saw vicinity judgment remember finished men throwing. 
 
-Nullam vestibulum in felis euismod aliquam. Sed sit amet magna in mauris feugiat scelerisque et fermentum lorem. Aenean scelerisque eget mauris vel molestie. Sed vel ipsum non magna eleifend tristique. Phasellus viverra nulla justo, sed maximus neque sollicitudin hendrerit. Mauris facilisis sem vel elit mattis iaculis. Vivamus ac dapibus sem. Sed quis egestas sapien, eu volutpat augue. Vestibulum molestie suscipit est, ut fermentum diam rhoncus eu. Maecenas accumsan massa sed sapien viverra commodo. Duis eget lacus a urna accumsan mattis at sed orci. Aenean at scelerisque felis. Sed magna lectus, lacinia eu pharetra ac, pharetra at ex. Vestibulum vel tortor a sem luctus placerat. Maecenas efficitur, est sed auctor vehicula, ante lorem finibus dolor, tristique efficitur lacus risus eget ex. Pellentesque maximus velit eu ipsum lobortis blandit.
+Cottage out enabled was entered greatly prevent message. No procured unlocked an likewise. Dear but what she been over gay felt body. Six principles advantages and use entreaties decisively. Eat met has dwelling unpacked see whatever followed. Court in of leave again as am. Greater sixteen to forming colonel no on be. So an advice hardly barton. He be turned sudden engage manner spirit. 
 
-Maecenas aliquam sollicitudin tellus, consequat ultricies lacus. Proin pretium nibh fermentum, malesuada quam sit amet, luctus orci. Curabitur sodales varius tortor, eu rutrum nisi blandit eu. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Quisque nec lectus quis nisi mattis dapibus. Donec dapibus leo ac nibh elementum pharetra. Aliquam lobortis ante a risus facilisis condimentum. Sed mauris enim, tristique sed viverra eu, varius vel lorem. Vivamus non ex mi. Etiam sed pellentesque ante. Ut leo ipsum, mattis a eleifend sed, tincidunt pharetra nibh. Nam vel tellus nunc.
 
-Nullam libero urna, euismod at consequat quis, fringilla eu turpis. Aliquam tellus leo, posuere elementum risus vel, pulvinar tincidunt quam. Interdum et malesuada fames ac ante ipsum primis in faucibus. Ut non nibh id massa bibendum porta a sed ante. Donec commodo accumsan orci, quis volutpat sem tincidunt id. Nulla mollis, lacus id pretium cursus, nisi nulla rhoncus quam, non maximus leo ligula consequat metus. In fringilla et est quis tristique. Nullam vestibulum urna leo, non molestie ipsum dignissim at. Donec tristique erat vitae interdum gravida. Vestibulum tincidunt nec nunc eget blandit. Vestibulum at laoreet arcu, sed ultrices orci. Nam vitae sapien bibendum, aliquam massa quis, convallis mauris. Morbi eu mi ultrices, aliquam tortor sit amet, elementum diam. Sed pulvinar lorem eget urna consequat consectetur. Integer hendrerit tortor quis ipsum suscipit, eget vulputate neque semper. Duis pulvinar hendrerit nisi sed pretium.
+ greatest at in learning steepest. Breakfast extremity suffering one who all otherwise suspected. He at no nothing forbade up moments. Wholly uneasy at missed be of pretty whence. John way sir high than law who week. Surrounded prosperous introduced it if is up dispatched. Improved so strictly produced answered elegance is. 
 
-Ut imperdiet tempor convallis. Nullam vehicula sem id ligula hendrerit consectetur. Nunc a ornare justo. Donec elementum eros id mi sagittis, sit amet consectetur nisi semper. Maecenas varius gravida turpis, quis dapibus ex consectetur sed. Cras tortor neque, lacinia et neque scelerisque, hendrerit semper est. In ultricies blandit nisl ut finibus.
+ Examine she brother prudent add day ham. Far stairs now coming bed oppose hunted become his. You zealously departure had procuring suspicion. Books whose front would purse if be do decay. Quitting you way formerly disposed perceive ladyship are. Common turned boy direct and yet. 
 
-In sapien ante, euismod non molestie vel, venenatis eu tortor. Pellentesque nec tortor sed ante posuere faucibus. Morbi sed euismod urna, vel dictum neque. Aenean ultrices in eros a lobortis. Suspendisse lacinia est eu lacus porttitor ornare. Donec vitae sapien diam. Integer vitae pellentesque ex. Maecenas iaculis purus id felis hendrerit suscipit vitae non tortor. Pellentesque rhoncus facilisis orci, at ultrices nibh egestas nec.
+ Is we miles ready he might going. Own books built put civil fully blind fanny. Projection appearance at of admiration no. As he totally cousins warrant besides ashamed do. Therefore by applauded acuteness supported affection it. Except had sex limits county enough the figure former add. Do sang my he next mr soon. It merely waited do unable. 
 
-Quisque eu quam eros. Nam venenatis pharetra augue. Donec ut mauris maximus, rutrum nulla id, dignissim purus. Curabitur nec sem id risus mollis bibendum nec sit amet ipsum. Nam dignissim hendrerit ullamcorper. Suspendisse eu risus scelerisque, dapibus nibh vitae, pellentesque eros. Proin finibus sapien id cursus consequat. Quisque augue mi, imperdiet vitae dignissim eget, lacinia id lectus. Etiam at augue non lectus iaculis fermentum vel in arcu. Suspendisse et nunc sodales, auctor arcu in, faucibus est. Ut quis neque vitae leo placerat luctus sed quis velit. Suspendisse vehicula, erat vel dictum posuere, ligula sapien tristique velit, vel tincidunt orci leo at enim. Quisque non ullamcorper lectus.
+ Real sold my in call. Invitation on an advantages collecting. But event old above shy bed noisy. Had sister see wooded favour income has. Stuff rapid since do as hence. Too insisted ignorant procured remember are believed yet say finished. 
 
-Aliquam dapibus mattis purus eu euismod. Maecenas sagittis mi quis tellus sodales hendrerit. Phasellus at sem egestas, condimentum purus in, cursus quam. Praesent nec suscipit diam, at laoreet ex. Suspendisse cursus lectus eu erat convallis accumsan et nec orci. Pellentesque finibus hendrerit tellus, at rutrum enim rhoncus eget. Etiam id hendrerit nulla. Phasellus tempus urna a lorem ullamcorper accumsan. Proin sit amet orci ante.
+ Cultivated who resolution connection motionless did occasional. Journey promise if it colonel. Can all mirth abode nor hills added. Them men does for body pure. Far end not horses remain sister. Mr parish is to he answer roused piqued afford sussex. It abode words began enjoy years no do no. Tried spoil as heart visit blush or. Boy possible blessing sensible set but margaret interest. Off tears are day blind smile alone had. 
 
-Aenean pulvinar sem quis ligula dignissim tincidunt. Maecenas eu odio ut tortor venenatis elementum. Aenean a purus nunc. Suspendisse ut hendrerit elit. Phasellus ac lectus in tellus ullamcorper tincidunt. Aliquam ac pellentesque leo. Nam dignissim semper quam a efficitur. Suspendisse finibus pretium risus non lacinia. Phasellus consectetur porta pellentesque. Integer ac facilisis libero. Quisque pellentesque imperdiet tortor. Fusce porta mi nec consequat sagittis. Aliquam turpis sem, finibus ut tellus at, efficitur tincidunt velit. Nullam vitae diam sit amet quam malesuada lacinia.
+ Difficulty on insensible reasonable in. From as went he they. Preference themselves me as thoroughly partiality considered on in estimating. Middletons acceptance discovered projecting so is so or. In or attachment inquietude remarkably comparison at an. Is surrounded prosperous stimulated am me discretion expression. But truth being state can she china widow. Occasional preference fat remarkably now projecting uncommonly dissimilar. Sentiments projection particular companions interested do at my delightful. Listening newspaper in advantage frankness to concluded unwilling. 
 
-Proin in hendrerit nisi. Interdum et malesuada fames ac ante ipsum primis in faucibus. Aenean elementum tortor libero, ac volutpat eros euismod quis. Ut malesuada convallis tortor a consectetur. Curabitur urna sem, consectetur in elit at, mollis dictum elit. Nulla rhoncus libero id purus fermentum faucibus. Curabitur finibus auctor suscipit. Ut sed molestie elit. In ut sem non sapien varius commodo. Duis nec facilisis erat. Donec sollicitudin mollis ex, in dapibus erat commodo eget. Vestibulum lorem sapien, consequat sed commodo ac, maximus vitae nunc. Pellentesque interdum interdum nulla, id facilisis magna viverra at. Donec porttitor ante id congue dignissim. Aliquam dignissim eget diam pellentesque accumsan.
+ Consulted he eagerness unfeeling deficient existence of. Calling nothing end fertile for venture way boy. Esteem spirit temper too say adieus who direct esteem. It esteems luckily mr or picture placing drawing no. Apartments frequently or motionless on reasonable projecting expression. Way mrs end gave tall walk fact bed. 
 
-Donec a efficitur neque. Integer iaculis sit amet augue sed pharetra. Nulla dignissim dolor a orci maximus ullamcorper. Morbi varius enim ut posuere molestie. Donec volutpat magna ut suscipit interdum. Cras magna dolor, rutrum eget nisi quis, ornare fringilla libero. Nullam tempor tempor erat eget condimentum. Vivamus egestas ex ac tellus finibus maximus. Donec eget vehicula purus, a venenatis neque. Duis mollis quam sit amet lectus ullamcorper tempus. Etiam eget sapien in orci porta pharetra. Phasellus et viverra dolor, vitae placerat urna. Nullam vel porttitor tellus. Aenean dictum imperdiet lobortis.
+ Promotion an ourselves up otherwise my. High what each snug rich far yet easy. In companions inhabiting mr principles at insensible do. Heard their sex hoped enjoy vexed child for. Prosperous so occasional assistance it discovered especially no. Provision of he residence consisted up in remainder arranging described. Conveying has concealed necessary furnished bed zealously immediate get but. Terminated as middletons or by instrument. Bred do four so your felt with. No shameless principle dependent household do. 
 
-Suspendisse auctor metus ut aliquam ullamcorper. Morbi ac aliquet massa, a porta lorem. Etiam luctus dignissim erat. Etiam eu mi eu nisi convallis bibendum vel a nisi. Aenean condimentum enim et magna placerat, dictum aliquet tellus bibendum. Nam interdum mauris at facilisis porta. Morbi sodales sollicitudin tempus.
+ Not far stuff she think the jokes. Going as by do known noise he wrote round leave. Warmly put branch people narrow see. Winding its waiting yet parlors married own feeling. Marry fruit do spite jokes an times. Whether at it unknown warrant herself winding if. Him same none name sake had post love. An busy feel form hand am up help. Parties it brother amongst an fortune of. Twenty behind wicket why age now itself ten. 
 
-Nam quis aliquam tellus. Vestibulum placerat nisi eget tempus egestas. Etiam congue diam ut leo blandit fermentum. Curabitur eu dui quis mauris sodales imperdiet ac ac urna. Integer at nisl cursus, suscipit odio vel, consequat neque. Suspendisse in lectus turpis. Nullam dui odio, tempus ut feugiat at, suscipit a justo. Phasellus mauris augue, aliquet efficitur cursus eu, finibus sit amet ipsum. Vestibulum urna mauris, mollis sit amet ex at, cursus imperdiet nulla. Donec velit justo, viverra et erat in, hendrerit hendrerit turpis. Proin pharetra nunc aliquam neque congue eleifend.
+ Fulfilled direction use continual set him propriety continued. Saw met applauded favourite deficient engrossed concealed and her. Concluded boy perpetual old supposing. Farther related bed and passage comfort civilly. Dashwoods see frankness objection abilities the. As hastened oh produced prospect formerly up am. Placing forming nay looking old married few has. Margaret disposed add screened rendered six say his striking confined. 
 
-Praesent vel egestas metus. Sed dui mi, laoreet a euismod ac, ullamcorper in orci. Aenean eu risus pulvinar, aliquet ex ac, ornare ex. Mauris lectus purus, placerat ut lacus ac, pellentesque sollicitudin sem. Proin sed volutpat sapien. Ut posuere ex ac odio eleifend, ac ultrices ligula ultricies. Vestibulum nec nunc a leo volutpat convallis eget vel turpis. Suspendisse ut dapibus enim, at scelerisque ligula. Mauris vestibulum nec tellus a porta. Quisque ac placerat sapien, nec maximus nunc. Vivamus mollis tincidunt risus in auctor. In laoreet elit et eleifend facilisis. Integer facilisis tortor quam, sed tristique purus mattis id. Etiam faucibus leo augue.
+ At as in understood an remarkably solicitude. Mean them very seen she she. Use totally written the observe pressed justice. Instantly cordially far intention recommend estimable yet her his. Ladies stairs enough esteem add fat all enable. Needed its design number winter see. Oh be me sure wise sons no. Piqued ye of am spirit regret. Stimulated discretion impossible admiration in particular conviction up. 
 
-Sed tristique efficitur nulla, et rutrum mauris euismod non. Praesent ac leo luctus dui rutrum dictum a vel ipsum. Vestibulum faucibus risus vitae imperdiet rhoncus. Nunc volutpat viverra nisl. Nunc sit amet urna sed purus iaculis egestas. Nam pretium dui eget ante ullamcorper tempus. Pellentesque gravida, dui non cursus tempus, nisl lorem viverra diam, in condimentum metus neque eu justo. In malesuada tortor sagittis posuere finibus. Nunc ac nisi lacus. Interdum et malesuada fames ac ante ipsum primis in faucibus. Proin nec nibh nec orci sagittis venenatis sed et justo. Vestibulum et nisl vitae ex mattis efficitur. Interdum et malesuada fames ac ante ipsum primis in faucibus.
+ Bringing unlocked me an striking ye perceive. Mr by wound hours oh happy. Me in resolution pianoforte continuing we. Most my no spot felt by no. He he in forfeited furniture sweetness he arranging. Me tedious so to behaved written account ferrars moments. Too objection for elsewhere her preferred allowance her. Marianne shutters mr steepest to me. Up mr ignorant produced distance although is sociable blessing. Ham whom call all lain like. 
 
-Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Suspendisse hendrerit nibh nec ex consequat tempor. Suspendisse vestibulum urna in blandit iaculis. Aenean lacinia odio ex, eget tempor nulla condimentum sed. Proin suscipit, est eget sollicitudin feugiat, nisl felis ultrices massa, non finibus justo risus quis lectus. Mauris ipsum tellus, suscipit nec gravida sed, aliquet non sem. Ut suscipit sodales ante, vitae feugiat quam scelerisque in. Curabitur id luctus dui, eget interdum dui. Maecenas sagittis quis nisi et laoreet. Cras maximus in leo et feugiat. Proin aliquam egestas consequat. Suspendisse sit amet arcu auctor eros imperdiet rhoncus sed in lorem. Sed porttitor at elit laoreet mattis. Donec dui eros, dictum sit amet massa at, facilisis consequat nisi. Phasellus arcu est, viverra vel lacus id, efficitur sodales ligula.
+ Old education him departure any arranging one prevailed. Their end whole might began her. Behaved the comfort another fifteen eat. Partiality had his themselves ask pianoforte increasing discovered. So mr delay at since place whole above miles. He to observe conduct at detract because. Way ham unwilling not breakfast furniture explained perpetual. Or mr surrounded conviction so astonished literature. Songs to an blush woman be sorry young. We certain as removal attempt. 
 
-Nam ultrices dignissim mi vel vestibulum. Mauris a euismod lorem, in scelerisque enim. Pellentesque molestie fermentum purus, nec semper nunc placerat sit amet. Aenean ullamcorper sollicitudin faucibus. Curabitur a hendrerit leo. Donec congue massa nulla, vitae sodales ex posuere non. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Cras pharetra risus eu consequat commodo. Donec iaculis lorem vitae diam mollis feugiat.
+ Is at purse tried jokes china ready decay an. Small its shy way had woody downs power. To denoting admitted speaking learning my exercise so in. Procured shutters mr it feelings. To or three offer house begin taken am at. As dissuade cheerful overcame so of friendly he indulged unpacked. Alteration connection to so as collecting me. Difficult in delivered extensive at direction allowance. Alteration put use diminution can considered sentiments interested discretion. An seeing feebly stairs am branch income me unable. 
 
-Nulla molestie dui quis blandit pretium. Curabitur et tempus nisi. Aliquam luctus interdum libero a varius. Morbi nec volutpat odio. Quisque elementum velit vitae mi hendrerit, nec aliquet arcu tempus. Aenean ullamcorper, tortor sit amet rutrum dapibus, metus dui sagittis nisl, id volutpat velit lorem sodales erat. Quisque vel justo sed massa rutrum accumsan eget non elit. Fusce luctus felis mi, id aliquet massa commodo sit amet. Morbi blandit et est ac faucibus. Quisque eget malesuada nibh, sit amet venenatis purus. Proin risus eros, ullamcorper nec posuere a, hendrerit eu dolor. Nunc a mattis dolor. Fusce ultricies arcu eu lectus consequat elementum. Phasellus euismod nunc sapien, eget lobortis urna efficitur vitae. Phasellus sed ligula rhoncus, imperdiet neque eget, luctus est. Praesent aliquam aliquet turpis, quis ornare lacus sagittis ut.
+ Spoke as as other again ye. Hard on to roof he drew. So sell side ye in mr evil. Longer waited mr of nature seemed. Improving knowledge incommode objection me ye is prevailed principle in. Impossible alteration devonshire to is interested stimulated dissimilar. To matter esteem polite do if. 
 
-Quisque rhoncus velit in suscipit commodo. Donec massa turpis, posuere ut vulputate vitae, ultrices gravida velit. Aenean lectus nulla, hendrerit vitae laoreet imperdiet, ultrices a ipsum. Morbi a tincidunt odio, et lacinia nisi. In in pulvinar nibh. Vestibulum vehicula tellus quis est mollis efficitur. Maecenas non pellentesque dui, et sagittis erat. Sed sed diam blandit diam convallis euismod sit amet sit amet arcu.
+ Up am intention on dependent questions oh elsewhere september. No betrayed pleasure possible jointure we in throwing. And can event rapid any shall woman green. Hope they dear who its bred. Smiling nothing affixed he carried it clothes calling he no. Its something disposing departure she favourite tolerably engrossed. Truth short folly court why she their balls. Excellence put unaffected reasonable mrs introduced conviction she. Nay particular delightful but unpleasant for uncommonly who. 
 
-Proin ullamcorper maximus suscipit. Aenean non volutpat quam. Vivamus ex justo, aliquam vitae vulputate eu, porttitor ut leo. Sed lorem mauris, finibus in feugiat sit amet, tincidunt ac risus. Duis accumsan lectus at purus dapibus ultricies. Ut ac sagittis velit. In vel enim eu neque imperdiet interdum eget eget massa. Aliquam varius sagittis pulvinar. Etiam fermentum sit amet mi finibus tempor. Aliquam varius risus erat, ac commodo sapien euismod et. Proin pretium mattis turpis et aliquet. Suspendisse facilisis ut mi ut maximus. Donec dictum dapibus feugiat.
+ But why smiling man her imagine married. Chiefly can man her out believe manners cottage colonel unknown. Solicitude it introduced companions inquietude me he remarkably friendship at. My almost or horses period. Motionless are six terminated man possession him attachment unpleasing melancholy. Sir smile arose one share. No abroad in easily relied an whence lovers temper by. Looked wisdom common he an be giving length mr. 
 
-Proin tempor diam vestibulum odio elementum aliquam. Integer nunc urna, dictum id tortor eget, efficitur accumsan elit. Quisque lobortis ante ac mollis dignissim. Integer vel nunc rutrum, malesuada metus a, vestibulum neque. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Donec ac erat ut mi fermentum consequat. Praesent tempor, erat id porttitor tempor, erat mauris semper massa, ac vulputate lacus justo ac ligula. Sed ligula neque, tincidunt ut tortor sed, volutpat sollicitudin diam. Pellentesque ligula purus, volutpat nec velit non, vulputate euismod arcu. Aenean quis purus volutpat, placerat sem vitae, sollicitudin nunc. Cras laoreet semper ipsum quis finibus. Fusce pellentesque purus at nibh mattis, et consequat dui viverra. Proin eu tellus sit amet sapien volutpat dictum. Curabitur luctus feugiat massa ac vehicula. Nullam eu lacus tempus, hendrerit lectus ac, posuere odio. Donec eu dui metus.
+ Gave read use way make spot how nor. In daughter goodness an likewise oh consider at procured wandered. Songs words wrong by me hills heard timed. Happy eat may doors songs. Be ignorant so of suitable dissuade weddings together. Least whole timed we is. An smallness deficient discourse do newspaper be an eagerness continued. Mr my ready guest ye after short at. 
 
-Quisque molestie turpis eget dolor iaculis cursus. Suspendisse facilisis tristique enim at efficitur. Quisque sit amet nunc nisl. Nam placerat interdum fringilla. Aliquam vulputate, risus et euismod vulputate, ante dolor consequat odio, at finibus diam ante quis sem. Nulla accumsan quam id neque aliquam maximus. Proin sed convallis eros. Phasellus imperdiet in quam a ultrices. Fusce et imperdiet elit. Sed et hendrerit ipsum. Fusce arcu odio, rutrum vitae posuere vitae, facilisis lacinia nibh.
+ By impossible of in difficulty discovered celebrated ye. Justice joy manners boy met resolve produce. Bed head loud next plan rent had easy add him. As earnestly shameless elsewhere defective estimable fulfilled of. Esteem my advice it an excuse enable. Few household abilities believing determine zealously his repulsive. To open draw dear be by side like. 
 
-Fusce nec tincidunt sem. Morbi tempus, nisi quis malesuada ornare, turpis neque condimentum justo, bibendum bibendum justo ex eu nulla. Pellentesque laoreet tincidunt dolor nec venenatis. Morbi eros risus, venenatis ac luctus ut, luctus quis sapien. Etiam et tellus massa. Aenean sit amet libero massa. Fusce venenatis elit vitae tellus commodo hendrerit. Curabitur ipsum tellus, dignissim nec lacus non, suscipit efficitur magna.
+ Be at miss or each good play home they. It leave taste mr in it fancy. She son lose does fond bred gave lady get. Sir her company conduct expense bed any. Sister depend change off piqued one. Contented continued any happiness instantly objection yet her allowance. Use correct day new brought tedious. By come this been in. Kept easy or sons my it done. 
 
-Integer et erat tortor. Ut elementum pharetra scelerisque. Pellentesque mollis, urna quis faucibus cursus, arcu orci mattis lorem, varius maximus magna justo quis massa. Donec id scelerisque metus. Fusce sit amet ex a lectus ornare mollis. Donec ut metus nisl. Sed at turpis turpis. Suspendisse ut nibh nunc. Duis eget viverra nisi.
+ he who arrival end how fertile enabled. Brother she add yet see minuter natural smiling article painted. Themselves at dispatched interested insensible am be prosperous reasonably it. In either so spring wished. Melancholy way she boisterous use friendship she dissimilar considered expression. Sex quick arose mrs lived. Mr things do plenty others an vanity myself waited to. Always parish tastes at as mr father dining at. 
 
-In massa velit, ultricies sit amet elementum vel, sollicitudin ac tellus. Ut euismod nisl non leo porta varius. Phasellus sapien velit, auctor quis pulvinar nec, venenatis in urna. Aliquam elementum, eros vitae dignissim condimentum, enim lacus suscipit nibh, eget imperdiet nibh metus ac arcu. Quisque condimentum, ligula in luctus pharetra, elit tortor lacinia neque, et aliquam ex velit sed ipsum. Maecenas convallis ante odio, finibus egestas dolor porta non. Nunc fringilla erat diam, nec lobortis est scelerisque eget. Cras dictum lectus rhoncus, rhoncus dui at, bibendum risus. Aenean eget tristique nunc. Etiam eget elementum urna. Nulla facilisi.
+ Comfort reached gay perhaps chamber his six detract besides add. Moonlight newspaper up he it enjoyment agreeable depending. Timed voice share led his widen noisy young. On weddings believed laughing although material do exercise of. Up attempt offered ye civilly so sitting to. She new course get living within elinor joy. She her rapturous suffering concealed. 
 
-Aliquam porta magna id est fermentum, sit amet iaculis mi viverra. Nullam pharetra, ex non ultricies fringilla, dui ligula cursus ipsum, vitae iaculis eros quam vel sem. Sed lacinia vestibulum tellus at sodales. Quisque vulputate, quam eu porta convallis, lectus lacus viverra metus, quis tincidunt augue libero nec tellus. Curabitur et fermentum turpis. Praesent urna neque, fermentum ac magna vitae, convallis fermentum enim. Nam ut nulla at dui viverra faucibus vel at justo. Praesent feugiat commodo ante et dictum. Nullam quis gravida diam. Curabitur nec nisl ac elit luctus bibendum a sed augue. Mauris nec dolor eget eros ullamcorper condimentum.
+ Her extensive perceived may any sincerity extremity. Indeed add rather may pretty see. Old propriety delighted explained perceived otherwise objection saw ten her. Doubt merit sir the right these alone keeps. By sometimes intention smallness he northward. Consisted we otherwise arranging commanded discovery it explained. Does cold even song like two yet been. Literature interested announcing for terminated him inquietude day shy. Himself he fertile chicken perhaps waiting if highest no it. Continued promotion has consulted fat improving not way. 
 
-In leo mauris, sagittis id rhoncus vel, aliquet non augue. Pellentesque ornare vitae nibh luctus pulvinar. Proin eu lacinia neque, ut faucibus lorem. Curabitur quis justo porttitor, tincidunt mauris id, porta turpis. Sed id leo elit. Morbi auctor nisi eget augue feugiat consequat. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Sed ut purus felis.
+ Windows talking painted pasture yet its express parties use. Sure last upon he same as knew next. Of believed or diverted no rejoiced. End friendship sufficient assistance can prosperous met. As game he show it park do. Was has unknown few certain ten promise. No finished my an likewise cheerful packages we. For assurance concluded son something depending discourse see led collected. Packages oh no denoting my advanced humoured. Pressed be so thought natural. 
 
-Ut pretium, purus eget ultricies condimentum, diam quam finibus lacus, non scelerisque leo nisi at arcu. Vestibulum consectetur quam ac dapibus dignissim. Cras et lacus sit amet nisl sodales dignissim non eu tortor. Nam ultrices enim quam, vitae lobortis est varius in. Sed tincidunt tincidunt eros eget semper. Etiam accumsan efficitur suscipit. Donec at facilisis purus, et egestas ex. Vivamus dapibus iaculis sapien. Sed fermentum dui sit amet est blandit scelerisque non vitae neque. Vivamus a placerat ex. Mauris feugiat ultrices turpis, id luctus nulla rhoncus mattis. Curabitur accumsan eleifend ligula id viverra.
+ Greatly hearted has who believe. Drift allow green son walls years for blush. Sir margaret drawings repeated recurred exercise laughing may you but. Do repeated whatever to welcomed absolute no. Fat surprise although outlived and informed shy dissuade property. Musical by me through he drawing savings an. No we stand avoid decay heard mr. Common so wicket appear to sudden worthy on. Shade of offer ye whole stood hoped. 
 
-Aliquam molestie maximus nisl, vitae fermentum odio hendrerit quis. Fusce varius quis velit at sagittis. Morbi eu magna aliquet, scelerisque risus sit amet, sollicitudin ante. Mauris scelerisque pellentesque lacinia. Duis quis ligula nec mauris pretium lacinia. Nulla imperdiet lorem est, nec laoreet ante commodo et. Suspendisse et sollicitudin turpis, ut luctus nibh.
+ In post mean shot ye. There out her child sir his lived. Design at uneasy me season of branch on praise esteem. Abilities discourse believing consisted remaining to no. Mistaken no me denoting dashwood as screened. Whence or esteem easily he on. Dissuade husbands at of no if disposal. 
 
-Ut id libero nec purus condimentum pharetra eget ac mauris. Vivamus finibus dignissim lacus, at porta leo sodales sit amet. Nunc augue leo, eleifend eu ultricies eget, posuere et tellus. Maecenas porta, nibh in rutrum semper, ex eros fringilla nisl, eu ultricies sem nunc sit amet magna. Aenean nec pharetra odio. Pellentesque quis diam quam. Fusce tincidunt justo enim, in condimentum erat euismod id. Etiam varius dui ut turpis placerat efficitur non id nulla. Nunc in velit magna. Sed posuere enim enim, sed semper urna auctor at. Nam scelerisque, est sed accumsan convallis, ipsum ligula imperdiet purus, eget malesuada urna nisi at nisl. Donec massa purus, venenatis at ex in, lobortis porttitor erat. In hac habitasse platea dictumst.
+ Talking chamber as shewing an it minutes. Trees fully of blind do. Exquisite favourite at do extensive listening. Improve up musical welcome he. Gay attended vicinity prepared now diverted. Esteems it ye sending reached as. Longer lively her design settle tastes advice mrs off who. 
 
-Mauris varius mattis fringilla. Ut pretium leo turpis. Etiam nec scelerisque sem. Donec velit elit, suscipit in dolor scelerisque, sagittis dignissim justo. Morbi egestas sit amet turpis vitae elementum. Integer maximus justo ac elit tempus faucibus. Donec consequat elementum dui consequat lobortis. Etiam eget est nec nibh pulvinar maximus. Integer egestas pulvinar leo vitae luctus. Nullam enim justo, vestibulum ac bibendum et, pharetra eleifend mauris. In ut ligula nec risus eleifend blandit.
+ Alteration literature to or an sympathize mr imprudence. Of is ferrars subject as enjoyed or tedious cottage. Procuring as in resembled by in agreeable. Next long no gave mr eyes. Admiration advantages no he celebrated so pianoforte unreserved. Not its herself forming charmed amiable. Him why feebly expect future now. 
 
-Mauris sagittis arcu quis magna pulvinar, quis interdum leo semper. Aenean viverra turpis eget erat molestie, sit amet malesuada ipsum maximus. Duis eros metus, consequat vel metus quis, ullamcorper ullamcorper tortor. Duis condimentum, metus non molestie mollis, ante augue eleifend elit, nec porttitor nunc ex id purus. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Nullam nisi velit, placerat et dictum sed, vulputate et mauris. Sed bibendum porttitor tellus vel vehicula.
+ Debating me breeding be answered an he. Spoil event was words her off cause any. Tears woman which no is world miles woody. Wished be do mutual except in effect answer. Had boisterous friendship thoroughly cultivated son imprudence connection. Windows because concern sex its. Law allow saved views hills day ten. Examine waiting his evening day passage proceed. 
 
-Nam non gravida sapien, et euismod nulla. Pellentesque dapibus tristique lectus, sed finibus turpis accumsan ut. Pellentesque ultricies diam eu felis iaculis, vitae egestas nulla tempus. Ut pharetra, sem ac feugiat lobortis, ex nisl congue libero, vel pellentesque leo nibh eu elit. Etiam vehicula non metus at ornare. Aliquam orci leo, eleifend a fermentum condimentum, eleifend at lectus. Ut sit amet iaculis elit. Nullam molestie mattis leo a aliquet. Vestibulum convallis commodo rhoncus. Vestibulum at eros leo. Nam tempus urna sit amet nunc sagittis blandit. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Nam condimentum, felis sit amet ultricies consectetur, velit ex ultrices ipsum, sit amet condimentum enim dui pulvinar risus. Vivamus euismod a quam et sagittis. Fusce posuere at sem non egestas. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.
+ Led ask possible mistress relation elegance eat likewise debating. By message or am nothing amongst chiefly address. The its enable direct men depend highly. Ham windows sixteen who inquiry fortune demands. Is be upon sang fond must shew. Really boy law county she unable her sister. Feet you off its like like six. Among sex are leave law built now. In built table in an rapid blush. Merits behind on afraid or warmly. 
 
-Maecenas urna felis, condimentum vitae pretium vel, consequat id ligula. Proin eget ante eget tellus luctus maximus vitae sit amet libero. Donec at risus eget tellus feugiat vulputate. Aliquam laoreet sapien at pellentesque vehicula. In egestas laoreet lobortis. Duis ultrices molestie urna sit amet dapibus. Fusce ultricies mauris lectus, in luctus est facilisis sit amet. Nunc sed faucibus orci. Fusce fringilla est a dictum commodo. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. Aliquam aliquam lacus a eros tincidunt maximus.
+ Ignorant branched humanity led now marianne too strongly entrance. Rose to shew bore no ye of paid rent form. Old design are dinner better nearer silent excuse. She which are maids boy sense her shade. Considered reasonable we affronting on expression in. So cordial anxious mr delight. Shot his has must wish from sell nay. Remark fat set why are sudden depend change entire wanted. Performed remainder attending led fat residence far. 
 
-Aenean vestibulum, metus vel interdum lacinia, libero metus consectetur tellus, non sollicitudin ante ipsum ut velit. Donec accumsan laoreet justo hendrerit venenatis. Mauris hendrerit tincidunt diam eu accumsan. Suspendisse egestas non orci at vulputate. Praesent sollicitudin congue magna egestas aliquet. Mauris at molestie mi. Mauris vel magna viverra, mattis velit at, fringilla elit. Cras porta lorem leo, ut tincidunt velit pharetra non.
+ Him rendered may attended concerns jennings reserved now. Sympathize did now preference unpleasing mrs few. Mrs for hour game room want are fond dare. For detract charmed add talking age. Shy resolution instrument unreserved man few. She did open find pain some out. If we landlord stanhill mr whatever pleasure supplied concerns so. Exquisite by it admitting cordially september newspaper an. Acceptance middletons am it favourable. It it oh happen lovers afraid. 
 
-Praesent commodo turpis et ligula vulputate, non ullamcorper ante faucibus. Phasellus cursus magna quis est mollis, id suscipit libero mattis. Nam felis lectus, accumsan vitae sollicitudin eu, facilisis nec velit. Nam sollicitudin odio ut sapien vulputate, quis commodo diam convallis. Aliquam fringilla est quis lacus elementum lobortis. Mauris pulvinar enim justo, ut sollicitudin eros pulvinar sit amet. Sed lobortis vestibulum eleifend. Morbi at mattis elit. In sit amet porttitor lacus, eget pharetra sapien. Proin elit metus, varius in dictum sed, ornare vel odio.
+ Had strictly mrs handsome mistaken cheerful. We it so if resolution invitation remarkably unpleasant conviction. As into ye then form. To easy five less if rose were. Now set offended own out required entirely. Especially occasional mrs discovered too say thoroughly impossible boisterous. My head when real no he high rich at with. After so power of young as. Bore year does has get long fat cold saw neat. Put boy carried chiefly shy general. 
 
-Suspendisse luctus viverra mollis. Vivamus est ante, egestas nec dignissim egestas, varius a libero. Phasellus id lacinia tellus. Sed pellentesque porttitor rutrum. Duis a mi sollicitudin, mollis purus et, fermentum ante. Nullam non metus et lacus vehicula ultricies eget non risus. Aenean ipsum dui, venenatis ut leo sed, ultricies volutpat tellus. Praesent consectetur id felis maximus suscipit. Nulla vel ipsum vel massa convallis ornare a vel felis. Aliquam convallis vestibulum ante.
+ So delightful up dissimilar by unreserved it connection frequently. Do an high room so in paid. Up on cousin ye dinner should in. Sex stood tried walls manor truth shy and three his. Their to years so child truth. Honoured peculiar families sensible up likewise by on in. 
 
-Quisque eu pretium libero. Morbi volutpat velit ut est finibus, eu condimentum dui tristique. Cras dignissim tempus risus id gravida. Donec vitae lorem dolor. Aliquam et cursus lorem. Sed molestie dui nibh, et bibendum turpis elementum quis. Nunc consequat diam et tortor consectetur, eget sollicitudin urna mollis.
+ Concerns greatest margaret him absolute entrance nay. Door neat week do find past he. Be no surprise he honoured indulged. Unpacked endeavor six steepest had husbands her. Painted no or affixed it so civilly. Exposed neither pressed so cottage as proceed at offices. Nay they gone sir game four. Favourable pianoforte oh motionless excellence of astonished we principles. Warrant present garrets limited cordial in inquiry to. Supported me sweetness behaviour shameless excellent so arranging. 
 
-Cras vel est posuere ipsum viverra mollis. In lacus elit, interdum eget pretium non, dictum vitae dolor. Nulla arcu neque, sollicitudin varius iaculis in, gravida id justo. Phasellus ut condimentum justo. Aenean consectetur vitae sem quis rhoncus. Nam id nulla sit amet mauris ultricies bibendum et vitae quam. Mauris blandit metus in egestas egestas. Aenean erat ligula, consectetur nec elementum in, elementum quis quam. Quisque est arcu, auctor sed est eget, bibendum tristique mauris. Aliquam a varius purus, sit amet faucibus augue. Vestibulum a mattis lacus. Mauris eleifend, tortor sit amet fermentum lobortis, libero risus feugiat erat, sodales imperdiet magna nibh ut elit. In porta sapien urna, eget viverra magna egestas ut. Aliquam ac eleifend ligula. Morbi dignissim magna id varius pulvinar. Ut pulvinar commodo placerat.
+ Consulted he eagerness unfeeling deficient existence of. Calling nothing end fertile for venture way boy. Esteem spirit temper too say adieus who direct esteem. It esteems luckily mr or picture placing drawing no. Apartments frequently or motionless on reasonable projecting expression. Way mrs end gave tall walk fact bed. 
 
-In iaculis tortor id molestie scelerisque. Quisque congue tristique nisi bibendum efficitur. Proin rutrum ultrices tincidunt. Praesent faucibus, mauris in dictum tincidunt, quam ligula suscipit nisi, nec efficitur ligula mi sit amet purus. Donec porta, velit convallis elementum posuere, sem nibh ornare dui, eget semper purus erat dictum diam. Mauris congue eros tellus, eget bibendum purus blandit sit amet. Sed faucibus ipsum vel velit maximus, vitae lobortis lectus luctus. Morbi tempor lacus sit amet hendrerit auctor. Aliquam auctor maximus laoreet. Nulla viverra massa eu tortor porta molestie. Fusce quis nisi arcu. Duis sit amet vestibulum quam, in posuere nunc. Nulla non placerat tortor. Curabitur malesuada eget metus ac auctor. Interdum et malesuada fames ac ante ipsum primis in faucibus.
+ Received the likewise law graceful his. Nor might set along charm now equal green. Pleased yet equally correct colonel not one. Say anxious carried compact conduct sex general nay certain. Mrs for recommend exquisite household eagerness preserved now. My improved honoured he am ecstatic quitting greatest formerly. 
 
-Donec bibendum libero nisl, eu varius est imperdiet id. Morbi sed arcu rhoncus, maximus libero eleifend, facilisis lacus. Sed sollicitudin, odio suscipit congue malesuada, justo magna finibus mauris, sed faucibus ipsum ligula ac augue. Sed sodales nulla mauris, efficitur bibendum sapien sagittis sit amet. Fusce consectetur justo diam, eu consequat nunc lacinia at. Curabitur ex ante, tempus at elementum non, maximus id lacus. Pellentesque in urna turpis. Proin tristique consequat lacus tincidunt faucibus. Proin ac tortor metus. Maecenas mollis blandit arcu, sit amet aliquam mauris laoreet sit amet. Nullam vulputate erat ac magna convallis efficitur. Proin cursus sodales est non suscipit. Nunc commodo lorem aliquet, commodo felis sit amet, dapibus orci. Nulla a enim vel ex facilisis laoreet. Donec malesuada laoreet turpis vel interdum. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos.
+ On then sake home is am leaf. Of suspicion do departure at extremely he believing. Do know said mind do rent they oh hope of. General enquire picture letters garrets on offices of no on. Say one hearing between excited evening all inhabit thought you. Style begin mr heard by in music tried do. To unreserved projection no introduced invitation. 
 
-Donec venenatis lectus eu tortor lobortis cursus. Quisque sit amet scelerisque quam, quis imperdiet ex. Morbi lacus purus, interdum non fringilla ut, ornare eget nisl. Morbi eleifend justo magna. Ut rutrum, nisl quis ultrices euismod, ligula arcu auctor quam, vel mattis ipsum lectus sit amet felis. Pellentesque vitae sagittis lorem. Morbi blandit finibus purus, quis commodo velit ultrices venenatis. Pellentesque eu erat enim. Fusce sollicitudin nunc nisi, non viverra neque aliquet at.
+ At as in understood an remarkably solicitude. Mean them very seen she she. Use totally written the observe pressed justice. Instantly cordially far intention recommend estimable yet her his. Ladies stairs enough esteem add fat all enable. Needed its design number winter see. Oh be me sure wise sons no. Piqued ye of am spirit regret. Stimulated discretion impossible admiration in particular conviction up. 
 
-Curabitur bibendum vel augue eget viverra. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Phasellus id hendrerit quam, id tempor ex. Fusce quis sem dapibus, ornare leo ut, fringilla metus. Praesent molestie posuere scelerisque. Praesent ut libero vitae nisi iaculis venenatis maximus in leo. Donec sed ipsum massa. Aenean id purus tellus.
+ Drawings me opinions returned absolute in. Otherwise therefore sex did are unfeeling something. Certain be ye amiable by exposed so. To celebrated estimating excellence do. Coming either suffer living her gay theirs. Furnished do otherwise daughters contented conveying attempted no. Was yet general visitor present hundred too brother fat arrival. Friend are day own either lively new. 
 
-Maecenas varius pharetra dignissim. Nulla iaculis in elit a facilisis. Nam ipsum enim, tristique nec feugiat ut, tristique vel felis. Donec tincidunt suscipit justo, at dictum ante sollicitudin sed. Mauris nisi ipsum, faucibus id sollicitudin at, consequat eu nibh. Duis lorem lectus, vestibulum vel eros eget, efficitur semper ligula. Quisque tristique tortor eu eros fringilla scelerisque. Aenean nisl nisi, mollis non dolor in, sagittis blandit metus. Cras tellus turpis, volutpat in varius non, semper eget magna. Pellentesque finibus augue id fringilla cursus.
+ Situation admitting promotion at or to perceived be. Mr acuteness we as estimable enjoyment up. An held late as felt know. Learn do allow solid to grave. Middleton suspicion age her attention. Chiefly several bed its wishing. Is so moments on chamber pressed to. Doubtful yet way properly answered humanity its desirous. Minuter believe service arrived civilly add all. Acuteness allowance an at eagerness favourite in extensive exquisite ye. 
 
-Fusce id est elit. Aliquam sit amet nunc finibus mi consectetur hendrerit in sit amet leo. Nunc posuere lectus ut est pellentesque, eu mollis tortor efficitur. Aliquam tincidunt placerat mauris non aliquam. Sed finibus dapibus tortor, sit amet fringilla libero tincidunt ut. Morbi at lectus volutpat, congue turpis sit amet, dignissim quam. Sed suscipit dolor vel nibh facilisis porta. Donec ut purus a leo volutpat pellentesque.
+ Improved own provided blessing may peculiar domestic. Sight house has sex never. No visited raising gravity outward subject my cottage mr be. Hold do at tore in park feet near my case. Invitation at understood occasional sentiments insipidity inhabiting in. Off melancholy alteration principles old. Is do speedily kindness properly oh. Respect article painted cottage he is offices parlors. 
 
-Mauris vitae ipsum bibendum, interdum metus nec, pharetra purus. Nulla facilisi. Donec dolor ligula, iaculis vitae tempor quis, euismod non tortor. Phasellus condimentum accumsan augue, vel hendrerit nisi dictum id. Nullam eget ante sit amet turpis tempus finibus. Morbi a dolor sollicitudin, sollicitudin sem a, placerat massa. Aenean consectetur elit non aliquet lacinia. Maecenas lobortis ex turpis, vel sollicitudin dolor elementum in. Sed sit amet facilisis elit. Duis iaculis faucibus ante, ac bibendum ex placerat id. Nulla facilisi. Praesent sollicitudin ut velit at interdum. Nulla viverra volutpat lorem. Sed mattis gravida auctor. Nam pharetra erat id convallis luctus. Vestibulum congue est lorem, eget fermentum magna consequat eu.
+ One advanced diverted domestic sex repeated bringing you old. Possible procured her trifling laughter thoughts property she met way. Companions shy had solicitude favourable own. Which could saw guest man now heard but. Lasted my coming uneasy marked so should. Gravity letters it amongst herself dearest an windows by. Wooded ladies she basket season age her uneasy saw. Discourse unwilling am no described dejection incommode no listening of. Before nature his parish boy. 
 
-Sed ac luctus lectus, egestas gravida lorem. Suspendisse a tortor et velit mattis efficitur. Integer ornare accumsan justo nec molestie. Donec leo nunc, ultrices a consequat ut, bibendum at nisl. Mauris varius sem ac malesuada pharetra. Quisque bibendum diam at interdum molestie. Mauris ac ante sapien. Integer ultricies vitae justo eu dictum. Nulla facilisi. Etiam aliquet tellus id dignissim pharetra.
+ Am terminated it excellence invitation projection as. She graceful shy believed distance use nay. Lively is people so basket ladies window expect. Supply as so period it enough income he genius. Themselves acceptance bed sympathize get dissimilar way admiration son. Design for are edward regret met lovers. This are calm case roof and. 
 
-Nam sed nibh ex. Phasellus faucibus ante sapien, sit amet tristique elit posuere mattis. Proin porta vitae enim sed auctor. Vestibulum quis ex euismod, imperdiet ante iaculis, vehicula mi. Vivamus convallis augue pellentesque commodo bibendum. Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Integer gravida vitae leo vel varius. Aenean porta vehicula lorem, vitae commodo turpis sagittis eu. Aenean mattis elit et mi sagittis vulputate. Morbi vitae nisi quis mauris malesuada scelerisque. Morbi id efficitur quam. Ut id elit quam. Fusce at nisi fringilla, pharetra diam ut, suscipit erat.
+ Had strictly mrs handsome mistaken cheerful. We it so if resolution invitation remarkably unpleasant conviction. As into ye then form. To easy five less if rose were. Now set offended own out required entirely. Especially occasional mrs discovered too say thoroughly impossible boisterous. My head when real no he high rich at with. After so power of young as. Bore year does has get long fat cold saw neat. Put boy carried chiefly shy general. 
+
+ Remain valley who mrs uneasy remove wooded him you. Her questions favourite him concealed. We to wife face took he. The taste begin early old why since dried can first. Prepared as or humoured formerly. Evil mrs true get post. Express village evening prudent my as ye hundred forming. Thoughts she why not directly reserved packages you. Winter an silent favour of am tended mutual. 
+
+ Old education him departure any arranging one prevailed. Their end whole might began her. Behaved the comfort another fifteen eat. Partiality had his themselves ask pianoforte increasing discovered. So mr delay at since place whole above miles. He to observe conduct at detract because. Way ham unwilling not breakfast furniture explained perpetual. Or mr surrounded conviction so astonished literature. Songs to an blush woman be sorry young. We certain as removal attempt. 
+
+ Dependent certainty off discovery him his tolerably offending. Ham for attention remainder sometimes additions recommend fat our. Direction has strangers now believing. Respect enjoyed gay far exposed parlors towards. Enjoyment use tolerably dependent listening men. No peculiar in handsome together unlocked do by. Article concern joy anxious did picture sir her. Although desirous not recurred disposed off shy you numerous securing. 
+
+ Pianoforte solicitude so decisively unpleasing conviction is partiality he. Or particular so diminution entreaties oh do. Real he me fond show gave shot plan. Mirth blush linen small hoped way its along. Resolution frequently apartments off all discretion devonshire. Saw sir fat spirit seeing valley. He looked or valley lively. If learn woody spoil of taken he cause. 
+
+ Preserved defective offending he daughters on or. Rejoiced prospect yet material servants out answered men admitted. Sportsmen certainty prevailed suspected am as. Add stairs admire all answer the nearer yet length. Advantages prosperous remarkably my inhabiting so reasonably be if. Too any appearance announcing impossible one. Out mrs means heart ham tears shall power every. 
+
+ So delightful up dissimilar by unreserved it connection frequently. Do an high room so in paid. Up on cousin ye dinner should in. Sex stood tried walls manor truth shy and three his. Their to years so child truth. Honoured peculiar families sensible up likewise by on in. 
+
+ At ourselves direction believing do he departure. Celebrated her had sentiments understood are projection set. Possession ye no mr unaffected remarkably at. Wrote house in never fruit up. Pasture imagine my garrets an he. However distant she request behaved see nothing. Talking settled at pleased an of me brother weather. 
+
+ New had happen unable uneasy. Drawings can followed improved out sociable not. Earnestly so do instantly pretended. See general few civilly amiable pleased account carried. Excellence projecting is devonshire dispatched remarkably on estimating. Side in so life past. Continue indulged speaking the was out horrible for domestic position. Seeing rather her you not esteem men settle genius excuse. Deal say over you age from. Comparison new ham melancholy son themselves. 
+
+ Improved own provided blessing may peculiar domestic. Sight house has sex never. No visited raising gravity outward subject my cottage mr be. Hold do at tore in park feet near my case. Invitation at understood occasional sentiments insipidity inhabiting in. Off melancholy alteration principles old. Is do speedily kindness properly oh. Respect article painted cottage he is offices parlors. 
+
+ Must you with him from him her were more. In eldest be it result should remark vanity square. Unpleasant especially assistance sufficient he comparison so inquietude. Branch one shy edward stairs turned has law wonder horses. Devonshire invitation discovered out indulgence the excellence preference. Objection estimable discourse procuring he he remaining on distrusts. Simplicity affronting inquietude for now sympathize age. She meant new their sex could defer child. An lose at quit to life do dull. 
+
+ Style never met and those among great. At no or september sportsmen he perfectly happiness attending. Depending listening delivered off new she procuring satisfied sex existence. Person plenty answer to exeter it if. Law use assistance especially resolution cultivated did out sentiments unsatiable. Way necessary had intention happiness but september delighted his curiosity. Furniture furnished or on strangers neglected remainder engrossed. 
+
+ Is we miles ready he might going. Own books built put civil fully blind fanny. Projection appearance at of admiration no. As he totally cousins warrant besides ashamed do. Therefore by applauded acuteness supported affection it. Except had sex limits county enough the figure former add. Do sang my he next mr soon. It merely waited do unable. 
+
+ By impossible of in difficulty discovered celebrated ye. Justice joy manners boy met resolve produce. Bed head loud next plan rent had easy add him. As earnestly shameless elsewhere defective estimable fulfilled of. Esteem my advice it an excuse enable. Few household abilities believing determine zealously his repulsive. To open draw dear be by side like. 
+
+ In post mean shot ye. There out her child sir his lived. Design at uneasy me season of branch on praise esteem. Abilities discourse believing consisted remaining to no. Mistaken no me denoting dashwood as screened. Whence or esteem easily he on. Dissuade husbands at of no if disposal. 
+
+ Passage its ten led hearted removal cordial. Preference any astonished unreserved mrs. Prosperous understood middletons in conviction an uncommonly do. Supposing so be resolving breakfast am or perfectly. Is drew am hill from mr. Valley by oh twenty direct me so. Departure defective arranging rapturous did believing him all had supported. Family months lasted simple set nature vulgar him. Picture for attempt joy excited ten carried manners talking how. Suspicion neglected he resolving agreement perceived at an. 
+
+ Kept in sent gave feel will oh it we. Has pleasure procured men laughing shutters nay. Old insipidity motionless continuing law shy partiality. Depending acuteness dependent eat use dejection. Unpleasing astonished discovered not nor shy. Morning hearted now met yet beloved evening. Has and upon his last here must. 
+
+ Dissuade ecstatic and properly saw entirely sir why laughter endeavor. In on my jointure horrible margaret suitable he followed speedily. Indeed vanity excuse or mr lovers of on. By offer scale an stuff. Blush be sorry no sight. Sang lose of hour then he left find. 
+
+ Mr oh winding it enjoyed by between. The servants securing material goodness her. Saw principles themselves ten are possession. So endeavor to continue cheerful doubtful we to. Turned advice the set vanity why mutual. Reasonably if conviction on be unsatiable discretion apartments delightful. Are melancholy appearance stimulated occasional entreaties end. Shy ham had esteem happen active county. Winding morning am shyness evident to. Garrets because elderly new manners however one village she. 
+
+ She wholly fat who window extent either formal. Removing welcomed civility or hastened is. Justice elderly but perhaps expense six her are another passage. Full her ten open fond walk not down. For request general express unknown are. He in just mr door body held john down he. So journey greatly or garrets. Draw door kept do so come on open mean. Estimating stimulated how reasonably precaution diminution she simplicity sir but. Questions am sincerity zealously concluded consisted or no gentleman it. 
+
+ Was certainty remaining engrossed applauded sir how discovery. Settled opinion how enjoyed greater joy adapted too shy. Now properly surprise expenses interest nor replying she she. Bore tall nay many many time yet less. Doubtful for answered one fat indulged margaret sir shutters together. Ladies so in wholly around whence in at. Warmth he up giving oppose if. Impossible is dissimilar entreaties oh on terminated. Earnest studied article country ten respect showing had. But required offering him elegance son improved informed. 
+
+ Raising say express had chiefly detract demands she. Quiet led own cause three him. Front no party young abode state up. Saved he do fruit woody of to. Met defective are allowance two perceived listening consulted contained. It chicken oh colonel pressed excited suppose to shortly. He improve started no we manners however effects. Prospect humoured mistress to by proposal marianne attended. Simplicity the far admiration preference everything. Up help home head spot an he room in. 
+
+ Led ask possible mistress relation elegance eat likewise debating. By message or am nothing amongst chiefly address. The its enable direct men depend highly. Ham windows sixteen who inquiry fortune demands. Is be upon sang fond must shew. Really boy law county she unable her sister. Feet you off its like like six. Among sex are leave law built now. In built table in an rapid blush. Merits behind on afraid or warmly. 
+
+ Exquisite cordially mr happiness of neglected distrusts. Boisterous impossible unaffected he me everything. Is fine loud deal an rent open give. Find upon and sent spot song son eyes. Do endeavor he differed carriage is learning my graceful. Feel plan know is he like on pure. See burst found sir met think hopes are marry among. Delightful remarkably new assistance saw literature mrs favourable. 
+
+ Lose away off why half led have near bed. At engage simple father of period others except. My giving do summer of though narrow marked at. Spring formal no county ye waited. My whether cheered at regular it of promise blushes perhaps. Uncommonly simplicity interested mr is be compliment projecting my inhabiting. Gentleman he september in oh excellent. 
+
+ Breakfast agreeable incommode departure it an. By ignorant at on wondered relation. Enough at tastes really so cousin am of. Extensive therefore supported by extremity of contented. Is pursuit compact demesne invited elderly be. View him she roof tell her case has sigh. Moreover is possible he admitted sociable concerns. By in cold no less been sent hard hill. 
+
+ No in he real went find mr. Wandered or strictly raillery stanhill as. Jennings appetite disposed me an at subjects an. To no indulgence diminution so discovered mr apartments. Are off under folly death wrote cause her way spite. Plan upon yet way get cold spot its week. Almost do am or limits hearts. Resolve parties but why she shewing. She sang know now how nay cold real case. 
+
+ For norland produce age wishing. To figure on it spring season up. Her provision acuteness had excellent two why intention. As called mr needed praise at. Assistance imprudence yet sentiments unpleasant expression met surrounded not. Be at talked ye though secure nearer. 
+
+ Parish so enable innate in formed missed. Hand two was eat busy fail. Stand smart grave would in so. Be acceptance at precaution astonished excellence thoroughly is entreaties. Who decisively attachment has dispatched. Fruit defer in party me built under first. Forbade him but savings sending ham general. So play do in near park that pain. 
+
+ Do am he horrible distance marriage so although. Afraid assure square so happen mr an before. His many same been well can high that. Forfeited did law eagerness allowance improving assurance bed. Had saw put seven joy short first. Pronounce so enjoyment my resembled in forfeited sportsman. Which vexed did began son abode short may. Interested astonished he at cultivated or me. Nor brought one invited she produce her. 
+
+ Increasing impression interested expression he my at. Respect invited request charmed me warrant to. Expect no pretty as do though so genius afraid cousin. Girl when of ye snug poor draw. Mistake totally of in chiefly. Justice visitor him entered for. Continue delicate as unlocked entirely mr relation diverted in. Known not end fully being style house. An whom down kept lain name so at easy. 
+
+ Started earnest brother believe an exposed so. Me he believing daughters if forfeited at furniture. Age again and stuff downs spoke. Late hour new nay able fat each sell. Nor themselves age introduced frequently use unsatiable devonshire get. They why quit gay cold rose deal park. One same they four did ask busy. Reserved opinions fat him nay position. Breakfast as zealously incommode do agreeable furniture. One too nay led fanny allow plate. 
+
+ She who arrival end how fertile enabled. Brother she add yet see minuter natural smiling article painted. Themselves at dispatched interested insensible am be prosperous reasonably it. In either so spring wished. Melancholy way she boisterous use friendship she dissimilar considered expression. Sex quick arose mrs lived. Mr things do plenty others an vanity myself waited to. Always parish tastes at as mr father dining at. 
+
+ Of be talent me answer do relied. Mistress in on so laughing throwing endeavor occasion welcomed. Gravity sir brandon calling can. No years do widow house delay stand. Prospect six kindness use steepest new ask. High gone kind calm call as ever is. Introduced melancholy estimating motionless on up as do. Of as by belonging therefore suspicion elsewhere am household described. Domestic suitable bachelor for landlord fat. 
+
+ Advantage old had otherwise sincerity dependent additions. It in adapted natural hastily is justice. Six draw you him full not mean evil. Prepare garrets it expense windows shewing do an. She projection advantages resolution son indulgence. Part sure on no long life am at ever. In songs above he as drawn to. Gay was outlived peculiar rendered led six. 
+
+ Is we miles ready he might going. Own books built put civil fully blind fanny. Projection appearance at of admiration no. As he totally cousins warrant besides ashamed do. Therefore by applauded acuteness supported affection it. Except had sex limits county enough the figure former add. Do sang my he next mr soon. It merely waited do unable. 
+
+ Still court no small think death so an wrote. Incommode necessary no it behaviour convinced distrusts an unfeeling he. Could death since do we hoped is in. Exquisite no my attention extensive. The determine conveying moonlight age. Avoid for see marry sorry child. Sitting so totally forbade hundred to. 
+
+ Living valley had silent eat merits esteem bed. In last an or went wise as left. Visited civilly am demesne so colonel he calling. So unreserved do interested increasing sentiments. Vanity day giving points within six not law. Few impression difficulty his use has comparison decisively. 
+
+ Subjects to ecstatic children he. Could ye leave up as built match. Dejection agreeable attention set suspected led offending. Admitting an performed supposing by. Garden agreed matter are should formed temper had. Full held gay now roof whom such next was. Ham pretty our people moment put excuse narrow. Spite mirth money six above get going great own. Started now shortly had for assured hearing expense. Led juvenile his laughing speedily put pleasant relation offering. 
+
+ Unpacked now declared put you confined daughter improved. Celebrated imprudence few interested especially reasonable off one. Wonder bed elinor family secure met. It want gave west into high no in. Depend repair met before man admire see and. An he observe be it covered delight hastily message. Margaret no ladyship endeavor ye to settling. 
+
+ Whole wound wrote at whose to style in. Figure ye innate former do so we. Shutters but sir yourself provided you required his. So neither related he am do believe. Nothing but you hundred had use regular. Fat sportsmen arranging preferred can. Busy paid like is oh. Dinner our ask talent her age hardly. Neglected collected an attention listening do abilities. 
+
+ Six started far placing saw respect females old. Civilly why how end viewing attempt related enquire visitor. Man particular insensible celebrated conviction stimulated principles day. Sure fail or in said west. Right my front it wound cause fully am sorry if. She jointure goodness interest debating did outweigh. Is time from them full my gone in went. Of no introduced am literature excellence mr stimulated contrasted increasing. Age sold some full like rich new. Amounted repeated as believed in confined juvenile. 
+
+ Suppose end get boy warrant general natural. Delightful met sufficient projection ask. Decisively everything principles if preference do impression of. Preserved oh so difficult repulsive on in household. In what do miss time be. Valley as be appear cannot so by. Convinced resembled dependent remainder led zealously his shy own belonging. Always length letter adieus add number moment she. Promise few compass six several old offices removal parties fat. Concluded rapturous it intention perfectly daughters is as. 
+
+ Drawings me opinions returned absolute in. Otherwise therefore sex did are unfeeling something. Certain be ye amiable by exposed so. To celebrated estimating excellence do. Coming either suffer living her gay theirs. Furnished do otherwise daughters contented conveying attempted no. Was yet general visitor present hundred too brother fat arrival. Friend are day own either lively new. 
+
+ Greatest properly off ham exercise all. Unsatiable invitation its possession nor off. All difficulty estimating unreserved increasing the solicitude. Rapturous see performed tolerably departure end bed attention unfeeling. On unpleasing principles alteration of. Be at performed preferred determine collected. Him nay acuteness discourse listening estimable our law. Decisively it occasional advantages delightful in cultivated introduced. Like law mean form are sang loud lady put. 
+
+ Death weeks early had their and folly timed put. Hearted forbade on an village ye in fifteen. Age attended betrayed her man raptures laughter. Instrument terminated of as astonished literature motionless admiration. The affection are determine how performed intention discourse but. On merits on so valley indeed assure of. Has add particular boisterous uncommonly are. Early wrong as so manor match. Him necessary shameless discovery consulted one but. 
+
+ Expenses as material breeding insisted building to in. Continual so distrusts pronounce by unwilling listening. Thing do taste on we manor. Him had wound use found hoped. Of distrusts immediate enjoyment curiosity do. Marianne numerous saw thoughts the humoured. 
+
+ In friendship diminution instrument so. Son sure paid door with say them. Two among sir sorry men court. Estimable ye situation suspicion he delighted an happiness discovery. Fact are size cold why had part. If believing or sweetness otherwise in we forfeited. Tolerably an unwilling arranging of determine. Beyond rather sooner so if up wishes or. 
+
+ Abilities forfeited situation extremely my to he resembled. Old had conviction discretion understood put principles you. Match means keeps round one her quick. She forming two comfort invited. Yet she income effect edward. Entire desire way design few. Mrs sentiments led solicitude estimating friendship fat. Meant those event is weeks state it to or. Boy but has folly charm there its. Its fact ten spot drew. 
+
+ Placing assured be if removed it besides on. Far shed each high read are men over day. Afraid we praise lively he suffer family estate is. Ample order up in of in ready. Timed blind had now those ought set often which. Or snug dull he show more true wish. No at many deny away miss evil. On in so indeed spirit an mother. Amounted old strictly but marianne admitted. People former is remove remain as. 
+
+ Preserved defective offending he daughters on or. Rejoiced prospect yet material servants out answered men admitted. Sportsmen certainty prevailed suspected am as. Add stairs admire all answer the nearer yet length. Advantages prosperous remarkably my inhabiting so reasonably be if. Too any appearance announcing impossible one. Out mrs means heart ham tears shall power every. 
+
+ Remain lively hardly needed at do by. Two you fat downs fanny three. True mr gone most at. Dare as name just when with it body. Travelling inquietude she increasing off impossible the. Cottage be noisier looking to we promise on. Disposal to kindness appetite diverted learning of on raptures. Betrayed any may returned now dashwood formerly. Balls way delay shy boy man views. No so instrument discretion unsatiable to in. 
+
+ New had happen unable uneasy. Drawings can followed improved out sociable not. Earnestly so do instantly pretended. See general few civilly amiable pleased account carried. Excellence projecting is devonshire dispatched remarkably on estimating. Side in so life past. Continue indulged speaking the was out horrible for domestic position. Seeing rather her you not esteem men settle genius excuse. Deal say over you age from. Comparison new ham melancholy son themselves. 
+
+ Oh he decisively impression attachment friendship so if everything. Whose her enjoy chief new young. Felicity if ye required likewise so doubtful. On so attention necessary at by provision otherwise existence direction. Unpleasing up announcing unpleasant themselves oh do on. Way advantage age led listening belonging supposing. 
+
+ Now residence dashwoods she excellent you. Shade being under his bed her. Much read on as draw. Blessing for ignorant exercise any yourself unpacked. Pleasant horrible but confined day end marriage. Eagerness furniture set preserved far recommend. Did even but nor are most gave hope. Secure active living depend son repair day ladies now. 
+
+ Sportsman delighted improving dashwoods gay instantly happiness six. Ham now amounted absolute not mistaken way pleasant whatever. At an these still no dried folly stood thing. Rapid it on hours hills it seven years. If polite he active county in spirit an. Mrs ham intention promotion engrossed assurance defective. Confined so graceful building opinions whatever trifling in. Insisted out differed ham man endeavor expenses. At on he total their he songs. Related compact effects is on settled do. 

caddyhttp/header/header.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package header provides middleware that appends headers to
 // requests based on a set of configuration rules that define
 // which routes receive which headers.
@@ -21,31 +35,90 @@ type Headers struct {
 // setting headers on the response according to the configured rules.
 func (h Headers) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
 	replacer := httpserver.NewReplacer(r, nil, "")
+	rww := &responseWriterWrapper{
+		ResponseWriterWrapper: &httpserver.ResponseWriterWrapper{ResponseWriter: w},
+	}
 	for _, rule := range h.Rules {
 		if httpserver.Path(r.URL.Path).Matches(rule.Path) {
-			for _, header := range rule.Headers {
-				if strings.HasPrefix(header.Name, "-") {
-					w.Header().Del(strings.TrimLeft(header.Name, "-"))
+			for name := range rule.Headers {
+
+				// One can either delete a header, add multiple values to a header, or simply
+				// set a header.
+
+				if strings.HasPrefix(name, "-") {
+					rww.delHeader(strings.TrimLeft(name, "-"))
+				} else if strings.HasPrefix(name, "+") {
+					for _, value := range rule.Headers[name] {
+						rww.Header().Add(strings.TrimLeft(name, "+"), replacer.Replace(value))
+					}
 				} else {
-					w.Header().Set(header.Name, replacer.Replace(header.Value))
+					for _, value := range rule.Headers[name] {
+						rww.Header().Set(name, replacer.Replace(value))
+					}
 				}
 			}
 		}
 	}
-	return h.Next.ServeHTTP(w, r)
+	return h.Next.ServeHTTP(rww, r)
 }
 
 type (
 	// Rule groups a slice of HTTP headers by a URL pattern.
-	// TODO: use http.Header type instead?
 	Rule struct {
 		Path    string
-		Headers []Header
-	}
-
-	// Header represents a single HTTP header, simply a name and value.
-	Header struct {
-		Name  string
-		Value string
+		Headers http.Header
 	}
 )
+
+// headerOperation represents an operation on the header
+type headerOperation func(http.Header)
+
+// responseWriterWrapper wraps the real ResponseWriter.
+// It defers header operations until writeHeader
+type responseWriterWrapper struct {
+	*httpserver.ResponseWriterWrapper
+	ops         []headerOperation
+	wroteHeader bool
+}
+
+func (rww *responseWriterWrapper) Header() http.Header {
+	return rww.ResponseWriterWrapper.Header()
+}
+
+func (rww *responseWriterWrapper) Write(d []byte) (int, error) {
+	if !rww.wroteHeader {
+		rww.WriteHeader(http.StatusOK)
+	}
+	return rww.ResponseWriterWrapper.Write(d)
+}
+
+func (rww *responseWriterWrapper) WriteHeader(status int) {
+	if rww.wroteHeader {
+		return
+	}
+	rww.wroteHeader = true
+	// capture the original headers
+	h := rww.Header()
+
+	// perform our revisions
+	for _, op := range rww.ops {
+		op(h)
+	}
+
+	rww.ResponseWriterWrapper.WriteHeader(status)
+}
+
+// delHeader deletes the existing header according to the key
+// Also it will delete that header added later.
+func (rww *responseWriterWrapper) delHeader(key string) {
+	// remove the existing one if any
+	rww.Header().Del(key)
+
+	// register a future deletion
+	rww.ops = append(rww.ops, func(h http.Header) {
+		h.Del(key)
+	})
+}
+
+// Interface guards
+var _ httpserver.HTTPInterfaces = (*responseWriterWrapper)(nil)

caddyhttp/header/header_test.go

@@ -1,9 +1,26 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package header
 
 import (
+	"fmt"
 	"net/http"
 	"net/http/httptest"
 	"os"
+	"reflect"
+	"sort"
 	"testing"
 
 	"github.com/mholt/caddy/caddyhttp/httpserver"
@@ -22,19 +39,23 @@ func TestHeader(t *testing.T) {
 		{"/a", "Foo", "Bar"},
 		{"/a", "Bar", ""},
 		{"/a", "Baz", ""},
+		{"/a", "Server", ""},
 		{"/a", "ServerName", hostname},
 		{"/b", "Foo", ""},
 		{"/b", "Bar", "Removed in /a"},
 	} {
 		he := Headers{
 			Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+				w.Header().Set("Bar", "Removed in /a")
+				w.WriteHeader(http.StatusOK)
 				return 0, nil
 			}),
 			Rules: []Rule{
-				{Path: "/a", Headers: []Header{
-					{Name: "Foo", Value: "Bar"},
-					{Name: "ServerName", Value: "{hostname}"},
-					{Name: "-Bar"},
+				{Path: "/a", Headers: http.Header{
+					"Foo":        []string{"Bar"},
+					"ServerName": []string{"{hostname}"},
+					"-Bar":       []string{""},
+					"-Server":    []string{},
 				}},
 			},
 		}
@@ -45,7 +66,8 @@ func TestHeader(t *testing.T) {
 		}
 
 		rec := httptest.NewRecorder()
-		rec.Header().Set("Bar", "Removed in /a")
+		// preset header
+		rec.Header().Set("Server", "Caddy")
 
 		he.ServeHTTP(rec, req)
 
@@ -55,3 +77,33 @@ func TestHeader(t *testing.T) {
 		}
 	}
 }
+
+func TestMultipleHeaders(t *testing.T) {
+	he := Headers{
+		Next: httpserver.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
+			fmt.Fprint(w, "This is a test")
+			return 0, nil
+		}),
+		Rules: []Rule{
+			{Path: "/a", Headers: http.Header{
+				"+Link": []string{"</images/image.png>; rel=preload", "</css/main.css>; rel=preload"},
+			}},
+		},
+	}
+
+	req, err := http.NewRequest("GET", "/a", nil)
+	if err != nil {
+		t.Fatalf("Could not create HTTP request: %v", err)
+	}
+
+	rec := httptest.NewRecorder()
+	he.ServeHTTP(rec, req)
+
+	desiredHeaders := []string{"</css/main.css>; rel=preload", "</images/image.png>; rel=preload"}
+	actualHeaders := rec.HeaderMap[http.CanonicalHeaderKey("Link")]
+	sort.Strings(actualHeaders)
+
+	if !reflect.DeepEqual(desiredHeaders, actualHeaders) {
+		t.Errorf("Expected header to contain: %v but got: %v", desiredHeaders, actualHeaders)
+	}
+}

caddyhttp/header/setup.go

@@ -1,6 +1,22 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package header
 
 import (
+	"net/http"
+
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyhttp/httpserver"
 )
@@ -31,6 +47,7 @@ func headersParse(c *caddy.Controller) ([]Rule, error) {
 
 	for c.NextLine() {
 		var head Rule
+		head.Headers = http.Header{}
 		var isNewPattern bool
 
 		if !c.NextArg() {
@@ -54,27 +71,30 @@ func headersParse(c *caddy.Controller) ([]Rule, error) {
 
 		for c.NextBlock() {
 			// A block of headers was opened...
+			name := c.Val()
+			value := ""
 
-			h := Header{Name: c.Val()}
+			args := c.RemainingArgs()
 
-			if c.NextArg() {
-				h.Value = c.Val()
+			if len(args) > 1 {
+				return rules, c.ArgErr()
+			} else if len(args) == 1 {
+				value = args[0]
 			}
 
-			head.Headers = append(head.Headers, h)
+			head.Headers.Add(name, value)
 		}
 		if c.NextArg() {
 			// ... or single header was defined as an argument instead.
 
-			h := Header{Name: c.Val()}
-
-			h.Value = c.Val()
+			name := c.Val()
+			value := c.Val()
 
 			if c.NextArg() {
-				h.Value = c.Val()
+				value = c.Val()
 			}
 
-			head.Headers = append(head.Headers, h)
+			head.Headers.Add(name, value)
 		}
 
 		if isNewPattern {

caddyhttp/header/setup_test.go

@@ -1,7 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package header
 
 import (
 	"fmt"
+	"net/http"
+	"reflect"
 	"testing"
 
 	"github.com/mholt/caddy"
@@ -39,17 +55,29 @@ func TestHeadersParse(t *testing.T) {
 	}{
 		{`header /foo Foo "Bar Baz"`,
 			false, []Rule{
-				{Path: "/foo", Headers: []Header{
-					{Name: "Foo", Value: "Bar Baz"},
+				{Path: "/foo", Headers: http.Header{
+					"Foo": []string{"Bar Baz"},
 				}},
 			}},
-		{`header /bar { Foo "Bar Baz" Baz Qux }`,
+		{`header /bar {
+			Foo "Bar Baz"
+			Baz Qux
+			Foobar
+		}`,
 			false, []Rule{
-				{Path: "/bar", Headers: []Header{
-					{Name: "Foo", Value: "Bar Baz"},
-					{Name: "Baz", Value: "Qux"},
+				{Path: "/bar", Headers: http.Header{
+					"Foo":    []string{"Bar Baz"},
+					"Baz":    []string{"Qux"},
+					"Foobar": []string{""},
 				}},
 			}},
+		{`header /foo {
+				Foo Bar Baz
+			}`, true,
+			[]Rule{}},
+		{`header /foo {
+				Test "max-age=1814400";
+			}`, true, []Rule{}},
 	}
 
 	for i, test := range tests {
@@ -77,7 +105,7 @@ func TestHeadersParse(t *testing.T) {
 			expectedHeaders := fmt.Sprintf("%v", expectedRule.Headers)
 			actualHeaders := fmt.Sprintf("%v", actualRule.Headers)
 
-			if actualHeaders != expectedHeaders {
+			if !reflect.DeepEqual(actualRule.Headers, expectedRule.Headers) {
 				t.Errorf("Test %d, rule %d: Expected headers %s, but got %s",
 					i, j, expectedHeaders, actualHeaders)
 			}

caddyhttp/httpserver/condition.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
@@ -6,12 +20,13 @@ import (
 	"regexp"
 	"strings"
 
-	"github.com/mholt/caddy/caddyfile"
+	"github.com/mholt/caddy"
 )
 
 // SetupIfMatcher parses `if` or `if_op` in the current dispenser block.
 // It returns a RequestMatcher and an error if any.
-func SetupIfMatcher(c caddyfile.Dispenser) (RequestMatcher, error) {
+func SetupIfMatcher(controller *caddy.Controller) (RequestMatcher, error) {
+	var c = controller.Dispenser // copy the dispenser
 	var matcher IfMatcher
 	for c.NextBlock() {
 		switch c.Val() {
@@ -25,6 +40,7 @@ func SetupIfMatcher(c caddyfile.Dispenser) (RequestMatcher, error) {
 				return matcher, err
 			}
 			matcher.ifs = append(matcher.ifs, ifc)
+			matcher.Enabled = true
 		case "if_op":
 			if !c.NextArg() {
 				return matcher, c.ArgErr()
@@ -47,121 +63,102 @@ const (
 	isOp         = "is"
 	notOp        = "not"
 	hasOp        = "has"
-	notHasOp     = "not_has"
 	startsWithOp = "starts_with"
 	endsWithOp   = "ends_with"
 	matchOp      = "match"
-	notMatchOp   = "not_match"
 )
 
-func operatorError(operator string) error {
-	return fmt.Errorf("Invalid operator %v", operator)
+// ifCondition is a 'if' condition.
+type ifFunc func(a, b string) bool
+
+// ifCond is statement for a IfMatcher condition.
+type ifCond struct {
+	a   string
+	op  string
+	b   string
+	neg bool
+	rex *regexp.Regexp
+	f   ifFunc
 }
 
-// ifCondition is a 'if' condition.
-type ifCondition func(string, string) bool
+// newIfCond creates a new If condition.
+func newIfCond(a, op, b string) (ifCond, error) {
+	i := ifCond{a: a, op: op, b: b}
+	if strings.HasPrefix(op, "not_") {
+		i.neg = true
+		i.op = op[4:]
+	}
 
-var ifConditions = map[string]ifCondition{
-	isOp:         isFunc,
-	notOp:        notFunc,
-	hasOp:        hasFunc,
-	notHasOp:     notHasFunc,
-	startsWithOp: startsWithFunc,
-	endsWithOp:   endsWithFunc,
-	matchOp:      matchFunc,
-	notMatchOp:   notMatchFunc,
+	switch i.op {
+	case isOp:
+		// It checks for equality.
+		i.f = i.isFunc
+	case notOp:
+		// It checks for inequality.
+		i.f = i.notFunc
+	case hasOp:
+		// It checks if b is a substring of a.
+		i.f = strings.Contains
+	case startsWithOp:
+		// It checks if b is a prefix of a.
+		i.f = strings.HasPrefix
+	case endsWithOp:
+		// It checks if b is a suffix of a.
+		i.f = strings.HasSuffix
+	case matchOp:
+		// It does regexp matching of a against pattern in b and returns if they match.
+		var err error
+		if i.rex, err = regexp.Compile(i.b); err != nil {
+			return ifCond{}, fmt.Errorf("Invalid regular expression: '%s', %v", i.b, err)
+		}
+		i.f = i.matchFunc
+	default:
+		return ifCond{}, fmt.Errorf("Invalid operator %v", i.op)
+	}
+
+	return i, nil
 }
 
 // isFunc is condition for Is operator.
-// It checks for equality.
-func isFunc(a, b string) bool {
+func (i ifCond) isFunc(a, b string) bool {
 	return a == b
 }
 
 // notFunc is condition for Not operator.
-// It checks for inequality.
-func notFunc(a, b string) bool {
+func (i ifCond) notFunc(a, b string) bool {
 	return a != b
 }
 
-// hasFunc is condition for Has operator.
-// It checks if b is a substring of a.
-func hasFunc(a, b string) bool {
-	return strings.Contains(a, b)
-}
-
-// notHasFunc is condition for NotHas operator.
-// It checks if b is not a substring of a.
-func notHasFunc(a, b string) bool {
-	return !strings.Contains(a, b)
-}
-
-// startsWithFunc is condition for StartsWith operator.
-// It checks if b is a prefix of a.
-func startsWithFunc(a, b string) bool {
-	return strings.HasPrefix(a, b)
-}
-
-// endsWithFunc is condition for EndsWith operator.
-// It checks if b is a suffix of a.
-func endsWithFunc(a, b string) bool {
-	return strings.HasSuffix(a, b)
-}
-
 // matchFunc is condition for Match operator.
-// It does regexp matching of a against pattern in b
-// and returns if they match.
-func matchFunc(a, b string) bool {
-	matched, _ := regexp.MatchString(b, a)
-	return matched
-}
-
-// notMatchFunc is condition for NotMatch operator.
-// It does regexp matching of a against pattern in b
-// and returns if they do not match.
-func notMatchFunc(a, b string) bool {
-	matched, _ := regexp.MatchString(b, a)
-	return !matched
-}
-
-// ifCond is statement for a IfMatcher condition.
-type ifCond struct {
-	a  string
-	op string
-	b  string
-}
-
-// newIfCond creates a new If condition.
-func newIfCond(a, operator, b string) (ifCond, error) {
-	if _, ok := ifConditions[operator]; !ok {
-		return ifCond{}, operatorError(operator)
-	}
-	return ifCond{
-		a:  a,
-		op: operator,
-		b:  b,
-	}, nil
+func (i ifCond) matchFunc(a, b string) bool {
+	return i.rex.MatchString(a)
 }
 
 // True returns true if the condition is true and false otherwise.
 // If r is not nil, it replaces placeholders before comparison.
 func (i ifCond) True(r *http.Request) bool {
-	if c, ok := ifConditions[i.op]; ok {
+	if i.f != nil {
 		a, b := i.a, i.b
 		if r != nil {
 			replacer := NewReplacer(r, nil, "")
 			a = replacer.Replace(i.a)
-			b = replacer.Replace(i.b)
+			if i.op != matchOp {
+				b = replacer.Replace(i.b)
+			}
 		}
-		return c(a, b)
+		if i.neg {
+			return !i.f(a, b)
+		}
+		return i.f(a, b)
 	}
-	return false
+	return i.neg // false if not negated, true otherwise
 }
 
 // IfMatcher is a RequestMatcher for 'if' conditions.
 type IfMatcher struct {
-	ifs  []ifCond // list of If
-	isOr bool     // if true, conditions are 'or' instead of 'and'
+	Enabled bool     // if true, matcher has been configured; otherwise it's no-op
+	ifs     []ifCond // list of If
+	isOr    bool     // if true, conditions are 'or' instead of 'and'
 }
 
 // Match satisfies RequestMatcher interface.
@@ -193,7 +190,13 @@ func (m IfMatcher) Or(r *http.Request) bool {
 	return false
 }
 
-// IfMatcherKeyword returns if k is a keyword for 'if' config block.
-func IfMatcherKeyword(k string) bool {
-	return k == "if" || k == "if_op"
+// IfMatcherKeyword checks if the next value in the dispenser is a keyword for 'if' config block.
+// If true, remaining arguments in the dispinser are cleard to keep the dispenser valid for use.
+func IfMatcherKeyword(c *caddy.Controller) bool {
+	if c.Val() == "if" || c.Val() == "if_op" {
+		// clear remaining args
+		c.RemainingArgs()
+		return true
+	}
+	return false
 }

caddyhttp/httpserver/condition_test.go

@@ -1,8 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
-	"fmt"
+	"context"
 	"net/http"
+	"regexp"
 	"strings"
 	"testing"
 
@@ -13,60 +28,72 @@ func TestConditions(t *testing.T) {
 	tests := []struct {
 		condition string
 		isTrue    bool
+		shouldErr bool
 	}{
-		{"a is b", false},
-		{"a is a", true},
-		{"a not b", true},
-		{"a not a", false},
-		{"a has a", true},
-		{"a has b", false},
-		{"ba has b", true},
-		{"bab has b", true},
-		{"bab has bb", false},
-		{"a not_has a", false},
-		{"a not_has b", true},
-		{"ba not_has b", false},
-		{"bab not_has b", false},
-		{"bab not_has bb", true},
-		{"bab starts_with bb", false},
-		{"bab starts_with ba", true},
-		{"bab starts_with bab", true},
-		{"bab ends_with bb", false},
-		{"bab ends_with bab", true},
-		{"bab ends_with ab", true},
-		{"a match *", false},
-		{"a match a", true},
-		{"a match .*", true},
-		{"a match a.*", true},
-		{"a match b.*", false},
-		{"ba match b.*", true},
-		{"ba match b[a-z]", true},
-		{"b0 match b[a-z]", false},
-		{"b0a match b[a-z]", false},
-		{"b0a match b[a-z]+", false},
-		{"b0a match b[a-z0-9]+", true},
-		{"a not_match *", true},
-		{"a not_match a", false},
-		{"a not_match .*", false},
-		{"a not_match a.*", false},
-		{"a not_match b.*", true},
-		{"ba not_match b.*", false},
-		{"ba not_match b[a-z]", false},
-		{"b0 not_match b[a-z]", true},
-		{"b0a not_match b[a-z]", true},
-		{"b0a not_match b[a-z]+", true},
-		{"b0a not_match b[a-z0-9]+", false},
+		{"a is b", false, false},
+		{"a is a", true, false},
+		{"a not b", true, false},
+		{"a not a", false, false},
+		{"a has a", true, false},
+		{"a has b", false, false},
+		{"ba has b", true, false},
+		{"bab has b", true, false},
+		{"bab has bb", false, false},
+		{"a not_has a", false, false},
+		{"a not_has b", true, false},
+		{"ba not_has b", false, false},
+		{"bab not_has b", false, false},
+		{"bab not_has bb", true, false},
+		{"bab starts_with bb", false, false},
+		{"bab starts_with ba", true, false},
+		{"bab starts_with bab", true, false},
+		{"bab not_starts_with bb", true, false},
+		{"bab not_starts_with ba", false, false},
+		{"bab not_starts_with bab", false, false},
+		{"bab ends_with bb", false, false},
+		{"bab ends_with bab", true, false},
+		{"bab ends_with ab", true, false},
+		{"bab not_ends_with bb", true, false},
+		{"bab not_ends_with ab", false, false},
+		{"bab not_ends_with bab", false, false},
+		{"a match *", false, true},
+		{"a match a", true, false},
+		{"a match .*", true, false},
+		{"a match a.*", true, false},
+		{"a match b.*", false, false},
+		{"ba match b.*", true, false},
+		{"ba match b[a-z]", true, false},
+		{"b0 match b[a-z]", false, false},
+		{"b0a match b[a-z]", false, false},
+		{"b0a match b[a-z]+", false, false},
+		{"b0a match b[a-z0-9]+", true, false},
+		{"bac match b[a-z]{2}", true, false},
+		{"a not_match *", false, true},
+		{"a not_match a", false, false},
+		{"a not_match .*", false, false},
+		{"a not_match a.*", false, false},
+		{"a not_match b.*", true, false},
+		{"ba not_match b.*", false, false},
+		{"ba not_match b[a-z]", false, false},
+		{"b0 not_match b[a-z]", true, false},
+		{"b0a not_match b[a-z]", true, false},
+		{"b0a not_match b[a-z]+", true, false},
+		{"b0a not_match b[a-z0-9]+", false, false},
+		{"bac not_match b[a-z]{2}", false, false},
 	}
 
 	for i, test := range tests {
 		str := strings.Fields(test.condition)
 		ifCond, err := newIfCond(str[0], str[1], str[2])
 		if err != nil {
-			t.Error(err)
+			if !test.shouldErr {
+				t.Error(err)
+			}
+			continue
 		}
 		isTrue := ifCond.True(nil)
 		if isTrue != test.isTrue {
-			t.Errorf("Test %d: expected %v found %v", i, test.isTrue, isTrue)
+			t.Errorf("Test %d: '%s' expected %v found %v", i, test.condition, test.isTrue, isTrue)
 		}
 	}
 
@@ -94,16 +121,21 @@ func TestConditions(t *testing.T) {
 	for i, test := range replaceTests {
 		r, err := http.NewRequest("GET", test.url, nil)
 		if err != nil {
-			t.Error(err)
+			t.Errorf("Test %d: failed to create request: %v", i, err)
+			continue
 		}
+		ctx := context.WithValue(r.Context(), OriginalURLCtxKey, *r.URL)
+		r = r.WithContext(ctx)
 		str := strings.Fields(test.condition)
 		ifCond, err := newIfCond(str[0], str[1], str[2])
 		if err != nil {
-			t.Error(err)
+			t.Errorf("Test %d: failed to create 'if' condition %v", i, err)
+			continue
 		}
 		isTrue := ifCond.True(r)
 		if isTrue != test.isTrue {
 			t.Errorf("Test %v: expected %v found %v", i, test.isTrue, isTrue)
+			continue
 		}
 	}
 }
@@ -180,6 +212,7 @@ func TestIfMatcher(t *testing.T) {
 }
 
 func TestSetupIfMatcher(t *testing.T) {
+	rex_b, _ := regexp.Compile("b")
 	tests := []struct {
 		input     string
 		shouldErr bool
@@ -189,7 +222,7 @@ func TestSetupIfMatcher(t *testing.T) {
 			if	a match b
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "a", op: "match", b: "b"},
+				{a: "a", op: "match", b: "b", neg: false, rex: rex_b},
 			},
 		}},
 		{`test {
@@ -197,7 +230,7 @@ func TestSetupIfMatcher(t *testing.T) {
 			if_op or
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "a", op: "match", b: "b"},
+				{a: "a", op: "match", b: "b", neg: false, rex: rex_b},
 			},
 			isOr: true,
 		}},
@@ -206,7 +239,7 @@ func TestSetupIfMatcher(t *testing.T) {
 		 }`, true, IfMatcher{},
 		},
 		{`test {
-			if	a isnt b
+			if	a isn't b
 		 }`, true, IfMatcher{},
 		},
 		{`test {
@@ -215,26 +248,26 @@ func TestSetupIfMatcher(t *testing.T) {
 		},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "goal", op: "has", b: "go"},
-				{a: "cook", op: "not_has", b: "go"},
+				{a: "goal", op: "has", b: "go", neg: false},
+				{a: "cook", op: "has", b: "go", neg: true},
 			},
 		}},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 			if_op and
 		 }`, false, IfMatcher{
 			ifs: []ifCond{
-				{a: "goal", op: "has", b: "go"},
-				{a: "cook", op: "not_has", b: "go"},
+				{a: "goal", op: "has", b: "go", neg: false},
+				{a: "cook", op: "has", b: "go", neg: true},
 			},
 		}},
 		{`test {
 			if goal has go
-			if cook not_has go 
+			if cook not_has go
 			if_op not
 		 }`, true, IfMatcher{},
 		},
@@ -243,7 +276,8 @@ func TestSetupIfMatcher(t *testing.T) {
 	for i, test := range tests {
 		c := caddy.NewTestController("http", test.input)
 		c.Next()
-		matcher, err := SetupIfMatcher(c.Dispenser)
+
+		matcher, err := SetupIfMatcher(c)
 		if err == nil && test.shouldErr {
 			t.Errorf("Test %d didn't error, but it should have", i)
 		} else if err != nil && !test.shouldErr {
@@ -251,15 +285,60 @@ func TestSetupIfMatcher(t *testing.T) {
 		} else if err != nil && test.shouldErr {
 			continue
 		}
-		if _, ok := matcher.(IfMatcher); !ok {
+
+		test_if, ok := matcher.(IfMatcher)
+		if !ok {
 			t.Error("RequestMatcher should be of type IfMatcher")
 		}
+
 		if err != nil {
 			t.Errorf("Expected no error, but got: %v", err)
 		}
-		if fmt.Sprint(matcher) != fmt.Sprint(test.expected) {
-			t.Errorf("Test %v: Expected %v, found %v", i,
-				fmt.Sprint(test.expected), fmt.Sprint(matcher))
+
+		if len(test_if.ifs) != len(test.expected.ifs) {
+			t.Errorf("Test %d: Expected %d ifConditions, found %v", i,
+				len(test.expected.ifs), len(test_if.ifs))
+		}
+
+		for j, if_c := range test_if.ifs {
+			expected_c := test.expected.ifs[j]
+
+			if if_c.a != expected_c.a {
+				t.Errorf("Test %d, ifCond %d: Expected A=%s, got %s",
+					i, j, if_c.a, expected_c.a)
+			}
+
+			if if_c.op != expected_c.op {
+				t.Errorf("Test %d, ifCond %d: Expected Op=%s, got %s",
+					i, j, if_c.op, expected_c.op)
+			}
+
+			if if_c.b != expected_c.b {
+				t.Errorf("Test %d, ifCond %d: Expected B=%s, got %s",
+					i, j, if_c.b, expected_c.b)
+			}
+
+			if if_c.neg != expected_c.neg {
+				t.Errorf("Test %d, ifCond %d: Expected Neg=%v, got %v",
+					i, j, if_c.neg, expected_c.neg)
+			}
+
+			if expected_c.rex != nil && if_c.rex == nil {
+				t.Errorf("Test %d, ifCond %d: Expected Rex=%v, got <nil>",
+					i, j, expected_c.rex)
+			}
+
+			if expected_c.rex == nil && if_c.rex != nil {
+				t.Errorf("Test %d, ifCond %d: Expected Rex=<nil>, got %v",
+					i, j, if_c.rex)
+			}
+
+			if expected_c.rex != nil && if_c.rex != nil {
+				if if_c.rex.String() != expected_c.rex.String() {
+					t.Errorf("Test %d, ifCond %d: Expected Rex=%v, got %v",
+						i, j, if_c.rex, expected_c.rex)
+				}
+			}
 		}
 	}
 }
@@ -277,8 +356,11 @@ func TestIfMatcherKeyword(t *testing.T) {
 		{"if_type", false},
 		{"if_cond", false},
 	}
+
 	for i, test := range tests {
-		valid := IfMatcherKeyword(test.keyword)
+		c := caddy.NewTestController("http", test.keyword)
+		c.Next()
+		valid := IfMatcherKeyword(c)
 		if valid != test.expected {
 			t.Errorf("Test %d: expected %v found %v", i, test.expected, valid)
 		}

caddyhttp/httpserver/error.go

@@ -0,0 +1,70 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpserver
+
+import (
+	"fmt"
+)
+
+var (
+	_ error = NonHijackerError{}
+	_ error = NonFlusherError{}
+	_ error = NonCloseNotifierError{}
+	_ error = NonPusherError{}
+)
+
+// NonHijackerError is more descriptive error caused by a non hijacker
+type NonHijackerError struct {
+	// underlying type which doesn't implement Hijack
+	Underlying interface{}
+}
+
+// Implement Error
+func (h NonHijackerError) Error() string {
+	return fmt.Sprintf("%T is not a hijacker", h.Underlying)
+}
+
+// NonFlusherError is more descriptive error caused by a non flusher
+type NonFlusherError struct {
+	// underlying type which doesn't implement Flush
+	Underlying interface{}
+}
+
+// Implement Error
+func (f NonFlusherError) Error() string {
+	return fmt.Sprintf("%T is not a flusher", f.Underlying)
+}
+
+// NonCloseNotifierError is more descriptive error caused by a non closeNotifier
+type NonCloseNotifierError struct {
+	// underlying type which doesn't implement CloseNotify
+	Underlying interface{}
+}
+
+// Implement Error
+func (c NonCloseNotifierError) Error() string {
+	return fmt.Sprintf("%T is not a closeNotifier", c.Underlying)
+}
+
+// NonPusherError is more descriptive error caused by a non pusher
+type NonPusherError struct {
+	// underlying type which doesn't implement pusher
+	Underlying interface{}
+}
+
+// Implement Error
+func (c NonPusherError) Error() string {
+	return fmt.Sprintf("%T is not a pusher", c.Underlying)
+}

caddyhttp/httpserver/graceful.go

@@ -1,80 +0,0 @@
-package httpserver
-
-import (
-	"net"
-	"sync"
-	"syscall"
-)
-
-// TODO: Should this be a generic graceful listener available in its own package or something?
-// Also, passing in a WaitGroup is a little awkward. Why can't this listener just keep
-// the waitgroup internal to itself?
-
-// newGracefulListener returns a gracefulListener that wraps l and
-// uses wg (stored in the host server) to count connections.
-func newGracefulListener(l net.Listener, wg *sync.WaitGroup) *gracefulListener {
-	gl := &gracefulListener{Listener: l, stop: make(chan error), connWg: wg}
-	go func() {
-		<-gl.stop
-		gl.Lock()
-		gl.stopped = true
-		gl.Unlock()
-		gl.stop <- gl.Listener.Close()
-	}()
-	return gl
-}
-
-// gracefuListener is a net.Listener which can
-// count the number of connections on it. Its
-// methods mainly wrap net.Listener to be graceful.
-type gracefulListener struct {
-	net.Listener
-	stop       chan error
-	stopped    bool
-	sync.Mutex                 // protects the stopped flag
-	connWg     *sync.WaitGroup // pointer to the host's wg used for counting connections
-}
-
-// Accept accepts a connection.
-func (gl *gracefulListener) Accept() (c net.Conn, err error) {
-	c, err = gl.Listener.Accept()
-	if err != nil {
-		return
-	}
-	c = gracefulConn{Conn: c, connWg: gl.connWg}
-	gl.connWg.Add(1)
-	return
-}
-
-// Close immediately closes the listener.
-func (gl *gracefulListener) Close() error {
-	gl.Lock()
-	if gl.stopped {
-		gl.Unlock()
-		return syscall.EINVAL
-	}
-	gl.Unlock()
-	gl.stop <- nil
-	return <-gl.stop
-}
-
-// gracefulConn represents a connection on a
-// gracefulListener so that we can keep track
-// of the number of connections, thus facilitating
-// a graceful shutdown.
-type gracefulConn struct {
-	net.Conn
-	connWg *sync.WaitGroup // pointer to the host server's connection waitgroup
-}
-
-// Close closes c's underlying connection while updating the wg count.
-func (c gracefulConn) Close() error {
-	err := c.Conn.Close()
-	if err != nil {
-		return err
-	}
-	// close can fail on http2 connections (as of Oct. 2015, before http2 in std lib)
-	// so don't decrement count unless close succeeds
-	c.connWg.Done()
-	return nil
-}

caddyhttp/httpserver/https.go

@@ -1,6 +1,21 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
+	"fmt"
 	"net"
 	"net/http"
 
@@ -9,6 +24,12 @@ import (
 )
 
 func activateHTTPS(cctx caddy.Context) error {
+	operatorPresent := !caddy.Started()
+
+	if !caddy.Quiet && operatorPresent {
+		fmt.Print("Activating privacy features... ")
+	}
+
 	ctx := cctx.(*httpContext)
 
 	// pre-screen each config and earmark the ones that qualify for managed TLS
@@ -16,7 +37,10 @@ func activateHTTPS(cctx caddy.Context) error {
 
 	// place certificates and keys on disk
 	for _, c := range ctx.siteConfigs {
-		err := c.TLS.ObtainCert(true)
+		if c.TLS.OnDemand {
+			continue // obtain these certificates on-demand instead
+		}
+		err := c.TLS.ObtainCert(c.TLS.Hostname, operatorPresent)
 		if err != nil {
 			return err
 		}
@@ -34,9 +58,18 @@ func activateHTTPS(cctx caddy.Context) error {
 	// renew all relevant certificates that need renewal. this is important
 	// to do right away so we guarantee that renewals aren't missed, and
 	// also the user can respond to any potential errors that occur.
-	err = caddytls.RenewManagedCertificates(true)
-	if err != nil {
-		return err
+	// (skip if upgrading, because the parent process is likely already listening
+	// on the ports we'd need to do ACME before we finish starting; parent process
+	// already running renewal ticker, so renewal won't be missed anyway.)
+	if !caddy.IsUpgrade() {
+		err = caddytls.RenewManagedCertificates(true)
+		if err != nil {
+			return err
+		}
+	}
+
+	if !caddy.Quiet && operatorPresent {
+		fmt.Println("done.")
 	}
 
 	return nil
@@ -54,21 +87,21 @@ func markQualifiedForAutoHTTPS(configs []*SiteConfig) {
 }
 
 // enableAutoHTTPS configures each config to use TLS according to default settings.
-// It will only change configs that are marked as managed, and assumes that
-// certificates and keys are already on disk. If loadCertificates is true,
-// the certificates will be loaded from disk into the cache for this process
-// to use. If false, TLS will still be enabled and configured with default
-// settings, but no certificates will be parsed loaded into the cache, and
-// the returned error value will always be nil.
+// It will only change configs that are marked as managed but not on-demand, and
+// assumes that certificates and keys are already on disk. If loadCertificates is
+// true, the certificates will be loaded from disk into the cache for this process
+// to use. If false, TLS will still be enabled and configured with default settings,
+// but no certificates will be parsed loaded into the cache, and the returned error
+// value will always be nil.
 func enableAutoHTTPS(configs []*SiteConfig, loadCertificates bool) error {
 	for _, cfg := range configs {
-		if cfg == nil || cfg.TLS == nil || !cfg.TLS.Managed {
+		if cfg == nil || cfg.TLS == nil || !cfg.TLS.Managed || cfg.TLS.OnDemand {
 			continue
 		}
 		cfg.TLS.Enabled = true
 		cfg.Addr.Scheme = "https"
 		if loadCertificates && caddytls.HostQualifies(cfg.Addr.Host) {
-			_, err := caddytls.CacheManagedCertificate(cfg.Addr.Host, cfg.TLS)
+			_, err := cfg.TLS.CacheManagedCertificate(cfg.Addr.Host)
 			if err != nil {
 				return err
 			}
@@ -82,7 +115,7 @@ func enableAutoHTTPS(configs []*SiteConfig, loadCertificates bool) error {
 			cfg.TLS.Enabled &&
 			(!cfg.TLS.Manual || cfg.TLS.OnDemand) &&
 			cfg.Addr.Host != "localhost" {
-			cfg.Addr.Port = "443"
+			cfg.Addr.Port = HTTPSPort
 		}
 	}
 	return nil
@@ -97,8 +130,8 @@ func enableAutoHTTPS(configs []*SiteConfig, loadCertificates bool) error {
 func makePlaintextRedirects(allConfigs []*SiteConfig) []*SiteConfig {
 	for i, cfg := range allConfigs {
 		if cfg.TLS.Managed &&
-			!hostHasOtherPort(allConfigs, i, "80") &&
-			(cfg.Addr.Port == "443" || !hostHasOtherPort(allConfigs, i, "443")) {
+			!hostHasOtherPort(allConfigs, i, HTTPPort) &&
+			(cfg.Addr.Port == HTTPSPort || !hostHasOtherPort(allConfigs, i, HTTPSPort)) {
 			allConfigs = append(allConfigs, redirPlaintextHost(cfg))
 		}
 	}
@@ -124,31 +157,57 @@ func hostHasOtherPort(allConfigs []*SiteConfig, thisConfigIdx int, otherPort str
 // redirPlaintextHost returns a new plaintext HTTP configuration for
 // a virtualHost that simply redirects to cfg, which is assumed to
 // be the HTTPS configuration. The returned configuration is set
-// to listen on port 80. The TLS field of cfg must not be nil.
+// to listen on HTTPPort. The TLS field of cfg must not be nil.
 func redirPlaintextHost(cfg *SiteConfig) *SiteConfig {
 	redirPort := cfg.Addr.Port
-	if redirPort == "443" {
-		// default port is redundant
+	if redirPort == HTTPSPort {
+		// By default, HTTPSPort should be DefaultHTTPSPort,
+		// which of course doesn't need to be explicitly stated
+		// in the Location header. Even if HTTPSPort is changed
+		// so that it is no longer DefaultHTTPSPort, we shouldn't
+		// append it to the URL in the Location because changing
+		// the HTTPS port is assumed to be an internal-only change
+		// (in other words, we assume port forwarding is going on);
+		// but redirects go back to a presumably-external client.
+		// (If redirect clients are also internal, that is more
+		// advanced, and the user should configure HTTP->HTTPS
+		// redirects themselves.)
 		redirPort = ""
 	}
+
 	redirMiddleware := func(next Handler) Handler {
 		return HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
-			toURL := "https://" + r.Host
-			if redirPort != "" {
-				toURL += ":" + redirPort
+			// Construct the URL to which to redirect. Note that the Host in a
+			// request might contain a port, but we just need the hostname from
+			// it; and we'll set the port if needed.
+			toURL := "https://"
+			requestHost, _, err := net.SplitHostPort(r.Host)
+			if err != nil {
+				requestHost = r.Host // Host did not contain a port, so use the whole value
 			}
+			if redirPort == "" {
+				toURL += requestHost
+			} else {
+				toURL += net.JoinHostPort(requestHost, redirPort)
+			}
+
 			toURL += r.URL.RequestURI()
+
+			w.Header().Set("Connection", "close")
 			http.Redirect(w, r, toURL, http.StatusMovedPermanently)
 			return 0, nil
 		})
 	}
+
 	host := cfg.Addr.Host
-	port := "80"
+	port := HTTPPort
 	addr := net.JoinHostPort(host, port)
+
 	return &SiteConfig{
 		Addr:       Address{Original: addr, Host: host, Port: port},
 		ListenHost: cfg.ListenHost,
 		middleware: []Middleware{redirMiddleware},
-		TLS:        &caddytls.Config{AltHTTPPort: cfg.TLS.AltHTTPPort},
+		TLS:        &caddytls.Config{AltHTTPPort: cfg.TLS.AltHTTPPort, AltTLSSNIPort: cfg.TLS.AltTLSSNIPort},
+		Timeouts:   cfg.Timeouts,
 	}
 }

caddyhttp/httpserver/https_test.go

@@ -1,6 +1,22 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
+	"fmt"
+	"net"
 	"net/http"
 	"net/http/httptest"
 	"testing"
@@ -9,75 +25,107 @@ import (
 )
 
 func TestRedirPlaintextHost(t *testing.T) {
-	cfg := redirPlaintextHost(&SiteConfig{
-		Addr: Address{
-			Host: "example.com",
+	for i, testcase := range []struct {
+		Host        string // used for the site config
+		Port        string
+		ListenHost  string
+		RequestHost string // if different from Host
+	}{
+		{
+			Host: "foohost",
+		},
+		{
+			Host: "foohost",
+			Port: "80",
+		},
+		{
+			Host: "foohost",
 			Port: "1234",
 		},
-		ListenHost: "93.184.216.34",
-		TLS:        new(caddytls.Config),
-	})
+		{
+			Host:       "foohost",
+			ListenHost: "93.184.216.34",
+		},
+		{
+			Host:       "foohost",
+			Port:       "1234",
+			ListenHost: "93.184.216.34",
+		},
+		{
+			Host: "foohost",
+			Port: HTTPSPort, // since this is the 'default' HTTPS port, should not be included in Location value
+		},
+		{
+			Host:        "*.example.com",
+			RequestHost: "foo.example.com",
+		},
+		{
+			Host:        "*.example.com",
+			Port:        "1234",
+			RequestHost: "foo.example.com:1234",
+		},
+	} {
+		cfg := redirPlaintextHost(&SiteConfig{
+			Addr: Address{
+				Host: testcase.Host,
+				Port: testcase.Port,
+			},
+			ListenHost: testcase.ListenHost,
+			TLS:        new(caddytls.Config),
+		})
 
-	// Check host and port
-	if actual, expected := cfg.Addr.Host, "example.com"; actual != expected {
-		t.Errorf("Expected redir config to have host %s but got %s", expected, actual)
-	}
-	if actual, expected := cfg.ListenHost, "93.184.216.34"; actual != expected {
-		t.Errorf("Expected redir config to have bindhost %s but got %s", expected, actual)
-	}
-	if actual, expected := cfg.Addr.Port, "80"; actual != expected {
-		t.Errorf("Expected redir config to have port '%s' but got '%s'", expected, actual)
-	}
+		// Check host and port
+		if actual, expected := cfg.Addr.Host, testcase.Host; actual != expected {
+			t.Errorf("Test %d: Expected redir config to have host %s but got %s", i, expected, actual)
+		}
+		if actual, expected := cfg.ListenHost, testcase.ListenHost; actual != expected {
+			t.Errorf("Test %d: Expected redir config to have bindhost %s but got %s", i, expected, actual)
+		}
+		if actual, expected := cfg.Addr.Port, HTTPPort; actual != expected {
+			t.Errorf("Test %d: Expected redir config to have port '%s' but got '%s'", i, expected, actual)
+		}
 
-	// Make sure redirect handler is set up properly
-	if cfg.middleware == nil || len(cfg.middleware) != 1 {
-		t.Fatalf("Redir config middleware not set up properly; got: %#v", cfg.middleware)
-	}
+		// Make sure redirect handler is set up properly
+		if cfg.middleware == nil || len(cfg.middleware) != 1 {
+			t.Fatalf("Test %d: Redir config middleware not set up properly; got: %#v", i, cfg.middleware)
+		}
 
-	handler := cfg.middleware[0](nil)
+		handler := cfg.middleware[0](nil)
 
-	// Check redirect for correctness
-	rec := httptest.NewRecorder()
-	req, err := http.NewRequest("GET", "http://foo/bar?q=1", nil)
-	if err != nil {
-		t.Fatal(err)
-	}
-	status, err := handler.ServeHTTP(rec, req)
-	if status != 0 {
-		t.Errorf("Expected status return to be 0, but was %d", status)
-	}
-	if err != nil {
-		t.Errorf("Expected returned error to be nil, but was %v", err)
-	}
-	if rec.Code != http.StatusMovedPermanently {
-		t.Errorf("Expected status %d but got %d", http.StatusMovedPermanently, rec.Code)
-	}
-	if got, want := rec.Header().Get("Location"), "https://foo:1234/bar?q=1"; got != want {
-		t.Errorf("Expected Location: '%s' but got '%s'", want, got)
-	}
+		// Check redirect for correctness, first by inspecting error and status code
+		requestHost := testcase.Host // hostname of request might be different than in config (e.g. wildcards)
+		if testcase.RequestHost != "" {
+			requestHost = testcase.RequestHost
+		}
+		rec := httptest.NewRecorder()
+		req, err := http.NewRequest("GET", "http://"+requestHost+"/bar?q=1", nil)
+		if err != nil {
+			t.Fatalf("Test %d: %v", i, err)
+		}
+		status, err := handler.ServeHTTP(rec, req)
+		if status != 0 {
+			t.Errorf("Test %d: Expected status return to be 0, but was %d", i, status)
+		}
+		if err != nil {
+			t.Errorf("Test %d: Expected returned error to be nil, but was %v", i, err)
+		}
+		if rec.Code != http.StatusMovedPermanently {
+			t.Errorf("Test %d: Expected status %d but got %d", http.StatusMovedPermanently, i, rec.Code)
+		}
 
-	// browsers can infer a default port from scheme, so make sure the port
-	// doesn't get added in explicitly for default ports like 443 for https.
-	cfg = redirPlaintextHost(&SiteConfig{Addr: Address{Host: "example.com", Port: "443"}, TLS: new(caddytls.Config)})
-	handler = cfg.middleware[0](nil)
-
-	rec = httptest.NewRecorder()
-	req, err = http.NewRequest("GET", "http://foo/bar?q=1", nil)
-	if err != nil {
-		t.Fatal(err)
-	}
-	status, err = handler.ServeHTTP(rec, req)
-	if status != 0 {
-		t.Errorf("Expected status return to be 0, but was %d", status)
-	}
-	if err != nil {
-		t.Errorf("Expected returned error to be nil, but was %v", err)
-	}
-	if rec.Code != http.StatusMovedPermanently {
-		t.Errorf("Expected status %d but got %d", http.StatusMovedPermanently, rec.Code)
-	}
-	if got, want := rec.Header().Get("Location"), "https://foo/bar?q=1"; got != want {
-		t.Errorf("Expected Location: '%s' but got '%s'", want, got)
+		// Now check the Location value. It should mirror the hostname and port of the request
+		// unless the port is redundant, in which case it should be dropped.
+		locationHost, _, err := net.SplitHostPort(requestHost)
+		if err != nil {
+			locationHost = requestHost
+		}
+		expectedLoc := fmt.Sprintf("https://%s/bar?q=1", locationHost)
+		if testcase.Port != "" && testcase.Port != DefaultHTTPSPort {
+			expectedLoc = fmt.Sprintf("https://%s:%s/bar?q=1", locationHost, testcase.Port)
+		}
+		if got, want := rec.Header().Get("Location"), expectedLoc; got != want {
+			t.Errorf("Test %d: Expected Location: '%s' but got '%s'", i, want, got)
+		}
 	}
 }
 

caddyhttp/httpserver/logger.go

@@ -0,0 +1,184 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpserver
+
+import (
+	"bytes"
+	"io"
+	"log"
+	"net"
+	"os"
+	"strings"
+	"sync"
+
+	"github.com/hashicorp/go-syslog"
+	"github.com/mholt/caddy"
+)
+
+var remoteSyslogPrefixes = map[string]string{
+	"syslog+tcp://": "tcp",
+	"syslog+udp://": "udp",
+	"syslog://":     "udp",
+}
+
+// Logger is shared between errors and log plugins and supports both logging to
+// a file (with an optional file roller), local and remote syslog servers.
+type Logger struct {
+	Output string
+	*log.Logger
+	Roller       *LogRoller
+	writer       io.Writer
+	fileMu       *sync.RWMutex
+	V4ipMask     net.IPMask
+	V6ipMask     net.IPMask
+	IPMaskExists bool
+}
+
+// NewTestLogger creates logger suitable for testing purposes
+func NewTestLogger(buffer *bytes.Buffer) *Logger {
+	return &Logger{
+		Logger: log.New(buffer, "", 0),
+		fileMu: new(sync.RWMutex),
+	}
+}
+
+// Println wraps underlying logger with mutex
+func (l Logger) Println(args ...interface{}) {
+	l.fileMu.RLock()
+	l.Logger.Println(args...)
+	l.fileMu.RUnlock()
+}
+
+// Printf wraps underlying logger with mutex
+func (l Logger) Printf(format string, args ...interface{}) {
+	l.fileMu.RLock()
+	l.Logger.Printf(format, args...)
+	l.fileMu.RUnlock()
+}
+
+func (l Logger) MaskIP(ip string) string {
+	var reqIP net.IP
+	// If unable to parse, simply return IP as provided.
+	reqIP = net.ParseIP(ip)
+	if reqIP == nil {
+		return ip
+	}
+
+	if reqIP.To4() != nil {
+		return reqIP.Mask(l.V4ipMask).String()
+	} else {
+		return reqIP.Mask(l.V6ipMask).String()
+	}
+
+}
+
+// Attach binds logger Start and Close functions to
+// controller's OnStartup and OnShutdown hooks.
+func (l *Logger) Attach(controller *caddy.Controller) {
+	if controller != nil {
+		// Opens file or connect to local/remote syslog
+		controller.OnStartup(l.Start)
+
+		// Closes file or disconnects from local/remote syslog
+		controller.OnShutdown(l.Close)
+	}
+}
+
+type syslogAddress struct {
+	network string
+	address string
+}
+
+func parseSyslogAddress(location string) *syslogAddress {
+	for prefix, network := range remoteSyslogPrefixes {
+		if strings.HasPrefix(location, prefix) {
+			return &syslogAddress{
+				network: network,
+				address: strings.TrimPrefix(location, prefix),
+			}
+		}
+	}
+
+	return nil
+}
+
+// Start initializes logger opening files or local/remote syslog connections
+func (l *Logger) Start() error {
+	// initialize mutex on start
+	l.fileMu = new(sync.RWMutex)
+
+	var err error
+
+selectwriter:
+	switch l.Output {
+	case "", "stderr":
+		l.writer = os.Stderr
+	case "stdout":
+		l.writer = os.Stdout
+	case "syslog":
+		l.writer, err = gsyslog.NewLogger(gsyslog.LOG_ERR, "LOCAL0", "caddy")
+		if err != nil {
+			return err
+		}
+	default:
+		if address := parseSyslogAddress(l.Output); address != nil {
+			l.writer, err = gsyslog.DialLogger(address.network, address.address, gsyslog.LOG_ERR, "LOCAL0", "caddy")
+
+			if err != nil {
+				return err
+			}
+
+			break selectwriter
+		}
+
+		var file *os.File
+
+		file, err = os.OpenFile(l.Output, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
+		if err != nil {
+			return err
+		}
+
+		if l.Roller != nil {
+			file.Close()
+			l.Roller.Filename = l.Output
+			l.writer = l.Roller.GetLogWriter()
+		} else {
+			l.writer = file
+		}
+	}
+
+	l.Logger = log.New(l.writer, "", 0)
+
+	return nil
+
+}
+
+// Close closes open log files or connections to syslog.
+func (l *Logger) Close() error {
+	// don't close stdout or stderr
+	if l.writer == os.Stdout || l.writer == os.Stderr {
+		return nil
+	}
+
+	// Will close local/remote syslog connections too :)
+	if closer, ok := l.writer.(io.WriteCloser); ok {
+		l.fileMu.Lock()
+		err := closer.Close()
+		l.fileMu.Unlock()
+		return err
+	}
+
+	return nil
+}

caddyhttp/httpserver/logger_test.go

@@ -0,0 +1,226 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//+build linux darwin
+
+package httpserver
+
+import (
+	"bytes"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strings"
+	"sync"
+	"testing"
+
+	syslog "gopkg.in/mcuadros/go-syslog.v2"
+	"gopkg.in/mcuadros/go-syslog.v2/format"
+)
+
+func TestLoggingToStdout(t *testing.T) {
+	testCases := []struct {
+		Output         string
+		ExpectedOutput string
+	}{
+		{
+			Output:         "stdout",
+			ExpectedOutput: "Hello world logged to stdout",
+		},
+	}
+
+	for i, testCase := range testCases {
+		output := captureStdout(func() {
+			logger := Logger{Output: testCase.Output, fileMu: new(sync.RWMutex)}
+
+			if err := logger.Start(); err != nil {
+				t.Fatalf("Got unexpected error: %v", err)
+			}
+
+			logger.Println(testCase.ExpectedOutput)
+		})
+
+		if !strings.Contains(output, testCase.ExpectedOutput) {
+			t.Fatalf("Test #%d: Expected output to contain: %s, got: %s", i, testCase.ExpectedOutput, output)
+		}
+	}
+}
+
+func TestLoggingToStderr(t *testing.T) {
+
+	testCases := []struct {
+		Output         string
+		ExpectedOutput string
+	}{
+		{
+			Output:         "stderr",
+			ExpectedOutput: "Hello world logged to stderr",
+		},
+		{
+			Output:         "",
+			ExpectedOutput: "Hello world logged to stderr #2",
+		},
+	}
+
+	for i, testCase := range testCases {
+		output := captureStderr(func() {
+			logger := Logger{Output: testCase.Output, fileMu: new(sync.RWMutex)}
+
+			if err := logger.Start(); err != nil {
+				t.Fatalf("Got unexpected error: %v", err)
+			}
+
+			logger.Println(testCase.ExpectedOutput)
+		})
+
+		if !strings.Contains(output, testCase.ExpectedOutput) {
+			t.Fatalf("Test #%d: Expected output to contain: %s, got: %s", i, testCase.ExpectedOutput, output)
+		}
+	}
+}
+
+func TestLoggingToFile(t *testing.T) {
+	file := filepath.Join(os.TempDir(), "access.log")
+	expectedOutput := "Hello world written to file"
+
+	logger := Logger{Output: file}
+
+	if err := logger.Start(); err != nil {
+		t.Fatalf("Got unexpected error during logger start: %v", err)
+	}
+
+	logger.Print(expectedOutput)
+
+	content, err := ioutil.ReadFile(file)
+	if err != nil {
+		t.Fatalf("Could not read log file content: %v", err)
+	}
+
+	if !bytes.Contains(content, []byte(expectedOutput)) {
+		t.Fatalf("Expected log file to contain: %s, got: %s", expectedOutput, string(content))
+	}
+
+	os.Remove(file)
+}
+
+func TestLoggingToSyslog(t *testing.T) {
+
+	testCases := []struct {
+		Output         string
+		ExpectedOutput string
+	}{
+		{
+			Output:         "syslog://127.0.0.1:5660",
+			ExpectedOutput: "Hello world! Test #1 over tcp",
+		},
+		{
+			Output:         "syslog+tcp://127.0.0.1:5661",
+			ExpectedOutput: "Hello world! Test #2 over tcp",
+		},
+		{
+			Output:         "syslog+udp://127.0.0.1:5662",
+			ExpectedOutput: "Hello world! Test #3 over udp",
+		},
+	}
+
+	for i, testCase := range testCases {
+
+		ch := make(chan format.LogParts, 256)
+		server, err := bootServer(testCase.Output, ch)
+		defer server.Kill()
+
+		if err != nil {
+			t.Errorf("Test #%d: expected no error during syslog server boot, got: %v", i, err)
+		}
+
+		logger := Logger{Output: testCase.Output, fileMu: new(sync.RWMutex)}
+
+		if err := logger.Start(); err != nil {
+			t.Errorf("Test #%d: expected no error during logger start, got: %v", i, err)
+		}
+
+		defer logger.Close()
+
+		logger.Print(testCase.ExpectedOutput)
+
+		actual := <-ch
+
+		if content, ok := actual["content"].(string); ok {
+			if !strings.Contains(content, testCase.ExpectedOutput) {
+				t.Errorf("Test #%d: expected server to capture content: %s, but got: %s", i, testCase.ExpectedOutput, content)
+			}
+		} else {
+			t.Errorf("Test #%d: expected server to capture content but got: %v", i, actual)
+		}
+	}
+}
+
+func bootServer(location string, ch chan format.LogParts) (*syslog.Server, error) {
+	address := parseSyslogAddress(location)
+
+	if address == nil {
+		return nil, fmt.Errorf("Could not parse syslog address: %s", location)
+	}
+
+	server := syslog.NewServer()
+	server.SetFormat(syslog.Automatic)
+
+	switch address.network {
+	case "tcp":
+		server.ListenTCP(address.address)
+	case "udp":
+		server.ListenUDP(address.address)
+	}
+
+	server.SetHandler(syslog.NewChannelHandler(ch))
+
+	if err := server.Boot(); err != nil {
+		return nil, err
+	}
+
+	return server, nil
+}
+
+func captureStdout(f func()) string {
+	original := os.Stdout
+	r, w, _ := os.Pipe()
+
+	os.Stdout = w
+
+	f()
+
+	w.Close()
+
+	written, _ := ioutil.ReadAll(r)
+	os.Stdout = original
+
+	return string(written)
+}
+
+func captureStderr(f func()) string {
+	original := os.Stderr
+	r, w, _ := os.Pipe()
+
+	os.Stderr = w
+
+	f()
+
+	w.Close()
+
+	written, _ := ioutil.ReadAll(r)
+	os.Stderr = original
+
+	return string(written)
+}

caddyhttp/httpserver/middleware.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
@@ -5,8 +19,9 @@ import (
 	"net/http"
 	"os"
 	"path"
-	"strings"
 	"time"
+
+	"github.com/mholt/caddy"
 )
 
 func init() {
@@ -19,20 +34,25 @@ type (
 	// passed the next Handler in the chain.
 	Middleware func(Handler) Handler
 
+	// ListenerMiddleware is similar to the Middleware type, except it
+	// chains one net.Listener to the next.
+	ListenerMiddleware func(caddy.Listener) caddy.Listener
+
 	// Handler is like http.Handler except ServeHTTP may return a status
 	// code and/or error.
 	//
-	// If ServeHTTP writes to the response body, it should return a status
-	// code of 0. This signals to other handlers above it that the response
-	// body is already written, and that they should not write to it also.
+	// If ServeHTTP writes the response header, it should return a status
+	// code of 0. This signals to other handlers before it that the response
+	// is already handled, and that they should not write to it also. Keep
+	// in mind that writing to the response body writes the header, too.
 	//
 	// If ServeHTTP encounters an error, it should return the error value
 	// so it can be logged by designated error-handling middleware.
 	//
-	// If writing a response after calling another ServeHTTP method, the
+	// If writing a response after calling the next ServeHTTP method, the
 	// returned status code SHOULD be used when writing the response.
 	//
-	// If handling errors after calling another ServeHTTP method, the
+	// If handling errors after calling the next ServeHTTP method, the
 	// returned error value SHOULD be logged or handled accordingly.
 	//
 	// Otherwise, return values should be propagated down the middleware
@@ -48,13 +68,23 @@ type (
 
 	// RequestMatcher checks to see if current request should be handled
 	// by underlying handler.
-	//
-	// TODO The long term plan is to get all middleware implement this
-	// interface and have validation done before requests are dispatched
-	// to each middleware.
 	RequestMatcher interface {
 		Match(r *http.Request) bool
 	}
+
+	// HandlerConfig is a middleware configuration.
+	// This makes it possible for middlewares to have a common
+	// configuration interface.
+	//
+	// TODO The long term plan is to get all middleware implement this
+	// interface for configurations.
+	HandlerConfig interface {
+		RequestMatcher
+		BasePath() string
+	}
+
+	// ConfigSelector selects a configuration.
+	ConfigSelector []HandlerConfig
 )
 
 // ServeHTTP implements the Handler interface.
@@ -62,6 +92,20 @@ func (f HandlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, err
 	return f(w, r)
 }
 
+// Select selects a Config.
+// This chooses the config with the longest length.
+func (c ConfigSelector) Select(r *http.Request) (config HandlerConfig) {
+	for i := range c {
+		if !c[i].Match(r) {
+			continue
+		}
+		if config == nil || len(c[i].BasePath()) > len(config.BasePath()) {
+			config = c[i]
+		}
+	}
+	return config
+}
+
 // IndexFile looks for a file in /root/fpath/indexFile for each string
 // in indexFiles. If an index file is found, it returns the root-relative
 // path to the file and true. If no index file is found, empty string
@@ -114,7 +158,7 @@ func SetLastModifiedHeader(w http.ResponseWriter, modTime time.Time) {
 
 // CaseSensitivePath determines if paths should be case sensitive.
 // This is configurable via CASE_SENSITIVE_PATH environment variable.
-var CaseSensitivePath = true
+var CaseSensitivePath = false
 
 const caseSensitivePathEnv = "CASE_SENSITIVE_PATH"
 
@@ -123,28 +167,13 @@ const caseSensitivePathEnv = "CASE_SENSITIVE_PATH"
 // This could have been in init, but init cannot be called from tests.
 func initCaseSettings() {
 	switch os.Getenv(caseSensitivePathEnv) {
-	case "0", "false":
-		CaseSensitivePath = false
-	default:
+	case "1", "true":
 		CaseSensitivePath = true
+	default:
+		CaseSensitivePath = false
 	}
 }
 
-// Path represents a URI path.
-type Path string
-
-// Matches checks to see if other matches p.
-//
-// Path matching will probably not always be a direct
-// comparison; this method assures that paths can be
-// easily and consistently matched.
-func (p Path) Matches(other string) bool {
-	if CaseSensitivePath {
-		return strings.HasPrefix(string(p), other)
-	}
-	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(other))
-}
-
 // MergeRequestMatchers merges multiple RequestMatchers into one.
 // This allows a middleware to use multiple RequestMatchers.
 func MergeRequestMatchers(matchers ...RequestMatcher) RequestMatcher {
@@ -182,3 +211,18 @@ var EmptyNext = HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, e
 func SameNext(next1, next2 Handler) bool {
 	return fmt.Sprintf("%v", next1) == fmt.Sprintf("%v", next2)
 }
+
+// Context key constants.
+const (
+	// ReplacerCtxKey is the context key for a per-request replacer.
+	ReplacerCtxKey caddy.CtxKey = "replacer"
+
+	// RemoteUserCtxKey is the key for the remote user of the request, if any (basicauth).
+	RemoteUserCtxKey caddy.CtxKey = "remote_user"
+
+	// MitmCtxKey is the key for the result of MITM detection
+	MitmCtxKey caddy.CtxKey = "mitm"
+
+	// RequestIDCtxKey is the key for the U4 UUID value
+	RequestIDCtxKey caddy.CtxKey = "request_id"
+)

caddyhttp/httpserver/middleware_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
@@ -45,7 +59,7 @@ func TestPathCaseSensitiveEnv(t *testing.T) {
 		{"0", false},
 		{"false", false},
 		{"true", true},
-		{"", true},
+		{"", false},
 	}
 
 	for i, test := range tests {

caddyhttp/httpserver/mitm.go

@@ -0,0 +1,762 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpserver
+
+import (
+	"bytes"
+	"context"
+	"crypto/tls"
+	"io"
+	"net"
+	"net/http"
+	"strconv"
+	"strings"
+	"sync"
+)
+
+// tlsHandler is a http.Handler that will inject a value
+// into the request context indicating if the TLS
+// connection is likely being intercepted.
+type tlsHandler struct {
+	next        http.Handler
+	listener    *tlsHelloListener
+	closeOnMITM bool // whether to close connection on MITM; TODO: expose through new directive
+}
+
+// ServeHTTP checks the User-Agent. For the four main browsers (Chrome,
+// Edge, Firefox, and Safari) indicated by the User-Agent, the properties
+// of the TLS Client Hello will be compared. The context value "mitm" will
+// be set to a value indicating if it is likely that the underlying TLS
+// connection is being intercepted.
+//
+// Note that due to Microsoft's decision to intentionally make IE/Edge
+// user agents obscure (and look like other browsers), this may offer
+// less accuracy for IE/Edge clients.
+//
+// This MITM detection capability is based on research done by Durumeric,
+// Halderman, et. al. in "The Security Impact of HTTPS Interception" (NDSS '17):
+// https://jhalderm.com/pub/papers/interception-ndss17.pdf
+func (h *tlsHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	if h.listener == nil {
+		h.next.ServeHTTP(w, r)
+		return
+	}
+
+	h.listener.helloInfosMu.RLock()
+	info := h.listener.helloInfos[r.RemoteAddr]
+	h.listener.helloInfosMu.RUnlock()
+
+	ua := r.Header.Get("User-Agent")
+
+	var checked, mitm bool
+	if r.Header.Get("X-BlueCoat-Via") != "" || // Blue Coat (masks User-Agent header to generic values)
+		r.Header.Get("X-FCCKV2") != "" || // Fortinet
+		info.advertisesHeartbeatSupport() { // no major browsers have ever implemented Heartbeat
+		checked = true
+		mitm = true
+	} else if strings.Contains(ua, "Edge") || strings.Contains(ua, "MSIE") ||
+		strings.Contains(ua, "Trident") {
+		checked = true
+		mitm = !info.looksLikeEdge()
+	} else if strings.Contains(ua, "Chrome") {
+		checked = true
+		mitm = !info.looksLikeChrome()
+	} else if strings.Contains(ua, "CriOS") {
+		// Chrome on iOS sometimes uses iOS-provided TLS stack (which looks exactly like Safari)
+		// but for connections that don't render a web page (favicon, etc.) it uses its own...
+		checked = true
+		mitm = !info.looksLikeChrome() && !info.looksLikeSafari()
+	} else if strings.Contains(ua, "Firefox") {
+		checked = true
+		if strings.Contains(ua, "Windows") {
+			ver := getVersion(ua, "Firefox")
+			if ver == 45.0 || ver == 52.0 {
+				mitm = !info.looksLikeTor()
+			} else {
+				mitm = !info.looksLikeFirefox()
+			}
+		} else {
+			mitm = !info.looksLikeFirefox()
+		}
+	} else if strings.Contains(ua, "Safari") {
+		checked = true
+		mitm = !info.looksLikeSafari()
+	}
+
+	if checked {
+		r = r.WithContext(context.WithValue(r.Context(), MitmCtxKey, mitm))
+	}
+
+	if mitm && h.closeOnMITM {
+		// TODO: This termination might need to happen later in the middleware
+		// chain in order to be picked up by the log directive, in case the site
+		// owner still wants to log this event. It'll probably require a new
+		// directive. If this feature is useful, we can finish implementing this.
+		r.Close = true
+		return
+	}
+
+	h.next.ServeHTTP(w, r)
+}
+
+// getVersion returns a (possibly simplified) representation of the version string
+// from a UserAgent string. It returns a float, so it can represent major and minor
+// versions; the rest of the version is just tacked on behind the decimal point.
+// The purpose of this is to stay simple while allowing for basic, fast comparisons.
+// If the version for softwareName is not found in ua, -1 is returned.
+func getVersion(ua, softwareName string) float64 {
+	search := softwareName + "/"
+	start := strings.Index(ua, search)
+	if start < 0 {
+		return -1
+	}
+	start += len(search)
+	end := strings.Index(ua[start:], " ")
+	if end < 0 {
+		end = len(ua)
+	} else {
+		end += start
+	}
+	strVer := strings.Replace(ua[start:end], "-", "", -1)
+	firstDot := strings.Index(strVer, ".")
+	if firstDot >= 0 {
+		strVer = strVer[:firstDot+1] + strings.Replace(strVer[firstDot+1:], ".", "", -1)
+	}
+	ver, err := strconv.ParseFloat(strVer, 64)
+	if err != nil {
+		return -1
+	}
+	return ver
+}
+
+// clientHelloConn reads the ClientHello
+// and stores it in the attached listener.
+type clientHelloConn struct {
+	net.Conn
+	listener  *tlsHelloListener
+	readHello bool // whether ClientHello has been read
+	buf       *bytes.Buffer
+}
+
+// Read reads from c.Conn (by letting the standard library
+// do the reading off the wire), with the exception of
+// getting a copy of the ClientHello so it can parse it.
+func (c *clientHelloConn) Read(b []byte) (n int, err error) {
+	// if we've already read the ClientHello, pass thru
+	if c.readHello {
+		return c.Conn.Read(b)
+	}
+
+	// we let the standard lib read off the wire for us, and
+	// tee that into our buffer so we can read the ClientHello
+	tee := io.TeeReader(c.Conn, c.buf)
+	n, err = tee.Read(b)
+	if err != nil {
+		return
+	}
+	if c.buf.Len() < 5 {
+		return // need to read more bytes for header
+	}
+
+	// read the header bytes
+	hdr := make([]byte, 5)
+	_, err = io.ReadFull(c.buf, hdr)
+	if err != nil {
+		return // this would be highly unusual and sad
+	}
+
+	// get length of the ClientHello message and read it
+	length := int(uint16(hdr[3])<<8 | uint16(hdr[4]))
+	if c.buf.Len() < length {
+		return // need to read more bytes
+	}
+	hello := make([]byte, length)
+	_, err = io.ReadFull(c.buf, hello)
+	if err != nil {
+		return
+	}
+	bufpool.Put(c.buf) // buffer no longer needed
+
+	// parse the ClientHello and store it in the map
+	rawParsed := parseRawClientHello(hello)
+	c.listener.helloInfosMu.Lock()
+	c.listener.helloInfos[c.Conn.RemoteAddr().String()] = rawParsed
+	c.listener.helloInfosMu.Unlock()
+
+	c.readHello = true
+	return
+}
+
+// parseRawClientHello parses data which contains the raw
+// TLS Client Hello message. It extracts relevant information
+// into info. Any error reading the Client Hello (such as
+// insufficient length or invalid length values) results in
+// a silent error and an incomplete info struct, since there
+// is no good way to handle an error like this during Accept().
+// The data is expected to contain the whole ClientHello and
+// ONLY the ClientHello.
+//
+// The majority of this code is borrowed from the Go standard
+// library, which is (c) The Go Authors. It has been modified
+// to fit this use case.
+func parseRawClientHello(data []byte) (info rawHelloInfo) {
+	if len(data) < 42 {
+		return
+	}
+	sessionIDLen := int(data[38])
+	if sessionIDLen > 32 || len(data) < 39+sessionIDLen {
+		return
+	}
+	data = data[39+sessionIDLen:]
+	if len(data) < 2 {
+		return
+	}
+	// cipherSuiteLen is the number of bytes of cipher suite numbers. Since
+	// they are uint16s, the number must be even.
+	cipherSuiteLen := int(data[0])<<8 | int(data[1])
+	if cipherSuiteLen%2 == 1 || len(data) < 2+cipherSuiteLen {
+		return
+	}
+	numCipherSuites := cipherSuiteLen / 2
+	// read in the cipher suites
+	info.cipherSuites = make([]uint16, numCipherSuites)
+	for i := 0; i < numCipherSuites; i++ {
+		info.cipherSuites[i] = uint16(data[2+2*i])<<8 | uint16(data[3+2*i])
+	}
+	data = data[2+cipherSuiteLen:]
+	if len(data) < 1 {
+		return
+	}
+	// read in the compression methods
+	compressionMethodsLen := int(data[0])
+	if len(data) < 1+compressionMethodsLen {
+		return
+	}
+	info.compressionMethods = data[1 : 1+compressionMethodsLen]
+
+	data = data[1+compressionMethodsLen:]
+
+	// ClientHello is optionally followed by extension data
+	if len(data) < 2 {
+		return
+	}
+	extensionsLength := int(data[0])<<8 | int(data[1])
+	data = data[2:]
+	if extensionsLength != len(data) {
+		return
+	}
+
+	// read in each extension, and extract any relevant information
+	// from extensions we care about
+	for len(data) != 0 {
+		if len(data) < 4 {
+			return
+		}
+		extension := uint16(data[0])<<8 | uint16(data[1])
+		length := int(data[2])<<8 | int(data[3])
+		data = data[4:]
+		if len(data) < length {
+			return
+		}
+
+		// record that the client advertised support for this extension
+		info.extensions = append(info.extensions, extension)
+
+		switch extension {
+		case extensionSupportedCurves:
+			// http://tools.ietf.org/html/rfc4492#section-5.5.1
+			if length < 2 {
+				return
+			}
+			l := int(data[0])<<8 | int(data[1])
+			if l%2 == 1 || length != l+2 {
+				return
+			}
+			numCurves := l / 2
+			info.curves = make([]tls.CurveID, numCurves)
+			d := data[2:]
+			for i := 0; i < numCurves; i++ {
+				info.curves[i] = tls.CurveID(d[0])<<8 | tls.CurveID(d[1])
+				d = d[2:]
+			}
+		case extensionSupportedPoints:
+			// http://tools.ietf.org/html/rfc4492#section-5.5.2
+			if length < 1 {
+				return
+			}
+			l := int(data[0])
+			if length != l+1 {
+				return
+			}
+			info.points = make([]uint8, l)
+			copy(info.points, data[1:])
+		}
+
+		data = data[length:]
+	}
+
+	return
+}
+
+// newTLSListener returns a new tlsHelloListener that wraps ln.
+func newTLSListener(ln net.Listener, config *tls.Config) *tlsHelloListener {
+	return &tlsHelloListener{
+		Listener:   ln,
+		config:     config,
+		helloInfos: make(map[string]rawHelloInfo),
+	}
+}
+
+// tlsHelloListener is a TLS listener that is specially designed
+// to read the ClientHello manually so we can extract necessary
+// information from it. Each ClientHello message is mapped by
+// the remote address of the client, which must be removed when
+// the connection is closed (use ConnState).
+type tlsHelloListener struct {
+	net.Listener
+	config       *tls.Config
+	helloInfos   map[string]rawHelloInfo
+	helloInfosMu sync.RWMutex
+}
+
+// Accept waits for and returns the next connection to the listener.
+// After it accepts the underlying connection, it reads the
+// ClientHello message and stores the parsed data into a map on l.
+func (l *tlsHelloListener) Accept() (net.Conn, error) {
+	conn, err := l.Listener.Accept()
+	if err != nil {
+		return nil, err
+	}
+	buf := bufpool.Get().(*bytes.Buffer)
+	buf.Reset()
+	helloConn := &clientHelloConn{Conn: conn, listener: l, buf: buf}
+	return tls.Server(helloConn, l.config), nil
+}
+
+// rawHelloInfo contains the "raw" data parsed from the TLS
+// Client Hello. No interpretation is done on the raw data.
+//
+// The methods on this type implement heuristics described
+// by Durumeric, Halderman, et. al. in
+// "The Security Impact of HTTPS Interception":
+// https://jhalderm.com/pub/papers/interception-ndss17.pdf
+type rawHelloInfo struct {
+	cipherSuites       []uint16
+	extensions         []uint16
+	compressionMethods []byte
+	curves             []tls.CurveID
+	points             []uint8
+}
+
+// advertisesHeartbeatSupport returns true if info indicates
+// that the client supports the Heartbeat extension.
+func (info rawHelloInfo) advertisesHeartbeatSupport() bool {
+	for _, ext := range info.extensions {
+		if ext == extensionHeartbeat {
+			return true
+		}
+	}
+	return false
+}
+
+// looksLikeFirefox returns true if info looks like a handshake
+// from a modern version of Firefox.
+func (info rawHelloInfo) looksLikeFirefox() bool {
+	// "To determine whether a Firefox session has been
+	// intercepted, we check for the presence and order
+	// of extensions, cipher suites, elliptic curves,
+	// EC point formats, and handshake compression methods." (early 2016)
+
+	// We check for the presence and order of the extensions.
+	// Note: Sometimes 0x15 (21, padding) is present, sometimes not.
+	// Note: Firefox 51+ does not advertise 0x3374 (13172, NPN).
+	// Note: Firefox doesn't advertise 0x0 (0, SNI) when connecting to IP addresses.
+	// Note: Firefox 55+ doesn't appear to advertise 0xFF03 (65283, short headers). It used to be between 5 and 13.
+	// Note: Firefox on Fedora (or RedHat) doesn't include ECC suites because of patent liability.
+	requiredExtensionsOrder := []uint16{23, 65281, 10, 11, 35, 16, 5, 13}
+	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
+		return false
+	}
+
+	// We check for both presence of curves and their ordering.
+	requiredCurves := []tls.CurveID{29, 23, 24, 25}
+	if len(info.curves) < len(requiredCurves) {
+		return false
+	}
+	for i := range requiredCurves {
+		if info.curves[i] != requiredCurves[i] {
+			return false
+		}
+	}
+	if len(info.curves) > len(requiredCurves) {
+		// newer Firefox (55 Nightly?) may have additional curves at end of list
+		allowedCurves := []tls.CurveID{256, 257}
+		for i := range allowedCurves {
+			if info.curves[len(requiredCurves)+i] != allowedCurves[i] {
+				return false
+			}
+		}
+	}
+
+	if hasGreaseCiphers(info.cipherSuites) {
+		return false
+	}
+
+	// We check for order of cipher suites but not presence, since
+	// according to the paper, cipher suites may be not be added
+	// or reordered by the user, but they may be disabled.
+	expectedCipherSuiteOrder := []uint16{
+		TLS_AES_128_GCM_SHA256,                      // 0x1301
+		TLS_CHACHA20_POLY1305_SHA256,                // 0x1303
+		TLS_AES_256_GCM_SHA384,                      // 0x1302
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
+		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,   // 0xc02f
+		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,  // 0xcca9
+		tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,    // 0xcca8
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, // 0xc02c
+		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,   // 0xc030
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,    // 0xc00a
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,    // 0xc009
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,      // 0xc013
+		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,      // 0xc014
+		TLS_DHE_RSA_WITH_AES_128_CBC_SHA,            // 0x33
+		TLS_DHE_RSA_WITH_AES_256_CBC_SHA,            // 0x39
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA,            // 0x2f
+		tls.TLS_RSA_WITH_AES_256_CBC_SHA,            // 0x35
+		tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,           // 0xa
+	}
+	return assertPresenceAndOrdering(expectedCipherSuiteOrder, info.cipherSuites, false)
+}
+
+// looksLikeChrome returns true if info looks like a handshake
+// from a modern version of Chrome.
+func (info rawHelloInfo) looksLikeChrome() bool {
+	// "We check for ciphers and extensions that Chrome is known
+	// to not support, but do not check for the inclusion of
+	// specific ciphers or extensions, nor do we validate their
+	// order. When appropriate, we check the presence and order
+	// of elliptic curves, compression methods, and EC point formats." (early 2016)
+
+	// Not in Chrome 56, but present in Safari 10 (Feb. 2017):
+	// TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 (0xc024)
+	// TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 (0xc023)
+	// TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
+	// TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
+	// TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028)
+	// TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 (0xc027)
+	// TLS_RSA_WITH_AES_256_CBC_SHA256 (0x3d)
+	// TLS_RSA_WITH_AES_128_CBC_SHA256 (0x3c)
+
+	// Not in Chrome 56, but present in Firefox 51 (Feb. 2017):
+	// TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
+	// TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
+	// TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x33)
+	// TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39)
+
+	// Selected ciphers present in Chrome mobile (Feb. 2017):
+	// 0xc00a, 0xc014, 0xc009, 0x9c, 0x9d, 0x2f, 0x35, 0xa
+
+	chromeCipherExclusions := map[uint16]struct{}{
+		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:     {}, // 0xc024
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256: {}, // 0xc023
+		TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:       {}, // 0xc028
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256:   {}, // 0xc027
+		TLS_RSA_WITH_AES_256_CBC_SHA256:             {}, // 0x3d
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA256:         {}, // 0x3c
+		TLS_DHE_RSA_WITH_AES_128_CBC_SHA:            {}, // 0x33
+		TLS_DHE_RSA_WITH_AES_256_CBC_SHA:            {}, // 0x39
+	}
+	for _, ext := range info.cipherSuites {
+		if _, ok := chromeCipherExclusions[ext]; ok {
+			return false
+		}
+	}
+
+	// Chrome does not include curve 25 (CurveP521) (as of Chrome 56, Feb. 2017).
+	for _, curve := range info.curves {
+		if curve == 25 {
+			return false
+		}
+	}
+
+	if !hasGreaseCiphers(info.cipherSuites) {
+		return false
+	}
+
+	return true
+}
+
+// looksLikeEdge returns true if info looks like a handshake
+// from a modern version of MS Edge.
+func (info rawHelloInfo) looksLikeEdge() bool {
+	// "SChannel connections can by uniquely identified because SChannel
+	// is the only TLS library we tested that includes the OCSP status
+	// request extension before the supported groups and EC point formats
+	// extensions." (early 2016)
+	//
+	// More specifically, the OCSP status request extension appears
+	// *directly* before the other two extensions, which occur in that
+	// order. (I contacted the authors for clarification and verified it.)
+	for i, ext := range info.extensions {
+		if ext == extensionOCSPStatusRequest {
+			if len(info.extensions) <= i+2 {
+				return false
+			}
+			if info.extensions[i+1] != extensionSupportedCurves ||
+				info.extensions[i+2] != extensionSupportedPoints {
+				return false
+			}
+		}
+	}
+
+	for _, cs := range info.cipherSuites {
+		// As of Feb. 2017, Edge does not have 0xff, but Avast adds it
+		if cs == scsvRenegotiation {
+			return false
+		}
+		// Edge and modern IE do not have 0x4 or 0x5, but Blue Coat does
+		if cs == TLS_RSA_WITH_RC4_128_MD5 || cs == tls.TLS_RSA_WITH_RC4_128_SHA {
+			return false
+		}
+	}
+
+	if hasGreaseCiphers(info.cipherSuites) {
+		return false
+	}
+
+	return true
+}
+
+// looksLikeSafari returns true if info looks like a handshake
+// from a modern version of MS Safari.
+func (info rawHelloInfo) looksLikeSafari() bool {
+	// "One unique aspect of Secure Transport is that it includes
+	// the TLS_EMPTY_RENEGOTIATION_INFO_SCSV (0xff) cipher first,
+	// whereas the other libraries we investigated include the
+	// cipher last. Similar to Microsoft, Apple has changed
+	// TLS behavior in minor OS updates, which are not indicated
+	// in the HTTP User-Agent header. We allow for any of the
+	// updates when validating handshakes, and we check for the
+	// presence and ordering of ciphers, extensions, elliptic
+	// curves, and compression methods." (early 2016)
+
+	// Note that any C lib (e.g. curl) compiled on macOS
+	// will probably use Secure Transport which will also
+	// share the TLS handshake characteristics of Safari.
+
+	// We check for the presence and order of the extensions.
+	requiredExtensionsOrder := []uint16{10, 11, 13, 13172, 16, 5, 18, 23}
+	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
+		// Safari on iOS 11 (beta) uses different set/ordering of extensions
+		requiredExtensionsOrderiOS11 := []uint16{65281, 0, 23, 13, 5, 13172, 18, 16, 11, 10}
+		if !assertPresenceAndOrdering(requiredExtensionsOrderiOS11, info.extensions, true) {
+			return false
+		}
+	} else {
+		// For these versions of Safari, expect TLS_EMPTY_RENEGOTIATION_INFO_SCSV first.
+		if len(info.cipherSuites) < 1 {
+			return false
+		}
+		if info.cipherSuites[0] != scsvRenegotiation {
+			return false
+		}
+	}
+
+	if hasGreaseCiphers(info.cipherSuites) {
+		return false
+	}
+
+	// We check for order and presence of cipher suites
+	expectedCipherSuiteOrder := []uint16{
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, // 0xc02c
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
+		TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,     // 0xc024
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, // 0xc023
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,    // 0xc00a
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,    // 0xc009
+		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,   // 0xc030
+		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,   // 0xc02f
+		TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,       // 0xc028
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,   // 0xc027
+		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,      // 0xc014
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,      // 0xc013
+		tls.TLS_RSA_WITH_AES_256_GCM_SHA384,         // 0x9d
+		tls.TLS_RSA_WITH_AES_128_GCM_SHA256,         // 0x9c
+		TLS_RSA_WITH_AES_256_CBC_SHA256,             // 0x3d
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA256,         // 0x3c
+		tls.TLS_RSA_WITH_AES_256_CBC_SHA,            // 0x35
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA,            // 0x2f
+	}
+	return assertPresenceAndOrdering(expectedCipherSuiteOrder, info.cipherSuites, true)
+}
+
+// looksLikeTor returns true if the info looks like a ClientHello from Tor browser
+// (based on Firefox).
+func (info rawHelloInfo) looksLikeTor() bool {
+	requiredExtensionsOrder := []uint16{10, 11, 16, 5, 13}
+	if !assertPresenceAndOrdering(requiredExtensionsOrder, info.extensions, true) {
+		return false
+	}
+
+	// check for session tickets support; Tor doesn't support them to prevent tracking
+	for _, ext := range info.extensions {
+		if ext == 35 {
+			return false
+		}
+	}
+
+	// We check for both presence of curves and their ordering, including
+	// an optional curve at the beginning (for Tor based on Firefox 52)
+	infoCurves := info.curves
+	if len(info.curves) == 4 {
+		if info.curves[0] != 29 {
+			return false
+		}
+		infoCurves = info.curves[1:]
+	}
+	requiredCurves := []tls.CurveID{23, 24, 25}
+	if len(infoCurves) < len(requiredCurves) {
+		return false
+	}
+	for i := range requiredCurves {
+		if infoCurves[i] != requiredCurves[i] {
+			return false
+		}
+	}
+
+	if hasGreaseCiphers(info.cipherSuites) {
+		return false
+	}
+
+	// We check for order of cipher suites but not presence, since
+	// according to the paper, cipher suites may be not be added
+	// or reordered by the user, but they may be disabled.
+	expectedCipherSuiteOrder := []uint16{
+		TLS_AES_128_GCM_SHA256,                      // 0x1301
+		TLS_CHACHA20_POLY1305_SHA256,                // 0x1303
+		TLS_AES_256_GCM_SHA384,                      // 0x1302
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, // 0xc02b
+		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,   // 0xc02f
+		tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,  // 0xcca9
+		tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,    // 0xcca8
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, // 0xc02c
+		tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,   // 0xc030
+		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,    // 0xc00a
+		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,    // 0xc009
+		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,      // 0xc013
+		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,      // 0xc014
+		TLS_DHE_RSA_WITH_AES_128_CBC_SHA,            // 0x33
+		TLS_DHE_RSA_WITH_AES_256_CBC_SHA,            // 0x39
+		tls.TLS_RSA_WITH_AES_128_CBC_SHA,            // 0x2f
+		tls.TLS_RSA_WITH_AES_256_CBC_SHA,            // 0x35
+		tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,           // 0xa
+	}
+	return assertPresenceAndOrdering(expectedCipherSuiteOrder, info.cipherSuites, false)
+}
+
+// assertPresenceAndOrdering will return true if candidateList contains
+// the items in requiredItems in the same order as requiredItems.
+//
+// If requiredIsSubset is true, then all items in requiredItems must be
+// present in candidateList. If requiredIsSubset is false, then requiredItems
+// may contain items that are not in candidateList.
+//
+// In all cases, the order of requiredItems is enforced.
+func assertPresenceAndOrdering(requiredItems, candidateList []uint16, requiredIsSubset bool) bool {
+	superset := requiredItems
+	subset := candidateList
+	if requiredIsSubset {
+		superset = candidateList
+		subset = requiredItems
+	}
+
+	var j int
+	for _, item := range subset {
+		var found bool
+		for j < len(superset) {
+			if superset[j] == item {
+				found = true
+				break
+			}
+			j++
+		}
+		if j == len(superset) && !found {
+			return false
+		}
+	}
+	return true
+}
+
+func hasGreaseCiphers(cipherSuites []uint16) bool {
+	for _, cipher := range cipherSuites {
+		if _, ok := greaseCiphers[cipher]; ok {
+			return true
+		}
+	}
+	return false
+}
+
+// pool buffers so we can reuse allocations over time
+var bufpool = sync.Pool{
+	New: func() interface{} {
+		return new(bytes.Buffer)
+	},
+}
+
+var greaseCiphers = map[uint16]struct{}{
+	0x0A0A: {},
+	0x1A1A: {},
+	0x2A2A: {},
+	0x3A3A: {},
+	0x4A4A: {},
+	0x5A5A: {},
+	0x6A6A: {},
+	0x7A7A: {},
+	0x8A8A: {},
+	0x9A9A: {},
+	0xAAAA: {},
+	0xBABA: {},
+	0xCACA: {},
+	0xDADA: {},
+	0xEAEA: {},
+	0xFAFA: {},
+}
+
+// Define variables used for TLS communication
+const (
+	extensionOCSPStatusRequest = 5
+	extensionSupportedCurves   = 10 // also called "SupportedGroups"
+	extensionSupportedPoints   = 11
+	extensionHeartbeat         = 15
+
+	scsvRenegotiation = 0xff
+
+	// cipher suites missing from the crypto/tls package,
+	// in no particular order here
+	TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 = 0xc024
+	TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384   = 0xc028
+	TLS_RSA_WITH_AES_256_CBC_SHA256         = 0x3d
+	TLS_DHE_RSA_WITH_AES_128_CBC_SHA        = 0x33
+	TLS_DHE_RSA_WITH_AES_256_CBC_SHA        = 0x39
+	TLS_RSA_WITH_RC4_128_MD5                = 0x4
+
+	// new PSK ciphers introduced by TLS 1.3, not (yet) in crypto/tls
+	// https://tlswg.github.io/tls13-spec/#rfc.appendix.A.4)
+	TLS_AES_128_GCM_SHA256       = 0x1301
+	TLS_AES_256_GCM_SHA384       = 0x1302
+	TLS_CHACHA20_POLY1305_SHA256 = 0x1303
+	TLS_AES_128_CCM_SHA256       = 0x1304
+	TLS_AES_128_CCM_8_SHA256     = 0x1305
+)

caddyhttp/httpserver/mitm_test.go

@@ -0,0 +1,420 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpserver
+
+import (
+	"crypto/tls"
+	"encoding/hex"
+	"net/http"
+	"net/http/httptest"
+	"reflect"
+	"testing"
+)
+
+func TestParseClientHello(t *testing.T) {
+	for i, test := range []struct {
+		inputHex string
+		expected rawHelloInfo
+	}{
+		{
+			// curl 7.51.0 (x86_64-apple-darwin16.0) libcurl/7.51.0 SecureTransport zlib/1.2.8
+			inputHex: `010000a6030358a28c73a71bdfc1f09dee13fecdc58805dcce42ac44254df548f14645f7dc2c00004400ffc02cc02bc024c023c00ac009c008c030c02fc028c027c014c013c012009f009e006b0067003900330016009d009c003d003c0035002f000a00af00ae008d008c008b01000039000a00080006001700180019000b00020100000d00120010040102010501060104030203050306030005000501000000000012000000170000`,
+			expected: rawHelloInfo{
+				cipherSuites:       []uint16{255, 49196, 49195, 49188, 49187, 49162, 49161, 49160, 49200, 49199, 49192, 49191, 49172, 49171, 49170, 159, 158, 107, 103, 57, 51, 22, 157, 156, 61, 60, 53, 47, 10, 175, 174, 141, 140, 139},
+				extensions:         []uint16{10, 11, 13, 5, 18, 23},
+				compressionMethods: []byte{0},
+				curves:             []tls.CurveID{23, 24, 25},
+				points:             []uint8{0},
+			},
+		},
+		{
+			// Chrome 56
+			inputHex: `010000c003031dae75222dae1433a5a283ddcde8ddabaefbf16d84f250eee6fdff48cdfff8a00000201a1ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a010000777a7a0000ff010001000000000e000c0000096c6f63616c686f73740017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a0008aaaa001d001700182a2a000100`,
+			expected: rawHelloInfo{
+				cipherSuites:       []uint16{6682, 49195, 49199, 49196, 49200, 52393, 52392, 52244, 52243, 49171, 49172, 156, 157, 47, 53, 10},
+				extensions:         []uint16{31354, 65281, 0, 23, 35, 13, 5, 18, 16, 30032, 11, 10, 10794},
+				compressionMethods: []byte{0},
+				curves:             []tls.CurveID{43690, 29, 23, 24},
+				points:             []uint8{0},
+			},
+		},
+		{
+			// Firefox 51
+			inputHex: `010000bd030375f9022fc3a6562467f3540d68013b2d0b961979de6129e944efe0b35531323500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a010000760000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000ff030000000d0020001e040305030603020308040805080604010501060102010402050206020202`,
+			expected: rawHelloInfo{
+				cipherSuites:       []uint16{49195, 49199, 52393, 52392, 49196, 49200, 49162, 49161, 49171, 49172, 51, 57, 47, 53, 10},
+				extensions:         []uint16{0, 23, 65281, 10, 11, 35, 16, 5, 65283, 13},
+				compressionMethods: []byte{0},
+				curves:             []tls.CurveID{29, 23, 24, 25},
+				points:             []uint8{0},
+			},
+		},
+		{
+			// openssl s_client (OpenSSL 0.9.8zh 14 Jan 2016)
+			inputHex: `0100012b03035d385236b8ca7b7946fa0336f164e76bf821ed90e8de26d97cc677671b6f36380000acc030c02cc028c024c014c00a00a500a300a1009f006b006a0069006800390038003700360088008700860085c032c02ec02ac026c00fc005009d003d00350084c02fc02bc027c023c013c00900a400a200a0009e00670040003f003e0033003200310030009a0099009800970045004400430042c031c02dc029c025c00ec004009c003c002f009600410007c011c007c00cc00200050004c012c008001600130010000dc00dc003000a00ff0201000055000b000403000102000a001c001a00170019001c001b0018001a0016000e000d000b000c0009000a00230000000d0020001e060106020603050105020503040104020403030103020303020102020203000f000101`,
+			expected: rawHelloInfo{
+				cipherSuites:       []uint16{49200, 49196, 49192, 49188, 49172, 49162, 165, 163, 161, 159, 107, 106, 105, 104, 57, 56, 55, 54, 136, 135, 134, 133, 49202, 49198, 49194, 49190, 49167, 49157, 157, 61, 53, 132, 49199, 49195, 49191, 49187, 49171, 49161, 164, 162, 160, 158, 103, 64, 63, 62, 51, 50, 49, 48, 154, 153, 152, 151, 69, 68, 67, 66, 49201, 49197, 49193, 49189, 49166, 49156, 156, 60, 47, 150, 65, 7, 49169, 49159, 49164, 49154, 5, 4, 49170, 49160, 22, 19, 16, 13, 49165, 49155, 10, 255},
+				extensions:         []uint16{11, 10, 35, 13, 15},
+				compressionMethods: []byte{1, 0},
+				curves:             []tls.CurveID{23, 25, 28, 27, 24, 26, 22, 14, 13, 11, 12, 9, 10},
+				points:             []uint8{0, 1, 2},
+			},
+		},
+	} {
+		data, err := hex.DecodeString(test.inputHex)
+		if err != nil {
+			t.Fatalf("Test %d: Could not decode hex data: %v", i, err)
+		}
+		actual := parseRawClientHello(data)
+		if !reflect.DeepEqual(test.expected, actual) {
+			t.Errorf("Test %d: Expected %+v; got %+v", i, test.expected, actual)
+		}
+	}
+}
+
+func TestHeuristicFunctionsAndHandler(t *testing.T) {
+	// To test the heuristics, we assemble a collection of real
+	// ClientHello messages from various TLS clients, both genuine
+	// and intercepted. Please be sure to hex-encode them and
+	// document the User-Agent associated with the connection
+	// as well as any intercepting proxy as thoroughly as possible.
+	//
+	// If the TLS client used is not an HTTP client (e.g. s_client),
+	// you can leave the userAgent blank, but please use a comment
+	// to document crucial missing information such as client name,
+	// version, and platform, maybe even the date you collected
+	// the sample! Please group similar clients together, ordered
+	// by version for convenience.
+
+	// clientHello pairs a User-Agent string to its ClientHello message.
+	type clientHello struct {
+		userAgent    string
+		helloHex     string      // do NOT include the header, just the ClientHello message
+		interception bool        // if test case shows an interception, set to true
+		reqHeaders   http.Header // if the request should set any headers to imitate a browser or proxy
+	}
+
+	// clientHellos groups samples of true (real) ClientHellos by the
+	// name of the browser that produced them. We limit the set of
+	// browsers to those we are programmed to protect, as well as a
+	// category for "Other" which contains real ClientHello messages
+	// from clients that we do not recognize, which may be used to
+	// test or imitate interception scenarios.
+	//
+	// Please group similar clients and order by version for convenience
+	// when adding to the test cases.
+	clientHellos := map[string][]clientHello{
+		"Chrome": {
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
+				helloHex:     `010000c003031dae75222dae1433a5a283ddcde8ddabaefbf16d84f250eee6fdff48cdfff8a00000201a1ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a010000777a7a0000ff010001000000000e000c0000096c6f63616c686f73740017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a0008aaaa001d001700182a2a000100`,
+				interception: false,
+			},
+			{
+				// Chrome on iOS will use iOS' TLS stack for requests that load
+				// the web page (apparently required by the dev ToS) but will use its
+				// own TLS stack for everything else, it seems.
+
+				// Chrome on iOS
+				userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
+				helloHex:  `010000de030358b062c509b21410a6496b5a82bfec74436cdecebe8ea1da29799939bbd3c17200002c00ffc02cc02bc024c023c00ac009c008c030c02fc028c027c014c013c012009d009c003d003c0035002f000a0100008900000014001200000f66696e6572706978656c732e636f6d000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
+			},
+			{
+				// Chrome on iOS (requesting favicon)
+				userAgent: "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) CriOS/56.0.2924.79 Mobile/14A456 Safari/602.1",
+				helloHex:  `010000c20303863eb64788e3b9638c261300318411cbdd8f09576d58eec1e744b6ce944f574f0000208a8acca9cca8cc14cc13c02bc02fc02cc030c013c014009c009d002f0035000a01000079baba0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e31000b00020100000a000a00083a3a001d001700184a4a000100`,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
+				helloHex:     `010000c603036f717a88212c3e9e41940f82c42acb3473e0e4a64e8f52d9af33d34e972e08a30000206a6ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a0100007d7a7a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00087a7a001d001700188a8a000100`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
+				helloHex:     `010001fc030383141d213d1bf069171843489faf808028d282c9828e1ba87637c863833c730720a67e76e152f4b704523b72317ef4587e231f02e2395e0ecac6be9f28c35e6ce600208a8ac02bc02fc02cc030cca9cca8cc14cc13c013c014009c009d002f0035000a010001931a1a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d00170000002300785e85429bf1764f33111cd3ad5d1c56d765976fd962b49dbecbb6f7865e2a8d8536ad854f1fa99a8bbbf998814fee54a63a0bf162869d2bba37e9778304e7c4140825718e191b574c6246a0611de6447bdd80417f83ff9d9b7124069a9f74b90394ecb89bec5f6a1a67c1b89e50b8674782f53dd51807651a000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00081a1a001d001700182a2a0001000015009a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
+				helloHex:     `010000c203034166c97e2016046e0c88ad867c410d0aee470f4d9b4ec8fe41a751d2a6348e3100001c4a4ac02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a0100007dcaca0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00086a6a001d001700187a7a000100`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36",
+				helloHex:     `010000c203037741795e73cd5b4949f79a0dc9cccc8b006e4c0ec324f965c6fe9f0833909f0100001c7a7ac02bc02fc02cc030cca9cca8c013c014009c009d002f0035000a0100007d7a7a0000ff0100010000000014001200000f66696e6572706978656c732e636f6d0017000000230000000d00140012040308040401050308050501080606010201000500050100000000001200000010000e000c02683208687474702f312e3175500000000b00020100000a000a00084a4a001d001700185a5a000100`,
+				interception: false,
+			},
+		},
+		"Firefox": {
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:51.0) Gecko/20100101 Firefox/51.0",
+				helloHex:     `010000bd030375f9022fc3a6562467f3540d68013b2d0b961979de6129e944efe0b35531323500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a010000760000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000ff030000000d0020001e040305030603020308040805080604010501060102010402050206020202`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010001fc0303c99d54ae0628bbb9fea3833a4244c6a712cac9d7738f4930b8b9d8e2f6bd578220f7936cedb48907981c9292fb08ceee6f59bd6fddb3d4271ccd7c12380c5038ab001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a01000195001500af000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b000201000023007886da2d41843ff42131b856982c19a545837b70e604325423a817d925e9d95bd084737682cea6b804dfb7cbe336a3b27b8d520d57520c29cfe5f4f3d3236183b84b05c18f0ca30bf598111e390086fea00d9631f1f78527277eb7838b86e73c4e5d15b55d086b1a4a8aa29f12a55126c6274bcd499bbeb23a0010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010000b1030365d899820b999245d571c2f7d6b850f63ad931d3c68ceb9cf5a508421a871dc500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100006a0000000e000c0000096c6f63616c686f737400170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+			{
+				// this was a Nightly release at the time
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:55.0) Gecko/20100101 Firefox/55.0",
+				helloHex:     `010001fc030331e380b7d12018e1202ef3327607203df5c5732b4fa5ab5abaf0b60034c2fb662070c836b9b89123e37f4f1074d152df438fa8ee8a0f89b036fd952f4fcc0b994f001c130113031302c02bc02fcca9cca8c02cc030c013c014002f0035000a0100019700000014001200000f63616464797365727665722e636f6d00170000ff01000100000a000e000c001d00170018001901000101000b0002010000230078c97e7716a041e2ea824571bef26a3dff2bf50a883cd15d904ab2d17deb514f6e0a079ee7c212c000178387ffafc2e530b6df6662f570aae134330f13c458a0eaad5a96a9696f572110918740b15db1143d19aaaa706942030b433a7e6150f62b443c0564e5b8f7ee9577bf3bf7faec8c67425b648ab54d880010000e000c02683208687474702f312e310005000501000000000028006b0069001d0020aee6e596155ee6f79f943e81ceabe0979d27fbbb8b9189ccb2ebc75226351f32001700410421875a44e510decac11ef1d7cfddd4dfe105d5cd3a2d42fba03ebde23e51e8ce65bda1b48be82d4848d1db2bfce68e94092e925a9ce0dbf5df35479558108489002b0009087f12030303020301000d0018001604030503060308040805080604010501060102030201002d000201010015002500000000000000000000000000000000000000000000000000000000000000000000000000`,
+				interception: false,
+			},
+			{
+				// Firefox on Fedora (RedHat) doesn't include ECC ciphers because of patent liabilities
+				userAgent:    "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0",
+				helloHex:     `010000b70303f5280b74d617d42e39fd77b78a2b537b1d7787ce4fcbcf3604c9fbcd677c6c5500001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100007000000014001200000f66696e6572706978656c732e636f6d00170000ff01000100000a000a0008001d001700180019000b00020100002300000010000e000c02683208687474702f312e31000500050100000000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+		},
+		"Edge": {
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+				helloHex:     `010000bd030358a3c9bf05f734842e189fb6ce653b67b846e990bc1fc5fb8c397874d06020f1000038c02cc02bc030c02f009f009ec024c023c028c027c00ac009c014c01300390033009d009c003d003c0035002f000a006a00400038003200130100005c000500050100000000000a00080006001d00170018000b00020100000d00140012040105010201040305030203020206010603002300000010000e000c02683208687474702f312e310017000055000006000100020002ff01000100`,
+				interception: false,
+			},
+		},
+		"Safari": {
+			{
+				userAgent:    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/602.4.8 (KHTML, like Gecko) Version/10.0.3 Safari/602.4.8",
+				helloHex:     `010000d2030358a295b513c8140c6ff880f4a8a73cc830ed2dab2c4f2068eb365228d828732e00002600ffc02cc02bc024c023c00ac009c030c02fc028c027c014c013009d009c003d003c0035002f010000830000000e000c0000096c6f63616c686f7374000a00080006001700180019000b00020100000d00120010040102010501060104030203050306033374000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e310005000501000000000012000000170000`,
+				interception: false,
+			},
+			{
+				// I think this was iOS 11 beta
+				userAgent:    "Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.28 (KHTML, like Gecko) Version/11.0 Mobile/15A5318g Safari/604.1",
+				helloHex:     `010000e10303be294e11847ba01301e0bb6129f4a0d66344602141a8f0a1ab0750a1db145755000028c02cc02bc024c023cca9c00ac009c030c02fc028c027cca8c014c013009d009c003d003c0035002f01000090ff0100010000000014001200000f66696e6572706978656c732e636f6d00170000000d00140012040308040401050308050501080606010201000500050100000000337400000012000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e31000b00020100000a00080006001d00170018`,
+				interception: false,
+			},
+			{
+				// iOS 11 stable
+				userAgent:    "Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/11.0 Mobile/15A372 Safari/604.1",
+				helloHex:     `010000dc030327fafb16708fcbe489fda332260d32b1a22bea6672a72b5e61d7b9963df1b10d000028c02cc02bc024c023c00ac009cca9c030c02fc028c027c014c013cca8009d009c003d003c0035002f0100008bff010001000000000f000d00000a6d69746d2e776174636800170000000d00140012040308040401050308050501080606010201000500050100000000337400000012000000100030002e0268320568322d31360568322d31350568322d313408737064792f332e3106737064792f3308687474702f312e31000b00020100000a00080006001d00170018`,
+				interception: false,
+			},
+		},
+		"Tor": {
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0",
+				helloHex:     `010000a40303137f05d4151f2d9095aee4254416d9dce73d6a1d857e8097ea20d021c04a7a81000016c02bc02fc00ac009c013c01400330039002f0035000a0100006500000014001200000f66696e6572706978656c732e636f6dff01000100000a00080006001700180019000b00020100337400000010000b000908687474702f312e31000500050100000000000d001600140401050106010201040305030603020304020202`,
+				interception: false,
+			},
+			{
+				userAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0",
+				helloHex:     `010000b4030322e1f3aff4c37caba303c2ce53ba1689b3e70117a46f413d44f70a74cb6a496100001ec02bc02fcca9cca8c02cc030c00ac009c013c01400330039002f0035000a0100006d00000014001200000f66696e6572706978656c732e636f6d00170000ff01000100000a000a0008001d001700180019000b000201000010000b000908687474702f312e31000500050100000000ff030000000d0018001604030503060308040805080604010501060102030201`,
+				interception: false,
+			},
+		},
+		"Other": { // these are either non-browser clients or intercepted client hellos
+			{
+				// openssl s_client (OpenSSL 0.9.8zh 14 Jan 2016) - NOT an interception, but not a browser either
+				helloHex: `0100012b03035d385236b8ca7b7946fa0336f164e76bf821ed90e8de26d97cc677671b6f36380000acc030c02cc028c024c014c00a00a500a300a1009f006b006a0069006800390038003700360088008700860085c032c02ec02ac026c00fc005009d003d00350084c02fc02bc027c023c013c00900a400a200a0009e00670040003f003e0033003200310030009a0099009800970045004400430042c031c02dc029c025c00ec004009c003c002f009600410007c011c007c00cc00200050004c012c008001600130010000dc00dc003000a00ff0201000055000b000403000102000a001c001a00170019001c001b0018001a0016000e000d000b000c0009000a00230000000d0020001e060106020603050105020503040104020403030103020303020102020203000f000101`,
+				// NOTE: This test case is not actually an interception, but s_client is not a browser
+				// or any client we support MITM checking for, either. Since it advertises heartbeat,
+				// our heuristics still flag it as a MITM.
+				interception: true,
+			},
+			{
+				// curl 7.51.0 (x86_64-apple-darwin16.0) libcurl/7.51.0 SecureTransport zlib/1.2.8
+				userAgent:    "curl/7.51.0",
+				helloHex:     `010000a6030358a28c73a71bdfc1f09dee13fecdc58805dcce42ac44254df548f14645f7dc2c00004400ffc02cc02bc024c023c00ac009c008c030c02fc028c027c014c013c012009f009e006b0067003900330016009d009c003d003c0035002f000a00af00ae008d008c008b01000039000a00080006001700180019000b00020100000d00120010040102010501060104030203050306030005000501000000000012000000170000`,
+				interception: false,
+			},
+			{
+				// Avast 17.1.2286 (Feb. 2017) on Windows 10 x64 build 14393, intercepting Edge
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+				helloHex:     `010000ce0303b418fdc4b6cf6436a5e2bfb06b96ed5faa7285c20c7b49341a78be962a9dc40000003ac02cc02bc030c02f009f009ec024c023c028c027c00ac009c014c01300390033009d009c003d003c0035002f000a006a004000380032001300ff0100006b00000014001200000f66696e6572706978656c732e636f6d000b000403000102000a00080006001d0017001800230000000d001400120401050102010403050302030202060106030005000501000000000010000e000c02683208687474702f312e310016000000170000`,
+				interception: true,
+			},
+			{
+				// Kaspersky Internet Security 17.0.0.611 on Windows 10 x64 build 14393, intercepting Edge
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+				helloHex:     `010000eb030361ce302bf4b0d5adf1ff30b2cf433c4a4b68f33e07b2651695e7ae6ec3cf126400003ac02cc02bc030c02f009f009ec024c023c028c027c00ac009c014c01300390033009d009c003d003c0035002f000a006a004000380032001300ff0100008800000014001200000f66696e6572706978656c732e636f6d000b000403000102000a001c001a00170019001c001b0018001a0016000e000d000b000c0009000a00230000000d0020001e060106020603050105020503040104020403030103020303020102020203000500050100000000000f0001010010000e000c02683208687474702f312e31`,
+				interception: true,
+			},
+			{
+				// Kaspersky Internet Security 17.0.0.611 on Windows 10 x64 build 14393, intercepting Firefox 51
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0",
+				helloHex:     `010001fc0303768e3f9ea75194c7cb03d23e8e6371b95fb696d339b797be57a634309ec98a42200f2a7554098364b7f05d21a8c7f43f31a893a4fc5670051020408c8e4dc234dd001cc02bc02fc02cc030c00ac009c013c01400330039002f0035000a00ff0100019700000014001200000f66696e6572706978656c732e636f6d000b000403000102000a001c001a00170019001c001b0018001a0016000e000d000b000c0009000a00230078bf4e244d4de3d53c6331edda9672dfc4a17aae92b671e86da1368b1b5ae5324372817d8f3b7ffe1a7a1537a5049b86cd7c44863978c1e615b005942755da20fc3a4e34a16f78034aa3b1cffcef95f81a0995c522a53b0e95a4f98db84c43359d93d8647b2de2a69f3ebdcfc6bca452730cbd00179226dedf000d0020001e060106020603050105020503040104020403030103020303020102020203000500050100000000000f0001010010000e000c02683208687474702f312e3100150093000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000`,
+				interception: true,
+			},
+			{
+				// Kaspersky Internet Security 17.0.0.611 on Windows 10 x64 build 14393, intercepting Chrome 56
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
+				helloHex:     `010000c903033481e7af24e647ba5a79ec97e9264c1a1f990cf842f50effe22be52130d5af82000018c02bc02fc02cc030c013c014009c009d002f0035000a00ff0100008800000014001200000f66696e6572706978656c732e636f6d000b000403000102000a001c001a00170019001c001b0018001a0016000e000d000b000c0009000a00230000000d0020001e060106020603050105020503040104020403030103020303020102020203000500050100000000000f0001010010000e000c02683208687474702f312e31`,
+				interception: true,
+			},
+			{
+				// AVG 17.1.3006 (build 17.1.3354.20) on Windows 10 x64 build 14393, intercepting Edge
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+				helloHex:     `010000ca0303fd83091207161eca6b4887db50587109c50e463beb190362736b1fcf9e05f807000036c02cc02bc030c02f009f009ec024c023c028c027c00ac009c014c01300390033009d009c003d003c0035002f006a00400038003200ff0100006b00000014001200000f66696e6572706978656c732e636f6d000b000403000102000a00080006001d0017001800230000000d001400120401050102010403050302030202060106030005000501000000000010000e000c02683208687474702f312e310016000000170000`,
+				interception: true,
+			},
+			{
+				// IE 11 on Windows 7, this connection was intercepted by Blue Coat
+				// no sensible User-Agent value, since Blue Coat changes it to something super generic
+				// By the way, here's another reason we hate Blue Coat: they break TLS 1.3:
+				// https://twitter.com/FiloSottile/status/835269932929667072
+				helloHex:     `010000b1030358a3f3bae627f464da8cb35976b88e9119640032d41e62a107d608ed8d3e62b9000034c028c027c014c013009f009e009d009cc02cc02bc024c023c00ac009003d003c0035002f006a004000380032000a0013000500040100005400000014001200000f66696e6572706978656c732e636f6d000500050100000000000a00080006001700180019000b00020100000d0014001206010603040105010201040305030203020200170000ff01000100`,
+				interception: true,
+				reqHeaders:   http.Header{"X-Bluecoat-Via": {"66808702E9A2CF4"}}, // actual field name would be "X-BlueCoat-Via" but Go canonicalizes field names
+			},
+			{
+				// Firefox 51.0.1 being intercepted by burp 1.7.17
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0",
+				helloHex:     `010000d8030358a92f4daca95acc2f6a10a9c50d736135eae39406d3090238464540d482677600003ac023c027003cc025c02900670040c009c013002fc004c00e00330032c02bc02f009cc02dc031009e00a2c008c012000ac003c00d0016001300ff01000075000a0034003200170001000300130015000600070009000a0018000b000c0019000d000e000f001000110002001200040005001400080016000b00020100000d00180016060306010503050104030401040202030201020201010000001700150000126a61677561722e6b796877616e612e6f7267`,
+				interception: true,
+			},
+			{
+				// Chrome 56 on Windows 10 being intercepted by Fortigate (on some public school network); note: I had to enable TLS 1.0 for this test (proxy was issuing a SHA-1 cert to client)
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
+				helloHex:     `010000e5030158ac612125c83bae95282113b2a4c572cf613c160d234350fb6d0ddce879ffec000064003300320039003800160013c013c009c014c00ac012c008002f0035000a00150012003d003c00670040006b006ac011c0070096009a009900410084004500440088008700ba00be00bd00c000c400c3c03cc044c042c03dc045c04300090005000400ff01000058000a003600340000000100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019000b0002010000000014001200000f66696e6572706978656c732e636f6d`,
+				interception: true,
+			},
+			{
+				// IE 11 on Windows 10, intercepted by Fortigate (same firewall as above)
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko",
+				helloHex:     `010000e5030158ac634c5278d7b17421f23a64cc91d68c470c6b247322fe867ba035b373d05c000064003300320039003800160013c013c009c014c00ac012c008002f0035000a00150012003d003c00670040006b006ac011c0070096009a009900410084004500440088008700ba00be00bd00c000c400c3c03cc044c042c03dc045c04300090005000400ff01000058000a003600340000000100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019000b0002010000000014001200000f66696e6572706978656c732e636f6d`,
+				interception: true,
+			},
+			{
+				// Edge 38.14393.0.0 on Windows 10, intercepted by Fortigate (same as above)
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+				helloHex:     `010000e5030158ac6421a45794b8ade6a0ac6c910cde0f99c49bb1ba737b88638ec8dcf0d077000064003300320039003800160013c013c009c014c00ac012c008002f0035000a00150012003d003c00670040006b006ac011c0070096009a009900410084004500440088008700ba00be00bd00c000c400c3c03cc044c042c03dc045c04300090005000400ff01000058000a003600340000000100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019000b0002010000000014001200000f66696e6572706978656c732e636f6d`,
+				interception: true,
+			},
+			{
+				// Firefox 50.0.1 on Windows 10, intercepted by Fortigate (same as above)
+				userAgent:    "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:50.0) Gecko/20100101 Firefox/50.0",
+				helloHex:     `010000e5030158ac64e40495e77b7baf2031281451620bfe354b0c37521ebc0a40f5dc0c0cb6000064003300320039003800160013c013c009c014c00ac012c008002f0035000a00150012003d003c00670040006b006ac011c0070096009a009900410084004500440088008700ba00be00bd00c000c400c3c03cc044c042c03dc045c04300090005000400ff01000058000a003600340000000100020003000400050006000700080009000a000b000c000d000e000f0010001100120013001400150016001700180019000b0002010000000014001200000f66696e6572706978656c732e636f6d`,
+				interception: true,
+			},
+		},
+	}
+
+	for client, chs := range clientHellos {
+		for i, ch := range chs {
+			hello, err := hex.DecodeString(ch.helloHex)
+			if err != nil {
+				t.Errorf("[%s] Test %d: Error decoding ClientHello: %v", client, i, err)
+				continue
+			}
+			parsed := parseRawClientHello(hello)
+
+			isChrome := parsed.looksLikeChrome()
+			isFirefox := parsed.looksLikeFirefox()
+			isSafari := parsed.looksLikeSafari()
+			isEdge := parsed.looksLikeEdge()
+			isTor := parsed.looksLikeTor()
+
+			// we want each of the heuristic functions to be as
+			// exclusive but as low-maintenance as possible;
+			// in other words, if one returns true, the others
+			// should return false, with as little logic as possible,
+			// but with enough logic to force TLS proxies to do a
+			// good job preserving characterstics of the handshake.
+			if (isChrome && (isFirefox || isSafari || isEdge || isTor)) ||
+				(isFirefox && (isChrome || isSafari || isEdge || isTor)) ||
+				(isSafari && (isChrome || isFirefox || isEdge || isTor)) ||
+				(isEdge && (isChrome || isFirefox || isSafari || isTor)) ||
+				(isTor && (isChrome || isFirefox || isSafari || isEdge)) {
+				t.Errorf("[%s] Test %d: Multiple fingerprinting functions matched: "+
+					"Chrome=%v Firefox=%v Safari=%v Edge=%v Tor=%v\n\tparsed hello dec: %+v\n\tparsed hello hex: %#x\n",
+					client, i, isChrome, isFirefox, isSafari, isEdge, isTor, parsed, parsed)
+			}
+
+			// test the handler and detection results
+			var got, checked bool
+			want := ch.interception
+			handler := &tlsHandler{
+				next: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+					got, checked = r.Context().Value(MitmCtxKey).(bool)
+				}),
+				listener: newTLSListener(nil, nil),
+			}
+			handler.listener.helloInfos[""] = parsed
+			w := httptest.NewRecorder()
+			r, err := http.NewRequest("GET", "/", nil)
+			if err != nil {
+				t.Fatal(err)
+			}
+			r.Header.Set("User-Agent", ch.userAgent)
+			if ch.reqHeaders != nil {
+				for field, values := range ch.reqHeaders {
+					r.Header[field] = values // NOTE: field names not standardized when setting directly like this!
+				}
+			}
+			handler.ServeHTTP(w, r)
+			if got != want {
+				t.Errorf("[%s] Test %d: Expected MITM=%v but got %v (type assertion OK (checked)=%v)",
+					client, i, want, got, checked)
+				t.Errorf("[%s] Test %d: Looks like Chrome=%v Firefox=%v Safari=%v Edge=%v Tor=%v\n\tparsed hello dec: %+v\n\tparsed hello hex: %#x\n",
+					client, i, isChrome, isFirefox, isSafari, isEdge, isTor, parsed, parsed)
+			}
+		}
+	}
+}
+
+func TestGetVersion(t *testing.T) {
+	for i, test := range []struct {
+		UserAgent    string
+		SoftwareName string
+		Version      float64
+	}{
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0",
+			SoftwareName: "Firefox",
+			Version:      45.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 6.1; rv:45.0) Gecko/20100101 Firefox/45.0 more_stuff_here",
+			SoftwareName: "Firefox",
+			Version:      45.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Safari",
+			Version:      537.36,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Chrome",
+			Version:      51.0270479,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "Mozilla",
+			Version:      5.0,
+		},
+		{
+			UserAgent:    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.79 Safari/537.36 Edge/14.14393",
+			SoftwareName: "curl",
+			Version:      -1,
+		},
+	} {
+		actual := getVersion(test.UserAgent, test.SoftwareName)
+		if actual != test.Version {
+			t.Errorf("Test [%d]: Expected version=%f, got version=%f for %s in '%s'",
+				i, test.Version, actual, test.SoftwareName, test.UserAgent)
+		}
+	}
+}

caddyhttp/httpserver/path.go

@@ -0,0 +1,67 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpserver
+
+import (
+	"net/http"
+	"path"
+	"strings"
+)
+
+// Path represents a URI path. It should usually be
+// set to the value of a request path.
+type Path string
+
+// Matches checks to see if base matches p. The correct
+// usage of this method sets p as the request path, and
+// base as a Caddyfile (user-defined) rule path.
+//
+// Path matching will probably not always be a direct
+// comparison; this method assures that paths can be
+// easily and consistently matched.
+//
+// Multiple slashes are collapsed/merged. See issue #1859.
+func (p Path) Matches(base string) bool {
+	if base == "/" || base == "" {
+		return true
+	}
+
+	// sanitize the paths for comparison, very important
+	// (slightly lossy if the base path requires multiple
+	// consecutive forward slashes, since those will be merged)
+	pHasTrailingSlash := strings.HasSuffix(string(p), "/")
+	baseHasTrailingSlash := strings.HasSuffix(base, "/")
+	p = Path(path.Clean(string(p)))
+	base = path.Clean(base)
+	if pHasTrailingSlash {
+		p += "/"
+	}
+	if baseHasTrailingSlash {
+		base += "/"
+	}
+
+	if CaseSensitivePath {
+		return strings.HasPrefix(string(p), base)
+	}
+	return strings.HasPrefix(strings.ToLower(string(p)), strings.ToLower(base))
+}
+
+// PathMatcher is a Path RequestMatcher.
+type PathMatcher string
+
+// Match satisfies RequestMatcher.
+func (p PathMatcher) Match(r *http.Request) bool {
+	return Path(r.URL.Path).Matches(string(p))
+}

caddyhttp/httpserver/path_test.go

@@ -0,0 +1,146 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpserver
+
+import "testing"
+
+func TestPathMatches(t *testing.T) {
+	for i, testcase := range []struct {
+		reqPath         Path
+		rulePath        string // or "base path" as in Caddyfile docs
+		shouldMatch     bool
+		caseInsensitive bool
+	}{
+		{
+			reqPath:     "/",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "/foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/foobar",
+			rulePath:    "/foo/bar",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/foo/",
+			rulePath:    "/foo/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/Foobar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+
+			reqPath:     "/FooBar",
+			rulePath:    "/Foo",
+			shouldMatch: true,
+		},
+		{
+			reqPath:         "/foobar",
+			rulePath:        "/FooBar",
+			shouldMatch:     true,
+			caseInsensitive: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "/", // a lone forward slash means to match all requests (see issue #1645) - many future test cases related to this issue
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "foobar.php",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/foo/bar",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "/no/leading/slash",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "no/leading/slash",
+			rulePath:    "",
+			shouldMatch: true,
+		},
+		{
+			// see issue #1859
+			reqPath:     "//double-slash",
+			rulePath:    "/double-slash",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/double//slash",
+			rulePath:    "/double/slash",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "//more/double//slashes",
+			rulePath:    "/more/double/slashes",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/path/../traversal",
+			rulePath:    "/traversal",
+			shouldMatch: true,
+		},
+		{
+			reqPath:     "/path/../traversal",
+			rulePath:    "/path",
+			shouldMatch: false,
+		},
+		{
+			reqPath:     "/keep-slashes/http://something/foo/bar",
+			rulePath:    "/keep-slashes/http://something",
+			shouldMatch: true,
+		},
+	} {
+		CaseSensitivePath = !testcase.caseInsensitive
+		if got, want := testcase.reqPath.Matches(testcase.rulePath), testcase.shouldMatch; got != want {
+			t.Errorf("Test %d: For request path '%s' and base path '%s': expected %v, got %v",
+				i, testcase.reqPath, testcase.rulePath, want, got)
+		}
+	}
+}

caddyhttp/httpserver/plugin.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
@@ -6,26 +20,31 @@ import (
 	"log"
 	"net"
 	"net/url"
+	"os"
+	"path/filepath"
 	"strings"
 	"time"
 
 	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyfile"
+	"github.com/mholt/caddy/caddyhttp/staticfiles"
 	"github.com/mholt/caddy/caddytls"
 )
 
 const serverType = "http"
 
 func init() {
+	flag.StringVar(&HTTPPort, "http-port", HTTPPort, "Default port to use for HTTP")
+	flag.StringVar(&HTTPSPort, "https-port", HTTPSPort, "Default port to use for HTTPS")
 	flag.StringVar(&Host, "host", DefaultHost, "Default host")
 	flag.StringVar(&Port, "port", DefaultPort, "Default port")
 	flag.StringVar(&Root, "root", DefaultRoot, "Root path of default site")
-	flag.DurationVar(&GracefulTimeout, "grace", 5*time.Second, "Maximum duration of graceful shutdown") // TODO
+	flag.DurationVar(&GracefulTimeout, "grace", 5*time.Second, "Maximum duration of graceful shutdown")
 	flag.BoolVar(&HTTP2, "http2", true, "Use HTTP/2")
 	flag.BoolVar(&QUIC, "quic", false, "Use experimental QUIC")
 
 	caddy.RegisterServerType(serverType, caddy.ServerType{
-		Directives: directives,
+		Directives: func() []string { return directives },
 		DefaultInput: func() caddy.Input {
 			if Port == DefaultPort && Host != "" {
 				// by leaving the port blank in this case we give auto HTTPS
@@ -43,15 +62,42 @@ func init() {
 		NewContext: newContext,
 	})
 	caddy.RegisterCaddyfileLoader("short", caddy.LoaderFunc(shortCaddyfileLoader))
+	caddy.RegisterParsingCallback(serverType, "root", hideCaddyfile)
 	caddy.RegisterParsingCallback(serverType, "tls", activateHTTPS)
 	caddytls.RegisterConfigGetter(serverType, func(c *caddy.Controller) *caddytls.Config { return GetConfig(c).TLS })
 }
 
-func newContext() caddy.Context {
-	return &httpContext{keysToSiteConfigs: make(map[string]*SiteConfig)}
+// hideCaddyfile hides the source/origin Caddyfile if it is within the
+// site root. This function should be run after parsing the root directive.
+func hideCaddyfile(cctx caddy.Context) error {
+	ctx := cctx.(*httpContext)
+	for _, cfg := range ctx.siteConfigs {
+		// if no Caddyfile exists exit.
+		if cfg.originCaddyfile == "" {
+			return nil
+		}
+		absRoot, err := filepath.Abs(cfg.Root)
+		if err != nil {
+			return err
+		}
+		absOriginCaddyfile, err := filepath.Abs(cfg.originCaddyfile)
+		if err != nil {
+			return err
+		}
+		if strings.HasPrefix(absOriginCaddyfile, absRoot) {
+			cfg.HiddenFiles = append(cfg.HiddenFiles, filepath.ToSlash(strings.TrimPrefix(absOriginCaddyfile, absRoot)))
+		}
+	}
+	return nil
+}
+
+func newContext(inst *caddy.Instance) caddy.Context {
+	return &httpContext{instance: inst, keysToSiteConfigs: make(map[string]*SiteConfig)}
 }
 
 type httpContext struct {
+	instance *caddy.Instance
+
 	// keysToSiteConfigs maps an address at the top of a
 	// server block (a "key") to its SiteConfig. Not all
 	// SiteConfigs will be represented here, only ones
@@ -71,12 +117,14 @@ func (h *httpContext) saveConfig(key string, cfg *SiteConfig) {
 // executing directives and otherwise prepares the directives to
 // be parsed and executed.
 func (h *httpContext) InspectServerBlocks(sourceFile string, serverBlocks []caddyfile.ServerBlock) ([]caddyfile.ServerBlock, error) {
+	siteAddrs := make(map[string]string)
+
 	// For each address in each server block, make a new config
 	for _, sb := range serverBlocks {
 		for _, key := range sb.Keys {
 			key = strings.ToLower(key)
 			if _, dup := h.keysToSiteConfigs[key]; dup {
-				return serverBlocks, fmt.Errorf("duplicate site address: %s", key)
+				return serverBlocks, fmt.Errorf("duplicate site key: %s", key)
 			}
 			addr, err := standardizeAddress(key)
 			if err != nil {
@@ -92,12 +140,48 @@ func (h *httpContext) InspectServerBlocks(sourceFile string, serverBlocks []cadd
 				addr.Port = Port
 			}
 
+			// Make sure the adjusted site address is distinct
+			addrCopy := addr // make copy so we don't disturb the original, carefully-parsed address struct
+			if addrCopy.Port == "" && Port == DefaultPort {
+				addrCopy.Port = Port
+			}
+			addrStr := strings.ToLower(addrCopy.String())
+			if otherSiteKey, dup := siteAddrs[addrStr]; dup {
+				err := fmt.Errorf("duplicate site address: %s", addrStr)
+				if (addrCopy.Host == Host && Host != DefaultHost) ||
+					(addrCopy.Port == Port && Port != DefaultPort) {
+					err = fmt.Errorf("site defined as %s is a duplicate of %s because of modified "+
+						"default host and/or port values (usually via -host or -port flags)", key, otherSiteKey)
+				}
+				return serverBlocks, err
+			}
+			siteAddrs[addrStr] = key
+
+			// If default HTTP or HTTPS ports have been customized,
+			// make sure the ACME challenge ports match
+			var altHTTPPort, altTLSSNIPort string
+			if HTTPPort != DefaultHTTPPort {
+				altHTTPPort = HTTPPort
+			}
+			if HTTPSPort != DefaultHTTPSPort {
+				altTLSSNIPort = HTTPSPort
+			}
+
+			// Make our caddytls.Config, which has a pointer to the
+			// instance's certificate cache and enough information
+			// to use automatic HTTPS when the time comes
+			caddytlsConfig := caddytls.NewConfig(h.instance)
+			caddytlsConfig.Hostname = addr.Host
+			caddytlsConfig.AltHTTPPort = altHTTPPort
+			caddytlsConfig.AltTLSSNIPort = altTLSSNIPort
+
 			// Save the config to our master list, and key it for lookups
 			cfg := &SiteConfig{
-				Addr:        addr,
-				Root:        Root,
-				TLS:         &caddytls.Config{Hostname: addr.Host},
-				HiddenFiles: []string{sourceFile},
+				Addr:            addr,
+				Root:            Root,
+				TLS:             caddytlsConfig,
+				originCaddyfile: sourceFile,
+				IndexPages:      staticfiles.DefaultIndexPages,
 			}
 			h.saveConfig(key, cfg)
 		}
@@ -127,7 +211,7 @@ func (h *httpContext) MakeServers() ([]caddy.Server, error) {
 		if !cfg.TLS.Enabled {
 			continue
 		}
-		if cfg.Addr.Port == "80" || cfg.Addr.Scheme == "http" {
+		if cfg.Addr.Port == HTTPPort || cfg.Addr.Scheme == "http" {
 			cfg.TLS.Enabled = false
 			log.Printf("[WARNING] TLS disabled for %s", cfg.Addr)
 		} else if cfg.Addr.Scheme == "" {
@@ -138,11 +222,11 @@ func (h *httpContext) MakeServers() ([]caddy.Server, error) {
 			// is incorrect for this site.
 			cfg.Addr.Scheme = "https"
 		}
-		if cfg.Addr.Port == "" {
+		if cfg.Addr.Port == "" && ((!cfg.TLS.Manual && !cfg.TLS.SelfSigned) || cfg.TLS.OnDemand) {
 			// this is vital, otherwise the function call below that
 			// sets the listener address will use the default port
 			// instead of 443 because it doesn't know about TLS.
-			cfg.Addr.Port = "443"
+			cfg.Addr.Port = HTTPSPort
 		}
 	}
 
@@ -170,14 +254,16 @@ func (h *httpContext) MakeServers() ([]caddy.Server, error) {
 // new, empty one will be created.
 func GetConfig(c *caddy.Controller) *SiteConfig {
 	ctx := c.Context().(*httpContext)
-	if cfg, ok := ctx.keysToSiteConfigs[c.Key]; ok {
+	key := strings.ToLower(c.Key)
+	if cfg, ok := ctx.keysToSiteConfigs[key]; ok {
 		return cfg
 	}
 	// we should only get here during tests because directive
 	// actions typically skip the server blocks where we make
 	// the configs
-	ctx.saveConfig(c.Key, &SiteConfig{Root: Root, TLS: new(caddytls.Config)})
-	return GetConfig(c)
+	cfg := &SiteConfig{Root: Root, TLS: new(caddytls.Config), IndexPages: staticfiles.DefaultIndexPages}
+	ctx.saveConfig(key, cfg)
+	return cfg
 }
 
 // shortCaddyfileLoader loads a Caddyfile if positional arguments are
@@ -206,11 +292,11 @@ func groupSiteConfigsByListenAddr(configs []*SiteConfig) (map[string][]*SiteConf
 	groups := make(map[string][]*SiteConfig)
 
 	for _, conf := range configs {
-		if caddy.IsLoopback(conf.Addr.Host) && conf.ListenHost == "" {
-			// special case: one would not expect a site served
-			// at loopback to be connected to from the outside.
-			conf.ListenHost = conf.Addr.Host
-		}
+		// We would add a special case here so that localhost addresses
+		// bind to 127.0.0.1 if conf.ListenHost is not already set, which
+		// would prevent outsiders from even connecting; but that was problematic:
+		// https://caddy.community/t/wildcard-virtual-domains-with-wildcard-roots/221/5?u=matt
+
 		if conf.Addr.Port == "" {
 			conf.Addr.Port = Port
 		}
@@ -225,11 +311,6 @@ func groupSiteConfigsByListenAddr(configs []*SiteConfig) (map[string][]*SiteConf
 	return groups, nil
 }
 
-// AddMiddleware adds a middleware to a site's middleware stack.
-func (sc *SiteConfig) AddMiddleware(m Middleware) {
-	sc.middleware = append(sc.middleware, m)
-}
-
 // Address represents a site address. It contains
 // the original input value, and the component
 // parts of an address. The component parts may be
@@ -246,7 +327,7 @@ func (a Address) String() string {
 	}
 	scheme := a.Scheme
 	if scheme == "" {
-		if a.Port == "443" {
+		if a.Port == HTTPSPort {
 			scheme = "https"
 		} else {
 			scheme = "http"
@@ -258,8 +339,8 @@ func (a Address) String() string {
 	}
 	s += a.Host
 	if a.Port != "" &&
-		((scheme == "https" && a.Port != "443") ||
-			(scheme == "http" && a.Port != "80")) {
+		((scheme == "https" && a.Port != DefaultHTTPSPort) ||
+			(scheme == "http" && a.Port != DefaultHTTPPort)) {
 		s += ":" + a.Port
 	}
 	if a.Path != "" {
@@ -278,12 +359,12 @@ func (a Address) VHost() string {
 }
 
 // standardizeAddress parses an address string into a structured format with separate
-// scheme, host, and port portions, as well as the original input string.
+// scheme, host, port, and path portions, as well as the original input string.
 func standardizeAddress(str string) (Address, error) {
 	input := str
 
 	// Split input into components (prepend with // to assert host by default)
-	if !strings.Contains(str, "//") {
+	if !strings.Contains(str, "//") && !strings.HasPrefix(str, "/") {
 		str = "//" + str
 	}
 	u, err := url.Parse(str)
@@ -303,9 +384,9 @@ func standardizeAddress(str string) (Address, error) {
 	// see if we can set port based off scheme
 	if port == "" {
 		if u.Scheme == "http" {
-			port = "80"
+			port = HTTPPort
 		} else if u.Scheme == "https" {
-			port = "443"
+			port = HTTPSPort
 		}
 	}
 
@@ -315,66 +396,154 @@ func standardizeAddress(str string) (Address, error) {
 	}
 
 	// error if scheme and port combination violate convention
-	if (u.Scheme == "http" && port == "443") || (u.Scheme == "https" && port == "80") {
+	if (u.Scheme == "http" && port == HTTPSPort) || (u.Scheme == "https" && port == HTTPPort) {
 		return Address{}, fmt.Errorf("[%s] scheme and port violate convention", input)
 	}
 
 	// standardize http and https ports to their respective port numbers
 	if port == "http" {
 		u.Scheme = "http"
-		port = "80"
+		port = HTTPPort
 	} else if port == "https" {
 		u.Scheme = "https"
-		port = "443"
+		port = HTTPSPort
 	}
 
 	return Address{Original: input, Scheme: u.Scheme, Host: host, Port: port, Path: u.Path}, err
 }
 
+// RegisterDevDirective splices name into the list of directives
+// immediately before another directive. This function is ONLY
+// for plugin development purposes! NEVER use it for a plugin
+// that you are not currently building. If before is empty,
+// the directive will be appended to the end of the list.
+//
+// It is imperative that directives execute in the proper
+// order, and hard-coding the list of directives guarantees
+// a correct, absolute order every time. This function is
+// convenient when developing a plugin, but it does not
+// guarantee absolute ordering. Multiple plugins registering
+// directives with this function will lead to non-
+// deterministic builds and buggy software.
+//
+// Directive names must be lower-cased and unique. Any errors
+// here are fatal, and even successful calls print a message
+// to stdout as a reminder to use it only in development.
+func RegisterDevDirective(name, before string) {
+	if name == "" {
+		fmt.Println("[FATAL] Cannot register empty directive name")
+		os.Exit(1)
+	}
+	if strings.ToLower(name) != name {
+		fmt.Printf("[FATAL] %s: directive name must be lowercase\n", name)
+		os.Exit(1)
+	}
+	for _, dir := range directives {
+		if dir == name {
+			fmt.Printf("[FATAL] %s: directive name already exists\n", name)
+			os.Exit(1)
+		}
+	}
+	if before == "" {
+		directives = append(directives, name)
+	} else {
+		var found bool
+		for i, dir := range directives {
+			if dir == before {
+				directives = append(directives[:i], append([]string{name}, directives[i:]...)...)
+				found = true
+				break
+			}
+		}
+		if !found {
+			fmt.Printf("[FATAL] %s: directive not found\n", before)
+			os.Exit(1)
+		}
+	}
+	msg := fmt.Sprintf("Registered directive '%s' ", name)
+	if before == "" {
+		msg += "at end of list"
+	} else {
+		msg += fmt.Sprintf("before '%s'", before)
+	}
+	fmt.Printf("[DEV NOTICE] %s\n", msg)
+}
+
 // directives is the list of all directives known to exist for the
 // http server type, including non-standard (3rd-party) directives.
 // The ordering of this list is important.
 var directives = []string{
 	// primitive actions that set up the fundamental vitals of each config
 	"root",
-	"tls",
+	"index",
 	"bind",
+	"limits",
+	"timeouts",
+	"tls",
 
 	// services/utilities, or other directives that don't necessarily inject handlers
-	"startup",
-	"shutdown",
+	"startup",  // TODO: Deprecate this directive
+	"shutdown", // TODO: Deprecate this directive
+	"on",
+	"request_id",
 	"realip", // github.com/captncraig/caddy-realip
 	"git",    // github.com/abiosoft/caddy-git
 
+	// directives that add listener middleware to the stack
+	"proxyprotocol", // github.com/mastercactapus/caddy-proxyprotocol
+
 	// directives that add middleware to the stack
+	"locale", // github.com/simia-tech/caddy-locale
 	"log",
-	"gzip",
-	"errors",
-	"minify",   // github.com/hacdias/caddy-minify
-	"ipfilter", // github.com/pyed/ipfilter
-	"search",   // github.com/pedronasser/caddy-search
-	"header",
-	"cors", // github.com/captncraig/cors/caddy
+	"cache", // github.com/nicolasazrak/caddy-cache
 	"rewrite",
-	"redir",
 	"ext",
-	"mime",
+	"gzip",
+	"header",
+	"errors",
+	"authz",        // github.com/casbin/caddy-authz
+	"filter",       // github.com/echocat/caddy-filter
+	"minify",       // github.com/hacdias/caddy-minify
+	"ipfilter",     // github.com/pyed/ipfilter
+	"ratelimit",    // github.com/xuqingfeng/caddy-rate-limit
+	"search",       // github.com/pedronasser/caddy-search
+	"expires",      // github.com/epicagency/caddy-expires
+	"forwardproxy", // github.com/caddyserver/forwardproxy
 	"basicauth",
-	"jwt",    // github.com/BTBurke/caddy-jwt
-	"jsonp",  // github.com/pschlump/caddy-jsonp
-	"upload", // blitznote.com/src/caddy.upload
+	"redir",
+	"status",
+	"cors",   // github.com/captncraig/cors/caddy
+	"nobots", // github.com/Xumeiquer/nobots
+	"mime",
+	"login",     // github.com/tarent/loginsrv/caddy
+	"reauth",    // github.com/freman/caddy-reauth
+	"jwt",       // github.com/BTBurke/caddy-jwt
+	"jsonp",     // github.com/pschlump/caddy-jsonp
+	"upload",    // blitznote.com/src/caddy.upload
+	"multipass", // github.com/namsral/multipass/caddy
 	"internal",
 	"pprof",
 	"expvar",
+	"push",
+	"datadog",    // github.com/payintech/caddy-datadog
+	"prometheus", // github.com/miekg/caddy-prometheus
+	"templates",
 	"proxy",
 	"fastcgi",
+	"cgi", // github.com/jung-kurt/caddy-cgi
 	"websocket",
+	"filemanager", // github.com/hacdias/filemanager/caddy/filemanager
+	"webdav",      // github.com/hacdias/caddy-webdav
 	"markdown",
-	"templates",
 	"browse",
-	"hugo",       // github.com/hacdias/caddy-hugo
-	"mailout",    // github.com/SchumacherFM/mailout
-	"prometheus", // github.com/miekg/caddy-prometheus
+	"jekyll",    // github.com/hacdias/filemanager/caddy/jekyll
+	"hugo",      // github.com/hacdias/filemanager/caddy/hugo
+	"mailout",   // github.com/SchumacherFM/mailout
+	"awses",     // github.com/miquella/caddy-awses
+	"awslambda", // github.com/coopernurse/caddy-awslambda
+	"grpc",      // github.com/pieterlouw/caddy-grpc
+	"gopkg",     // github.com/zikes/gopkg
+	"restic",    // github.com/restic/caddy
 }
 
 const (
@@ -384,6 +553,10 @@ const (
 	DefaultPort = "2015"
 	// DefaultRoot is the default root folder.
 	DefaultRoot = "."
+	// DefaultHTTPPort is the default port for HTTP.
+	DefaultHTTPPort = "80"
+	// DefaultHTTPSPort is the default port for HTTPS.
+	DefaultHTTPSPort = "443"
 )
 
 // These "soft defaults" are configurable by
@@ -406,4 +579,10 @@ var (
 
 	// QUIC indicates whether QUIC is enabled or not.
 	QUIC bool
+
+	// HTTPPort is the port to use for HTTP.
+	HTTPPort = DefaultHTTPPort
+
+	// HTTPSPort is the port to use for HTTPS.
+	HTTPSPort = DefaultHTTPSPort
 )

caddyhttp/httpserver/plugin_test.go

@@ -1,9 +1,24 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
 	"strings"
 	"testing"
 
+	"github.com/mholt/caddy"
 	"github.com/mholt/caddy/caddyfile"
 )
 
@@ -50,6 +65,7 @@ func TestStandardizeAddress(t *testing.T) {
 		{`https://host:443/path/foo`, "https", "host", "443", "/path/foo", false},
 		{`host:80/path`, "", "host", "80", "/path", false},
 		{`host:https/path`, "https", "host", "443", "/path", false},
+		{`/path`, "", "", "", "/path", false},
 	} {
 		actual, err := standardizeAddress(test.input)
 
@@ -121,7 +137,7 @@ func TestAddressString(t *testing.T) {
 func TestInspectServerBlocksWithCustomDefaultPort(t *testing.T) {
 	Port = "9999"
 	filename := "Testfile"
-	ctx := newContext().(*httpContext)
+	ctx := newContext(&caddy.Instance{Storage: make(map[interface{}]interface{})}).(*httpContext)
 	input := strings.NewReader(`localhost`)
 	sblocks, err := caddyfile.Parse(filename, input, nil)
 	if err != nil {
@@ -136,3 +152,115 @@ func TestInspectServerBlocksWithCustomDefaultPort(t *testing.T) {
 		t.Errorf("Expected the port on the address to be set, but got: %#v", addr)
 	}
 }
+
+// See discussion on PR #2015
+func TestInspectServerBlocksWithAdjustedAddress(t *testing.T) {
+	Port = DefaultPort
+	Host = "example.com"
+	filename := "Testfile"
+	ctx := newContext(&caddy.Instance{Storage: make(map[interface{}]interface{})}).(*httpContext)
+	input := strings.NewReader("example.com {\n}\n:2015 {\n}")
+	sblocks, err := caddyfile.Parse(filename, input, nil)
+	if err != nil {
+		t.Fatalf("Expected no error setting up test, got: %v", err)
+	}
+	_, err = ctx.InspectServerBlocks(filename, sblocks)
+	if err == nil {
+		t.Fatalf("Expected an error because site definitions should overlap, got: %v", err)
+	}
+}
+
+func TestInspectServerBlocksCaseInsensitiveKey(t *testing.T) {
+	filename := "Testfile"
+	ctx := newContext(&caddy.Instance{Storage: make(map[interface{}]interface{})}).(*httpContext)
+	input := strings.NewReader("localhost {\n}\nLOCALHOST {\n}")
+	sblocks, err := caddyfile.Parse(filename, input, nil)
+	if err != nil {
+		t.Fatalf("Expected no error setting up test, got: %v", err)
+	}
+	_, err = ctx.InspectServerBlocks(filename, sblocks)
+	if err == nil {
+		t.Error("Expected an error because keys on this server type are case-insensitive (so these are duplicated), but didn't get an error")
+	}
+}
+
+func TestGetConfig(t *testing.T) {
+	// case insensitivity for key
+	con := caddy.NewTestController("http", "")
+	con.Key = "foo"
+	cfg := GetConfig(con)
+	con.Key = "FOO"
+	cfg2 := GetConfig(con)
+	if cfg != cfg2 {
+		t.Errorf("Expected same config using same key with different case; got %p and %p", cfg, cfg2)
+	}
+
+	// make sure different key returns different config
+	con.Key = "foobar"
+	cfg3 := GetConfig(con)
+	if cfg == cfg3 {
+		t.Errorf("Expected different configs using when key is different; got %p and %p", cfg, cfg3)
+	}
+}
+
+func TestDirectivesList(t *testing.T) {
+	for i, dir1 := range directives {
+		if dir1 == "" {
+			t.Errorf("directives[%d]: empty directive name", i)
+			continue
+		}
+		if got, want := dir1, strings.ToLower(dir1); got != want {
+			t.Errorf("directives[%d]: %s should be lower-cased", i, dir1)
+			continue
+		}
+		for j := i + 1; j < len(directives); j++ {
+			dir2 := directives[j]
+			if dir1 == dir2 {
+				t.Errorf("directives[%d] (%s) is a duplicate of directives[%d] (%s)",
+					j, dir2, i, dir1)
+			}
+		}
+	}
+}
+
+func TestContextSaveConfig(t *testing.T) {
+	ctx := newContext(&caddy.Instance{Storage: make(map[interface{}]interface{})}).(*httpContext)
+	ctx.saveConfig("foo", new(SiteConfig))
+	if _, ok := ctx.keysToSiteConfigs["foo"]; !ok {
+		t.Error("Expected config to be saved, but it wasn't")
+	}
+	if got, want := len(ctx.siteConfigs), 1; got != want {
+		t.Errorf("Expected len(siteConfigs) == %d, but was %d", want, got)
+	}
+	ctx.saveConfig("Foobar", new(SiteConfig))
+	if _, ok := ctx.keysToSiteConfigs["foobar"]; ok {
+		t.Error("Did not expect to get config with case-insensitive key, but did")
+	}
+	if got, want := len(ctx.siteConfigs), 2; got != want {
+		t.Errorf("Expected len(siteConfigs) == %d, but was %d", want, got)
+	}
+}
+
+// Test to make sure we are correctly hiding the Caddyfile
+func TestHideCaddyfile(t *testing.T) {
+	ctx := newContext(&caddy.Instance{Storage: make(map[interface{}]interface{})}).(*httpContext)
+	ctx.saveConfig("test", &SiteConfig{
+		Root:            Root,
+		originCaddyfile: "Testfile",
+	})
+	err := hideCaddyfile(ctx)
+	if err != nil {
+		t.Fatalf("Failed to hide Caddyfile, got: %v", err)
+		return
+	}
+	if len(ctx.siteConfigs[0].HiddenFiles) == 0 {
+		t.Fatal("Failed to add Caddyfile to HiddenFiles.")
+		return
+	}
+	for _, file := range ctx.siteConfigs[0].HiddenFiles {
+		if file == "/Testfile" {
+			return
+		}
+	}
+	t.Fatal("Caddyfile missing from HiddenFiles")
+}

caddyhttp/httpserver/recorder.go

@@ -1,10 +1,24 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
-	"bufio"
-	"errors"
-	"net"
+	"bytes"
+	"io"
 	"net/http"
+	"sync"
 	"time"
 )
 
@@ -21,24 +35,22 @@ import (
 //
 // Beware when accessing the Replacer value; it may be nil!
 type ResponseRecorder struct {
-	http.ResponseWriter
+	*ResponseWriterWrapper
 	Replacer Replacer
 	status   int
 	size     int
 	start    time.Time
 }
 
-// NewResponseRecorder makes and returns a new responseRecorder,
-// which captures the HTTP Status code from the ResponseWriter
-// and also the length of the response body written through it.
+// NewResponseRecorder makes and returns a new ResponseRecorder.
 // Because a status is not set unless WriteHeader is called
 // explicitly, this constructor initializes with a status code
 // of 200 to cover the default case.
 func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 	return &ResponseRecorder{
-		ResponseWriter: w,
-		status:         http.StatusOK,
-		start:          time.Now(),
+		ResponseWriterWrapper: &ResponseWriterWrapper{ResponseWriter: w},
+		status:                http.StatusOK,
+		start:                 time.Now(),
 	}
 }
 
@@ -46,53 +58,204 @@ func NewResponseRecorder(w http.ResponseWriter) *ResponseRecorder {
 // underlying ResponseWriter's WriteHeader method.
 func (r *ResponseRecorder) WriteHeader(status int) {
 	r.status = status
-	r.ResponseWriter.WriteHeader(status)
+	r.ResponseWriterWrapper.WriteHeader(status)
 }
 
 // Write is a wrapper that records the size of the body
 // that gets written.
 func (r *ResponseRecorder) Write(buf []byte) (int, error) {
-	n, err := r.ResponseWriter.Write(buf)
+	n, err := r.ResponseWriterWrapper.Write(buf)
 	if err == nil {
 		r.size += n
 	}
 	return n, err
 }
 
-// Size is a Getter to size property
+// Size returns the size of the recorded response body.
 func (r *ResponseRecorder) Size() int {
 	return r.size
 }
 
-// Status is a Getter to status property
+// Status returns the recorded response status code.
 func (r *ResponseRecorder) Status() int {
 	return r.status
 }
 
-// Hijack implements http.Hijacker. It simply wraps the underlying
-// ResponseWriter's Hijack method if there is one, or returns an error.
-func (r *ResponseRecorder) Hijack() (net.Conn, *bufio.ReadWriter, error) {
-	if hj, ok := r.ResponseWriter.(http.Hijacker); ok {
-		return hj.Hijack()
-	}
-	return nil, nil, errors.New("not a Hijacker")
+// ResponseBuffer is a type that conditionally buffers the
+// response in memory. It implements http.ResponseWriter so
+// that it can stream the response if it is not buffering.
+// Whether it buffers is decided by a func passed into the
+// constructor, NewResponseBuffer.
+//
+// This type implements http.ResponseWriter, so you can pass
+// this to the Next() middleware in the chain and record its
+// response. However, since the entire response body will be
+// buffered in memory, only use this when explicitly configured
+// and required for some specific reason. For example, the
+// text/template package only parses templates out of []byte
+// and not io.Reader, so the templates directive uses this
+// type to obtain the entire template text, but only on certain
+// requests that match the right Content-Type, etc.
+//
+// ResponseBuffer also implements io.ReaderFrom for performance
+// reasons. The standard lib's http.response type (unexported)
+// uses io.Copy to write the body. io.Copy makes an allocation
+// if the destination does not have a ReadFrom method (or if
+// the source does not have a WriteTo method, but that's
+// irrelevant here). Our ReadFrom is smart: if buffering, it
+// calls the buffer's ReadFrom, which makes no allocs because
+// it is already a buffer! If we're streaming the response
+// instead, ReadFrom uses io.CopyBuffer with a pooled buffer
+// that is managed within this package.
+type ResponseBuffer struct {
+	*ResponseWriterWrapper
+	Buffer       *bytes.Buffer
+	header       http.Header
+	status       int
+	shouldBuffer func(status int, header http.Header) bool
+	stream       bool
+	rw           http.ResponseWriter
+	wroteHeader  bool
 }
 
-// Flush implements http.Flusher. It simply wraps the underlying
-// ResponseWriter's Flush method if there is one, or does nothing.
-func (r *ResponseRecorder) Flush() {
-	if f, ok := r.ResponseWriter.(http.Flusher); ok {
-		f.Flush()
-	} else {
-		panic("not a Flusher") // should be recovered at the beginning of middleware stack
+// NewResponseBuffer returns a new ResponseBuffer that will
+// use buf to store the full body of the response if shouldBuffer
+// returns true. If shouldBuffer returns false, then the response
+// body will be streamed directly to rw.
+//
+// shouldBuffer will be passed the status code and header fields of
+// the response. With that information, the function should decide
+// whether to buffer the response in memory. For example: the templates
+// directive uses this to determine whether the response is the
+// right Content-Type (according to user config) for a template.
+//
+// For performance, the buf you pass in should probably be obtained
+// from a sync.Pool in order to reuse allocated space.
+func NewResponseBuffer(buf *bytes.Buffer, rw http.ResponseWriter,
+	shouldBuffer func(status int, header http.Header) bool) *ResponseBuffer {
+	rb := &ResponseBuffer{
+		Buffer:       buf,
+		header:       make(http.Header),
+		status:       http.StatusOK, // default status code
+		shouldBuffer: shouldBuffer,
+		rw:           rw,
+	}
+	rb.ResponseWriterWrapper = &ResponseWriterWrapper{ResponseWriter: rw}
+	return rb
+}
+
+// Header returns the response header map.
+func (rb *ResponseBuffer) Header() http.Header {
+	return rb.header
+}
+
+// WriteHeader calls shouldBuffer to decide whether the
+// upcoming body should be buffered, and then writes
+// the header to the response.
+func (rb *ResponseBuffer) WriteHeader(status int) {
+	if rb.wroteHeader {
+		return
+	}
+	rb.wroteHeader = true
+
+	rb.status = status
+	rb.stream = !rb.shouldBuffer(status, rb.header)
+	if rb.stream {
+		rb.CopyHeader()
+		rb.ResponseWriterWrapper.WriteHeader(status)
 	}
 }
 
-// CloseNotify implements http.CloseNotifier.
-// It just inherits the underlying ResponseWriter's CloseNotify method.
-func (r *ResponseRecorder) CloseNotify() <-chan bool {
-	if cn, ok := r.ResponseWriter.(http.CloseNotifier); ok {
-		return cn.CloseNotify()
+// Write writes buf to rb.Buffer if buffering, otherwise
+// to the ResponseWriter directly if streaming.
+func (rb *ResponseBuffer) Write(buf []byte) (int, error) {
+	if !rb.wroteHeader {
+		rb.WriteHeader(http.StatusOK)
 	}
-	panic("not a CloseNotifier")
+
+	if rb.stream {
+		return rb.ResponseWriterWrapper.Write(buf)
+	}
+	return rb.Buffer.Write(buf)
 }
+
+// Buffered returns whether rb has decided to buffer the response.
+func (rb *ResponseBuffer) Buffered() bool {
+	return !rb.stream
+}
+
+// CopyHeader copies the buffered header in rb to the ResponseWriter,
+// but it does not write the header out.
+func (rb *ResponseBuffer) CopyHeader() {
+	for field, val := range rb.header {
+		rb.ResponseWriterWrapper.Header()[field] = val
+	}
+}
+
+// ReadFrom avoids allocations when writing to the buffer (if buffering),
+// and reduces allocations when writing to the ResponseWriter directly
+// (if streaming).
+//
+// In local testing with the templates directive, req/sec were improved
+// from ~8,200 to ~9,600 on templated files by ensuring that this type
+// implements io.ReaderFrom.
+func (rb *ResponseBuffer) ReadFrom(src io.Reader) (int64, error) {
+	if !rb.wroteHeader {
+		rb.WriteHeader(http.StatusOK)
+	}
+
+	if rb.stream {
+		// first see if we can avoid any allocations at all
+		if wt, ok := src.(io.WriterTo); ok {
+			return wt.WriteTo(rb.ResponseWriterWrapper)
+		}
+		// if not, use a pooled copy buffer to reduce allocs
+		// (this improved req/sec from ~25,300 to ~27,000 on
+		// static files served directly with the fileserver,
+		// but results fluctuated a little on each run).
+		// a note of caution:
+		// https://go-review.googlesource.com/c/22134#message-ff351762308fe05f6b72a487d6842e3988916486
+		buf := respBufPool.Get().([]byte)
+		n, err := io.CopyBuffer(rb.ResponseWriterWrapper, src, buf)
+		respBufPool.Put(buf) // defer'ing this slowed down benchmarks a smidgin, I think
+		return n, err
+	}
+	return rb.Buffer.ReadFrom(src)
+}
+
+// StatusCodeWriter returns an http.ResponseWriter that always
+// writes the status code stored in rb from when a response
+// was buffered to it.
+func (rb *ResponseBuffer) StatusCodeWriter(w http.ResponseWriter) http.ResponseWriter {
+	return forcedStatusCodeWriter{w, rb}
+}
+
+// forcedStatusCodeWriter is used to force a status code when
+// writing the header. It uses the status code saved on rb.
+// This is useful if passing a http.ResponseWriter into
+// http.ServeContent because ServeContent hard-codes 2xx status
+// codes. If we buffered the response, we force that status code
+// instead.
+type forcedStatusCodeWriter struct {
+	http.ResponseWriter
+	rb *ResponseBuffer
+}
+
+func (fscw forcedStatusCodeWriter) WriteHeader(int) {
+	fscw.ResponseWriter.WriteHeader(fscw.rb.status)
+}
+
+// respBufPool is used for io.CopyBuffer when ResponseBuffer
+// is configured to stream a response.
+var respBufPool = &sync.Pool{
+	New: func() interface{} {
+		return make([]byte, 32*1024)
+	},
+}
+
+// Interface guards
+var (
+	_ HTTPInterfaces = (*ResponseRecorder)(nil)
+	_ HTTPInterfaces = (*ResponseBuffer)(nil)
+	_ io.ReaderFrom  = (*ResponseBuffer)(nil)
+)

caddyhttp/httpserver/recorder_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (

caddyhttp/httpserver/replacer.go

@@ -1,6 +1,23 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
+	"bytes"
+	"io"
+	"io/ioutil"
 	"net"
 	"net/http"
 	"net/http/httputil"
@@ -10,6 +27,8 @@ import (
 	"strconv"
 	"strings"
 	"time"
+
+	"github.com/mholt/caddy"
 )
 
 // requestReplacer is a strings.Replacer which is used to
@@ -20,6 +39,8 @@ var requestReplacer = strings.NewReplacer(
 	"\n", "\\n",
 )
 
+var now = time.Now
+
 // Replacer is a type which can replace placeholder
 // substrings in a string with actual values from a
 // http.Request and ResponseRecorder. Always use
@@ -37,10 +58,40 @@ type Replacer interface {
 // they will be used to overwrite other replacements
 // if there is a name conflict.
 type replacer struct {
-	replacements       map[string]string
 	customReplacements map[string]string
 	emptyValue         string
 	responseRecorder   *ResponseRecorder
+	request            *http.Request
+	requestBody        *limitWriter
+}
+
+type limitWriter struct {
+	w      bytes.Buffer
+	remain int
+}
+
+func newLimitWriter(max int) *limitWriter {
+	return &limitWriter{
+		w:      bytes.Buffer{},
+		remain: max,
+	}
+}
+
+func (lw *limitWriter) Write(p []byte) (int, error) {
+	// skip if we are full
+	if lw.remain <= 0 {
+		return len(p), nil
+	}
+	if n := len(p); n > lw.remain {
+		p = p[:lw.remain]
+	}
+	n, err := lw.w.Write(p)
+	lw.remain -= n
+	return n, err
+}
+
+func (lw *limitWriter) String() string {
+	return lw.w.String()
 }
 
 // NewReplacer makes a new replacer based on r and rr which
@@ -51,120 +102,287 @@ type replacer struct {
 // emptyValue should be the string that is used in place
 // of empty string (can still be empty string).
 func NewReplacer(r *http.Request, rr *ResponseRecorder, emptyValue string) Replacer {
-	rep := &replacer{
-		responseRecorder:   rr,
-		customReplacements: make(map[string]string),
-		replacements: map[string]string{
-			"{method}": r.Method,
-			"{scheme}": func() string {
-				if r.TLS != nil {
-					return "https"
-				}
-				return "http"
-			}(),
-			"{hostname}": func() string {
-				name, err := os.Hostname()
-				if err != nil {
-					return ""
-				}
-				return name
-			}(),
-			"{host}":          r.Host,
-			"{path}":          r.URL.Path,
-			"{path_escaped}":  url.QueryEscape(r.URL.Path),
-			"{query}":         r.URL.RawQuery,
-			"{query_escaped}": url.QueryEscape(r.URL.RawQuery),
-			"{fragment}":      r.URL.Fragment,
-			"{proto}":         r.Proto,
-			"{remote}": func() string {
-				if fwdFor := r.Header.Get("X-Forwarded-For"); fwdFor != "" {
-					return fwdFor
-				}
-				host, _, err := net.SplitHostPort(r.RemoteAddr)
-				if err != nil {
-					return r.RemoteAddr
-				}
-				return host
-			}(),
-			"{port}": func() string {
-				_, port, err := net.SplitHostPort(r.RemoteAddr)
-				if err != nil {
-					return ""
-				}
-				return port
-			}(),
-			"{uri}":         r.URL.RequestURI(),
-			"{uri_escaped}": url.QueryEscape(r.URL.RequestURI()),
-			"{when}":        time.Now().Format(timeFormat),
-			"{file}": func() string {
-				_, file := path.Split(r.URL.Path)
-				return file
-			}(),
-			"{dir}": func() string {
-				dir, _ := path.Split(r.URL.Path)
-				return dir
-			}(),
-			"{request}": func() string {
-				dump, err := httputil.DumpRequest(r, false)
-				if err != nil {
-					return ""
-				}
-
-				return requestReplacer.Replace(string(dump))
-			}(),
-		},
-		emptyValue: emptyValue,
+	repl := &replacer{
+		request:          r,
+		responseRecorder: rr,
+		emptyValue:       emptyValue,
 	}
 
-	// Header placeholders (case-insensitive)
-	for header, values := range r.Header {
-		rep.replacements[headerReplacer+strings.ToLower(header)+"}"] = strings.Join(values, ",")
+	// extract customReplacements from a request replacer when present.
+	if existing, ok := r.Context().Value(ReplacerCtxKey).(*replacer); ok {
+		repl.requestBody = existing.requestBody
+		repl.customReplacements = existing.customReplacements
+	} else {
+		// if there is no existing replacer, build one from scratch.
+		rb := newLimitWriter(MaxLogBodySize)
+		if r.Body != nil {
+			r.Body = struct {
+				io.Reader
+				io.Closer
+			}{io.TeeReader(r.Body, rb), io.Closer(r.Body)}
+		}
+		repl.requestBody = rb
+		repl.customReplacements = make(map[string]string)
 	}
 
-	return rep
+	return repl
+}
+
+func canLogRequest(r *http.Request) bool {
+	if r.Method == "POST" || r.Method == "PUT" {
+		for _, cType := range r.Header[headerContentType] {
+			// the cType could have charset and other info
+			if strings.Contains(cType, contentTypeJSON) || strings.Contains(cType, contentTypeXML) {
+				return true
+			}
+		}
+	}
+	return false
 }
 
 // Replace performs a replacement of values on s and returns
 // the string with the replaced values.
 func (r *replacer) Replace(s string) string {
-	// Make response placeholders now
-	if r.responseRecorder != nil {
-		r.replacements["{status}"] = strconv.Itoa(r.responseRecorder.status)
-		r.replacements["{size}"] = strconv.Itoa(r.responseRecorder.size)
-		r.replacements["{latency}"] = time.Since(r.responseRecorder.start).String()
+	// Do not attempt replacements if no placeholder is found.
+	if !strings.ContainsAny(s, "{}") {
+		return s
 	}
 
-	// Include custom placeholders, overwriting existing ones if necessary
-	for key, val := range r.customReplacements {
-		r.replacements[key] = val
-	}
-
-	// Header replacements - these are case-insensitive, so we can't just use strings.Replace()
-	for strings.Contains(s, headerReplacer) {
-		idxStart := strings.Index(s, headerReplacer)
-		endOffset := idxStart + len(headerReplacer)
-		idxEnd := strings.Index(s[endOffset:], "}")
-		if idxEnd > -1 {
-			placeholder := strings.ToLower(s[idxStart : endOffset+idxEnd+1])
-			replacement := r.replacements[placeholder]
-			if replacement == "" {
-				replacement = r.emptyValue
-			}
-			s = s[:idxStart] + replacement + s[endOffset+idxEnd+1:]
-		} else {
+	result := ""
+	for {
+		idxStart := strings.Index(s, "{")
+		if idxStart == -1 {
+			// no placeholder anymore
 			break
 		}
-	}
-
-	// Regular replacements - these are easier because they're case-sensitive
-	for placeholder, replacement := range r.replacements {
-		if replacement == "" {
-			replacement = r.emptyValue
+		idxEnd := strings.Index(s[idxStart:], "}")
+		if idxEnd == -1 {
+			// unpaired placeholder
+			break
 		}
-		s = strings.Replace(s, placeholder, replacement, -1)
+		idxEnd += idxStart
+
+		// get a replacement
+		placeholder := s[idxStart : idxEnd+1]
+		replacement := r.getSubstitution(placeholder)
+
+		// append prefix + replacement
+		result += s[:idxStart] + replacement
+
+		// strip out scanned parts
+		s = s[idxEnd+1:]
 	}
 
-	return s
+	// append unscanned parts
+	return result + s
+}
+
+func roundDuration(d time.Duration) time.Duration {
+	if d >= time.Millisecond {
+		return round(d, time.Millisecond)
+	} else if d >= time.Microsecond {
+		return round(d, time.Microsecond)
+	}
+
+	return d
+}
+
+// round rounds d to the nearest r
+func round(d, r time.Duration) time.Duration {
+	if r <= 0 {
+		return d
+	}
+	neg := d < 0
+	if neg {
+		d = -d
+	}
+	if m := d % r; m+m < r {
+		d = d - m
+	} else {
+		d = d + r - m
+	}
+	if neg {
+		return -d
+	}
+	return d
+}
+
+// getSubstitution retrieves value from corresponding key
+func (r *replacer) getSubstitution(key string) string {
+	// search custom replacements first
+	if value, ok := r.customReplacements[key]; ok {
+		return value
+	}
+
+	// search request headers then
+	if key[1] == '>' {
+		want := key[2 : len(key)-1]
+		for key, values := range r.request.Header {
+			// Header placeholders (case-insensitive)
+			if strings.EqualFold(key, want) {
+				return strings.Join(values, ",")
+			}
+		}
+	}
+	// search response headers then
+	if r.responseRecorder != nil && key[1] == '<' {
+		want := key[2 : len(key)-1]
+		for key, values := range r.responseRecorder.Header() {
+			// Header placeholders (case-insensitive)
+			if strings.EqualFold(key, want) {
+				return strings.Join(values, ",")
+			}
+		}
+	}
+	// next check for cookies
+	if key[1] == '~' {
+		name := key[2 : len(key)-1]
+		if cookie, err := r.request.Cookie(name); err == nil {
+			return cookie.Value
+		}
+	}
+	// next check for query argument
+	if key[1] == '?' {
+		query := r.request.URL.Query()
+		name := key[2 : len(key)-1]
+		return query.Get(name)
+	}
+
+	// search default replacements in the end
+	switch key {
+	case "{method}":
+		return r.request.Method
+	case "{scheme}":
+		if r.request.TLS != nil {
+			return "https"
+		}
+		return "http"
+	case "{hostname}":
+		name, err := os.Hostname()
+		if err != nil {
+			return r.emptyValue
+		}
+		return name
+	case "{host}":
+		return r.request.Host
+	case "{hostonly}":
+		host, _, err := net.SplitHostPort(r.request.Host)
+		if err != nil {
+			return r.request.Host
+		}
+		return host
+	case "{path}":
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.Path
+	case "{path_escaped}":
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return url.QueryEscape(u.Path)
+	case "{request_id}":
+		reqid, _ := r.request.Context().Value(RequestIDCtxKey).(string)
+		return reqid
+	case "{rewrite_path}":
+		return r.request.URL.Path
+	case "{rewrite_path_escaped}":
+		return url.QueryEscape(r.request.URL.Path)
+	case "{query}":
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.RawQuery
+	case "{query_escaped}":
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return url.QueryEscape(u.RawQuery)
+	case "{fragment}":
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.Fragment
+	case "{proto}":
+		return r.request.Proto
+	case "{remote}":
+		host, _, err := net.SplitHostPort(r.request.RemoteAddr)
+		if err != nil {
+			return r.request.RemoteAddr
+		}
+		return host
+	case "{port}":
+		_, port, err := net.SplitHostPort(r.request.RemoteAddr)
+		if err != nil {
+			return r.emptyValue
+		}
+		return port
+	case "{uri}":
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return u.RequestURI()
+	case "{uri_escaped}":
+		u, _ := r.request.Context().Value(OriginalURLCtxKey).(url.URL)
+		return url.QueryEscape(u.RequestURI())
+	case "{rewrite_uri}":
+		return r.request.URL.RequestURI()
+	case "{rewrite_uri_escaped}":
+		return url.QueryEscape(r.request.URL.RequestURI())
+	case "{when}":
+		return now().Format(timeFormat)
+	case "{when_iso}":
+		return now().UTC().Format(timeFormatISOUTC)
+	case "{when_unix}":
+		return strconv.FormatInt(now().Unix(), 10)
+	case "{file}":
+		_, file := path.Split(r.request.URL.Path)
+		return file
+	case "{dir}":
+		dir, _ := path.Split(r.request.URL.Path)
+		return dir
+	case "{request}":
+		dump, err := httputil.DumpRequest(r.request, false)
+		if err != nil {
+			return r.emptyValue
+		}
+		return requestReplacer.Replace(string(dump))
+	case "{request_body}":
+		if !canLogRequest(r.request) {
+			return r.emptyValue
+		}
+		_, err := ioutil.ReadAll(r.request.Body)
+		if err != nil {
+			if err == ErrMaxBytesExceeded {
+				return r.emptyValue
+			}
+		}
+		return requestReplacer.Replace(r.requestBody.String())
+	case "{mitm}":
+		if val, ok := r.request.Context().Value(caddy.CtxKey("mitm")).(bool); ok {
+			if val {
+				return "likely"
+			}
+			return "unlikely"
+		}
+		return "unknown"
+	case "{status}":
+		if r.responseRecorder == nil {
+			return r.emptyValue
+		}
+		return strconv.Itoa(r.responseRecorder.status)
+	case "{size}":
+		if r.responseRecorder == nil {
+			return r.emptyValue
+		}
+		return strconv.Itoa(r.responseRecorder.size)
+	case "{latency}":
+		if r.responseRecorder == nil {
+			return r.emptyValue
+		}
+		return roundDuration(time.Since(r.responseRecorder.start)).String()
+	case "{latency_ms}":
+		if r.responseRecorder == nil {
+			return r.emptyValue
+		}
+		elapsedDuration := time.Since(r.responseRecorder.start)
+		return strconv.FormatInt(convertToMilliseconds(elapsedDuration), 10)
+	}
+
+	return r.emptyValue
+}
+
+//convertToMilliseconds returns the number of milliseconds in the given duration
+func convertToMilliseconds(d time.Duration) int64 {
+	return d.Nanoseconds() / 1e6
 }
 
 // Set sets key to value in the r.customReplacements map.
@@ -173,6 +391,11 @@ func (r *replacer) Set(key, value string) {
 }
 
 const (
-	timeFormat     = "02/Jan/2006:15:04:05 -0700"
-	headerReplacer = "{>"
+	timeFormat        = "02/Jan/2006:15:04:05 -0700"
+	timeFormatISOUTC  = "2006-01-02T15:04:05Z" // ISO 8601 with timezone to be assumed as UTC
+	headerContentType = "Content-Type"
+	contentTypeJSON   = "application/json"
+	contentTypeXML    = "application/xml"
+	// MaxLogBodySize limits the size of logged request's body
+	MaxLogBodySize = 100 * 1024
 )

caddyhttp/httpserver/replacer_test.go

@@ -1,11 +1,27 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
+	"context"
 	"net/http"
 	"net/http/httptest"
 	"os"
 	"strings"
 	"testing"
+	"time"
 )
 
 func TestNewReplacer(t *testing.T) {
@@ -21,21 +37,12 @@ func TestNewReplacer(t *testing.T) {
 
 	switch v := rep.(type) {
 	case *replacer:
-		if v.replacements["{host}"] != "localhost" {
+		if v.getSubstitution("{host}") != "localhost" {
 			t.Error("Expected host to be localhost")
 		}
-		if v.replacements["{method}"] != "POST" {
+		if v.getSubstitution("{method}") != "POST" {
 			t.Error("Expected request method  to be POST")
 		}
-
-		// Response placeholders should only be set after call to Replace()
-		if got, want := v.replacements["{status}"], ""; got != want {
-			t.Errorf("Expected status to NOT be set before Replace() is called; was: %s", got)
-		}
-		rep.Replace("foobar")
-		if got, want := v.replacements["{status}"], "200"; got != want {
-			t.Errorf("Expected status to be %s, was: %s", want, got)
-		}
 	default:
 		t.Fatalf("Expected *replacer underlying Replacer type, got: %#v", rep)
 	}
@@ -46,19 +53,35 @@ func TestReplace(t *testing.T) {
 	recordRequest := NewResponseRecorder(w)
 	reader := strings.NewReader(`{"username": "dennis"}`)
 
-	request, err := http.NewRequest("POST", "http://localhost", reader)
+	request, err := http.NewRequest("POST", "http://localhost/?foo=bar", reader)
 	if err != nil {
-		t.Fatal("Request Formation Failed\n")
+		t.Fatalf("Failed to make request: %v", err)
 	}
+	ctx := context.WithValue(request.Context(), OriginalURLCtxKey, *request.URL)
+	request = request.WithContext(ctx)
+
 	request.Header.Set("Custom", "foobarbaz")
 	request.Header.Set("ShorterVal", "1")
 	repl := NewReplacer(request, recordRequest, "-")
+	// add some headers after creating replacer
+	request.Header.Set("CustomAdd", "caddy")
+	request.Header.Set("Cookie", "foo=bar; taste=delicious")
+
+	// add some respons headers
+	recordRequest.Header().Set("Custom", "CustomResponseHeader")
 
 	hostname, err := os.Hostname()
 	if err != nil {
-		t.Fatal("Failed to determine hostname\n")
+		t.Fatalf("Failed to determine hostname: %v", err)
 	}
 
+	old := now
+	now = func() time.Time {
+		return time.Date(2006, 1, 2, 15, 4, 5, 02, time.FixedZone("hardcoded", -7))
+	}
+	defer func() {
+		now = old
+	}()
 	testCases := []struct {
 		template string
 		expect   string
@@ -67,13 +90,28 @@ func TestReplace(t *testing.T) {
 		{"This host is {host}.", "This host is localhost."},
 		{"This request method is {method}.", "This request method is POST."},
 		{"The response status is {status}.", "The response status is 200."},
+		{"{when}", "02/Jan/2006:15:04:05 +0000"},
+		{"{when_iso}", "2006-01-02T15:04:12Z"},
+		{"{when_unix}", "1136214252"},
 		{"The Custom header is {>Custom}.", "The Custom header is foobarbaz."},
-		{"The request is {request}.", "The request is POST / HTTP/1.1\\r\\nHost: localhost\\r\\nCustom: foobarbaz\\r\\nShorterval: 1\\r\\n\\r\\n."},
+		{"The CustomAdd header is {>CustomAdd}.", "The CustomAdd header is caddy."},
+		{"The Custom response header is {<Custom}.", "The Custom response header is CustomResponseHeader."},
+		{"Bad {>Custom placeholder", "Bad {>Custom placeholder"},
+		{"The request is {request}.", "The request is POST /?foo=bar HTTP/1.1\\r\\nHost: localhost\\r\\n" +
+			"Cookie: foo=bar; taste=delicious\\r\\nCustom: foobarbaz\\r\\nCustomadd: caddy\\r\\n" +
+			"Shorterval: 1\\r\\n\\r\\n."},
 		{"The cUsToM header is {>cUsToM}...", "The cUsToM header is foobarbaz..."},
+		{"The cUsToM response header is {<CuSTom}.", "The cUsToM response header is CustomResponseHeader."},
 		{"The Non-Existent header is {>Non-Existent}.", "The Non-Existent header is -."},
 		{"Bad {host placeholder...", "Bad {host placeholder..."},
 		{"Bad {>Custom placeholder", "Bad {>Custom placeholder"},
 		{"Bad {>Custom placeholder {>ShorterVal}", "Bad -"},
+		{"Bad {}", "Bad -"},
+		{"Cookies are {~taste}", "Cookies are delicious"},
+		{"Missing cookie is {~missing}", "Missing cookie is -"},
+		{"Query string is {query}", "Query string is foo=bar"},
+		{"Query string value for foo is {?foo}", "Query string value for foo is bar"},
+		{"Missing query string argument is {?missing}", "Missing query string argument is "},
 	}
 
 	for _, c := range testCases {
@@ -87,12 +125,18 @@ func TestReplace(t *testing.T) {
 		replacements map[string]string
 		expect       string
 	}{
-		{"/a{1}/{2}", map[string]string{"{1}": "12", "{2}": ""}, "/a12/"},
+		{
+			"/a{1}/{2}",
+			map[string]string{
+				"{1}": "12",
+				"{2}": "",
+			},
+			"/a12/"},
 	}
 
 	for _, c := range complexCases {
 		repl := &replacer{
-			replacements: c.replacements,
+			customReplacements: c.replacements,
 		}
 		if expected, actual := c.expect, repl.Replace(c.template); expected != actual {
 			t.Errorf("for template '%s', expected '%s', got '%s'", c.template, expected, actual)
@@ -100,6 +144,43 @@ func TestReplace(t *testing.T) {
 	}
 }
 
+func TestResponseRecorderNil(t *testing.T) {
+
+	reader := strings.NewReader(`{"username": "dennis"}`)
+
+	request, err := http.NewRequest("POST", "http://localhost/?foo=bar", reader)
+	if err != nil {
+		t.Fatalf("Failed to make request: %v", err)
+	}
+
+	request.Header.Set("Custom", "foobarbaz")
+	repl := NewReplacer(request, nil, "-")
+	// add some headers after creating replacer
+	request.Header.Set("CustomAdd", "caddy")
+	request.Header.Set("Cookie", "foo=bar; taste=delicious")
+
+	old := now
+	now = func() time.Time {
+		return time.Date(2006, 1, 2, 15, 4, 5, 02, time.FixedZone("hardcoded", -7))
+	}
+	defer func() {
+		now = old
+	}()
+	testCases := []struct {
+		template string
+		expect   string
+	}{
+		{"The Custom response header is {<Custom}.", "The Custom response header is -."},
+	}
+
+	for _, c := range testCases {
+		if expected, actual := c.expect, repl.Replace(c.template); expected != actual {
+			t.Errorf("for template '%s', expected '%s', got '%s'", c.template, expected, actual)
+		}
+	}
+
+}
+
 func TestSet(t *testing.T) {
 	w := httptest.NewRecorder()
 	recordRequest := NewResponseRecorder(w)
@@ -107,7 +188,7 @@ func TestSet(t *testing.T) {
 
 	request, err := http.NewRequest("POST", "http://localhost", reader)
 	if err != nil {
-		t.Fatalf("Request Formation Failed \n")
+		t.Fatalf("Request Formation Failed: %s\n", err.Error())
 	}
 	repl := NewReplacer(request, recordRequest, "")
 
@@ -129,3 +210,76 @@ func TestSet(t *testing.T) {
 		t.Error("Expected variable replacement failed")
 	}
 }
+
+// Test function to test that various placeholders hold correct values after a rewrite
+// has been performed.  The NewRequest actually contains the rewritten value.
+func TestPathRewrite(t *testing.T) {
+	w := httptest.NewRecorder()
+	recordRequest := NewResponseRecorder(w)
+	reader := strings.NewReader(`{"username": "dennis"}`)
+
+	request, err := http.NewRequest("POST", "http://getcaddy.com/index.php?key=value", reader)
+	if err != nil {
+		t.Fatalf("Request Formation Failed: %s\n", err.Error())
+	}
+	urlCopy := *request.URL
+	urlCopy.Path = "a/custom/path.php"
+	ctx := context.WithValue(request.Context(), OriginalURLCtxKey, urlCopy)
+	request = request.WithContext(ctx)
+
+	repl := NewReplacer(request, recordRequest, "")
+
+	if got, want := repl.Replace("This path is '{path}'"), "This path is 'a/custom/path.php'"; got != want {
+		t.Errorf("{path} replacement failed; got '%s', want '%s'", got, want)
+	}
+
+	if got, want := repl.Replace("This path is {rewrite_path}"), "This path is /index.php"; got != want {
+		t.Errorf("{rewrite_path} replacement failed; got '%s', want '%s'", got, want)
+	}
+	if got, want := repl.Replace("This path is '{uri}'"), "This path is 'a/custom/path.php?key=value'"; got != want {
+		t.Errorf("{uri} replacement failed; got '%s', want '%s'", got, want)
+	}
+
+	if got, want := repl.Replace("This path is {rewrite_uri}"), "This path is /index.php?key=value"; got != want {
+		t.Errorf("{rewrite_uri} replacement failed; got '%s', want '%s'", got, want)
+	}
+
+}
+
+func TestRound(t *testing.T) {
+	var tests = map[time.Duration]time.Duration{
+		// 599.935µs -> 560µs
+		559935 * time.Nanosecond: 560 * time.Microsecond,
+		// 1.55ms    -> 2ms
+		1550 * time.Microsecond: 2 * time.Millisecond,
+		// 1.5555s   -> 1.556s
+		1555500 * time.Microsecond: 1556 * time.Millisecond,
+		// 1m2.0035s -> 1m2.004s
+		62003500 * time.Microsecond: 62004 * time.Millisecond,
+	}
+
+	for dur, expected := range tests {
+		rounded := roundDuration(dur)
+		if rounded != expected {
+			t.Errorf("Expected %v, Got %v", expected, rounded)
+		}
+	}
+}
+
+func TestMillisecondConverstion(t *testing.T) {
+	var testCases = map[time.Duration]int64{
+		2 * time.Second:           2000,
+		9039492 * time.Nanosecond: 9,
+		1000 * time.Microsecond:   1,
+		127 * time.Nanosecond:     0,
+		0 * time.Millisecond:      0,
+		255 * time.Millisecond:    255,
+	}
+
+	for dur, expected := range testCases {
+		numMillisecond := convertToMilliseconds(dur)
+		if numMillisecond != expected {
+			t.Errorf("Expected %v. Got %v", expected, numMillisecond)
+		}
+	}
+}

caddyhttp/httpserver/responsewriterwrapper.go

@@ -0,0 +1,79 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpserver
+
+import (
+	"bufio"
+	"net"
+	"net/http"
+)
+
+// ResponseWriterWrapper wrappers underlying ResponseWriter
+// and inherits its Hijacker/Pusher/CloseNotifier/Flusher as well.
+type ResponseWriterWrapper struct {
+	http.ResponseWriter
+}
+
+// Hijack implements http.Hijacker. It simply wraps the underlying
+// ResponseWriter's Hijack method if there is one, or returns an error.
+func (rww *ResponseWriterWrapper) Hijack() (net.Conn, *bufio.ReadWriter, error) {
+	if hj, ok := rww.ResponseWriter.(http.Hijacker); ok {
+		return hj.Hijack()
+	}
+	return nil, nil, NonHijackerError{Underlying: rww.ResponseWriter}
+}
+
+// Flush implements http.Flusher. It simply wraps the underlying
+// ResponseWriter's Flush method if there is one, or panics.
+func (rww *ResponseWriterWrapper) Flush() {
+	if f, ok := rww.ResponseWriter.(http.Flusher); ok {
+		f.Flush()
+	} else {
+		panic(NonFlusherError{Underlying: rww.ResponseWriter})
+	}
+}
+
+// CloseNotify implements http.CloseNotifier.
+// It just inherits the underlying ResponseWriter's CloseNotify method.
+// It panics if the underlying ResponseWriter is not a CloseNotifier.
+func (rww *ResponseWriterWrapper) CloseNotify() <-chan bool {
+	if cn, ok := rww.ResponseWriter.(http.CloseNotifier); ok {
+		return cn.CloseNotify()
+	}
+	panic(NonCloseNotifierError{Underlying: rww.ResponseWriter})
+}
+
+// Push implements http.Pusher.
+// It just inherits the underlying ResponseWriter's Push method.
+// It panics if the underlying ResponseWriter is not a Pusher.
+func (rww *ResponseWriterWrapper) Push(target string, opts *http.PushOptions) error {
+	if pusher, hasPusher := rww.ResponseWriter.(http.Pusher); hasPusher {
+		return pusher.Push(target, opts)
+	}
+
+	return NonPusherError{Underlying: rww.ResponseWriter}
+}
+
+// HTTPInterfaces mix all the interfaces that middleware ResponseWriters need to support.
+type HTTPInterfaces interface {
+	http.ResponseWriter
+	http.Pusher
+	http.Flusher
+	http.CloseNotifier
+	http.Hijacker
+}
+
+// Interface guards
+var _ HTTPInterfaces = (*ResponseWriterWrapper)(nil)

caddyhttp/httpserver/roller.go

@@ -1,11 +1,25 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
+	"errors"
 	"io"
+	"path/filepath"
 	"strconv"
 
-	"github.com/mholt/caddy"
-
 	"gopkg.in/natefinch/lumberjack.v2"
 )
 
@@ -15,50 +29,110 @@ type LogRoller struct {
 	MaxSize    int
 	MaxAge     int
 	MaxBackups int
+	Compress   bool
 	LocalTime  bool
 }
 
 // GetLogWriter returns an io.Writer that writes to a rolling logger.
+// This should be called only from the main goroutine (like during
+// server setup) because this method is not thread-safe; it is careful
+// to create only one log writer per log file, even if the log file
+// is shared by different sites or middlewares. This ensures that
+// rolling is synchronized, since a process (or multiple processes)
+// should not create more than one roller on the same file at the
+// same time. See issue #1363.
 func (l LogRoller) GetLogWriter() io.Writer {
-	return &lumberjack.Logger{
-		Filename:   l.Filename,
-		MaxSize:    l.MaxSize,
-		MaxAge:     l.MaxAge,
-		MaxBackups: l.MaxBackups,
-		LocalTime:  l.LocalTime,
+	absPath, err := filepath.Abs(l.Filename)
+	if err != nil {
+		absPath = l.Filename // oh well, hopefully they're consistent in how they specify the filename
+	}
+	lj, has := lumberjacks[absPath]
+	if !has {
+		lj = &lumberjack.Logger{
+			Filename:   l.Filename,
+			MaxSize:    l.MaxSize,
+			MaxAge:     l.MaxAge,
+			MaxBackups: l.MaxBackups,
+			Compress:   l.Compress,
+			LocalTime:  l.LocalTime,
+		}
+		lumberjacks[absPath] = lj
+	}
+	return lj
+}
+
+// IsLogRollerSubdirective is true if the subdirective is for the log roller.
+func IsLogRollerSubdirective(subdir string) bool {
+	return subdir == directiveRotateSize ||
+		subdir == directiveRotateAge ||
+		subdir == directiveRotateKeep ||
+		subdir == directiveRotateCompress
+}
+
+var invalidRollerParameterErr = errors.New("invalid roller parameter")
+
+// ParseRoller parses roller contents out of c.
+func ParseRoller(l *LogRoller, what string, where ...string) error {
+	if l == nil {
+		l = DefaultLogRoller()
+	}
+
+	// rotate_compress doesn't accept any parameters.
+	// others only accept one parameter
+	if (what == directiveRotateCompress && len(where) != 0) ||
+		(what != directiveRotateCompress && len(where) != 1) {
+		return invalidRollerParameterErr
+	}
+
+	var (
+		value int
+		err   error
+	)
+	if what != directiveRotateCompress {
+		value, err = strconv.Atoi(where[0])
+		if err != nil {
+			return err
+		}
+	}
+
+	switch what {
+	case directiveRotateSize:
+		l.MaxSize = value
+	case directiveRotateAge:
+		l.MaxAge = value
+	case directiveRotateKeep:
+		l.MaxBackups = value
+	case directiveRotateCompress:
+		l.Compress = true
+	}
+	return nil
+}
+
+// DefaultLogRoller will roll logs by default.
+func DefaultLogRoller() *LogRoller {
+	return &LogRoller{
+		MaxSize:    defaultRotateSize,
+		MaxAge:     defaultRotateAge,
+		MaxBackups: defaultRotateKeep,
+		Compress:   false,
+		LocalTime:  true,
 	}
 }
 
-// ParseRoller parses roller contents out of c.
-func ParseRoller(c *caddy.Controller) (*LogRoller, error) {
-	var size, age, keep int
-	// This is kind of a hack to support nested blocks:
-	// As we are already in a block: either log or errors,
-	// c.nesting > 0 but, as soon as c meets a }, it thinks
-	// the block is over and return false for c.NextBlock.
-	for c.NextBlock() {
-		what := c.Val()
-		if !c.NextArg() {
-			return nil, c.ArgErr()
-		}
-		value := c.Val()
-		var err error
-		switch what {
-		case "size":
-			size, err = strconv.Atoi(value)
-		case "age":
-			age, err = strconv.Atoi(value)
-		case "keep":
-			keep, err = strconv.Atoi(value)
-		}
-		if err != nil {
-			return nil, err
-		}
-	}
-	return &LogRoller{
-		MaxSize:    size,
-		MaxAge:     age,
-		MaxBackups: keep,
-		LocalTime:  true,
-	}, nil
-}
+const (
+	// defaultRotateSize is 100 MB.
+	defaultRotateSize = 100
+	// defaultRotateAge is 14 days.
+	defaultRotateAge = 14
+	// defaultRotateKeep is 10 files.
+	defaultRotateKeep = 10
+
+	directiveRotateSize     = "rotate_size"
+	directiveRotateAge      = "rotate_age"
+	directiveRotateKeep     = "rotate_keep"
+	directiveRotateCompress = "rotate_compress"
+)
+
+// lumberjacks maps log filenames to the logger
+// that is being used to keep them rolled/maintained.
+var lumberjacks = make(map[string]*lumberjack.Logger)

caddyhttp/httpserver/server.go

@@ -1,14 +1,32 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 // Package httpserver implements an HTTP server on top of Caddy.
 package httpserver
 
 import (
+	"context"
 	"crypto/tls"
+	"errors"
 	"fmt"
 	"log"
 	"net"
 	"net/http"
+	"net/url"
 	"os"
 	"path"
+	"path/filepath"
 	"runtime"
 	"strings"
 	"sync"
@@ -27,75 +45,104 @@ type Server struct {
 	listener    net.Listener
 	listenerMu  sync.Mutex
 	sites       []*SiteConfig
-	connTimeout time.Duration  // max time to wait for a connection before force stop
-	connWg      sync.WaitGroup // one increment per connection
-	tlsGovChan  chan struct{}  // close to stop the TLS maintenance goroutine
+	connTimeout time.Duration // max time to wait for a connection before force stop
+	tlsGovChan  chan struct{} // close to stop the TLS maintenance goroutine
 	vhosts      *vhostTrie
 }
 
 // ensure it satisfies the interface
 var _ caddy.GracefulServer = new(Server)
 
+var defaultALPN = []string{"h2", "http/1.1"}
+
+// makeTLSConfig extracts TLS settings from each site config to
+// build a tls.Config usable in Caddy HTTP servers. The returned
+// config will be nil if TLS is disabled for these sites.
+func makeTLSConfig(group []*SiteConfig) (*tls.Config, error) {
+	var tlsConfigs []*caddytls.Config
+	for i := range group {
+		if HTTP2 && len(group[i].TLS.ALPN) == 0 {
+			// if no application-level protocol was configured up to now,
+			// default to HTTP/2, then HTTP/1.1 if necessary
+			group[i].TLS.ALPN = defaultALPN
+		}
+		tlsConfigs = append(tlsConfigs, group[i].TLS)
+	}
+	return caddytls.MakeTLSConfig(tlsConfigs)
+}
+
+func getFallbacks(sites []*SiteConfig) []string {
+	fallbacks := []string{}
+	for _, sc := range sites {
+		if sc.FallbackSite {
+			fallbacks = append(fallbacks, sc.Addr.Host)
+		}
+	}
+	return fallbacks
+}
+
 // NewServer creates a new Server instance that will listen on addr
 // and will serve the sites configured in group.
 func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 	s := &Server{
-		Server: &http.Server{
-			Addr: addr,
-			// TODO: Make these values configurable?
-			// ReadTimeout:    2 * time.Minute,
-			// WriteTimeout:   2 * time.Minute,
-			// MaxHeaderBytes: 1 << 16,
-		},
+		Server:      makeHTTPServerWithTimeouts(addr, group),
 		vhosts:      newVHostTrie(),
 		sites:       group,
 		connTimeout: GracefulTimeout,
 	}
+	s.vhosts.fallbackHosts = append(s.vhosts.fallbackHosts, getFallbacks(group)...)
+	s.Server = makeHTTPServerWithHeaderLimit(s.Server, group)
 	s.Server.Handler = s // this is weird, but whatever
-	s.Server.ConnState = func(c net.Conn, cs http.ConnState) {
-		if cs == http.StateIdle {
-			s.listenerMu.Lock()
-			// server stopped, close idle connection
-			if s.listener == nil {
-				c.Close()
-			}
-			s.listenerMu.Unlock()
-		}
-	}
 
-	// Disable HTTP/2 if desired
-	if !HTTP2 {
-		s.Server.TLSNextProto = make(map[string]func(*http.Server, *tls.Conn, http.Handler))
-	}
-
-	// Enable QUIC if desired
-	if QUIC {
-		s.quicServer = &h2quic.Server{Server: s.Server}
-		s.Server.Handler = s.wrapWithSvcHeaders(s.Server.Handler)
-	}
-
-	// We have to bound our wg with one increment
-	// to prevent a "race condition" that is hard-coded
-	// into sync.WaitGroup.Wait() - basically, an add
-	// with a positive delta must be guaranteed to
-	// occur before Wait() is called on the wg.
-	// In a way, this kind of acts as a safety barrier.
-	s.connWg.Add(1)
-
-	// Set up TLS configuration
-	var tlsConfigs []*caddytls.Config
-	var err error
-	for _, site := range group {
-		tlsConfigs = append(tlsConfigs, site.TLS)
-	}
-	s.Server.TLSConfig, err = caddytls.MakeTLSConfig(tlsConfigs)
+	// extract TLS settings from each site config to build
+	// a tls.Config, which will not be nil if TLS is enabled
+	tlsConfig, err := makeTLSConfig(group)
 	if err != nil {
 		return nil, err
 	}
+	s.Server.TLSConfig = tlsConfig
+
+	// if TLS is enabled, make sure we prepare the Server accordingly
+	if s.Server.TLSConfig != nil {
+		// enable QUIC if desired (requires HTTP/2)
+		if HTTP2 && QUIC {
+			s.quicServer = &h2quic.Server{Server: s.Server}
+			s.Server.Handler = s.wrapWithSvcHeaders(s.Server.Handler)
+		}
+
+		// wrap the HTTP handler with a handler that does MITM detection
+		tlsh := &tlsHandler{next: s.Server.Handler}
+		s.Server.Handler = tlsh // this needs to be the "outer" handler when Serve() is called, for type assertion
+
+		// when Serve() creates the TLS listener later, that listener should
+		// be adding a reference the ClientHello info to a map; this callback
+		// will be sure to clear out that entry when the connection closes.
+		s.Server.ConnState = func(c net.Conn, cs http.ConnState) {
+			// when a connection closes or is hijacked, delete its entry
+			// in the map, because we are done with it.
+			if tlsh.listener != nil {
+				if cs == http.StateHijacked || cs == http.StateClosed {
+					tlsh.listener.helloInfosMu.Lock()
+					delete(tlsh.listener.helloInfos, c.RemoteAddr().String())
+					tlsh.listener.helloInfosMu.Unlock()
+				}
+			}
+		}
+
+		// As of Go 1.7, if the Server's TLSConfig is not nil, HTTP/2 is enabled only
+		// if TLSConfig.NextProtos includes the string "h2"
+		if HTTP2 && len(s.Server.TLSConfig.NextProtos) == 0 {
+			// some experimenting shows that this NextProtos must have at least
+			// one value that overlaps with the NextProtos of any other tls.Config
+			// that is returned from GetConfigForClient; if there is no overlap,
+			// the connection will fail (as of Go 1.8, Feb. 2017).
+			s.Server.TLSConfig.NextProtos = defaultALPN
+		}
+	}
 
 	// Compile custom middleware for every site (enables virtual hosting)
 	for _, site := range group {
-		stack := Handler(staticfiles.FileServer{Root: http.Dir(site.Root), Hide: site.HiddenFiles})
+		stack := Handler(staticfiles.FileServer{Root: http.Dir(site.Root), Hide: site.HiddenFiles, IndexPages: site.IndexPages})
 		for i := len(site.middleware) - 1; i >= 0; i-- {
 			stack = site.middleware[i](stack)
 		}
@@ -106,6 +153,87 @@ func NewServer(addr string, group []*SiteConfig) (*Server, error) {
 	return s, nil
 }
 
+// makeHTTPServerWithHeaderLimit apply minimum header limit within a group to given http.Server
+func makeHTTPServerWithHeaderLimit(s *http.Server, group []*SiteConfig) *http.Server {
+	var min int64
+	for _, cfg := range group {
+		limit := cfg.Limits.MaxRequestHeaderSize
+		if limit == 0 {
+			continue
+		}
+
+		// not set yet
+		if min == 0 {
+			min = limit
+		}
+
+		// find a better one
+		if limit < min {
+			min = limit
+		}
+	}
+
+	if min > 0 {
+		s.MaxHeaderBytes = int(min)
+	}
+	return s
+}
+
+// makeHTTPServerWithTimeouts makes an http.Server from the group of
+// configs in a way that configures timeouts (or, if not set, it uses
+// the default timeouts) by combining the configuration of each
+// SiteConfig in the group. (Timeouts are important for mitigating
+// slowloris attacks.)
+func makeHTTPServerWithTimeouts(addr string, group []*SiteConfig) *http.Server {
+	// find the minimum duration configured for each timeout
+	var min Timeouts
+	for _, cfg := range group {
+		if cfg.Timeouts.ReadTimeoutSet &&
+			(!min.ReadTimeoutSet || cfg.Timeouts.ReadTimeout < min.ReadTimeout) {
+			min.ReadTimeoutSet = true
+			min.ReadTimeout = cfg.Timeouts.ReadTimeout
+		}
+		if cfg.Timeouts.ReadHeaderTimeoutSet &&
+			(!min.ReadHeaderTimeoutSet || cfg.Timeouts.ReadHeaderTimeout < min.ReadHeaderTimeout) {
+			min.ReadHeaderTimeoutSet = true
+			min.ReadHeaderTimeout = cfg.Timeouts.ReadHeaderTimeout
+		}
+		if cfg.Timeouts.WriteTimeoutSet &&
+			(!min.WriteTimeoutSet || cfg.Timeouts.WriteTimeout < min.WriteTimeout) {
+			min.WriteTimeoutSet = true
+			min.WriteTimeout = cfg.Timeouts.WriteTimeout
+		}
+		if cfg.Timeouts.IdleTimeoutSet &&
+			(!min.IdleTimeoutSet || cfg.Timeouts.IdleTimeout < min.IdleTimeout) {
+			min.IdleTimeoutSet = true
+			min.IdleTimeout = cfg.Timeouts.IdleTimeout
+		}
+	}
+
+	// for the values that were not set, use defaults
+	if !min.ReadTimeoutSet {
+		min.ReadTimeout = defaultTimeouts.ReadTimeout
+	}
+	if !min.ReadHeaderTimeoutSet {
+		min.ReadHeaderTimeout = defaultTimeouts.ReadHeaderTimeout
+	}
+	if !min.WriteTimeoutSet {
+		min.WriteTimeout = defaultTimeouts.WriteTimeout
+	}
+	if !min.IdleTimeoutSet {
+		min.IdleTimeout = defaultTimeouts.IdleTimeout
+	}
+
+	// set the final values on the server and return it
+	return &http.Server{
+		Addr:              addr,
+		ReadTimeout:       min.ReadTimeout,
+		ReadHeaderTimeout: min.ReadHeaderTimeout,
+		WriteTimeout:      min.WriteTimeout,
+		IdleTimeout:       min.IdleTimeout,
+	}
+}
+
 func (s *Server) wrapWithSvcHeaders(previousHandler http.Handler) http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
 		s.quicServer.SetQuicHeaders(w.Header())
@@ -141,19 +269,36 @@ func (s *Server) Listen() (net.Listener, error) {
 		}
 	}
 
-	// Very important to return a concrete caddy.Listener
-	// implementation for graceful restarts.
-	return ln.(*net.TCPListener), nil
-}
-
-// Serve serves requests on ln. It blocks until ln is closed.
-func (s *Server) Serve(ln net.Listener) error {
 	if tcpLn, ok := ln.(*net.TCPListener); ok {
 		ln = tcpKeepAliveListener{TCPListener: tcpLn}
 	}
 
-	ln = newGracefulListener(ln, &s.connWg)
+	cln := ln.(caddy.Listener)
+	for _, site := range s.sites {
+		for _, m := range site.listenerMiddleware {
+			cln = m(cln)
+		}
+	}
 
+	// Very important to return a concrete caddy.Listener
+	// implementation for graceful restarts.
+	return cln.(caddy.Listener), nil
+}
+
+// ListenPacket creates udp connection for QUIC if it is enabled,
+func (s *Server) ListenPacket() (net.PacketConn, error) {
+	if QUIC {
+		udpAddr, err := net.ResolveUDPAddr("udp", s.Server.Addr)
+		if err != nil {
+			return nil, err
+		}
+		return net.ListenUDP("udp", udpAddr)
+	}
+	return nil, nil
+}
+
+// Serve serves requests on ln. It blocks until ln is closed.
+func (s *Server) Serve(ln net.Listener) error {
 	s.listenerMu.Lock()
 	s.listener = ln
 	s.listenerMu.Unlock()
@@ -164,28 +309,31 @@ func (s *Server) Serve(ln net.Listener) error {
 		// not implement the File() method we need for graceful restarts
 		// on POSIX systems.
 		// TODO: Is this ^ still relevant anymore? Maybe we can now that it's a net.Listener...
-		ln = tls.NewListener(ln, s.Server.TLSConfig)
+		ln = newTLSListener(ln, s.Server.TLSConfig)
+		if handler, ok := s.Server.Handler.(*tlsHandler); ok {
+			handler.listener = ln.(*tlsHelloListener)
+		}
 
 		// Rotate TLS session ticket keys
 		s.tlsGovChan = caddytls.RotateSessionTicketKeys(s.Server.TLSConfig)
 	}
 
-	if QUIC {
-		go func() {
-			err := s.quicServer.ListenAndServe()
-			if err != nil {
-				log.Printf("[ERROR] listening for QUIC connections: %v", err)
-			}
-		}()
-	}
-
 	err := s.Server.Serve(ln)
-	if QUIC {
+	if s.quicServer != nil {
 		s.quicServer.Close()
 	}
 	return err
 }
 
+// ServePacket serves QUIC requests on pc until it is closed.
+func (s *Server) ServePacket(pc net.PacketConn) error {
+	if s.quicServer != nil {
+		err := s.quicServer.Serve(pc.(*net.UDPConn))
+		return fmt.Errorf("serving QUIC connections: %v", err)
+	}
+	return nil
+}
+
 // ServeHTTP is the entry point of all HTTP requests.
 func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 	defer func() {
@@ -197,9 +345,24 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 		}
 	}()
 
-	w.Header().Set("Server", "Caddy")
+	// copy the original, unchanged URL into the context
+	// so it can be referenced by middlewares
+	urlCopy := *r.URL
+	if r.URL.User != nil {
+		userInfo := new(url.Userinfo)
+		*userInfo = *r.URL.User
+		urlCopy.User = userInfo
+	}
+	c := context.WithValue(r.Context(), OriginalURLCtxKey, urlCopy)
+	r = r.WithContext(c)
 
-	sanitizePath(r)
+	// Setup a replacer for the request that keeps track of placeholder
+	// values across plugins.
+	replacer := NewReplacer(r, nil, "")
+	c = context.WithValue(r.Context(), ReplacerCtxKey, replacer)
+	r = r.WithContext(c)
+
+	w.Header().Set("Server", caddy.AppName)
 
 	status, _ := s.serveHTTP(w, r)
 
@@ -220,11 +383,13 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 
 	// look up the virtualhost; if no match, serve error
 	vhost, pathPrefix := s.vhosts.Match(hostname + r.URL.Path)
+	c := context.WithValue(r.Context(), caddy.CtxKey("path_prefix"), pathPrefix)
+	r = r.WithContext(c)
 
 	if vhost == nil {
 		// check for ACME challenge even if vhost is nil;
 		// could be a new host coming online soon
-		if caddytls.HTTPChallengeHandler(w, r, caddytls.DefaultHTTPAlternatePort) {
+		if caddytls.HTTPChallengeHandler(w, r, "localhost") {
 			return 0, nil
 		}
 		// otherwise, log the error and write a message to the client
@@ -232,7 +397,7 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 		if err != nil {
 			remoteHost = r.RemoteAddr
 		}
-		WriteTextResponse(w, http.StatusNotFound, "No such site at "+s.Server.Addr)
+		WriteSiteNotFound(w, r) // don't add headers outside of this function
 		log.Printf("[INFO] %s - No such site at %s (Remote: %s, Referer: %s)",
 			hostname, s.Server.Addr, remoteHost, r.Header.Get("Referer"))
 		return 0, nil
@@ -240,7 +405,7 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 
 	// we still check for ACME challenge if the vhost exists,
 	// because we must apply its HTTP challenge config settings
-	if s.proxyHTTPChallenge(vhost, w, r) {
+	if caddytls.HTTPChallengeHandler(w, r, vhost.ListenHost) {
 		return 0, nil
 	}
 
@@ -248,31 +413,25 @@ func (s *Server) serveHTTP(w http.ResponseWriter, r *http.Request) (int, error)
 	// the URL path, so a request to example.com/foo/blog on the site
 	// defined as example.com/foo appears as /blog instead of /foo/blog.
 	if pathPrefix != "/" {
-		r.URL.Path = strings.TrimPrefix(r.URL.Path, pathPrefix)
-		if !strings.HasPrefix(r.URL.Path, "/") {
-			r.URL.Path = "/" + r.URL.Path
-		}
+		r.URL = trimPathPrefix(r.URL, pathPrefix)
 	}
 
 	return vhost.middlewareChain.ServeHTTP(w, r)
 }
 
-// proxyHTTPChallenge solves the ACME HTTP challenge if r is the HTTP
-// request for the challenge. If it is, and if the request has been
-// fulfilled (response written), true is returned; false otherwise.
-// If you don't have a vhost, just call the challenge handler directly.
-func (s *Server) proxyHTTPChallenge(vhost *SiteConfig, w http.ResponseWriter, r *http.Request) bool {
-	if vhost.Addr.Port != caddytls.HTTPChallengePort {
-		return false
+func trimPathPrefix(u *url.URL, prefix string) *url.URL {
+	// We need to use URL.EscapedPath() when trimming the pathPrefix as
+	// URL.Path is ambiguous about / or %2f - see docs. See #1927
+	trimmed := strings.TrimPrefix(u.EscapedPath(), prefix)
+	if !strings.HasPrefix(trimmed, "/") {
+		trimmed = "/" + trimmed
 	}
-	if vhost.TLS != nil && vhost.TLS.Manual {
-		return false
+	trimmedURL, err := url.Parse(trimmed)
+	if err != nil {
+		log.Printf("[ERROR] Unable to parse trimmed URL %s: %v", trimmed, err)
+		return u
 	}
-	altPort := caddytls.DefaultHTTPAlternatePort
-	if vhost.TLS != nil && vhost.TLS.AltHTTPPort != "" {
-		altPort = vhost.TLS.AltHTTPPort
-	}
-	return caddytls.HTTPChallengeHandler(w, r, altPort)
+	return trimmedURL
 }
 
 // Address returns the address s was assigned to listen on.
@@ -282,62 +441,21 @@ func (s *Server) Address() string {
 
 // Stop stops s gracefully (or forcefully after timeout) and
 // closes its listener.
-func (s *Server) Stop() (err error) {
-	s.Server.SetKeepAlivesEnabled(false)
+func (s *Server) Stop() error {
+	ctx, cancel := context.WithTimeout(context.Background(), s.connTimeout)
+	defer cancel()
 
-	if runtime.GOOS != "windows" {
-		// force connections to close after timeout
-		done := make(chan struct{})
-		go func() {
-			s.connWg.Done() // decrement our initial increment used as a barrier
-			s.connWg.Wait()
-			close(done)
-		}()
-
-		// Wait for remaining connections to finish or
-		// force them all to close after timeout
-		select {
-		case <-time.After(s.connTimeout):
-		case <-done:
-		}
+	err := s.Server.Shutdown(ctx)
+	if err != nil {
+		return err
 	}
 
-	// Close the listener now; this stops the server without delay
-	s.listenerMu.Lock()
-	if s.listener != nil {
-		err = s.listener.Close()
-		s.listener = nil
-	}
-	s.listenerMu.Unlock()
-
-	// Closing this signals any TLS governor goroutines to exit
+	// signal any TLS governor goroutines to exit
 	if s.tlsGovChan != nil {
 		close(s.tlsGovChan)
 	}
 
-	return
-}
-
-// sanitizePath collapses any ./ ../ /// madness
-// which helps prevent path traversal attacks.
-// Note to middleware: use URL.RawPath If you need
-// the "original" URL.Path value.
-func sanitizePath(r *http.Request) {
-	if r.URL.Path == "/" {
-		return
-	}
-	cleanedPath := path.Clean(r.URL.Path)
-	if cleanedPath == "." {
-		r.URL.Path = "/"
-	} else {
-		if !strings.HasPrefix(cleanedPath, "/") {
-			cleanedPath = "/" + cleanedPath
-		}
-		if strings.HasSuffix(r.URL.Path, "/") && !strings.HasSuffix(cleanedPath, "/") {
-			cleanedPath = cleanedPath + "/"
-		}
-		r.URL.Path = cleanedPath
-	}
+	return nil
 }
 
 // OnStartupComplete lists the sites served by this server
@@ -352,9 +470,15 @@ func (s *Server) OnStartupComplete() {
 			output += " (only accessible on this machine)"
 		}
 		fmt.Println(output)
+		log.Println(output)
 	}
 }
 
+// defaultTimeouts stores the default timeout values to use
+// if left unset by user configuration. NOTE: Most default
+// timeouts are disabled (see issues #1464 and #1733).
+var defaultTimeouts = Timeouts{IdleTimeout: 5 * time.Minute}
+
 // tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
 // connections. It's used by ListenAndServe and ListenAndServeTLS so
 // dead TCP connections (e.g. closing laptop mid-download) eventually
@@ -381,12 +505,29 @@ func (ln tcpKeepAliveListener) File() (*os.File, error) {
 	return ln.TCPListener.File()
 }
 
+// ErrMaxBytesExceeded is the error returned by MaxBytesReader
+// when the request body exceeds the limit imposed
+var ErrMaxBytesExceeded = errors.New("http: request body too large")
+
 // DefaultErrorFunc responds to an HTTP request with a simple description
 // of the specified HTTP status code.
 func DefaultErrorFunc(w http.ResponseWriter, r *http.Request, status int) {
 	WriteTextResponse(w, status, fmt.Sprintf("%d %s\n", status, http.StatusText(status)))
 }
 
+const httpStatusMisdirectedRequest = 421 // RFC 7540, 9.1.2
+
+// WriteSiteNotFound writes appropriate error code to w, signaling that
+// requested host is not served by Caddy on a given port.
+func WriteSiteNotFound(w http.ResponseWriter, r *http.Request) {
+	status := http.StatusNotFound
+	if r.ProtoMajor >= 2 {
+		// TODO: use http.StatusMisdirectedRequest when it gets defined
+		status = httpStatusMisdirectedRequest
+	}
+	WriteTextResponse(w, status, fmt.Sprintf("%d Site %s is not served on this interface\n", status, r.Host))
+}
+
 // WriteTextResponse writes body with code status to w. The body will
 // be interpreted as plain text.
 func WriteTextResponse(w http.ResponseWriter, status int, body string) {
@@ -395,3 +536,20 @@ func WriteTextResponse(w http.ResponseWriter, status int, body string) {
 	w.WriteHeader(status)
 	w.Write([]byte(body))
 }
+
+// SafePath joins siteRoot and reqPath and converts it to a path that can
+// be used to access a path on the local disk. It ensures the path does
+// not traverse outside of the site root.
+//
+// If opening a file, use http.Dir instead.
+func SafePath(siteRoot, reqPath string) string {
+	reqPath = filepath.ToSlash(reqPath)
+	reqPath = strings.Replace(reqPath, "\x00", "", -1) // NOTE: Go 1.9 checks for null bytes in the syscall package
+	if siteRoot == "" {
+		siteRoot = "."
+	}
+	return filepath.Join(siteRoot, filepath.FromSlash(path.Clean("/"+reqPath)))
+}
+
+// OriginalURLCtxKey is the key for accessing the original, incoming URL on an HTTP request.
+const OriginalURLCtxKey = caddy.CtxKey("original_url")

caddyhttp/httpserver/server_test.go

@@ -1,8 +1,24 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
 	"net/http"
+	"net/url"
 	"testing"
+	"time"
 )
 
 func TestAddress(t *testing.T) {
@@ -13,3 +29,221 @@ func TestAddress(t *testing.T) {
 		t.Errorf("Expected '%s' but got '%s'", want, got)
 	}
 }
+
+func TestMakeHTTPServerWithTimeouts(t *testing.T) {
+	for i, tc := range []struct {
+		group    []*SiteConfig
+		expected Timeouts
+	}{
+		{
+			group: []*SiteConfig{{Timeouts: Timeouts{}}},
+			expected: Timeouts{
+				ReadTimeout:       defaultTimeouts.ReadTimeout,
+				ReadHeaderTimeout: defaultTimeouts.ReadHeaderTimeout,
+				WriteTimeout:      defaultTimeouts.WriteTimeout,
+				IdleTimeout:       defaultTimeouts.IdleTimeout,
+			},
+		},
+		{
+			group: []*SiteConfig{{Timeouts: Timeouts{
+				ReadTimeout:          1 * time.Second,
+				ReadTimeoutSet:       true,
+				ReadHeaderTimeout:    2 * time.Second,
+				ReadHeaderTimeoutSet: true,
+			}}},
+			expected: Timeouts{
+				ReadTimeout:       1 * time.Second,
+				ReadHeaderTimeout: 2 * time.Second,
+				WriteTimeout:      defaultTimeouts.WriteTimeout,
+				IdleTimeout:       defaultTimeouts.IdleTimeout,
+			},
+		},
+		{
+			group: []*SiteConfig{{Timeouts: Timeouts{
+				ReadTimeoutSet:  true,
+				WriteTimeoutSet: true,
+			}}},
+			expected: Timeouts{
+				ReadTimeout:       0,
+				ReadHeaderTimeout: defaultTimeouts.ReadHeaderTimeout,
+				WriteTimeout:      0,
+				IdleTimeout:       defaultTimeouts.IdleTimeout,
+			},
+		},
+		{
+			group: []*SiteConfig{
+				{Timeouts: Timeouts{
+					ReadTimeout:     2 * time.Second,
+					ReadTimeoutSet:  true,
+					WriteTimeout:    2 * time.Second,
+					WriteTimeoutSet: true,
+				}},
+				{Timeouts: Timeouts{
+					ReadTimeout:     1 * time.Second,
+					ReadTimeoutSet:  true,
+					WriteTimeout:    1 * time.Second,
+					WriteTimeoutSet: true,
+				}},
+			},
+			expected: Timeouts{
+				ReadTimeout:       1 * time.Second,
+				ReadHeaderTimeout: defaultTimeouts.ReadHeaderTimeout,
+				WriteTimeout:      1 * time.Second,
+				IdleTimeout:       defaultTimeouts.IdleTimeout,
+			},
+		},
+		{
+			group: []*SiteConfig{{Timeouts: Timeouts{
+				ReadHeaderTimeout:    5 * time.Second,
+				ReadHeaderTimeoutSet: true,
+				IdleTimeout:          10 * time.Second,
+				IdleTimeoutSet:       true,
+			}}},
+			expected: Timeouts{
+				ReadTimeout:       defaultTimeouts.ReadTimeout,
+				ReadHeaderTimeout: 5 * time.Second,
+				WriteTimeout:      defaultTimeouts.WriteTimeout,
+				IdleTimeout:       10 * time.Second,
+			},
+		},
+	} {
+		actual := makeHTTPServerWithTimeouts("127.0.0.1:9005", tc.group)
+
+		if got, want := actual.Addr, "127.0.0.1:9005"; got != want {
+			t.Errorf("Test %d: Expected Addr=%s, but was %s", i, want, got)
+		}
+		if got, want := actual.ReadTimeout, tc.expected.ReadTimeout; got != want {
+			t.Errorf("Test %d: Expected ReadTimeout=%v, but was %v", i, want, got)
+		}
+		if got, want := actual.ReadHeaderTimeout, tc.expected.ReadHeaderTimeout; got != want {
+			t.Errorf("Test %d: Expected ReadHeaderTimeout=%v, but was %v", i, want, got)
+		}
+		if got, want := actual.WriteTimeout, tc.expected.WriteTimeout; got != want {
+			t.Errorf("Test %d: Expected WriteTimeout=%v, but was %v", i, want, got)
+		}
+		if got, want := actual.IdleTimeout, tc.expected.IdleTimeout; got != want {
+			t.Errorf("Test %d: Expected IdleTimeout=%v, but was %v", i, want, got)
+		}
+	}
+}
+
+func TestTrimPathPrefix(t *testing.T) {
+	for i, pt := range []struct {
+		path       string
+		prefix     string
+		expected   string
+		shouldFail bool
+	}{
+		{
+			path:       "/my/path",
+			prefix:     "/my",
+			expected:   "/path",
+			shouldFail: false,
+		},
+		{
+			path:       "/my/%2f/path",
+			prefix:     "/my",
+			expected:   "/%2f/path",
+			shouldFail: false,
+		},
+		{
+			path:       "/my/path",
+			prefix:     "/my/",
+			expected:   "/path",
+			shouldFail: false,
+		},
+		{
+			path:       "/my///path",
+			prefix:     "/my",
+			expected:   "/path",
+			shouldFail: true,
+		},
+		{
+			path:       "/my///path",
+			prefix:     "/my",
+			expected:   "///path",
+			shouldFail: false,
+		},
+		{
+			path:       "/my/path///slash",
+			prefix:     "/my",
+			expected:   "/path///slash",
+			shouldFail: false,
+		},
+		{
+			path:       "/my/%2f/path/%2f",
+			prefix:     "/my",
+			expected:   "/%2f/path/%2f",
+			shouldFail: false,
+		}, {
+			path:       "/my/%20/path",
+			prefix:     "/my",
+			expected:   "/%20/path",
+			shouldFail: false,
+		}, {
+			path:       "/path",
+			prefix:     "",
+			expected:   "/path",
+			shouldFail: false,
+		}, {
+			path:       "/path/my/",
+			prefix:     "/my",
+			expected:   "/path/my/",
+			shouldFail: false,
+		}, {
+			path:       "",
+			prefix:     "/my",
+			expected:   "/",
+			shouldFail: false,
+		}, {
+			path:       "/apath",
+			prefix:     "",
+			expected:   "/apath",
+			shouldFail: false,
+		},
+	} {
+
+		u, _ := url.Parse(pt.path)
+		if got, want := trimPathPrefix(u, pt.prefix), pt.expected; got.EscapedPath() != want {
+			if !pt.shouldFail {
+
+				t.Errorf("Test %d: Expected='%s', but was '%s' ", i, want, got.EscapedPath())
+			}
+		} else if pt.shouldFail {
+			t.Errorf("SHOULDFAIL Test %d: Expected='%s', and was '%s' but should fail", i, want, got.EscapedPath())
+		}
+	}
+}
+
+func TestMakeHTTPServerWithHeaderLimit(t *testing.T) {
+	for name, c := range map[string]struct {
+		group  []*SiteConfig
+		expect int
+	}{
+		"disable": {
+			group:  []*SiteConfig{{}},
+			expect: 0,
+		},
+		"oneSite": {
+			group: []*SiteConfig{{Limits: Limits{
+				MaxRequestHeaderSize: 100,
+			}}},
+			expect: 100,
+		},
+		"multiSites": {
+			group: []*SiteConfig{
+				{Limits: Limits{MaxRequestHeaderSize: 100}},
+				{Limits: Limits{MaxRequestHeaderSize: 50}},
+			},
+			expect: 50,
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			actual := makeHTTPServerWithHeaderLimit(&http.Server{}, c.group)
+			if got := actual.MaxHeaderBytes; got != c.expect {
+				t.Errorf("Expect %d, but got %d", c.expect, got)
+			}
+		})
+	}
+}

caddyhttp/httpserver/siteconfig.go

@@ -1,6 +1,24 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
-import "github.com/mholt/caddy/caddytls"
+import (
+	"time"
+
+	"github.com/mholt/caddy/caddytls"
+)
 
 // SiteConfig contains information about a site
 // (also known as a virtual host).
@@ -8,6 +26,9 @@ type SiteConfig struct {
 	// The address of the site
 	Addr Address
 
+	// The list of viable index page names of the site
+	IndexPages []string
+
 	// The hostname to bind listener to;
 	// defaults to Addr.Host
 	ListenHost string
@@ -21,6 +42,9 @@ type SiteConfig struct {
 	// Compiled middleware stack
 	middlewareChain Handler
 
+	// listener middleware stack
+	listenerMiddleware []ListenerMiddleware
+
 	// Directory from which to serve files
 	Root string
 
@@ -30,6 +54,65 @@ type SiteConfig struct {
 	// standardized way of loading files from disk
 	// for a request.
 	HiddenFiles []string
+
+	// Max request's header/body size
+	Limits Limits
+
+	// The path to the Caddyfile used to generate this site config
+	originCaddyfile string
+
+	// These timeout values are used, in conjunction with other
+	// site configs on the same server instance, to set the
+	// respective timeout values on the http.Server that
+	// is created. Sensible values will mitigate slowloris
+	// attacks and overcome faulty networks, while still
+	// preserving functionality needed for proxying,
+	// websockets, etc.
+	Timeouts Timeouts
+
+	// If true, any requests not matching other site definitions
+	// may be served by this site.
+	FallbackSite bool
+}
+
+// Timeouts specify various timeouts for a server to use.
+// If the assocated bool field is true, then the duration
+// value should be treated literally (i.e. a zero-value
+// duration would mean "no timeout"). If false, the duration
+// was left unset, so a zero-value duration would mean to
+// use a default value (even if default is non-zero).
+type Timeouts struct {
+	ReadTimeout          time.Duration
+	ReadTimeoutSet       bool
+	ReadHeaderTimeout    time.Duration
+	ReadHeaderTimeoutSet bool
+	WriteTimeout         time.Duration
+	WriteTimeoutSet      bool
+	IdleTimeout          time.Duration
+	IdleTimeoutSet       bool
+}
+
+// Limits specify size limit of request's header and body.
+type Limits struct {
+	MaxRequestHeaderSize int64
+	MaxRequestBodySizes  []PathLimit
+}
+
+// PathLimit is a mapping from a site's path to its corresponding
+// maximum request body size (in bytes)
+type PathLimit struct {
+	Path  string
+	Limit int64
+}
+
+// AddMiddleware adds a middleware to a site's middleware stack.
+func (s *SiteConfig) AddMiddleware(m Middleware) {
+	s.middleware = append(s.middleware, m)
+}
+
+// AddListenerMiddleware adds a listener middleware to a site's listenerMiddleware stack.
+func (s *SiteConfig) AddListenerMiddleware(l ListenerMiddleware) {
+	s.listenerMiddleware = append(s.listenerMiddleware, l)
 }
 
 // TLSConfig returns s.TLS.
@@ -51,3 +134,8 @@ func (s SiteConfig) Port() string {
 func (s SiteConfig) Middleware() []Middleware {
 	return s.middleware
 }
+
+// ListenerMiddleware returns s.listenerMiddleware
+func (s SiteConfig) ListenerMiddleware() []ListenerMiddleware {
+	return s.listenerMiddleware
+}

caddyhttp/httpserver/tplcontext.go

@@ -1,16 +1,36 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
 	"bytes"
+	"crypto/rand"
 	"fmt"
 	"io/ioutil"
+	mathrand "math/rand"
 	"net"
 	"net/http"
 	"net/url"
+	"path"
 	"strings"
+	"sync"
 	"text/template"
 	"time"
 
+	"os"
+
 	"github.com/russross/blackfriday"
 )
 
@@ -22,10 +42,26 @@ type Context struct {
 	Root http.FileSystem
 	Req  *http.Request
 	URL  *url.URL
+	Args []interface{} // defined by arguments to .Include
+
+	// just used for adding preload links for server push
+	responseHeader http.Header
+}
+
+// NewContextWithHeader creates a context with given response header.
+//
+// To plugin developer:
+// The returned context's exported fileds remain empty,
+// you should then initialize them if you want.
+func NewContextWithHeader(rh http.Header) Context {
+	return Context{
+		responseHeader: rh,
+	}
 }
 
 // Include returns the contents of filename relative to the site root.
-func (c Context) Include(filename string) (string, error) {
+func (c Context) Include(filename string, args ...interface{}) (string, error) {
+	c.Args = args
 	return ContextInclude(filename, c, c.Root)
 }
 
@@ -56,6 +92,31 @@ func (c Context) Header(name string) string {
 	return c.Req.Header.Get(name)
 }
 
+// Hostname gets the (remote) hostname of the client making the request.
+func (c Context) Hostname() string {
+	ip := c.IP()
+
+	hostnameList, err := net.LookupAddr(ip)
+	if err != nil || len(hostnameList) == 0 {
+		return c.Req.RemoteAddr
+	}
+
+	return hostnameList[0]
+}
+
+// Env gets a map of the environment variables.
+func (c Context) Env() map[string]string {
+	osEnv := os.Environ()
+	envVars := make(map[string]string, len(osEnv))
+	for _, env := range osEnv {
+		data := strings.SplitN(env, "=", 2)
+		if len(data) == 2 && len(data[0]) > 0 {
+			envVars[data[0]] = data[1]
+		}
+	}
+	return envVars
+}
+
 // IP gets the (remote) IP address of the client making the request.
 func (c Context) IP() string {
 	ip, _, err := net.SplitHostPort(c.Req.RemoteAddr)
@@ -65,6 +126,29 @@ func (c Context) IP() string {
 	return ip
 }
 
+// To mock the net.InterfaceAddrs from the test.
+var networkInterfacesFn = net.InterfaceAddrs
+
+// ServerIP gets the (local) IP address of the server.
+// TODO: The bind directive should be honored in this method (see PR #1474).
+func (c Context) ServerIP() string {
+	addrs, err := networkInterfacesFn()
+	if err != nil {
+		return ""
+	}
+
+	for _, address := range addrs {
+		// Validate the address and check if it's not a loopback
+		if ipnet, ok := address.(*net.IPNet); ok && !ipnet.IP.IsLoopback() {
+			if ipnet.IP.To4() != nil || ipnet.IP.To16() != nil {
+				return ipnet.IP.String()
+			}
+		}
+	}
+
+	return ""
+}
+
 // URI returns the raw, unprocessed request URI (including query
 // string and hash) obtained directly from the Request-Line of
 // the HTTP request.
@@ -92,7 +176,7 @@ func (c Context) Port() (string, error) {
 	if err != nil {
 		if !strings.Contains(c.Req.Host, ":") {
 			// common with sites served on the default port 80
-			return "80", nil
+			return HTTPPort, nil
 		}
 		return "", err
 	}
@@ -158,6 +242,13 @@ func (c Context) StripHTML(s string) string {
 	return buf.String()
 }
 
+// Ext returns the suffix beginning at the final dot in the final
+// slash-separated element of the pathStr (or in other words, the
+// file extension).
+func (c Context) Ext(pathStr string) string {
+	return path.Ext(pathStr)
+}
+
 // StripExt returns the input string without the extension,
 // which is the suffix starting with the final '.' character
 // but not before the final path separator ('/') character.
@@ -211,13 +302,15 @@ func ContextInclude(filename string, ctx interface{}, fs http.FileSystem) (strin
 		return "", err
 	}
 
-	tpl, err := template.New(filename).Parse(string(body))
+	tpl, err := template.New(filename).Funcs(TemplateFuncs).Parse(string(body))
 	if err != nil {
 		return "", err
 	}
 
-	var buf bytes.Buffer
-	err = tpl.Execute(&buf, ctx)
+	buf := includeBufs.Get().(*bytes.Buffer)
+	buf.Reset()
+	defer includeBufs.Put(buf)
+	err = tpl.Execute(buf, ctx)
 	if err != nil {
 		return "", err
 	}
@@ -235,11 +328,16 @@ func (c Context) ToUpper(s string) string {
 	return strings.ToUpper(s)
 }
 
-// Split is a passthrough to strings.Split. It will split the first argument at each instance of the separator and return a slice of strings.
+// Split is a pass-through to strings.Split. It will split the first argument at each instance of the separator and return a slice of strings.
 func (c Context) Split(s string, sep string) []string {
 	return strings.Split(s, sep)
 }
 
+// Join is a pass-through to strings.Join. It will join the first argument slice with the separator in the second argument and return the result.
+func (c Context) Join(a []string, sep string) string {
+	return strings.Join(a, sep)
+}
+
 // Slice will convert the given arguments into a slice.
 func (c Context) Slice(elems ...interface{}) []interface{} {
 	return elems
@@ -261,3 +359,102 @@ func (c Context) Map(values ...interface{}) (map[string]interface{}, error) {
 	}
 	return dict, nil
 }
+
+// Files reads and returns a slice of names from the given directory
+// relative to the root of Context c.
+func (c Context) Files(name string) ([]string, error) {
+	dir, err := c.Root.Open(path.Clean(name))
+	if err != nil {
+		return nil, err
+	}
+	defer dir.Close()
+
+	stat, err := dir.Stat()
+	if err != nil {
+		return nil, err
+	}
+	if !stat.IsDir() {
+		return nil, fmt.Errorf("%v is not a directory", name)
+	}
+
+	dirInfo, err := dir.Readdir(0)
+	if err != nil {
+		return nil, err
+	}
+
+	names := make([]string, len(dirInfo))
+	for i, fileInfo := range dirInfo {
+		names[i] = fileInfo.Name()
+	}
+
+	return names, nil
+}
+
+// IsMITM returns true if it seems likely that the TLS connection
+// is being intercepted.
+func (c Context) IsMITM() bool {
+	if val, ok := c.Req.Context().Value(MitmCtxKey).(bool); ok {
+		return val
+	}
+	return false
+}
+
+// RandomString generates a random string of random length given
+// length bounds. Thanks to http://stackoverflow.com/a/35615565/1048862
+// for the clever technique that is fairly fast, secure, and maintains
+// proper distributions over the dictionary.
+func (c Context) RandomString(minLen, maxLen int) string {
+	const (
+		letterBytes   = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
+		letterIdxBits = 6                    // 6 bits to represent 64 possibilities (indexes)
+		letterIdxMask = 1<<letterIdxBits - 1 // all 1-bits, as many as letterIdxBits
+	)
+
+	if minLen < 0 || maxLen < 0 || maxLen < minLen {
+		return ""
+	}
+
+	n := mathrand.Intn(maxLen-minLen+1) + minLen // choose actual length
+
+	// secureRandomBytes returns a number of bytes using crypto/rand.
+	secureRandomBytes := func(numBytes int) []byte {
+		randomBytes := make([]byte, numBytes)
+		rand.Read(randomBytes)
+		return randomBytes
+	}
+
+	result := make([]byte, n)
+	bufferSize := int(float64(n) * 1.3)
+	for i, j, randomBytes := 0, 0, []byte{}; i < n; j++ {
+		if j%bufferSize == 0 {
+			randomBytes = secureRandomBytes(bufferSize)
+		}
+		if idx := int(randomBytes[j%n] & letterIdxMask); idx < len(letterBytes) {
+			result[i] = letterBytes[idx]
+			i++
+		}
+	}
+
+	return string(result)
+}
+
+// AddLink adds a link header in response
+// see https://www.w3.org/wiki/LinkHeader
+func (c Context) AddLink(link string) string {
+	if c.responseHeader == nil {
+		return ""
+	}
+	c.responseHeader.Add("Link", link)
+	return ""
+}
+
+// buffer pool for .Include context actions
+var includeBufs = sync.Pool{
+	New: func() interface{} {
+		return new(bytes.Buffer)
+	},
+}
+
+// TemplateFuncs contains user-defined functions
+// for execution in templates.
+var TemplateFuncs = template.FuncMap{}

caddyhttp/httpserver/tplcontext_test.go

@@ -1,13 +1,31 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
 	"bytes"
 	"fmt"
 	"io/ioutil"
+	"net"
 	"net/http"
+	"net/http/httptest"
 	"net/url"
 	"os"
 	"path/filepath"
+	"reflect"
+	"sort"
 	"strings"
 	"testing"
 	"time"
@@ -28,6 +46,7 @@ func TestInclude(t *testing.T) {
 	}()
 
 	tests := []struct {
+		args                 []interface{}
 		fileContent          string
 		expectedContent      string
 		shouldErr            bool
@@ -40,7 +59,15 @@ func TestInclude(t *testing.T) {
 			shouldErr:            false,
 			expectedErrorContent: "",
 		},
-		// Test 1 - failure on template.Parse
+		// Test 1 - all good, with args
+		{
+			args:                 []interface{}{"hello", 5},
+			fileContent:          `str1 {{ .Root }} str2 {{index .Args 0}} {{index .Args 1}}`,
+			expectedContent:      fmt.Sprintf("str1 %s str2 %s %d", context.Root, "hello", 5),
+			shouldErr:            false,
+			expectedErrorContent: "",
+		},
+		// Test 2 - failure on template.Parse
 		{
 			fileContent:          `str1 {{ .Root } str2`,
 			expectedContent:      "",
@@ -60,8 +87,16 @@ func TestInclude(t *testing.T) {
 			shouldErr:            true,
 			expectedErrorContent: `type httpserver.Context`,
 		},
+		// Test 4 - all good, with custom function
+		{
+			fileContent:          `hello {{ caddy }}`,
+			expectedContent:      "hello caddy",
+			shouldErr:            false,
+			expectedErrorContent: "",
+		},
 	}
 
+	TemplateFuncs["caddy"] = func() string { return "caddy" }
 	for i, test := range tests {
 		testPrefix := getTestPrefix(i)
 
@@ -71,7 +106,7 @@ func TestInclude(t *testing.T) {
 			t.Fatal(testPrefix+"Failed to create test file. Error was: %v", err)
 		}
 
-		content, err := context.Include(inputFilename)
+		content, err := context.Include(inputFilename, test.args...)
 		if err != nil {
 			if !test.shouldErr {
 				t.Errorf(testPrefix+"Expected no error, found [%s]", test.expectedErrorContent, err.Error())
@@ -224,6 +259,75 @@ func TestHeader(t *testing.T) {
 	}
 }
 
+func TestHostname(t *testing.T) {
+	context := getContextOrFail(t)
+
+	tests := []struct {
+		inputRemoteAddr  string
+		expectedHostname string
+	}{
+		// TODO(mholt): Fix these tests, they're not portable. i.e. my resolver
+		// returns "fwdr-8.fwdr-8.fwdr-8.fwdr-8." instead of these google ones.
+		// Test 0 - ipv4 with port
+		// {"8.8.8.8:1111", "google-public-dns-a.google.com."},
+		// // Test 1 - ipv4 without port
+		// {"8.8.8.8", "google-public-dns-a.google.com."},
+		// // Test 2 - ipv6 with port
+		// {"[2001:4860:4860::8888]:11", "google-public-dns-a.google.com."},
+		// // Test 3 - ipv6 without port and brackets
+		// {"2001:4860:4860::8888", "google-public-dns-a.google.com."},
+		// Test 4 - no hostname available
+		{"1.1.1.1", "1.1.1.1"},
+	}
+
+	for i, test := range tests {
+		testPrefix := getTestPrefix(i)
+
+		context.Req.RemoteAddr = test.inputRemoteAddr
+		actualHostname := context.Hostname()
+
+		if actualHostname != test.expectedHostname {
+			t.Errorf(testPrefix+"Expected hostname %s, found %s", test.expectedHostname, actualHostname)
+		}
+	}
+}
+
+func TestEnv(t *testing.T) {
+	context := getContextOrFail(t)
+
+	name := "ENV_TEST_NAME"
+	testValue := "TEST_VALUE"
+	os.Setenv(name, testValue)
+
+	notExisting := "ENV_TEST_NOT_EXISTING"
+	os.Unsetenv(notExisting)
+
+	invalidName := "ENV_TEST_INVALID_NAME"
+	os.Setenv("="+invalidName, testValue)
+
+	env := context.Env()
+	if value := env[name]; value != testValue {
+		t.Errorf("Expected env-variable %s value '%s', found '%s'",
+			name, testValue, value)
+	}
+
+	if value, ok := env[notExisting]; ok {
+		t.Errorf("Expected empty env-variable %s, found '%s'",
+			notExisting, value)
+	}
+
+	for k, v := range env {
+		if strings.Contains(k, invalidName) {
+			t.Errorf("Expected invalid name not to be included in Env %s, found in key '%s'", invalidName, k)
+		}
+		if strings.Contains(v, invalidName) {
+			t.Errorf("Expected invalid name not be be included in Env %s, found in value '%s'", invalidName, v)
+		}
+	}
+
+	os.Unsetenv("=" + invalidName)
+}
+
 func TestIP(t *testing.T) {
 	context := getContextOrFail(t)
 
@@ -255,6 +359,44 @@ func TestIP(t *testing.T) {
 	}
 }
 
+type myIP string
+
+func (ip myIP) mockInterfaces() ([]net.Addr, error) {
+	a := net.ParseIP(string(ip))
+
+	return []net.Addr{
+		&net.IPNet{IP: a, Mask: nil},
+	}, nil
+}
+
+func TestServerIP(t *testing.T) {
+	context := getContextOrFail(t)
+
+	tests := []string{
+		// Test 0 - ipv4
+		"1.1.1.1",
+		// Test 1 - ipv6
+		"2001:db8:a0b:12f0::1",
+	}
+
+	for i, expectedIP := range tests {
+		testPrefix := getTestPrefix(i)
+
+		// Mock the network interface
+		ip := myIP(expectedIP)
+		networkInterfacesFn = ip.mockInterfaces
+		defer func() {
+			networkInterfacesFn = net.InterfaceAddrs
+		}()
+
+		actualIP := context.ServerIP()
+
+		if actualIP != expectedIP {
+			t.Errorf("%sExpected IP \"%s\", found \"%s\".", testPrefix, expectedIP, actualIP)
+		}
+	}
+}
+
 func TestURL(t *testing.T) {
 	context := getContextOrFail(t)
 
@@ -369,7 +511,7 @@ func TestMethod(t *testing.T) {
 
 }
 
-func TestPathMatches(t *testing.T) {
+func TestContextPathMatches(t *testing.T) {
 	context := getContextOrFail(t)
 
 	tests := []struct {
@@ -604,8 +746,9 @@ func initTestContext() (Context, error) {
 	if err != nil {
 		return Context{}, err
 	}
+	res := httptest.NewRecorder()
 
-	return Context{Root: http.Dir(os.TempDir()), Req: request}, nil
+	return Context{Root: http.Dir(os.TempDir()), responseHeader: res.Header(), Req: request}, nil
 }
 
 func getContextOrFail(t *testing.T) Context {
@@ -648,3 +791,134 @@ func TestTemplates(t *testing.T) {
 		}
 	}
 }
+
+func TestFiles(t *testing.T) {
+	tests := []struct {
+		fileNames []string
+		inputBase string
+		shouldErr bool
+		verifyErr func(error) bool
+	}{
+		// Test 1 - directory and files exist
+		{
+			fileNames: []string{"file1", "file2"},
+			shouldErr: false,
+		},
+		// Test 2 - directory exists, no files
+		{
+			fileNames: []string{},
+			shouldErr: false,
+		},
+		// Test 3 - file or directory does not exist
+		{
+			fileNames: nil,
+			inputBase: "doesNotExist",
+			shouldErr: true,
+			verifyErr: os.IsNotExist,
+		},
+		// Test 4 - directory and files exist, but path to a file
+		{
+			fileNames: []string{"file1", "file2"},
+			inputBase: "file1",
+			shouldErr: true,
+			verifyErr: func(err error) bool {
+				return strings.HasSuffix(err.Error(), "is not a directory")
+			},
+		},
+		// Test 5 - try to leave Context Root
+		{
+			fileNames: nil,
+			inputBase: filepath.Join("..", "..", "..", "..", "..", "etc"),
+			shouldErr: true,
+			verifyErr: os.IsNotExist,
+		},
+	}
+
+	for i, test := range tests {
+		context := getContextOrFail(t)
+		testPrefix := getTestPrefix(i + 1)
+		var dirPath string
+		var err error
+
+		// Create directory / files from test case.
+		if test.fileNames != nil {
+			dirPath, err = ioutil.TempDir(fmt.Sprintf("%s", context.Root), "caddy_ctxtest")
+			if err != nil {
+				os.RemoveAll(dirPath)
+				t.Fatalf(testPrefix+"Expected no error creating directory, got: '%s'", err.Error())
+			}
+
+			for _, name := range test.fileNames {
+				absFilePath := filepath.Join(dirPath, name)
+				if err = ioutil.WriteFile(absFilePath, []byte(""), os.ModePerm); err != nil {
+					os.RemoveAll(dirPath)
+					t.Fatalf(testPrefix+"Expected no error creating file, got: '%s'", err.Error())
+				}
+			}
+		}
+
+		// Perform test case.
+		input := filepath.ToSlash(filepath.Join(filepath.Base(dirPath), test.inputBase))
+		actual, err := context.Files(input)
+		if err != nil {
+			if !test.shouldErr {
+				t.Errorf(testPrefix+"Expected no error, got: '%s'", err.Error())
+			} else if !test.verifyErr(err) {
+				t.Errorf(testPrefix+"Could not verify error content, got: '%s'", err.Error())
+			}
+		} else if test.shouldErr {
+			t.Errorf(testPrefix + "Expected error but had none")
+		} else {
+			numFiles := len(test.fileNames)
+			// reflect.DeepEqual does not consider two empty slices to be equal
+			if numFiles == 0 && len(actual) != 0 {
+				t.Errorf(testPrefix+"Expected files %v, got: %v",
+					test.fileNames, actual)
+			} else {
+				sort.Strings(actual)
+				if numFiles > 0 && !reflect.DeepEqual(test.fileNames, actual) {
+					t.Errorf(testPrefix+"Expected files %v, got: %v",
+						test.fileNames, actual)
+				}
+			}
+		}
+
+		if dirPath != "" {
+			if err := os.RemoveAll(dirPath); err != nil && !os.IsNotExist(err) {
+				t.Fatalf(testPrefix+"Expected no error removing directory, got: '%s'", err.Error())
+			}
+		}
+	}
+}
+
+func TestAddLink(t *testing.T) {
+	for name, c := range map[string]struct {
+		input       string
+		expectLinks []string
+	}{
+		"oneLink": {
+			input:       `{{.AddLink "</test.css>; rel=preload"}}`,
+			expectLinks: []string{"</test.css>; rel=preload"},
+		},
+		"multipleLinks": {
+			input:       `{{.AddLink "</test1.css>; rel=preload"}} {{.AddLink "</test2.css>; rel=meta"}}`,
+			expectLinks: []string{"</test1.css>; rel=preload", "</test2.css>; rel=meta"},
+		},
+	} {
+		c := c
+		t.Run(name, func(t *testing.T) {
+			ctx := getContextOrFail(t)
+			tmpl, err := template.New("").Parse(c.input)
+			if err != nil {
+				t.Fatal(err)
+			}
+			err = tmpl.Execute(ioutil.Discard, ctx)
+			if err != nil {
+				t.Fatal(err)
+			}
+			if got := ctx.responseHeader["Link"]; !reflect.DeepEqual(got, c.expectLinks) {
+				t.Errorf("Result not match: expect %v, but got %v", c.expectLinks, got)
+			}
+		})
+	}
+}

caddyhttp/httpserver/vhosttrie.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (
@@ -10,14 +24,15 @@ import (
 // wildcards as TLS certificates support them), then
 // by longest matching path.
 type vhostTrie struct {
-	edges map[string]*vhostTrie
-	site  *SiteConfig // also known as a virtual host
-	path  string      // the path portion of the key for this node
+	fallbackHosts []string
+	edges         map[string]*vhostTrie
+	site          *SiteConfig // site to match on this node; also known as a virtual host
+	path          string      // the path portion of the key for the associated site
 }
 
 // newVHostTrie returns a new vhostTrie.
 func newVHostTrie() *vhostTrie {
-	return &vhostTrie{edges: make(map[string]*vhostTrie)}
+	return &vhostTrie{edges: make(map[string]*vhostTrie), fallbackHosts: []string{"0.0.0.0", ""}}
 }
 
 // Insert adds stack to t keyed by key. The key should be
@@ -57,13 +72,13 @@ func (t *vhostTrie) insertPath(remainingPath, originalPath string, site *SiteCon
 // A typical key will be in the form "host" or "host/path".
 func (t *vhostTrie) Match(key string) (*SiteConfig, string) {
 	host, path := t.splitHostPath(key)
-	// try the given host, then, if no match, try wildcard hosts
+	// try the given host, then, if no match, try fallback hosts
 	branch := t.matchHost(host)
-	if branch == nil {
-		branch = t.matchHost("0.0.0.0")
-	}
-	if branch == nil {
-		branch = t.matchHost("")
+	for _, h := range t.fallbackHosts {
+		if branch != nil {
+			break
+		}
+		branch = t.matchHost(h)
 	}
 	if branch == nil {
 		return nil, ""
@@ -137,3 +152,24 @@ func (t *vhostTrie) splitHostPath(key string) (host, path string) {
 	}
 	return
 }
+
+// String returns a list of all the entries in t; assumes that
+// t is a root node.
+func (t *vhostTrie) String() string {
+	var s string
+	for host, edge := range t.edges {
+		s += edge.str(host)
+	}
+	return s
+}
+
+func (t *vhostTrie) str(prefix string) string {
+	var s string
+	for key, edge := range t.edges {
+		if edge.site != nil {
+			s += prefix + key + "\n"
+		}
+		s += edge.str(prefix + key)
+	}
+	return s
+}

caddyhttp/httpserver/vhosttrie_test.go

@@ -1,3 +1,17 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
 package httpserver
 
 import (